2023-11-08T19:07:43.858 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-08T19:07:43.859 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi --owner scheduled_pdonnell@teuthology --verbose --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} --archive /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101 -- /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/orig.config.yaml 2023-11-08T19:07:43.909 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T19:07:43.999 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101 branch: main description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} email: pdonnell@redhat.com first_in_suite: false job_id: '7452101' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - evicting unresponsive client - RECENT_CRASH sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: dry_run: true install: ceph: flavor: default sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-batrick-testing-20231108.185857 sha1: e797689ab88984658d73bbc928b2ed06c676d14a owner: scheduled_pdonnell@teuthology priority: 50 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3044 sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-batrick-testing-20231108.185857 suite_path: /home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: e797689ab88984658d73bbc928b2ed06c676d14a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi119.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPBNva1EB0X+Rk9GaiKcTAed2A2VkQxzqBk9e/XzaaFlee/0pzNH2bS7q+Q+rsPjvt+wMPeHsePwojxmU8NtGY= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOkKx9IMFOrO+AuaZwDyc1aHT05j90gu48HEtBN8zF7Nk+xxLdFUunqvdnYQFJjED4nXkaViAUM+irEx681lgs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - exec: mon.a: - ceph config set mgr mgr/crash/warn_recent_interval 0 - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_snapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-08_19:01:12 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-08T19:07:43.999 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa; will attempt to use it 2023-11-08T19:07:44.000 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks 2023-11-08T19:07:44.000 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-08T19:07:44.002 INFO:teuthology.task.internal:Checking packages... 2023-11-08T19:07:44.028 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4' 2023-11-08T19:07:44.028 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-08T19:07:44.029 INFO:teuthology.packaging:ref: None 2023-11-08T19:07:44.029 INFO:teuthology.packaging:tag: None 2023-11-08T19:07:44.029 INFO:teuthology.packaging:branch: main 2023-11-08T19:07:44.030 INFO:teuthology.packaging:sha1: 7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:07:44.030 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2023-11-08T19:07:44.281 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7224.g8858839c 2023-11-08T19:07:44.282 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-08T19:07:44.290 INFO:teuthology.task.internal:no buildpackages task found 2023-11-08T19:07:44.291 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-08T19:07:44.309 INFO:teuthology.task.internal:Saving configuration 2023-11-08T19:07:44.331 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-08T19:07:44.336 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T19:07:44.363 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:02:51.385441', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPBNva1EB0X+Rk9GaiKcTAed2A2VkQxzqBk9e/XzaaFlee/0pzNH2bS7q+Q+rsPjvt+wMPeHsePwojxmU8NtGY='} 2023-11-08T19:07:44.385 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:02:51.383928', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOkKx9IMFOrO+AuaZwDyc1aHT05j90gu48HEtBN8zF7Nk+xxLdFUunqvdnYQFJjED4nXkaViAUM+irEx681lgs='} 2023-11-08T19:07:44.385 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-08T19:07:44.391 INFO:teuthology.task.internal:roles: ubuntu@smithi119.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-08T19:07:44.391 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-08T19:07:44.391 INFO:teuthology.run_tasks:Running task console_log... 2023-11-08T19:07:44.465 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f18ae9bdb80>, signals=[15]) 2023-11-08T19:07:44.466 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-08T19:07:44.472 INFO:teuthology.task.internal:Opening connections... 2023-11-08T19:07:44.473 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:07:44.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:07:44.562 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:07:44.563 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:07:44.648 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-08T19:07:44.654 DEBUG:teuthology.orchestra.run.smithi119:> uname -m 2023-11-08T19:07:44.672 INFO:teuthology.orchestra.run.smithi119.stdout:x86_64 2023-11-08T19:07:44.673 DEBUG:teuthology.orchestra.run.smithi119:> cat /etc/os-release 2023-11-08T19:07:44.729 INFO:teuthology.orchestra.run.smithi119.stdout:NAME="CentOS Stream" 2023-11-08T19:07:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION="9" 2023-11-08T19:07:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:ID="centos" 2023-11-08T19:07:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:ID_LIKE="rhel fedora" 2023-11-08T19:07:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:VERSION_ID="9" 2023-11-08T19:07:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:PLATFORM_ID="platform:el9" 2023-11-08T19:07:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-08T19:07:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:ANSI_COLOR="0;31" 2023-11-08T19:07:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:LOGO="fedora-logo-icon" 2023-11-08T19:07:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-08T19:07:44.731 INFO:teuthology.orchestra.run.smithi119.stdout:HOME_URL="https://centos.org/" 2023-11-08T19:07:44.732 INFO:teuthology.orchestra.run.smithi119.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-08T19:07:44.732 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-08T19:07:44.732 INFO:teuthology.orchestra.run.smithi119.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-08T19:07:44.733 INFO:teuthology.lock.ops:Updating smithi119.front.sepia.ceph.com on lock server 2023-11-08T19:07:44.778 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2023-11-08T19:07:44.796 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2023-11-08T19:07:44.796 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2023-11-08T19:07:44.853 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2023-11-08T19:07:44.853 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2023-11-08T19:07:44.854 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2023-11-08T19:07:44.854 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2023-11-08T19:07:44.854 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2023-11-08T19:07:44.854 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2023-11-08T19:07:44.854 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-08T19:07:44.855 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2023-11-08T19:07:44.855 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2023-11-08T19:07:44.855 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-08T19:07:44.855 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2023-11-08T19:07:44.855 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-08T19:07:44.856 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-08T19:07:44.856 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-08T19:07:44.856 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2023-11-08T19:07:44.886 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-08T19:07:44.895 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-08T19:07:44.903 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-08T19:07:44.903 DEBUG:teuthology.orchestra.run.smithi119:> test '!' -e /home/ubuntu/cephtest 2023-11-08T19:07:44.908 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2023-11-08T19:07:44.924 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-08T19:07:44.930 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-08T19:07:44.931 DEBUG:teuthology.orchestra.run.smithi119:> test -z $(ls -A /var/lib/ceph) 2023-11-08T19:07:44.966 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2023-11-08T19:07:45.163 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-08T19:07:45.216 INFO:teuthology.run_tasks:Running task kernel... 2023-11-08T19:07:45.232 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-11-08T19:07:45.233 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-08T19:07:45.233 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-08T19:07:45.233 DEBUG:teuthology.orchestra.run.smithi119:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T19:07:45.234 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T19:07:45.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:07:45.251 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-11-08T19:07:45.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:07:45.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-11-08T19:07:45.412 DEBUG:teuthology.task.kernel:sha1 for client.0 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T19:07:45.412 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-08T19:07:45.412 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-11-08T19:07:45.417 DEBUG:teuthology.task.kernel:sha1 for client.1 is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T19:07:45.418 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-08T19:07:45.418 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-11-08T19:07:45.431 INFO:teuthology.orchestra.run.smithi119.stdout:5.14.0-361.el9.x86_64 2023-11-08T19:07:45.432 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T19:07:45.433 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-08T19:07:45.434 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum upgrade -y linux-firmware 2023-11-08T19:07:45.436 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-361.el9.x86_64 2023-11-08T19:07:45.436 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T19:07:45.437 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-08T19:07:45.437 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum upgrade -y linux-firmware 2023-11-08T19:07:47.501 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 14 MB/s | 6.3 MB 00:00 2023-11-08T19:07:47.539 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-11-08T19:07:48.817 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2023-11-08T19:07:48.831 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 8.0 MB/s | 2.0 MB 00:00 2023-11-08T19:07:50.072 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 8.1 MB/s | 7.9 MB 00:00 2023-11-08T19:07:50.350 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 7.9 MB 00:01 2023-11-08T19:07:52.281 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 16 MB/s | 18 MB 00:01 2023-11-08T19:07:56.834 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 18 MB 00:05 2023-11-08T19:07:57.073 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 5.8 MB 00:01 2023-11-08T19:07:58.440 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 56 kB/s | 15 kB 00:00 2023-11-08T19:08:00.751 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 5.8 MB 00:00 2023-11-08T19:08:02.502 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2023-11-08T19:08:04.061 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 14 MB/s | 19 MB 00:01 2023-11-08T19:08:08.544 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2023-11-08T19:08:09.156 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 1.8 MB/s | 19 MB 00:10 2023-11-08T19:08:10.843 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-11-08T19:08:10.853 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-11-08T19:08:10.853 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-11-08T19:08:10.853 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-11-08T19:08:10.853 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-11-08T19:08:10.854 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-11-08T19:08:10.854 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:08:10.854 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-11-08T19:08:10.854 INFO:teuthology.orchestra.run.smithi119.stdout:================================================================================ 2023-11-08T19:08:10.855 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 1 Package 2023-11-08T19:08:10.855 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:08:10.855 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 326 M 2023-11-08T19:08:10.855 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-11-08T19:08:13.772 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-11-08T19:08:14.562 INFO:teuthology.orchestra.run.smithi119.stdout:linux-firmware-20230926-140.el9.noarch.rpm 88 MB/s | 326 MB 00:03 2023-11-08T19:08:14.563 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-11-08T19:08:14.564 INFO:teuthology.orchestra.run.smithi119.stdout:Total 88 MB/s | 326 MB 00:03 2023-11-08T19:08:14.564 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-11-08T19:08:14.662 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-11-08T19:08:14.662 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-11-08T19:08:16.108 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-11-08T19:08:16.119 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-11-08T19:08:16.120 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2023-11-08T19:08:16.120 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-11-08T19:08:16.120 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2023-11-08T19:08:16.120 INFO:teuthology.orchestra.run.smithi188.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-11-08T19:08:16.121 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:08:16.121 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2023-11-08T19:08:16.121 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2023-11-08T19:08:16.121 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 1 Package 2023-11-08T19:08:16.121 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:08:16.122 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 326 M 2023-11-08T19:08:16.122 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2023-11-08T19:08:16.723 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-11-08T19:08:16.724 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-11-08T19:08:20.031 INFO:teuthology.orchestra.run.smithi188.stdout:linux-firmware-20230926-140.el9.noarch.rpm 83 MB/s | 326 MB 00:03 2023-11-08T19:08:20.032 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2023-11-08T19:08:20.033 INFO:teuthology.orchestra.run.smithi188.stdout:Total 83 MB/s | 326 MB 00:03 2023-11-08T19:08:20.033 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2023-11-08T19:08:20.142 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2023-11-08T19:08:20.142 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2023-11-08T19:08:22.404 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-11-08T19:08:22.451 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2023-11-08T19:08:22.451 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2023-11-08T19:08:22.874 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T19:08:22.971 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T19:08:23.898 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T19:08:23.898 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T19:08:24.288 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T19:08:24.288 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:08:24.288 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-11-08T19:08:24.289 INFO:teuthology.orchestra.run.smithi119.stdout: linux-firmware-20230926-140.el9.noarch 2023-11-08T19:08:24.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:08:24.289 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-11-08T19:08:24.575 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-08T19:08:24.576 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T19:08:24.721 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-11-08T19:08:24.722 DEBUG:teuthology.orchestra.run.smithi119:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-08T19:08:24.724 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi119.front.sepia.ceph.com to complete... 2023-11-08T19:08:28.226 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2023-11-08T19:08:28.606 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T19:08:28.695 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T19:08:29.562 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T19:08:29.562 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T19:08:29.941 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T19:08:29.941 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:08:29.941 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2023-11-08T19:08:29.942 INFO:teuthology.orchestra.run.smithi188.stdout: linux-firmware-20230926-140.el9.noarch 2023-11-08T19:08:29.942 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:08:29.942 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-11-08T19:08:30.248 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.1... 2023-11-08T19:08:30.249 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T19:08:30.473 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-11-08T19:08:30.474 DEBUG:teuthology.orchestra.run.smithi188:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-08T19:08:30.477 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi188.front.sepia.ceph.com to complete... 2023-11-08T19:08:38.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08 19:08:38 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-08T19:08:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:FINISHED --2023-11-08 19:08:38-- 2023-11-08T19:08:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:Total wall clock time: 14s 2023-11-08T19:08:38.739 INFO:teuthology.orchestra.run.smithi119.stderr:Downloaded: 1 files, 306M in 14s (22.3 MB/s) 2023-11-08T19:08:38.740 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.0... 2023-11-08T19:08:38.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-08T19:08:40.589 INFO:teuthology.orchestra.run.smithi119.stdout:Verifying... ######################################## 2023-11-08T19:08:40.666 INFO:teuthology.orchestra.run.smithi119.stdout:Preparing... ######################################## 2023-11-08T19:08:40.745 INFO:teuthology.orchestra.run.smithi119.stdout:Updating / installing... 2023-11-08T19:08:47.784 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-08T19:09:03.083 INFO:teuthology.orchestra.run.smithi119.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-08T19:09:06.671 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08 19:09:06 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-08T19:09:06.672 INFO:teuthology.orchestra.run.smithi188.stderr:FINISHED --2023-11-08 19:09:06-- 2023-11-08T19:09:06.672 INFO:teuthology.orchestra.run.smithi188.stderr:Total wall clock time: 36s 2023-11-08T19:09:06.672 INFO:teuthology.orchestra.run.smithi188.stderr:Downloaded: 1 files, 306M in 36s (8.61 MB/s) 2023-11-08T19:09:06.725 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on client.1... 2023-11-08T19:09:06.726 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-08T19:09:08.561 INFO:teuthology.orchestra.run.smithi188.stdout:Verifying... ######################################## 2023-11-08T19:09:08.647 INFO:teuthology.orchestra.run.smithi188.stdout:Preparing... ######################################## 2023-11-08T19:09:08.727 INFO:teuthology.orchestra.run.smithi188.stdout:Updating / installing... 2023-11-08T19:09:15.821 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-08T19:09:24.265 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi119.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-08T19:09:24.266 DEBUG:teuthology.orchestra.run.smithi119:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-08T19:09:24.327 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.327 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.327 INFO:teuthology.orchestra.run.smithi119.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.327 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-08T19:09:24.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-08T19:09:24.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-08T19:09:24.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-08T19:09:24.328 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-08T19:09:24.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-08T19:09:24.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-08T19:09:24.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-08T19:09:24.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-08T19:09:24.329 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-08T19:09:24.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-08T19:09:24.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-08T19:09:24.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-08T19:09:24.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-08T19:09:24.330 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-08T19:09:24.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-08T19:09:24.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-08T19:09:24.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-08T19:09:24.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-08T19:09:24.331 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-08T19:09:24.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-08T19:09:24.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-08T19:09:24.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-08T19:09:24.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-08T19:09:24.332 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-08T19:09:24.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-08T19:09:24.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-08T19:09:24.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-08T19:09:24.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-08T19:09:24.333 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-08T19:09:24.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-08T19:09:24.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-08T19:09:24.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-08T19:09:24.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-08T19:09:24.334 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-08T19:09:24.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-08T19:09:24.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-08T19:09:24.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-08T19:09:24.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-08T19:09:24.335 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-08T19:09:24.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-08T19:09:24.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-08T19:09:24.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-08T19:09:24.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-08T19:09:24.336 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-08T19:09:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-08T19:09:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-08T19:09:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-08T19:09:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-08T19:09:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-08T19:09:24.337 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-08T19:09:24.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-08T19:09:24.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-08T19:09:24.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-08T19:09:24.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-08T19:09:24.338 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-08T19:09:24.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-08T19:09:24.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-08T19:09:24.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-08T19:09:24.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-08T19:09:24.339 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-08T19:09:24.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-08T19:09:24.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-08T19:09:24.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-08T19:09:24.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-08T19:09:24.340 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-08T19:09:24.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-08T19:09:24.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-08T19:09:24.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-08T19:09:24.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-08T19:09:24.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-08T19:09:24.341 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-08T19:09:24.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-08T19:09:24.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-08T19:09:24.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-08T19:09:24.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-08T19:09:24.342 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-08T19:09:24.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-08T19:09:24.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-08T19:09:24.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-08T19:09:24.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-08T19:09:24.343 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-08T19:09:24.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-08T19:09:24.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-08T19:09:24.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-08T19:09:24.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-08T19:09:24.344 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-08T19:09:24.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-08T19:09:24.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-08T19:09:24.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-08T19:09:24.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-08T19:09:24.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-08T19:09:24.345 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-08T19:09:24.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-08T19:09:24.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-08T19:09:24.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-08T19:09:24.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-08T19:09:24.346 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-08T19:09:24.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-08T19:09:24.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-08T19:09:24.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-08T19:09:24.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-08T19:09:24.347 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-08T19:09:24.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-08T19:09:24.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-08T19:09:24.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-08T19:09:24.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-08T19:09:24.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-08T19:09:24.348 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-08T19:09:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-08T19:09:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-08T19:09:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-08T19:09:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-08T19:09:24.349 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-08T19:09:24.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-08T19:09:24.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-08T19:09:24.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-08T19:09:24.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-08T19:09:24.350 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-08T19:09:24.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-08T19:09:24.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-08T19:09:24.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-08T19:09:24.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-08T19:09:24.351 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-08T19:09:24.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-08T19:09:24.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-08T19:09:24.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-08T19:09:24.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-08T19:09:24.352 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-08T19:09:24.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-08T19:09:24.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-08T19:09:24.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-08T19:09:24.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-08T19:09:24.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-08T19:09:24.353 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-08T19:09:24.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-08T19:09:24.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-08T19:09:24.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-08T19:09:24.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-08T19:09:24.354 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-08T19:09:24.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-08T19:09:24.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-08T19:09:24.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-08T19:09:24.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-08T19:09:24.355 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-08T19:09:24.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-08T19:09:24.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-08T19:09:24.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-08T19:09:24.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-08T19:09:24.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-08T19:09:24.356 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-08T19:09:24.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-08T19:09:24.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-08T19:09:24.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-08T19:09:24.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-08T19:09:24.357 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-08T19:09:24.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-08T19:09:24.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-08T19:09:24.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-08T19:09:24.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-08T19:09:24.358 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-08T19:09:24.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-08T19:09:24.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-08T19:09:24.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-08T19:09:24.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-08T19:09:24.359 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-08T19:09:24.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-08T19:09:24.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-08T19:09:24.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-08T19:09:24.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-08T19:09:24.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-08T19:09:24.360 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-08T19:09:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-08T19:09:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-08T19:09:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-08T19:09:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-08T19:09:24.361 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-08T19:09:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-08T19:09:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-08T19:09:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-08T19:09:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-08T19:09:24.362 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-08T19:09:24.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-08T19:09:24.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-08T19:09:24.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-08T19:09:24.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-08T19:09:24.363 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-08T19:09:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-08T19:09:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-08T19:09:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-08T19:09:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-08T19:09:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-08T19:09:24.364 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-08T19:09:24.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-08T19:09:24.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-08T19:09:24.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-08T19:09:24.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-08T19:09:24.365 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-08T19:09:24.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-08T19:09:24.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-08T19:09:24.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-08T19:09:24.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-08T19:09:24.366 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-08T19:09:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-08T19:09:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-08T19:09:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-08T19:09:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-08T19:09:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-08T19:09:24.367 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-08T19:09:24.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-08T19:09:24.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-08T19:09:24.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-08T19:09:24.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-08T19:09:24.368 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-08T19:09:24.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-08T19:09:24.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-08T19:09:24.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-08T19:09:24.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-08T19:09:24.369 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-08T19:09:24.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-08T19:09:24.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-08T19:09:24.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-08T19:09:24.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-08T19:09:24.370 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-08T19:09:24.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-08T19:09:24.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-08T19:09:24.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-08T19:09:24.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-08T19:09:24.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-08T19:09:24.371 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-08T19:09:24.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-08T19:09:24.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-08T19:09:24.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-08T19:09:24.372 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-08T19:09:24.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-08T19:09:24.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-08T19:09:24.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-08T19:09:24.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-08T19:09:24.373 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-08T19:09:24.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-08T19:09:24.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-08T19:09:24.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-08T19:09:24.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-08T19:09:24.374 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-08T19:09:24.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-08T19:09:24.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-08T19:09:24.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-08T19:09:24.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-08T19:09:24.375 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-08T19:09:24.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-08T19:09:24.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-08T19:09:24.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-08T19:09:24.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-08T19:09:24.376 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-08T19:09:24.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-08T19:09:24.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-08T19:09:24.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-08T19:09:24.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-08T19:09:24.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-08T19:09:24.377 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-08T19:09:24.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-08T19:09:24.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-08T19:09:24.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-08T19:09:24.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-08T19:09:24.378 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-08T19:09:24.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-08T19:09:24.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-08T19:09:24.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-08T19:09:24.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-08T19:09:24.379 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-08T19:09:24.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-08T19:09:24.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-08T19:09:24.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-08T19:09:24.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-08T19:09:24.380 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-08T19:09:24.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-08T19:09:24.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-08T19:09:24.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-08T19:09:24.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-08T19:09:24.381 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-08T19:09:24.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-08T19:09:24.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-08T19:09:24.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-08T19:09:24.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-08T19:09:24.382 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-08T19:09:24.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-08T19:09:24.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-08T19:09:24.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-08T19:09:24.383 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-08T19:09:24.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-08T19:09:24.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-08T19:09:24.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-08T19:09:24.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-08T19:09:24.384 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-08T19:09:24.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-08T19:09:24.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-08T19:09:24.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-08T19:09:24.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-08T19:09:24.385 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-08T19:09:24.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-08T19:09:24.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-08T19:09:24.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-08T19:09:24.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-08T19:09:24.386 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-08T19:09:24.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-08T19:09:24.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-08T19:09:24.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-08T19:09:24.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-08T19:09:24.387 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-08T19:09:24.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-08T19:09:24.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-08T19:09:24.388 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-08T19:09:24.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-08T19:09:24.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-08T19:09:24.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-08T19:09:24.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-08T19:09:24.389 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-08T19:09:24.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-08T19:09:24.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-08T19:09:24.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-08T19:09:24.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-08T19:09:24.390 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-08T19:09:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-08T19:09:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-08T19:09:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-08T19:09:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-08T19:09:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-08T19:09:24.391 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-08T19:09:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-08T19:09:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-08T19:09:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-08T19:09:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-08T19:09:24.392 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-08T19:09:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-08T19:09:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-08T19:09:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-08T19:09:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-08T19:09:24.393 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-08T19:09:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-08T19:09:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-08T19:09:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-08T19:09:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-08T19:09:24.394 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-08T19:09:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-08T19:09:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-08T19:09:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-08T19:09:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-08T19:09:24.395 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-08T19:09:24.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-08T19:09:24.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-08T19:09:24.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-08T19:09:24.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-08T19:09:24.396 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-08T19:09:24.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-08T19:09:24.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-08T19:09:24.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-08T19:09:24.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-08T19:09:24.397 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-08T19:09:24.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-08T19:09:24.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-08T19:09:24.398 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-08T19:09:24.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-08T19:09:24.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-08T19:09:24.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-08T19:09:24.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-08T19:09:24.400 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-08T19:09:24.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-08T19:09:24.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-08T19:09:24.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-08T19:09:24.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-08T19:09:24.401 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-08T19:09:24.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-08T19:09:24.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-08T19:09:24.402 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-08T19:09:24.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-08T19:09:24.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-08T19:09:24.403 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-08T19:09:24.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-08T19:09:24.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-08T19:09:24.404 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-08T19:09:24.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-08T19:09:24.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-08T19:09:24.405 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-08T19:09:24.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-08T19:09:24.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-08T19:09:24.406 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-08T19:09:24.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-08T19:09:24.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-08T19:09:24.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-08T19:09:24.407 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-08T19:09:24.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-08T19:09:24.408 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-08T19:09:24.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-08T19:09:24.409 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-08T19:09:24.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-08T19:09:24.410 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-08T19:09:24.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-08T19:09:24.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-08T19:09:24.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-08T19:09:24.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-08T19:09:24.411 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-08T19:09:24.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-08T19:09:24.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-08T19:09:24.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-08T19:09:24.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-08T19:09:24.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-08T19:09:24.412 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-08T19:09:24.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-08T19:09:24.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-08T19:09:24.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-08T19:09:24.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-08T19:09:24.413 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-08T19:09:24.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-08T19:09:24.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-08T19:09:24.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-08T19:09:24.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-08T19:09:24.414 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-08T19:09:24.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-08T19:09:24.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-08T19:09:24.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-08T19:09:24.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-08T19:09:24.415 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-08T19:09:24.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-08T19:09:24.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-08T19:09:24.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-08T19:09:24.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-08T19:09:24.416 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-08T19:09:24.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-08T19:09:24.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-08T19:09:24.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-08T19:09:24.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-08T19:09:24.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-08T19:09:24.417 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-08T19:09:24.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-08T19:09:24.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-08T19:09:24.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-08T19:09:24.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-08T19:09:24.418 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-08T19:09:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-08T19:09:24.419 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-08T19:09:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-08T19:09:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-08T19:09:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-08T19:09:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-08T19:09:24.420 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-08T19:09:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-08T19:09:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-08T19:09:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-08T19:09:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-08T19:09:24.421 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-08T19:09:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-08T19:09:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-08T19:09:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-08T19:09:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-08T19:09:24.422 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-08T19:09:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-08T19:09:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-08T19:09:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-08T19:09:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-08T19:09:24.423 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-08T19:09:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-08T19:09:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-08T19:09:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-08T19:09:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-08T19:09:24.424 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-08T19:09:24.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-08T19:09:24.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-08T19:09:24.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-08T19:09:24.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-08T19:09:24.425 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-08T19:09:24.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-08T19:09:24.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-08T19:09:24.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-08T19:09:24.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-08T19:09:24.426 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-08T19:09:24.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-08T19:09:24.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-08T19:09:24.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-08T19:09:24.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-08T19:09:24.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-08T19:09:24.427 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-08T19:09:24.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-08T19:09:24.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-08T19:09:24.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-08T19:09:24.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-08T19:09:24.428 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-08T19:09:24.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-08T19:09:24.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-08T19:09:24.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-08T19:09:24.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-08T19:09:24.429 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-08T19:09:24.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-08T19:09:24.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-08T19:09:24.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-08T19:09:24.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-08T19:09:24.430 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-08T19:09:24.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-08T19:09:24.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-08T19:09:24.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-08T19:09:24.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-08T19:09:24.431 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-08T19:09:24.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-08T19:09:24.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-08T19:09:24.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-08T19:09:24.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-08T19:09:24.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-08T19:09:24.432 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-08T19:09:24.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-08T19:09:24.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-08T19:09:24.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-08T19:09:24.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-08T19:09:24.433 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-08T19:09:24.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-08T19:09:24.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-08T19:09:24.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-08T19:09:24.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-08T19:09:24.434 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-08T19:09:24.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-08T19:09:24.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-08T19:09:24.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-08T19:09:24.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-08T19:09:24.435 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-08T19:09:24.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-08T19:09:24.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-08T19:09:24.436 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-08T19:09:24.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-08T19:09:24.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-08T19:09:24.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-08T19:09:24.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-08T19:09:24.437 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-08T19:09:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-08T19:09:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-08T19:09:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-08T19:09:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-08T19:09:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-08T19:09:24.438 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-08T19:09:24.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-08T19:09:24.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-08T19:09:24.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-08T19:09:24.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-08T19:09:24.439 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-08T19:09:24.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-08T19:09:24.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-08T19:09:24.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-08T19:09:24.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-08T19:09:24.440 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-08T19:09:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-08T19:09:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-08T19:09:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-08T19:09:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-08T19:09:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-08T19:09:24.441 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-08T19:09:24.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-08T19:09:24.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-08T19:09:24.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-08T19:09:24.442 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-08T19:09:24.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-08T19:09:24.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-08T19:09:24.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-08T19:09:24.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-08T19:09:24.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-08T19:09:24.443 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-08T19:09:24.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-08T19:09:24.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-08T19:09:24.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-08T19:09:24.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-08T19:09:24.444 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-08T19:09:24.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-08T19:09:24.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-08T19:09:24.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-08T19:09:24.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-08T19:09:24.445 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-08T19:09:24.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-08T19:09:24.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-08T19:09:24.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-08T19:09:24.446 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-08T19:09:24.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-08T19:09:24.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-08T19:09:24.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-08T19:09:24.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-08T19:09:24.447 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-08T19:09:24.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-08T19:09:24.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-08T19:09:24.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-08T19:09:24.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-08T19:09:24.448 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-08T19:09:24.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-08T19:09:24.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-08T19:09:24.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-08T19:09:24.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-08T19:09:24.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-08T19:09:24.449 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-08T19:09:24.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-08T19:09:24.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-08T19:09:24.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-08T19:09:24.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-08T19:09:24.450 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-08T19:09:24.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-08T19:09:24.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-08T19:09:24.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-08T19:09:24.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-08T19:09:24.451 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-08T19:09:24.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-08T19:09:24.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-08T19:09:24.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-08T19:09:24.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-08T19:09:24.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-08T19:09:24.452 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-08T19:09:24.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-08T19:09:24.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-08T19:09:24.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-08T19:09:24.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-08T19:09:24.453 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-08T19:09:24.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-08T19:09:24.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-08T19:09:24.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-08T19:09:24.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-08T19:09:24.454 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-08T19:09:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-08T19:09:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-08T19:09:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-08T19:09:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-08T19:09:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-08T19:09:24.455 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-08T19:09:24.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-08T19:09:24.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-08T19:09:24.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-08T19:09:24.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-08T19:09:24.456 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-08T19:09:24.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-08T19:09:24.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-08T19:09:24.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-08T19:09:24.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-08T19:09:24.457 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-08T19:09:24.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-08T19:09:24.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-08T19:09:24.458 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-08T19:09:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-08T19:09:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-08T19:09:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-08T19:09:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-08T19:09:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-08T19:09:24.459 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-08T19:09:24.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-08T19:09:24.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-08T19:09:24.460 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-08T19:09:24.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-08T19:09:24.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-08T19:09:24.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-08T19:09:24.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-08T19:09:24.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-08T19:09:24.461 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-08T19:09:24.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-08T19:09:24.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-08T19:09:24.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-08T19:09:24.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-08T19:09:24.462 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-08T19:09:24.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-08T19:09:24.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-08T19:09:24.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-08T19:09:24.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-08T19:09:24.463 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-08T19:09:24.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-08T19:09:24.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-08T19:09:24.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-08T19:09:24.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-08T19:09:24.464 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-08T19:09:24.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-08T19:09:24.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-08T19:09:24.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-08T19:09:24.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-08T19:09:24.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-08T19:09:24.465 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-08T19:09:24.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-08T19:09:24.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-08T19:09:24.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-08T19:09:24.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-08T19:09:24.466 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-08T19:09:24.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-08T19:09:24.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-08T19:09:24.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-08T19:09:24.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-08T19:09:24.467 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-08T19:09:24.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-08T19:09:24.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-08T19:09:24.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-08T19:09:24.468 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-08T19:09:24.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-08T19:09:24.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-08T19:09:24.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-08T19:09:24.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-08T19:09:24.469 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-08T19:09:24.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-08T19:09:24.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-08T19:09:24.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-08T19:09:24.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-08T19:09:24.470 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-08T19:09:24.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-08T19:09:24.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-08T19:09:24.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-08T19:09:24.471 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-08T19:09:24.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-08T19:09:24.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-08T19:09:24.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-08T19:09:24.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-08T19:09:24.472 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-08T19:09:24.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-08T19:09:24.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-08T19:09:24.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-08T19:09:24.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-08T19:09:24.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-08T19:09:24.473 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-08T19:09:24.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-08T19:09:24.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-08T19:09:24.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-08T19:09:24.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-08T19:09:24.474 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-08T19:09:24.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-08T19:09:24.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-08T19:09:24.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-08T19:09:24.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-08T19:09:24.475 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-08T19:09:24.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-08T19:09:24.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-08T19:09:24.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-08T19:09:24.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-08T19:09:24.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-08T19:09:24.476 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-08T19:09:24.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-08T19:09:24.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-08T19:09:24.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-08T19:09:24.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-08T19:09:24.477 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-08T19:09:24.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-08T19:09:24.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-08T19:09:24.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-08T19:09:24.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-08T19:09:24.478 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-08T19:09:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-08T19:09:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-08T19:09:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-08T19:09:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-08T19:09:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-08T19:09:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-08T19:09:24.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-08T19:09:24.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-08T19:09:24.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-08T19:09:24.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-08T19:09:24.480 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-08T19:09:24.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-08T19:09:24.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-08T19:09:24.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-08T19:09:24.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-08T19:09:24.481 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-08T19:09:24.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-08T19:09:24.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-08T19:09:24.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-08T19:09:24.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-08T19:09:24.482 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-08T19:09:24.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-08T19:09:24.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-08T19:09:24.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-08T19:09:24.483 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-08T19:09:24.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-08T19:09:24.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-08T19:09:24.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-08T19:09:24.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-08T19:09:24.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-08T19:09:24.484 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-08T19:09:24.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-08T19:09:24.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-08T19:09:24.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-08T19:09:24.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-08T19:09:24.485 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-08T19:09:24.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-08T19:09:24.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-08T19:09:24.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-08T19:09:24.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-08T19:09:24.486 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-08T19:09:24.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-08T19:09:24.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-08T19:09:24.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-08T19:09:24.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-08T19:09:24.487 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-08T19:09:24.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-08T19:09:24.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-08T19:09:24.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-08T19:09:24.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-08T19:09:24.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-08T19:09:24.488 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-08T19:09:24.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-08T19:09:24.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-08T19:09:24.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-08T19:09:24.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-08T19:09:24.489 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-08T19:09:24.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-08T19:09:24.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-08T19:09:24.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-08T19:09:24.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-08T19:09:24.490 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-08T19:09:24.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-08T19:09:24.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-08T19:09:24.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-08T19:09:24.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-08T19:09:24.491 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-08T19:09:24.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-08T19:09:24.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-08T19:09:24.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-08T19:09:24.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-08T19:09:24.492 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-08T19:09:24.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-08T19:09:24.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-08T19:09:24.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-08T19:09:24.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-08T19:09:24.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-08T19:09:24.493 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-08T19:09:24.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-08T19:09:24.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-08T19:09:24.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-08T19:09:24.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-08T19:09:24.494 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-08T19:09:24.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-08T19:09:24.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-08T19:09:24.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-08T19:09:24.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-08T19:09:24.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-08T19:09:24.495 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-08T19:09:24.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-08T19:09:24.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-08T19:09:24.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-08T19:09:24.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-08T19:09:24.496 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-08T19:09:24.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-08T19:09:24.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-08T19:09:24.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-08T19:09:24.497 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-08T19:09:24.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-08T19:09:24.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-08T19:09:24.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-08T19:09:24.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-08T19:09:24.498 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-08T19:09:24.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-08T19:09:24.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-08T19:09:24.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-08T19:09:24.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-08T19:09:24.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-08T19:09:24.499 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-08T19:09:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-08T19:09:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-08T19:09:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-08T19:09:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-08T19:09:24.500 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-08T19:09:24.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-08T19:09:24.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-08T19:09:24.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-08T19:09:24.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-08T19:09:24.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-08T19:09:24.501 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-08T19:09:24.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-08T19:09:24.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-08T19:09:24.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-08T19:09:24.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-08T19:09:24.502 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-08T19:09:24.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-08T19:09:24.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-08T19:09:24.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-08T19:09:24.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-08T19:09:24.503 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-08T19:09:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-08T19:09:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-08T19:09:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-08T19:09:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-08T19:09:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-08T19:09:24.504 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-08T19:09:24.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-08T19:09:24.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-08T19:09:24.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-08T19:09:24.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-08T19:09:24.505 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-08T19:09:24.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-08T19:09:24.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-08T19:09:24.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-08T19:09:24.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-08T19:09:24.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-08T19:09:24.506 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-08T19:09:24.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-08T19:09:24.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-08T19:09:24.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-08T19:09:24.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-08T19:09:24.507 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-08T19:09:24.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-08T19:09:24.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-08T19:09:24.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-08T19:09:24.508 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-08T19:09:24.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-08T19:09:24.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-08T19:09:24.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-08T19:09:24.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-08T19:09:24.509 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-08T19:09:24.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-08T19:09:24.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-08T19:09:24.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-08T19:09:24.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-08T19:09:24.510 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-08T19:09:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-08T19:09:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-08T19:09:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-08T19:09:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-08T19:09:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-08T19:09:24.511 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-08T19:09:24.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-08T19:09:24.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-08T19:09:24.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-08T19:09:24.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-08T19:09:24.512 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-08T19:09:24.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-08T19:09:24.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-08T19:09:24.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-08T19:09:24.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-08T19:09:24.513 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-08T19:09:24.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-08T19:09:24.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-08T19:09:24.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-08T19:09:24.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-08T19:09:24.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-08T19:09:24.514 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-08T19:09:24.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-08T19:09:24.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-08T19:09:24.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-08T19:09:24.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-08T19:09:24.515 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-08T19:09:24.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-08T19:09:24.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-08T19:09:24.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-08T19:09:24.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-08T19:09:24.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-08T19:09:24.516 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-08T19:09:24.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-08T19:09:24.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-08T19:09:24.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-08T19:09:24.517 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-08T19:09:24.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-08T19:09:24.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-08T19:09:24.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-08T19:09:24.518 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-08T19:09:24.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-08T19:09:24.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-08T19:09:24.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-08T19:09:24.519 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-08T19:09:24.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-08T19:09:24.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-08T19:09:24.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-08T19:09:24.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-08T19:09:24.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-08T19:09:24.520 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-08T19:09:24.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-08T19:09:24.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-08T19:09:24.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-08T19:09:24.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-08T19:09:24.521 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-08T19:09:24.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-08T19:09:24.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-08T19:09:24.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-08T19:09:24.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-08T19:09:24.522 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-08T19:09:24.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-08T19:09:24.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-08T19:09:24.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-08T19:09:24.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-08T19:09:24.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-08T19:09:24.523 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-08T19:09:24.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-08T19:09:24.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-08T19:09:24.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-08T19:09:24.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-08T19:09:24.524 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-08T19:09:24.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-08T19:09:24.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-08T19:09:24.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-08T19:09:24.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-08T19:09:24.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-08T19:09:24.525 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-08T19:09:24.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-08T19:09:24.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-08T19:09:24.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-08T19:09:24.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-08T19:09:24.526 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-08T19:09:24.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-08T19:09:24.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-08T19:09:24.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-08T19:09:24.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-08T19:09:24.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-08T19:09:24.527 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-08T19:09:24.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-08T19:09:24.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-08T19:09:24.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-08T19:09:24.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-08T19:09:24.528 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-08T19:09:24.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-08T19:09:24.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-08T19:09:24.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-08T19:09:24.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-08T19:09:24.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-08T19:09:24.529 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-08T19:09:24.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-08T19:09:24.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-08T19:09:24.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-08T19:09:24.530 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-08T19:09:24.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-08T19:09:24.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-08T19:09:24.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-08T19:09:24.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-08T19:09:24.531 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-08T19:09:24.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-08T19:09:24.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-08T19:09:24.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-08T19:09:24.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-08T19:09:24.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-08T19:09:24.532 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-08T19:09:24.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-08T19:09:24.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-08T19:09:24.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-08T19:09:24.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-08T19:09:24.533 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-08T19:09:24.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-08T19:09:24.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-08T19:09:24.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-08T19:09:24.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-08T19:09:24.534 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-08T19:09:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-08T19:09:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-08T19:09:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-08T19:09:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-08T19:09:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-08T19:09:24.535 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-08T19:09:24.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-08T19:09:24.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-08T19:09:24.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-08T19:09:24.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-08T19:09:24.536 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-08T19:09:24.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-08T19:09:24.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-08T19:09:24.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-08T19:09:24.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-08T19:09:24.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-08T19:09:24.537 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-08T19:09:24.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-08T19:09:24.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-08T19:09:24.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-08T19:09:24.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-08T19:09:24.538 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-08T19:09:24.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-08T19:09:24.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-08T19:09:24.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-08T19:09:24.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-08T19:09:24.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-08T19:09:24.539 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-08T19:09:24.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-08T19:09:24.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-08T19:09:24.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-08T19:09:24.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-08T19:09:24.540 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-08T19:09:24.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-08T19:09:24.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-08T19:09:24.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-08T19:09:24.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-08T19:09:24.541 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-08T19:09:24.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-08T19:09:24.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-08T19:09:24.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-08T19:09:24.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-08T19:09:24.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-08T19:09:24.542 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-08T19:09:24.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-08T19:09:24.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-08T19:09:24.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-08T19:09:24.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-08T19:09:24.543 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-08T19:09:24.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-08T19:09:24.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-08T19:09:24.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-08T19:09:24.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-08T19:09:24.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-08T19:09:24.544 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-08T19:09:24.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-08T19:09:24.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-08T19:09:24.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-08T19:09:24.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-08T19:09:24.545 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-08T19:09:24.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-08T19:09:24.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-08T19:09:24.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-08T19:09:24.546 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-08T19:09:24.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-08T19:09:24.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-08T19:09:24.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-08T19:09:24.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-08T19:09:24.547 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-08T19:09:24.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-08T19:09:24.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-08T19:09:24.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-08T19:09:24.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-08T19:09:24.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-08T19:09:24.548 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-08T19:09:24.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-08T19:09:24.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-08T19:09:24.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-08T19:09:24.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-08T19:09:24.549 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-08T19:09:24.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-08T19:09:24.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-08T19:09:24.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-08T19:09:24.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-08T19:09:24.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-08T19:09:24.550 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-08T19:09:24.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-08T19:09:24.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-08T19:09:24.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-08T19:09:24.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-08T19:09:24.551 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-08T19:09:24.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-08T19:09:24.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-08T19:09:24.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-08T19:09:24.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-08T19:09:24.552 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-08T19:09:24.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-08T19:09:24.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-08T19:09:24.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-08T19:09:24.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-08T19:09:24.553 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-08T19:09:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-08T19:09:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-08T19:09:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-08T19:09:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-08T19:09:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-08T19:09:24.554 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-08T19:09:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-08T19:09:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-08T19:09:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-08T19:09:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-08T19:09:24.555 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-08T19:09:24.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-08T19:09:24.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-08T19:09:24.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-08T19:09:24.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-08T19:09:24.556 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-08T19:09:24.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-08T19:09:24.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-08T19:09:24.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-08T19:09:24.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-08T19:09:24.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-08T19:09:24.557 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-08T19:09:24.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-08T19:09:24.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-08T19:09:24.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-08T19:09:24.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-08T19:09:24.558 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-08T19:09:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-08T19:09:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-08T19:09:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-08T19:09:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-08T19:09:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-08T19:09:24.559 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-08T19:09:24.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-08T19:09:24.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-08T19:09:24.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-08T19:09:24.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-08T19:09:24.560 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-08T19:09:24.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-08T19:09:24.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-08T19:09:24.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-08T19:09:24.561 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-08T19:09:24.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-08T19:09:24.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-08T19:09:24.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-08T19:09:24.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-08T19:09:24.562 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-08T19:09:24.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-08T19:09:24.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-08T19:09:24.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-08T19:09:24.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-08T19:09:24.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-08T19:09:24.563 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-08T19:09:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-08T19:09:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-08T19:09:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-08T19:09:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-08T19:09:24.564 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-08T19:09:24.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-08T19:09:24.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-08T19:09:24.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-08T19:09:24.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-08T19:09:24.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-08T19:09:24.565 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-08T19:09:24.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-08T19:09:24.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-08T19:09:24.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-08T19:09:24.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-08T19:09:24.566 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-08T19:09:24.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-08T19:09:24.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-08T19:09:24.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-08T19:09:24.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-08T19:09:24.567 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-08T19:09:24.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-08T19:09:24.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-08T19:09:24.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-08T19:09:24.568 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-08T19:09:24.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-08T19:09:24.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-08T19:09:24.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-08T19:09:24.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-08T19:09:24.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-08T19:09:24.569 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-08T19:09:24.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-08T19:09:24.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-08T19:09:24.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-08T19:09:24.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-08T19:09:24.570 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-08T19:09:24.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-08T19:09:24.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-08T19:09:24.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-08T19:09:24.571 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-08T19:09:24.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-08T19:09:24.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-08T19:09:24.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-08T19:09:24.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-08T19:09:24.572 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-08T19:09:24.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-08T19:09:24.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-08T19:09:24.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-08T19:09:24.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-08T19:09:24.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-08T19:09:24.573 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-08T19:09:24.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-08T19:09:24.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-08T19:09:24.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-08T19:09:24.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-08T19:09:24.574 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-08T19:09:24.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-08T19:09:24.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-08T19:09:24.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-08T19:09:24.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-08T19:09:24.575 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-08T19:09:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-08T19:09:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-08T19:09:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-08T19:09:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-08T19:09:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-08T19:09:24.576 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-08T19:09:24.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-08T19:09:24.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-08T19:09:24.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-08T19:09:24.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-08T19:09:24.577 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-08T19:09:24.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-08T19:09:24.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-08T19:09:24.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-08T19:09:24.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-08T19:09:24.578 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-08T19:09:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-08T19:09:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-08T19:09:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-08T19:09:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-08T19:09:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-08T19:09:24.579 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-08T19:09:24.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-08T19:09:24.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-08T19:09:24.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-08T19:09:24.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-08T19:09:24.580 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-08T19:09:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-08T19:09:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-08T19:09:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-08T19:09:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-08T19:09:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-08T19:09:24.581 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-08T19:09:24.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-08T19:09:24.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-08T19:09:24.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-08T19:09:24.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-08T19:09:24.582 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-08T19:09:24.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-08T19:09:24.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-08T19:09:24.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-08T19:09:24.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-08T19:09:24.583 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-08T19:09:24.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-08T19:09:24.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-08T19:09:24.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-08T19:09:24.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-08T19:09:24.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-08T19:09:24.584 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-08T19:09:24.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-08T19:09:24.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-08T19:09:24.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-08T19:09:24.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-08T19:09:24.585 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-08T19:09:24.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-08T19:09:24.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-08T19:09:24.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-08T19:09:24.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-08T19:09:24.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-08T19:09:24.586 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-08T19:09:24.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-08T19:09:24.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-08T19:09:24.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-08T19:09:24.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-08T19:09:24.587 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-08T19:09:24.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-08T19:09:24.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-08T19:09:24.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-08T19:09:24.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-08T19:09:24.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-08T19:09:24.588 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-08T19:09:24.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-08T19:09:24.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-08T19:09:24.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-08T19:09:24.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-08T19:09:24.589 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-08T19:09:24.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-08T19:09:24.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-08T19:09:24.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-08T19:09:24.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-08T19:09:24.590 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-08T19:09:24.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-08T19:09:24.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-08T19:09:24.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-08T19:09:24.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-08T19:09:24.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-08T19:09:24.591 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-08T19:09:24.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-08T19:09:24.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-08T19:09:24.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-08T19:09:24.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-08T19:09:24.592 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-08T19:09:24.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-08T19:09:24.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-08T19:09:24.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-08T19:09:24.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-08T19:09:24.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-08T19:09:24.593 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-08T19:09:24.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-08T19:09:24.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-08T19:09:24.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-08T19:09:24.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-08T19:09:24.594 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-08T19:09:24.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-08T19:09:24.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-08T19:09:24.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-08T19:09:24.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-08T19:09:24.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-08T19:09:24.595 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-08T19:09:24.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-08T19:09:24.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-08T19:09:24.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-08T19:09:24.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-08T19:09:24.596 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-08T19:09:24.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-08T19:09:24.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-08T19:09:24.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-08T19:09:24.597 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-08T19:09:24.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-08T19:09:24.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-08T19:09:24.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-08T19:09:24.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-08T19:09:24.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-08T19:09:24.598 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-08T19:09:24.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-08T19:09:24.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-08T19:09:24.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-08T19:09:24.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-08T19:09:24.599 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-08T19:09:24.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-08T19:09:24.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-08T19:09:24.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-08T19:09:24.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-08T19:09:24.600 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-08T19:09:24.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-08T19:09:24.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-08T19:09:24.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-08T19:09:24.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-08T19:09:24.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-08T19:09:24.601 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-08T19:09:24.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-08T19:09:24.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-08T19:09:24.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-08T19:09:24.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-08T19:09:24.602 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-08T19:09:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-08T19:09:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-08T19:09:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-08T19:09:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-08T19:09:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-08T19:09:24.603 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-08T19:09:24.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-08T19:09:24.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-08T19:09:24.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-08T19:09:24.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-08T19:09:24.604 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-08T19:09:24.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-08T19:09:24.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-08T19:09:24.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-08T19:09:24.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-08T19:09:24.605 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-08T19:09:24.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-08T19:09:24.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-08T19:09:24.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-08T19:09:24.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-08T19:09:24.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-08T19:09:24.606 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-08T19:09:24.607 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-08T19:09:24.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-08T19:09:24.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-08T19:09:24.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-08T19:09:24.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-08T19:09:24.608 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-08T19:09:24.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-08T19:09:24.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-08T19:09:24.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-08T19:09:24.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-08T19:09:24.609 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-08T19:09:24.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-08T19:09:24.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-08T19:09:24.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-08T19:09:24.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-08T19:09:24.610 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-08T19:09:24.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-08T19:09:24.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-08T19:09:24.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-08T19:09:24.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-08T19:09:24.611 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-08T19:09:24.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-08T19:09:24.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-08T19:09:24.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-08T19:09:24.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-08T19:09:24.612 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-08T19:09:24.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-08T19:09:24.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-08T19:09:24.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-08T19:09:24.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-08T19:09:24.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-08T19:09:24.613 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-08T19:09:24.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-08T19:09:24.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-08T19:09:24.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-08T19:09:24.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-08T19:09:24.614 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-08T19:09:24.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-08T19:09:24.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-08T19:09:24.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-08T19:09:24.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-08T19:09:24.615 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-08T19:09:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-08T19:09:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-08T19:09:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-08T19:09:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-08T19:09:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-08T19:09:24.616 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-08T19:09:24.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-08T19:09:24.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-08T19:09:24.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-08T19:09:24.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-08T19:09:24.617 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-08T19:09:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-08T19:09:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-08T19:09:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-08T19:09:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-08T19:09:24.618 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-08T19:09:24.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-08T19:09:24.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-08T19:09:24.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-08T19:09:24.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-08T19:09:24.619 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-08T19:09:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-08T19:09:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-08T19:09:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-08T19:09:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-08T19:09:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-08T19:09:24.620 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-08T19:09:24.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-08T19:09:24.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-08T19:09:24.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-08T19:09:24.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-08T19:09:24.621 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-08T19:09:24.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-08T19:09:24.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-08T19:09:24.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-08T19:09:24.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-08T19:09:24.622 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-08T19:09:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-08T19:09:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-08T19:09:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-08T19:09:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-08T19:09:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-08T19:09:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-08T19:09:24.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-08T19:09:24.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-08T19:09:24.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-08T19:09:24.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-08T19:09:24.624 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-08T19:09:24.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-08T19:09:24.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-08T19:09:24.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-08T19:09:24.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-08T19:09:24.625 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-08T19:09:24.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-08T19:09:24.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-08T19:09:24.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-08T19:09:24.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-08T19:09:24.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-08T19:09:24.626 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-08T19:09:24.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-08T19:09:24.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-08T19:09:24.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-08T19:09:24.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-08T19:09:24.627 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-08T19:09:24.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-08T19:09:24.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-08T19:09:24.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-08T19:09:24.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-08T19:09:24.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-08T19:09:24.628 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-08T19:09:24.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-08T19:09:24.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-08T19:09:24.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-08T19:09:24.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-08T19:09:24.629 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-08T19:09:24.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-08T19:09:24.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-08T19:09:24.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-08T19:09:24.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-08T19:09:24.630 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-08T19:09:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-08T19:09:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-08T19:09:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-08T19:09:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-08T19:09:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-08T19:09:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-08T19:09:24.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-08T19:09:24.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-08T19:09:24.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-08T19:09:24.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-08T19:09:24.632 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-08T19:09:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-08T19:09:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-08T19:09:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-08T19:09:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-08T19:09:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-08T19:09:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-08T19:09:24.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-08T19:09:24.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-08T19:09:24.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-08T19:09:24.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-08T19:09:24.634 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-08T19:09:24.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-08T19:09:24.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-08T19:09:24.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-08T19:09:24.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-08T19:09:24.635 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-08T19:09:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-08T19:09:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-08T19:09:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-08T19:09:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-08T19:09:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-08T19:09:24.636 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-08T19:09:24.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-08T19:09:24.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-08T19:09:24.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-08T19:09:24.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-08T19:09:24.637 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-08T19:09:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-08T19:09:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-08T19:09:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-08T19:09:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-08T19:09:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-08T19:09:24.638 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-08T19:09:24.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-08T19:09:24.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-08T19:09:24.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-08T19:09:24.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-08T19:09:24.639 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-08T19:09:24.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-08T19:09:24.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-08T19:09:24.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-08T19:09:24.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-08T19:09:24.640 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-08T19:09:24.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-08T19:09:24.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-08T19:09:24.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-08T19:09:24.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-08T19:09:24.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-08T19:09:24.641 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-08T19:09:24.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-08T19:09:24.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-08T19:09:24.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-08T19:09:24.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-08T19:09:24.642 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-08T19:09:24.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-08T19:09:24.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-08T19:09:24.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-08T19:09:24.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-08T19:09:24.643 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-08T19:09:24.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-08T19:09:24.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-08T19:09:24.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-08T19:09:24.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-08T19:09:24.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-08T19:09:24.644 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-08T19:09:24.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-08T19:09:24.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-08T19:09:24.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-08T19:09:24.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-08T19:09:24.645 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-08T19:09:24.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-08T19:09:24.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-08T19:09:24.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-08T19:09:24.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-08T19:09:24.646 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-08T19:09:24.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-08T19:09:24.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-08T19:09:24.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-08T19:09:24.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-08T19:09:24.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-08T19:09:24.647 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-08T19:09:24.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-08T19:09:24.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-08T19:09:24.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-08T19:09:24.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-08T19:09:24.648 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-08T19:09:24.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-08T19:09:24.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-08T19:09:24.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-08T19:09:24.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-08T19:09:24.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-08T19:09:24.649 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-08T19:09:24.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-08T19:09:24.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-08T19:09:24.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-08T19:09:24.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-08T19:09:24.650 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-08T19:09:24.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-08T19:09:24.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-08T19:09:24.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-08T19:09:24.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-08T19:09:24.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-08T19:09:24.651 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-08T19:09:24.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-08T19:09:24.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-08T19:09:24.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-08T19:09:24.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-08T19:09:24.652 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-08T19:09:24.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-08T19:09:24.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-08T19:09:24.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-08T19:09:24.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-08T19:09:24.653 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-08T19:09:24.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-08T19:09:24.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-08T19:09:24.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-08T19:09:24.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-08T19:09:24.654 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-08T19:09:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-08T19:09:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-08T19:09:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-08T19:09:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-08T19:09:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-08T19:09:24.655 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-08T19:09:24.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-08T19:09:24.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-08T19:09:24.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-08T19:09:24.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-08T19:09:24.656 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-08T19:09:24.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-08T19:09:24.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-08T19:09:24.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-08T19:09:24.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-08T19:09:24.657 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-08T19:09:24.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-08T19:09:24.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-08T19:09:24.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-08T19:09:24.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-08T19:09:24.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-08T19:09:24.658 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-08T19:09:24.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-08T19:09:24.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-08T19:09:24.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-08T19:09:24.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-08T19:09:24.659 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-08T19:09:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-08T19:09:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-08T19:09:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-08T19:09:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-08T19:09:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-08T19:09:24.660 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-08T19:09:24.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-08T19:09:24.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-08T19:09:24.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-08T19:09:24.661 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-08T19:09:24.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-08T19:09:24.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-08T19:09:24.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-08T19:09:24.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-08T19:09:24.662 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-08T19:09:24.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-08T19:09:24.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-08T19:09:24.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-08T19:09:24.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-08T19:09:24.663 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-08T19:09:24.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-08T19:09:24.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-08T19:09:24.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-08T19:09:24.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-08T19:09:24.664 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-08T19:09:24.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-08T19:09:24.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-08T19:09:24.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-08T19:09:24.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-08T19:09:24.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-08T19:09:24.665 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-08T19:09:24.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-08T19:09:24.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-08T19:09:24.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-08T19:09:24.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-08T19:09:24.666 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-08T19:09:24.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-08T19:09:24.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-08T19:09:24.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-08T19:09:24.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-08T19:09:24.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-08T19:09:24.667 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-08T19:09:24.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-08T19:09:24.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-08T19:09:24.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-08T19:09:24.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-08T19:09:24.668 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-08T19:09:24.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-08T19:09:24.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-08T19:09:24.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-08T19:09:24.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-08T19:09:24.669 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-08T19:09:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-08T19:09:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-08T19:09:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-08T19:09:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-08T19:09:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-08T19:09:24.670 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-08T19:09:24.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-08T19:09:24.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-08T19:09:24.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-08T19:09:24.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-08T19:09:24.671 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-08T19:09:24.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-08T19:09:24.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-08T19:09:24.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-08T19:09:24.672 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-08T19:09:24.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-08T19:09:24.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-08T19:09:24.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-08T19:09:24.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-08T19:09:24.673 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-08T19:09:24.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-08T19:09:24.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-08T19:09:24.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-08T19:09:24.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-08T19:09:24.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-08T19:09:24.674 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-08T19:09:24.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-08T19:09:24.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-08T19:09:24.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-08T19:09:24.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-08T19:09:24.675 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-08T19:09:24.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-08T19:09:24.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-08T19:09:24.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-08T19:09:24.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-08T19:09:24.676 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-08T19:09:24.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-08T19:09:24.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-08T19:09:24.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-08T19:09:24.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-08T19:09:24.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-08T19:09:24.677 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-08T19:09:24.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-08T19:09:24.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-08T19:09:24.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-08T19:09:24.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-08T19:09:24.678 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-08T19:09:24.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-08T19:09:24.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-08T19:09:24.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-08T19:09:24.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-08T19:09:24.679 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-08T19:09:24.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-08T19:09:24.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-08T19:09:24.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-08T19:09:24.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-08T19:09:24.680 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-08T19:09:24.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-08T19:09:24.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-08T19:09:24.681 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-08T19:09:24.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-08T19:09:24.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-08T19:09:24.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-08T19:09:24.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-08T19:09:24.682 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-08T19:09:24.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-08T19:09:24.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-08T19:09:24.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-08T19:09:24.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-08T19:09:24.683 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-08T19:09:24.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-08T19:09:24.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-08T19:09:24.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-08T19:09:24.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-08T19:09:24.684 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-08T19:09:24.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-08T19:09:24.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-08T19:09:24.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-08T19:09:24.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-08T19:09:24.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-08T19:09:24.685 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-08T19:09:24.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-08T19:09:24.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-08T19:09:24.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-08T19:09:24.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-08T19:09:24.686 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-08T19:09:24.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-08T19:09:24.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-08T19:09:24.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-08T19:09:24.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-08T19:09:24.687 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-08T19:09:24.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-08T19:09:24.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-08T19:09:24.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-08T19:09:24.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-08T19:09:24.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-08T19:09:24.688 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-08T19:09:24.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-08T19:09:24.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-08T19:09:24.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-08T19:09:24.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-08T19:09:24.689 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-08T19:09:24.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-08T19:09:24.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-08T19:09:24.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-08T19:09:24.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-08T19:09:24.690 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-08T19:09:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-08T19:09:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-08T19:09:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-08T19:09:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-08T19:09:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-08T19:09:24.691 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-08T19:09:24.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-08T19:09:24.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-08T19:09:24.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-08T19:09:24.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-08T19:09:24.692 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-08T19:09:24.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-08T19:09:24.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-08T19:09:24.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-08T19:09:24.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-08T19:09:24.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-08T19:09:24.693 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-08T19:09:24.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-08T19:09:24.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-08T19:09:24.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-08T19:09:24.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-08T19:09:24.694 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-08T19:09:24.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-08T19:09:24.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-08T19:09:24.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-08T19:09:24.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-08T19:09:24.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-08T19:09:24.695 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-08T19:09:24.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-08T19:09:24.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-08T19:09:24.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-08T19:09:24.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-08T19:09:24.696 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-08T19:09:24.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-08T19:09:24.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-08T19:09:24.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-08T19:09:24.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-08T19:09:24.697 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-08T19:09:24.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-08T19:09:24.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-08T19:09:24.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-08T19:09:24.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-08T19:09:24.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-08T19:09:24.698 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-08T19:09:24.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-08T19:09:24.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-08T19:09:24.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-08T19:09:24.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-08T19:09:24.699 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-08T19:09:24.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-08T19:09:24.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-08T19:09:24.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-08T19:09:24.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-08T19:09:24.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-08T19:09:24.700 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-08T19:09:24.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-08T19:09:24.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-08T19:09:24.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-08T19:09:24.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-08T19:09:24.701 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-08T19:09:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-08T19:09:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-08T19:09:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-08T19:09:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-08T19:09:24.702 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-08T19:09:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-08T19:09:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-08T19:09:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-08T19:09:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-08T19:09:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-08T19:09:24.703 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-08T19:09:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-08T19:09:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-08T19:09:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-08T19:09:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-08T19:09:24.704 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-08T19:09:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-08T19:09:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-08T19:09:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-08T19:09:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-08T19:09:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-08T19:09:24.705 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-08T19:09:24.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-08T19:09:24.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-08T19:09:24.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-08T19:09:24.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-08T19:09:24.706 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-08T19:09:24.707 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-08T19:09:24.707 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-11-08T19:09:24.707 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-08T19:09:24.707 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-08T19:09:24.707 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-08T19:09:24.707 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-08T19:09:24.708 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-08T19:09:24.708 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-08T19:09:24.708 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-08T19:09:24.708 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-08T19:09:24.708 INFO:teuthology.orchestra.run.smithi119.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-08T19:09:24.709 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.710 DEBUG:teuthology.orchestra.run.smithi119:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-08T19:09:24.732 INFO:teuthology.orchestra.run.smithi119.stdout:postinstall scriptlet (using /bin/sh): 2023-11-08T19:09:24.732 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-08T19:09:24.732 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:24.733 INFO:teuthology.orchestra.run.smithi119.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:24.733 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.733 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:24.733 INFO:teuthology.orchestra.run.smithi119.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:24.733 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-11-08T19:09:24.734 INFO:teuthology.orchestra.run.smithi119.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-08T19:09:24.734 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-08T19:09:24.734 INFO:teuthology.orchestra.run.smithi119.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-08T19:09:24.734 INFO:teuthology.orchestra.run.smithi119.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-08T19:09:24.734 INFO:teuthology.orchestra.run.smithi119.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.735 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-11-08T19:09:24.735 INFO:teuthology.orchestra.run.smithi119.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-08T19:09:24.735 INFO:teuthology.orchestra.run.smithi119.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-08T19:09:24.735 INFO:teuthology.orchestra.run.smithi119.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.735 INFO:teuthology.orchestra.run.smithi119.stdout:fi 2023-11-08T19:09:24.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo rpm -qi grub2-tools 2023-11-08T19:09:24.811 INFO:teuthology.orchestra.run.smithi119.stdout:Name : grub2-tools 2023-11-08T19:09:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:Epoch : 1 2023-11-08T19:09:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:Version : 2.06 2023-11-08T19:09:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:Release : 68.el9 2023-11-08T19:09:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:Architecture: x86_64 2023-11-08T19:09:24.812 INFO:teuthology.orchestra.run.smithi119.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-08T19:09:24.813 INFO:teuthology.orchestra.run.smithi119.stdout:Group : Unspecified 2023-11-08T19:09:24.813 INFO:teuthology.orchestra.run.smithi119.stdout:Size : 8264254 2023-11-08T19:09:24.813 INFO:teuthology.orchestra.run.smithi119.stdout:License : GPLv3+ 2023-11-08T19:09:24.813 INFO:teuthology.orchestra.run.smithi119.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-08T19:09:24.813 INFO:teuthology.orchestra.run.smithi119.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-08T19:09:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-08T19:09:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-08T19:09:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:Packager : builder@centos.org 2023-11-08T19:09:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:Vendor : CentOS 2023-11-08T19:09:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-08T19:09:24.814 INFO:teuthology.orchestra.run.smithi119.stdout:Summary : Support tools for GRUB. 2023-11-08T19:09:24.815 INFO:teuthology.orchestra.run.smithi119.stdout:Description : 2023-11-08T19:09:24.815 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:09:24.815 INFO:teuthology.orchestra.run.smithi119.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-08T19:09:24.815 INFO:teuthology.orchestra.run.smithi119.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-08T19:09:24.815 INFO:teuthology.orchestra.run.smithi119.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-08T19:09:24.816 INFO:teuthology.orchestra.run.smithi119.stdout:hardware devices. 2023-11-08T19:09:24.816 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:09:24.816 INFO:teuthology.orchestra.run.smithi119.stdout:This subpackage provides tools for support of all platforms. 2023-11-08T19:09:24.817 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-08T19:09:24.817 INFO:teuthology.task.kernel:Updating grub on smithi119 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:24.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-08T19:09:25.409 INFO:teuthology.orchestra.run.smithi119.stderr:Generating grub configuration file ... 2023-11-08T19:09:26.808 INFO:teuthology.orchestra.run.smithi119.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-08T19:09:26.844 INFO:teuthology.orchestra.run.smithi119.stderr:done 2023-11-08T19:09:26.846 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-11-08T19:09:26.862 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.FFJqnQWS2E 2023-11-08T19:09:26.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.FFJqnQWS2E 2023-11-08T19:09:26.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.FFJqnQWS2E 2023-11-08T19:09:27.072 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.FFJqnQWS2E is 6KB 2023-11-08T19:09:27.120 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.FFJqnQWS2E 2023-11-08T19:09:27.139 DEBUG:teuthology.orchestra.run.smithi119:> sudo /bin/ls /boot/loader/entries || true 2023-11-08T19:09:27.211 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-08T19:09:27.212 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-08T19:09:27.212 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-08T19:09:27.212 INFO:teuthology.orchestra.run.smithi119.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-08T19:09:27.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:27.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo shutdown -r now 2023-11-08T19:09:29.408 INFO:teuthology.orchestra.run.smithi188.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-08T19:09:50.584 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-08T19:09:50.585 DEBUG:teuthology.orchestra.run.smithi188:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-08T19:09:50.657 INFO:teuthology.orchestra.run.smithi188.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:50.657 INFO:teuthology.orchestra.run.smithi188.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:50.658 INFO:teuthology.orchestra.run.smithi188.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:50.658 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:50.658 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-08T19:09:50.658 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-08T19:09:50.658 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-08T19:09:50.659 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-08T19:09:50.659 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-08T19:09:50.659 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-08T19:09:50.659 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-08T19:09:50.660 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-08T19:09:50.660 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-08T19:09:50.660 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-08T19:09:50.660 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-08T19:09:50.660 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-08T19:09:50.661 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-08T19:09:50.661 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-08T19:09:50.661 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-08T19:09:50.661 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-08T19:09:50.661 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-08T19:09:50.662 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-08T19:09:50.662 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-08T19:09:50.662 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-08T19:09:50.664 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-08T19:09:50.664 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-08T19:09:50.664 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-08T19:09:50.665 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-08T19:09:50.665 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-08T19:09:50.665 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-08T19:09:50.665 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-08T19:09:50.666 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-08T19:09:50.666 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-08T19:09:50.666 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-08T19:09:50.666 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-08T19:09:50.666 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-08T19:09:50.667 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-08T19:09:50.667 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-08T19:09:50.667 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-08T19:09:50.667 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-08T19:09:50.668 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-08T19:09:50.668 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-08T19:09:50.668 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-08T19:09:50.668 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-08T19:09:50.668 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-08T19:09:50.669 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-08T19:09:50.669 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-08T19:09:50.669 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-08T19:09:50.669 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-08T19:09:50.669 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-08T19:09:50.670 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-08T19:09:50.670 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-08T19:09:50.670 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-08T19:09:50.670 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-08T19:09:50.671 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-08T19:09:50.671 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-08T19:09:50.671 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-08T19:09:50.671 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-08T19:09:50.671 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-08T19:09:50.672 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-08T19:09:50.672 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-08T19:09:50.672 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-08T19:09:50.672 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-08T19:09:50.672 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-08T19:09:50.673 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-08T19:09:50.673 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-08T19:09:50.673 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-08T19:09:50.673 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-08T19:09:50.673 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-08T19:09:50.674 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-08T19:09:50.674 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-08T19:09:50.674 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-08T19:09:50.674 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-08T19:09:50.674 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-08T19:09:50.675 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-08T19:09:50.675 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-08T19:09:50.675 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-08T19:09:50.675 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-08T19:09:50.675 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-08T19:09:50.675 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-08T19:09:50.676 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-08T19:09:50.676 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-08T19:09:50.676 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-08T19:09:50.676 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-08T19:09:50.676 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-08T19:09:50.677 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-08T19:09:50.677 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-08T19:09:50.677 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-08T19:09:50.677 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-08T19:09:50.677 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-08T19:09:50.678 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-08T19:09:50.678 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-08T19:09:50.678 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-08T19:09:50.678 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-08T19:09:50.678 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-08T19:09:50.679 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-08T19:09:50.679 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-08T19:09:50.679 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-08T19:09:50.679 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-08T19:09:50.679 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-08T19:09:50.680 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-08T19:09:50.680 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-08T19:09:50.680 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-08T19:09:50.680 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-08T19:09:50.680 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-08T19:09:50.681 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-08T19:09:50.681 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-08T19:09:50.681 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-08T19:09:50.681 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-08T19:09:50.681 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-08T19:09:50.682 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-08T19:09:50.682 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-08T19:09:50.682 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-08T19:09:50.682 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-08T19:09:50.683 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-08T19:09:50.683 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-08T19:09:50.683 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-08T19:09:50.683 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-08T19:09:50.683 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-08T19:09:50.684 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-08T19:09:50.684 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-08T19:09:50.684 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-08T19:09:50.684 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-08T19:09:50.684 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-08T19:09:50.685 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-08T19:09:50.685 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-08T19:09:50.685 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-08T19:09:50.685 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-08T19:09:50.685 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-08T19:09:50.686 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-08T19:09:50.686 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-08T19:09:50.686 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-08T19:09:50.686 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-08T19:09:50.686 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-08T19:09:50.687 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-08T19:09:50.687 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-08T19:09:50.687 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-08T19:09:50.687 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-08T19:09:50.687 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-08T19:09:50.688 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-08T19:09:50.688 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-08T19:09:50.688 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-08T19:09:50.688 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-08T19:09:50.688 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-08T19:09:50.688 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-08T19:09:50.689 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-08T19:09:50.689 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-08T19:09:50.689 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-08T19:09:50.689 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-08T19:09:50.689 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-08T19:09:50.690 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-08T19:09:50.690 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-08T19:09:50.690 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-08T19:09:50.690 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-08T19:09:50.691 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-08T19:09:50.691 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-08T19:09:50.691 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-08T19:09:50.691 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-08T19:09:50.691 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-08T19:09:50.692 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-08T19:09:50.692 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-08T19:09:50.692 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-08T19:09:50.692 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-08T19:09:50.692 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-08T19:09:50.693 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-08T19:09:50.693 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-08T19:09:50.693 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-08T19:09:50.693 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-08T19:09:50.693 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-08T19:09:50.694 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-08T19:09:50.694 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-08T19:09:50.694 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-08T19:09:50.694 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-08T19:09:50.694 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-08T19:09:50.694 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-08T19:09:50.695 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-08T19:09:50.695 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-08T19:09:50.695 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-08T19:09:50.695 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-08T19:09:50.695 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-08T19:09:50.696 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-08T19:09:50.696 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-08T19:09:50.696 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-08T19:09:50.696 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-08T19:09:50.696 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-08T19:09:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-08T19:09:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-08T19:09:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-08T19:09:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-08T19:09:50.697 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-08T19:09:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-08T19:09:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-08T19:09:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-08T19:09:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-08T19:09:50.698 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-08T19:09:50.699 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-08T19:09:50.699 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-08T19:09:50.699 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-08T19:09:50.699 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-08T19:09:50.699 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-08T19:09:50.700 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-08T19:09:50.700 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-08T19:09:50.700 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-08T19:09:50.700 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-08T19:09:50.700 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-08T19:09:50.700 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-08T19:09:50.701 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-08T19:09:50.701 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-08T19:09:50.701 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-08T19:09:50.701 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-08T19:09:50.701 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-08T19:09:50.702 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-08T19:09:50.702 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-08T19:09:50.702 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-08T19:09:50.702 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-08T19:09:50.702 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-08T19:09:50.703 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-08T19:09:50.703 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-08T19:09:50.703 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-08T19:09:50.703 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-08T19:09:50.703 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-08T19:09:50.704 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-08T19:09:50.704 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-08T19:09:50.704 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-08T19:09:50.704 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-08T19:09:50.704 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-08T19:09:50.705 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-08T19:09:50.705 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-08T19:09:50.705 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-08T19:09:50.705 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-08T19:09:50.705 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-08T19:09:50.705 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-08T19:09:50.706 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-08T19:09:50.706 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-08T19:09:50.706 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-08T19:09:50.706 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-08T19:09:50.707 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-08T19:09:50.707 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-08T19:09:50.707 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-08T19:09:50.707 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-08T19:09:50.707 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-08T19:09:50.708 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-08T19:09:50.708 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-08T19:09:50.708 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-08T19:09:50.708 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-08T19:09:50.708 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-08T19:09:50.709 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-08T19:09:50.709 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-08T19:09:50.709 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-08T19:09:50.710 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-08T19:09:50.710 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-08T19:09:50.710 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-08T19:09:50.710 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-08T19:09:50.710 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-08T19:09:50.711 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-08T19:09:50.711 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-08T19:09:50.711 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-08T19:09:50.711 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-08T19:09:50.711 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-08T19:09:50.712 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-08T19:09:50.712 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-08T19:09:50.712 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-08T19:09:50.712 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-08T19:09:50.712 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-08T19:09:50.713 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-08T19:09:50.713 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-08T19:09:50.713 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-08T19:09:50.713 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-08T19:09:50.713 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-08T19:09:50.714 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-08T19:09:50.714 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-08T19:09:50.714 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-08T19:09:50.714 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-08T19:09:50.714 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-08T19:09:50.715 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-08T19:09:50.715 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-08T19:09:50.715 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-08T19:09:50.715 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-08T19:09:50.715 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-08T19:09:50.716 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-08T19:09:50.716 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-08T19:09:50.716 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-08T19:09:50.716 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-08T19:09:50.716 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-08T19:09:50.717 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-08T19:09:50.717 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-08T19:09:50.717 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-08T19:09:50.718 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-08T19:09:50.718 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-08T19:09:50.718 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-08T19:09:50.718 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-08T19:09:50.718 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-08T19:09:50.719 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-08T19:09:50.719 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-08T19:09:50.719 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-08T19:09:50.719 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-08T19:09:50.720 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-08T19:09:50.720 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-08T19:09:50.720 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-08T19:09:50.720 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-08T19:09:50.720 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-08T19:09:50.721 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-08T19:09:50.721 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-08T19:09:50.721 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-08T19:09:50.721 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-08T19:09:50.721 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-08T19:09:50.722 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-08T19:09:50.722 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-08T19:09:50.722 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-08T19:09:50.722 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-08T19:09:50.722 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-08T19:09:50.723 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-08T19:09:50.723 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-08T19:09:50.723 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-08T19:09:50.723 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-08T19:09:50.724 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-08T19:09:50.724 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-08T19:09:50.724 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-08T19:09:50.724 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-08T19:09:50.724 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-08T19:09:50.725 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-08T19:09:50.725 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-08T19:09:50.725 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-08T19:09:50.725 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-08T19:09:50.725 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-08T19:09:50.726 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-08T19:09:50.726 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-08T19:09:50.726 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-08T19:09:50.726 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-08T19:09:50.727 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-08T19:09:50.727 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-08T19:09:50.727 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-08T19:09:50.727 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-08T19:09:50.727 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-08T19:09:50.728 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-08T19:09:50.728 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-08T19:09:50.728 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-08T19:09:50.728 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-08T19:09:50.728 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-08T19:09:50.729 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-08T19:09:50.729 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-08T19:09:50.729 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-08T19:09:50.729 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-08T19:09:50.729 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-08T19:09:50.730 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-08T19:09:50.730 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-08T19:09:50.730 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-08T19:09:50.730 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-08T19:09:50.731 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-08T19:09:50.731 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-08T19:09:50.731 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-08T19:09:50.731 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-08T19:09:50.731 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-08T19:09:50.732 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-08T19:09:50.732 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-08T19:09:50.732 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-08T19:09:50.732 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-08T19:09:50.732 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-08T19:09:50.733 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-08T19:09:50.733 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-08T19:09:50.733 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-08T19:09:50.733 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-08T19:09:50.733 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-08T19:09:50.734 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-08T19:09:50.734 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-08T19:09:50.734 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-08T19:09:50.734 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-08T19:09:50.734 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-08T19:09:50.735 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-08T19:09:50.735 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-08T19:09:50.735 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-08T19:09:50.735 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-08T19:09:50.735 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-08T19:09:50.735 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-08T19:09:50.736 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-08T19:09:50.736 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-08T19:09:50.736 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-08T19:09:50.736 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-08T19:09:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-08T19:09:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-08T19:09:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-08T19:09:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-08T19:09:50.737 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-08T19:09:50.738 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-08T19:09:50.738 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-08T19:09:50.738 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-08T19:09:50.738 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-08T19:09:50.738 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-08T19:09:50.739 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-08T19:09:50.739 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-08T19:09:50.739 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-08T19:09:50.739 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-08T19:09:50.739 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-08T19:09:50.740 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-08T19:09:50.740 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-08T19:09:50.740 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-08T19:09:50.740 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-08T19:09:50.740 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-08T19:09:50.741 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-08T19:09:50.741 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-08T19:09:50.741 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-08T19:09:50.741 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-08T19:09:50.741 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-08T19:09:50.742 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-08T19:09:50.742 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-08T19:09:50.742 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-08T19:09:50.742 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-08T19:09:50.742 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-08T19:09:50.743 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-08T19:09:50.743 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-08T19:09:50.743 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-08T19:09:50.743 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-08T19:09:50.743 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-08T19:09:50.744 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-08T19:09:50.744 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-08T19:09:50.744 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-08T19:09:50.744 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-08T19:09:50.744 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-08T19:09:50.745 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-08T19:09:50.745 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-08T19:09:50.745 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-08T19:09:50.745 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-08T19:09:50.745 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-08T19:09:50.746 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-08T19:09:50.746 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-08T19:09:50.746 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-08T19:09:50.746 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-08T19:09:50.746 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-08T19:09:50.747 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-08T19:09:50.747 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-08T19:09:50.747 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-08T19:09:50.747 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-08T19:09:50.748 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-08T19:09:50.748 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-08T19:09:50.748 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-08T19:09:50.748 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-08T19:09:50.748 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-08T19:09:50.749 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-08T19:09:50.749 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-08T19:09:50.749 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-08T19:09:50.749 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-08T19:09:50.749 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-08T19:09:50.750 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-08T19:09:50.750 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-08T19:09:50.750 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-08T19:09:50.750 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-08T19:09:50.750 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-08T19:09:50.751 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-08T19:09:50.751 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-08T19:09:50.751 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-08T19:09:50.751 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-08T19:09:50.751 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-08T19:09:50.752 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-08T19:09:50.752 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-08T19:09:50.752 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-08T19:09:50.752 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-08T19:09:50.752 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-08T19:09:50.753 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-08T19:09:50.753 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-08T19:09:50.753 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-08T19:09:50.753 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-08T19:09:50.753 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-08T19:09:50.753 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-08T19:09:50.754 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-08T19:09:50.754 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-08T19:09:50.754 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-08T19:09:50.754 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-08T19:09:50.755 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-08T19:09:50.755 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-08T19:09:50.755 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-08T19:09:50.755 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-08T19:09:50.755 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-08T19:09:50.756 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-08T19:09:50.756 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-08T19:09:50.756 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-08T19:09:50.756 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-08T19:09:50.756 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-08T19:09:50.757 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-08T19:09:50.757 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-08T19:09:50.757 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-08T19:09:50.757 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-08T19:09:50.757 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-08T19:09:50.758 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-08T19:09:50.758 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-08T19:09:50.758 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-08T19:09:50.758 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-08T19:09:50.758 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-08T19:09:50.759 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-08T19:09:50.759 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-08T19:09:50.759 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-08T19:09:50.759 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-08T19:09:50.759 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-08T19:09:50.760 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-08T19:09:50.760 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-08T19:09:50.760 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-08T19:09:50.760 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-08T19:09:50.760 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-08T19:09:50.761 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-08T19:09:50.761 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-08T19:09:50.761 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-08T19:09:50.761 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-08T19:09:50.761 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-08T19:09:50.762 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-08T19:09:50.762 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-08T19:09:50.762 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-08T19:09:50.762 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-08T19:09:50.762 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-08T19:09:50.763 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-08T19:09:50.763 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-08T19:09:50.763 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-08T19:09:50.763 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-08T19:09:50.763 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-08T19:09:50.764 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-08T19:09:50.764 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-08T19:09:50.764 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-08T19:09:50.764 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-08T19:09:50.764 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-08T19:09:50.765 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-08T19:09:50.765 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-08T19:09:50.765 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-08T19:09:50.765 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-08T19:09:50.765 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-08T19:09:50.766 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-08T19:09:50.766 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-08T19:09:50.766 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-08T19:09:50.766 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-08T19:09:50.766 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-08T19:09:50.767 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-08T19:09:50.767 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-08T19:09:50.767 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-08T19:09:50.767 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-08T19:09:50.767 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-08T19:09:50.768 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-08T19:09:50.768 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-08T19:09:50.768 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-08T19:09:50.768 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-08T19:09:50.768 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-08T19:09:50.769 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-08T19:09:50.769 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-08T19:09:50.769 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-08T19:09:50.769 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-08T19:09:50.769 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-08T19:09:50.769 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-08T19:09:50.770 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-08T19:09:50.770 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-08T19:09:50.770 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-08T19:09:50.770 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-08T19:09:50.771 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-08T19:09:50.771 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-08T19:09:50.771 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-08T19:09:50.771 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-08T19:09:50.771 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-08T19:09:50.772 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-08T19:09:50.772 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-08T19:09:50.772 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-08T19:09:50.772 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-08T19:09:50.772 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-08T19:09:50.773 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-08T19:09:50.773 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-08T19:09:50.773 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-08T19:09:50.773 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-08T19:09:50.773 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-08T19:09:50.774 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-08T19:09:50.774 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-08T19:09:50.774 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-08T19:09:50.774 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-08T19:09:50.774 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-08T19:09:50.775 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-08T19:09:50.775 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-08T19:09:50.775 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-08T19:09:50.775 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-08T19:09:50.775 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-08T19:09:50.776 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-08T19:09:50.776 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-08T19:09:50.776 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-08T19:09:50.776 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-08T19:09:50.776 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-08T19:09:50.777 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-08T19:09:50.777 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-08T19:09:50.777 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-08T19:09:50.777 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-08T19:09:50.777 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-08T19:09:50.778 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-08T19:09:50.778 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-08T19:09:50.778 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-08T19:09:50.778 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-08T19:09:50.778 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-08T19:09:50.779 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-08T19:09:50.779 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-08T19:09:50.779 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-08T19:09:50.779 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-08T19:09:50.779 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-08T19:09:50.780 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-08T19:09:50.780 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-08T19:09:50.780 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-08T19:09:50.780 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-08T19:09:50.780 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-08T19:09:50.780 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-08T19:09:50.781 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-08T19:09:50.781 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-08T19:09:50.781 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-08T19:09:50.781 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-08T19:09:50.782 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-08T19:09:50.782 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-08T19:09:50.782 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-08T19:09:50.782 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-08T19:09:50.782 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-08T19:09:50.783 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-08T19:09:50.783 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-08T19:09:50.783 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-08T19:09:50.783 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-08T19:09:50.783 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-08T19:09:50.784 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-08T19:09:50.784 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-08T19:09:50.784 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-08T19:09:50.784 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-08T19:09:50.784 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-08T19:09:50.785 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-08T19:09:50.785 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-08T19:09:50.785 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-08T19:09:50.785 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-08T19:09:50.785 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-08T19:09:50.786 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-08T19:09:50.786 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-08T19:09:50.786 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-08T19:09:50.786 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-08T19:09:50.786 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-08T19:09:50.787 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-08T19:09:50.787 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-08T19:09:50.787 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-08T19:09:50.787 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-08T19:09:50.787 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-08T19:09:50.788 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-08T19:09:50.788 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-08T19:09:50.788 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-08T19:09:50.788 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-08T19:09:50.789 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-08T19:09:50.789 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-08T19:09:50.789 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-08T19:09:50.789 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-08T19:09:50.789 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-08T19:09:50.790 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-08T19:09:50.790 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-08T19:09:50.790 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-08T19:09:50.790 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-08T19:09:50.790 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-08T19:09:50.791 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-08T19:09:50.791 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-08T19:09:50.791 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-08T19:09:50.791 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-08T19:09:50.791 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-08T19:09:50.792 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-08T19:09:50.792 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-08T19:09:50.792 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-08T19:09:50.792 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-08T19:09:50.793 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-08T19:09:50.793 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-08T19:09:50.793 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-08T19:09:50.793 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-08T19:09:50.793 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-08T19:09:50.794 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-08T19:09:50.794 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-08T19:09:50.794 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-08T19:09:50.794 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-08T19:09:50.794 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-08T19:09:50.795 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-08T19:09:50.795 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-08T19:09:50.795 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-08T19:09:50.795 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-08T19:09:50.795 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-08T19:09:50.796 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-08T19:09:50.796 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-08T19:09:50.796 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-08T19:09:50.796 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-08T19:09:50.796 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-08T19:09:50.797 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-08T19:09:50.797 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-08T19:09:50.797 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-08T19:09:50.797 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-08T19:09:50.797 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-08T19:09:50.798 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-08T19:09:50.798 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-08T19:09:50.798 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-08T19:09:50.798 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-08T19:09:50.798 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-08T19:09:50.799 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-08T19:09:50.799 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-08T19:09:50.799 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-08T19:09:50.799 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-08T19:09:50.799 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-08T19:09:50.800 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-08T19:09:50.800 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-08T19:09:50.800 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-08T19:09:50.800 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-08T19:09:50.801 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-08T19:09:50.801 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-08T19:09:50.801 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-08T19:09:50.801 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-08T19:09:50.801 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-08T19:09:50.802 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-08T19:09:50.802 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-08T19:09:50.802 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-08T19:09:50.802 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-08T19:09:50.802 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-08T19:09:50.802 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-08T19:09:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-08T19:09:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-08T19:09:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-08T19:09:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-08T19:09:50.803 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-08T19:09:50.804 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-08T19:09:50.804 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-08T19:09:50.804 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-08T19:09:50.804 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-08T19:09:50.804 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-08T19:09:50.805 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-08T19:09:50.805 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-08T19:09:50.805 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-08T19:09:50.805 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-08T19:09:50.805 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-08T19:09:50.806 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-08T19:09:50.806 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-08T19:09:50.806 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-08T19:09:50.806 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-08T19:09:50.807 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-08T19:09:50.807 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-08T19:09:50.807 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-08T19:09:50.807 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-08T19:09:50.807 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-08T19:09:50.808 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-08T19:09:50.808 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-08T19:09:50.808 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-08T19:09:50.808 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-08T19:09:50.808 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-08T19:09:50.808 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-08T19:09:50.809 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-08T19:09:50.809 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-08T19:09:50.809 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-08T19:09:50.809 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-08T19:09:50.810 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-08T19:09:50.810 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-08T19:09:50.810 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-08T19:09:50.810 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-08T19:09:50.811 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-08T19:09:50.811 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-08T19:09:50.811 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-08T19:09:50.811 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-08T19:09:50.811 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-08T19:09:50.812 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-08T19:09:50.812 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-08T19:09:50.812 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-08T19:09:50.812 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-08T19:09:50.812 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-08T19:09:50.813 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-08T19:09:50.813 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-08T19:09:50.813 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-08T19:09:50.813 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-08T19:09:50.813 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-08T19:09:50.814 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-08T19:09:50.814 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-08T19:09:50.814 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-08T19:09:50.814 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-08T19:09:50.814 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-08T19:09:50.815 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-08T19:09:50.815 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-08T19:09:50.815 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-08T19:09:50.815 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-08T19:09:50.815 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-08T19:09:50.816 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-08T19:09:50.816 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-08T19:09:50.816 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-08T19:09:50.816 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-08T19:09:50.816 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-08T19:09:50.817 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-08T19:09:50.817 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-08T19:09:50.817 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-08T19:09:50.817 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-08T19:09:50.817 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-08T19:09:50.818 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-08T19:09:50.818 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-08T19:09:50.818 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-08T19:09:50.818 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-08T19:09:50.819 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-08T19:09:50.819 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-08T19:09:50.819 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-08T19:09:50.819 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-08T19:09:50.819 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-08T19:09:50.820 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-08T19:09:50.820 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-08T19:09:50.820 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-08T19:09:50.820 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-08T19:09:50.820 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-08T19:09:50.821 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-08T19:09:50.821 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-08T19:09:50.821 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-08T19:09:50.821 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-08T19:09:50.821 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-08T19:09:50.822 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-08T19:09:50.822 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-08T19:09:50.822 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-08T19:09:50.822 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-08T19:09:50.823 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-08T19:09:50.823 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-08T19:09:50.823 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-08T19:09:50.823 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-08T19:09:50.823 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-08T19:09:50.824 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-08T19:09:50.824 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-08T19:09:50.824 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-08T19:09:50.824 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-08T19:09:50.824 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-08T19:09:50.825 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-08T19:09:50.825 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-08T19:09:50.825 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-08T19:09:50.825 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-08T19:09:50.825 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-08T19:09:50.826 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-08T19:09:50.826 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-08T19:09:50.826 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-08T19:09:50.826 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-08T19:09:50.826 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-08T19:09:50.827 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-08T19:09:50.827 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-08T19:09:50.827 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-08T19:09:50.827 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-08T19:09:50.827 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-08T19:09:50.828 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-08T19:09:50.828 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-08T19:09:50.828 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-08T19:09:50.828 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-08T19:09:50.828 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-08T19:09:50.829 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-08T19:09:50.829 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-08T19:09:50.829 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-08T19:09:50.829 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-08T19:09:50.829 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-08T19:09:50.830 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-08T19:09:50.830 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-08T19:09:50.830 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-08T19:09:50.830 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-08T19:09:50.830 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-08T19:09:50.831 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-08T19:09:50.831 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-08T19:09:50.831 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-08T19:09:50.831 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-08T19:09:50.831 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-08T19:09:50.832 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-08T19:09:50.832 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-08T19:09:50.832 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-08T19:09:50.832 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-08T19:09:50.832 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-08T19:09:50.833 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-08T19:09:50.833 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-08T19:09:50.833 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-08T19:09:50.833 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-08T19:09:50.833 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-08T19:09:50.834 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-08T19:09:50.834 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-08T19:09:50.834 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-08T19:09:50.834 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-08T19:09:50.834 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-08T19:09:50.835 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-08T19:09:50.835 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-08T19:09:50.835 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-08T19:09:50.835 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-08T19:09:50.835 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-08T19:09:50.836 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-08T19:09:50.836 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-08T19:09:50.836 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-08T19:09:50.836 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-08T19:09:50.836 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-08T19:09:50.837 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-08T19:09:50.837 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-08T19:09:50.837 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-08T19:09:50.837 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-08T19:09:50.837 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-08T19:09:50.837 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-08T19:09:50.838 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-08T19:09:50.838 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-08T19:09:50.838 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-08T19:09:50.838 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-08T19:09:50.838 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-08T19:09:50.839 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-08T19:09:50.839 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-08T19:09:50.839 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-08T19:09:50.839 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-08T19:09:50.839 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-08T19:09:50.840 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-08T19:09:50.840 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-08T19:09:50.840 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-08T19:09:50.840 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-08T19:09:50.840 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-08T19:09:50.841 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-08T19:09:50.841 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-08T19:09:50.841 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-08T19:09:50.841 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-08T19:09:50.841 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-08T19:09:50.842 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-08T19:09:50.842 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-08T19:09:50.842 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-08T19:09:50.842 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-08T19:09:50.842 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-08T19:09:50.842 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-08T19:09:50.843 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-08T19:09:50.843 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-08T19:09:50.843 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-08T19:09:50.843 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-08T19:09:50.843 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-08T19:09:50.844 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-08T19:09:50.844 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-08T19:09:50.844 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-08T19:09:50.844 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-08T19:09:50.844 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-08T19:09:50.845 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-08T19:09:50.845 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-08T19:09:50.845 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-08T19:09:50.845 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-08T19:09:50.845 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-08T19:09:50.846 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-08T19:09:50.846 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-08T19:09:50.846 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-08T19:09:50.846 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-08T19:09:50.846 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-08T19:09:50.847 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-08T19:09:50.847 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-08T19:09:50.847 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-08T19:09:50.847 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-08T19:09:50.847 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-08T19:09:50.848 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-08T19:09:50.848 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-08T19:09:50.848 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-08T19:09:50.848 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-08T19:09:50.848 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-08T19:09:50.849 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-08T19:09:50.849 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-08T19:09:50.849 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-08T19:09:50.849 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-08T19:09:50.849 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-08T19:09:50.850 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-08T19:09:50.850 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-08T19:09:50.850 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-08T19:09:50.850 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-08T19:09:50.850 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-08T19:09:50.851 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-08T19:09:50.851 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-08T19:09:50.851 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-08T19:09:50.851 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-08T19:09:50.851 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-08T19:09:50.851 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-08T19:09:50.852 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-08T19:09:50.852 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-08T19:09:50.852 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-08T19:09:50.852 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-08T19:09:50.852 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-08T19:09:50.853 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-08T19:09:50.853 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-08T19:09:50.853 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-08T19:09:50.853 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-08T19:09:50.853 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-08T19:09:50.854 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-08T19:09:50.854 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-08T19:09:50.854 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-08T19:09:50.854 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-08T19:09:50.854 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-08T19:09:50.855 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-08T19:09:50.855 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-08T19:09:50.855 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-08T19:09:50.855 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-08T19:09:50.855 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-08T19:09:50.856 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-08T19:09:50.856 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-08T19:09:50.856 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-08T19:09:50.856 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-08T19:09:50.856 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-08T19:09:50.857 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-08T19:09:50.857 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-08T19:09:50.857 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-08T19:09:50.857 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-08T19:09:50.857 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-08T19:09:50.858 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-08T19:09:50.858 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-08T19:09:50.858 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-08T19:09:50.858 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-08T19:09:50.858 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-08T19:09:50.859 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-08T19:09:50.859 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-08T19:09:50.859 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-08T19:09:50.859 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-08T19:09:50.859 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-08T19:09:50.860 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-08T19:09:50.860 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-08T19:09:50.860 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-08T19:09:50.860 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-08T19:09:50.860 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-08T19:09:50.860 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-08T19:09:50.861 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-08T19:09:50.861 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-08T19:09:50.861 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-08T19:09:50.861 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-08T19:09:50.861 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-08T19:09:50.862 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-08T19:09:50.862 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-08T19:09:50.862 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-08T19:09:50.862 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-08T19:09:50.862 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-08T19:09:50.863 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-08T19:09:50.863 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-08T19:09:50.863 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-08T19:09:50.863 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-08T19:09:50.863 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-08T19:09:50.864 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-08T19:09:50.864 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-08T19:09:50.864 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-08T19:09:50.864 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-08T19:09:50.864 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-08T19:09:50.865 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-08T19:09:50.865 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-08T19:09:50.865 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-08T19:09:50.865 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-08T19:09:50.865 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-08T19:09:50.866 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-08T19:09:50.866 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-08T19:09:50.866 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-08T19:09:50.866 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-08T19:09:50.866 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-08T19:09:50.867 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-08T19:09:50.867 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-08T19:09:50.867 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-08T19:09:50.867 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-08T19:09:50.867 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-08T19:09:50.868 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-08T19:09:50.868 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-08T19:09:50.868 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-08T19:09:50.868 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-08T19:09:50.868 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-08T19:09:50.869 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-08T19:09:50.869 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-08T19:09:50.869 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-08T19:09:50.869 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-08T19:09:50.869 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-08T19:09:50.869 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-08T19:09:50.870 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-08T19:09:50.870 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-08T19:09:50.870 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-08T19:09:50.870 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-08T19:09:50.870 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-08T19:09:50.871 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-08T19:09:50.871 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-08T19:09:50.871 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-08T19:09:50.871 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-08T19:09:50.871 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-08T19:09:50.872 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-08T19:09:50.872 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-08T19:09:50.872 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-08T19:09:50.872 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-08T19:09:50.872 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-08T19:09:50.873 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-08T19:09:50.873 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-08T19:09:50.873 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-08T19:09:50.873 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-08T19:09:50.873 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-08T19:09:50.874 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-08T19:09:50.874 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-08T19:09:50.874 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-08T19:09:50.874 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-08T19:09:50.874 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-08T19:09:50.875 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-08T19:09:50.875 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-08T19:09:50.875 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-08T19:09:50.875 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-08T19:09:50.875 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-08T19:09:50.876 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-08T19:09:50.876 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-08T19:09:50.876 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-08T19:09:50.876 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-08T19:09:50.876 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-08T19:09:50.877 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-08T19:09:50.877 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-08T19:09:50.877 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-08T19:09:50.877 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-08T19:09:50.877 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-08T19:09:50.878 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-08T19:09:50.878 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-08T19:09:50.878 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-08T19:09:50.878 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-08T19:09:50.878 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-08T19:09:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-08T19:09:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-08T19:09:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-08T19:09:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-08T19:09:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-08T19:09:50.879 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-08T19:09:50.880 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-08T19:09:50.880 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-08T19:09:50.880 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-08T19:09:50.880 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-08T19:09:50.881 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-08T19:09:50.881 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-08T19:09:50.881 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-08T19:09:50.881 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-08T19:09:50.881 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-08T19:09:50.882 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-08T19:09:50.882 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-08T19:09:50.882 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-08T19:09:50.882 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-08T19:09:50.882 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-08T19:09:50.883 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-08T19:09:50.883 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-08T19:09:50.883 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-08T19:09:50.883 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-08T19:09:50.883 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-08T19:09:50.883 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-08T19:09:50.884 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-08T19:09:50.884 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-08T19:09:50.884 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-08T19:09:50.884 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-08T19:09:50.884 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-08T19:09:50.885 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-08T19:09:50.885 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-08T19:09:50.885 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-08T19:09:50.885 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-08T19:09:50.886 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-08T19:09:50.886 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-08T19:09:50.886 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-08T19:09:50.886 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-08T19:09:50.886 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-08T19:09:50.887 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-08T19:09:50.887 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-08T19:09:50.887 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-08T19:09:50.887 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-08T19:09:50.887 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-08T19:09:50.888 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-08T19:09:50.888 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-08T19:09:50.888 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-08T19:09:50.888 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-08T19:09:50.888 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-08T19:09:50.889 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-08T19:09:50.889 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-08T19:09:50.889 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-08T19:09:50.889 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-08T19:09:50.889 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-08T19:09:50.889 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-08T19:09:50.890 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-08T19:09:50.890 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-08T19:09:50.890 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-08T19:09:50.890 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-08T19:09:50.891 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-08T19:09:50.891 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-08T19:09:50.891 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-08T19:09:50.891 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-08T19:09:50.891 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-08T19:09:50.892 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-08T19:09:50.892 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-08T19:09:50.892 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-08T19:09:50.892 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-08T19:09:50.892 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-08T19:09:50.893 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-08T19:09:50.893 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-08T19:09:50.893 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-08T19:09:50.893 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-08T19:09:50.893 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-08T19:09:50.894 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-08T19:09:50.894 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-08T19:09:50.894 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-08T19:09:50.894 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-08T19:09:50.894 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-08T19:09:50.894 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-08T19:09:50.895 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-08T19:09:50.895 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-08T19:09:50.895 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-08T19:09:50.895 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-08T19:09:50.895 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-08T19:09:50.896 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-08T19:09:50.896 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-08T19:09:50.896 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-08T19:09:50.896 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-08T19:09:50.896 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-08T19:09:50.897 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-08T19:09:50.897 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-08T19:09:50.897 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-08T19:09:50.897 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-08T19:09:50.897 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-08T19:09:50.898 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-08T19:09:50.898 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-08T19:09:50.898 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-08T19:09:50.898 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-08T19:09:50.898 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-08T19:09:50.899 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-08T19:09:50.899 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-08T19:09:50.899 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-08T19:09:50.899 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-08T19:09:50.899 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-08T19:09:50.899 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-08T19:09:50.900 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-08T19:09:50.900 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-08T19:09:50.900 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-08T19:09:50.900 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-08T19:09:50.900 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-08T19:09:50.901 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-08T19:09:50.901 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-08T19:09:50.901 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-08T19:09:50.901 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-08T19:09:50.901 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-08T19:09:50.902 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-08T19:09:50.902 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-08T19:09:50.902 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-08T19:09:50.902 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-08T19:09:50.902 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-08T19:09:50.903 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-08T19:09:50.903 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-08T19:09:50.903 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-08T19:09:50.903 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-08T19:09:50.903 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-08T19:09:50.904 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-08T19:09:50.904 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-08T19:09:50.904 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-08T19:09:50.904 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-08T19:09:50.904 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-08T19:09:50.905 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-08T19:09:50.905 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-08T19:09:50.905 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-08T19:09:50.905 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-08T19:09:50.905 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-08T19:09:50.906 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-08T19:09:50.906 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-08T19:09:50.906 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-08T19:09:50.906 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-08T19:09:50.906 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-08T19:09:50.907 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-08T19:09:50.907 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-08T19:09:50.907 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-08T19:09:50.907 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-08T19:09:50.907 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-08T19:09:50.908 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-08T19:09:50.908 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-08T19:09:50.908 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-08T19:09:50.908 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-08T19:09:50.908 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-08T19:09:50.909 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-08T19:09:50.909 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-08T19:09:50.909 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-08T19:09:50.909 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-08T19:09:50.909 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-08T19:09:50.910 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-08T19:09:50.910 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-08T19:09:50.910 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-08T19:09:50.910 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-08T19:09:50.910 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-08T19:09:50.911 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-08T19:09:50.911 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-08T19:09:50.911 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-08T19:09:50.911 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-08T19:09:50.911 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-08T19:09:50.912 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-08T19:09:50.912 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-08T19:09:50.912 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-08T19:09:50.912 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-08T19:09:50.912 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-08T19:09:50.913 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-08T19:09:50.913 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-08T19:09:50.913 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-08T19:09:50.913 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-08T19:09:50.913 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-08T19:09:50.914 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-08T19:09:50.914 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-08T19:09:50.914 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-08T19:09:50.914 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-08T19:09:50.914 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-08T19:09:50.915 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-08T19:09:50.915 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-08T19:09:50.915 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-08T19:09:50.915 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-08T19:09:50.915 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-08T19:09:50.916 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-08T19:09:50.916 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-08T19:09:50.916 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-08T19:09:50.916 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-08T19:09:50.916 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-08T19:09:50.917 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-08T19:09:50.917 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-08T19:09:50.917 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-08T19:09:50.917 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-08T19:09:50.917 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-08T19:09:50.917 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-08T19:09:50.918 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-08T19:09:50.918 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-08T19:09:50.918 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-08T19:09:50.918 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-08T19:09:50.919 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-08T19:09:50.919 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-08T19:09:50.919 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-08T19:09:50.919 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-08T19:09:50.919 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-08T19:09:50.920 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-08T19:09:50.920 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-08T19:09:50.920 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-08T19:09:50.920 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-08T19:09:50.920 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-08T19:09:50.920 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-08T19:09:50.921 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-08T19:09:50.921 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-08T19:09:50.921 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-08T19:09:50.921 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-08T19:09:50.922 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-08T19:09:50.922 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-08T19:09:50.922 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-08T19:09:50.922 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-08T19:09:50.922 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-08T19:09:50.923 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-08T19:09:50.923 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-08T19:09:50.923 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-08T19:09:50.923 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-08T19:09:50.923 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-08T19:09:50.924 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-08T19:09:50.924 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-08T19:09:50.924 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-08T19:09:50.924 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-08T19:09:50.924 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-08T19:09:50.925 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-08T19:09:50.925 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-08T19:09:50.925 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-08T19:09:50.925 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-08T19:09:50.926 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-08T19:09:50.926 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-08T19:09:50.926 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-08T19:09:50.926 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-08T19:09:50.926 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-08T19:09:50.927 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-08T19:09:50.927 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-08T19:09:50.927 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-08T19:09:50.927 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-08T19:09:50.927 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-08T19:09:50.928 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-08T19:09:50.928 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-08T19:09:50.928 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-08T19:09:50.928 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-08T19:09:50.928 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-08T19:09:50.929 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-08T19:09:50.929 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-08T19:09:50.929 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-08T19:09:50.929 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-08T19:09:50.930 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-08T19:09:50.930 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-08T19:09:50.930 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-08T19:09:50.930 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-08T19:09:50.930 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-08T19:09:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-08T19:09:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-08T19:09:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-08T19:09:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-08T19:09:50.931 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-08T19:09:50.932 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-08T19:09:50.932 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-08T19:09:50.932 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-08T19:09:50.932 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-08T19:09:50.932 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-08T19:09:50.933 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-08T19:09:50.933 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-08T19:09:50.933 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-08T19:09:50.933 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-08T19:09:50.933 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-08T19:09:50.934 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-08T19:09:50.934 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-08T19:09:50.934 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-08T19:09:50.934 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-08T19:09:50.934 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-08T19:09:50.935 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-08T19:09:50.935 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-08T19:09:50.935 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-08T19:09:50.935 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-08T19:09:50.935 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-08T19:09:50.936 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-08T19:09:50.936 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-08T19:09:50.936 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-08T19:09:50.936 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-08T19:09:50.936 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-08T19:09:50.937 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-08T19:09:50.937 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-08T19:09:50.937 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-08T19:09:50.937 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-08T19:09:50.937 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-08T19:09:50.938 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-08T19:09:50.938 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-08T19:09:50.938 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-08T19:09:50.938 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-08T19:09:50.938 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-08T19:09:50.939 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-08T19:09:50.939 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-08T19:09:50.939 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-08T19:09:50.939 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-08T19:09:50.939 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-08T19:09:50.940 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-08T19:09:50.940 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-08T19:09:50.940 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-08T19:09:50.940 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-08T19:09:50.940 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-08T19:09:50.940 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-08T19:09:50.941 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-08T19:09:50.941 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-08T19:09:50.941 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-08T19:09:50.941 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-08T19:09:50.941 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-08T19:09:50.942 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-08T19:09:50.942 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-08T19:09:50.942 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-08T19:09:50.942 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-08T19:09:50.942 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-08T19:09:50.943 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-08T19:09:50.943 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-08T19:09:50.943 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-08T19:09:50.943 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-08T19:09:50.943 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-08T19:09:50.944 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-08T19:09:50.944 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-08T19:09:50.944 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-08T19:09:50.944 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-08T19:09:50.944 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-08T19:09:50.944 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-08T19:09:50.945 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-08T19:09:50.945 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-08T19:09:50.945 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-08T19:09:50.945 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-08T19:09:50.946 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-08T19:09:50.946 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-08T19:09:50.946 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-08T19:09:50.946 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-08T19:09:50.946 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-08T19:09:50.947 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-08T19:09:50.947 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-08T19:09:50.947 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-08T19:09:50.947 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-08T19:09:50.947 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-08T19:09:50.948 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-08T19:09:50.948 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-08T19:09:50.948 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-08T19:09:50.948 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-08T19:09:50.948 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-08T19:09:50.948 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-08T19:09:50.949 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-08T19:09:50.949 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-08T19:09:50.949 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-08T19:09:50.949 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-08T19:09:50.949 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-08T19:09:50.950 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-08T19:09:50.950 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-08T19:09:50.950 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-08T19:09:50.950 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-08T19:09:50.950 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-08T19:09:50.951 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-08T19:09:50.951 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-08T19:09:50.951 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-08T19:09:50.951 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-08T19:09:50.952 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-08T19:09:50.952 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-08T19:09:50.952 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-08T19:09:50.952 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-08T19:09:50.952 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-08T19:09:50.952 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-08T19:09:50.953 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-08T19:09:50.953 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-08T19:09:50.953 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-08T19:09:50.953 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-08T19:09:50.953 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-08T19:09:50.954 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-08T19:09:50.954 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-08T19:09:50.954 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-08T19:09:50.954 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-08T19:09:50.954 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-08T19:09:50.955 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-08T19:09:50.955 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-08T19:09:50.955 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-08T19:09:50.955 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-08T19:09:50.955 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-08T19:09:50.956 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-08T19:09:50.956 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-08T19:09:50.956 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-08T19:09:50.956 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-08T19:09:50.956 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-08T19:09:50.957 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-08T19:09:50.957 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-08T19:09:50.957 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-08T19:09:50.957 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-08T19:09:50.957 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-08T19:09:50.958 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-08T19:09:50.958 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-08T19:09:50.958 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-08T19:09:50.958 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-08T19:09:50.958 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-08T19:09:50.958 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-08T19:09:50.959 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-08T19:09:50.959 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-08T19:09:50.959 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-08T19:09:50.959 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-08T19:09:50.959 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-08T19:09:50.960 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-08T19:09:50.960 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-08T19:09:50.960 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-08T19:09:50.960 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-08T19:09:50.960 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-08T19:09:50.961 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-08T19:09:50.961 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-08T19:09:50.961 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-08T19:09:50.961 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-08T19:09:50.961 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-08T19:09:50.962 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-08T19:09:50.962 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-08T19:09:50.962 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-08T19:09:50.962 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-08T19:09:50.962 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-08T19:09:50.963 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-08T19:09:50.963 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-08T19:09:50.963 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-08T19:09:50.963 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-08T19:09:50.963 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-08T19:09:50.964 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-08T19:09:50.964 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-08T19:09:50.964 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-08T19:09:50.964 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-08T19:09:50.964 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-08T19:09:50.965 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-08T19:09:50.965 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-08T19:09:50.965 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-08T19:09:50.965 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-08T19:09:50.965 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-08T19:09:50.966 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-08T19:09:50.966 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-08T19:09:50.966 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-08T19:09:50.966 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-08T19:09:50.966 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-08T19:09:50.967 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-08T19:09:50.967 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-08T19:09:50.967 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-08T19:09:50.967 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-08T19:09:50.967 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-08T19:09:50.968 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-08T19:09:50.968 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-08T19:09:50.968 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-08T19:09:50.968 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-08T19:09:50.968 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-08T19:09:50.968 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-08T19:09:50.969 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-08T19:09:50.969 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-08T19:09:50.969 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-08T19:09:50.969 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-08T19:09:50.969 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-08T19:09:50.970 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-08T19:09:50.970 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-08T19:09:50.970 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-08T19:09:50.970 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-08T19:09:50.970 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-08T19:09:50.971 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-08T19:09:50.971 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-08T19:09:50.971 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-08T19:09:50.971 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-08T19:09:50.971 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-08T19:09:50.971 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-08T19:09:50.972 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-08T19:09:50.972 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-08T19:09:50.972 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-08T19:09:50.972 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-08T19:09:50.972 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-08T19:09:50.973 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-08T19:09:50.973 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-08T19:09:50.973 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-08T19:09:50.973 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-08T19:09:50.974 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-08T19:09:50.974 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-08T19:09:50.974 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-08T19:09:50.974 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-08T19:09:50.974 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-08T19:09:50.975 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-08T19:09:50.975 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-08T19:09:50.975 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-08T19:09:50.975 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-08T19:09:50.975 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-08T19:09:50.976 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-08T19:09:50.976 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-08T19:09:50.976 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-08T19:09:50.976 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-08T19:09:50.976 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-08T19:09:50.977 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-08T19:09:50.977 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-08T19:09:50.977 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-08T19:09:50.977 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-08T19:09:50.977 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-08T19:09:50.978 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-08T19:09:50.978 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-08T19:09:50.978 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-08T19:09:50.978 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-08T19:09:50.978 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-08T19:09:50.979 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-08T19:09:50.979 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-08T19:09:50.979 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-08T19:09:50.979 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-08T19:09:50.979 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-08T19:09:50.980 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-08T19:09:50.980 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-08T19:09:50.980 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-08T19:09:50.980 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-08T19:09:50.980 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-08T19:09:50.981 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-08T19:09:50.981 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-08T19:09:50.981 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-08T19:09:50.981 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-08T19:09:50.981 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-08T19:09:50.981 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-08T19:09:50.982 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-08T19:09:50.982 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-08T19:09:50.982 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-08T19:09:50.982 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-08T19:09:50.983 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-08T19:09:50.983 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-08T19:09:50.983 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-08T19:09:50.983 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-08T19:09:50.983 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-08T19:09:50.984 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-08T19:09:50.984 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-08T19:09:50.984 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-08T19:09:50.984 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-08T19:09:50.984 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-08T19:09:50.985 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-08T19:09:50.985 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-08T19:09:50.985 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-08T19:09:50.985 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-08T19:09:50.985 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-08T19:09:50.986 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-08T19:09:50.986 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-08T19:09:50.986 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-08T19:09:50.986 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-08T19:09:50.986 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-08T19:09:50.987 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-08T19:09:50.987 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-08T19:09:50.987 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-08T19:09:50.987 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-08T19:09:50.987 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-08T19:09:50.988 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-08T19:09:50.988 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-08T19:09:50.988 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-08T19:09:50.988 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-08T19:09:50.988 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-08T19:09:50.989 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-08T19:09:50.989 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-08T19:09:50.989 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-08T19:09:50.989 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-08T19:09:50.989 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-08T19:09:50.990 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-08T19:09:50.990 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-08T19:09:50.990 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-08T19:09:50.990 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-08T19:09:50.990 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-08T19:09:50.991 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-08T19:09:50.991 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-08T19:09:50.991 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-08T19:09:50.991 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-08T19:09:50.991 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-08T19:09:50.991 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-08T19:09:50.992 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-08T19:09:50.992 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-08T19:09:50.992 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-08T19:09:50.992 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-08T19:09:50.992 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-08T19:09:50.993 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-08T19:09:50.993 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-08T19:09:50.993 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-08T19:09:50.993 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-08T19:09:50.993 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-08T19:09:50.994 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-08T19:09:50.994 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-08T19:09:50.994 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-08T19:09:50.994 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-08T19:09:50.994 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-08T19:09:50.995 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-08T19:09:50.995 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-08T19:09:50.995 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-08T19:09:50.995 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-08T19:09:50.995 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-08T19:09:50.996 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-08T19:09:50.996 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-08T19:09:50.996 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-08T19:09:50.996 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-08T19:09:50.996 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-08T19:09:50.996 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-08T19:09:50.997 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-08T19:09:50.997 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-08T19:09:50.997 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-08T19:09:50.997 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-08T19:09:50.997 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-08T19:09:50.998 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-08T19:09:50.998 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-08T19:09:50.998 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-08T19:09:50.998 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-08T19:09:50.998 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-08T19:09:50.999 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-08T19:09:50.999 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-08T19:09:50.999 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-08T19:09:50.999 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-08T19:09:50.999 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-08T19:09:51.000 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-08T19:09:51.000 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-08T19:09:51.000 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-08T19:09:51.000 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-08T19:09:51.000 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-08T19:09:51.000 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-08T19:09:51.001 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-08T19:09:51.001 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-08T19:09:51.001 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-08T19:09:51.001 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-08T19:09:51.001 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-08T19:09:51.002 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-08T19:09:51.002 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-08T19:09:51.002 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-08T19:09:51.002 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-08T19:09:51.002 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-08T19:09:51.003 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-08T19:09:51.003 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-08T19:09:51.003 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-08T19:09:51.003 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-08T19:09:51.003 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-08T19:09:51.004 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-08T19:09:51.004 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-08T19:09:51.004 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-08T19:09:51.004 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-08T19:09:51.004 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-08T19:09:51.005 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-08T19:09:51.005 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-08T19:09:51.005 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-08T19:09:51.005 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-08T19:09:51.005 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-08T19:09:51.006 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-08T19:09:51.006 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-08T19:09:51.006 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-08T19:09:51.006 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-08T19:09:51.006 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-08T19:09:51.007 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-08T19:09:51.007 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-08T19:09:51.007 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-08T19:09:51.007 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-08T19:09:51.007 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-08T19:09:51.008 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-08T19:09:51.008 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-08T19:09:51.008 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-08T19:09:51.008 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-08T19:09:51.008 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-08T19:09:51.009 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-08T19:09:51.009 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-08T19:09:51.009 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-08T19:09:51.009 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-08T19:09:51.009 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-08T19:09:51.010 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-08T19:09:51.010 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-08T19:09:51.010 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-08T19:09:51.010 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-08T19:09:51.010 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-08T19:09:51.011 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-08T19:09:51.011 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-08T19:09:51.011 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-08T19:09:51.011 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-08T19:09:51.011 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-08T19:09:51.012 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-08T19:09:51.012 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-08T19:09:51.012 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-08T19:09:51.012 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-08T19:09:51.012 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-08T19:09:51.013 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-08T19:09:51.013 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-08T19:09:51.013 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-08T19:09:51.013 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-08T19:09:51.013 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-08T19:09:51.014 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-08T19:09:51.014 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-08T19:09:51.014 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-08T19:09:51.014 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-08T19:09:51.014 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-08T19:09:51.015 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-08T19:09:51.015 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-08T19:09:51.015 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-08T19:09:51.015 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-08T19:09:51.015 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-08T19:09:51.016 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-08T19:09:51.016 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-08T19:09:51.016 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-08T19:09:51.016 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-08T19:09:51.016 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-08T19:09:51.017 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-08T19:09:51.017 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-08T19:09:51.017 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-08T19:09:51.017 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-08T19:09:51.018 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-08T19:09:51.018 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-08T19:09:51.018 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-08T19:09:51.018 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-08T19:09:51.018 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-08T19:09:51.019 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-08T19:09:51.019 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-08T19:09:51.019 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-08T19:09:51.019 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-08T19:09:51.019 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-08T19:09:51.020 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-08T19:09:51.020 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-08T19:09:51.020 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-08T19:09:51.020 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-08T19:09:51.020 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-08T19:09:51.021 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-08T19:09:51.021 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-08T19:09:51.021 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-08T19:09:51.021 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-08T19:09:51.021 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-08T19:09:51.022 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-08T19:09:51.022 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-08T19:09:51.022 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-08T19:09:51.022 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-08T19:09:51.022 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-08T19:09:51.022 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-08T19:09:51.023 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-08T19:09:51.023 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-08T19:09:51.023 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-08T19:09:51.023 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-08T19:09:51.024 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-08T19:09:51.024 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-08T19:09:51.024 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-08T19:09:51.024 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-08T19:09:51.024 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-08T19:09:51.025 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-08T19:09:51.025 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-08T19:09:51.025 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-08T19:09:51.025 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-08T19:09:51.025 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-08T19:09:51.026 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-08T19:09:51.026 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-08T19:09:51.026 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-08T19:09:51.026 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-08T19:09:51.026 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-08T19:09:51.027 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-08T19:09:51.027 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-08T19:09:51.027 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-08T19:09:51.027 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-08T19:09:51.027 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-08T19:09:51.028 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-08T19:09:51.028 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-08T19:09:51.028 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-08T19:09:51.028 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-08T19:09:51.028 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-08T19:09:51.029 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-08T19:09:51.029 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-08T19:09:51.029 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-08T19:09:51.029 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-08T19:09:51.029 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-08T19:09:51.030 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-08T19:09:51.030 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-08T19:09:51.030 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-08T19:09:51.030 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-08T19:09:51.030 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-08T19:09:51.031 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-08T19:09:51.031 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-08T19:09:51.031 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-08T19:09:51.031 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-08T19:09:51.031 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-08T19:09:51.032 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-08T19:09:51.032 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-08T19:09:51.032 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-08T19:09:51.032 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-08T19:09:51.032 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-08T19:09:51.033 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-08T19:09:51.033 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-08T19:09:51.033 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-08T19:09:51.033 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-08T19:09:51.033 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-08T19:09:51.033 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-08T19:09:51.034 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-08T19:09:51.034 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-08T19:09:51.034 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-08T19:09:51.034 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-08T19:09:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-08T19:09:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-08T19:09:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-08T19:09:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-08T19:09:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-08T19:09:51.035 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-08T19:09:51.036 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-08T19:09:51.036 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-08T19:09:51.036 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-08T19:09:51.036 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-08T19:09:51.036 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-08T19:09:51.037 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-08T19:09:51.037 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-08T19:09:51.037 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-08T19:09:51.037 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-08T19:09:51.038 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-08T19:09:51.038 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-08T19:09:51.038 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-08T19:09:51.038 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-08T19:09:51.038 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-08T19:09:51.039 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-08T19:09:51.039 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-08T19:09:51.039 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-08T19:09:51.039 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-08T19:09:51.039 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-08T19:09:51.040 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-08T19:09:51.040 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-08T19:09:51.040 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-08T19:09:51.040 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-08T19:09:51.040 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-08T19:09:51.041 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-08T19:09:51.041 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-08T19:09:51.041 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-08T19:09:51.041 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-08T19:09:51.041 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-08T19:09:51.042 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-08T19:09:51.042 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-08T19:09:51.042 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-08T19:09:51.042 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-08T19:09:51.042 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-08T19:09:51.043 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-08T19:09:51.043 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-08T19:09:51.043 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-08T19:09:51.043 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-08T19:09:51.043 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-08T19:09:51.043 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-08T19:09:51.044 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-08T19:09:51.044 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-08T19:09:51.044 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-08T19:09:51.044 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-08T19:09:51.045 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-08T19:09:51.045 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-08T19:09:51.045 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-08T19:09:51.045 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-08T19:09:51.045 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-08T19:09:51.046 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-08T19:09:51.046 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-08T19:09:51.046 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-08T19:09:51.046 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-08T19:09:51.046 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-08T19:09:51.047 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-08T19:09:51.047 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-08T19:09:51.047 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-08T19:09:51.047 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-08T19:09:51.047 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-08T19:09:51.048 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-08T19:09:51.048 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-08T19:09:51.048 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-08T19:09:51.048 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-11-08T19:09:51.048 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-08T19:09:51.049 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-08T19:09:51.049 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-08T19:09:51.049 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-08T19:09:51.049 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-08T19:09:51.049 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-08T19:09:51.050 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-08T19:09:51.050 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-08T19:09:51.050 INFO:teuthology.orchestra.run.smithi188.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-08T19:09:51.051 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:51.051 DEBUG:teuthology.orchestra.run.smithi188:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-08T19:09:51.074 INFO:teuthology.orchestra.run.smithi188.stdout:postinstall scriptlet (using /bin/sh): 2023-11-08T19:09:51.075 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-08T19:09:51.075 INFO:teuthology.orchestra.run.smithi188.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:51.075 INFO:teuthology.orchestra.run.smithi188.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:51.075 INFO:teuthology.orchestra.run.smithi188.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:51.076 INFO:teuthology.orchestra.run.smithi188.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:51.076 INFO:teuthology.orchestra.run.smithi188.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T19:09:51.076 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2023-11-08T19:09:51.076 INFO:teuthology.orchestra.run.smithi188.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-08T19:09:51.076 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-08T19:09:51.077 INFO:teuthology.orchestra.run.smithi188.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-08T19:09:51.077 INFO:teuthology.orchestra.run.smithi188.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-08T19:09:51.077 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:51.077 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2023-11-08T19:09:51.077 INFO:teuthology.orchestra.run.smithi188.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-08T19:09:51.078 INFO:teuthology.orchestra.run.smithi188.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-08T19:09:51.078 INFO:teuthology.orchestra.run.smithi188.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:51.078 INFO:teuthology.orchestra.run.smithi188.stdout:fi 2023-11-08T19:09:51.079 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2023-11-08T19:09:51.154 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2023-11-08T19:09:51.155 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2023-11-08T19:09:51.155 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2023-11-08T19:09:51.155 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2023-11-08T19:09:51.155 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2023-11-08T19:09:51.156 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-08T19:09:51.156 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2023-11-08T19:09:51.156 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2023-11-08T19:09:51.156 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2023-11-08T19:09:51.157 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-08T19:09:51.157 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-08T19:09:51.157 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-08T19:09:51.157 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-08T19:09:51.157 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2023-11-08T19:09:51.157 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2023-11-08T19:09:51.158 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-08T19:09:51.158 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2023-11-08T19:09:51.158 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2023-11-08T19:09:51.158 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:09:51.158 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-08T19:09:51.159 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-08T19:09:51.159 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-08T19:09:51.159 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2023-11-08T19:09:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:09:51.159 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2023-11-08T19:09:51.160 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-08T19:09:51.161 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:51.161 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-08T19:09:51.785 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2023-11-08T19:09:53.153 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-08T19:09:53.192 INFO:teuthology.orchestra.run.smithi188.stderr:done 2023-11-08T19:09:53.194 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2023-11-08T19:09:53.209 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.o6bKsVfnw3 2023-11-08T19:09:53.210 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.o6bKsVfnw3 2023-11-08T19:09:53.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.o6bKsVfnw3 2023-11-08T19:09:53.424 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.o6bKsVfnw3 is 6KB 2023-11-08T19:09:53.472 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.o6bKsVfnw3 2023-11-08T19:09:53.486 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2023-11-08T19:09:53.552 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-08T19:09:53.552 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-08T19:09:53.552 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-08T19:09:53.552 INFO:teuthology.orchestra.run.smithi188.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-08T19:09:53.553 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:09:53.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2023-11-08T19:09:57.374 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-08T19:09:57.375 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-11-08T19:09:57.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:10:15.930 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-11-08T19:10:23.706 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-08T19:10:23.707 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-08T19:10:23.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:10:24.933 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-11-08T19:10:24.934 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:10:27.994 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.119 2023-11-08T19:10:40.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-11-08T19:10:40.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:10:42.042 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-08T19:10:51.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-08T19:10:51.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:10:54.106 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.188 2023-11-08T19:11:06.118 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-08T19:11:06.119 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:11:11.935 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-11-08T19:11:12.615 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-11-08T19:11:12.616 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-08T19:11:12.616 DEBUG:teuthology.orchestra.run.smithi119:> uname -r 2023-11-08T19:11:12.660 INFO:teuthology.orchestra.run.smithi119.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:11:12.660 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:11:12.661 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-08T19:11:12.661 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-08T19:11:13.663 DEBUG:teuthology.parallel:result is None 2023-11-08T19:11:37.902 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-11-08T19:11:38.605 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-08T19:11:38.606 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-08T19:11:38.606 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2023-11-08T19:11:38.649 INFO:teuthology.orchestra.run.smithi188.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:11:38.650 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-08T19:11:38.650 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-08T19:11:38.650 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-08T19:11:39.651 DEBUG:teuthology.parallel:result is None 2023-11-08T19:11:39.652 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-08T19:11:39.659 INFO:teuthology.task.internal:Creating test directory... 2023-11-08T19:11:39.659 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T19:11:39.670 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T19:11:39.721 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-08T19:11:39.732 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-08T19:11:39.792 INFO:teuthology.task.internal:Creating archive directory... 2023-11-08T19:11:39.793 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T19:11:39.802 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T19:11:39.865 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-08T19:11:39.872 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-08T19:11:39.873 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T19:11:39.881 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T19:11:40.036 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:40.050 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:40.102 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:40.129 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:11:40.134 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-08T19:11:40.141 INFO:teuthology.task.internal:Configuring sudo... 2023-11-08T19:11:40.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T19:11:40.150 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T19:11:40.240 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-08T19:11:40.249 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-08T19:11:40.250 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T19:11:40.259 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T19:11:40.313 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:40.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:40.511 DEBUG:teuthology.orchestra.run.smithi119:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:40.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:40.697 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:11:40.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T19:11:40.801 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:40.883 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T19:11:40.976 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:41.045 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T19:11:41.135 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:11:41.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T19:11:41.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo service rsyslog restart 2023-11-08T19:11:41.220 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2023-11-08T19:11:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:11:41.371 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:11:41.869 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-08T19:11:41.882 INFO:teuthology.task.internal:Starting timer... 2023-11-08T19:11:41.882 INFO:teuthology.run_tasks:Running task pcp... 2023-11-08T19:11:41.897 INFO:teuthology.run_tasks:Running task selinux... 2023-11-08T19:11:41.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo service auditd rotate 2023-11-08T19:11:42.057 INFO:teuthology.orchestra.run.smithi119.stdout:Rotating logs: 2023-11-08T19:11:42.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2023-11-08T19:11:42.211 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2023-11-08T19:11:42.214 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-08T19:11:42.215 DEBUG:teuthology.orchestra.run.smithi119:> /usr/sbin/getenforce 2023-11-08T19:11:42.275 INFO:teuthology.orchestra.run.smithi119.stdout:Permissive 2023-11-08T19:11:42.275 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2023-11-08T19:11:42.340 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2023-11-08T19:11:42.340 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi119.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2023-11-08T19:11:42.341 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:11:42.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:11:42.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:11:42.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:11:42.519 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-08T19:11:42.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo /usr/sbin/setenforce permissive 2023-11-08T19:11:42.649 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2023-11-08T19:11:42.744 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-08T19:11:42.754 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-08T19:11:42.761 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-08T19:11:42.920 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-08T19:11:42.938 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-08T19:11:42.940 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi119.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-08T19:21:36.833 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi119.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2023-11-08T19:21:36.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-11-08T19:21:36.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi119.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:21:36.979 DEBUG:teuthology.orchestra.run.smithi119:> true 2023-11-08T19:21:37.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi119.front.sepia.ceph.com' 2023-11-08T19:21:37.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-08T19:21:37.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T19:21:37.271 DEBUG:teuthology.orchestra.run.smithi188:> true 2023-11-08T19:21:37.423 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2023-11-08T19:21:37.424 INFO:teuthology.run_tasks:Running task clock... 2023-11-08T19:21:37.435 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-08T19:21:37.436 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T19:21:37.436 DEBUG:teuthology.orchestra.run.smithi119:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:21:37.446 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T19:21:37.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:21:37.544 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-08T19:21:37.552 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-08T19:21:37.599 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-08T19:21:37.604 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-08T19:21:37.689 INFO:teuthology.orchestra.run.smithi119.stderr:sudo: ntpd: command not found 2023-11-08T19:21:37.701 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2023-11-08T19:21:37.726 INFO:teuthology.orchestra.run.smithi119.stdout:506 Cannot talk to daemon 2023-11-08T19:21:37.743 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2023-11-08T19:21:37.774 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-08T19:21:37.794 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-08T19:21:37.827 INFO:teuthology.orchestra.run.smithi119.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-08T19:21:37.847 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-08T19:21:37.953 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2023-11-08T19:21:37.963 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:21:37.963 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-11-08T19:21:37.963 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.964 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.964 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.964 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.973 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2023-11-08T19:21:37.981 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:21:37.981 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2023-11-08T19:21:37.981 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.982 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.982 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.982 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:21:37.983 INFO:teuthology.run_tasks:Running task install... 2023-11-08T19:21:37.996 DEBUG:teuthology.task.install:project ceph 2023-11-08T19:21:37.997 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}} 2023-11-08T19:21:37.997 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'} 2023-11-08T19:21:37.998 INFO:teuthology.task.install:Using flavor: default 2023-11-08T19:21:38.004 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-08T19:21:38.005 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-08T19:21:38.005 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'tag': None, 'wait_for_package': False} 2023-11-08T19:21:38.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:21:38.008 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'tag': None, 'wait_for_package': False} 2023-11-08T19:21:38.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:21:38.152 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/ 2023-11-08T19:21:38.153 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7035.g7532011a 2023-11-08T19:21:38.351 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/ 2023-11-08T19:21:38.352 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7035.g7532011a 2023-11-08T19:21:38.503 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-08T19:21:38.503 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:21:38.505 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-08T19:21:38.609 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-08T19:21:38.610 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-08T19:21:38.681 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-08T19:21:38.682 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:21:38.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-08T19:21:38.756 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-08T19:21:38.786 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-08T19:21:38.786 DEBUG:teuthology.orchestra.run.smithi119:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-08T19:21:38.884 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-08T19:21:38.931 DEBUG:teuthology.orchestra.run.smithi119:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-08T19:21:38.977 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2023-11-08T19:21:38.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2023-11-08T19:21:39.068 DEBUG:teuthology.orchestra.run.smithi119:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-08T19:21:39.151 INFO:teuthology.orchestra.run.smithi119.stdout:check_obsoletes = 1 2023-11-08T19:21:39.155 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum clean all 2023-11-08T19:21:39.333 INFO:teuthology.orchestra.run.smithi188.stdout:72 files removed 2023-11-08T19:21:39.372 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-08T19:21:39.499 INFO:teuthology.orchestra.run.smithi119.stdout:72 files removed 2023-11-08T19:21:39.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-08T19:21:40.405 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 150 kB/s | 78 kB 00:00 2023-11-08T19:21:40.774 INFO:teuthology.orchestra.run.smithi119.stdout:ceph packages for x86_64 108 kB/s | 78 kB 00:00 2023-11-08T19:21:41.147 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 15 kB/s | 9.4 kB 00:00 2023-11-08T19:21:41.683 INFO:teuthology.orchestra.run.smithi119.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-11-08T19:21:42.006 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 2.4 kB/s | 1.9 kB 00:00 2023-11-08T19:21:42.559 INFO:teuthology.orchestra.run.smithi119.stdout:ceph source packages 2.3 kB/s | 1.9 kB 00:00 2023-11-08T19:21:42.592 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-11-08T19:21:43.121 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-11-08T19:21:44.237 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS-9 - BaseOS 4.8 MB/s | 2.0 MB 00:00 2023-11-08T19:21:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2023-11-08T19:21:46.325 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 7.9 MB 00:01 2023-11-08T19:21:52.138 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 7.9 MB 00:07 2023-11-08T19:21:55.880 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 7.1 MB/s | 18 MB 00:02 2023-11-08T19:22:00.824 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 4.5 MB/s | 5.8 MB 00:01 2023-11-08T19:22:02.742 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 15 kB 00:00 2023-11-08T19:22:04.186 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 16 MB/s | 19 MB 00:01 2023-11-08T19:22:09.515 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2023-11-08T19:22:11.957 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - AppStream 753 kB/s | 18 MB 00:24 2023-11-08T19:22:12.363 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T19:22:12.365 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T19:22:12.385 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-08T19:22:12.386 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-08T19:22:12.387 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-08T19:22:12.389 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-08T19:22:12.391 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-08T19:22:12.395 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-08T19:22:12.396 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-08T19:22:12.474 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2023-11-08T19:22:12.497 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2023-11-08T19:22:12.497 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2023-11-08T19:22:12.497 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2023-11-08T19:22:12.497 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2023-11-08T19:22:12.498 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-08T19:22:12.498 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:18.0.0-7035.g7532011a.el9 ceph 6.5 k 2023-11-08T19:22:12.498 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:18.0.0-7035.g7532011a.el9 ceph 5.2 M 2023-11-08T19:22:12.498 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 859 k 2023-11-08T19:22:12.499 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7035.g7532011a.el9 ceph 140 k 2023-11-08T19:22:12.499 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:18.0.0-7035.g7532011a.el9 ceph 1.5 M 2023-11-08T19:22:12.499 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 132 k 2023-11-08T19:22:12.499 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 1.8 M 2023-11-08T19:22:12.499 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 7.4 M 2023-11-08T19:22:12.500 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 48 k 2023-11-08T19:22:12.500 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 8.7 M 2023-11-08T19:22:12.500 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:18.0.0-7035.g7532011a.el9 ceph 41 M 2023-11-08T19:22:12.500 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 262 k 2023-11-08T19:22:12.500 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 270 k 2023-11-08T19:22:12.501 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 217 k 2023-11-08T19:22:12.501 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 18 k 2023-11-08T19:22:12.501 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-08T19:22:12.501 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-08T19:22:12.501 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-08T19:22:12.502 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 31 k 2023-11-08T19:22:12.502 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 718 k 2023-11-08T19:22:12.502 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 126 k 2023-11-08T19:22:12.502 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-08T19:22:12.503 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-08T19:22:12.503 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-08T19:22:12.504 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-08T19:22:12.504 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-08T19:22:12.504 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-08T19:22:12.504 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:18.0.0-7035.g7532011a.el9 ceph 162 k 2023-11-08T19:22:12.505 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:18.0.0-7035.g7532011a.el9 ceph 320 k 2023-11-08T19:22:12.505 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 299 k 2023-11-08T19:22:12.505 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 99 k 2023-11-08T19:22:12.505 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 84 k 2023-11-08T19:22:12.505 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-08T19:22:12.506 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 169 k 2023-11-08T19:22:12.506 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-08T19:22:12.506 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2023-11-08T19:22:12.506 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-08T19:22:12.506 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-08T19:22:12.507 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-08T19:22:12.507 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-08T19:22:12.507 INFO:teuthology.orchestra.run.smithi188.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-08T19:22:12.507 INFO:teuthology.orchestra.run.smithi188.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-08T19:22:12.507 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-08T19:22:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.3 M 2023-11-08T19:22:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-08T19:22:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-08T19:22:12.508 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2023-11-08T19:22:12.508 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-08T19:22:12.509 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-08T19:22:12.509 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 19 M 2023-11-08T19:22:12.509 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 23 k 2023-11-08T19:22:12.509 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:18.0.0-7035.g7532011a.el9 ceph 2.1 M 2023-11-08T19:22:12.509 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 243 k 2023-11-08T19:22:12.510 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.4 M 2023-11-08T19:22:12.510 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 18 M 2023-11-08T19:22:12.510 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 15 k 2023-11-08T19:22:12.510 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:18.0.0-7035.g7532011a.el9 ceph 24 k 2023-11-08T19:22:12.510 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-08T19:22:12.511 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-08T19:22:12.513 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-08T19:22:12.513 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-08T19:22:12.514 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-08T19:22:12.514 INFO:teuthology.orchestra.run.smithi188.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-08T19:22:12.514 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-08T19:22:12.514 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-08T19:22:12.514 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:18.0.0-7035.g7532011a.el9 ceph 163 k 2023-11-08T19:22:12.515 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-08T19:22:12.515 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-08T19:22:12.515 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-08T19:22:12.515 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-08T19:22:12.515 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-08T19:22:12.516 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 476 k 2023-11-08T19:22:12.516 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-08T19:22:12.516 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.8 M 2023-11-08T19:22:12.516 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-08T19:22:12.516 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-08T19:22:12.516 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-08T19:22:12.517 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-08T19:22:12.517 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-08T19:22:12.517 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-08T19:22:12.517 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-08T19:22:12.517 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-08T19:22:12.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-08T19:22:12.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-08T19:22:12.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-08T19:22:12.518 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-08T19:22:12.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 45 k 2023-11-08T19:22:12.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 127 k 2023-11-08T19:22:12.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-08T19:22:12.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-08T19:22:12.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-08T19:22:12.519 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-08T19:22:12.520 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-08T19:22:12.520 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-08T19:22:12.520 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-08T19:22:12.520 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-08T19:22:12.520 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-08T19:22:12.521 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-08T19:22:12.521 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-08T19:22:12.521 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-08T19:22:12.521 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-08T19:22:12.521 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-08T19:22:12.522 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-08T19:22:12.522 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-08T19:22:12.522 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-08T19:22:12.522 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-08T19:22:12.522 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-08T19:22:12.523 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-08T19:22:12.523 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-08T19:22:12.523 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-08T19:22:12.523 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-08T19:22:12.523 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-08T19:22:12.524 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-08T19:22:12.524 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-08T19:22:12.524 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-08T19:22:12.524 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-08T19:22:12.524 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-08T19:22:12.525 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-08T19:22:12.525 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-08T19:22:12.525 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-08T19:22:12.525 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-08T19:22:12.525 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-08T19:22:12.525 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-08T19:22:12.526 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-08T19:22:12.526 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-08T19:22:12.526 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-08T19:22:12.526 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-08T19:22:12.527 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-08T19:22:12.527 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-08T19:22:12.527 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-08T19:22:12.527 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-08T19:22:12.527 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-08T19:22:12.528 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-08T19:22:12.528 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-08T19:22:12.528 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-08T19:22:12.528 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-08T19:22:12.528 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-08T19:22:12.528 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-08T19:22:12.529 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-08T19:22:12.529 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-08T19:22:12.529 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-08T19:22:12.529 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-08T19:22:12.529 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2023-11-08T19:22:12.530 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-08T19:22:12.530 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:22:12.530 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2023-11-08T19:22:12.530 INFO:teuthology.orchestra.run.smithi188.stdout:============================================================================================ 2023-11-08T19:22:12.530 INFO:teuthology.orchestra.run.smithi188.stdout:Install 132 Packages 2023-11-08T19:22:12.531 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 10 Packages 2023-11-08T19:22:12.531 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:22:12.532 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 200 M 2023-11-08T19:22:12.532 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2023-11-08T19:22:13.561 INFO:teuthology.orchestra.run.smithi188.stdout:(1/142): ceph-18.0.0-7035.g7532011a.el9.x86_64. 15 kB/s | 6.5 kB 00:00 2023-11-08T19:22:14.186 INFO:teuthology.orchestra.run.smithi188.stdout:(2/142): ceph-fuse-18.0.0-7035.g7532011a.el9.x8 1.3 MB/s | 859 kB 00:00 2023-11-08T19:22:14.620 INFO:teuthology.orchestra.run.smithi188.stdout:(3/142): ceph-base-18.0.0-7035.g7532011a.el9.x8 3.5 MB/s | 5.2 MB 00:01 2023-11-08T19:22:14.962 INFO:teuthology.orchestra.run.smithi188.stdout:(4/142): ceph-mds-18.0.0-7035.g7532011a.el9.x86 6.1 MB/s | 2.1 MB 00:00 2023-11-08T19:22:15.170 INFO:teuthology.orchestra.run.smithi188.stdout:(5/142): ceph-immutable-object-cache-18.0.0-703 143 kB/s | 140 kB 00:00 2023-11-08T19:22:15.412 INFO:teuthology.orchestra.run.smithi188.stdout:(6/142): ceph-common-18.0.0-7035.g7532011a.el9. 8.3 MB/s | 19 MB 00:02 2023-11-08T19:22:15.496 INFO:teuthology.orchestra.run.smithi188.stdout:(7/142): ceph-mgr-18.0.0-7035.g7532011a.el9.x86 2.7 MB/s | 1.5 MB 00:00 2023-11-08T19:22:16.004 INFO:teuthology.orchestra.run.smithi188.stdout:(8/142): ceph-mon-18.0.0-7035.g7532011a.el9.x86 5.3 MB/s | 4.4 MB 00:00 2023-11-08T19:22:16.138 INFO:teuthology.orchestra.run.smithi188.stdout:(9/142): ceph-selinux-18.0.0-7035.g7532011a.el9 182 kB/s | 24 kB 00:00 2023-11-08T19:22:16.829 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - CRB 5.1 MB/s | 5.8 MB 00:01 2023-11-08T19:22:16.971 INFO:teuthology.orchestra.run.smithi188.stdout:(10/142): ceph-osd-18.0.0-7035.g7532011a.el9.x8 11 MB/s | 18 MB 00:01 2023-11-08T19:22:17.147 INFO:teuthology.orchestra.run.smithi188.stdout:(11/142): ceph-radosgw-18.0.0-7035.g7532011a.el 5.3 MB/s | 8.7 MB 00:01 2023-11-08T19:22:17.180 INFO:teuthology.orchestra.run.smithi188.stdout:(12/142): cephfs-mirror-18.0.0-7035.g7532011a.e 1.0 MB/s | 217 kB 00:00 2023-11-08T19:22:17.247 INFO:teuthology.orchestra.run.smithi188.stdout:(13/142): libcephfs-devel-18.0.0-7035.g7532011a 314 kB/s | 31 kB 00:00 2023-11-08T19:22:17.364 INFO:teuthology.orchestra.run.smithi188.stdout:(14/142): libcephsqlite-18.0.0-7035.g7532011a.e 1.4 MB/s | 163 kB 00:00 2023-11-08T19:22:17.431 INFO:teuthology.orchestra.run.smithi188.stdout:(15/142): libcephfs2-18.0.0-7035.g7532011a.el9. 2.8 MB/s | 718 kB 00:00 2023-11-08T19:22:17.473 INFO:teuthology.orchestra.run.smithi188.stdout:(16/142): librados-devel-18.0.0-7035.g7532011a. 1.1 MB/s | 126 kB 00:00 2023-11-08T19:22:17.565 INFO:teuthology.orchestra.run.smithi188.stdout:(17/142): libradosstriper1-18.0.0-7035.g7532011 3.5 MB/s | 476 kB 00:00 2023-11-08T19:22:17.674 INFO:teuthology.orchestra.run.smithi188.stdout:(18/142): python3-ceph-argparse-18.0.0-7035.g75 420 kB/s | 45 kB 00:00 2023-11-08T19:22:17.790 INFO:teuthology.orchestra.run.smithi188.stdout:(19/142): python3-ceph-common-18.0.0-7035.g7532 1.1 MB/s | 127 kB 00:00 2023-11-08T19:22:17.899 INFO:teuthology.orchestra.run.smithi188.stdout:(20/142): python3-cephfs-18.0.0-7035.g7532011a. 1.5 MB/s | 162 kB 00:00 2023-11-08T19:22:18.049 INFO:teuthology.orchestra.run.smithi188.stdout:(21/142): python3-rados-18.0.0-7035.g7532011a.e 2.1 MB/s | 320 kB 00:00 2023-11-08T19:22:18.166 INFO:teuthology.orchestra.run.smithi188.stdout:(22/142): python3-rbd-18.0.0-7035.g7532011a.el9 2.5 MB/s | 299 kB 00:00 2023-11-08T19:22:18.333 INFO:teuthology.orchestra.run.smithi188.stdout:(23/142): librgw2-18.0.0-7035.g7532011a.el9.x86 5.6 MB/s | 4.8 MB 00:00 2023-11-08T19:22:18.358 INFO:teuthology.orchestra.run.smithi119.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 15 kB 00:00 2023-11-08T19:22:18.359 INFO:teuthology.orchestra.run.smithi188.stdout:(24/142): python3-rgw-18.0.0-7035.g7532011a.el9 519 kB/s | 99 kB 00:00 2023-11-08T19:22:18.480 INFO:teuthology.orchestra.run.smithi188.stdout:(25/142): rbd-fuse-18.0.0-7035.g7532011a.el9.x8 578 kB/s | 84 kB 00:00 2023-11-08T19:22:18.651 INFO:teuthology.orchestra.run.smithi188.stdout:(26/142): rbd-nbd-18.0.0-7035.g7532011a.el9.x86 999 kB/s | 169 kB 00:00 2023-11-08T19:22:18.743 INFO:teuthology.orchestra.run.smithi188.stdout:(27/142): rbd-mirror-18.0.0-7035.g7532011a.el9. 7.9 MB/s | 3.0 MB 00:00 2023-11-08T19:22:18.769 INFO:teuthology.orchestra.run.smithi188.stdout:(28/142): ceph-grafana-dashboards-18.0.0-7035.g 200 kB/s | 23 kB 00:00 2023-11-08T19:22:18.871 INFO:teuthology.orchestra.run.smithi188.stdout:(29/142): ceph-mgr-cephadm-18.0.0-7035.g7532011 1.0 MB/s | 132 kB 00:00 2023-11-08T19:22:19.238 INFO:teuthology.orchestra.run.smithi188.stdout:(30/142): ceph-mgr-dashboard-18.0.0-7035.g75320 3.8 MB/s | 1.8 MB 00:00 2023-11-08T19:22:19.405 INFO:teuthology.orchestra.run.smithi188.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7035.g75 1.4 MB/s | 243 kB 00:00 2023-11-08T19:22:19.639 INFO:teuthology.orchestra.run.smithi188.stdout:(32/142): ceph-mgr-diskprediction-local-18.0.0- 9.6 MB/s | 7.4 MB 00:00 2023-11-08T19:22:19.664 INFO:teuthology.orchestra.run.smithi188.stdout:(33/142): ceph-mgr-rook-18.0.0-7035.g7532011a.e 185 kB/s | 48 kB 00:00 2023-11-08T19:22:19.740 INFO:teuthology.orchestra.run.smithi188.stdout:(34/142): ceph-prometheus-alerts-18.0.0-7035.g7 152 kB/s | 15 kB 00:00 2023-11-08T19:22:19.773 INFO:teuthology.orchestra.run.smithi188.stdout:(35/142): ceph-volume-18.0.0-7035.g7532011a.el9 2.4 MB/s | 262 kB 00:00 2023-11-08T19:22:19.873 INFO:teuthology.orchestra.run.smithi188.stdout:(36/142): cephadm-18.0.0-7035.g7532011a.el9.noa 2.0 MB/s | 270 kB 00:00 2023-11-08T19:22:19.898 INFO:teuthology.orchestra.run.smithi188.stdout:(37/142): cephfs-top-18.0.0-7035.g7532011a.el9. 146 kB/s | 18 kB 00:00 2023-11-08T19:22:20.090 INFO:teuthology.orchestra.run.smithi188.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 856 kB/s | 106 kB 00:00 2023-11-08T19:22:20.141 INFO:teuthology.orchestra.run.smithi188.stdout:(39/142): bison-3.7.4-5.el9.x86_64.rpm 3.4 MB/s | 920 kB 00:00 2023-11-08T19:22:20.168 INFO:teuthology.orchestra.run.smithi119.stdout:Extra Packages for Enterprise Linux 12 MB/s | 19 MB 00:01 2023-11-08T19:22:20.174 INFO:teuthology.orchestra.run.smithi188.stdout:(40/142): flex-2.6.4-9.el9.x86_64.rpm 3.8 MB/s | 307 kB 00:00 2023-11-08T19:22:20.225 INFO:teuthology.orchestra.run.smithi188.stdout:(41/142): flexiblas-3.0.4-8.el9.x86_64.rpm 393 kB/s | 32 kB 00:00 2023-11-08T19:22:20.333 INFO:teuthology.orchestra.run.smithi188.stdout:(42/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2023-11-08T19:22:20.375 INFO:teuthology.orchestra.run.smithi188.stdout:(43/142): flexiblas-openblas-openmp-3.0.4-8.el9 125 kB/s | 17 kB 00:00 2023-11-08T19:22:20.417 INFO:teuthology.orchestra.run.smithi188.stdout:(44/142): jq-1.6-16.el9.x86_64.rpm 4.5 MB/s | 186 kB 00:00 2023-11-08T19:22:20.459 INFO:teuthology.orchestra.run.smithi188.stdout:(45/142): libaio-devel-0.3.111-13.el9.x86_64.rp 122 kB/s | 10 kB 00:00 2023-11-08T19:22:20.501 INFO:teuthology.orchestra.run.smithi188.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2023-11-08T19:22:20.542 INFO:teuthology.orchestra.run.smithi188.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 7.9 MB/s | 661 kB 00:00 2023-11-08T19:22:20.593 INFO:teuthology.orchestra.run.smithi188.stdout:(48/142): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2023-11-08T19:22:20.677 INFO:teuthology.orchestra.run.smithi188.stdout:(49/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 349 kB/s | 46 kB 00:00 2023-11-08T19:22:20.719 INFO:teuthology.orchestra.run.smithi188.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.8 MB/s | 282 kB 00:00 2023-11-08T19:22:20.744 INFO:teuthology.orchestra.run.smithi188.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.2 MB/s | 217 kB 00:00 2023-11-08T19:22:20.811 INFO:teuthology.orchestra.run.smithi188.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 523 kB/s | 34 kB 00:00 2023-11-08T19:22:20.936 INFO:teuthology.orchestra.run.smithi188.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2023-11-08T19:22:20.988 INFO:teuthology.orchestra.run.smithi188.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 9.4 MB/s | 1.5 MB 00:00 2023-11-08T19:22:21.045 INFO:teuthology.orchestra.run.smithi188.stdout:(55/142): postgresql-contrib-13.11-1.el9.x86_64 7.4 MB/s | 811 kB 00:00 2023-11-08T19:22:21.071 INFO:teuthology.orchestra.run.smithi188.stdout:(56/142): postgresql-private-libs-13.11-1.el9.x 1.7 MB/s | 134 kB 00:00 2023-11-08T19:22:21.221 INFO:teuthology.orchestra.run.smithi188.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-11-08T19:22:21.380 INFO:teuthology.orchestra.run.smithi188.stdout:(58/142): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2023-11-08T19:22:21.413 INFO:teuthology.orchestra.run.smithi188.stdout:(59/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.1 MB/s | 205 kB 00:00 2023-11-08T19:22:21.455 INFO:teuthology.orchestra.run.smithi188.stdout:(60/142): python3-jinja2-2.11.3-4.el9.noarch.rp 5.5 MB/s | 228 kB 00:00 2023-11-08T19:22:21.497 INFO:teuthology.orchestra.run.smithi188.stdout:(61/142): python3-jsonpatch-1.21-16.el9.noarch. 297 kB/s | 24 kB 00:00 2023-11-08T19:22:21.547 INFO:teuthology.orchestra.run.smithi188.stdout:(62/142): python3-jsonpointer-2.0-4.el9.noarch. 370 kB/s | 18 kB 00:00 2023-11-08T19:22:21.573 INFO:teuthology.orchestra.run.smithi188.stdout:(63/142): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-11-08T19:22:21.615 INFO:teuthology.orchestra.run.smithi188.stdout:(64/142): python3-markupsafe-1.1.1-12.el9.x86_6 486 kB/s | 32 kB 00:00 2023-11-08T19:22:21.790 INFO:teuthology.orchestra.run.smithi188.stdout:(65/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 4.8 MB 00:00 2023-11-08T19:22:21.815 INFO:teuthology.orchestra.run.smithi188.stdout:(66/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-11-08T19:22:21.857 INFO:teuthology.orchestra.run.smithi188.stdout:(67/142): python3-oauthlib-3.1.1-5.el9.noarch.r 4.2 MB/s | 174 kB 00:00 2023-11-08T19:22:21.899 INFO:teuthology.orchestra.run.smithi188.stdout:(68/142): python3-prettytable-0.7.2-27.el9.noar 502 kB/s | 41 kB 00:00 2023-11-08T19:22:21.941 INFO:teuthology.orchestra.run.smithi188.stdout:(69/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.2 MB/s | 132 kB 00:00 2023-11-08T19:22:21.966 INFO:teuthology.orchestra.run.smithi188.stdout:(70/142): python3-pyasn1-modules-0.4.8-6.el9.no 3.2 MB/s | 211 kB 00:00 2023-11-08T19:22:22.016 INFO:teuthology.orchestra.run.smithi188.stdout:(71/142): python3-requests-oauthlib-1.3.0-12.el 875 kB/s | 43 kB 00:00 2023-11-08T19:22:22.317 INFO:teuthology.orchestra.run.smithi188.stdout:(72/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 45 MB/s | 16 MB 00:00 2023-11-08T19:22:22.367 INFO:teuthology.orchestra.run.smithi188.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 115 kB/s | 39 kB 00:00 2023-11-08T19:22:22.409 INFO:teuthology.orchestra.run.smithi188.stdout:(74/142): socat-1.7.4.1-5.el9.x86_64.rpm 7.3 MB/s | 300 kB 00:00 2023-11-08T19:22:22.459 INFO:teuthology.orchestra.run.smithi188.stdout:(75/142): uuid-1.6.2-55.el9.x86_64.rpm 624 kB/s | 56 kB 00:00 2023-11-08T19:22:22.551 INFO:teuthology.orchestra.run.smithi188.stdout:(76/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 691 kB/s | 63 kB 00:00 2023-11-08T19:22:22.576 INFO:teuthology.orchestra.run.smithi188.stdout:(77/142): fuse-2.9.9-15.el9.x86_64.rpm 681 kB/s | 79 kB 00:00 2023-11-08T19:22:22.627 INFO:teuthology.orchestra.run.smithi188.stdout:(78/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 16 MB/s | 803 kB 00:00 2023-11-08T19:22:22.652 INFO:teuthology.orchestra.run.smithi188.stdout:(79/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.6 MB/s | 195 kB 00:00 2023-11-08T19:22:22.694 INFO:teuthology.orchestra.run.smithi188.stdout:(80/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.8 MB/s | 241 kB 00:00 2023-11-08T19:22:22.719 INFO:teuthology.orchestra.run.smithi188.stdout:(81/142): python3-ply-3.11-14.el9.noarch.rpm 1.5 MB/s | 103 kB 00:00 2023-11-08T19:22:22.761 INFO:teuthology.orchestra.run.smithi188.stdout:(82/142): python3-pycparser-2.20-6.el9.noarch.r 3.0 MB/s | 124 kB 00:00 2023-11-08T19:22:22.786 INFO:teuthology.orchestra.run.smithi188.stdout:(83/142): python3-requests-2.25.1-7.el9.noarch. 1.7 MB/s | 113 kB 00:00 2023-11-08T19:22:22.828 INFO:teuthology.orchestra.run.smithi188.stdout:(84/142): python3-urllib3-1.26.5-4.el9.noarch.r 4.5 MB/s | 187 kB 00:00 2023-11-08T19:22:23.312 INFO:teuthology.orchestra.run.smithi188.stdout:(85/142): ceph-test-18.0.0-7035.g7532011a.el9.x 5.8 MB/s | 41 MB 00:07 2023-11-08T19:22:23.479 INFO:teuthology.orchestra.run.smithi188.stdout:(86/142): python3-cryptography-36.0.1-4.el9.x86 1.8 MB/s | 1.2 MB 00:00 2023-11-08T19:22:23.621 INFO:teuthology.orchestra.run.smithi188.stdout:(87/142): elfutils-libelf-devel-0.189-3.el9.x86 73 kB/s | 22 kB 00:00 2023-11-08T19:22:23.796 INFO:teuthology.orchestra.run.smithi188.stdout:(88/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 146 kB/s | 26 kB 00:00 2023-11-08T19:22:23.846 INFO:teuthology.orchestra.run.smithi188.stdout:(89/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 123 kB/s | 45 kB 00:00 2023-11-08T19:22:23.913 INFO:teuthology.orchestra.run.smithi188.stdout:(90/142): automake-1.16.2-8.el9.noarch.rpm 636 kB/s | 689 kB 00:01 2023-11-08T19:22:24.039 INFO:teuthology.orchestra.run.smithi188.stdout:(91/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 693 kB/s | 86 kB 00:00 2023-11-08T19:22:24.081 INFO:teuthology.orchestra.run.smithi188.stdout:(92/142): python3-pytz-2021.1-5.el9.noarch.rpm 218 kB/s | 51 kB 00:00 2023-11-08T19:22:24.164 INFO:teuthology.orchestra.run.smithi188.stdout:(93/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 362 kB/s | 45 kB 00:00 2023-11-08T19:22:24.615 INFO:teuthology.orchestra.run.smithi188.stdout:(94/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 5.0 MB/s | 4.1 MB 00:00 2023-11-08T19:22:24.761 INFO:teuthology.orchestra.run.smithi188.stdout:(95/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 455 kB/s | 309 kB 00:00 2023-11-08T19:22:24.907 INFO:teuthology.orchestra.run.smithi188.stdout:(96/142): liboath-2.6.7-2.el9.x86_64.rpm 336 kB/s | 49 kB 00:00 2023-11-08T19:22:24.974 INFO:teuthology.orchestra.run.smithi188.stdout:(97/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 68 kB/s | 24 kB 00:00 2023-11-08T19:22:25.024 INFO:teuthology.orchestra.run.smithi188.stdout:(98/142): libunwind-1.6.2-1.el9.x86_64.rpm 583 kB/s | 67 kB 00:00 2023-11-08T19:22:25.174 INFO:teuthology.orchestra.run.smithi188.stdout:(99/142): libarrow-9.0.0-9.el9.x86_64.rpm 4.4 MB/s | 4.4 MB 00:01 2023-11-08T19:22:25.241 INFO:teuthology.orchestra.run.smithi188.stdout:(100/142): python3-asyncssh-2.13.2-3.el9.noarch 2.5 MB/s | 548 kB 00:00 2023-11-08T19:22:25.317 INFO:teuthology.orchestra.run.smithi188.stdout:(101/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2023-11-08T19:22:25.367 INFO:teuthology.orchestra.run.smithi188.stdout:(102/142): python3-cachetools-4.2.4-1.el9.noarc 258 kB/s | 32 kB 00:00 2023-11-08T19:22:25.443 INFO:teuthology.orchestra.run.smithi119.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2023-11-08T19:22:25.450 INFO:teuthology.orchestra.run.smithi188.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2023-11-08T19:22:25.518 INFO:teuthology.orchestra.run.smithi188.stdout:(104/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 1.5 MB/s | 837 kB 00:00 2023-11-08T19:22:25.551 INFO:teuthology.orchestra.run.smithi188.stdout:(105/142): python3-cheroot-8.6.0-4.el9.noarch.r 937 kB/s | 172 kB 00:00 2023-11-08T19:22:25.602 INFO:teuthology.orchestra.run.smithi188.stdout:(106/142): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2023-11-08T19:22:25.702 INFO:teuthology.orchestra.run.smithi188.stdout:(107/142): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2023-11-08T19:22:25.744 INFO:teuthology.orchestra.run.smithi188.stdout:(108/142): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2023-11-08T19:22:25.769 INFO:teuthology.orchestra.run.smithi188.stdout:(109/142): python3-google-auth-2.23.3-1.el9.noa 787 kB/s | 197 kB 00:00 2023-11-08T19:22:25.820 INFO:teuthology.orchestra.run.smithi188.stdout:(110/142): python3-jaraco-collections-3.0.0-8.e 198 kB/s | 23 kB 00:00 2023-11-08T19:22:25.887 INFO:teuthology.orchestra.run.smithi188.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2023-11-08T19:22:25.929 INFO:teuthology.orchestra.run.smithi188.stdout:(112/142): python3-jaraco-text-3.2.0-6.el9.noar 124 kB/s | 20 kB 00:00 2023-11-08T19:22:25.970 INFO:teuthology.orchestra.run.smithi188.stdout:(113/142): python3-jwt+crypto-2.4.0-1.el9.noarc 60 kB/s | 9.0 kB 00:00 2023-11-08T19:22:26.012 INFO:teuthology.orchestra.run.smithi188.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 328 kB/s | 41 kB 00:00 2023-11-08T19:22:26.104 INFO:teuthology.orchestra.run.smithi188.stdout:(115/142): python3-logutils-0.3.5-21.el9.noarch 348 kB/s | 46 kB 00:00 2023-11-08T19:22:26.130 INFO:teuthology.orchestra.run.smithi188.stdout:(116/142): python3-more-itertools-8.12.0-2.el9. 677 kB/s | 79 kB 00:00 2023-11-08T19:22:26.172 INFO:teuthology.orchestra.run.smithi188.stdout:(117/142): python3-kubernetes-26.1.0-2.el9.noar 4.2 MB/s | 1.0 MB 00:00 2023-11-08T19:22:26.264 INFO:teuthology.orchestra.run.smithi188.stdout:(118/142): python3-natsort-7.1.1-5.el9.noarch.r 365 kB/s | 58 kB 00:00 2023-11-08T19:22:26.297 INFO:teuthology.orchestra.run.smithi188.stdout:(119/142): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2023-11-08T19:22:26.339 INFO:teuthology.orchestra.run.smithi188.stdout:(120/142): python3-portend-3.1.0-2.el9.noarch.r 98 kB/s | 16 kB 00:00 2023-11-08T19:22:26.364 INFO:teuthology.orchestra.run.smithi188.stdout:(121/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 900 kB/s | 90 kB 00:00 2023-11-08T19:22:26.432 INFO:teuthology.orchestra.run.smithi188.stdout:(122/142): python3-repoze-lru-0.7-16.el9.noarch 231 kB/s | 31 kB 00:00 2023-11-08T19:22:26.465 INFO:teuthology.orchestra.run.smithi188.stdout:(123/142): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2023-11-08T19:22:26.524 INFO:teuthology.orchestra.run.smithi188.stdout:(124/142): python3-rsa-4.9-2.el9.noarch.rpm 373 kB/s | 59 kB 00:00 2023-11-08T19:22:26.598 INFO:teuthology.orchestra.run.smithi188.stdout:(125/142): python3-tempora-5.0.0-2.el9.noarch.r 215 kB/s | 36 kB 00:00 2023-11-08T19:22:26.657 INFO:teuthology.orchestra.run.smithi188.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 266 kB/s | 51 kB 00:00 2023-11-08T19:22:26.691 INFO:teuthology.orchestra.run.smithi188.stdout:(127/142): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2023-11-08T19:22:26.716 INFO:teuthology.orchestra.run.smithi188.stdout:(128/142): python3-websocket-client-1.2.3-2.el9 765 kB/s | 90 kB 00:00 2023-11-08T19:22:26.784 INFO:teuthology.orchestra.run.smithi188.stdout:(129/142): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2023-11-08T19:22:26.834 INFO:teuthology.orchestra.run.smithi188.stdout:(130/142): python3-zc-lockfile-2.0-10.el9.noarc 141 kB/s | 20 kB 00:00 2023-11-08T19:22:26.867 INFO:teuthology.orchestra.run.smithi188.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2023-11-08T19:22:26.984 INFO:teuthology.orchestra.run.smithi188.stdout:(132/142): thrift-0.15.0-2.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2023-11-08T19:22:27.018 INFO:teuthology.orchestra.run.smithi188.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-08T19:22:27.102 INFO:teuthology.orchestra.run.smithi188.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 27 MB/s | 2.2 MB 00:00 2023-11-08T19:22:27.185 INFO:teuthology.orchestra.run.smithi188.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 21 MB/s | 1.8 MB 00:00 2023-11-08T19:22:27.235 INFO:teuthology.orchestra.run.smithi188.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 857 kB/s | 42 kB 00:00 2023-11-08T19:22:27.311 INFO:teuthology.orchestra.run.smithi188.stdout:(137/142): NetworkManager-tui-1.45.6-1.el9.x86_ 3.2 MB/s | 245 kB 00:00 2023-11-08T19:22:27.345 INFO:teuthology.orchestra.run.smithi188.stdout:(138/142): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 69 kB 00:00 2023-11-08T19:22:27.395 INFO:teuthology.orchestra.run.smithi188.stdout:(139/142): libacl-2.3.1-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2023-11-08T19:22:27.437 INFO:teuthology.orchestra.run.smithi188.stdout:(140/142): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2023-11-08T19:22:27.796 INFO:teuthology.orchestra.run.smithi188.stdout:(141/142): librados2-18.0.0-7035.g7532011a.el9. 3.4 MB/s | 3.3 MB 00:00 2023-11-08T19:22:27.879 INFO:teuthology.orchestra.run.smithi188.stdout:(142/142): librbd1-18.0.0-7035.g7532011a.el9.x8 3.0 MB/s | 3.0 MB 00:01 2023-11-08T19:22:27.887 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2023-11-08T19:22:27.888 INFO:teuthology.orchestra.run.smithi188.stdout:Total 13 MB/s | 200 MB 00:15 2023-11-08T19:22:28.213 INFO:teuthology.orchestra.run.smithi119.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T19:22:28.214 INFO:teuthology.orchestra.run.smithi119.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T19:22:28.233 INFO:teuthology.orchestra.run.smithi119.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-08T19:22:28.234 INFO:teuthology.orchestra.run.smithi119.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-08T19:22:28.236 INFO:teuthology.orchestra.run.smithi119.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-08T19:22:28.237 INFO:teuthology.orchestra.run.smithi119.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-08T19:22:28.239 INFO:teuthology.orchestra.run.smithi119.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-08T19:22:28.243 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-08T19:22:28.244 INFO:teuthology.orchestra.run.smithi119.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-08T19:22:28.276 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2023-11-08T19:22:28.322 INFO:teuthology.orchestra.run.smithi119.stdout:Dependencies resolved. 2023-11-08T19:22:28.349 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2023-11-08T19:22:28.349 INFO:teuthology.orchestra.run.smithi119.stdout: Package Arch Version Repository Size 2023-11-08T19:22:28.349 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2023-11-08T19:22:28.349 INFO:teuthology.orchestra.run.smithi119.stdout:Installing: 2023-11-08T19:22:28.350 INFO:teuthology.orchestra.run.smithi119.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-08T19:22:28.350 INFO:teuthology.orchestra.run.smithi119.stdout: ceph x86_64 2:18.0.0-7035.g7532011a.el9 ceph 6.5 k 2023-11-08T19:22:28.350 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base x86_64 2:18.0.0-7035.g7532011a.el9 ceph 5.2 M 2023-11-08T19:22:28.350 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 859 k 2023-11-08T19:22:28.351 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7035.g7532011a.el9 ceph 140 k 2023-11-08T19:22:28.351 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr x86_64 2:18.0.0-7035.g7532011a.el9 ceph 1.5 M 2023-11-08T19:22:28.351 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 132 k 2023-11-08T19:22:28.351 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 1.8 M 2023-11-08T19:22:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 7.4 M 2023-11-08T19:22:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 48 k 2023-11-08T19:22:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 8.7 M 2023-11-08T19:22:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test x86_64 2:18.0.0-7035.g7532011a.el9 ceph 41 M 2023-11-08T19:22:28.352 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 262 k 2023-11-08T19:22:28.353 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 270 k 2023-11-08T19:22:28.353 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 217 k 2023-11-08T19:22:28.353 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 18 k 2023-11-08T19:22:28.354 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-08T19:22:28.354 INFO:teuthology.orchestra.run.smithi119.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-08T19:22:28.355 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-08T19:22:28.355 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 31 k 2023-11-08T19:22:28.355 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 718 k 2023-11-08T19:22:28.356 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel x86_64 2:18.0.0-7035.g7532011a.el9 ceph 126 k 2023-11-08T19:22:28.356 INFO:teuthology.orchestra.run.smithi119.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-08T19:22:28.356 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-08T19:22:28.356 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-08T19:22:28.357 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-08T19:22:28.357 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-08T19:22:28.357 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-08T19:22:28.357 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs x86_64 2:18.0.0-7035.g7532011a.el9 ceph 162 k 2023-11-08T19:22:28.358 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados x86_64 2:18.0.0-7035.g7532011a.el9 ceph 320 k 2023-11-08T19:22:28.358 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 299 k 2023-11-08T19:22:28.358 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw x86_64 2:18.0.0-7035.g7532011a.el9 ceph 99 k 2023-11-08T19:22:28.359 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 84 k 2023-11-08T19:22:28.359 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-08T19:22:28.359 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 169 k 2023-11-08T19:22:28.359 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-08T19:22:28.359 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrading: 2023-11-08T19:22:28.360 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-08T19:22:28.360 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-08T19:22:28.360 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-08T19:22:28.360 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-08T19:22:28.360 INFO:teuthology.orchestra.run.smithi119.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-08T19:22:28.361 INFO:teuthology.orchestra.run.smithi119.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-08T19:22:28.361 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-08T19:22:28.361 INFO:teuthology.orchestra.run.smithi119.stdout: librados2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.3 M 2023-11-08T19:22:28.361 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 3.0 M 2023-11-08T19:22:28.362 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-08T19:22:28.362 INFO:teuthology.orchestra.run.smithi119.stdout:Installing dependencies: 2023-11-08T19:22:28.362 INFO:teuthology.orchestra.run.smithi119.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-08T19:22:28.362 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-08T19:22:28.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 19 M 2023-11-08T19:22:28.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 23 k 2023-11-08T19:22:28.363 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds x86_64 2:18.0.0-7035.g7532011a.el9 ceph 2.1 M 2023-11-08T19:22:28.364 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 243 k 2023-11-08T19:22:28.364 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.4 M 2023-11-08T19:22:28.365 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd x86_64 2:18.0.0-7035.g7532011a.el9 ceph 18 M 2023-11-08T19:22:28.365 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7035.g7532011a.el9 ceph-noarch 15 k 2023-11-08T19:22:28.365 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux x86_64 2:18.0.0-7035.g7532011a.el9 ceph 24 k 2023-11-08T19:22:28.366 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-08T19:22:28.366 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-08T19:22:28.366 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-08T19:22:28.367 INFO:teuthology.orchestra.run.smithi119.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-08T19:22:28.367 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-08T19:22:28.367 INFO:teuthology.orchestra.run.smithi119.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-08T19:22:28.367 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-08T19:22:28.368 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-08T19:22:28.368 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite x86_64 2:18.0.0-7035.g7532011a.el9 ceph 163 k 2023-11-08T19:22:28.368 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-08T19:22:28.369 INFO:teuthology.orchestra.run.smithi119.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-08T19:22:28.369 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-08T19:22:28.369 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-08T19:22:28.369 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-08T19:22:28.370 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 476 k 2023-11-08T19:22:28.370 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-08T19:22:28.370 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2 x86_64 2:18.0.0-7035.g7532011a.el9 ceph 4.8 M 2023-11-08T19:22:28.370 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-08T19:22:28.370 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-08T19:22:28.371 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-08T19:22:28.371 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-08T19:22:28.371 INFO:teuthology.orchestra.run.smithi119.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-08T19:22:28.371 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-08T19:22:28.372 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-08T19:22:28.372 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-08T19:22:28.372 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-08T19:22:28.372 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-08T19:22:28.372 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-08T19:22:28.372 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-08T19:22:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse x86_64 2:18.0.0-7035.g7532011a.el9 ceph 45 k 2023-11-08T19:22:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common x86_64 2:18.0.0-7035.g7532011a.el9 ceph 127 k 2023-11-08T19:22:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-08T19:22:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-08T19:22:28.373 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-08T19:22:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-08T19:22:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-08T19:22:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-08T19:22:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-08T19:22:28.374 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-08T19:22:28.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-08T19:22:28.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-08T19:22:28.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-08T19:22:28.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-08T19:22:28.375 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-08T19:22:28.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-08T19:22:28.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-08T19:22:28.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-08T19:22:28.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-08T19:22:28.376 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-08T19:22:28.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-08T19:22:28.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-08T19:22:28.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-08T19:22:28.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-08T19:22:28.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-08T19:22:28.377 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-08T19:22:28.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-08T19:22:28.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-08T19:22:28.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-08T19:22:28.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-08T19:22:28.378 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-08T19:22:28.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-08T19:22:28.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-08T19:22:28.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-08T19:22:28.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-08T19:22:28.379 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-08T19:22:28.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-08T19:22:28.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-08T19:22:28.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-08T19:22:28.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-08T19:22:28.380 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-08T19:22:28.381 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-08T19:22:28.381 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-08T19:22:28.381 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-08T19:22:28.381 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-08T19:22:28.381 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-08T19:22:28.382 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-08T19:22:28.382 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-08T19:22:28.382 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-08T19:22:28.382 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-08T19:22:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-08T19:22:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-08T19:22:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-08T19:22:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-08T19:22:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-08T19:22:28.383 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-08T19:22:28.384 INFO:teuthology.orchestra.run.smithi119.stdout:Installing weak dependencies: 2023-11-08T19:22:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-08T19:22:28.384 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:22:28.384 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction Summary 2023-11-08T19:22:28.384 INFO:teuthology.orchestra.run.smithi119.stdout:============================================================================================ 2023-11-08T19:22:28.385 INFO:teuthology.orchestra.run.smithi119.stdout:Install 132 Packages 2023-11-08T19:22:28.385 INFO:teuthology.orchestra.run.smithi119.stdout:Upgrade 10 Packages 2023-11-08T19:22:28.385 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:22:28.386 INFO:teuthology.orchestra.run.smithi119.stdout:Total download size: 200 M 2023-11-08T19:22:28.386 INFO:teuthology.orchestra.run.smithi119.stdout:Downloading Packages: 2023-11-08T19:22:28.475 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2023-11-08T19:22:28.475 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2023-11-08T19:22:29.228 INFO:teuthology.orchestra.run.smithi119.stdout:(1/142): ceph-18.0.0-7035.g7532011a.el9.x86_64. 26 kB/s | 6.5 kB 00:00 2023-11-08T19:22:29.429 INFO:teuthology.orchestra.run.smithi119.stdout:(2/142): ceph-fuse-18.0.0-7035.g7532011a.el9.x8 4.2 MB/s | 859 kB 00:00 2023-11-08T19:22:29.496 INFO:teuthology.orchestra.run.smithi119.stdout:(3/142): ceph-immutable-object-cache-18.0.0-703 2.1 MB/s | 140 kB 00:00 2023-11-08T19:22:29.754 INFO:teuthology.orchestra.run.smithi119.stdout:(4/142): ceph-mds-18.0.0-7035.g7532011a.el9.x86 8.1 MB/s | 2.1 MB 00:00 2023-11-08T19:22:29.971 INFO:teuthology.orchestra.run.smithi119.stdout:(5/142): ceph-mgr-18.0.0-7035.g7532011a.el9.x86 6.8 MB/s | 1.5 MB 00:00 2023-11-08T19:22:30.472 INFO:teuthology.orchestra.run.smithi119.stdout:(6/142): ceph-base-18.0.0-7035.g7532011a.el9.x8 3.5 MB/s | 5.2 MB 00:01 2023-11-08T19:22:30.572 INFO:teuthology.orchestra.run.smithi119.stdout:(7/142): ceph-mon-18.0.0-7035.g7532011a.el9.x86 7.3 MB/s | 4.4 MB 00:00 2023-11-08T19:22:30.747 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2023-11-08T19:22:30.749 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2023-11-08T19:22:30.822 INFO:teuthology.orchestra.run.smithi119.stdout:(8/142): ceph-common-18.0.0-7035.g7532011a.el9. 10 MB/s | 19 MB 00:01 2023-11-08T19:22:30.897 INFO:teuthology.orchestra.run.smithi119.stdout:(9/142): ceph-selinux-18.0.0-7035.g7532011a.el9 325 kB/s | 24 kB 00:00 2023-11-08T19:22:31.796 INFO:teuthology.orchestra.run.smithi119.stdout:(10/142): ceph-radosgw-18.0.0-7035.g7532011a.el 7.1 MB/s | 8.7 MB 00:01 2023-11-08T19:22:31.890 INFO:teuthology.orchestra.run.smithi119.stdout:(11/142): cephfs-mirror-18.0.0-7035.g7532011a.e 2.3 MB/s | 217 kB 00:00 2023-11-08T19:22:31.965 INFO:teuthology.orchestra.run.smithi119.stdout:(12/142): libcephfs-devel-18.0.0-7035.g7532011a 421 kB/s | 31 kB 00:00 2023-11-08T19:22:32.132 INFO:teuthology.orchestra.run.smithi119.stdout:(13/142): libcephfs2-18.0.0-7035.g7532011a.el9. 4.2 MB/s | 718 kB 00:00 2023-11-08T19:22:32.232 INFO:teuthology.orchestra.run.smithi119.stdout:(14/142): libcephsqlite-18.0.0-7035.g7532011a.e 1.6 MB/s | 163 kB 00:00 2023-11-08T19:22:32.324 INFO:teuthology.orchestra.run.smithi119.stdout:(15/142): librados-devel-18.0.0-7035.g7532011a. 1.4 MB/s | 126 kB 00:00 2023-11-08T19:22:32.483 INFO:teuthology.orchestra.run.smithi119.stdout:(16/142): libradosstriper1-18.0.0-7035.g7532011 3.0 MB/s | 476 kB 00:00 2023-11-08T19:22:33.425 INFO:teuthology.orchestra.run.smithi119.stdout:(17/142): librgw2-18.0.0-7035.g7532011a.el9.x86 5.1 MB/s | 4.8 MB 00:00 2023-11-08T19:22:33.508 INFO:teuthology.orchestra.run.smithi119.stdout:(18/142): python3-ceph-argparse-18.0.0-7035.g75 549 kB/s | 45 kB 00:00 2023-11-08T19:22:33.592 INFO:teuthology.orchestra.run.smithi119.stdout:(19/142): python3-ceph-common-18.0.0-7035.g7532 1.5 MB/s | 127 kB 00:00 2023-11-08T19:22:33.675 INFO:teuthology.orchestra.run.smithi119.stdout:(20/142): python3-cephfs-18.0.0-7035.g7532011a. 1.9 MB/s | 162 kB 00:00 2023-11-08T19:22:33.776 INFO:teuthology.orchestra.run.smithi119.stdout:(21/142): python3-rados-18.0.0-7035.g7532011a.e 3.1 MB/s | 320 kB 00:00 2023-11-08T19:22:33.867 INFO:teuthology.orchestra.run.smithi119.stdout:(22/142): python3-rbd-18.0.0-7035.g7532011a.el9 3.2 MB/s | 299 kB 00:00 2023-11-08T19:22:33.959 INFO:teuthology.orchestra.run.smithi119.stdout:(23/142): python3-rgw-18.0.0-7035.g7532011a.el9 1.1 MB/s | 99 kB 00:00 2023-11-08T19:22:34.052 INFO:teuthology.orchestra.run.smithi119.stdout:(24/142): rbd-fuse-18.0.0-7035.g7532011a.el9.x8 928 kB/s | 84 kB 00:00 2023-11-08T19:22:34.677 INFO:teuthology.orchestra.run.smithi119.stdout:(25/142): rbd-mirror-18.0.0-7035.g7532011a.el9. 4.8 MB/s | 3.0 MB 00:00 2023-11-08T19:22:34.760 INFO:teuthology.orchestra.run.smithi119.stdout:(26/142): rbd-nbd-18.0.0-7035.g7532011a.el9.x86 2.0 MB/s | 169 kB 00:00 2023-11-08T19:22:34.844 INFO:teuthology.orchestra.run.smithi119.stdout:(27/142): ceph-grafana-dashboards-18.0.0-7035.g 281 kB/s | 23 kB 00:00 2023-11-08T19:22:34.919 INFO:teuthology.orchestra.run.smithi119.stdout:(28/142): ceph-mgr-cephadm-18.0.0-7035.g7532011 1.7 MB/s | 132 kB 00:00 2023-11-08T19:22:35.336 INFO:teuthology.orchestra.run.smithi119.stdout:(29/142): ceph-osd-18.0.0-7035.g7532011a.el9.x8 3.6 MB/s | 18 MB 00:04 2023-11-08T19:22:35.412 INFO:teuthology.orchestra.run.smithi119.stdout:(30/142): ceph-mgr-dashboard-18.0.0-7035.g75320 3.6 MB/s | 1.8 MB 00:00 2023-11-08T19:22:35.520 INFO:teuthology.orchestra.run.smithi119.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7035.g75 2.2 MB/s | 243 kB 00:00 2023-11-08T19:22:35.621 INFO:teuthology.orchestra.run.smithi119.stdout:(32/142): ceph-mgr-rook-18.0.0-7035.g7532011a.e 482 kB/s | 48 kB 00:00 2023-11-08T19:22:35.721 INFO:teuthology.orchestra.run.smithi119.stdout:(33/142): ceph-prometheus-alerts-18.0.0-7035.g7 152 kB/s | 15 kB 00:00 2023-11-08T19:22:36.214 INFO:teuthology.orchestra.run.smithi119.stdout:(34/142): ceph-test-18.0.0-7035.g7532011a.el9.x 7.8 MB/s | 41 MB 00:05 2023-11-08T19:22:36.246 INFO:teuthology.orchestra.run.smithi119.stdout:(35/142): ceph-volume-18.0.0-7035.g7532011a.el9 500 kB/s | 262 kB 00:00 2023-11-08T19:22:36.355 INFO:teuthology.orchestra.run.smithi119.stdout:(36/142): cephfs-top-18.0.0-7035.g7532011a.el9. 169 kB/s | 18 kB 00:00 2023-11-08T19:22:36.547 INFO:teuthology.orchestra.run.smithi119.stdout:(37/142): bison-3.7.4-5.el9.x86_64.rpm 4.7 MB/s | 920 kB 00:00 2023-11-08T19:22:36.589 INFO:teuthology.orchestra.run.smithi119.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 106 kB 00:00 2023-11-08T19:22:36.631 INFO:teuthology.orchestra.run.smithi119.stdout:(39/142): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 307 kB 00:00 2023-11-08T19:22:36.668 INFO:teuthology.orchestra.run.smithi119.stdout:(40/142): flexiblas-3.0.4-8.el9.x86_64.rpm 873 kB/s | 32 kB 00:00 2023-11-08T19:22:36.702 INFO:teuthology.orchestra.run.smithi119.stdout:(41/142): cephadm-18.0.0-7035.g7532011a.el9.noa 555 kB/s | 270 kB 00:00 2023-11-08T19:22:36.765 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2023-11-08T19:22:36.819 INFO:teuthology.orchestra.run.smithi119.stdout:(42/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2023-11-08T19:22:36.844 INFO:teuthology.orchestra.run.smithi119.stdout:(43/142): flexiblas-openblas-openmp-3.0.4-8.el9 133 kB/s | 17 kB 00:00 2023-11-08T19:22:36.870 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-08T19:22:36.886 INFO:teuthology.orchestra.run.smithi119.stdout:(44/142): jq-1.6-16.el9.x86_64.rpm 4.5 MB/s | 186 kB 00:00 2023-11-08T19:22:36.899 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-08T19:22:36.911 INFO:teuthology.orchestra.run.smithi119.stdout:(45/142): libaio-devel-0.3.111-13.el9.x86_64.rp 154 kB/s | 10 kB 00:00 2023-11-08T19:22:36.928 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-08T19:22:36.953 INFO:teuthology.orchestra.run.smithi119.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 159 kB 00:00 2023-11-08T19:22:36.986 INFO:teuthology.orchestra.run.smithi119.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 8.8 MB/s | 661 kB 00:00 2023-11-08T19:22:37.020 INFO:teuthology.orchestra.run.smithi119.stdout:(48/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2023-11-08T19:22:37.037 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-08T19:22:37.062 INFO:teuthology.orchestra.run.smithi119.stdout:(49/142): libtool-2.4.6-45.el9.x86_64.rpm 7.6 MB/s | 578 kB 00:00 2023-11-08T19:22:37.096 INFO:teuthology.orchestra.run.smithi119.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.9 MB/s | 282 kB 00:00 2023-11-08T19:22:37.130 INFO:teuthology.orchestra.run.smithi119.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-11-08T19:22:37.163 INFO:teuthology.orchestra.run.smithi119.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 529 kB/s | 34 kB 00:00 2023-11-08T19:22:37.289 INFO:teuthology.orchestra.run.smithi119.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 30 MB/s | 4.7 MB 00:00 2023-11-08T19:22:37.339 INFO:teuthology.orchestra.run.smithi119.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 9.4 MB/s | 1.5 MB 00:00 2023-11-08T19:22:37.389 INFO:teuthology.orchestra.run.smithi119.stdout:(55/142): postgresql-contrib-13.11-1.el9.x86_64 8.0 MB/s | 811 kB 00:00 2023-11-08T19:22:37.415 INFO:teuthology.orchestra.run.smithi119.stdout:(56/142): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2023-11-08T19:22:37.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-08T19:22:37.557 INFO:teuthology.orchestra.run.smithi119.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 35 MB/s | 5.7 MB 00:00 2023-11-08T19:22:37.682 INFO:teuthology.orchestra.run.smithi119.stdout:(58/142): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2023-11-08T19:22:37.691 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-08T19:22:37.707 INFO:teuthology.orchestra.run.smithi119.stdout:(59/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 205 kB 00:00 2023-11-08T19:22:37.816 INFO:teuthology.orchestra.run.smithi119.stdout:(60/142): ceph-mgr-diskprediction-local-18.0.0- 3.0 MB/s | 7.4 MB 00:02 2023-11-08T19:22:37.841 INFO:teuthology.orchestra.run.smithi119.stdout:(61/142): python3-jsonpatch-1.21-16.el9.noarch. 183 kB/s | 24 kB 00:00 2023-11-08T19:22:37.872 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-08T19:22:37.875 INFO:teuthology.orchestra.run.smithi119.stdout:(62/142): python3-jinja2-2.11.3-4.el9.noarch.rp 1.3 MB/s | 228 kB 00:00 2023-11-08T19:22:37.891 INFO:teuthology.orchestra.run.smithi119.stdout:(63/142): python3-jsonpointer-2.0-4.el9.noarch. 243 kB/s | 18 kB 00:00 2023-11-08T19:22:37.908 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-08T19:22:37.917 INFO:teuthology.orchestra.run.smithi119.stdout:(64/142): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2023-11-08T19:22:37.959 INFO:teuthology.orchestra.run.smithi119.stdout:(65/142): python3-markupsafe-1.1.1-12.el9.x86_6 487 kB/s | 32 kB 00:00 2023-11-08T19:22:38.062 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-08T19:22:38.076 INFO:teuthology.orchestra.run.smithi119.stdout:(66/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2023-11-08T19:22:38.101 INFO:teuthology.orchestra.run.smithi119.stdout:(67/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.2 MB/s | 217 kB 00:00 2023-11-08T19:22:38.126 INFO:teuthology.orchestra.run.smithi119.stdout:(68/142): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 174 kB 00:00 2023-11-08T19:22:38.160 INFO:teuthology.orchestra.run.smithi119.stdout:(69/142): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 41 kB 00:00 2023-11-08T19:22:38.162 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-08T19:22:38.185 INFO:teuthology.orchestra.run.smithi119.stdout:(70/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-11-08T19:22:38.210 INFO:teuthology.orchestra.run.smithi119.stdout:(71/142): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2023-11-08T19:22:38.235 INFO:teuthology.orchestra.run.smithi119.stdout:(72/142): python3-requests-oauthlib-1.3.0-12.el 1.7 MB/s | 43 kB 00:00 2023-11-08T19:22:38.241 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-08T19:22:38.269 INFO:teuthology.orchestra.run.smithi119.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 671 kB/s | 39 kB 00:00 2023-11-08T19:22:38.496 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-08T19:22:38.550 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-08T19:22:38.569 INFO:teuthology.orchestra.run.smithi119.stdout:(74/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2023-11-08T19:22:38.594 INFO:teuthology.orchestra.run.smithi119.stdout:(75/142): uuid-1.6.2-55.el9.x86_64.rpm 187 kB/s | 56 kB 00:00 2023-11-08T19:22:38.614 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-08T19:22:38.628 INFO:teuthology.orchestra.run.smithi119.stdout:(76/142): socat-1.7.4.1-5.el9.x86_64.rpm 896 kB/s | 300 kB 00:00 2023-11-08T19:22:38.653 INFO:teuthology.orchestra.run.smithi119.stdout:(77/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 63 kB 00:00 2023-11-08T19:22:38.678 INFO:teuthology.orchestra.run.smithi119.stdout:(78/142): fuse-2.9.9-15.el9.x86_64.rpm 984 kB/s | 79 kB 00:00 2023-11-08T19:22:38.729 INFO:teuthology.orchestra.run.smithi119.stdout:(79/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 8.0 MB/s | 803 kB 00:00 2023-11-08T19:22:38.753 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-08T19:22:38.762 INFO:teuthology.orchestra.run.smithi119.stdout:(80/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2023-11-08T19:22:38.787 INFO:teuthology.orchestra.run.smithi119.stdout:(81/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 241 kB 00:00 2023-11-08T19:22:38.790 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-08T19:22:38.804 INFO:teuthology.orchestra.run.smithi119.stdout:(82/142): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 103 kB 00:00 2023-11-08T19:22:38.830 INFO:teuthology.orchestra.run.smithi119.stdout:(83/142): python3-requests-2.25.1-7.el9.noarch. 2.8 MB/s | 113 kB 00:00 2023-11-08T19:22:38.863 INFO:teuthology.orchestra.run.smithi119.stdout:(84/142): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 124 kB 00:00 2023-11-08T19:22:38.865 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-08T19:22:38.889 INFO:teuthology.orchestra.run.smithi119.stdout:(85/142): python3-urllib3-1.26.5-4.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2023-11-08T19:22:39.122 INFO:teuthology.orchestra.run.smithi119.stdout:(86/142): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2023-11-08T19:22:39.156 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-08T19:22:39.156 INFO:teuthology.orchestra.run.smithi119.stdout:(87/142): elfutils-libelf-devel-0.189-3.el9.x86 84 kB/s | 22 kB 00:00 2023-11-08T19:22:39.294 INFO:teuthology.orchestra.run.smithi119.stdout:(88/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 186 kB/s | 26 kB 00:00 2023-11-08T19:22:39.371 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T19:22:39.459 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T19:22:39.474 INFO:teuthology.orchestra.run.smithi119.stdout:(89/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 129 kB/s | 45 kB 00:00 2023-11-08T19:22:39.657 INFO:teuthology.orchestra.run.smithi119.stdout:(90/142): python3-pytz-2021.1-5.el9.noarch.rpm 279 kB/s | 51 kB 00:00 2023-11-08T19:22:39.676 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T19:22:39.690 INFO:teuthology.orchestra.run.smithi119.stdout:(91/142): automake-1.16.2-8.el9.noarch.rpm 861 kB/s | 689 kB 00:00 2023-11-08T19:22:39.766 INFO:teuthology.orchestra.run.smithi119.stdout:(92/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 798 kB/s | 86 kB 00:00 2023-11-08T19:22:39.766 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T19:22:39.799 INFO:teuthology.orchestra.run.smithi119.stdout:(93/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 418 kB/s | 45 kB 00:00 2023-11-08T19:22:39.958 INFO:teuthology.orchestra.run.smithi119.stdout:(94/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.6 MB/s | 309 kB 00:00 2023-11-08T19:22:40.008 INFO:teuthology.orchestra.run.smithi119.stdout:(95/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 494 kB/s | 24 kB 00:00 2023-11-08T19:22:40.142 INFO:teuthology.orchestra.run.smithi119.stdout:(96/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 4.9 MB/s | 4.1 MB 00:00 2023-11-08T19:22:40.167 INFO:teuthology.orchestra.run.smithi119.stdout:(97/142): liboath-2.6.7-2.el9.x86_64.rpm 308 kB/s | 49 kB 00:00 2023-11-08T19:22:40.213 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T19:22:40.271 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-08T19:22:40.284 INFO:teuthology.orchestra.run.smithi119.stdout:(98/142): libarrow-9.0.0-9.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2023-11-08T19:22:40.309 INFO:teuthology.orchestra.run.smithi119.stdout:(99/142): libunwind-1.6.2-1.el9.x86_64.rpm 405 kB/s | 67 kB 00:00 2023-11-08T19:22:40.328 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-08T19:22:40.360 INFO:teuthology.orchestra.run.smithi119.stdout:(100/142): python3-asyncssh-2.13.2-3.el9.noarch 7.2 MB/s | 548 kB 00:00 2023-11-08T19:22:40.401 INFO:teuthology.orchestra.run.smithi119.stdout:(101/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 3.5 MB/s | 837 kB 00:00 2023-11-08T19:22:40.427 INFO:teuthology.orchestra.run.smithi119.stdout:(102/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 372 kB/s | 43 kB 00:00 2023-11-08T19:22:40.452 INFO:teuthology.orchestra.run.smithi119.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 303 kB/s | 15 kB 00:00 2023-11-08T19:22:40.453 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-08T19:22:40.477 INFO:teuthology.orchestra.run.smithi119.stdout:(104/142): python3-cachetools-4.2.4-1.el9.noarc 276 kB/s | 32 kB 00:00 2023-11-08T19:22:40.511 INFO:teuthology.orchestra.run.smithi119.stdout:(105/142): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-11-08T19:22:40.530 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-08T19:22:40.544 INFO:teuthology.orchestra.run.smithi119.stdout:(106/142): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2023-11-08T19:22:40.567 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-08T19:22:40.570 INFO:teuthology.orchestra.run.smithi119.stdout:(107/142): python3-jaraco-8.2.1-3.el9.noarch.rp 185 kB/s | 11 kB 00:00 2023-11-08T19:22:40.595 INFO:teuthology.orchestra.run.smithi119.stdout:(108/142): python3-jaraco-classes-3.2.1-5.el9.n 356 kB/s | 18 kB 00:00 2023-11-08T19:22:40.612 INFO:teuthology.orchestra.run.smithi119.stdout:(109/142): python3-jaraco-collections-3.0.0-8.e 563 kB/s | 23 kB 00:00 2023-11-08T19:22:40.624 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-08T19:22:40.646 INFO:teuthology.orchestra.run.smithi119.stdout:(110/142): python3-google-auth-2.23.3-1.el9.noa 1.1 MB/s | 197 kB 00:00 2023-11-08T19:22:40.663 INFO:teuthology.orchestra.run.smithi119.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 293 kB/s | 19 kB 00:00 2023-11-08T19:22:40.679 INFO:teuthology.orchestra.run.smithi119.stdout:(112/142): python3-jaraco-text-3.2.0-6.el9.noar 296 kB/s | 20 kB 00:00 2023-11-08T19:22:40.696 INFO:teuthology.orchestra.run.smithi119.stdout:(113/142): python3-jwt+crypto-2.4.0-1.el9.noarc 182 kB/s | 9.0 kB 00:00 2023-11-08T19:22:40.713 INFO:teuthology.orchestra.run.smithi119.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 1.2 MB/s | 41 kB 00:00 2023-11-08T19:22:40.733 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-08T19:22:40.739 INFO:teuthology.orchestra.run.smithi119.stdout:(115/142): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2023-11-08T19:22:40.764 INFO:teuthology.orchestra.run.smithi119.stdout:(116/142): python3-more-itertools-8.12.0-2.el9. 1.6 MB/s | 79 kB 00:00 2023-11-08T19:22:40.789 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-08T19:22:40.790 INFO:teuthology.orchestra.run.smithi119.stdout:(117/142): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2023-11-08T19:22:40.823 INFO:teuthology.orchestra.run.smithi119.stdout:(118/142): python3-pecan-1.4.2-3.el9.noarch.rpm 4.6 MB/s | 272 kB 00:00 2023-11-08T19:22:40.828 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-08T19:22:40.848 INFO:teuthology.orchestra.run.smithi119.stdout:(119/142): python3-portend-3.1.0-2.el9.noarch.r 283 kB/s | 16 kB 00:00 2023-11-08T19:22:40.874 INFO:teuthology.orchestra.run.smithi119.stdout:(120/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2023-11-08T19:22:40.899 INFO:teuthology.orchestra.run.smithi119.stdout:(121/142): python3-repoze-lru-0.7-16.el9.noarch 619 kB/s | 31 kB 00:00 2023-11-08T19:22:40.943 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-08T19:22:40.966 INFO:teuthology.orchestra.run.smithi119.stdout:(122/142): python3-kubernetes-26.1.0-2.el9.noar 3.6 MB/s | 1.0 MB 00:00 2023-11-08T19:22:40.982 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 27/152 2023-11-08T19:22:40.991 INFO:teuthology.orchestra.run.smithi119.stdout:(123/142): python3-rsa-4.9-2.el9.noarch.rpm 647 kB/s | 59 kB 00:00 2023-11-08T19:22:41.017 INFO:teuthology.orchestra.run.smithi119.stdout:(124/142): python3-tempora-5.0.0-2.el9.noarch.r 723 kB/s | 36 kB 00:00 2023-11-08T19:22:41.032 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 28/152 2023-11-08T19:22:41.041 INFO:teuthology.orchestra.run.smithi119.stdout:(125/142): python3-routes-2.5.1-5.el9.noarch.rp 1.1 MB/s | 188 kB 00:00 2023-11-08T19:22:41.067 INFO:teuthology.orchestra.run.smithi119.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2023-11-08T19:22:41.100 INFO:teuthology.orchestra.run.smithi119.stdout:(127/142): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-11-08T19:22:41.126 INFO:teuthology.orchestra.run.smithi119.stdout:(128/142): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-11-08T19:22:41.141 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 29/152 2023-11-08T19:22:41.151 INFO:teuthology.orchestra.run.smithi119.stdout:(129/142): python3-zc-lockfile-2.0-10.el9.noarc 402 kB/s | 20 kB 00:00 2023-11-08T19:22:41.185 INFO:teuthology.orchestra.run.smithi119.stdout:(130/142): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2023-11-08T19:22:41.218 INFO:teuthology.orchestra.run.smithi119.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2023-11-08T19:22:41.285 INFO:teuthology.orchestra.run.smithi119.stdout:(132/142): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2023-11-08T19:22:41.319 INFO:teuthology.orchestra.run.smithi119.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-08T19:22:41.351 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-08T19:22:41.411 INFO:teuthology.orchestra.run.smithi119.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-11-08T19:22:41.419 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-08T19:22:41.486 INFO:teuthology.orchestra.run.smithi119.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 24 MB/s | 1.8 MB 00:00 2023-11-08T19:22:41.520 INFO:teuthology.orchestra.run.smithi119.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 1.3 MB/s | 42 kB 00:00 2023-11-08T19:22:41.562 INFO:teuthology.orchestra.run.smithi119.stdout:(137/142): NetworkManager-tui-1.45.6-1.el9.x86_ 5.9 MB/s | 245 kB 00:00 2023-11-08T19:22:41.595 INFO:teuthology.orchestra.run.smithi119.stdout:(138/142): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 69 kB 00:00 2023-11-08T19:22:41.637 INFO:teuthology.orchestra.run.smithi119.stdout:(139/142): libacl-2.3.1-4.el9.x86_64.rpm 536 kB/s | 22 kB 00:00 2023-11-08T19:22:41.713 INFO:teuthology.orchestra.run.smithi119.stdout:(140/142): librados2-18.0.0-7035.g7532011a.el9. 6.3 MB/s | 3.3 MB 00:00 2023-11-08T19:22:41.746 INFO:teuthology.orchestra.run.smithi119.stdout:(141/142): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2023-11-08T19:22:41.921 INFO:teuthology.orchestra.run.smithi119.stdout:(142/142): librbd1-18.0.0-7035.g7532011a.el9.x8 4.3 MB/s | 3.0 MB 00:00 2023-11-08T19:22:41.929 INFO:teuthology.orchestra.run.smithi119.stdout:-------------------------------------------------------------------------------- 2023-11-08T19:22:41.930 INFO:teuthology.orchestra.run.smithi119.stdout:Total 15 MB/s | 200 MB 00:13 2023-11-08T19:22:42.074 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-08T19:22:42.175 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-08T19:22:42.229 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-08T19:22:42.301 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 35/152 2023-11-08T19:22:42.322 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction check 2023-11-08T19:22:42.340 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-08T19:22:42.491 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-08T19:22:42.520 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction check succeeded. 2023-11-08T19:22:42.520 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction test 2023-11-08T19:22:42.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-08T19:22:42.611 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-08T19:22:42.795 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-08T19:22:42.836 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-08T19:22:42.876 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-08T19:22:42.936 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-08T19:22:42.989 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-08T19:22:43.074 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-08T19:22:43.276 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-08T19:22:43.324 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-08T19:22:43.410 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-08T19:22:43.542 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-08T19:22:43.579 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/152 2023-11-08T19:22:43.622 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-08T19:22:43.657 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-08T19:22:43.685 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-08T19:22:43.718 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-08T19:22:43.890 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-08T19:22:43.945 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-08T19:22:44.020 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-08T19:22:44.103 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-08T19:22:44.367 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-08T19:22:44.420 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-08T19:22:44.612 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-08T19:22:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:Transaction test succeeded. 2023-11-08T19:22:44.646 INFO:teuthology.orchestra.run.smithi119.stdout:Running transaction 2023-11-08T19:22:44.656 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-08T19:22:44.674 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-08T19:22:44.710 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-08T19:22:44.753 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-08T19:22:44.796 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-08T19:22:44.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-08T19:22:44.943 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-08T19:22:45.003 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-08T19:22:45.043 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-08T19:22:45.170 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-08T19:22:45.232 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-08T19:22:45.276 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-08T19:22:45.342 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-08T19:22:45.378 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-08T19:22:45.522 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-08T19:22:45.583 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-08T19:22:45.731 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-08T19:22:45.799 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-08T19:22:46.787 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-08T19:22:46.839 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-08T19:22:46.872 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-08T19:22:46.998 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-08T19:22:47.048 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-08T19:22:47.280 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-08T19:22:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-08T19:22:47.425 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-08T19:22:48.233 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-08T19:22:48.367 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-08T19:22:49.990 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-08T19:22:50.083 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-08T19:22:50.125 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-08T19:22:50.160 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-08T19:22:50.202 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-08T19:22:50.562 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-08T19:22:50.599 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-08T19:22:50.615 INFO:teuthology.orchestra.run.smithi119.stdout: Preparing : 1/1 2023-11-08T19:22:50.696 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-08T19:22:50.726 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-08T19:22:50.754 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-08T19:22:50.770 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-08T19:22:50.804 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 97/152 2023-11-08T19:22:50.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-08T19:22:50.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-08T19:22:51.236 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-08T19:22:51.300 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-08T19:22:51.346 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-08T19:22:51.384 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-08T19:22:51.458 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-08T19:22:51.493 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-08T19:22:51.509 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-08T19:22:51.537 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 101/152 2023-11-08T19:22:51.617 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-08T19:22:51.676 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-08T19:22:51.719 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-08T19:22:51.988 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-08T19:22:52.056 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-08T19:22:52.081 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-08T19:22:52.243 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-08T19:22:52.274 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-08T19:22:52.358 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 13/152 2023-11-08T19:22:52.732 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-08T19:22:52.873 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T19:22:52.959 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T19:22:53.177 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T19:22:53.232 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T19:22:53.734 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T19:22:53.771 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-08T19:22:53.817 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-08T19:22:53.927 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-08T19:22:53.994 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-08T19:22:54.021 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-08T19:22:54.025 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-08T19:22:54.066 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-08T19:22:54.077 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-08T19:22:54.177 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-08T19:22:54.226 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-08T19:22:54.277 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-08T19:22:54.393 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-08T19:22:54.431 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 27/152 2023-11-08T19:22:54.474 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 28/152 2023-11-08T19:22:54.570 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 29/152 2023-11-08T19:22:54.698 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-08T19:22:54.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-08T19:22:54.747 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-08T19:22:54.818 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-08T19:22:54.934 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-08T19:22:54.934 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-08T19:22:54.935 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-08T19:22:54.935 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:22:54.988 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-08T19:22:55.534 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-08T19:22:55.623 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-08T19:22:55.668 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-08T19:22:55.706 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 35/152 2023-11-08T19:22:55.745 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-08T19:22:55.894 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-08T19:22:55.949 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-08T19:22:56.068 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-08T19:22:56.321 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-08T19:22:56.368 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-08T19:22:56.409 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-08T19:22:56.480 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-08T19:22:56.539 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-08T19:22:56.600 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-08T19:22:56.792 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-08T19:22:56.840 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-08T19:22:56.886 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-08T19:22:57.013 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-08T19:22:57.366 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/152 2023-11-08T19:22:57.450 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-08T19:22:57.490 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-08T19:22:57.530 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-08T19:22:57.582 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-08T19:22:57.744 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-08T19:22:57.797 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-08T19:22:57.877 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-08T19:22:57.954 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-08T19:22:58.223 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-08T19:22:58.282 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-08T19:22:58.459 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-08T19:22:58.527 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-08T19:22:58.558 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-08T19:22:58.598 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-08T19:22:58.642 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-08T19:22:58.685 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-08T19:22:58.730 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-08T19:22:58.810 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-08T19:22:58.859 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-08T19:22:58.901 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-08T19:22:59.027 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-08T19:22:59.064 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-08T19:22:59.099 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-08T19:22:59.152 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-08T19:22:59.194 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-08T19:22:59.336 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-08T19:22:59.385 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-08T19:22:59.525 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-08T19:22:59.599 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-08T19:23:00.592 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-08T19:23:00.656 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-08T19:23:00.690 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-08T19:23:00.820 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-08T19:23:00.856 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-08T19:23:01.090 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-08T19:23:01.120 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-08T19:23:01.213 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-08T19:23:02.035 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-08T19:23:02.167 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-08T19:23:03.750 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-08T19:23:03.849 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-08T19:23:03.886 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-08T19:23:03.918 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-08T19:23:03.959 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-08T19:23:04.324 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-08T19:23:04.355 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-08T19:23:04.477 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 96/152 2023-11-08T19:23:04.511 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 97/152 2023-11-08T19:23:04.575 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-08T19:23:05.062 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-08T19:23:05.100 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-08T19:23:05.192 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 100/152 2023-11-08T19:23:05.218 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 101/152 2023-11-08T19:23:07.735 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-08T19:23:07.773 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-08T19:23:07.776 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-08T19:23:07.777 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2023-11-08T19:23:07.777 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2023-11-08T19:23:07.777 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2023-11-08T19:23:07.777 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2023-11-08T19:23:07.778 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2023-11-08T19:23:07.778 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2023-11-08T19:23:07.778 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2023-11-08T19:23:07.778 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:08.033 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-08T19:23:08.080 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-08T19:23:08.080 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:08.080 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-08T19:23:08.081 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:08.400 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 102/152 2023-11-08T19:23:08.443 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-08T19:23:08.614 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 103/152 2023-11-08T19:23:08.615 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-08T19:23:08.615 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-08T19:23:08.615 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:08.652 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-08T19:23:09.796 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-08T19:23:09.882 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-08T19:23:09.882 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:09.882 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-08T19:23:09.882 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T19:23:09.883 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T19:23:09.883 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:10.113 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-08T19:23:10.208 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-08T19:23:10.208 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:10.208 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-08T19:23:10.208 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T19:23:10.209 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T19:23:10.209 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:10.705 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-08T19:23:10.776 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-08T19:23:10.777 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:10.777 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-08T19:23:10.778 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T19:23:10.778 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T19:23:10.779 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:11.440 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-08T19:23:11.485 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-08T19:23:11.557 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 110/152 2023-11-08T19:23:11.672 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 111/152 2023-11-08T19:23:11.703 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-08T19:23:12.029 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-08T19:23:12.062 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-08T19:23:13.189 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-08T19:23:13.221 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-08T19:23:13.365 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-08T19:23:13.537 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 115/152 2023-11-08T19:23:13.574 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-08T19:23:13.646 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-08T19:23:13.646 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:13.647 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-08T19:23:13.647 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T19:23:13.647 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T19:23:13.647 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:13.723 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-08T19:23:13.764 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-08T19:23:14.639 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 118/152 2023-11-08T19:23:14.675 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-08T19:23:14.749 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-08T19:23:14.749 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:14.750 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-08T19:23:14.750 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T19:23:14.750 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T19:23:14.750 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:14.801 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-08T19:23:14.877 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-08T19:23:14.877 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:14.877 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-08T19:23:14.877 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:14.935 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-08T19:23:15.016 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-08T19:23:15.017 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:15.017 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-08T19:23:15.017 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T19:23:15.017 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T19:23:15.018 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:15.394 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-08T19:23:15.467 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-08T19:23:15.468 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:15.468 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-08T19:23:15.468 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T19:23:15.468 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T19:23:15.469 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:19.609 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 123/152 2023-11-08T19:23:19.665 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 124/152 2023-11-08T19:23:19.705 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 125/152 2023-11-08T19:23:19.987 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-08T19:23:20.106 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-08T19:23:20.169 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 128/152 2023-11-08T19:23:20.233 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-08T19:23:20.263 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-08T19:23:20.294 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 131/152 2023-11-08T19:23:20.340 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-08T19:23:20.463 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-08T19:23:20.539 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-08T19:23:20.615 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-08T19:23:20.941 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 104/152 2023-11-08T19:23:20.941 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /sys 2023-11-08T19:23:20.941 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /proc 2023-11-08T19:23:20.941 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /mnt 2023-11-08T19:23:20.942 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /var/tmp 2023-11-08T19:23:20.942 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /home 2023-11-08T19:23:20.942 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /root 2023-11-08T19:23:20.942 INFO:teuthology.orchestra.run.smithi119.stdout:skipping the directory /tmp 2023-11-08T19:23:20.942 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:21.169 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-08T19:23:21.218 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 105/152 2023-11-08T19:23:21.218 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:21.219 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-08T19:23:21.219 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:21.567 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T19:23:21.646 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T19:23:21.707 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T19:23:21.775 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 137/152 2023-11-08T19:23:22.918 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-08T19:23:23.002 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 106/152 2023-11-08T19:23:23.003 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:23.003 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-08T19:23:23.003 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T19:23:23.003 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T19:23:23.004 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:23.172 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-08T19:23:23.244 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-08T19:23:23.270 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-08T19:23:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-08T19:23:23.315 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 107/152 2023-11-08T19:23:23.315 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:23.315 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-08T19:23:23.316 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T19:23:23.316 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T19:23:23.316 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:23.436 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-08T19:23:23.480 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-08T19:23:23.517 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-08T19:23:23.517 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T19:23:23.620 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T19:23:23.650 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-08T19:23:23.684 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-08T19:23:23.717 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-08T19:23:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T19:23:23.760 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T19:23:23.853 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-08T19:23:23.929 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 108/152 2023-11-08T19:23:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-08T19:23:23.929 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T19:23:23.930 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T19:23:23.930 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:24.092 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T19:23:24.093 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T19:23:24.248 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T19:23:24.279 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T19:23:24.279 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T19:23:24.409 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T19:23:24.532 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T19:23:24.578 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-08T19:23:24.616 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 109/152 2023-11-08T19:23:24.665 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 110/152 2023-11-08T19:23:24.787 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 111/152 2023-11-08T19:23:24.804 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T19:23:24.805 INFO:teuthology.orchestra.run.smithi188.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-08T19:23:24.805 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:24.819 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-08T19:23:25.140 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 112/152 2023-11-08T19:23:25.203 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-08T19:23:26.361 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 113/152 2023-11-08T19:23:26.395 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-08T19:23:26.537 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 114/152 2023-11-08T19:23:26.705 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 115/152 2023-11-08T19:23:26.739 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-08T19:23:26.809 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 116/152 2023-11-08T19:23:26.809 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:26.810 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-08T19:23:26.810 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T19:23:26.810 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T19:23:26.810 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:26.864 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-08T19:23:26.903 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 117/152 2023-11-08T19:23:27.833 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 118/152 2023-11-08T19:23:27.865 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-08T19:23:27.937 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 119/152 2023-11-08T19:23:27.938 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:27.938 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-08T19:23:27.938 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T19:23:27.939 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T19:23:27.939 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:27.991 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-08T19:23:28.092 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7035.g75320 120/152 2023-11-08T19:23:28.092 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:28.093 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-08T19:23:28.093 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:28.167 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-08T19:23:28.275 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 121/152 2023-11-08T19:23:28.275 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:28.275 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-08T19:23:28.275 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T19:23:28.276 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T19:23:28.276 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:28.676 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-08T19:23:28.778 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 122/152 2023-11-08T19:23:28.778 INFO:teuthology.orchestra.run.smithi119.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T19:23:28.778 INFO:teuthology.orchestra.run.smithi119.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-08T19:23:28.778 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T19:23:28.779 INFO:teuthology.orchestra.run.smithi119.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T19:23:28.779 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:29.666 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 1/152 2023-11-08T19:23:29.666 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2/152 2023-11-08T19:23:29.667 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 3/152 2023-11-08T19:23:29.667 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 4/152 2023-11-08T19:23:29.667 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7035.g75320 5/152 2023-11-08T19:23:29.667 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-08T19:23:29.668 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-08T19:23:29.668 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-08T19:23:29.668 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 9/152 2023-11-08T19:23:29.668 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 10/152 2023-11-08T19:23:29.668 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 11/152 2023-11-08T19:23:29.669 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-08T19:23:29.669 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 13/152 2023-11-08T19:23:29.669 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 14/152 2023-11-08T19:23:29.669 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 15/152 2023-11-08T19:23:29.669 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 16/152 2023-11-08T19:23:29.670 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 17/152 2023-11-08T19:23:29.670 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 18/152 2023-11-08T19:23:29.670 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 19/152 2023-11-08T19:23:29.670 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 20/152 2023-11-08T19:23:29.670 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 21/152 2023-11-08T19:23:29.671 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 22/152 2023-11-08T19:23:29.671 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 23/152 2023-11-08T19:23:29.671 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 24/152 2023-11-08T19:23:29.671 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 25/152 2023-11-08T19:23:29.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 26/152 2023-11-08T19:23:29.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 27/152 2023-11-08T19:23:29.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 28/152 2023-11-08T19:23:29.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 29/152 2023-11-08T19:23:29.683 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 30/152 2023-11-08T19:23:29.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 31/152 2023-11-08T19:23:29.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 32/152 2023-11-08T19:23:29.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 33/152 2023-11-08T19:23:29.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 34/152 2023-11-08T19:23:29.684 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 35/152 2023-11-08T19:23:29.685 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 36/152 2023-11-08T19:23:29.685 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 37/152 2023-11-08T19:23:29.685 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 38/152 2023-11-08T19:23:29.685 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-08T19:23:29.685 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-08T19:23:29.686 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-08T19:23:29.686 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-08T19:23:29.686 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-08T19:23:29.686 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-08T19:23:29.686 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-08T19:23:29.687 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-08T19:23:29.687 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-08T19:23:29.687 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-08T19:23:29.687 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-08T19:23:29.687 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-08T19:23:29.687 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-08T19:23:29.688 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-08T19:23:29.688 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-08T19:23:29.688 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-08T19:23:29.688 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-08T19:23:29.688 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-08T19:23:29.689 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-08T19:23:29.689 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-08T19:23:29.689 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-08T19:23:29.689 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-08T19:23:29.689 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-08T19:23:29.690 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-08T19:23:29.690 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-08T19:23:29.690 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-08T19:23:29.690 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-08T19:23:29.691 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-08T19:23:29.691 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-08T19:23:29.691 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-08T19:23:29.691 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-08T19:23:29.691 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-08T19:23:29.692 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-08T19:23:29.692 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-08T19:23:29.692 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-08T19:23:29.692 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-08T19:23:29.692 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-08T19:23:29.692 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-08T19:23:29.693 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-08T19:23:29.693 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-08T19:23:29.693 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-08T19:23:29.693 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-08T19:23:29.693 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-08T19:23:29.694 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-08T19:23:29.694 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-08T19:23:29.694 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-08T19:23:29.694 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-08T19:23:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-08T19:23:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-08T19:23:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-08T19:23:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-08T19:23:29.695 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-08T19:23:29.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-08T19:23:29.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-08T19:23:29.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-08T19:23:29.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-08T19:23:29.696 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-08T19:23:29.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-08T19:23:29.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-08T19:23:29.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-08T19:23:29.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-08T19:23:29.697 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-08T19:23:29.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-08T19:23:29.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-08T19:23:29.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-08T19:23:29.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-08T19:23:29.698 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-08T19:23:29.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-08T19:23:29.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 107/152 2023-11-08T19:23:29.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-08T19:23:29.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-08T19:23:29.699 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-08T19:23:29.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-08T19:23:29.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-08T19:23:29.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-08T19:23:29.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-08T19:23:29.700 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-08T19:23:29.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-08T19:23:29.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-08T19:23:29.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-08T19:23:29.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-08T19:23:29.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-08T19:23:29.701 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-08T19:23:29.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-08T19:23:29.702 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-08T19:23:29.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-08T19:23:29.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-08T19:23:29.704 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-08T19:23:29.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-08T19:23:29.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-08T19:23:29.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-08T19:23:29.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-08T19:23:29.705 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-08T19:23:29.706 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-08T19:23:29.706 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 133/152 2023-11-08T19:23:29.706 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-08T19:23:29.706 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 135/152 2023-11-08T19:23:29.706 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-08T19:23:29.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-08T19:23:29.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-08T19:23:29.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-08T19:23:29.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-08T19:23:29.707 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-08T19:23:29.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-08T19:23:29.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-08T19:23:29.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-08T19:23:29.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-08T19:23:29.708 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-08T19:23:29.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-08T19:23:29.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-08T19:23:29.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-08T19:23:29.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T19:23:29.709 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-08T19:23:31.822 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T19:23:31.823 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:31.823 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2023-11-08T19:23:31.823 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:31.823 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:31.824 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:31.824 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:31.824 INFO:teuthology.orchestra.run.smithi188.stdout: acl-2.3.1-4.el9.x86_64 2023-11-08T19:23:31.824 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-08T19:23:31.824 INFO:teuthology.orchestra.run.smithi188.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-08T19:23:31.825 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.825 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.825 INFO:teuthology.orchestra.run.smithi188.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-08T19:23:31.825 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2023-11-08T19:23:31.825 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2023-11-08T19:23:31.826 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2023-11-08T19:23:31.826 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-08T19:23:31.826 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.826 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.827 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.827 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.827 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.828 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.828 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.828 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.828 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.828 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.829 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.829 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.829 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.829 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.829 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.830 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.830 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.830 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.830 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.831 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.831 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.831 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.831 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:31.831 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-08T19:23:31.832 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2023-11-08T19:23:31.832 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-08T19:23:31.832 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-08T19:23:31.832 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-08T19:23:31.832 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-08T19:23:31.833 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-08T19:23:31.833 INFO:teuthology.orchestra.run.smithi188.stdout: jq-1.6-16.el9.x86_64 2023-11-08T19:23:31.833 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-08T19:23:31.833 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-08T19:23:31.833 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-08T19:23:31.834 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.834 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.834 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.834 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-08T19:23:31.835 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-08T19:23:31.835 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-08T19:23:31.835 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-08T19:23:31.835 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-08T19:23:31.835 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.836 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.836 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-08T19:23:31.836 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.836 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-08T19:23:31.836 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-08T19:23:31.837 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-08T19:23:31.837 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-08T19:23:31.837 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-08T19:23:31.837 INFO:teuthology.orchestra.run.smithi188.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-08T19:23:31.837 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-08T19:23:31.838 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-08T19:23:31.838 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-08T19:23:31.838 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-08T19:23:31.838 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.11-1.el9.x86_64 2023-11-08T19:23:31.839 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-08T19:23:31.839 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-08T19:23:31.839 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-08T19:23:31.839 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-08T19:23:31.839 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-08T19:23:31.839 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-08T19:23:31.840 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-08T19:23:31.840 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.840 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.840 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.840 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-08T19:23:31.841 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-08T19:23:31.841 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-08T19:23:31.841 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-08T19:23:31.841 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-08T19:23:31.841 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-08T19:23:31.842 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-08T19:23:31.842 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-08T19:23:31.842 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-08T19:23:31.842 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-08T19:23:31.843 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-08T19:23:31.843 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-08T19:23:31.843 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-08T19:23:31.843 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-08T19:23:31.843 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-08T19:23:31.844 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-08T19:23:31.844 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-08T19:23:31.844 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-08T19:23:31.844 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-08T19:23:31.845 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-08T19:23:31.845 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-08T19:23:31.845 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-08T19:23:31.845 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-08T19:23:31.845 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-08T19:23:31.846 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-08T19:23:31.846 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-08T19:23:31.846 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-08T19:23:31.846 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2023-11-08T19:23:31.846 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-08T19:23:31.847 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-08T19:23:31.847 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-08T19:23:31.847 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-08T19:23:31.847 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-08T19:23:31.847 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-08T19:23:31.848 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-08T19:23:31.848 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.848 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.848 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-08T19:23:31.848 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-08T19:23:31.849 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-08T19:23:31.849 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.849 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-08T19:23:31.849 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-08T19:23:31.849 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-08T19:23:31.850 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-08T19:23:31.850 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-08T19:23:31.850 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-08T19:23:31.850 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-08T19:23:31.850 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-08T19:23:31.851 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-08T19:23:31.851 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-08T19:23:31.851 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-08T19:23:31.851 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.851 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.852 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:31.852 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2023-11-08T19:23:31.852 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-08T19:23:31.852 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-08T19:23:31.852 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-08T19:23:31.853 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-08T19:23:31.853 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-08T19:23:31.853 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-08T19:23:31.853 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:31.854 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2023-11-08T19:23:32.399 DEBUG:teuthology.parallel:result is None 2023-11-08T19:23:33.015 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 123/152 2023-11-08T19:23:33.073 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 124/152 2023-11-08T19:23:33.112 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 125/152 2023-11-08T19:23:33.392 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-08T19:23:33.513 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-08T19:23:33.582 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 128/152 2023-11-08T19:23:33.659 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-08T19:23:33.694 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-08T19:23:33.714 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 131/152 2023-11-08T19:23:33.760 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-08T19:23:33.863 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-08T19:23:33.945 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-08T19:23:33.994 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-08T19:23:34.881 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T19:23:34.925 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T19:23:34.995 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T19:23:35.063 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 137/152 2023-11-08T19:23:36.424 INFO:teuthology.orchestra.run.smithi119.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-08T19:23:36.523 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-08T19:23:36.571 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-08T19:23:36.709 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-08T19:23:36.750 INFO:teuthology.orchestra.run.smithi119.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-08T19:23:36.795 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-08T19:23:36.796 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T19:23:36.915 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T19:23:36.963 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-08T19:23:36.995 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-08T19:23:37.028 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-08T19:23:37.078 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T19:23:37.079 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T19:23:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T19:23:37.411 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T19:23:37.550 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T19:23:37.581 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T19:23:37.581 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T19:23:37.720 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T19:23:37.842 INFO:teuthology.orchestra.run.smithi119.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T19:23:38.097 INFO:teuthology.orchestra.run.smithi119.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T19:23:38.098 INFO:teuthology.orchestra.run.smithi119.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-08T19:23:38.098 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:42.583 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-2:18.0.0-7035.g7532011a.el9.x86_64 1/152 2023-11-08T19:23:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2/152 2023-11-08T19:23:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 3/152 2023-11-08T19:23:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 4/152 2023-11-08T19:23:42.584 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7035.g75320 5/152 2023-11-08T19:23:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 6/152 2023-11-08T19:23:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 7/152 2023-11-08T19:23:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 8/152 2023-11-08T19:23:42.585 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 9/152 2023-11-08T19:23:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 10/152 2023-11-08T19:23:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 11/152 2023-11-08T19:23:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 12/152 2023-11-08T19:23:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 13/152 2023-11-08T19:23:42.586 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_ 14/152 2023-11-08T19:23:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 15/152 2023-11-08T19:23:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 16/152 2023-11-08T19:23:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados-devel-2:18.0.0-7035.g7532011a.el9.x86_6 17/152 2023-11-08T19:23:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86 18/152 2023-11-08T19:23:42.587 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 19/152 2023-11-08T19:23:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7035.g7532011a.el 20/152 2023-11-08T19:23:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ceph-common-2:18.0.0-7035.g7532011a.el9. 21/152 2023-11-08T19:23:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_6 22/152 2023-11-08T19:23:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 23/152 2023-11-08T19:23:42.588 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 24/152 2023-11-08T19:23:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 25/152 2023-11-08T19:23:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 26/152 2023-11-08T19:23:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 27/152 2023-11-08T19:23:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 28/152 2023-11-08T19:23:42.589 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7035.g7532011a. 29/152 2023-11-08T19:23:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noa 30/152 2023-11-08T19:23:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.n 31/152 2023-11-08T19:23:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7035.g753 32/152 2023-11-08T19:23:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el 33/152 2023-11-08T19:23:42.590 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 34/152 2023-11-08T19:23:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.e 35/152 2023-11-08T19:23:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 36/152 2023-11-08T19:23:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephadm-2:18.0.0-7035.g7532011a.el9.noarch 37/152 2023-11-08T19:23:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 38/152 2023-11-08T19:23:42.591 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-08T19:23:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-08T19:23:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-08T19:23:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-08T19:23:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-08T19:23:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-08T19:23:42.592 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-08T19:23:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-08T19:23:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-08T19:23:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-08T19:23:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-08T19:23:42.593 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-08T19:23:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-08T19:23:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-08T19:23:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-08T19:23:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-08T19:23:42.594 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-08T19:23:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-08T19:23:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-08T19:23:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-08T19:23:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-08T19:23:42.595 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-08T19:23:42.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-08T19:23:42.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-08T19:23:42.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-08T19:23:42.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-08T19:23:42.596 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-08T19:23:42.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-08T19:23:42.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-08T19:23:42.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-08T19:23:42.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-08T19:23:42.597 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-08T19:23:42.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-08T19:23:42.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-08T19:23:42.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-08T19:23:42.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-08T19:23:42.598 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-08T19:23:42.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-08T19:23:42.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-08T19:23:42.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-08T19:23:42.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-08T19:23:42.599 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-08T19:23:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-08T19:23:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-08T19:23:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-08T19:23:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-08T19:23:42.600 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-08T19:23:42.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-08T19:23:42.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-08T19:23:42.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-08T19:23:42.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-08T19:23:42.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-08T19:23:42.601 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-08T19:23:42.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-08T19:23:42.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-08T19:23:42.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-08T19:23:42.602 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-08T19:23:42.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-08T19:23:42.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-08T19:23:42.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-08T19:23:42.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-08T19:23:42.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-08T19:23:42.603 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-08T19:23:42.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-08T19:23:42.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-08T19:23:42.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-08T19:23:42.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-08T19:23:42.604 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-08T19:23:42.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 107/152 2023-11-08T19:23:42.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-08T19:23:42.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-08T19:23:42.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-08T19:23:42.605 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-08T19:23:42.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-08T19:23:42.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-08T19:23:42.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-08T19:23:42.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-08T19:23:42.606 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-08T19:23:42.607 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-08T19:23:42.607 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-08T19:23:42.607 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-08T19:23:42.607 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-08T19:23:42.607 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-08T19:23:42.607 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-08T19:23:42.608 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-08T19:23:42.608 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-08T19:23:42.608 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-08T19:23:42.608 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-08T19:23:42.608 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-08T19:23:42.609 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-08T19:23:42.609 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-08T19:23:42.609 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-08T19:23:42.609 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-08T19:23:42.609 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-08T19:23:42.610 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:18.0.0-7035.g7532011a.el9.x86_64 133/152 2023-11-08T19:23:42.610 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-08T19:23:42.610 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 135/152 2023-11-08T19:23:42.610 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-08T19:23:42.610 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-08T19:23:42.611 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-08T19:23:42.611 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-08T19:23:42.611 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-08T19:23:42.611 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-08T19:23:42.611 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-08T19:23:42.611 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-08T19:23:42.612 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-08T19:23:42.612 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-08T19:23:42.612 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-08T19:23:42.612 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-08T19:23:42.612 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-08T19:23:42.613 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-08T19:23:42.613 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T19:23:42.613 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-08T19:23:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T19:23:44.702 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:44.702 INFO:teuthology.orchestra.run.smithi119.stdout:Upgraded: 2023-11-08T19:23:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-08T19:23:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: acl-2.3.1-4.el9.x86_64 2023-11-08T19:23:44.703 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-08T19:23:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-08T19:23:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: librados2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: librbd1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.704 INFO:teuthology.orchestra.run.smithi119.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-08T19:23:44.705 INFO:teuthology.orchestra.run.smithi119.stdout:Installed: 2023-11-08T19:23:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: automake-1.16.2-8.el9.noarch 2023-11-08T19:23:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: bison-3.7.4-5.el9.x86_64 2023-11-08T19:23:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-08T19:23:44.705 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-base-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-grafana-dashboards-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.706 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-immutable-object-cache-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mds-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-dashboard-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.707 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-modules-core-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mgr-rook-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-mon-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-osd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-prometheus-alerts-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.708 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-radosgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-selinux-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-test-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: ceph-volume-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: cephadm-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.709 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: cephfs-top-2:18.0.0-7035.g7532011a.el9.noarch 2023-11-08T19:23:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-08T19:23:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: flex-2.6.4-9.el9.x86_64 2023-11-08T19:23:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-08T19:23:44.710 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-08T19:23:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-08T19:23:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-08T19:23:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-08T19:23:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: jq-1.6-16.el9.x86_64 2023-11-08T19:23:44.711 INFO:teuthology.orchestra.run.smithi119.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-08T19:23:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-08T19:23:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-08T19:23:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: libcephfs2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: libcephsqlite-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.712 INFO:teuthology.orchestra.run.smithi119.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-08T19:23:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-08T19:23:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-08T19:23:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-08T19:23:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-08T19:23:44.713 INFO:teuthology.orchestra.run.smithi119.stdout: librados-devel-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: libradosstriper1-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-08T19:23:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: librgw2-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-08T19:23:44.714 INFO:teuthology.orchestra.run.smithi119.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-08T19:23:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-08T19:23:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-08T19:23:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-08T19:23:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-08T19:23:44.715 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-08T19:23:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-08T19:23:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-08T19:23:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-08T19:23:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-13.11-1.el9.x86_64 2023-11-08T19:23:44.716 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-08T19:23:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-08T19:23:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-08T19:23:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-08T19:23:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-08T19:23:44.717 INFO:teuthology.orchestra.run.smithi119.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-08T19:23:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-08T19:23:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-argparse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ceph-common-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cephfs-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.718 INFO:teuthology.orchestra.run.smithi119.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-08T19:23:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-08T19:23:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-08T19:23:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-08T19:23:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-08T19:23:44.719 INFO:teuthology.orchestra.run.smithi119.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-08T19:23:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-08T19:23:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-08T19:23:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-08T19:23:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-08T19:23:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-08T19:23:44.720 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-08T19:23:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-08T19:23:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-08T19:23:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-08T19:23:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-08T19:23:44.721 INFO:teuthology.orchestra.run.smithi119.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-08T19:23:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-08T19:23:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-08T19:23:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-08T19:23:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-08T19:23:44.722 INFO:teuthology.orchestra.run.smithi119.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-08T19:23:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-08T19:23:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-08T19:23:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-08T19:23:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-08T19:23:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-08T19:23:44.723 INFO:teuthology.orchestra.run.smithi119.stdout: python3-ply-3.11-14.el9.noarch 2023-11-08T19:23:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-08T19:23:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-08T19:23:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-08T19:23:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-08T19:23:44.724 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-08T19:23:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-08T19:23:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-08T19:23:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rados-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-08T19:23:44.725 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-08T19:23:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-08T19:23:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rgw-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-08T19:23:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-08T19:23:44.726 INFO:teuthology.orchestra.run.smithi119.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-08T19:23:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-08T19:23:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-08T19:23:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-08T19:23:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-08T19:23:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-08T19:23:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-08T19:23:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-08T19:23:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-08T19:23:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-fuse-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-mirror-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: rbd-nbd-2:18.0.0-7035.g7532011a.el9.x86_64 2023-11-08T19:23:44.728 INFO:teuthology.orchestra.run.smithi119.stdout: re2-1:20211101-3.el9.x86_64 2023-11-08T19:23:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-08T19:23:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-08T19:23:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-08T19:23:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-08T19:23:44.729 INFO:teuthology.orchestra.run.smithi119.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-08T19:23:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-08T19:23:44.730 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:44.730 INFO:teuthology.orchestra.run.smithi119.stdout:Complete! 2023-11-08T19:23:45.229 DEBUG:teuthology.parallel:result is None 2023-11-08T19:23:45.230 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:23:45.372 DEBUG:teuthology.orchestra.run.smithi119:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-08T19:23:45.441 INFO:teuthology.orchestra.run.smithi119.stdout:18.0.0-7035.g7532011a.el9 2023-11-08T19:23:45.442 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7035.g7532011a.el9 2023-11-08T19:23:45.442 INFO:teuthology.task.install:The correct ceph version 18.0.0-7035.g7532011a is installed. 2023-11-08T19:23:45.443 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4 2023-11-08T19:23:45.587 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-08T19:23:45.663 INFO:teuthology.orchestra.run.smithi188.stdout:18.0.0-7035.g7532011a.el9 2023-11-08T19:23:45.664 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7035.g7532011a.el9 2023-11-08T19:23:45.664 INFO:teuthology.task.install:The correct ceph version 18.0.0-7035.g7532011a is installed. 2023-11-08T19:23:45.664 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-08T19:23:45.665 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:45.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T19:23:45.747 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:45.748 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T19:23:45.855 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-08T19:23:45.856 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:45.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T19:23:45.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T19:23:46.061 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:46.062 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T19:23:46.167 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T19:23:46.308 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-08T19:23:46.308 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:46.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T19:23:46.394 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T19:23:46.517 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:46.518 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T19:23:46.619 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T19:23:46.763 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-08T19:23:46.764 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:46.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T19:23:46.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T19:23:46.929 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:46.930 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T19:23:47.034 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T19:23:47.130 INFO:teuthology.run_tasks:Running task ceph... 2023-11-08T19:23:47.198 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-08T19:23:47.199 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 777 /var/log/ceph 2023-11-08T19:23:47.205 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2023-11-08T19:23:47.306 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-08T19:23:47.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T19:23:47.312 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T19:23:47.412 INFO:tasks.ceph:Creating extra log directories... 2023-11-08T19:23:47.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T19:23:47.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T19:23:47.522 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-08T19:23:47.523 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-08T19:23:47.523 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-08T19:23:47.523 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:47.524 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T19:23:47.565 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:47.782 DEBUG:teuthology.orchestra.run.smithi119:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:47.859 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:47.860 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T19:23:47.920 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:48.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-08T19:23:48.300 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-08T19:23:48.301 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-08T19:23:48.301 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101', 'branch': 'main', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7452101', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'dry_run': True}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-batrick-testing-20231108.185857', 'sha1': 'e797689ab88984658d73bbc928b2ed06c676d14a'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 50, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3044, 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-batrick-testing-20231108.185857', 'suite_path': '/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/batrick/ceph.git', 'suite_sha1': 'e797689ab88984658d73bbc928b2ed06c676d14a', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi119.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPBNva1EB0X+Rk9GaiKcTAed2A2VkQxzqBk9e/XzaaFlee/0pzNH2bS7q+Q+rsPjvt+wMPeHsePwojxmU8NtGY=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOkKx9IMFOrO+AuaZwDyc1aHT05j90gu48HEtBN8zF7Nk+xxLdFUunqvdnYQFJjED4nXkaViAUM+irEx681lgs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'evicting unresponsive client', 'RECENT_CRASH'], 'sha1': '7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4', 'cluster': 'ceph'}}, {'kclient': None}, {'exec': {'mon.a': ['ceph config set mgr mgr/crash/warn_recent_interval 0']}}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_snapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-08_19:01:12', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-08T19:23:48.302 DEBUG:teuthology.orchestra.run.smithi119:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T19:23:48.311 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T19:23:48.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T19:23:48.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T19:23:48.480 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:48.481 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T19:23:48.525 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T19:23:48.525 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_1 2023-11-08T19:23:48.611 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T19:23:48.612 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:48.612 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 640 Links: 1 2023-11-08T19:23:48.612 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:48.612 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:48.613 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-11-08 19:23:38.520199453 +0000 2023-11-08T19:23:48.613 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-11-08 19:20:53.711149566 +0000 2023-11-08T19:23:48.613 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-11-08 19:20:53.711149566 +0000 2023-11-08T19:23:48.613 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-11-08 19:20:53.711149566 +0000 2023-11-08T19:23:48.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T19:23:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-11-08T19:23:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-11-08T19:23:48.729 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000456693 s, 1.1 MB/s 2023-11-08T19:23:48.733 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T19:23:48.776 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_2 2023-11-08T19:23:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T19:23:48.859 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:48.859 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 660 Links: 1 2023-11-08T19:23:48.860 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:48.860 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:48.860 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-11-08 19:23:38.521199429 +0000 2023-11-08T19:23:48.860 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-11-08 19:20:54.743124643 +0000 2023-11-08T19:23:48.860 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-11-08 19:20:54.743124643 +0000 2023-11-08T19:23:48.861 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-11-08 19:20:54.743124643 +0000 2023-11-08T19:23:48.861 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T19:23:48.985 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-11-08T19:23:48.985 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-11-08T19:23:48.985 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000404762 s, 1.3 MB/s 2023-11-08T19:23:48.990 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T19:23:49.050 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_3 2023-11-08T19:23:49.101 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T19:23:49.102 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:49.102 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 680 Links: 1 2023-11-08T19:23:49.102 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:49.102 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:49.102 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-11-08 19:23:38.522199405 +0000 2023-11-08T19:23:49.103 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-11-08 19:20:55.762100034 +0000 2023-11-08T19:23:49.103 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-11-08 19:20:55.762100034 +0000 2023-11-08T19:23:49.103 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-11-08 19:20:55.762100034 +0000 2023-11-08T19:23:49.103 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T19:23:49.193 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-11-08T19:23:49.193 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-11-08T19:23:49.193 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.00036674 s, 1.4 MB/s 2023-11-08T19:23:49.196 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T19:23:49.255 DEBUG:teuthology.orchestra.run.smithi119:> stat /dev/vg_nvme/lv_4 2023-11-08T19:23:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T19:23:49.308 INFO:teuthology.orchestra.run.smithi119.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:49.309 INFO:teuthology.orchestra.run.smithi119.stdout:Device: 5h/5d Inode: 696 Links: 1 2023-11-08T19:23:49.309 INFO:teuthology.orchestra.run.smithi119.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:49.309 INFO:teuthology.orchestra.run.smithi119.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:49.310 INFO:teuthology.orchestra.run.smithi119.stdout:Access: 2023-11-08 19:23:38.522199405 +0000 2023-11-08T19:23:49.310 INFO:teuthology.orchestra.run.smithi119.stdout:Modify: 2023-11-08 19:20:56.772075642 +0000 2023-11-08T19:23:49.310 INFO:teuthology.orchestra.run.smithi119.stdout:Change: 2023-11-08 19:20:56.772075642 +0000 2023-11-08T19:23:49.310 INFO:teuthology.orchestra.run.smithi119.stdout: Birth: 2023-11-08 19:20:56.772075642 +0000 2023-11-08T19:23:49.311 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T19:23:49.401 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records in 2023-11-08T19:23:49.401 INFO:teuthology.orchestra.run.smithi119.stderr:1+0 records out 2023-11-08T19:23:49.401 INFO:teuthology.orchestra.run.smithi119.stderr:512 bytes copied, 0.000327494 s, 1.6 MB/s 2023-11-08T19:23:49.404 DEBUG:teuthology.orchestra.run.smithi119:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T19:23:49.461 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:49.461 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:49.461 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T19:23:49.522 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T19:23:49.522 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2023-11-08T19:23:49.577 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T19:23:49.578 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:49.578 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-08T19:23:49.578 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:49.578 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:49.579 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-08 19:23:25.260429857 +0000 2023-11-08T19:23:49.579 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-08 19:20:53.716358493 +0000 2023-11-08T19:23:49.579 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-08 19:20:53.716358493 +0000 2023-11-08T19:23:49.579 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-08 19:20:53.716358493 +0000 2023-11-08T19:23:49.580 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T19:23:49.673 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-08T19:23:49.673 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-08T19:23:49.674 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000351519 s, 1.5 MB/s 2023-11-08T19:23:49.678 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T19:23:49.737 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2023-11-08T19:23:49.789 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T19:23:49.790 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:49.790 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-11-08T19:23:49.790 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:49.791 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:49.791 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-08 19:23:25.261429838 +0000 2023-11-08T19:23:49.791 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-08 19:20:54.778339049 +0000 2023-11-08T19:23:49.791 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-08 19:20:54.778339049 +0000 2023-11-08T19:23:49.791 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-08 19:20:54.778339049 +0000 2023-11-08T19:23:49.792 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T19:23:49.874 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-08T19:23:49.875 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-08T19:23:49.875 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000328205 s, 1.6 MB/s 2023-11-08T19:23:49.878 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T19:23:49.932 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2023-11-08T19:23:49.979 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T19:23:49.979 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:49.979 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 670 Links: 1 2023-11-08T19:23:49.979 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:49.980 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:49.980 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-08 19:23:25.262429820 +0000 2023-11-08T19:23:49.980 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-08 19:20:55.797320392 +0000 2023-11-08T19:23:49.980 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-08 19:20:55.798320374 +0000 2023-11-08T19:23:49.980 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-08 19:20:55.797320392 +0000 2023-11-08T19:23:49.981 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T19:23:50.063 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-08T19:23:50.063 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-08T19:23:50.063 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000328707 s, 1.6 MB/s 2023-11-08T19:23:50.066 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T19:23:50.122 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2023-11-08T19:23:50.176 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T19:23:50.176 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T19:23:50.177 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-11-08T19:23:50.177 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T19:23:50.177 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T19:23:50.177 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2023-11-08 19:23:25.263429802 +0000 2023-11-08T19:23:50.177 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2023-11-08 19:20:56.786302286 +0000 2023-11-08T19:23:50.178 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2023-11-08 19:20:56.786302286 +0000 2023-11-08T19:23:50.178 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2023-11-08 19:20:56.786302286 +0000 2023-11-08T19:23:50.178 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T19:23:50.264 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2023-11-08T19:23:50.264 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2023-11-08T19:23:50.264 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000340801 s, 1.5 MB/s 2023-11-08T19:23:50.267 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T19:23:50.323 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:50.324 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T19:23:50.324 INFO:tasks.ceph:Generating config... 2023-11-08T19:23:50.325 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-08T19:23:50.326 INFO:tasks.ceph:[client] debug client = 20 2023-11-08T19:23:50.326 INFO:tasks.ceph:[client] debug ms = 1 2023-11-08T19:23:50.326 INFO:tasks.ceph:[client] pid file = 2023-11-08T19:23:50.326 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-08T19:23:50.327 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-08T19:23:50.327 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-08T19:23:50.327 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-08T19:23:50.328 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-08T19:23:50.328 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-08T19:23:50.328 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-08T19:23:50.328 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-08T19:23:50.329 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-08T19:23:50.329 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-08T19:23:50.329 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-08T19:23:50.329 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-08T19:23:50.330 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-08T19:23:50.330 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-08T19:23:50.330 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-08T19:23:50.330 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-08T19:23:50.331 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-08T19:23:50.331 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-08T19:23:50.331 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-08T19:23:50.332 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-08T19:23:50.332 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-08T19:23:50.332 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-08T19:23:50.332 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-08T19:23:50.333 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-08T19:23:50.333 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-08T19:23:50.333 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-08T19:23:50.333 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-08T19:23:50.334 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-08T19:23:50.334 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-08T19:23:50.334 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-08T19:23:50.334 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-08T19:23:50.335 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-08T19:23:50.335 INFO:tasks.ceph:Setting up mon.a... 2023-11-08T19:23:50.336 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-08T19:23:50.480 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.keyring 2023-11-08T19:23:50.488 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-08T19:23:50.616 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T19:23:50.697 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.119'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2023-11-08T19:23:50.697 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-08T19:23:50.698 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:50.699 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T19:23:50.753 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.119 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:50.911 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:50.911 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: generated fsid ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:23:50.911 INFO:teuthology.orchestra.run.smithi119.stdout:setting min_mon_release = pacific 2023-11-08T19:23:50.912 INFO:teuthology.orchestra.run.smithi119.stdout:epoch 0 2023-11-08T19:23:50.912 INFO:teuthology.orchestra.run.smithi119.stdout:fsid ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:23:50.912 INFO:teuthology.orchestra.run.smithi119.stdout:last_changed 2023-11-08T19:23:50.910786+0000 2023-11-08T19:23:50.912 INFO:teuthology.orchestra.run.smithi119.stdout:created 2023-11-08T19:23:50.910786+0000 2023-11-08T19:23:50.912 INFO:teuthology.orchestra.run.smithi119.stdout:min_mon_release 16 (pacific) 2023-11-08T19:23:50.913 INFO:teuthology.orchestra.run.smithi119.stdout:election_strategy: 1 2023-11-08T19:23:50.913 INFO:teuthology.orchestra.run.smithi119.stdout:0: [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] mon.a 2023-11-08T19:23:50.913 INFO:teuthology.orchestra.run.smithi119.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2023-11-08T19:23:50.913 INFO:teuthology.orchestra.run.smithi119.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2023-11-08T19:23:50.913 INFO:teuthology.orchestra.run.smithi119.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-08T19:23:50.915 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T19:23:50.964 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d... 2023-11-08T19:23:50.965 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:23:50.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:23:51.117 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-11-08T19:23:51.118 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2023-11-08T19:23:51.118 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:23:51.118 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2023-11-08T19:23:51.118 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.119 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2023-11-08T19:23:51.119 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.119 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2023-11-08T19:23:51.119 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.120 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2023-11-08T19:23:51.120 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2023-11-08T19:23:51.120 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.120 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2023-11-08T19:23:51.120 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2023-11-08T19:23:51.121 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.121 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:23:51.121 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2023-11-08T19:23:51.121 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.121 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:23:51.121 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:23:51.122 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.122 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2023-11-08T19:23:51.122 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.122 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2023-11-08T19:23:51.123 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2023-11-08T19:23:51.123 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:23:51.123 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:23:51.123 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2023-11-08T19:23:51.123 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:23:51.124 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2023-11-08T19:23:51.124 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:23:51.124 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:23:51.124 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2023-11-08T19:23:51.124 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.124 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:23:51.125 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.125 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2023-11-08T19:23:51.125 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.125 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2023-11-08T19:23:51.125 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.126 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2023-11-08T19:23:51.126 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2023-11-08T19:23:51.126 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2023-11-08T19:23:51.126 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-08T19:23:51.126 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2023-11-08T19:23:51.127 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:23:51.127 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.127 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2023-11-08T19:23:51.127 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2023-11-08T19:23:51.127 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.127 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2023-11-08T19:23:51.128 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2023-11-08T19:23:51.128 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:23:51.128 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.128 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2023-11-08T19:23:51.128 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2023-11-08T19:23:51.129 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.129 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2023-11-08T19:23:51.129 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2023-11-08T19:23:51.129 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2023-11-08T19:23:51.129 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2023-11-08T19:23:51.129 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.130 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2023-11-08T19:23:51.130 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2023-11-08T19:23:51.130 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.130 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:23:51.130 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.131 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2023-11-08T19:23:51.131 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.131 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2023-11-08T19:23:51.131 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.131 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:23:51.132 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2023-11-08T19:23:51.132 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.132 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2023-11-08T19:23:51.132 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2023-11-08T19:23:51.132 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2023-11-08T19:23:51.132 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2023-11-08T19:23:51.133 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2023-11-08T19:23:51.133 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2023-11-08T19:23:51.133 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:23:51.133 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2023-11-08T19:23:51.133 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2023-11-08T19:23:51.134 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:23:51.134 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2023-11-08T19:23:51.134 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:23:51.134 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:23:51.134 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2023-11-08T19:23:51.135 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-11-08T19:23:51.135 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.135 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2023-11-08T19:23:51.135 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:23:51.135 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2023-11-08T19:23:51.135 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-11-08T19:23:51.136 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-11-08T19:23:51.136 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:51.136 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:51.136 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2023-11-08T19:23:51.136 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.137 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2023-11-08T19:23:51.137 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:23:51.137 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-11-08T19:23:51.137 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2023-11-08T19:23:51.137 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-11-08T19:23:51.138 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2023-11-08T19:23:51.138 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2023-11-08T19:23:51.138 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:51.138 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2023-11-08T19:23:51.138 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2023-11-08T19:23:51.139 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:51.139 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.139 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:23:51.139 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:23:51.139 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:23:51.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.140 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:23:51.140 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:23:51.140 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:23:51.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.140 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2023-11-08T19:23:51.141 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:23:51.141 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.141 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:23:51.141 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2023-11-08T19:23:51.141 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:23:51.142 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2023-11-08T19:23:51.142 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2023-11-08T19:23:51.142 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2023-11-08T19:23:51.142 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2023-11-08T19:23:51.142 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:23:51.143 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:23:51.143 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2023-11-08T19:23:51.143 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-11-08T19:23:51.143 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:23:51.143 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2023-11-08T19:23:51.144 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-11-08T19:23:51.144 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-11-08T19:23:51.144 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2023-11-08T19:23:51.144 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2023-11-08T19:23:51.144 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2023-11-08T19:23:51.145 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-11-08T19:23:51.145 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-11-08T19:23:51.145 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-11-08T19:23:51.145 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-11-08T19:23:51.145 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2023-11-08T19:23:51.146 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2023-11-08T19:23:51.146 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2023-11-08T19:23:51.146 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:23:51.146 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2023-11-08T19:23:51.146 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2023-11-08T19:23:51.147 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2023-11-08T19:23:51.147 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2023-11-08T19:23:51.147 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-11-08T19:23:51.147 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-11-08T19:23:51.147 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-11-08T19:23:51.152 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2023-11-08T19:23:51.152 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2023-11-08T19:23:51.152 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:23:51.153 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2023-11-08T19:23:51.153 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.153 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2023-11-08T19:23:51.153 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.153 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2023-11-08T19:23:51.154 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.154 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2023-11-08T19:23:51.154 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2023-11-08T19:23:51.154 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.154 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2023-11-08T19:23:51.155 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2023-11-08T19:23:51.155 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.155 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:23:51.155 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2023-11-08T19:23:51.155 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:23:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:23:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2023-11-08T19:23:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2023-11-08T19:23:51.157 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2023-11-08T19:23:51.157 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:23:51.157 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:23:51.157 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2023-11-08T19:23:51.158 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:23:51.158 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2023-11-08T19:23:51.158 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:23:51.158 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:23:51.158 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2023-11-08T19:23:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:23:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2023-11-08T19:23:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.159 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2023-11-08T19:23:51.160 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.160 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2023-11-08T19:23:51.160 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2023-11-08T19:23:51.160 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2023-11-08T19:23:51.160 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-08T19:23:51.160 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2023-11-08T19:23:51.161 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:23:51.161 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.161 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2023-11-08T19:23:51.161 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2023-11-08T19:23:51.161 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.161 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2023-11-08T19:23:51.162 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2023-11-08T19:23:51.162 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:23:51.162 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.162 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2023-11-08T19:23:51.162 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2023-11-08T19:23:51.163 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.163 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2023-11-08T19:23:51.163 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2023-11-08T19:23:51.163 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2023-11-08T19:23:51.163 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2023-11-08T19:23:51.164 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.164 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2023-11-08T19:23:51.164 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2023-11-08T19:23:51.164 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.164 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:23:51.164 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.165 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2023-11-08T19:23:51.165 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.165 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2023-11-08T19:23:51.165 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.165 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:23:51.165 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2023-11-08T19:23:51.166 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.166 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2023-11-08T19:23:51.166 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2023-11-08T19:23:51.166 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2023-11-08T19:23:51.166 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2023-11-08T19:23:51.167 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2023-11-08T19:23:51.167 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2023-11-08T19:23:51.167 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:23:51.167 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2023-11-08T19:23:51.167 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2023-11-08T19:23:51.168 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:23:51.168 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2023-11-08T19:23:51.168 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:23:51.168 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:23:51.168 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2023-11-08T19:23:51.169 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-08T19:23:51.169 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.169 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2023-11-08T19:23:51.169 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:23:51.169 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2023-11-08T19:23:51.169 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-08T19:23:51.170 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-08T19:23:51.170 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:51.170 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:51.170 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2023-11-08T19:23:51.170 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.171 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2023-11-08T19:23:51.171 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:23:51.171 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-08T19:23:51.171 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2023-11-08T19:23:51.171 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-08T19:23:51.172 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2023-11-08T19:23:51.172 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2023-11-08T19:23:51.172 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:23:51.172 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2023-11-08T19:23:51.172 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2023-11-08T19:23:51.173 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:23:51.173 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.173 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:23:51.173 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:23:51.173 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:23:51.173 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.174 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:23:51.174 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:23:51.174 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:23:51.174 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.175 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2023-11-08T19:23:51.175 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:23:51.175 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.175 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:23:51.175 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2023-11-08T19:23:51.175 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:23:51.176 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2023-11-08T19:23:51.176 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2023-11-08T19:23:51.176 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2023-11-08T19:23:51.176 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2023-11-08T19:23:51.176 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:23:51.177 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:23:51.177 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2023-11-08T19:23:51.177 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2023-11-08T19:23:51.177 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:23:51.177 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2023-11-08T19:23:51.178 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-08T19:23:51.178 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-08T19:23:51.178 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2023-11-08T19:23:51.178 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2023-11-08T19:23:51.178 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2023-11-08T19:23:51.178 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-08T19:23:51.179 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-08T19:23:51.179 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-08T19:23:51.179 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-08T19:23:51.179 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2023-11-08T19:23:51.179 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2023-11-08T19:23:51.179 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2023-11-08T19:23:51.180 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:23:51.180 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2023-11-08T19:23:51.180 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2023-11-08T19:23:51.180 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2023-11-08T19:23:51.180 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2023-11-08T19:23:51.180 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-08T19:23:51.181 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-08T19:23:51.181 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-08T19:23:51.196 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-08T19:23:51.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-08T19:23:51.343 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-08T19:23:51.344 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:51.344 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-08T19:23:51.396 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:51.396 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-08T19:23:51.448 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:23:51.449 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:51.449 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T19:23:51.450 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T19:23:51.597 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:23:51.598 DEBUG:teuthology.orchestra.run.smithi119:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:51.694 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:23:51.694 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:51.694 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T19:23:51.695 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T19:23:51.841 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:23:51.842 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:51.939 INFO:tasks.ceph:Setting up mon nodes... 2023-11-08T19:23:51.940 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-08T19:23:51.940 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T19:23:52.132 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T19:23:52.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T19:23:52.370 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T19:23:52.382 INFO:tasks.ceph:Setting up mds nodes... 2023-11-08T19:23:52.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T19:23:52.574 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T19:23:52.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-08T19:23:52.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T19:23:52.854 INFO:teuthology.orchestra.run.smithi119.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T19:23:52.863 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-08T19:23:52.958 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T19:23:53.147 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T19:23:53.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-08T19:23:53.255 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T19:23:53.431 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T19:23:53.441 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-08T19:23:53.536 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-08T19:23:53.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-08T19:23:53.666 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-08T19:23:53.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-08T19:23:53.859 INFO:teuthology.orchestra.run.smithi119.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-08T19:23:53.906 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-08T19:23:54.041 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-08T19:23:54.091 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-08T19:23:54.208 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-08T19:23:54.259 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-08T19:23:54.259 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi119.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T19:23:54.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-08T19:23:54.361 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:54.362 INFO:tasks.ceph:role: osd.0 2023-11-08T19:23:54.362 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:23:54.363 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T19:23:54.462 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:23:54.462 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:23:54.463 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:23:54.463 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:23:54.463 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:23:54.463 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:23:54.464 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:23:54.464 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:23:54.464 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:23:54.465 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:23:54.998 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-11-08T19:23:55.005 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-11-08T19:23:55.005 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-08T19:23:55.196 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-08T19:23:55.307 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-08T19:23:55.390 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:55.391 INFO:tasks.ceph:role: osd.1 2023-11-08T19:23:55.391 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:23:55.391 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T19:23:55.484 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:23:55.485 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:23:55.485 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:23:55.485 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:23:55.486 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:23:55.486 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:23:55.486 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:23:55.486 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:23:55.487 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:23:55.487 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:23:55.970 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-11-08T19:23:55.976 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-11-08T19:23:55.976 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-08T19:23:56.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-08T19:23:56.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-08T19:23:56.273 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:56.274 INFO:tasks.ceph:role: osd.2 2023-11-08T19:23:56.274 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:23:56.274 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T19:23:56.358 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:23:56.358 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:23:56.359 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:23:56.359 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:23:56.359 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:23:56.359 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:23:56.359 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:23:56.360 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:23:56.360 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:23:56.360 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:23:56.879 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-11-08T19:23:56.886 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-11-08T19:23:56.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-08T19:23:57.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-08T19:23:57.159 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-08T19:23:57.255 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T19:23:57.255 INFO:tasks.ceph:role: osd.3 2023-11-08T19:23:57.255 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:23:57.256 DEBUG:teuthology.orchestra.run.smithi119:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T19:23:57.358 INFO:teuthology.orchestra.run.smithi119.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:23:57.359 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:23:57.359 INFO:teuthology.orchestra.run.smithi119.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:23:57.359 INFO:teuthology.orchestra.run.smithi119.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:23:57.360 INFO:teuthology.orchestra.run.smithi119.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:23:57.360 INFO:teuthology.orchestra.run.smithi119.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:23:57.360 INFO:teuthology.orchestra.run.smithi119.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:23:57.360 INFO:teuthology.orchestra.run.smithi119.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:23:57.360 INFO:teuthology.orchestra.run.smithi119.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:23:57.361 INFO:teuthology.orchestra.run.smithi119.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:23:57.817 INFO:teuthology.orchestra.run.smithi119.stdout:Discarding blocks...Done. 2023-11-08T19:23:57.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi119.front.sepia.ceph.com -o noatime 2023-11-08T19:23:57.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-08T19:23:57.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-08T19:23:58.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:23:58.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:23:58.209+0000 7ff504012500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-08T19:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:23:58.210+0000 7ff504012500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-08T19:23:58.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:23:58.210+0000 7ff504012500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T19:23:58.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:23:58.210+0000 7ff504012500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T19:23:58.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:23:58.210+0000 7ff504012500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T19:23:58.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:23:58.214+0000 7ff504012500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-08T19:24:01.144 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-08T19:24:01.243 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:01.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:01.378+0000 7f7c1fa69500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-08T19:24:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:01.378+0000 7f7c1fa69500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-08T19:24:01.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:01.379+0000 7f7c1fa69500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T19:24:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:01.379+0000 7f7c1fa69500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T19:24:01.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:01.379+0000 7f7c1fa69500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T19:24:01.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:01.381+0000 7f7c1fa69500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-08T19:24:04.306 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-08T19:24:04.406 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:04.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:04.545+0000 7fdf2ee2b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-08T19:24:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:04.546+0000 7fdf2ee2b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-08T19:24:04.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:04.546+0000 7fdf2ee2b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T19:24:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:04.546+0000 7fdf2ee2b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T19:24:04.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:04.546+0000 7fdf2ee2b500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T19:24:04.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:04.548+0000 7fdf2ee2b500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-08T19:24:07.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-08T19:24:07.831 DEBUG:teuthology.orchestra.run.smithi119:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:07.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:07.972+0000 7fdff360d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-08T19:24:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:07.973+0000 7fdff360d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-08T19:24:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:07.973+0000 7fdff360d500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T19:24:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:07.973+0000 7fdff360d500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T19:24:07.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:07.973+0000 7fdff360d500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T19:24:07.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:07.975+0000 7fdff360d500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-08T19:24:11.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-08T19:24:11.250 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:11.354 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:11.354 INFO:tasks.ceph:role: osd.4 2023-11-08T19:24:11.354 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:24:11.355 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T19:24:11.457 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:11.457 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:11.457 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:11.457 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:24:11.457 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:11.458 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:11.458 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:11.458 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:24:11.458 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:11.458 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:11.979 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-08T19:24:11.986 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-08T19:24:11.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:12.171 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:12.264 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:12.342 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:12.342 INFO:tasks.ceph:role: osd.5 2023-11-08T19:24:12.343 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:24:12.343 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T19:24:12.429 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:12.429 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:12.429 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:12.430 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:24:12.430 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:12.430 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:12.430 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:12.431 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:24:12.431 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:12.431 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:12.909 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-08T19:24:12.915 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-08T19:24:12.916 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:13.035 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:13.121 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:13.199 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:13.200 INFO:tasks.ceph:role: osd.6 2023-11-08T19:24:13.200 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:24:13.200 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T19:24:13.287 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:24:13.288 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:13.288 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:13.289 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:13.289 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:24:13.289 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:13.289 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:13.811 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-08T19:24:13.817 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-08T19:24:13.818 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:14.025 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:14.135 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:14.234 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T19:24:14.234 INFO:tasks.ceph:role: osd.7 2023-11-08T19:24:14.235 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:24:14.235 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T19:24:14.337 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T19:24:14.338 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T19:24:14.338 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T19:24:14.338 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T19:24:14.338 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T19:24:14.339 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2023-11-08T19:24:14.339 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T19:24:14.339 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T19:24:14.339 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T19:24:14.340 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T19:24:14.802 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2023-11-08T19:24:14.807 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2023-11-08T19:24:14.808 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:14.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:15.063 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:15.228 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:15.226+0000 7fcbc248a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-08T19:24:15.229 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:15.227+0000 7fcbc248a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-08T19:24:15.229 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:15.227+0000 7fcbc248a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T19:24:15.229 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:15.227+0000 7fcbc248a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T19:24:15.229 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:15.227+0000 7fcbc248a500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T19:24:15.234 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:15.232+0000 7fcbc248a500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-08T19:24:18.182 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-08T19:24:18.277 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:18.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:18.313 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:18.407 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:18.407+0000 7fea8d413500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-08T19:24:18.408 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:18.407+0000 7fea8d413500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-08T19:24:18.409 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:18.407+0000 7fea8d413500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T19:24:18.409 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:18.407+0000 7fea8d413500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T19:24:18.409 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:18.407+0000 7fea8d413500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T19:24:18.410 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:18.409+0000 7fea8d413500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-08T19:24:21.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-08T19:24:21.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:21.571 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:21.570+0000 7f489840e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-08T19:24:21.572 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:21.571+0000 7f489840e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-08T19:24:21.572 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:21.571+0000 7f489840e500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T19:24:21.572 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:21.571+0000 7f489840e500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T19:24:21.573 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:21.571+0000 7f489840e500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T19:24:21.574 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:21.573+0000 7f489840e500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-08T19:24:24.503 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-08T19:24:24.600 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:24.739 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:24.738+0000 7f0f07e0d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-08T19:24:24.740 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:24.739+0000 7f0f07e0d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-08T19:24:24.741 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:24.739+0000 7f0f07e0d500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T19:24:24.741 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:24.739+0000 7f0f07e0d500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T19:24:24.741 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:24.739+0000 7f0f07e0d500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T19:24:24.742 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:24.742+0000 7f0f07e0d500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-08T19:24:27.666 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-08T19:24:27.764 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-08T19:24:27.765 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:27.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-08T19:24:27.867 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:27.868 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-08T19:24:27.964 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:27.965 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-08T19:24:28.059 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:28.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-08T19:24:28.148 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:28.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-08T19:24:28.289 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:28.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-08T19:24:28.385 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:28.385 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-08T19:24:28.479 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:28.479 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-08T19:24:28.581 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:28.582 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-08T19:24:28.676 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:28.676 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-08T19:24:28.768 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:28.768 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-08T19:24:28.907 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:28.908 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-08T19:24:29.005 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:29.006 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-08T19:24:29.103 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:29.103 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-08T19:24:29.197 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:29.197 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-08T19:24:29.258 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:29.259 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-08T19:24:29.313 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:29.314 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-08T19:24:29.375 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:29.375 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-08T19:24:29.429 INFO:tasks.ceph:Adding keys to all mons... 2023-11-08T19:24:29.430 DEBUG:teuthology.orchestra.run.smithi119:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T19:24:29.439 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T19:24:29.528 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.y] 2023-11-08T19:24:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ== 2023-11-08T19:24:29.529 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-11-08T19:24:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg== 2023-11-08T19:24:29.529 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-11-08T19:24:29.529 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw== 2023-11-08T19:24:29.530 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.0] 2023-11-08T19:24:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA== 2023-11-08T19:24:29.530 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.1] 2023-11-08T19:24:29.530 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A== 2023-11-08T19:24:29.530 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.2] 2023-11-08T19:24:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA== 2023-11-08T19:24:29.531 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.3] 2023-11-08T19:24:29.531 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ== 2023-11-08T19:24:29.531 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr.x] 2023-11-08T19:24:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA== 2023-11-08T19:24:29.532 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-11-08T19:24:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ== 2023-11-08T19:24:29.532 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-11-08T19:24:29.532 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ== 2023-11-08T19:24:29.533 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.4] 2023-11-08T19:24:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ== 2023-11-08T19:24:29.533 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.5] 2023-11-08T19:24:29.533 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw== 2023-11-08T19:24:29.533 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.6] 2023-11-08T19:24:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g== 2023-11-08T19:24:29.534 INFO:teuthology.orchestra.run.smithi119.stdout:[osd.7] 2023-11-08T19:24:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ== 2023-11-08T19:24:29.534 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-11-08T19:24:29.534 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:24:29.535 INFO:teuthology.orchestra.run.smithi119.stdout:[client.2] 2023-11-08T19:24:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q== 2023-11-08T19:24:29.535 INFO:teuthology.orchestra.run.smithi119.stdout:[client.1] 2023-11-08T19:24:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ== 2023-11-08T19:24:29.535 INFO:teuthology.orchestra.run.smithi119.stdout:[client.3] 2023-11-08T19:24:29.535 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ== 2023-11-08T19:24:29.537 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2023-11-08T19:24:29.537 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ== 2023-11-08T19:24:29.537 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-08T19:24:29.538 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg== 2023-11-08T19:24:29.538 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-08T19:24:29.538 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw== 2023-11-08T19:24:29.538 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2023-11-08T19:24:29.539 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA== 2023-11-08T19:24:29.539 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2023-11-08T19:24:29.539 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A== 2023-11-08T19:24:29.539 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2023-11-08T19:24:29.539 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA== 2023-11-08T19:24:29.540 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2023-11-08T19:24:29.540 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ== 2023-11-08T19:24:29.540 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2023-11-08T19:24:29.540 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA== 2023-11-08T19:24:29.540 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-08T19:24:29.540 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ== 2023-11-08T19:24:29.541 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-08T19:24:29.541 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ== 2023-11-08T19:24:29.541 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2023-11-08T19:24:29.541 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ== 2023-11-08T19:24:29.541 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2023-11-08T19:24:29.542 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw== 2023-11-08T19:24:29.542 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2023-11-08T19:24:29.542 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g== 2023-11-08T19:24:29.542 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2023-11-08T19:24:29.542 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ== 2023-11-08T19:24:29.543 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2023-11-08T19:24:29.543 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:24:29.543 INFO:teuthology.orchestra.run.smithi188.stdout:[client.2] 2023-11-08T19:24:29.543 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q== 2023-11-08T19:24:29.543 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2023-11-08T19:24:29.544 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ== 2023-11-08T19:24:29.544 INFO:teuthology.orchestra.run.smithi188.stdout:[client.3] 2023-11-08T19:24:29.544 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ== 2023-11-08T19:24:29.544 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:29.553 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:29.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:29.707 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:29.854 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:29.859 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:30.002 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.162 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.309 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.314 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.470 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:30.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:30.621 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T19:24:30.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:30.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:30.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:30.923 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T19:24:31.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.072 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.218 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.227 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.522 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.527 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T19:24:31.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.684 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.841 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.990 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:31.998 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:32.138 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:32.144 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T19:24:32.287 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-08T19:24:32.287 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-08T19:24:32.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T19:24:32.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-08T19:24:32.634 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-08T19:24:32.735 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T19:24:32.894 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-08T19:24:32.983 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-08T19:24:33.082 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T19:24:33.238 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-08T19:24:33.319 DEBUG:teuthology.orchestra.run.smithi119:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:33.328 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T19:24:33.381 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-08T19:24:33.382 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-08T19:24:33.383 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-08T19:24:33.388 INFO:tasks.ceph.mon.a:Started 2023-11-08T19:24:33.389 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-08T19:24:33.389 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-08T19:24:33.394 INFO:tasks.ceph.mon.b:Started 2023-11-08T19:24:33.395 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-08T19:24:33.395 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-08T19:24:33.400 INFO:tasks.ceph.mon.c:Started 2023-11-08T19:24:33.400 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-08T19:24:33.401 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-08T19:24:33.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-08T19:24:33.406 INFO:tasks.ceph.mgr.y:Started 2023-11-08T19:24:33.407 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-08T19:24:33.407 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-08T19:24:33.412 INFO:tasks.ceph.mgr.x:Started 2023-11-08T19:24:33.412 INFO:tasks.ceph:Setting crush tunables to default 2023-11-08T19:24:33.412 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph osd crush tunables default 2023-11-08T19:24:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.595+0000 7fe49304d640 1 Processor -- start 2023-11-08T19:24:33.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.595+0000 7fe49304d640 1 -- start start 2023-11-08T19:24:33.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49304d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe48c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49304d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 0x7fe48c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49304d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48c0b27c0 con 0x7fe48c0d3ac0 2023-11-08T19:24:33.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49304d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe48c0b2aa0 con 0x7fe48c160910 2023-11-08T19:24:33.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49304d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c0d1640 con 0x7fe48c160530 2023-11-08T19:24:33.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49204b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe48c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49204b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe48c0cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:35776/0 (socket says 172.21.15.119:35776) 2023-11-08T19:24:33.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.597+0000 7fe49204b640 1 -- 172.21.15.119:0/3691875681 learned_addr learned my addr 172.21.15.119:0/3691875681 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:33.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49204b640 1 -- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 msgr2=0x7fe48c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-08T19:24:33.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49204b640 1 -- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 msgr2=0x7fe48c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T19:24:33.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49204b640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe48c0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T19:24:33.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49204b640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe48c0cd960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T19:24:33.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 0x7fe48c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49184a640 1 -- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 msgr2=0x7fe48c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T19:24:33.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49184a640 1 -- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 msgr2=0x7fe48c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T19:24:33.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 0x7fe48c0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T19:24:33.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.598+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 0x7fe48c0cdea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.617+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552372665 0 0) 0x7fe48c0b27c0 con 0x7fe48c0d3ac0 2023-11-08T19:24:33.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.617+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe478003520 con 0x7fe48c0d3ac0 2023-11-08T19:24:33.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 601729067 0 0) 0x7fe478003520 con 0x7fe48c0d3ac0 2023-11-08T19:24:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 msgr2=0x7fe48c0cd960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:24:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe48c0cd960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 msgr2=0x7fe48c0cdea0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T19:24:33.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 --2- 172.21.15.119:0/3691875681 >> v2:172.21.15.119:3300/0 conn(0x7fe48c160910 0x7fe48c0cdea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c0d1920 con 0x7fe48c0d3ac0 2023-11-08T19:24:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe4840019f0 con 0x7fe48c0d3ac0 2023-11-08T19:24:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 >> v1:172.21.15.188:6789/0 conn(0x7fe48c0d3ac0 legacy=0x7fe48c0cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe478003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe478003ff0 0x7fe4780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478004d00 0x7fe478005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c0b2aa0 con 0x7fe478003ff0 2023-11-08T19:24:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c0d1640 con 0x7fe478004d00 2023-11-08T19:24:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47800bae0 con 0x7fe48c160530 2023-11-08T19:24:33.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe478003ff0 0x7fe4780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49204b640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe478003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49284c640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478004d00 0x7fe478005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49184a640 1 -- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 msgr2=0x7fe478003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe48c160530 0x7fe478003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49184a640 1 -- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478004d00 msgr2=0x7fe478005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:33.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe478004d00 0x7fe478005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.623+0000 7fe49184a640 1 -- 172.21.15.119:0/3691875681 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478003520 con 0x7fe478003ff0 2023-11-08T19:24:33.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.624+0000 7fe49184a640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe478003ff0 0x7fe4780047c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00b9e0 tx=0x7fe4780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b755133787837cc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.625+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c032630 con 0x7fe478003ff0 2023-11-08T19:24:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.625+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c032790 con 0x7fe478003ff0 2023-11-08T19:24:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.625+0000 7fe491049640 1 -- 172.21.15.119:0/3691875681 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c032b40 con 0x7fe478003ff0 2023-11-08T19:24:33.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49304d640 1 -- 172.21.15.119:0/3691875681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe478003ff0 msgr2=0x7fe4780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49304d640 1 --2- 172.21.15.119:0/3691875681 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe478003ff0 0x7fe4780047c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00b9e0 tx=0x7fe4780057f0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49284c640 1 -- 172.21.15.119:0/3691875681 reap_dead start 2023-11-08T19:24:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49304d640 1 -- 172.21.15.119:0/3691875681 shutdown_connections 2023-11-08T19:24:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49304d640 1 -- 172.21.15.119:0/3691875681 >> 172.21.15.119:0/3691875681 conn(0x7fe48c0d2ec0 msgr2=0x7fe48c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49304d640 1 -- 172.21.15.119:0/3691875681 shutdown_connections 2023-11-08T19:24:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.626+0000 7fe49304d640 1 -- 172.21.15.119:0/3691875681 wait complete. 2023-11-08T19:24:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.628+0000 7fe49304d640 1 Processor -- start 2023-11-08T19:24:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.628+0000 7fe49304d640 1 -- start start 2023-11-08T19:24:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49304d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe478003ff0 0x7fe48c15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49304d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 0x7fe48c0b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49304d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe48c0b11b0 0x7fe48c0b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:33.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49304d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c0b1630 con 0x7fe48c1f30c0 2023-11-08T19:24:33.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49304d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c15cd50 con 0x7fe48c0b11b0 2023-11-08T19:24:33.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49304d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c0ca3e0 con 0x7fe478003ff0 2023-11-08T19:24:33.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49184a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 0x7fe48c0b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49184a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 0x7fe48c0b0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:32964/0 (socket says 172.21.15.119:32964) 2023-11-08T19:24:33.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.629+0000 7fe49184a640 1 -- 172.21.15.119:0/336672732 learned_addr learned my addr 172.21.15.119:0/336672732 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:33.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49204b640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe478003ff0 0x7fe48c15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49284c640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe48c0b11b0 0x7fe48c0b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:33.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49184a640 1 -- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe478003ff0 msgr2=0x7fe48c15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49184a640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe478003ff0 0x7fe48c15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49184a640 1 -- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe48c0b11b0 msgr2=0x7fe48c0b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49184a640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe48c0b11b0 0x7fe48c0b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49184a640 1 -- 172.21.15.119:0/336672732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c0ca6c0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe49184a640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 0x7fe48c0b0c70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00b9e0 tx=0x7fe47c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:33.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.630+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c03ba60 con 0x7fe48c1f30c0 2023-11-08T19:24:33.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.631+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c03bbc0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.631+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c030400 con 0x7fe48c1f30c0 2023-11-08T19:24:33.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.631+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe48c0ca9a0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.631+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe48c0caf60 con 0x7fe48c1f30c0 2023-11-08T19:24:33.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.631+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe47c030860 con 0x7fe48c1f30c0 2023-11-08T19:24:33.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.631+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe47c04e8f0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.632+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48c0cb0f0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.635+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe47c036020 con 0x7fe48c1f30c0 2023-11-08T19:24:33.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.706+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe48c04ece0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.711+0000 7fe482ffd640 1 -- 172.21.15.119:0/336672732 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe47c00b5e0 con 0x7fe48c1f30c0 2023-11-08T19:24:33.713 INFO:teuthology.orchestra.run.smithi119.stderr:adjusted tunables profile to default 2023-11-08T19:24:33.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 msgr2=0x7fe48c0b0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:33.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 0x7fe48c0b0c70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00b9e0 tx=0x7fe47c004730 comp rx=0 tx=0).stop 2023-11-08T19:24:33.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 shutdown_connections 2023-11-08T19:24:33.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe48c0b11b0 0x7fe48c0b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe48c1f30c0 0x7fe48c0b0c70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 --2- 172.21.15.119:0/336672732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe478003ff0 0x7fe48c15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:33.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 >> 172.21.15.119:0/336672732 conn(0x7fe48c0d2ec0 msgr2=0x7fe48c06b300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:33.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 shutdown_connections 2023-11-08T19:24:33.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:33.713+0000 7fe49304d640 1 -- 172.21.15.119:0/336672732 wait complete. 2023-11-08T19:24:33.731 INFO:tasks.ceph:check_enable_crimson: False 2023-11-08T19:24:33.732 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-08T19:24:33.732 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:33.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-08T19:24:33.811 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:33.811 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-08T19:24:33.935 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:33.936 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-08T19:24:34.060 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:24:34.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-08T19:24:34.143 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:34.143 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-08T19:24:34.233 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:34.234 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-08T19:24:34.315 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:34.316 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-08T19:24:34.347 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:34.346+0000 7f2138c3b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T19:24:34.367 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:34.367+0000 7f49f8a1b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T19:24:34.406 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:34.405+0000 7f2138c3b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T19:24:34.433 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:34.432+0000 7f49f8a1b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T19:24:34.442 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:24:34.442 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-08T19:24:34.574 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f44842af-30d5-463b-bdab-3bf51ed607e1 0 2023-11-08T19:24:34.765 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.764+0000 7f756ec26640 1 Processor -- start 2023-11-08T19:24:34.766 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.765+0000 7f756ec26640 1 -- start start 2023-11-08T19:24:34.767 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756ec26640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f756815d4a0 0x7f75680d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.767 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756ec26640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f75680cfa00 0x7f75680cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756ec26640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7568161160 0x7f7568161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756ec26640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75680d03d0 con 0x7f75680cfa00 2023-11-08T19:24:34.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756ec26640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756815df50 con 0x7f756815d4a0 2023-11-08T19:24:34.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756ec26640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680d1e60 con 0x7f7568161160 2023-11-08T19:24:34.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756d1c3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7568161160 0x7f7568161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756c9c2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f756815d4a0 0x7f75680d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756d1c3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7568161160 0x7f7568161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:60146/0 (socket says 172.21.15.188:60146) 2023-11-08T19:24:34.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.766+0000 7f756d1c3640 1 -- 172.21.15.188:0/212581158 learned_addr learned my addr 172.21.15.188:0/212581158 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:34.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f7567fff640 1 --2- 172.21.15.188:0/212581158 >> v2:172.21.15.119:3300/0 conn(0x7f75680cfa00 0x7f75680cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f756c9c2640 1 -- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7568161160 msgr2=0x7f7568161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:34.771 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f756c9c2640 1 --2- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7568161160 0x7f7568161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f756c9c2640 1 -- 172.21.15.188:0/212581158 >> v2:172.21.15.119:3300/0 conn(0x7f75680cfa00 msgr2=0x7f75680cfe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:34.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f756c9c2640 1 --2- 172.21.15.188:0/212581158 >> v2:172.21.15.119:3300/0 conn(0x7f75680cfa00 0x7f75680cfe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f756c9c2640 1 -- 172.21.15.188:0/212581158 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75680d2830 con 0x7f756815d4a0 2023-11-08T19:24:34.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.767+0000 7f7567fff640 1 --2- 172.21.15.188:0/212581158 >> v2:172.21.15.119:3300/0 conn(0x7f75680cfa00 0x7f75680cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:34.772 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.768+0000 7f756c9c2640 1 --2- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680d22f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7568072830 tx=0x7f755c02b7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cc9e447222df8f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.768+0000 7f75677fe640 1 -- 172.21.15.188:0/212581158 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c02f050 con 0x7f756815d4a0 2023-11-08T19:24:34.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.768+0000 7f75677fe640 1 -- 172.21.15.188:0/212581158 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c031700 con 0x7f756815d4a0 2023-11-08T19:24:34.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.768+0000 7f75677fe640 1 -- 172.21.15.188:0/212581158 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c031a90 con 0x7f756815d4a0 2023-11-08T19:24:34.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 -- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 msgr2=0x7f75680d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.773 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 --2- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680d22f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7568072830 tx=0x7f755c02b7a0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 -- 172.21.15.188:0/212581158 shutdown_connections 2023-11-08T19:24:34.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 --2- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7568161160 0x7f7568161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 --2- 172.21.15.188:0/212581158 >> v2:172.21.15.119:3300/0 conn(0x7f75680cfa00 0x7f75680cfe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 --2- 172.21.15.188:0/212581158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680d22f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.774 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 -- 172.21.15.188:0/212581158 >> 172.21.15.188:0/212581158 conn(0x7f756806a9f0 msgr2=0x7f756806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.775 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 -- 172.21.15.188:0/212581158 shutdown_connections 2023-11-08T19:24:34.775 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.769+0000 7f756ec26640 1 -- 172.21.15.188:0/212581158 wait complete. 2023-11-08T19:24:34.777 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.771+0000 7f756ec26640 1 Processor -- start 2023-11-08T19:24:34.777 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.771+0000 7f756ec26640 1 -- start start 2023-11-08T19:24:34.777 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756ec26640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 0x7f75680c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756ec26640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756ec26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568161160 0x7f75680cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756ec26640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568152690 con 0x7f7568161160 2023-11-08T19:24:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756ec26640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7568154a50 con 0x7f756815d4a0 2023-11-08T19:24:34.778 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756ec26640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75680ca250 con 0x7f75680cfa00 2023-11-08T19:24:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756c9c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 0x7f75680c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756c9c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 0x7f75680c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:60160/0 (socket says 172.21.15.188:60160) 2023-11-08T19:24:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f756c9c2640 1 -- 172.21.15.188:0/3923731897 learned_addr learned my addr 172.21.15.188:0/3923731897 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.772+0000 7f7567fff640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.779 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756d1c3640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568161160 0x7f75680cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756c9c2640 1 -- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 msgr2=0x7f75680cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756c9c2640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756c9c2640 1 -- 172.21.15.188:0/3923731897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568161160 msgr2=0x7f75680cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756c9c2640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568161160 0x7f75680cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.780 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756c9c2640 1 -- 172.21.15.188:0/3923731897 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c009400 con 0x7f75680cfa00 2023-11-08T19:24:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756c9c2640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 0x7f75680c89f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f755c000c00 tx=0x7f755c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c0339b0 con 0x7f75680cfa00 2023-11-08T19:24:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f755c033b30 con 0x7f75680cfa00 2023-11-08T19:24:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755c03c400 con 0x7f75680cfa00 2023-11-08T19:24:34.781 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75681577a0 con 0x7f75680cfa00 2023-11-08T19:24:34.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.773+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7568154400 con 0x7f75680cfa00 2023-11-08T19:24:34.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.774+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f755c03a040 con 0x7f75680cfa00 2023-11-08T19:24:34.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.774+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f755c0485a0 con 0x7f75680cfa00 2023-11-08T19:24:34.782 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.774+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7568074430 con 0x7f75680cfa00 2023-11-08T19:24:34.783 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.779+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f755c043020 con 0x7f75680cfa00 2023-11-08T19:24:34.857 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.856+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f44842af-30d5-463b-bdab-3bf51ed607e1", "id": 0} v 0) v1 -- 0x7f75680d1ae0 con 0x7f75680cfa00 2023-11-08T19:24:34.862 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.862+0000 7f75657fa640 1 -- 172.21.15.188:0/3923731897 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f44842af-30d5-463b-bdab-3bf51ed607e1", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f755c038070 con 0x7f75680cfa00 2023-11-08T19:24:34.863 INFO:teuthology.orchestra.run.smithi188.stdout:0 2023-11-08T19:24:34.864 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 msgr2=0x7f75680c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:34.864 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 0x7f75680c89f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f755c000c00 tx=0x7f755c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.864 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 shutdown_connections 2023-11-08T19:24:34.864 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7568161160 0x7f75680cce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.865 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f756815d4a0 0x7f75680cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.865 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 --2- 172.21.15.188:0/3923731897 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75680cfa00 0x7f75680c89f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:34.865 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 >> 172.21.15.188:0/3923731897 conn(0x7f756806a9f0 msgr2=0x7f75680b8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:34.865 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 shutdown_connections 2023-11-08T19:24:34.865 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:34.863+0000 7f756ec26640 1 -- 172.21.15.188:0/3923731897 wait complete. 2023-11-08T19:24:34.880 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 9c1f24ae-f05b-4d9c-8867-fa908bd72d75 1 2023-11-08T19:24:34.995 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:34.994+0000 7f2138c3b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T19:24:35.056 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.054+0000 7f2138c3b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T19:24:35.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.070+0000 7f4008b6e640 1 Processor -- start 2023-11-08T19:24:35.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.070+0000 7f4008b6e640 1 -- start start 2023-11-08T19:24:35.073 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4008b6e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f40040d03c0 0x7f400406bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4008b6e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40040cb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4008b6e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40040cff30 con 0x7f400415d1e0 2023-11-08T19:24:35.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4008b6e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40040ce560 con 0x7f40040d03c0 2023-11-08T19:24:35.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4008b6e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400406c590 con 0x7f400406c100 2023-11-08T19:24:35.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f40037fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f400415d1e0 0x7f40040cde50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:60930/0 (socket says 172.21.15.188:60930) 2023-11-08T19:24:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f40037fe640 1 -- 172.21.15.188:0/710784912 learned_addr learned my addr 172.21.15.188:0/710784912 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-08T19:24:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f3ffbfff640 1 --2- 172.21.15.188:0/710784912 >> v2:172.21.15.188:3300/0 conn(0x7f40040d03c0 0x7f400406bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4003fff640 1 --2- 172.21.15.188:0/710784912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40040cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4002ffd640 1 -- 172.21.15.188:0/710784912 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695198803 0 0) 0x7f40040cff30 con 0x7f400415d1e0 2023-11-08T19:24:35.075 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.072+0000 7f4002ffd640 1 -- 172.21.15.188:0/710784912 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fe4003520 con 0x7f400415d1e0 2023-11-08T19:24:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.073+0000 7f4003fff640 1 -- 172.21.15.188:0/710784912 >> v2:172.21.15.188:3300/0 conn(0x7f40040d03c0 msgr2=0x7f400406bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.073+0000 7f4003fff640 1 --2- 172.21.15.188:0/710784912 >> v2:172.21.15.188:3300/0 conn(0x7f40040d03c0 0x7f400406bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.073+0000 7f4003fff640 1 -- 172.21.15.188:0/710784912 >> v1:172.21.15.188:6789/0 conn(0x7f400415d1e0 legacy=0x7f40040cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.073+0000 7f4003fff640 1 -- 172.21.15.188:0/710784912 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40040cf080 con 0x7f400406c100 2023-11-08T19:24:35.076 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.073+0000 7f3ffbfff640 1 --2- 172.21.15.188:0/710784912 >> v2:172.21.15.188:3300/0 conn(0x7f40040d03c0 0x7f400406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:35.078 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.074+0000 7f4003fff640 1 --2- 172.21.15.188:0/710784912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40040cb260 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009dc0 tx=0x7f3ffc02be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb3e7c4c39ae32ef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.078 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.074+0000 7f4002ffd640 1 -- 172.21.15.188:0/710784912 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc02cc30 con 0x7f400406c100 2023-11-08T19:24:35.078 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.074+0000 7f4002ffd640 1 -- 172.21.15.188:0/710784912 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ffc02cd90 con 0x7f400406c100 2023-11-08T19:24:35.078 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.074+0000 7f4002ffd640 1 -- 172.21.15.188:0/710784912 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc034660 con 0x7f400406c100 2023-11-08T19:24:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.074+0000 7f4008b6e640 1 -- 172.21.15.188:0/710784912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 msgr2=0x7f40040cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.074+0000 7f4008b6e640 1 --2- 172.21.15.188:0/710784912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40040cb260 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc009dc0 tx=0x7f3ffc02be80 comp rx=0 tx=0).stop 2023-11-08T19:24:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.075+0000 7f4008b6e640 1 -- 172.21.15.188:0/710784912 shutdown_connections 2023-11-08T19:24:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.075+0000 7f4008b6e640 1 --2- 172.21.15.188:0/710784912 >> v2:172.21.15.188:3300/0 conn(0x7f40040d03c0 0x7f400406bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.079 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.075+0000 7f4008b6e640 1 --2- 172.21.15.188:0/710784912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40040cb260 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.075+0000 7f4008b6e640 1 -- 172.21.15.188:0/710784912 >> 172.21.15.188:0/710784912 conn(0x7f40040d0c10 msgr2=0x7f40040d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.075+0000 7f4008b6e640 1 -- 172.21.15.188:0/710784912 shutdown_connections 2023-11-08T19:24:35.080 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.075+0000 7f4008b6e640 1 -- 172.21.15.188:0/710784912 wait complete. 2023-11-08T19:24:35.080 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.076+0000 7f49f8a1b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T19:24:35.082 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.076+0000 7f4008b6e640 1 Processor -- start 2023-11-08T19:24:35.083 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.077+0000 7f4008b6e640 1 -- start start 2023-11-08T19:24:35.083 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.077+0000 7f4008b6e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40041517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.083 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f4008b6e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 0x7f40040c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.083 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f4008b6e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400415d1e0 0x7f40040c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.083 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f4008b6e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40040c5630 con 0x7f400415d1e0 2023-11-08T19:24:35.084 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f4008b6e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40040c75e0 con 0x7f40040d03c0 2023-11-08T19:24:35.084 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f4008b6e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4004151d10 con 0x7f400406c100 2023-11-08T19:24:35.084 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40041517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.084 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 0x7f40040c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.084 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f3ffbfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 0x7f40040c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44174/0 (socket says 172.21.15.188:44174) 2023-11-08T19:24:35.085 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f3ffbfff640 1 -- 172.21.15.188:0/3479480041 learned_addr learned my addr 172.21.15.188:0/3479480041 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:35.085 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 -- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 msgr2=0x7f40040c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.085 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 0x7f40040c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.085 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 -- 172.21.15.188:0/3479480041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400415d1e0 msgr2=0x7f40040c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.085 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400415d1e0 0x7f40040c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.086 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 -- 172.21.15.188:0/3479480041 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec009400 con 0x7f400406c100 2023-11-08T19:24:35.086 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f3ffbfff640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 0x7f40040c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:35.086 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.078+0000 7f40037fe640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40041517d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c390 tx=0x7f3ff400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.086 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40074e0 con 0x7f400406c100 2023-11-08T19:24:35.086 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff4007640 con 0x7f400406c100 2023-11-08T19:24:35.086 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff40059c0 con 0x7f400406c100 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ffc009a70 con 0x7f400406c100 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4004159040 con 0x7f400406c100 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3ff4012400 con 0x7f400406c100 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.079+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f3ff4012a60 con 0x7f400406c100 2023-11-08T19:24:35.087 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.080+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400404ece0 con 0x7f400406c100 2023-11-08T19:24:35.088 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.084+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3ff4021060 con 0x7f400406c100 2023-11-08T19:24:35.106 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.105+0000 7f2138c3b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T19:24:35.143 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.141+0000 7f49f8a1b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T19:24:35.163 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.162+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c1f24ae-f05b-4d9c-8867-fa908bd72d75", "id": 1} v 0) v1 -- 0x7f400406cd90 con 0x7f400406c100 2023-11-08T19:24:35.168 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.167+0000 7f2138c3b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T19:24:35.170 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.169+0000 7f4000ff9640 1 -- 172.21.15.188:0/3479480041 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c1f24ae-f05b-4d9c-8867-fa908bd72d75", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3ff4017070 con 0x7f400406c100 2023-11-08T19:24:35.170 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-08T19:24:35.171 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.170+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 msgr2=0x7f40041517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.171 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.170+0000 7f4008b6e640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40041517d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400c390 tx=0x7f3ff400c850 comp rx=0 tx=0).stop 2023-11-08T19:24:35.171 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.170+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 shutdown_connections 2023-11-08T19:24:35.172 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.170+0000 7f4008b6e640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f400415d1e0 0x7f40040c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.172 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.170+0000 7f4008b6e640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f40040d03c0 0x7f40040c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.173 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.170+0000 7f4008b6e640 1 --2- 172.21.15.188:0/3479480041 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f400406c100 0x7f40041517d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.173 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.171+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 >> 172.21.15.188:0/3479480041 conn(0x7f40040d0c10 msgr2=0x7f400415c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.173 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.171+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 shutdown_connections 2023-11-08T19:24:35.173 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.171+0000 7f4008b6e640 1 -- 172.21.15.188:0/3479480041 wait complete. 2023-11-08T19:24:35.188 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new f623705e-975f-4df5-aa4d-61abed82929f 2 2023-11-08T19:24:35.197 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.196+0000 7f49f8a1b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T19:24:35.218 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.216+0000 7f2138c3b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T19:24:35.263 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.262+0000 7f49f8a1b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T19:24:35.315 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.314+0000 7f49f8a1b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T19:24:35.330 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.329+0000 7f2138c3b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-08T19:24:35.381 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.380+0000 7fa339b3c640 1 Processor -- start 2023-11-08T19:24:35.382 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.381+0000 7fa339b3c640 1 -- start start 2023-11-08T19:24:35.383 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa339b3c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa33415b1a0 0x7fa3340b2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.383 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa339b3c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa3340b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.383 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa339b3c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3340ccb80 con 0x7fa33415adc0 2023-11-08T19:24:35.384 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa339b3c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3340ccdb0 con 0x7fa33415b1a0 2023-11-08T19:24:35.384 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa339b3c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3340d0240 con 0x7fa3340cb220 2023-11-08T19:24:35.384 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa338b3a640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa33415adc0 0x7fa33415f550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:60936/0 (socket says 172.21.15.188:60936) 2023-11-08T19:24:35.384 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa338b3a640 1 -- 172.21.15.188:0/1770165785 learned_addr learned my addr 172.21.15.188:0/1770165785 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-08T19:24:35.384 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa33933b640 1 --2- 172.21.15.188:0/1770165785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa3340b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.385 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.382+0000 7fa333fff640 1 --2- 172.21.15.188:0/1770165785 >> v2:172.21.15.119:3300/0 conn(0x7fa33415b1a0 0x7fa3340b2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.386 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.383+0000 7fa33933b640 1 -- 172.21.15.188:0/1770165785 >> v2:172.21.15.119:3300/0 conn(0x7fa33415b1a0 msgr2=0x7fa3340b2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:35.387 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.383+0000 7fa33933b640 1 --2- 172.21.15.188:0/1770165785 >> v2:172.21.15.119:3300/0 conn(0x7fa33415b1a0 0x7fa3340b2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.387 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.383+0000 7fa33933b640 1 -- 172.21.15.188:0/1770165785 >> v1:172.21.15.188:6789/0 conn(0x7fa33415adc0 legacy=0x7fa33415f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.387 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.383+0000 7fa33933b640 1 -- 172.21.15.188:0/1770165785 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3340d1a10 con 0x7fa3340cb220 2023-11-08T19:24:35.387 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.383+0000 7fa3337fe640 1 -- 172.21.15.188:0/1770165785 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409918774 0 0) 0x7fa3340ccb80 con 0x7fa33415adc0 2023-11-08T19:24:35.388 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.384+0000 7fa33933b640 1 --2- 172.21.15.188:0/1770165785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa3340b2bd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009d90 tx=0x7fa32c02bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44fe3d2e0acd86aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.388 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.384+0000 7fa3337fe640 1 -- 172.21.15.188:0/1770165785 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c02cc50 con 0x7fa3340cb220 2023-11-08T19:24:35.388 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.384+0000 7fa3337fe640 1 -- 172.21.15.188:0/1770165785 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c02cdb0 con 0x7fa3340cb220 2023-11-08T19:24:35.388 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.384+0000 7fa3337fe640 1 -- 172.21.15.188:0/1770165785 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c034660 con 0x7fa3340cb220 2023-11-08T19:24:35.388 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.384+0000 7fa339b3c640 1 -- 172.21.15.188:0/1770165785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 msgr2=0x7fa3340b2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.384+0000 7fa339b3c640 1 --2- 172.21.15.188:0/1770165785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa3340b2bd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa32c009d90 tx=0x7fa32c02bea0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.385+0000 7fa339b3c640 1 -- 172.21.15.188:0/1770165785 shutdown_connections 2023-11-08T19:24:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.385+0000 7fa339b3c640 1 --2- 172.21.15.188:0/1770165785 >> v2:172.21.15.119:3300/0 conn(0x7fa33415b1a0 0x7fa3340b2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.385+0000 7fa339b3c640 1 --2- 172.21.15.188:0/1770165785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa3340b2bd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.389 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.385+0000 7fa339b3c640 1 -- 172.21.15.188:0/1770165785 >> 172.21.15.188:0/1770165785 conn(0x7fa33406a120 msgr2=0x7fa33406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.385+0000 7fa339b3c640 1 -- 172.21.15.188:0/1770165785 shutdown_connections 2023-11-08T19:24:35.390 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.385+0000 7fa339b3c640 1 -- 172.21.15.188:0/1770165785 wait complete. 2023-11-08T19:24:35.392 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.386+0000 7fa339b3c640 1 Processor -- start 2023-11-08T19:24:35.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.386+0000 7fa339b3c640 1 -- start start 2023-11-08T19:24:35.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.387+0000 7fa339b3c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa33415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.387+0000 7fa339b3c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa33415adc0 0x7fa334150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.387+0000 7fa339b3c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa33415b1a0 0x7fa3341537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.393 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.387+0000 7fa339b3c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33415c5d0 con 0x7fa33415b1a0 2023-11-08T19:24:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.387+0000 7fa339b3c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33415d560 con 0x7fa33415adc0 2023-11-08T19:24:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.387+0000 7fa339b3c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa334154220 con 0x7fa3340cb220 2023-11-08T19:24:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa33415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa333fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa33415adc0 0x7fa334150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.394 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa333fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa33415adc0 0x7fa334150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44188/0 (socket says 172.21.15.188:44188) 2023-11-08T19:24:35.395 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa33415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:60202/0 (socket says 172.21.15.188:60202) 2023-11-08T19:24:35.395 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa333fff640 1 -- 172.21.15.188:0/3870491971 learned_addr learned my addr 172.21.15.188:0/3870491971 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:35.395 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa33933b640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa33415b1a0 0x7fa3341537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.395 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 -- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa33415adc0 msgr2=0x7fa334150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.395 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa33415adc0 0x7fa334150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.395 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 -- 172.21.15.188:0/3870491971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa33415b1a0 msgr2=0x7fa3341537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.396 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa33415b1a0 0x7fa3341537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.396 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 -- 172.21.15.188:0/3870491971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c009a70 con 0x7fa3340cb220 2023-11-08T19:24:35.396 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa338b3a640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa33415d020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c450 tx=0x7fa32800c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.396 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.388+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3280074e0 con 0x7fa3340cb220 2023-11-08T19:24:35.396 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.389+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa328007640 con 0x7fa3340cb220 2023-11-08T19:24:35.397 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.389+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa328004a20 con 0x7fa3340cb220 2023-11-08T19:24:35.397 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.389+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa334154500 con 0x7fa3340cb220 2023-11-08T19:24:35.397 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.389+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33415cb00 con 0x7fa3340cb220 2023-11-08T19:24:35.397 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.389+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa328012400 con 0x7fa3340cb220 2023-11-08T19:24:35.398 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.389+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fa328012ad0 con 0x7fa3340cb220 2023-11-08T19:24:35.398 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.390+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33404ece0 con 0x7fa3340cb220 2023-11-08T19:24:35.398 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.394+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa328018070 con 0x7fa3340cb220 2023-11-08T19:24:35.410 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.409+0000 7f2138c3b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:35.436 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.435+0000 7f49f8a1b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-08T19:24:35.464 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.462+0000 7f2138c3b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T19:24:35.472 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.471+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f623705e-975f-4df5-aa4d-61abed82929f", "id": 2} v 0) v1 -- 0x7fa3340c74f0 con 0x7fa3340cb220 2023-11-08T19:24:35.480 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.479+0000 7fa3317fa640 1 -- 172.21.15.188:0/3870491971 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f623705e-975f-4df5-aa4d-61abed82929f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa3280126b0 con 0x7fa3340cb220 2023-11-08T19:24:35.480 INFO:teuthology.orchestra.run.smithi188.stdout:2 2023-11-08T19:24:35.480 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 msgr2=0x7fa33415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.481 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa33415d020 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa32800c450 tx=0x7fa32800c910 comp rx=0 tx=0).stop 2023-11-08T19:24:35.482 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 shutdown_connections 2023-11-08T19:24:35.482 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa33415b1a0 0x7fa3341537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.482 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa33415adc0 0x7fa334150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.482 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 --2- 172.21.15.188:0/3870491971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3340cb220 0x7fa33415d020 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.483 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 >> 172.21.15.188:0/3870491971 conn(0x7fa33406a120 msgr2=0x7fa3341560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.483 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 shutdown_connections 2023-11-08T19:24:35.483 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.480+0000 7fa339b3c640 1 -- 172.21.15.188:0/3870491971 wait complete. 2023-11-08T19:24:35.496 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new c2435256-2139-4515-ab1c-cdef186bd730 3 2023-11-08T19:24:35.521 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.519+0000 7f49f8a1b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:35.525 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.523+0000 7f2138c3b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T19:24:35.578 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.577+0000 7f49f8a1b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T19:24:35.579 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:35.578+0000 7f2138c3b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T19:24:35.642 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.641+0000 7f49f8a1b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T19:24:35.699 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:35.698+0000 7f49f8a1b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T19:24:35.721 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.720+0000 7fe91f240640 1 Processor -- start 2023-11-08T19:24:35.722 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.721+0000 7fe91f240640 1 -- start start 2023-11-08T19:24:35.723 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91f240640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe918160530 0x7fe9180cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.724 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91f240640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180d0830 con 0x7fe918160910 2023-11-08T19:24:35.724 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91f240640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9180b27c0 con 0x7fe9180ce5b0 2023-11-08T19:24:35.724 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91f240640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9180b2aa0 con 0x7fe918160530 2023-11-08T19:24:35.724 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91e23e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe918160530 0x7fe9180cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.725 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91e23e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe918160530 0x7fe9180cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44198/0 (socket says 172.21.15.188:44198) 2023-11-08T19:24:35.725 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91e23e640 1 -- 172.21.15.188:0/3182213136 learned_addr learned my addr 172.21.15.188:0/3182213136 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:35.725 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.722+0000 7fe91ea3f640 1 --1- 172.21.15.188:0/3182213136 >> v1:172.21.15.188:6789/0 conn(0x7fe9180ce5b0 0x7fe9180b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:60938/0 (socket says 172.21.15.188:60938) 2023-11-08T19:24:35.725 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.723+0000 7fe91e23e640 1 -- 172.21.15.188:0/3182213136 >> v1:172.21.15.188:6789/0 conn(0x7fe9180ce5b0 legacy=0x7fe9180b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.725 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.723+0000 7fe91e23e640 1 -- 172.21.15.188:0/3182213136 >> v1:172.21.15.119:6789/0 conn(0x7fe918160910 legacy=0x7fe9180cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:35.726 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.723+0000 7fe91d23c640 1 -- 172.21.15.188:0/3182213136 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401820146 0 0) 0x7fe9180b27c0 con 0x7fe9180ce5b0 2023-11-08T19:24:35.726 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.723+0000 7fe91e23e640 1 -- 172.21.15.188:0/3182213136 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180d1640 con 0x7fe918160530 2023-11-08T19:24:35.727 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.724+0000 7fe91e23e640 1 --2- 172.21.15.188:0/3182213136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160530 0x7fe9180cd960 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe914009850 tx=0x7fe91402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=398778adf3eaaa9b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.727 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.724+0000 7fe91d23c640 1 -- 172.21.15.188:0/3182213136 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914004230 con 0x7fe918160530 2023-11-08T19:24:35.727 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.724+0000 7fe91d23c640 1 -- 172.21.15.188:0/3182213136 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe914004390 con 0x7fe918160530 2023-11-08T19:24:35.728 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.724+0000 7fe91d23c640 1 -- 172.21.15.188:0/3182213136 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe914030840 con 0x7fe918160530 2023-11-08T19:24:35.728 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 -- 172.21.15.188:0/3182213136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160530 msgr2=0x7fe9180cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.728 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 --2- 172.21.15.188:0/3182213136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160530 0x7fe9180cd960 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe914009850 tx=0x7fe91402b980 comp rx=0 tx=0).stop 2023-11-08T19:24:35.728 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 -- 172.21.15.188:0/3182213136 shutdown_connections 2023-11-08T19:24:35.728 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 --2- 172.21.15.188:0/3182213136 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160530 0x7fe9180cd960 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.729 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 -- 172.21.15.188:0/3182213136 >> 172.21.15.188:0/3182213136 conn(0x7fe9180d2ec0 msgr2=0x7fe9180d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.729 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 -- 172.21.15.188:0/3182213136 shutdown_connections 2023-11-08T19:24:35.729 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.725+0000 7fe91f240640 1 -- 172.21.15.188:0/3182213136 wait complete. 2023-11-08T19:24:35.731 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.726+0000 7fe91f240640 1 Processor -- start 2023-11-08T19:24:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.727+0000 7fe91f240640 1 -- start start 2023-11-08T19:24:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.727+0000 7fe91f240640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 0x7fe9180c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.727+0000 7fe91f240640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe918160530 0x7fe9180b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.727+0000 7fe91f240640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160910 0x7fe9180b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:35.732 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91f240640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180b5370 con 0x7fe918160530 2023-11-08T19:24:35.733 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91f240640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9180b4770 con 0x7fe918160910 2023-11-08T19:24:35.733 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91f240640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe91815ab70 con 0x7fe9180ce5b0 2023-11-08T19:24:35.733 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91e23e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 0x7fe9180c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.733 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91ea3f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160910 0x7fe9180b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.734 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91e23e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 0x7fe9180c8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:60216/0 (socket says 172.21.15.188:60216) 2023-11-08T19:24:35.734 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91e23e640 1 -- 172.21.15.188:0/1784262427 learned_addr learned my addr 172.21.15.188:0/1784262427 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:35.734 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91da3d640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe918160530 0x7fe9180b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:35.734 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91ea3f640 1 -- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 msgr2=0x7fe9180c8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.734 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91ea3f640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 0x7fe9180c8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.735 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91ea3f640 1 -- 172.21.15.188:0/1784262427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe918160530 msgr2=0x7fe9180b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.735 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91ea3f640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe918160530 0x7fe9180b3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.735 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91ea3f640 1 -- 172.21.15.188:0/1784262427 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe914009460 con 0x7fe918160910 2023-11-08T19:24:35.735 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.728+0000 7fe91e23e640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 0x7fe9180c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:35.735 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe91ea3f640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160910 0x7fe9180b40d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe910001560 tx=0x7fe91000cbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:35.736 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91000ce30 con 0x7fe918160910 2023-11-08T19:24:35.736 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe91000a610 con 0x7fe918160910 2023-11-08T19:24:35.736 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91000a9e0 con 0x7fe918160910 2023-11-08T19:24:35.736 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe91815ae50 con 0x7fe918160910 2023-11-08T19:24:35.736 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9180c9510 con 0x7fe918160910 2023-11-08T19:24:35.737 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe9100070d0 con 0x7fe918160910 2023-11-08T19:24:35.737 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.729+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe910007810 con 0x7fe918160910 2023-11-08T19:24:35.737 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.730+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe91815a9d0 con 0x7fe918160910 2023-11-08T19:24:35.738 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.733+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe91815a9d0 con 0x7fe918160910 2023-11-08T19:24:35.812 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.811+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c2435256-2139-4515-ab1c-cdef186bd730", "id": 3} v 0) v1 -- 0x7fe91815b370 con 0x7fe918160910 2023-11-08T19:24:35.820 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.819+0000 7fe90effd640 1 -- 172.21.15.188:0/1784262427 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c2435256-2139-4515-ab1c-cdef186bd730", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe91815b370 con 0x7fe918160910 2023-11-08T19:24:35.820 INFO:teuthology.orchestra.run.smithi188.stdout:3 2023-11-08T19:24:35.821 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160910 msgr2=0x7fe9180b40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:35.821 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160910 0x7fe9180b40d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe910001560 tx=0x7fe91000cbe0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.821 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 shutdown_connections 2023-11-08T19:24:35.822 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe918160910 0x7fe9180b40d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.822 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe918160530 0x7fe9180b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.822 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 --2- 172.21.15.188:0/1784262427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9180ce5b0 0x7fe9180c8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:35.822 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 >> 172.21.15.188:0/1784262427 conn(0x7fe9180d2ec0 msgr2=0x7fe91815e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:35.822 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 shutdown_connections 2023-11-08T19:24:35.823 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:35.820+0000 7fe91f240640 1 -- 172.21.15.188:0/1784262427 wait complete. 2023-11-08T19:24:35.836 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 4591576b-2c35-457c-8790-d2e9af31a168 4 2023-11-08T19:24:36.023 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.022+0000 7f5a61fa9640 1 Processor -- start 2023-11-08T19:24:36.024 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.022+0000 7f5a61fa9640 1 -- start start 2023-11-08T19:24:36.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a61fa9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5a5c0d0f90 0x7f5a5c0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.025 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a61fa9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a5c0d1a80 0x7f5a5c0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a61fa9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a5c160c60 con 0x7f5a5c0cec10 2023-11-08T19:24:36.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a61fa9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c0d0260 con 0x7f5a5c0d0f90 2023-11-08T19:24:36.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a61fa9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cf080 con 0x7f5a5c0d1a80 2023-11-08T19:24:36.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a5bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a5c0d1a80 0x7f5a5c0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.026 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a5bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5a5c0d1a80 0x7f5a5c0cdbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44212/0 (socket says 172.21.15.188:44212) 2023-11-08T19:24:36.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a5bfff640 1 -- 172.21.15.188:0/3301058659 learned_addr learned my addr 172.21.15.188:0/3301058659 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.027 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.024+0000 7f5a5b7fe640 1 --2- 172.21.15.188:0/3301058659 >> v2:172.21.15.119:3300/0 conn(0x7f5a5c0d0f90 0x7f5a5c0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.025+0000 7f5a5bfff640 1 -- 172.21.15.188:0/3301058659 >> v2:172.21.15.119:3300/0 conn(0x7f5a5c0d0f90 msgr2=0x7f5a5c0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:36.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.025+0000 7f5a5affd640 1 -- 172.21.15.188:0/3301058659 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 829736731 0 0) 0x7f5a5c160c60 con 0x7f5a5c0cec10 2023-11-08T19:24:36.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.025+0000 7f5a5bfff640 1 --2- 172.21.15.188:0/3301058659 >> v2:172.21.15.119:3300/0 conn(0x7f5a5c0d0f90 0x7f5a5c0d0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.029 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.025+0000 7f5a5bfff640 1 -- 172.21.15.188:0/3301058659 >> v1:172.21.15.188:6789/0 conn(0x7f5a5c0cec10 legacy=0x7f5a5c0d1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.025+0000 7f5a5bfff640 1 -- 172.21.15.188:0/3301058659 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a5c0cf5e0 con 0x7f5a5c0d1a80 2023-11-08T19:24:36.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.026+0000 7f5a5bfff640 1 --2- 172.21.15.188:0/3301058659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0cdbf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c06a850 tx=0x7f5a5402be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d5c201d2112cdfc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.027+0000 7f5a5affd640 1 -- 172.21.15.188:0/3301058659 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5402c9f0 con 0x7f5a5c0d1a80 2023-11-08T19:24:36.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.027+0000 7f5a5affd640 1 -- 172.21.15.188:0/3301058659 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5402cb50 con 0x7f5a5c0d1a80 2023-11-08T19:24:36.030 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.027+0000 7f5a5affd640 1 -- 172.21.15.188:0/3301058659 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a540343f0 con 0x7f5a5c0d1a80 2023-11-08T19:24:36.031 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.027+0000 7f5a61fa9640 1 -- 172.21.15.188:0/3301058659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d1a80 msgr2=0x7f5a5c0cdbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.031 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.027+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/3301058659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0cdbf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c06a850 tx=0x7f5a5402be50 comp rx=0 tx=0).stop 2023-11-08T19:24:36.031 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.028+0000 7f5a61fa9640 1 -- 172.21.15.188:0/3301058659 shutdown_connections 2023-11-08T19:24:36.031 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.028+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/3301058659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0cdbf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.031 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.028+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/3301058659 >> v2:172.21.15.119:3300/0 conn(0x7f5a5c0d0f90 0x7f5a5c0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.028+0000 7f5a61fa9640 1 -- 172.21.15.188:0/3301058659 >> 172.21.15.188:0/3301058659 conn(0x7f5a5c0d3090 msgr2=0x7f5a5c0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.028+0000 7f5a61fa9640 1 -- 172.21.15.188:0/3301058659 shutdown_connections 2023-11-08T19:24:36.032 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.028+0000 7f5a61fa9640 1 -- 172.21.15.188:0/3301058659 wait complete. 2023-11-08T19:24:36.033 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.028+0000 7f2138c3b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-08T19:24:36.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.030+0000 7f5a61fa9640 1 Processor -- start 2023-11-08T19:24:36.035 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.030+0000 7f5a61fa9640 1 -- start start 2023-11-08T19:24:36.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a61fa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 0x7f5a5c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a61fa9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 0x7f5a5c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a61fa9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.036 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a61fa9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0c6ae0 con 0x7f5a5c0d1a80 2023-11-08T19:24:36.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a61fa9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0cb0b0 con 0x7f5a5c0d0f90 2023-11-08T19:24:36.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a61fa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c0c7b60 con 0x7f5a5c0cec10 2023-11-08T19:24:36.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 0x7f5a5c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 0x7f5a5c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.037 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a5b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 0x7f5a5c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:60220/0 (socket says 172.21.15.188:60220) 2023-11-08T19:24:36.038 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a53fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 0x7f5a5c0cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44220/0 (socket says 172.21.15.188:44220) 2023-11-08T19:24:36.038 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.031+0000 7f5a5b7fe640 1 -- 172.21.15.188:0/610571457 learned_addr learned my addr 172.21.15.188:0/610571457 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.038 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5bfff640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.038 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5b7fe640 1 -- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 msgr2=0x7f5a5c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.039 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5b7fe640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 0x7f5a5c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.039 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5b7fe640 1 -- 172.21.15.188:0/610571457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0d1a80 msgr2=0x7f5a5c0b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.039 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5b7fe640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.039 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5b7fe640 1 -- 172.21.15.188:0/610571457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a54009b30 con 0x7f5a5c0cec10 2023-11-08T19:24:36.039 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a53fff640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 0x7f5a5c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:36.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a5b7fe640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 0x7f5a5c0c8740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00e870 tx=0x7f5a4c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c0094f0 con 0x7f5a5c0cec10 2023-11-08T19:24:36.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a4c00c940 con 0x7f5a5c0cec10 2023-11-08T19:24:36.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c0c6da0 con 0x7f5a5c0cec10 2023-11-08T19:24:36.040 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.032+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a4c00ccf0 con 0x7f5a5c0cec10 2023-11-08T19:24:36.041 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.033+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c0c7360 con 0x7f5a5c0cec10 2023-11-08T19:24:36.041 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.033+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5a4c005c90 con 0x7f5a5c0cec10 2023-11-08T19:24:36.041 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.033+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f5a4c0067f0 con 0x7f5a5c0cec10 2023-11-08T19:24:36.041 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.034+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c074430 con 0x7f5a5c0cec10 2023-11-08T19:24:36.042 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.038+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5a4c021060 con 0x7f5a5c0cec10 2023-11-08T19:24:36.116 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.115+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4591576b-2c35-457c-8790-d2e9af31a168", "id": 4} v 0) v1 -- 0x7f5a5c0c7da0 con 0x7f5a5c0cec10 2023-11-08T19:24:36.123 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.122+0000 7f5a58ff9640 1 -- 172.21.15.188:0/610571457 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4591576b-2c35-457c-8790-d2e9af31a168", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5a4c029020 con 0x7f5a5c0cec10 2023-11-08T19:24:36.124 INFO:teuthology.orchestra.run.smithi188.stdout:4 2023-11-08T19:24:36.125 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 msgr2=0x7f5a5c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.125 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 0x7f5a5c0c8740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00e870 tx=0x7f5a4c00ed30 comp rx=0 tx=0).stop 2023-11-08T19:24:36.125 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 shutdown_connections 2023-11-08T19:24:36.125 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5a5c0d1a80 0x7f5a5c0b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5a5c0d0f90 0x7f5a5c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 --2- 172.21.15.188:0/610571457 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5a5c0cec10 0x7f5a5c0c8740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 >> 172.21.15.188:0/610571457 conn(0x7f5a5c0d3090 msgr2=0x7f5a5c0ce340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 shutdown_connections 2023-11-08T19:24:36.126 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.124+0000 7f5a61fa9640 1 -- 172.21.15.188:0/610571457 wait complete. 2023-11-08T19:24:36.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new e224dfe7-9d85-4a0e-b408-43df239741d3 5 2023-11-08T19:24:36.175 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.174+0000 7f49f8a1b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-08T19:24:36.191 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.189+0000 7f2138c3b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T19:24:36.330 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.329+0000 7f9ca9815640 1 Processor -- start 2023-11-08T19:24:36.331 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.329+0000 7f9ca9815640 1 -- start start 2023-11-08T19:24:36.333 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca9815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 0x7f9ca4071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.333 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca9815640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9ca40ca2e0 0x7f9ca406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.333 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca9815640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ca406a660 0x7f9ca40cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca9815640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca406aad0 con 0x7f9ca406a660 2023-11-08T19:24:36.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca9815640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca406dd90 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca9815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca406cff0 con 0x7f9ca415b040 2023-11-08T19:24:36.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca2ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 0x7f9ca4071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.334 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca2ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 0x7f9ca4071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:60234/0 (socket says 172.21.15.188:60234) 2023-11-08T19:24:36.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca2ffd640 1 -- 172.21.15.188:0/3664417495 learned_addr learned my addr 172.21.15.188:0/3664417495 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.331+0000 7f9ca27fc640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.188:3300/0 conn(0x7f9ca40ca2e0 0x7f9ca406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca37fe640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.119:3300/0 conn(0x7f9ca406a660 0x7f9ca40cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca2ffd640 1 -- 172.21.15.188:0/3664417495 >> v2:172.21.15.188:3300/0 conn(0x7f9ca40ca2e0 msgr2=0x7f9ca406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:36.335 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca2ffd640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.188:3300/0 conn(0x7f9ca40ca2e0 0x7f9ca406a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca2ffd640 1 -- 172.21.15.188:0/3664417495 >> v2:172.21.15.119:3300/0 conn(0x7f9ca406a660 msgr2=0x7f9ca40cfc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:36.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca2ffd640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.119:3300/0 conn(0x7f9ca406a660 0x7f9ca40cfc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca2ffd640 1 -- 172.21.15.188:0/3664417495 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4071cb0 con 0x7f9ca415b040 2023-11-08T19:24:36.336 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.332+0000 7f9ca37fe640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.119:3300/0 conn(0x7f9ca406a660 0x7f9ca40cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:36.337 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.331+0000 7f2138c3b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T19:24:36.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.333+0000 7f9ca2ffd640 1 --2- 172.21.15.188:0/3664417495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 0x7f9ca4071440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40cfbf0 tx=0x7f9c9402b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b207e615811c058 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.333+0000 7f9ca1ffb640 1 -- 172.21.15.188:0/3664417495 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94030850 con 0x7f9ca415b040 2023-11-08T19:24:36.340 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.333+0000 7f9ca1ffb640 1 -- 172.21.15.188:0/3664417495 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c940309b0 con 0x7f9ca415b040 2023-11-08T19:24:36.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.333+0000 7f9ca1ffb640 1 -- 172.21.15.188:0/3664417495 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94030d40 con 0x7f9ca415b040 2023-11-08T19:24:36.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 -- 172.21.15.188:0/3664417495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 msgr2=0x7f9ca4071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 --2- 172.21.15.188:0/3664417495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 0x7f9ca4071440 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9ca40cfbf0 tx=0x7f9c9402b990 comp rx=0 tx=0).stop 2023-11-08T19:24:36.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 -- 172.21.15.188:0/3664417495 shutdown_connections 2023-11-08T19:24:36.341 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.119:3300/0 conn(0x7f9ca406a660 0x7f9ca40cfc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 --2- 172.21.15.188:0/3664417495 >> v2:172.21.15.188:3300/0 conn(0x7f9ca40ca2e0 0x7f9ca406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 --2- 172.21.15.188:0/3664417495 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca415b040 0x7f9ca4071440 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 -- 172.21.15.188:0/3664417495 >> 172.21.15.188:0/3664417495 conn(0x7f9ca406b230 msgr2=0x7f9ca406b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 -- 172.21.15.188:0/3664417495 shutdown_connections 2023-11-08T19:24:36.342 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.334+0000 7f9ca9815640 1 -- 172.21.15.188:0/3664417495 wait complete. 2023-11-08T19:24:36.343 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.336+0000 7f9ca9815640 1 Processor -- start 2023-11-08T19:24:36.343 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.336+0000 7f9ca9815640 1 -- start start 2023-11-08T19:24:36.343 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca9815640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca406a660 0x7f9ca413fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.343 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca9815640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 0x7f9ca4140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.343 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca9815640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca415b040 0x7f9ca4154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.344 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca9815640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca415c510 con 0x7f9ca415b040 2023-11-08T19:24:36.344 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca9815640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca4140220 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.344 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca9815640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca415c7b0 con 0x7f9ca406a660 2023-11-08T19:24:36.344 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca27fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 0x7f9ca4140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.344 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca2ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca406a660 0x7f9ca413fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.345 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca27fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 0x7f9ca4140550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44234/0 (socket says 172.21.15.188:44234) 2023-11-08T19:24:36.345 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca27fc640 1 -- 172.21.15.188:0/1979440881 learned_addr learned my addr 172.21.15.188:0/1979440881 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.345 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca37fe640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca415b040 0x7f9ca4154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.345 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.337+0000 7f9ca27fc640 1 -- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca406a660 msgr2=0x7f9ca413fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.345 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca27fc640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca406a660 0x7f9ca413fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.346 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca27fc640 1 -- 172.21.15.188:0/1979440881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca415b040 msgr2=0x7f9ca4154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.346 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca27fc640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca415b040 0x7f9ca4154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.346 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca27fc640 1 -- 172.21.15.188:0/1979440881 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c940094d0 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.346 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca27fc640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 0x7f9ca4140550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800e870 tx=0x7f9c9800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.346 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c980094f0 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.347 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9800c940 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.347 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9800ccf0 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.347 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca415ca90 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.347 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca4140a90 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.347 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.338+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9c98013770 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.348 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.339+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f9c9801c850 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.348 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.339+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca4074430 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.349 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.342+0000 7f49f8a1b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T19:24:36.349 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.342+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9c98021060 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.385 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.383+0000 7f2138c3b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T19:24:36.420 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.419+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e224dfe7-9d85-4a0e-b408-43df239741d3", "id": 5} v 0) v1 -- 0x7f9ca40cab90 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.429 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.428+0000 7f9c8bfff640 1 -- 172.21.15.188:0/1979440881 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e224dfe7-9d85-4a0e-b408-43df239741d3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9c98017070 con 0x7f9ca40ca2e0 2023-11-08T19:24:36.429 INFO:teuthology.orchestra.run.smithi188.stdout:5 2023-11-08T19:24:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 msgr2=0x7f9ca4140550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.430 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 0x7f9ca4140550 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9c9800e870 tx=0x7f9c9800ed30 comp rx=0 tx=0).stop 2023-11-08T19:24:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 shutdown_connections 2023-11-08T19:24:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca415b040 0x7f9ca4154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca40ca2e0 0x7f9ca4140550 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 --2- 172.21.15.188:0/1979440881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca406a660 0x7f9ca413fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.431 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 >> 172.21.15.188:0/1979440881 conn(0x7f9ca406b230 msgr2=0x7f9ca415a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 shutdown_connections 2023-11-08T19:24:36.432 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.429+0000 7f9ca9815640 1 -- 172.21.15.188:0/1979440881 wait complete. 2023-11-08T19:24:36.446 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new a680ca11-7326-4758-9067-d74680391317 6 2023-11-08T19:24:36.471 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.470+0000 7f49f8a1b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T19:24:36.494 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.493+0000 7f2138c3b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.526 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.525+0000 7f49f8a1b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T19:24:36.604 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.603+0000 7f2138c3b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T19:24:36.641 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.640+0000 7f49f8a1b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.666 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.665+0000 7fe4bb514640 1 Processor -- start 2023-11-08T19:24:36.666 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.665+0000 7fe4bb514640 1 -- start start 2023-11-08T19:24:36.668 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4bb514640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b40b3f40 0x7fe4b4167de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.668 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4bb514640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b40d0830 con 0x7fe4b4160530 2023-11-08T19:24:36.668 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4bb514640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b40b27c0 con 0x7fe4b4160910 2023-11-08T19:24:36.669 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4bb514640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40b2aa0 con 0x7fe4b40b3f40 2023-11-08T19:24:36.669 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4b9d11640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fe4b4160910 0x7fe4b40ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:60950/0 (socket says 172.21.15.188:60950) 2023-11-08T19:24:36.669 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4b9d11640 1 -- 172.21.15.188:0/3586146016 learned_addr learned my addr 172.21.15.188:0/3586146016 (peer_addr_for_me v1:172.21.15.188:0/0) 2023-11-08T19:24:36.669 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4bad13640 1 --2- 172.21.15.188:0/3586146016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b40b3f40 0x7fe4b4167de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.671 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4b9510640 1 -- 172.21.15.188:0/3586146016 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168631274 0 0) 0x7fe4b40b27c0 con 0x7fe4b4160910 2023-11-08T19:24:36.671 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.667+0000 7fe4bad13640 1 -- 172.21.15.188:0/3586146016 >> v1:172.21.15.188:6789/0 conn(0x7fe4b4160910 legacy=0x7fe4b40ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.671 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.668+0000 7fe4bad13640 1 -- 172.21.15.188:0/3586146016 >> v1:172.21.15.119:6789/0 conn(0x7fe4b4160530 legacy=0x7fe4b40cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.671 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.668+0000 7fe4bad13640 1 -- 172.21.15.188:0/3586146016 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b40d1640 con 0x7fe4b40b3f40 2023-11-08T19:24:36.672 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.669+0000 7fe4bad13640 1 --2- 172.21.15.188:0/3586146016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b40b3f40 0x7fe4b4167de0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009dc0 tx=0x7fe4ac02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e2c919a076693f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.672 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.669+0000 7fe4b9510640 1 -- 172.21.15.188:0/3586146016 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac02ccd0 con 0x7fe4b40b3f40 2023-11-08T19:24:36.672 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.669+0000 7fe4b9510640 1 -- 172.21.15.188:0/3586146016 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac02ce30 con 0x7fe4b40b3f40 2023-11-08T19:24:36.672 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.669+0000 7fe4b9510640 1 -- 172.21.15.188:0/3586146016 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac034660 con 0x7fe4b40b3f40 2023-11-08T19:24:36.672 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.669+0000 7fe4bb514640 1 -- 172.21.15.188:0/3586146016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b40b3f40 msgr2=0x7fe4b4167de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.673 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.669+0000 7fe4bb514640 1 --2- 172.21.15.188:0/3586146016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b40b3f40 0x7fe4b4167de0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009dc0 tx=0x7fe4ac02bf20 comp rx=0 tx=0).stop 2023-11-08T19:24:36.673 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.670+0000 7fe4bb514640 1 -- 172.21.15.188:0/3586146016 shutdown_connections 2023-11-08T19:24:36.673 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.670+0000 7fe4bb514640 1 --2- 172.21.15.188:0/3586146016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b40b3f40 0x7fe4b4167de0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.673 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.670+0000 7fe4bb514640 1 -- 172.21.15.188:0/3586146016 >> 172.21.15.188:0/3586146016 conn(0x7fe4b40d2ec0 msgr2=0x7fe4b40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.673 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.670+0000 7fe4bb514640 1 -- 172.21.15.188:0/3586146016 shutdown_connections 2023-11-08T19:24:36.673 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.670+0000 7fe4bb514640 1 -- 172.21.15.188:0/3586146016 wait complete. 2023-11-08T19:24:36.674 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.672+0000 7fe4bb514640 1 Processor -- start 2023-11-08T19:24:36.674 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.672+0000 7fe4bb514640 1 -- start start 2023-11-08T19:24:36.676 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4bb514640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 0x7fe4b415c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.677 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4bb514640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b4160530 0x7fe4b40c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.677 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4bb514640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b4160910 0x7fe4b40cc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.677 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4bb514640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40c6c90 con 0x7fe4b4160910 2023-11-08T19:24:36.677 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4bb514640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40c6f70 con 0x7fe4b40b3f40 2023-11-08T19:24:36.677 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4bb514640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b40c7250 con 0x7fe4b4160530 2023-11-08T19:24:36.678 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4ba512640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 0x7fe4b415c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.678 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4ba512640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 0x7fe4b415c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44244/0 (socket says 172.21.15.188:44244) 2023-11-08T19:24:36.678 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4ba512640 1 -- 172.21.15.188:0/696821615 learned_addr learned my addr 172.21.15.188:0/696821615 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.678 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.674+0000 7fe4b9d11640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b4160530 0x7fe4b40c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.678 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4bad13640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b4160910 0x7fe4b40cc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.679 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4ba512640 1 -- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b4160530 msgr2=0x7fe4b40c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.679 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4ba512640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b4160530 0x7fe4b40c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.679 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4ba512640 1 -- 172.21.15.188:0/696821615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b4160910 msgr2=0x7fe4b40cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.679 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4ba512640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b4160910 0x7fe4b40cc4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.679 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4ba512640 1 -- 172.21.15.188:0/696821615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ac009a70 con 0x7fe4b40b3f40 2023-11-08T19:24:36.680 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4ba512640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 0x7fe4b415c5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000c450 tx=0x7fe4b000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.680 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b00052d0 con 0x7fe4b40b3f40 2023-11-08T19:24:36.680 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0005450 con 0x7fe4b40b3f40 2023-11-08T19:24:36.680 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b40c7530 con 0x7fe4b40b3f40 2023-11-08T19:24:36.680 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.675+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0003880 con 0x7fe4b40b3f40 2023-11-08T19:24:36.681 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.676+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b415cb30 con 0x7fe4b40b3f40 2023-11-08T19:24:36.681 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.676+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe4b0003ce0 con 0x7fe4b40b3f40 2023-11-08T19:24:36.681 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.676+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fe4b0011380 con 0x7fe4b40b3f40 2023-11-08T19:24:36.681 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.677+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b40c7a50 con 0x7fe4b40b3f40 2023-11-08T19:24:36.682 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.681+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe4b40c7a50 con 0x7fe4b40b3f40 2023-11-08T19:24:36.759 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.758+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a680ca11-7326-4758-9067-d74680391317", "id": 6} v 0) v1 -- 0x7fe4b404ece0 con 0x7fe4b40b3f40 2023-11-08T19:24:36.760 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.760+0000 7f49f8a1b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T19:24:36.766 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.764+0000 7f2138c3b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.766 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.765+0000 7fe4aaffd640 1 -- 172.21.15.188:0/696821615 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a680ca11-7326-4758-9067-d74680391317", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe4b0012630 con 0x7fe4b40b3f40 2023-11-08T19:24:36.767 INFO:teuthology.orchestra.run.smithi188.stdout:6 2023-11-08T19:24:36.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 msgr2=0x7fe4b415c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 0x7fe4b415c5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4b000c450 tx=0x7fe4b000c910 comp rx=0 tx=0).stop 2023-11-08T19:24:36.768 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 shutdown_connections 2023-11-08T19:24:36.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b4160910 0x7fe4b40cc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b4160530 0x7fe4b40c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 --2- 172.21.15.188:0/696821615 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4b40b3f40 0x7fe4b415c5f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 >> 172.21.15.188:0/696821615 conn(0x7fe4b40d2ec0 msgr2=0x7fe4b415e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.769 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 shutdown_connections 2023-11-08T19:24:36.770 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.767+0000 7fe4bb514640 1 -- 172.21.15.188:0/696821615 wait complete. 2023-11-08T19:24:36.784 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 2c7ab92b-e835-4319-88c7-2b4e829783ba 7 2023-11-08T19:24:36.815 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.814+0000 7f2138c3b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:36.931 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.929+0000 7f49f8a1b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T19:24:36.969 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.968+0000 7fb5c6cca640 1 Processor -- start 2023-11-08T19:24:36.970 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.969+0000 7fb5c6cca640 1 -- start start 2023-11-08T19:24:36.971 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c6cca640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c0160530 0x7fb5c00cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.971 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c6cca640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5c0160910 0x7fb5c00cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.971 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c6cca640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5c00b27c0 con 0x7fb5c00d3ac0 2023-11-08T19:24:36.971 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c6cca640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5c00b2aa0 con 0x7fb5c0160910 2023-11-08T19:24:36.972 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c6cca640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c00d1640 con 0x7fb5c0160530 2023-11-08T19:24:36.972 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c54c7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5c0160910 0x7fb5c00cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.972 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c5cc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c0160530 0x7fb5c00cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.972 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c54c7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5c0160910 0x7fb5c00cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44254/0 (socket says 172.21.15.188:44254) 2023-11-08T19:24:36.972 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.970+0000 7fb5c54c7640 1 -- 172.21.15.188:0/2338802691 learned_addr learned my addr 172.21.15.188:0/2338802691 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.973 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:36.970+0000 7f2138c3b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T19:24:36.974 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.971+0000 7fb5c54c7640 1 -- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c0160530 msgr2=0x7fb5c00cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:36.975 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.971+0000 7fb5c54c7640 1 --2- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c0160530 0x7fb5c00cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.975 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.971+0000 7fb5c54c7640 1 -- 172.21.15.188:0/2338802691 >> v1:172.21.15.119:6789/0 conn(0x7fb5c00d3ac0 legacy=0x7fb5c00cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:36.975 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.971+0000 7fb5c54c7640 1 -- 172.21.15.188:0/2338802691 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c00d1920 con 0x7fb5c0160910 2023-11-08T19:24:36.975 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.972+0000 7fb5c54c7640 1 --2- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160910 0x7fb5c00cdea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=837408625fd82d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.975 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.972+0000 7fb5c4cc6640 1 -- 172.21.15.188:0/2338802691 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc031640 con 0x7fb5c0160910 2023-11-08T19:24:36.976 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.972+0000 7fb5c4cc6640 1 -- 172.21.15.188:0/2338802691 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc0317a0 con 0x7fb5c0160910 2023-11-08T19:24:36.976 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.972+0000 7fb5c4cc6640 1 -- 172.21.15.188:0/2338802691 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc031b30 con 0x7fb5c0160910 2023-11-08T19:24:36.976 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.972+0000 7fb5c6cca640 1 -- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160910 msgr2=0x7fb5c00cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.976 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.972+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160910 0x7fb5c00cdea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.976 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.973+0000 7fb5c6cca640 1 -- 172.21.15.188:0/2338802691 shutdown_connections 2023-11-08T19:24:36.977 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.973+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160910 0x7fb5c00cdea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.977 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.973+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/2338802691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c0160530 0x7fb5c00cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.977 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.973+0000 7fb5c6cca640 1 -- 172.21.15.188:0/2338802691 >> 172.21.15.188:0/2338802691 conn(0x7fb5c00d2ec0 msgr2=0x7fb5c00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:36.977 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.973+0000 7fb5c6cca640 1 -- 172.21.15.188:0/2338802691 shutdown_connections 2023-11-08T19:24:36.977 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.973+0000 7fb5c6cca640 1 -- 172.21.15.188:0/2338802691 wait complete. 2023-11-08T19:24:36.980 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.974+0000 7fb5c6cca640 1 Processor -- start 2023-11-08T19:24:36.980 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.974+0000 7fb5c6cca640 1 -- start start 2023-11-08T19:24:36.980 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.975+0000 7fb5c6cca640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c00d3ac0 0x7fb5c0154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.980 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.975+0000 7fb5c6cca640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 0x7fb5c00b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.981 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.975+0000 7fb5c6cca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0160910 0x7fb5c00b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:36.981 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.975+0000 7fb5c6cca640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c00aff90 con 0x7fb5c0160910 2023-11-08T19:24:36.981 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.975+0000 7fb5c6cca640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c00b0230 con 0x7fb5c0160530 2023-11-08T19:24:36.981 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.975+0000 7fb5c6cca640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c00b0510 con 0x7fb5c00d3ac0 2023-11-08T19:24:36.981 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 0x7fb5c00b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.982 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c5cc8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c00d3ac0 0x7fb5c0154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.982 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 0x7fb5c00b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:44260/0 (socket says 172.21.15.188:44260) 2023-11-08T19:24:36.982 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 -- 172.21.15.188:0/3852543978 learned_addr learned my addr 172.21.15.188:0/3852543978 (peer_addr_for_me v2:172.21.15.188:0/0) 2023-11-08T19:24:36.982 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c64c9640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0160910 0x7fb5c00b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:36.982 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 -- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c00d3ac0 msgr2=0x7fb5c0154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.983 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c00d3ac0 0x7fb5c0154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.983 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 -- 172.21.15.188:0/3852543978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0160910 msgr2=0x7fb5c00b80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:36.983 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0160910 0x7fb5c00b80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:36.983 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 -- 172.21.15.188:0/3852543978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc0094d0 con 0x7fb5c0160530 2023-11-08T19:24:36.983 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c64c9640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0160910 0x7fb5c00b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:36.983 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c5cc8640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c00d3ac0 0x7fb5c0154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:36.984 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.976+0000 7fb5c54c7640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 0x7fb5c00b0ff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:36.984 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc03a930 con 0x7fb5c0160530 2023-11-08T19:24:36.984 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5bc03aa90 con 0x7fb5c0160530 2023-11-08T19:24:36.984 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5bc03ae40 con 0x7fb5c0160530 2023-11-08T19:24:36.984 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5c00b07f0 con 0x7fb5c0160530 2023-11-08T19:24:36.985 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5c00b1530 con 0x7fb5c0160530 2023-11-08T19:24:36.985 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb5bc02f050 con 0x7fb5c0160530 2023-11-08T19:24:36.985 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.977+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fb5bc044b90 con 0x7fb5c0160530 2023-11-08T19:24:36.985 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.978+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5c004ece0 con 0x7fb5c0160530 2023-11-08T19:24:36.986 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:36.982+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb5bc035020 con 0x7fb5c0160530 2023-11-08T19:24:36.986 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:36.984+0000 7f49f8a1b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T19:24:37.061 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.060+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2c7ab92b-e835-4319-88c7-2b4e829783ba", "id": 7} v 0) v1 -- 0x7fb5c00b0d10 con 0x7fb5c0160530 2023-11-08T19:24:37.071 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.070+0000 7fb5b27fc640 1 -- 172.21.15.188:0/3852543978 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2c7ab92b-e835-4319-88c7-2b4e829783ba", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb5bc02d040 con 0x7fb5c0160530 2023-11-08T19:24:37.071 INFO:teuthology.orchestra.run.smithi188.stdout:7 2023-11-08T19:24:37.072 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 msgr2=0x7fb5c00b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.072 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 0x7fb5c00b0ff0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc009870 tx=0x7fb5bc0046c0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 shutdown_connections 2023-11-08T19:24:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5c0160910 0x7fb5c00b80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5c0160530 0x7fb5c00b0ff0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 --2- 172.21.15.188:0/3852543978 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5c00d3ac0 0x7fb5c0154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.073 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 >> 172.21.15.188:0/3852543978 conn(0x7fb5c00d2ec0 msgr2=0x7fb5c0155100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 shutdown_connections 2023-11-08T19:24:37.074 INFO:teuthology.orchestra.run.smithi188.stderr:2023-11-08T19:24:37.071+0000 7fb5c6cca640 1 -- 172.21.15.188:0/3852543978 wait complete. 2023-11-08T19:24:37.088 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-08T19:24:37.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-08T19:24:37.097 INFO:tasks.ceph.osd.0:Started 2023-11-08T19:24:37.098 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-08T19:24:37.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-08T19:24:37.106 INFO:tasks.ceph.osd.1:Started 2023-11-08T19:24:37.107 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-08T19:24:37.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-08T19:24:37.115 INFO:tasks.ceph.osd.2:Started 2023-11-08T19:24:37.115 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-08T19:24:37.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-08T19:24:37.123 INFO:tasks.ceph.osd.3:Started 2023-11-08T19:24:37.123 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-08T19:24:37.124 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-08T19:24:37.132 INFO:tasks.ceph.osd.4:Started 2023-11-08T19:24:37.132 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-08T19:24:37.132 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-08T19:24:37.140 INFO:tasks.ceph.osd.5:Started 2023-11-08T19:24:37.141 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-08T19:24:37.141 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-08T19:24:37.146 INFO:tasks.ceph.osd.6:Started 2023-11-08T19:24:37.147 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-08T19:24:37.147 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-08T19:24:37.153 INFO:tasks.ceph.osd.7:Started 2023-11-08T19:24:37.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:37.155 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:37.149+0000 7f49f8a1b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T19:24:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.376+0000 7f18c7fff640 1 Processor -- start 2023-11-08T19:24:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.376+0000 7f18c7fff640 1 -- start start 2023-11-08T19:24:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f18c815d4c0 0x7f18c8167e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c7fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80c7100 con 0x7f18c80d1d50 2023-11-08T19:24:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c7fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18c80d04f0 con 0x7f18c815d0e0 2023-11-08T19:24:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c7fff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18c80d07d0 con 0x7f18c815d4c0 2023-11-08T19:24:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c67fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f18c80d1d50 0x7f18c80ce180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44448/0 (socket says 172.21.15.119:44448) 2023-11-08T19:24:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c67fc640 1 -- 172.21.15.119:0/4225143860 learned_addr learned my addr 172.21.15.119:0/4225143860 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.377+0000 7f18c77fe640 1 --2- 172.21.15.119:0/4225143860 >> v2:172.21.15.188:3300/0 conn(0x7f18c815d4c0 0x7f18c8167e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139433850 0 0) 0x7f18c80c7100 con 0x7f18c80d1d50 2023-11-08T19:24:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18b0003520 con 0x7f18c80d1d50 2023-11-08T19:24:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335834235 0 0) 0x7f18c80d04f0 con 0x7f18c815d0e0 2023-11-08T19:24:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f18c80c7100 con 0x7f18c815d0e0 2023-11-08T19:24:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c77fe640 1 -- 172.21.15.119:0/4225143860 >> v1:172.21.15.188:6789/0 conn(0x7f18c815d0e0 legacy=0x7f18c80cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c77fe640 1 -- 172.21.15.119:0/4225143860 >> v1:172.21.15.119:6789/0 conn(0x7f18c80d1d50 legacy=0x7f18c80ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.378+0000 7f18c77fe640 1 -- 172.21.15.119:0/4225143860 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c80c6ab0 con 0x7f18c815d4c0 2023-11-08T19:24:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.379+0000 7f18c77fe640 1 --2- 172.21.15.119:0/4225143860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d4c0 0x7f18c8167e10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009d90 tx=0x7f18c002bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebbb48867a438e51 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.379+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c002cd80 con 0x7f18c815d4c0 2023-11-08T19:24:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.379+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0034440 con 0x7f18c815d4c0 2023-11-08T19:24:37.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.379+0000 7f18c5ffb640 1 -- 172.21.15.119:0/4225143860 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c00347d0 con 0x7f18c815d4c0 2023-11-08T19:24:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 -- 172.21.15.119:0/4225143860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d4c0 msgr2=0x7f18c8167e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 --2- 172.21.15.119:0/4225143860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d4c0 0x7f18c8167e10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009d90 tx=0x7f18c002bfd0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 -- 172.21.15.119:0/4225143860 shutdown_connections 2023-11-08T19:24:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 --2- 172.21.15.119:0/4225143860 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d4c0 0x7f18c8167e10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 -- 172.21.15.119:0/4225143860 >> 172.21.15.119:0/4225143860 conn(0x7f18c80d2f60 msgr2=0x7f18c80d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:37.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 -- 172.21.15.119:0/4225143860 shutdown_connections 2023-11-08T19:24:37.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.380+0000 7f18c7fff640 1 -- 172.21.15.119:0/4225143860 wait complete. 2023-11-08T19:24:37.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.382+0000 7f18c7fff640 1 Processor -- start 2023-11-08T19:24:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.382+0000 7f18c7fff640 1 -- start start 2023-11-08T19:24:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 0x7f18c80b6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d0e0 0x7f18c815f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:37.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18c815d4c0 0x7f18c815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c7fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80b7880 con 0x7f18c80d1d50 2023-11-08T19:24:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c7fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18c815fa40 con 0x7f18c815d0e0 2023-11-08T19:24:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c7fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18c80b6c00 con 0x7f18c815d4c0 2023-11-08T19:24:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 0x7f18c80b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:37.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 0x7f18c80b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33006/0 (socket says 172.21.15.119:33006) 2023-11-08T19:24:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c6ffd640 1 -- 172.21.15.119:0/1311734100 learned_addr learned my addr 172.21.15.119:0/1311734100 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c67fc640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d0e0 0x7f18c815f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.383+0000 7f18c77fe640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18c815d4c0 0x7f18c815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c6ffd640 1 -- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18c815d4c0 msgr2=0x7f18c815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c6ffd640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18c815d4c0 0x7f18c815a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c6ffd640 1 -- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d0e0 msgr2=0x7f18c815f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c6ffd640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d0e0 0x7f18c815f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c6ffd640 1 -- 172.21.15.119:0/1311734100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009a70 con 0x7f18c80d1d50 2023-11-08T19:24:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c67fc640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d0e0 0x7f18c815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:24:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c6ffd640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 0x7f18c80b6590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c490 tx=0x7f18b800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:37.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b80052d0 con 0x7f18c80d1d50 2023-11-08T19:24:37.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18b8005430 con 0x7f18c80d1d50 2023-11-08T19:24:37.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18c80b6ee0 con 0x7f18c80d1d50 2023-11-08T19:24:37.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b8012870 con 0x7f18c80d1d50 2023-11-08T19:24:37.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18c80b74a0 con 0x7f18c80d1d50 2023-11-08T19:24:37.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.384+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f18b8012cd0 con 0x7f18c80d1d50 2023-11-08T19:24:37.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.385+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f18b801b400 con 0x7f18c80d1d50 2023-11-08T19:24:37.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.385+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18c804ece0 con 0x7f18c80d1d50 2023-11-08T19:24:37.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.388+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f18b80035f0 con 0x7f18c80d1d50 2023-11-08T19:24:37.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.451+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18c8155990 con 0x7f18c80d1d50 2023-11-08T19:24:37.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.452+0000 7f18af7fe640 1 -- 172.21.15.119:0/1311734100 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f18b8003790 con 0x7f18c80d1d50 2023-11-08T19:24:37.454 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:24:37.454 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":10,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:24:37.062283+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:37.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.453+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 msgr2=0x7f18c80b6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:37.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.453+0000 7f18c7fff640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 0x7f18c80b6590 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f18b800c490 tx=0x7f18b800c950 comp rx=0 tx=0).stop 2023-11-08T19:24:37.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 shutdown_connections 2023-11-08T19:24:37.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f18c815d4c0 0x7f18c815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f18c815d0e0 0x7f18c815f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 --2- 172.21.15.119:0/1311734100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f18c80d1d50 0x7f18c80b6590 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:37.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 >> 172.21.15.119:0/1311734100 conn(0x7f18c80d2f60 msgr2=0x7f18c815b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:37.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 shutdown_connections 2023-11-08T19:24:37.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:37.454+0000 7f18c7fff640 1 -- 172.21.15.119:0/1311734100 wait complete. 2023-11-08T19:24:37.471 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-08T19:24:37.472 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-08T19:24:37.820 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-11-08T19:24:37.818+0000 7f969f80d500 -1 Falling back to public interface 2023-11-08T19:24:37.829 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-08T19:24:37.828+0000 7f91997ad500 -1 Falling back to public interface 2023-11-08T19:24:37.833 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-11-08T19:24:37.832+0000 7fc89ac0e500 -1 Falling back to public interface 2023-11-08T19:24:37.833 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-11-08T19:24:37.832+0000 7ffbcb415500 -1 Falling back to public interface 2023-11-08T19:24:37.834 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-11-08T19:24:37.832+0000 7fc89955b640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6808/0: (98) Address already in use 2023-11-08T19:24:37.834 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-08T19:24:37.834+0000 7fbf27148500 -1 Falling back to public interface 2023-11-08T19:24:37.842 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-11-08T19:24:37.841+0000 7f6954423500 -1 Falling back to public interface 2023-11-08T19:24:37.868 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-08T19:24:37.867+0000 7f94d9c24500 -1 Falling back to public interface 2023-11-08T19:24:37.871 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-08T19:24:37.871+0000 7fd60bbad500 -1 Falling back to public interface 2023-11-08T19:24:38.128 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:38.127+0000 7f2138c3b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T19:24:38.220 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:38.219+0000 7f49f8a1b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T19:24:38.278 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:38.438 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:38.436+0000 7f2138c3b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T19:24:38.453 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.452+0000 7f0b44e90640 1 Processor -- start 2023-11-08T19:24:38.453 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.452+0000 7f0b44e90640 1 -- start start 2023-11-08T19:24:38.454 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b44e90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:38.455 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b44e90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b40161870 0x7f0b400cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:38.455 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b44e90640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4015df70 con 0x7f0b400cfa00 2023-11-08T19:24:38.455 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b44e90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4015daa0 con 0x7f0b40161870 2023-11-08T19:24:38.455 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b44e90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400d1e60 con 0x7f0b4015d6c0 2023-11-08T19:24:38.456 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b37fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0b400cfa00 0x7f0b40161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:44462/0 (socket says 172.21.15.119:44462) 2023-11-08T19:24:38.456 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b37fff640 1 -- 172.21.15.119:0/3866920850 learned_addr learned my addr 172.21.15.119:0/3866920850 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:38.456 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3866920850 >> v2:172.21.15.188:3300/0 conn(0x7f0b40161870 0x7f0b400cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:38.456 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.453+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:38.458 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/3866920850 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228471423 0 0) 0x7f0b4015df70 con 0x7f0b400cfa00 2023-11-08T19:24:38.458 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/3866920850 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b2c003520 con 0x7f0b400cfa00 2023-11-08T19:24:38.458 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 msgr2=0x7f0b400d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:38.458 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.459 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/3866920850 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 357228871 0 0) 0x7f0b2c003520 con 0x7f0b400cfa00 2023-11-08T19:24:38.459 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3866920850 >> v1:172.21.15.119:6789/0 conn(0x7f0b400cfa00 legacy=0x7f0b40161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:38.459 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3866920850 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400d03a0 con 0x7f0b40161870 2023-11-08T19:24:38.459 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.454+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:38.459 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.455+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b40161870 0x7f0b400cfe60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009d90 tx=0x7f0b3802bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26d412673eb8b1e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:38.460 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.455+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/3866920850 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28001670 con 0x7f0b40161870 2023-11-08T19:24:38.460 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.455+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/3866920850 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3802c810 con 0x7f0b40161870 2023-11-08T19:24:38.460 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.455+0000 7f0b3dd9b640 1 -- 172.21.15.119:0/3866920850 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3802cba0 con 0x7f0b40161870 2023-11-08T19:24:38.460 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 -- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b40161870 msgr2=0x7f0b400cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:38.460 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b40161870 0x7f0b400cfe60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009d90 tx=0x7f0b3802bce0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.461 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 -- 172.21.15.119:0/3866920850 shutdown_connections 2023-11-08T19:24:38.461 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b40161870 0x7f0b400cfe60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.461 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3866920850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.461 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 -- 172.21.15.119:0/3866920850 >> 172.21.15.119:0/3866920850 conn(0x7f0b4006a9f0 msgr2=0x7f0b4006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:38.462 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 -- 172.21.15.119:0/3866920850 shutdown_connections 2023-11-08T19:24:38.462 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.456+0000 7f0b44e90640 1 -- 172.21.15.119:0/3866920850 wait complete. 2023-11-08T19:24:38.464 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.458+0000 7f0b44e90640 1 Processor -- start 2023-11-08T19:24:38.464 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.458+0000 7f0b44e90640 1 -- start start 2023-11-08T19:24:38.465 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b44e90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b400cfa00 0x7f0b400b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:38.465 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b44e90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:38.465 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b44e90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 0x7f0b400aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:38.465 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b44e90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400aaac0 con 0x7f0b40161870 2023-11-08T19:24:38.465 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b44e90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400aada0 con 0x7f0b400cfa00 2023-11-08T19:24:38.466 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b44e90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b400ab080 con 0x7f0b4015d6c0 2023-11-08T19:24:38.466 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b3ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 0x7f0b400aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:38.466 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b3ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 0x7f0b400aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33018/0 (socket says 172.21.15.119:33018) 2023-11-08T19:24:38.466 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b37fff640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:38.467 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3173346249 learned_addr learned my addr 172.21.15.119:0/3173346249 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:38.467 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.459+0000 7f0b3e59c640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b400cfa00 0x7f0b400b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:38.467 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 msgr2=0x7f0b400a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:38.467 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.467 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b400cfa00 msgr2=0x7f0b400b20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:38.468 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b400cfa00 0x7f0b400b20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.468 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b3ed9d640 1 -- 172.21.15.119:0/3173346249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b30009400 con 0x7f0b40161870 2023-11-08T19:24:38.468 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b3ed9d640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 0x7f0b400aa490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009d90 tx=0x7f0b38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:38.468 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.460+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38009450 con 0x7f0b40161870 2023-11-08T19:24:38.468 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.461+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b38034c50 con 0x7f0b40161870 2023-11-08T19:24:38.469 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.461+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b38040450 con 0x7f0b40161870 2023-11-08T19:24:38.469 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.461+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b38009a70 con 0x7f0b40161870 2023-11-08T19:24:38.469 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.461+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b400ab640 con 0x7f0b40161870 2023-11-08T19:24:38.469 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.461+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0b380408b0 con 0x7f0b40161870 2023-11-08T19:24:38.469 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.461+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f0b38049400 con 0x7f0b40161870 2023-11-08T19:24:38.470 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.462+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b40074430 con 0x7f0b40161870 2023-11-08T19:24:38.470 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.466+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f0b38044020 con 0x7f0b40161870 2023-11-08T19:24:38.502 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:38.501+0000 7f2138c3b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T19:24:38.531 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:38.530+0000 7f49f8a1b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T19:24:38.532 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.531+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b400d3b40 con 0x7f0b40161870 2023-11-08T19:24:38.533 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.532+0000 7f0b36ffd640 1 -- 172.21.15.119:0/3173346249 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f0b38039070 con 0x7f0b40161870 2023-11-08T19:24:38.534 INFO:teuthology.misc.health.smithi119.stdout: 2023-11-08T19:24:38.534 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":10,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:24:37.062283+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:38.535 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.533+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 msgr2=0x7f0b400aa490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:38.535 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.533+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 0x7f0b400aa490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009d90 tx=0x7f0b38004a40 comp rx=0 tx=0).stop 2023-11-08T19:24:38.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 shutdown_connections 2023-11-08T19:24:38.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b40161870 0x7f0b400aa490 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4015d6c0 0x7f0b400a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 --2- 172.21.15.119:0/3173346249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b400cfa00 0x7f0b400b20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:38.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 >> 172.21.15.119:0/3173346249 conn(0x7f0b4006a9f0 msgr2=0x7f0b400b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:38.537 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 shutdown_connections 2023-11-08T19:24:38.537 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:38.534+0000 7f0b44e90640 1 -- 172.21.15.119:0/3173346249 wait complete. 2023-11-08T19:24:38.548 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T19:24:38.591 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:38.590+0000 7f49f8a1b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T19:24:38.619 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:38.618+0000 7f2138c3b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T19:24:38.713 INFO:tasks.ceph.mgr.x.smithi188.stderr:2023-11-08T19:24:38.712+0000 7f49f8a1b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T19:24:39.675 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:39.674+0000 7f20da011640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-08T19:24:39.676 INFO:tasks.ceph.mgr.y.smithi119.stderr:2023-11-08T19:24:39.674+0000 7f20da011640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-08T19:24:40.667 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-08T19:24:40.666+0000 7f91997ad500 -1 osd.4 0 log_to_monitors true 2023-11-08T19:24:41.405 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-11-08T19:24:41.404+0000 7fc89ac0e500 -1 osd.2 0 log_to_monitors true 2023-11-08T19:24:41.407 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-11-08T19:24:41.406+0000 7f6954423500 -1 osd.1 0 log_to_monitors true 2023-11-08T19:24:41.431 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-08T19:24:41.430+0000 7f94d9c24500 -1 osd.7 0 log_to_monitors true 2023-11-08T19:24:41.438 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-08T19:24:41.438+0000 7fd60bbad500 -1 osd.5 0 log_to_monitors true 2023-11-08T19:24:41.615 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-11-08T19:24:41.614+0000 7f969f80d500 -1 osd.0 0 log_to_monitors true 2023-11-08T19:24:41.634 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-08T19:24:41.633+0000 7fbf27148500 -1 osd.6 0 log_to_monitors true 2023-11-08T19:24:41.663 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-11-08T19:24:41.662+0000 7ffbcb415500 -1 osd.3 0 log_to_monitors true 2023-11-08T19:24:44.955 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-08T19:24:44.953+0000 7f9195f51640 -1 osd.4 0 waiting for initial osdmap 2023-11-08T19:24:44.967 INFO:tasks.ceph.osd.4.smithi188.stderr:2023-11-08T19:24:44.964+0000 7f918d564640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.019 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-08T19:24:45.017+0000 7fbf238ea640 -1 osd.6 0 waiting for initial osdmap 2023-11-08T19:24:45.032 INFO:tasks.ceph.osd.6.smithi188.stderr:2023-11-08T19:24:45.030+0000 7fbf1aefd640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.091 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-11-08T19:24:45.090+0000 7ffbc75f9640 -1 osd.3 0 waiting for initial osdmap 2023-11-08T19:24:45.094 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-08T19:24:45.092+0000 7f94d5568640 -1 osd.7 0 waiting for initial osdmap 2023-11-08T19:24:45.102 INFO:tasks.ceph.osd.7.smithi188.stderr:2023-11-08T19:24:45.100+0000 7f94cd38f640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.113 INFO:tasks.ceph.osd.3.smithi119.stderr:2023-11-08T19:24:45.111+0000 7ffbbec0c640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.116 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-08T19:24:45.114+0000 7fd607b3b640 -1 osd.5 0 waiting for initial osdmap 2023-11-08T19:24:45.123 INFO:tasks.ceph.osd.5.smithi188.stderr:2023-11-08T19:24:45.121+0000 7fd5ff962640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.146 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-11-08T19:24:45.145+0000 7f969b8d2640 -1 osd.0 0 waiting for initial osdmap 2023-11-08T19:24:45.152 INFO:tasks.ceph.osd.0.smithi119.stderr:2023-11-08T19:24:45.151+0000 7f9692ee5640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.177 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-11-08T19:24:45.176+0000 7fc896d56640 -1 osd.2 0 waiting for initial osdmap 2023-11-08T19:24:45.180 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-11-08T19:24:45.179+0000 7f694fe13640 -1 osd.1 0 waiting for initial osdmap 2023-11-08T19:24:45.189 INFO:tasks.ceph.osd.2.smithi119.stderr:2023-11-08T19:24:45.188+0000 7fc88e369640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.192 INFO:tasks.ceph.osd.1.smithi119.stderr:2023-11-08T19:24:45.191+0000 7f6947c3a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T19:24:45.355 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:45.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.534+0000 7f6bb4cc9640 1 Processor -- start 2023-11-08T19:24:45.536 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.534+0000 7f6bb4cc9640 1 -- start start 2023-11-08T19:24:45.538 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6bb4cc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb015af80 0x7f6bb00cea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.538 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6bb4cc9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6bb015b360 0x7f6bb00cefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.539 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6bb4cc9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6bb00d0af0 0x7f6bb00d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.539 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6bb4cc9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb00cfe90 con 0x7f6bb015b360 2023-11-08T19:24:45.539 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6bb4cc9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bb00c4a00 con 0x7f6bb00d0af0 2023-11-08T19:24:45.539 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6bb4cc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00cbd30 con 0x7f6bb015af80 2023-11-08T19:24:45.539 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6ba7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6bb015b360 0x7f6bb00cefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.540 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6ba7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6bb015b360 0x7f6bb00cefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50296/0 (socket says 172.21.15.119:50296) 2023-11-08T19:24:45.540 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.536+0000 7f6ba7fff640 1 -- 172.21.15.119:0/25597999 learned_addr learned my addr 172.21.15.119:0/25597999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:45.540 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6bae59c640 1 --2- 172.21.15.119:0/25597999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb015af80 0x7f6bb00cea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.540 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6baed9d640 1 --2- 172.21.15.119:0/25597999 >> v2:172.21.15.188:3300/0 conn(0x7f6bb00d0af0 0x7f6bb00d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.540 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6ba7fff640 1 -- 172.21.15.119:0/25597999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb015af80 msgr2=0x7f6bb00cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:45.541 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6ba7fff640 1 --2- 172.21.15.119:0/25597999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb015af80 0x7f6bb00cea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.541 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6ba7fff640 1 -- 172.21.15.119:0/25597999 >> v2:172.21.15.188:3300/0 conn(0x7f6bb00d0af0 msgr2=0x7f6bb00d0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:45.541 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6ba7fff640 1 --2- 172.21.15.119:0/25597999 >> v2:172.21.15.188:3300/0 conn(0x7f6bb00d0af0 0x7f6bb00d0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.541 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6ba7fff640 1 -- 172.21.15.119:0/25597999 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb00cf5f0 con 0x7f6bb015b360 2023-11-08T19:24:45.541 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.537+0000 7f6baed9d640 1 --2- 172.21.15.119:0/25597999 >> v2:172.21.15.188:3300/0 conn(0x7f6bb00d0af0 0x7f6bb00d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:45.544 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.538+0000 7f6ba7fff640 1 --2- 172.21.15.119:0/25597999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb00cefc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00cd5b0 tx=0x7f6b9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffa3536a790a47b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:45.545 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.538+0000 7f6badd9b640 1 -- 172.21.15.119:0/25597999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98031870 con 0x7f6bb015b360 2023-11-08T19:24:45.545 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.538+0000 7f6badd9b640 1 -- 172.21.15.119:0/25597999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b980319d0 con 0x7f6bb015b360 2023-11-08T19:24:45.545 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.538+0000 7f6badd9b640 1 -- 172.21.15.119:0/25597999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b98031d60 con 0x7f6bb015b360 2023-11-08T19:24:45.545 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/25597999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 msgr2=0x7f6bb00cefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:45.545 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/25597999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb00cefc0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00cd5b0 tx=0x7f6b9802b910 comp rx=0 tx=0).stop 2023-11-08T19:24:45.546 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/25597999 shutdown_connections 2023-11-08T19:24:45.546 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/25597999 >> v2:172.21.15.188:3300/0 conn(0x7f6bb00d0af0 0x7f6bb00d0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.546 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/25597999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb00cefc0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.546 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/25597999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb015af80 0x7f6bb00cea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.546 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/25597999 >> 172.21.15.119:0/25597999 conn(0x7f6bb008afe0 msgr2=0x7f6bb009a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:45.547 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/25597999 shutdown_connections 2023-11-08T19:24:45.547 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.539+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/25597999 wait complete. 2023-11-08T19:24:45.547 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.540+0000 7f6bb4cc9640 1 Processor -- start 2023-11-08T19:24:45.547 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.540+0000 7f6bb4cc9640 1 -- start start 2023-11-08T19:24:45.547 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.541+0000 7f6bb4cc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb00d0af0 0x7f6bb0154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.548 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.541+0000 7f6bb4cc9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bb015af80 0x7f6bb01502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.548 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.541+0000 7f6bb4cc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb015db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.548 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.541+0000 7f6bb4cc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00b5230 con 0x7f6bb015b360 2023-11-08T19:24:45.548 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.541+0000 7f6bb4cc9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00cb000 con 0x7f6bb015af80 2023-11-08T19:24:45.548 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.541+0000 7f6bb4cc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bb00ca760 con 0x7f6bb00d0af0 2023-11-08T19:24:45.549 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb015db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.549 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb015db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50302/0 (socket says 172.21.15.119:50302) 2023-11-08T19:24:45.549 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 -- 172.21.15.119:0/3277785515 learned_addr learned my addr 172.21.15.119:0/3277785515 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:45.549 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6bae59c640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb00d0af0 0x7f6bb0154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.549 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 -- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb00d0af0 msgr2=0x7f6bb0154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:45.550 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb00d0af0 0x7f6bb0154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.550 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 -- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bb015af80 msgr2=0x7f6bb01502b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:45.550 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6ba7fff640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bb015af80 0x7f6bb01502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.550 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bb015af80 0x7f6bb01502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.550 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 -- 172.21.15.119:0/3277785515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b980094d0 con 0x7f6bb015b360 2023-11-08T19:24:45.551 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6baed9d640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb015db20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800cf20 tx=0x7f6ba800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:45.551 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800d840 con 0x7f6bb015b360 2023-11-08T19:24:45.551 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba800d9a0 con 0x7f6bb015b360 2023-11-08T19:24:45.551 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.542+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bb00caa40 con 0x7f6bb015b360 2023-11-08T19:24:45.551 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.543+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800dd50 con 0x7f6bb015b360 2023-11-08T19:24:45.551 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.543+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bb00c8140 con 0x7f6bb015b360 2023-11-08T19:24:45.553 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.544+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bb00b53e0 con 0x7f6bb015b360 2023-11-08T19:24:45.553 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.544+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f6ba801e3b0 con 0x7f6bb015b360 2023-11-08T19:24:45.553 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.545+0000 7f6ba6ffd640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6b9006ae10 0x7f6b9006d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:45.553 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.545+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f6ba8089290 con 0x7f6bb015b360 2023-11-08T19:24:45.553 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.546+0000 7f6bae59c640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6b9006ae10 0x7f6b9006d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:45.554 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.546+0000 7f6bae59c640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6b9006ae10 0x7f6b9006d2d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006fd0 tx=0x7f6ba0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:45.554 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.550+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bb00b53e0 con 0x7f6bb015b360 2023-11-08T19:24:45.733 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.731+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bb00b53e0 con 0x7f6bb015b360 2023-11-08T19:24:45.733 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.732+0000 7f6ba6ffd640 1 -- 172.21.15.119:0/3277785515 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f6bb00b53e0 con 0x7f6bb015b360 2023-11-08T19:24:45.734 INFO:teuthology.misc.health.smithi119.stdout: 2023-11-08T19:24:45.734 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":12,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:24:42.684302+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:45.736 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.734+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6b9006ae10 msgr2=0x7f6b9006d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:45.737 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.734+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6b9006ae10 0x7f6b9006d2d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0006fd0 tx=0x7f6ba0008040 comp rx=0 tx=0).stop 2023-11-08T19:24:45.737 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 msgr2=0x7f6bb015db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:45.737 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb015db20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800cf20 tx=0x7f6ba800a7d0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.737 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 shutdown_connections 2023-11-08T19:24:45.737 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6b9006ae10 0x7f6b9006d2d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.738 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6bb015b360 0x7f6bb015db20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.738 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6bb015af80 0x7f6bb01502b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.738 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 --2- 172.21.15.119:0/3277785515 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6bb00d0af0 0x7f6bb0154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:45.738 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 >> 172.21.15.119:0/3277785515 conn(0x7f6bb008afe0 msgr2=0x7f6bb00b2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:45.738 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 shutdown_connections 2023-11-08T19:24:45.739 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:45.735+0000 7f6bb4cc9640 1 -- 172.21.15.119:0/3277785515 wait complete. 2023-11-08T19:24:45.749 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T19:24:48.460 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:48.469 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:24:52.556 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:52.741 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.740+0000 7f99c600e640 1 Processor -- start 2023-11-08T19:24:52.742 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.740+0000 7f99c600e640 1 -- start start 2023-11-08T19:24:52.743 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.741+0000 7f99c600e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 0x7f99c00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.743 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99c600e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00c4960 con 0x7f99c015d9f0 2023-11-08T19:24:52.743 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99c600e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c015df60 con 0x7f99c006cb40 2023-11-08T19:24:52.744 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99c600e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0143a40 con 0x7f99c0142f80 2023-11-08T19:24:52.744 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99b7fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f99c015d9f0 0x7f99c00c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57128/0 (socket says 172.21.15.119:57128) 2023-11-08T19:24:52.744 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99b7fff640 1 -- 172.21.15.119:0/681315551 learned_addr learned my addr 172.21.15.119:0/681315551 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:52.746 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 0x7f99c00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.746 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.742+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546770916 0 0) 0x7f99c00c4960 con 0x7f99c015d9f0 2023-11-08T19:24:52.747 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac003520 con 0x7f99c015d9f0 2023-11-08T19:24:52.747 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1931889312 0 0) 0x7f99ac003520 con 0x7f99c015d9f0 2023-11-08T19:24:52.747 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 msgr2=0x7f99c00cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:52.747 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 0x7f99c00cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.747 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 >> v1:172.21.15.188:6789/0 conn(0x7f99c006cb40 legacy=0x7f99c00c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.747 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c015e240 con 0x7f99c015d9f0 2023-11-08T19:24:52.748 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f99a8001670 con 0x7f99c015d9f0 2023-11-08T19:24:52.748 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 >> v1:172.21.15.119:6789/0 conn(0x7f99c015d9f0 legacy=0x7f99c00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.748 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac0039a0 0x7f99ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.748 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 0x7f99ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.748 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac004d00 0x7f99ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.749 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0143a40 con 0x7f99ac0039a0 2023-11-08T19:24:52.749 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac00bab0 con 0x7f99ac004d00 2023-11-08T19:24:52.749 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99ac00de80 con 0x7f99c0142f80 2023-11-08T19:24:52.749 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.743+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac0039a0 0x7f99ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.749 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.744+0000 7f99bf7fe640 1 -- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 msgr2=0x7f99ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:52.749 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.744+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c0142f80 0x7f99ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.750 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.744+0000 7f99bf7fe640 1 -- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac004d00 msgr2=0x7f99ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:52.750 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.744+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac004d00 0x7f99ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.750 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.744+0000 7f99bf7fe640 1 -- 172.21.15.119:0/681315551 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99ac003520 con 0x7f99ac0039a0 2023-11-08T19:24:52.750 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac0039a0 0x7f99ac003dc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009ac0 tx=0x7f99ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb89bf8392d56a40 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:52.750 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b002c630 con 0x7f99ac0039a0 2023-11-08T19:24:52.751 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b002c790 con 0x7f99ac0039a0 2023-11-08T19:24:52.751 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99beffd640 1 -- 172.21.15.119:0/681315551 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b002cb20 con 0x7f99ac0039a0 2023-11-08T19:24:52.753 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99c600e640 1 -- 172.21.15.119:0/681315551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac0039a0 msgr2=0x7f99ac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.754 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99c600e640 1 --2- 172.21.15.119:0/681315551 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99ac0039a0 0x7f99ac003dc0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f99b0009ac0 tx=0x7f99ac0057f0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.754 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.745+0000 7f99bffff640 1 -- 172.21.15.119:0/681315551 reap_dead start 2023-11-08T19:24:52.754 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.746+0000 7f99c600e640 1 -- 172.21.15.119:0/681315551 shutdown_connections 2023-11-08T19:24:52.754 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.746+0000 7f99c600e640 1 -- 172.21.15.119:0/681315551 >> 172.21.15.119:0/681315551 conn(0x7f99c00cb610 msgr2=0x7f99c00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:52.754 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.746+0000 7f99c600e640 1 -- 172.21.15.119:0/681315551 shutdown_connections 2023-11-08T19:24:52.755 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.746+0000 7f99c600e640 1 -- 172.21.15.119:0/681315551 wait complete. 2023-11-08T19:24:52.755 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.747+0000 7f99c600e640 1 Processor -- start 2023-11-08T19:24:52.755 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.747+0000 7f99c600e640 1 -- start start 2023-11-08T19:24:52.755 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.748+0000 7f99c600e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac0039a0 0x7f99c0140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.755 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.748+0000 7f99c600e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00c8690 0x7f99c015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.756 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.748+0000 7f99c600e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 0x7f99c00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.756 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.748+0000 7f99c600e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c01413d0 con 0x7f99c015ce80 2023-11-08T19:24:52.756 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.748+0000 7f99c600e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0140a00 con 0x7f99ac0039a0 2023-11-08T19:24:52.756 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.748+0000 7f99c600e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0159980 con 0x7f99c00c8690 2023-11-08T19:24:52.756 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 0x7f99c00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.757 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 0x7f99c00c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44308/0 (socket says 172.21.15.119:44308) 2023-11-08T19:24:52.757 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 -- 172.21.15.119:0/1972744942 learned_addr learned my addr 172.21.15.119:0/1972744942 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:52.757 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac0039a0 0x7f99c0140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.757 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99b7fff640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00c8690 0x7f99c015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.757 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 -- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00c8690 msgr2=0x7f99c015c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.758 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00c8690 0x7f99c015c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.758 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 -- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac0039a0 msgr2=0x7f99c0140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.758 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac0039a0 0x7f99c0140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.758 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 -- 172.21.15.119:0/1972744942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c013fdf0 con 0x7f99c015ce80 2023-11-08T19:24:52.758 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac0039a0 0x7f99c0140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:52.758 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bffff640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 0x7f99c00c5bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c9080 tx=0x7f99b800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:52.759 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.749+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8005d40 con 0x7f99c015ce80 2023-11-08T19:24:52.759 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.750+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b800a610 con 0x7f99c015ce80 2023-11-08T19:24:52.759 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.750+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800a9c0 con 0x7f99c015ce80 2023-11-08T19:24:52.759 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.750+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c01400d0 con 0x7f99c015ce80 2023-11-08T19:24:52.759 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.750+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c0140690 con 0x7f99c015ce80 2023-11-08T19:24:52.760 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.751+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0140850 con 0x7f99c015ce80 2023-11-08T19:24:52.760 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.752+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f99b800ab20 con 0x7f99c015ce80 2023-11-08T19:24:52.761 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.753+0000 7f99bcff9640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f99a006ae70 0x7f99a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:52.761 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.753+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f99b80882f0 con 0x7f99c015ce80 2023-11-08T19:24:52.761 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.753+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f99a006ae70 0x7f99a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:52.761 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.754+0000 7f99bf7fe640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f99a006ae70 0x7f99a006d330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f99b0036000 tx=0x7f99b0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:52.761 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.758+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99b8053650 con 0x7f99c015ce80 2023-11-08T19:24:52.937 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.935+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c0140b40 con 0x7f99c015ce80 2023-11-08T19:24:52.938 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.937+0000 7f99bcff9640 1 -- 172.21.15.119:0/1972744942 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f99b8056c70 con 0x7f99c015ce80 2023-11-08T19:24:52.939 INFO:teuthology.misc.health.smithi119.stdout: 2023-11-08T19:24:52.939 INFO:teuthology.misc.health.smithi119.stdout:{"epoch":16,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:24:49.581137+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:52.941 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.939+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f99a006ae70 msgr2=0x7f99a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.941 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.939+0000 7f99c600e640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f99a006ae70 0x7f99a006d330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f99b0036000 tx=0x7f99b0030000 comp rx=0 tx=0).stop 2023-11-08T19:24:52.942 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.939+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 msgr2=0x7f99c00c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:52.942 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.939+0000 7f99c600e640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 0x7f99c00c5bd0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c9080 tx=0x7f99b800cb90 comp rx=0 tx=0).stop 2023-11-08T19:24:52.942 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.939+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 shutdown_connections 2023-11-08T19:24:52.942 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.939+0000 7f99c600e640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f99a006ae70 0x7f99a006d330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.942 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.940+0000 7f99c600e640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99c015ce80 0x7f99c00c5bd0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.943 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.940+0000 7f99c600e640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99c00c8690 0x7f99c015c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.943 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.940+0000 7f99c600e640 1 --2- 172.21.15.119:0/1972744942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f99ac0039a0 0x7f99c0140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:52.943 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.940+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 >> 172.21.15.119:0/1972744942 conn(0x7f99c00cb610 msgr2=0x7f99c01420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:52.943 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.940+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 shutdown_connections 2023-11-08T19:24:52.943 INFO:teuthology.misc.health.smithi119.stderr:2023-11-08T19:24:52.940+0000 7f99c600e640 1 -- 172.21.15.119:0/1972744942 wait complete. 2023-11-08T19:24:52.955 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-08T19:24:52.955 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-08T19:24:52.955 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:24:52.956 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:24:52.964 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:24:52.964 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:24:52.964 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:24:52.973 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:24:52.973 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:24:52.973 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:24:52.982 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:24:52.983 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:24:52.983 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:24:52.993 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:24:52.994 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-08T19:24:52.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:53.118 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:24:53.120 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:24:53.146 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:24:53.147 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:24:53.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.180+0000 7f5fde355640 1 Processor -- start 2023-11-08T19:24:53.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.181+0000 7f5fde355640 1 -- start start 2023-11-08T19:24:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.182+0000 7f5fde355640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd81447a0 0x7f5fd815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.182+0000 7f5fde355640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fd80cad30 0x7f5fd806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.182+0000 7f5fde355640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd80c3a50 con 0x7f5fd80cb170 2023-11-08T19:24:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.182+0000 7f5fde355640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd8144b80 con 0x7f5fd80cad30 2023-11-08T19:24:53.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.182+0000 7f5fde355640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80c9150 con 0x7f5fd81447a0 2023-11-08T19:24:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.182+0000 7f5fcffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fd80cad30 0x7f5fd806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fcffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5fd80cad30 0x7f5fd806f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:55356/0 (socket says 172.21.15.119:55356) 2023-11-08T19:24:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd81447a0 0x7f5fd815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fcffff640 1 -- 172.21.15.119:0/668529782 learned_addr learned my addr 172.21.15.119:0/668529782 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:53.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fd77fe640 1 -- 172.21.15.119:0/668529782 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523689486 0 0) 0x7f5fd80c3a50 con 0x7f5fd80cb170 2023-11-08T19:24:53.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fd77fe640 1 -- 172.21.15.119:0/668529782 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fc4003520 con 0x7f5fd80cb170 2023-11-08T19:24:53.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fcffff640 1 -- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd81447a0 msgr2=0x7f5fd815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fcffff640 1 --2- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd81447a0 0x7f5fd815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fcffff640 1 -- 172.21.15.119:0/668529782 >> v1:172.21.15.188:6789/0 conn(0x7f5fd80cb170 legacy=0x7f5fd806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fcffff640 1 -- 172.21.15.119:0/668529782 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80c9410 con 0x7f5fd80cad30 2023-11-08T19:24:53.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.183+0000 7f5fd7fff640 1 --2- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd81447a0 0x7f5fd815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:24:53.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.184+0000 7f5fcffff640 1 --2- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cad30 0x7f5fd806f930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009870 tx=0x7f5fc002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11eac446504d23e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.184+0000 7f5fd77fe640 1 -- 172.21.15.119:0/668529782 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0030840 con 0x7f5fd80cad30 2023-11-08T19:24:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fd77fe640 1 -- 172.21.15.119:0/668529782 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc00309a0 con 0x7f5fd80cad30 2023-11-08T19:24:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fd77fe640 1 -- 172.21.15.119:0/668529782 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0030d30 con 0x7f5fd80cad30 2023-11-08T19:24:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 -- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cad30 msgr2=0x7f5fd806f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 --2- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cad30 0x7f5fd806f930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009870 tx=0x7f5fc002b980 comp rx=0 tx=0).stop 2023-11-08T19:24:53.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 -- 172.21.15.119:0/668529782 shutdown_connections 2023-11-08T19:24:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 --2- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cad30 0x7f5fd806f930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 --2- 172.21.15.119:0/668529782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd81447a0 0x7f5fd815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 -- 172.21.15.119:0/668529782 >> 172.21.15.119:0/668529782 conn(0x7f5fd80cb610 msgr2=0x7f5fd80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:53.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 -- 172.21.15.119:0/668529782 shutdown_connections 2023-11-08T19:24:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.185+0000 7f5fde355640 1 -- 172.21.15.119:0/668529782 wait complete. 2023-11-08T19:24:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.187+0000 7f5fde355640 1 Processor -- start 2023-11-08T19:24:53.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.187+0000 7f5fde355640 1 -- start start 2023-11-08T19:24:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fde355640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd80cad30 0x7f5fd814fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fde355640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cb170 0x7f5fd814a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fde355640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 0x7f5fd81f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fde355640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd81f5000 con 0x7f5fd81447a0 2023-11-08T19:24:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fde355640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80c7e30 con 0x7f5fd80cb170 2023-11-08T19:24:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fde355640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd80c7820 con 0x7f5fd80cad30 2023-11-08T19:24:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fdc8f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 0x7f5fd81f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fdc8f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 0x7f5fd81f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44344/0 (socket says 172.21.15.119:44344) 2023-11-08T19:24:53.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fdc8f2640 1 -- 172.21.15.119:0/3049563254 learned_addr learned my addr 172.21.15.119:0/3049563254 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fcffff640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cb170 0x7f5fd814a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.188+0000 7f5fd7fff640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd80cad30 0x7f5fd814fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fdc8f2640 1 -- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd80cad30 msgr2=0x7f5fd814fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fdc8f2640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd80cad30 0x7f5fd814fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fdc8f2640 1 -- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cb170 msgr2=0x7f5fd814a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fdc8f2640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cb170 0x7f5fd814a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fdc8f2640 1 -- 172.21.15.119:0/3049563254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc8009400 con 0x7f5fd81447a0 2023-11-08T19:24:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fdc8f2640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 0x7f5fd81f4060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c7a0 tx=0x7f5fd000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd0003b70 con 0x7f5fd81447a0 2023-11-08T19:24:53.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd0003cd0 con 0x7f5fd81447a0 2023-11-08T19:24:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc00094d0 con 0x7f5fd81447a0 2023-11-08T19:24:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.189+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd000aa90 con 0x7f5fd81447a0 2023-11-08T19:24:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.190+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd81f4a40 con 0x7f5fd81447a0 2023-11-08T19:24:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.191+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd80c7fc0 con 0x7f5fd81447a0 2023-11-08T19:24:53.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.191+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fd000abf0 con 0x7f5fd81447a0 2023-11-08T19:24:53.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.192+0000 7f5fd57fa640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fb806ae70 0x7f5fb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.192+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5fd0088fe0 con 0x7f5fd81447a0 2023-11-08T19:24:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.193+0000 7f5fd7fff640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fb806ae70 0x7f5fb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.193+0000 7f5fd7fff640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fb806ae70 0x7f5fb806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009c30 tx=0x7f5fc8009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.197+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fd00544c0 con 0x7f5fd81447a0 2023-11-08T19:24:53.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.382+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5fd8074430 con 0x7f5fd81447a0 2023-11-08T19:24:53.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.384+0000 7f5fd57fa640 1 -- 172.21.15.119:0/3049563254 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f5fd0057ae0 con 0x7f5fd81447a0 2023-11-08T19:24:53.386 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:24:53.387 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:24:49.581137+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:53.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fb806ae70 msgr2=0x7f5fb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fb806ae70 0x7f5fb806d330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8009c30 tx=0x7f5fc8009190 comp rx=0 tx=0).stop 2023-11-08T19:24:53.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 msgr2=0x7f5fd81f4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 0x7f5fd81f4060 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5fd000c7a0 tx=0x7f5fd000cc60 comp rx=0 tx=0).stop 2023-11-08T19:24:53.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 shutdown_connections 2023-11-08T19:24:53.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fb806ae70 0x7f5fb806d330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5fd81447a0 0x7f5fd81f4060 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5fd80cb170 0x7f5fd814a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 --2- 172.21.15.119:0/3049563254 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5fd80cad30 0x7f5fd814fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 >> 172.21.15.119:0/3049563254 conn(0x7f5fd80cb610 msgr2=0x7f5fd8140580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:53.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 shutdown_connections 2023-11-08T19:24:53.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.387+0000 7f5fde355640 1 -- 172.21.15.119:0/3049563254 wait complete. 2023-11-08T19:24:53.406 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:24:53.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:24:53.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.605+0000 7fd86bd0c640 1 Processor -- start 2023-11-08T19:24:53.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.606+0000 7fd86bd0c640 1 -- start start 2023-11-08T19:24:53.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.607+0000 7fd86bd0c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd864144960 0x7fd86415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.607+0000 7fd86bd0c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd86415ea60 con 0x7fd86415f1d0 2023-11-08T19:24:53.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.607+0000 7fd86bd0c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8640c9360 con 0x7fd864144580 2023-11-08T19:24:53.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.608+0000 7fd86bd0c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8640c9150 con 0x7fd864144960 2023-11-08T19:24:53.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.608+0000 7fd86a2a9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd86415f1d0 0x7fd8640c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57138/0 (socket says 172.21.15.119:57138) 2023-11-08T19:24:53.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.608+0000 7fd86a2a9640 1 -- 172.21.15.119:0/4160999811 learned_addr learned my addr 172.21.15.119:0/4160999811 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:53.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.608+0000 7fd8692a7640 1 --2- 172.21.15.119:0/4160999811 >> v2:172.21.15.188:3300/0 conn(0x7fd864144960 0x7fd86415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.608+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745935573 0 0) 0x7fd86415ea60 con 0x7fd86415f1d0 2023-11-08T19:24:53.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.608+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd850003520 con 0x7fd86415f1d0 2023-11-08T19:24:53.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3619880559 0 0) 0x7fd850003520 con 0x7fd86415f1d0 2023-11-08T19:24:53.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 >> v2:172.21.15.188:3300/0 conn(0x7fd864144960 msgr2=0x7fd86415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 --2- 172.21.15.119:0/4160999811 >> v2:172.21.15.188:3300/0 conn(0x7fd864144960 0x7fd86415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 >> v1:172.21.15.188:6789/0 conn(0x7fd864144580 legacy=0x7fd8640cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640c9640 con 0x7fd86415f1d0 2023-11-08T19:24:53.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd85c0019f0 con 0x7fd86415f1d0 2023-11-08T19:24:53.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 >> v1:172.21.15.119:6789/0 conn(0x7fd86415f1d0 legacy=0x7fd8640c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd850003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8500047c0 0x7fd850004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8500051d0 0x7fd85000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c9150 con 0x7fd8500039c0 2023-11-08T19:24:53.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85000c120 con 0x7fd8500047c0 2023-11-08T19:24:53.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85000c2e0 con 0x7fd8500051d0 2023-11-08T19:24:53.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd869aa8640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd850003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd8692a7640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8500047c0 0x7fd850004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.609+0000 7fd86a2a9640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8500051d0 0x7fd85000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.610+0000 7fd869aa8640 1 -- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8500051d0 msgr2=0x7fd85000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.610+0000 7fd869aa8640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8500051d0 0x7fd85000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.610+0000 7fd869aa8640 1 -- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8500047c0 msgr2=0x7fd850004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:53.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.610+0000 7fd869aa8640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8500047c0 0x7fd850004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.610+0000 7fd869aa8640 1 -- 172.21.15.119:0/4160999811 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850003520 con 0x7fd8500039c0 2023-11-08T19:24:53.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.610+0000 7fd86a2a9640 1 -- 172.21.15.119:0/4160999811 reap_dead start 2023-11-08T19:24:53.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd869aa8640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd850003de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd860001720 tx=0x7fd8500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd1ce7ca75274fb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8600052d0 con 0x7fd8500039c0 2023-11-08T19:24:53.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86002e050 con 0x7fd8500039c0 2023-11-08T19:24:53.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd868aa6640 1 -- 172.21.15.119:0/4160999811 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860003830 con 0x7fd8500039c0 2023-11-08T19:24:53.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd86bd0c640 1 -- 172.21.15.119:0/4160999811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 msgr2=0x7fd850003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd850003de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd860001720 tx=0x7fd8500057f0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd86bd0c640 1 -- 172.21.15.119:0/4160999811 shutdown_connections 2023-11-08T19:24:53.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/4160999811 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd850003de0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.611+0000 7fd86bd0c640 1 -- 172.21.15.119:0/4160999811 >> 172.21.15.119:0/4160999811 conn(0x7fd8640cb610 msgr2=0x7fd8640cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:53.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.612+0000 7fd86bd0c640 1 -- 172.21.15.119:0/4160999811 shutdown_connections 2023-11-08T19:24:53.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.612+0000 7fd86bd0c640 1 -- 172.21.15.119:0/4160999811 wait complete. 2023-11-08T19:24:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.613+0000 7fd86bd0c640 1 Processor -- start 2023-11-08T19:24:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.613+0000 7fd86bd0c640 1 -- start start 2023-11-08T19:24:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd86bd0c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd8641429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd86bd0c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd864143010 0x7fd8640c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd86bd0c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8640c85e0 0x7fd8640aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd86bd0c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c8a70 con 0x7fd8500039c0 2023-11-08T19:24:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd86bd0c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640adbb0 con 0x7fd8640c85e0 2023-11-08T19:24:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd86bd0c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c6680 con 0x7fd864143010 2023-11-08T19:24:53.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.614+0000 7fd869aa8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd8641429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd8641429f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44372/0 (socket says 172.21.15.119:44372) 2023-11-08T19:24:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 -- 172.21.15.119:0/3912084016 learned_addr learned my addr 172.21.15.119:0/3912084016 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd8692a7640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd864143010 0x7fd8640c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 -- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd864143010 msgr2=0x7fd8640c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd864143010 0x7fd8640c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd86a2a9640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8640c85e0 0x7fd8640aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 -- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8640c85e0 msgr2=0x7fd8640aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8640c85e0 0x7fd8640aecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 -- 172.21.15.119:0/3912084016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640c6960 con 0x7fd8500039c0 2023-11-08T19:24:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd8692a7640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd864143010 0x7fd8640c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd86a2a9640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8640c85e0 0x7fd8640aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.615+0000 7fd869aa8640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd8641429f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd860005040 tx=0x7fd860001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.616+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8600076c0 con 0x7fd8500039c0 2023-11-08T19:24:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.616+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8600069e0 con 0x7fd8500039c0 2023-11-08T19:24:53.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.616+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8640c6c40 con 0x7fd8500039c0 2023-11-08T19:24:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.616+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860006d90 con 0x7fd8500039c0 2023-11-08T19:24:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.616+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8640af210 con 0x7fd8500039c0 2023-11-08T19:24:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.617+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8640c67c0 con 0x7fd8500039c0 2023-11-08T19:24:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.618+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8600035f0 con 0x7fd8500039c0 2023-11-08T19:24:53.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.619+0000 7fd85a7fc640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd84806aea0 0x7fd84806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:53.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.619+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd8600ab130 con 0x7fd8500039c0 2023-11-08T19:24:53.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.619+0000 7fd8692a7640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd84806aea0 0x7fd84806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:53.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.620+0000 7fd8692a7640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd84806aea0 0x7fd84806d360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd864144d70 tx=0x7fd854008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:53.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.623+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8640c67c0 con 0x7fd8500039c0 2023-11-08T19:24:53.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.814+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8640c67c0 con 0x7fd8500039c0 2023-11-08T19:24:53.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.816+0000 7fd85a7fc640 1 -- 172.21.15.119:0/3912084016 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd86004b020 con 0x7fd8500039c0 2023-11-08T19:24:53.817 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:24:53.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd84806aea0 msgr2=0x7fd84806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd84806aea0 0x7fd84806d360 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd864144d70 tx=0x7fd854008040 comp rx=0 tx=0).stop 2023-11-08T19:24:53.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 msgr2=0x7fd8641429f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:53.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd8641429f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd860005040 tx=0x7fd860001120 comp rx=0 tx=0).stop 2023-11-08T19:24:53.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 shutdown_connections 2023-11-08T19:24:53.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd84806aea0 0x7fd84806d360 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8640c85e0 0x7fd8640aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd864143010 0x7fd8640c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 --2- 172.21.15.119:0/3912084016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd8500039c0 0x7fd8641429f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:53.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.818+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 >> 172.21.15.119:0/3912084016 conn(0x7fd8640cb610 msgr2=0x7fd8640cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:53.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.819+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 shutdown_connections 2023-11-08T19:24:53.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:53.819+0000 7fd86bd0c640 1 -- 172.21.15.119:0/3912084016 wait complete. 2023-11-08T19:24:53.839 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:24:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.047+0000 7f820938c640 1 Processor -- start 2023-11-08T19:24:54.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.047+0000 7f820938c640 1 -- start start 2023-11-08T19:24:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f820938c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 0x7f82040ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f820938c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f820415d4c0 0x7f82040c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f820938c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82040c7100 con 0x7f820415d0e0 2023-11-08T19:24:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f820938c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82040d04f0 con 0x7f820415d4c0 2023-11-08T19:24:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f820938c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82040d07d0 con 0x7f82040d1d50 2023-11-08T19:24:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f8203fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f820415d0e0 0x7f82040cda70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57142/0 (socket says 172.21.15.119:57142) 2023-11-08T19:24:54.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f8203fff640 1 -- 172.21.15.119:0/222229938 learned_addr learned my addr 172.21.15.119:0/222229938 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:54.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.049+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349564718 0 0) 0x7f82040c7100 con 0x7f820415d0e0 2023-11-08T19:24:54.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f81fb7fe640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 0x7f82040ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f8208b8b640 1 --2- 172.21.15.119:0/222229938 >> v2:172.21.15.188:3300/0 conn(0x7f820415d4c0 0x7f82040c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f0003520 con 0x7f820415d0e0 2023-11-08T19:24:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4153827889 0 0) 0x7f81f0003520 con 0x7f820415d0e0 2023-11-08T19:24:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 msgr2=0x7f82040ce180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 0x7f82040ce180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 >> v2:172.21.15.188:3300/0 conn(0x7f820415d4c0 msgr2=0x7f82040c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 --2- 172.21.15.119:0/222229938 >> v2:172.21.15.188:3300/0 conn(0x7f820415d4c0 0x7f82040c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82040c6ab0 con 0x7f820415d0e0 2023-11-08T19:24:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f81f4001670 con 0x7f820415d0e0 2023-11-08T19:24:54.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 >> v1:172.21.15.119:6789/0 conn(0x7f820415d0e0 legacy=0x7f82040cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81f0003970 0x7f81f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 0x7f81f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0004d00 0x7f81f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040d04f0 con 0x7f81f0003970 2023-11-08T19:24:54.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82040d07d0 con 0x7f81f0004d00 2023-11-08T19:24:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f000b980 con 0x7f82040d1d50 2023-11-08T19:24:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.050+0000 7f8203fff640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81f0003970 0x7f81f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.051+0000 7f8203fff640 1 -- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 msgr2=0x7f81f00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.051+0000 7f8203fff640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040d1d50 0x7f81f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.051+0000 7f8203fff640 1 -- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0004d00 msgr2=0x7f81f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.051+0000 7f8208b8b640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0004d00 0x7f81f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.051+0000 7f8203fff640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0004d00 0x7f81f0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.051+0000 7f8203fff640 1 -- 172.21.15.119:0/222229938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0003520 con 0x7f81f0003970 2023-11-08T19:24:54.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.052+0000 7f8203fff640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81f0003970 0x7f81f0003d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c290 tx=0x7f81f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76a612af47be0c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.052+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4005ad0 con 0x7f81f0003970 2023-11-08T19:24:54.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.052+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f4005c30 con 0x7f81f0003970 2023-11-08T19:24:54.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.052+0000 7f82037fe640 1 -- 172.21.15.119:0/222229938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f402c8d0 con 0x7f81f0003970 2023-11-08T19:24:54.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f820938c640 1 -- 172.21.15.119:0/222229938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81f0003970 msgr2=0x7f81f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f820938c640 1 --2- 172.21.15.119:0/222229938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81f0003970 0x7f81f0003d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f81f400c290 tx=0x7f81f00057f0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f8208b8b640 1 -- 172.21.15.119:0/222229938 reap_dead start 2023-11-08T19:24:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f820938c640 1 -- 172.21.15.119:0/222229938 shutdown_connections 2023-11-08T19:24:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f820938c640 1 -- 172.21.15.119:0/222229938 >> 172.21.15.119:0/222229938 conn(0x7f82040d2f60 msgr2=0x7f82040d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f820938c640 1 -- 172.21.15.119:0/222229938 shutdown_connections 2023-11-08T19:24:54.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.053+0000 7f820938c640 1 -- 172.21.15.119:0/222229938 wait complete. 2023-11-08T19:24:54.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.055+0000 7f820938c640 1 Processor -- start 2023-11-08T19:24:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.055+0000 7f820938c640 1 -- start start 2023-11-08T19:24:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f820938c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0003970 0x7f82040b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f820938c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 0x7f82041589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f820938c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f820415fff0 0x7f8204151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f820938c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820415e510 con 0x7f82040b5b80 2023-11-08T19:24:54.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f820938c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820415dec0 con 0x7f81f0003970 2023-11-08T19:24:54.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f820938c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82040b8620 con 0x7f820415fff0 2023-11-08T19:24:54.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 0x7f82041589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 0x7f82041589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44390/0 (socket says 172.21.15.119:44390) 2023-11-08T19:24:54.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 -- 172.21.15.119:0/2846270002 learned_addr learned my addr 172.21.15.119:0/2846270002 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:54.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f8203fff640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0003970 0x7f82040b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f8208b8b640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f820415fff0 0x7f8204151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 -- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f820415fff0 msgr2=0x7f8204151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f820415fff0 0x7f8204151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 -- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0003970 msgr2=0x7f82040b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0003970 0x7f82040b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f81fb7fe640 1 -- 172.21.15.119:0/2846270002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820415e130 con 0x7f82040b5b80 2023-11-08T19:24:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f8203fff640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0003970 0x7f82040b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.056+0000 7f8208b8b640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f820415fff0 0x7f8204151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:54.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.057+0000 7f81fb7fe640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 0x7f82041589c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00e7f0 tx=0x7f81ec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.057+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00c940 con 0x7f82040b5b80 2023-11-08T19:24:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.057+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec00caa0 con 0x7f82040b5b80 2023-11-08T19:24:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.057+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00ce30 con 0x7f82040b5b80 2023-11-08T19:24:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.057+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820415db80 con 0x7f82040b5b80 2023-11-08T19:24:54.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.057+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8204158f00 con 0x7f82040b5b80 2023-11-08T19:24:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.058+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820415e650 con 0x7f82040b5b80 2023-11-08T19:24:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.059+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81ec013470 con 0x7f82040b5b80 2023-11-08T19:24:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.060+0000 7f82017fa640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406ae70 0x7f81e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.060+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f81ec089340 con 0x7f82040b5b80 2023-11-08T19:24:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.060+0000 7f8203fff640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406ae70 0x7f81e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.061+0000 7f8203fff640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406ae70 0x7f81e406d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81f4034000 tx=0x7f81f4007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.064+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f820415e650 con 0x7f82040b5b80 2023-11-08T19:24:54.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.297+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f820415e650 con 0x7f82040b5b80 2023-11-08T19:24:54.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.298+0000 7f82017fa640 1 -- 172.21.15.119:0/2846270002 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f81ec021e90 con 0x7f82040b5b80 2023-11-08T19:24:54.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:24:54.300 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-08T19:24:54.301 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 3 2023-11-08T19:24:54.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406ae70 msgr2=0x7f81e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406ae70 0x7f81e406d330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81f4034000 tx=0x7f81f4007000 comp rx=0 tx=0).stop 2023-11-08T19:24:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 msgr2=0x7f82041589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 0x7f82041589c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00e7f0 tx=0x7f81ec00ecb0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 shutdown_connections 2023-11-08T19:24:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406ae70 0x7f81e406d330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f820415fff0 0x7f8204151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82040b5b80 0x7f82041589c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 --2- 172.21.15.119:0/2846270002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81f0003970 0x7f82040b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.301+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 >> 172.21.15.119:0/2846270002 conn(0x7f82040d2f60 msgr2=0x7f82040c85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.302+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 shutdown_connections 2023-11-08T19:24:54.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.302+0000 7f820938c640 1 -- 172.21.15.119:0/2846270002 wait complete. 2023-11-08T19:24:54.321 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:24:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.529+0000 7f3db919e640 1 Processor -- start 2023-11-08T19:24:54.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.529+0000 7f3db919e640 1 -- start start 2023-11-08T19:24:54.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db919e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3db40cfa00 0x7f3db40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db919e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db4161160 0x7f3db41615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db919e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db415df70 con 0x7f3db415d6c0 2023-11-08T19:24:54.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db919e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db415daa0 con 0x7f3db40cfa00 2023-11-08T19:24:54.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db919e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40d03b0 con 0x7f3db4161160 2023-11-08T19:24:54.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db2d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3db415d6c0 0x7f3db40d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57154/0 (socket says 172.21.15.119:57154) 2023-11-08T19:24:54.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3db2d9d640 1 -- 172.21.15.119:0/3885528206 learned_addr learned my addr 172.21.15.119:0/3885528206 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:54.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.531+0000 7f3dabfff640 1 --2- 172.21.15.119:0/3885528206 >> v2:172.21.15.119:3300/0 conn(0x7f3db40cfa00 0x7f3db40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3db359e640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db4161160 0x7f3db41615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3dabfff640 1 -- 172.21.15.119:0/3885528206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db4161160 msgr2=0x7f3db41615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3dabfff640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db4161160 0x7f3db41615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3dabfff640 1 -- 172.21.15.119:0/3885528206 >> v1:172.21.15.119:6789/0 conn(0x7f3db415d6c0 legacy=0x7f3db40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3dabfff640 1 -- 172.21.15.119:0/3885528206 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db4161b10 con 0x7f3db40cfa00 2023-11-08T19:24:54.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3db359e640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db4161160 0x7f3db41615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:54.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.532+0000 7f3db259c640 1 -- 172.21.15.119:0/3885528206 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3530753278 0 0) 0x7f3db415df70 con 0x7f3db415d6c0 2023-11-08T19:24:54.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.533+0000 7f3dabfff640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db40cfa00 0x7f3db40cfe70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0097a0 tx=0x7f3d9c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d29ba3248591042 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.533+0000 7f3db259c640 1 -- 172.21.15.119:0/3885528206 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c031780 con 0x7f3db40cfa00 2023-11-08T19:24:54.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.533+0000 7f3db259c640 1 -- 172.21.15.119:0/3885528206 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d9c0318e0 con 0x7f3db40cfa00 2023-11-08T19:24:54.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.533+0000 7f3db259c640 1 -- 172.21.15.119:0/3885528206 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c031c70 con 0x7f3db40cfa00 2023-11-08T19:24:54.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 -- 172.21.15.119:0/3885528206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db40cfa00 msgr2=0x7f3db40cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db40cfa00 0x7f3db40cfe70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0097a0 tx=0x7f3d9c02b820 comp rx=0 tx=0).stop 2023-11-08T19:24:54.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 -- 172.21.15.119:0/3885528206 shutdown_connections 2023-11-08T19:24:54.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db4161160 0x7f3db41615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 --2- 172.21.15.119:0/3885528206 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db40cfa00 0x7f3db40cfe70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 -- 172.21.15.119:0/3885528206 >> 172.21.15.119:0/3885528206 conn(0x7f3db406a9f0 msgr2=0x7f3db406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 -- 172.21.15.119:0/3885528206 shutdown_connections 2023-11-08T19:24:54.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.534+0000 7f3db919e640 1 -- 172.21.15.119:0/3885528206 wait complete. 2023-11-08T19:24:54.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.536+0000 7f3db919e640 1 Processor -- start 2023-11-08T19:24:54.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.536+0000 7f3db919e640 1 -- start start 2023-11-08T19:24:54.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.537+0000 7f3db919e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3db40cfa00 0x7f3db4155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.537+0000 7f3db919e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db415d6c0 0x7f3db4156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.537+0000 7f3db919e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 0x7f3db4152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.537+0000 7f3db919e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4156ab0 con 0x7f3db4161160 2023-11-08T19:24:54.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.537+0000 7f3db919e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40c9850 con 0x7f3db40cfa00 2023-11-08T19:24:54.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.537+0000 7f3db919e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4156ca0 con 0x7f3db415d6c0 2023-11-08T19:24:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 0x7f3db4152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 0x7f3db4152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44414/0 (socket says 172.21.15.119:44414) 2023-11-08T19:24:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 -- 172.21.15.119:0/889154718 learned_addr learned my addr 172.21.15.119:0/889154718 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3dabfff640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db415d6c0 0x7f3db4156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db2d9d640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3db40cfa00 0x7f3db4155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 -- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db415d6c0 msgr2=0x7f3db4156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db415d6c0 0x7f3db4156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 -- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3db40cfa00 msgr2=0x7f3db4155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3db40cfa00 0x7f3db4155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.538+0000 7f3db359e640 1 -- 172.21.15.119:0/889154718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9c009400 con 0x7f3db4161160 2023-11-08T19:24:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.539+0000 7f3db359e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 0x7f3db4152340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00cd50 tx=0x7f3dac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.539+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00f680 con 0x7f3db4161160 2023-11-08T19:24:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.539+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac00f800 con 0x7f3db4161160 2023-11-08T19:24:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.539+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00fbb0 con 0x7f3db4161160 2023-11-08T19:24:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.539+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db4156f80 con 0x7f3db4161160 2023-11-08T19:24:54.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.539+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db4157540 con 0x7f3db4161160 2023-11-08T19:24:54.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.540+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db4074430 con 0x7f3db4161160 2023-11-08T19:24:54.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.541+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3dac00b040 con 0x7f3db4161160 2023-11-08T19:24:54.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.542+0000 7f3dab7fe640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3d9406ae70 0x7f3d9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.542+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3dac089ce0 con 0x7f3db4161160 2023-11-08T19:24:54.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.542+0000 7f3db2d9d640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3d9406ae70 0x7f3d9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.543+0000 7f3db2d9d640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3d9406ae70 0x7f3d9406d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3da4007fd0 tx=0x7f3da4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.547+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dac0550c0 con 0x7f3db4161160 2023-11-08T19:24:54.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.742+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3db40d10f0 con 0x7f3db4161160 2023-11-08T19:24:54.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.744+0000 7f3dab7fe640 1 -- 172.21.15.119:0/889154718 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12850 (secure 0 0 0) 0x7f3dac01d090 con 0x7f3db4161160 2023-11-08T19:24:54.746 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:24:54.746 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":16,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:24:49.581137+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:24:54.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3d9406ae70 msgr2=0x7f3d9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3d9406ae70 0x7f3d9406d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3da4007fd0 tx=0x7f3da4009040 comp rx=0 tx=0).stop 2023-11-08T19:24:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 msgr2=0x7f3db4152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 0x7f3db4152340 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00cd50 tx=0x7f3dac00a650 comp rx=0 tx=0).stop 2023-11-08T19:24:54.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 shutdown_connections 2023-11-08T19:24:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3d9406ae70 0x7f3d9406d330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3db4161160 0x7f3db4152340 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3db415d6c0 0x7f3db4156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 --2- 172.21.15.119:0/889154718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3db40cfa00 0x7f3db4155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 >> 172.21.15.119:0/889154718 conn(0x7f3db406a9f0 msgr2=0x7f3db415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 shutdown_connections 2023-11-08T19:24:54.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.747+0000 7f3db919e640 1 -- 172.21.15.119:0/889154718 wait complete. 2023-11-08T19:24:54.766 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:24:54.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:24:54.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.976+0000 7fde669a9640 1 Processor -- start 2023-11-08T19:24:54.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.977+0000 7fde669a9640 1 -- start start 2023-11-08T19:24:54.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde669a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fde60143600 0x7fde600c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde669a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6015d9f0 0x7fde6015de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde669a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde6006cb40 0x7fde6006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde669a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde600c4f50 con 0x7fde6006cb40 2023-11-08T19:24:54.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde669a9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde6006d530 con 0x7fde60143600 2023-11-08T19:24:54.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde669a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c4960 con 0x7fde6015d9f0 2023-11-08T19:24:54.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde64f46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde6006cb40 0x7fde6006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde64f46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde6006cb40 0x7fde6006cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44418/0 (socket says 172.21.15.119:44418) 2023-11-08T19:24:54.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.978+0000 7fde64f46640 1 -- 172.21.15.119:0/978078103 learned_addr learned my addr 172.21.15.119:0/978078103 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:54.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde5f7fe640 1 --2- 172.21.15.119:0/978078103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6015d9f0 0x7fde6015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde64f46640 1 -- 172.21.15.119:0/978078103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6015d9f0 msgr2=0x7fde6015de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde5ffff640 1 --2- 172.21.15.119:0/978078103 >> v2:172.21.15.188:3300/0 conn(0x7fde60143600 0x7fde600c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde64f46640 1 --2- 172.21.15.119:0/978078103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6015d9f0 0x7fde6015de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde64f46640 1 -- 172.21.15.119:0/978078103 >> v2:172.21.15.188:3300/0 conn(0x7fde60143600 msgr2=0x7fde600c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:54.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde64f46640 1 --2- 172.21.15.119:0/978078103 >> v2:172.21.15.188:3300/0 conn(0x7fde60143600 0x7fde600c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.979+0000 7fde64f46640 1 -- 172.21.15.119:0/978078103 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6015e440 con 0x7fde6006cb40 2023-11-08T19:24:54.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.980+0000 7fde64f46640 1 --2- 172.21.15.119:0/978078103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6006cb40 0x7fde6006cff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde600c3f30 tx=0x7fde5802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510db431f12d0631 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.980+0000 7fde5effd640 1 -- 172.21.15.119:0/978078103 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580317c0 con 0x7fde6006cb40 2023-11-08T19:24:54.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.980+0000 7fde5effd640 1 -- 172.21.15.119:0/978078103 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde58031920 con 0x7fde6006cb40 2023-11-08T19:24:54.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.980+0000 7fde5effd640 1 -- 172.21.15.119:0/978078103 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde58031cb0 con 0x7fde6006cb40 2023-11-08T19:24:54.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.980+0000 7fde669a9640 1 -- 172.21.15.119:0/978078103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6006cb40 msgr2=0x7fde6006cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.980+0000 7fde669a9640 1 --2- 172.21.15.119:0/978078103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6006cb40 0x7fde6006cff0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fde600c3f30 tx=0x7fde5802b860 comp rx=0 tx=0).stop 2023-11-08T19:24:54.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 -- 172.21.15.119:0/978078103 shutdown_connections 2023-11-08T19:24:54.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 --2- 172.21.15.119:0/978078103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde6006cb40 0x7fde6006cff0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 --2- 172.21.15.119:0/978078103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6015d9f0 0x7fde6015de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 --2- 172.21.15.119:0/978078103 >> v2:172.21.15.188:3300/0 conn(0x7fde60143600 0x7fde600c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 -- 172.21.15.119:0/978078103 >> 172.21.15.119:0/978078103 conn(0x7fde600cb610 msgr2=0x7fde600cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 -- 172.21.15.119:0/978078103 shutdown_connections 2023-11-08T19:24:54.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.981+0000 7fde669a9640 1 -- 172.21.15.119:0/978078103 wait complete. 2023-11-08T19:24:54.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.982+0000 7fde669a9640 1 Processor -- start 2023-11-08T19:24:54.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.983+0000 7fde669a9640 1 -- start start 2023-11-08T19:24:54.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.983+0000 7fde669a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6006cb40 0x7fde6015c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.983+0000 7fde669a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 0x7fde6015f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde669a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde6015d9f0 0x7fde6006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde669a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6015cb00 con 0x7fde60143600 2023-11-08T19:24:54.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde669a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde6015ff60 con 0x7fde6015d9f0 2023-11-08T19:24:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde669a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde60160240 con 0x7fde6006cb40 2023-11-08T19:24:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 0x7fde6015f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 0x7fde6015f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44434/0 (socket says 172.21.15.119:44434) 2023-11-08T19:24:54.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 -- 172.21.15.119:0/1846802957 learned_addr learned my addr 172.21.15.119:0/1846802957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 -- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6006cb40 msgr2=0x7fde6015c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5ffff640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6006cb40 0x7fde6015c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde64f46640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde6015d9f0 0x7fde6006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6006cb40 0x7fde6015c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 -- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde6015d9f0 msgr2=0x7fde6006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:54.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde6015d9f0 0x7fde6006ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.984+0000 7fde5f7fe640 1 -- 172.21.15.119:0/1846802957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde580094d0 con 0x7fde60143600 2023-11-08T19:24:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.985+0000 7fde5f7fe640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 0x7fde6015f510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c7f0 tx=0x7fde4800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.985+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48004230 con 0x7fde60143600 2023-11-08T19:24:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.985+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde48004390 con 0x7fde60143600 2023-11-08T19:24:54.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.985+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48007860 con 0x7fde60143600 2023-11-08T19:24:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.985+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde60160520 con 0x7fde60143600 2023-11-08T19:24:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.985+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde60161e80 con 0x7fde60143600 2023-11-08T19:24:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.986+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde60162130 con 0x7fde60143600 2023-11-08T19:24:54.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.987+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde48007bd0 con 0x7fde60143600 2023-11-08T19:24:54.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.988+0000 7fde5cff9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde4006aea0 0x7fde4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:54.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.989+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fde48089320 con 0x7fde60143600 2023-11-08T19:24:54.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.989+0000 7fde5ffff640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde4006aea0 0x7fde4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:54.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.989+0000 7fde5ffff640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde4006aea0 0x7fde4006d360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde50008e70 tx=0x7fde500083c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:54.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:54.993+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde60162130 con 0x7fde60143600 2023-11-08T19:24:55.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.179+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde60162130 con 0x7fde60143600 2023-11-08T19:24:55.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.180+0000 7fde5cff9640 1 -- 172.21.15.119:0/1846802957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde4801e020 con 0x7fde60143600 2023-11-08T19:24:55.182 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:24:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde4006aea0 msgr2=0x7fde4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde4006aea0 0x7fde4006d360 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde50008e70 tx=0x7fde500083c0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 msgr2=0x7fde6015f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 0x7fde6015f510 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fde4800c7f0 tx=0x7fde4800ccb0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 shutdown_connections 2023-11-08T19:24:55.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde4006aea0 0x7fde4006d360 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde6015d9f0 0x7fde6006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde60143600 0x7fde6015f510 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 --2- 172.21.15.119:0/1846802957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde6006cb40 0x7fde6015c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 >> 172.21.15.119:0/1846802957 conn(0x7fde600cb610 msgr2=0x7fde600cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 shutdown_connections 2023-11-08T19:24:55.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.183+0000 7fde669a9640 1 -- 172.21.15.119:0/1846802957 wait complete. 2023-11-08T19:24:55.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:24:55.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.415+0000 7f412324b640 1 Processor -- start 2023-11-08T19:24:55.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.415+0000 7f412324b640 1 -- start start 2023-11-08T19:24:55.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.416+0000 7f412324b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f411c15f4c0 0x7f411c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f412324b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f411c06a120 0x7f411c06a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f412324b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f411c06aa70 con 0x7f411c1447a0 2023-11-08T19:24:55.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f412324b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c15f930 con 0x7f411c06a120 2023-11-08T19:24:55.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f412324b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f411c0c3a50 con 0x7f411c15f4c0 2023-11-08T19:24:55.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f4120fe7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f411c1447a0 0x7f411c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57162/0 (socket says 172.21.15.119:57162) 2023-11-08T19:24:55.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f4120fe7640 1 -- 172.21.15.119:0/4027171994 learned_addr learned my addr 172.21.15.119:0/4027171994 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:55.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f41217e8640 1 --2- 172.21.15.119:0/4027171994 >> v2:172.21.15.119:3300/0 conn(0x7f411c06a120 0x7f411c06a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f411bfff640 1 --2- 172.21.15.119:0/4027171994 >> v2:172.21.15.188:3300/0 conn(0x7f411c15f4c0 0x7f411c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f41217e8640 1 -- 172.21.15.119:0/4027171994 >> v2:172.21.15.188:3300/0 conn(0x7f411c15f4c0 msgr2=0x7f411c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f41217e8640 1 --2- 172.21.15.119:0/4027171994 >> v2:172.21.15.188:3300/0 conn(0x7f411c15f4c0 0x7f411c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f41217e8640 1 -- 172.21.15.119:0/4027171994 >> v1:172.21.15.119:6789/0 conn(0x7f411c1447a0 legacy=0x7f411c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f411b7fe640 1 -- 172.21.15.119:0/4027171994 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226549839 0 0) 0x7f411c06aa70 con 0x7f411c1447a0 2023-11-08T19:24:55.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.417+0000 7f41217e8640 1 -- 172.21.15.119:0/4027171994 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f411c144b80 con 0x7f411c06a120 2023-11-08T19:24:55.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.418+0000 7f41217e8640 1 --2- 172.21.15.119:0/4027171994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c06a120 0x7f411c06a530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f411c06df20 tx=0x7f411402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b1742c27bfbaea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f411b7fe640 1 -- 172.21.15.119:0/4027171994 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f411402cc70 con 0x7f411c06a120 2023-11-08T19:24:55.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f411b7fe640 1 -- 172.21.15.119:0/4027171994 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f411402cdd0 con 0x7f411c06a120 2023-11-08T19:24:55.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f411b7fe640 1 -- 172.21.15.119:0/4027171994 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114034660 con 0x7f411c06a120 2023-11-08T19:24:55.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 -- 172.21.15.119:0/4027171994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c06a120 msgr2=0x7f411c06a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 --2- 172.21.15.119:0/4027171994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c06a120 0x7f411c06a530 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f411c06df20 tx=0x7f411402bec0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 -- 172.21.15.119:0/4027171994 shutdown_connections 2023-11-08T19:24:55.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 --2- 172.21.15.119:0/4027171994 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c06a120 0x7f411c06a530 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 --2- 172.21.15.119:0/4027171994 >> v2:172.21.15.188:3300/0 conn(0x7f411c15f4c0 0x7f411c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 -- 172.21.15.119:0/4027171994 >> 172.21.15.119:0/4027171994 conn(0x7f411c0cb610 msgr2=0x7f411c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 -- 172.21.15.119:0/4027171994 shutdown_connections 2023-11-08T19:24:55.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.419+0000 7f412324b640 1 -- 172.21.15.119:0/4027171994 wait complete. 2023-11-08T19:24:55.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.421+0000 7f412324b640 1 Processor -- start 2023-11-08T19:24:55.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.421+0000 7f412324b640 1 -- start start 2023-11-08T19:24:55.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f412324b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f411c06a120 0x7f411c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f412324b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f411c1447a0 0x7f411c13d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f412324b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 0x7f411c15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f412324b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0cd680 con 0x7f411c15f4c0 2023-11-08T19:24:55.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f412324b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0cd960 con 0x7f411c06a120 2023-11-08T19:24:55.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f412324b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f411c0cdc40 con 0x7f411c1447a0 2023-11-08T19:24:55.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f41217e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 0x7f411c15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.422+0000 7f41217e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 0x7f411c15c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44444/0 (socket says 172.21.15.119:44444) 2023-11-08T19:24:55.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 -- 172.21.15.119:0/2086278792 learned_addr learned my addr 172.21.15.119:0/2086278792 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:55.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f411bfff640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f411c1447a0 0x7f411c13d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f4120fe7640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f411c06a120 0x7f411c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 -- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f411c1447a0 msgr2=0x7f411c13d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f411c1447a0 0x7f411c13d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 -- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f411c06a120 msgr2=0x7f411c0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f411c06a120 0x7f411c0b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 -- 172.21.15.119:0/2086278792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4114009a70 con 0x7f411c15f4c0 2023-11-08T19:24:55.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41217e8640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 0x7f411c15c520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4114009e80 tx=0x7f4114004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114040650 con 0x7f411c15f4c0 2023-11-08T19:24:55.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41140407d0 con 0x7f411c15f4c0 2023-11-08T19:24:55.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4114040b80 con 0x7f411c15f4c0 2023-11-08T19:24:55.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.423+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c0cdf20 con 0x7f411c15f4c0 2023-11-08T19:24:55.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.424+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c15db50 con 0x7f411c15f4c0 2023-11-08T19:24:55.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.425+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c0b1b10 con 0x7f411c15f4c0 2023-11-08T19:24:55.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.425+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4114040ce0 con 0x7f411c15f4c0 2023-11-08T19:24:55.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.426+0000 7f41197fa640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f410006ae70 0x7f410006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.426+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f41140ac0c0 con 0x7f411c15f4c0 2023-11-08T19:24:55.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.427+0000 7f4120fe7640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f410006ae70 0x7f410006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.427+0000 7f4120fe7640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f410006ae70 0x7f410006d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4110000f60 tx=0x7f4110009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.431+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f411c0b1b10 con 0x7f411c15f4c0 2023-11-08T19:24:55.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.622+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f411c0b1b10 con 0x7f411c15f4c0 2023-11-08T19:24:55.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.623+0000 7f41197fa640 1 -- 172.21.15.119:0/2086278792 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f411c0b1b10 con 0x7f411c15f4c0 2023-11-08T19:24:55.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:24:55.625 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-11-08T19:24:55.625 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:24:55.625 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-11-08T19:24:55.626 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-11-08T19:24:55.626 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:24:55.626 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-11-08T19:24:55.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f410006ae70 msgr2=0x7f410006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f410006ae70 0x7f410006d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4110000f60 tx=0x7f4110009040 comp rx=0 tx=0).stop 2023-11-08T19:24:55.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 msgr2=0x7f411c15c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 0x7f411c15c520 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4114009e80 tx=0x7f4114004a40 comp rx=0 tx=0).stop 2023-11-08T19:24:55.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 shutdown_connections 2023-11-08T19:24:55.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f410006ae70 0x7f410006d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f411c15f4c0 0x7f411c15c520 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f411c1447a0 0x7f411c13d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 --2- 172.21.15.119:0/2086278792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f411c06a120 0x7f411c0b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 >> 172.21.15.119:0/2086278792 conn(0x7f411c0cb610 msgr2=0x7f411c072d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 shutdown_connections 2023-11-08T19:24:55.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.625+0000 7f412324b640 1 -- 172.21.15.119:0/2086278792 wait complete. 2023-11-08T19:24:55.644 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:24:55.645 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:24:55.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.861+0000 7f2bcae4f640 1 Processor -- start 2023-11-08T19:24:55.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.861+0000 7f2bcae4f640 1 -- start start 2023-11-08T19:24:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.862+0000 7f2bcae4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 0x7f2bc40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.862+0000 7f2bcae4f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc415df50 con 0x7f2bc415d4a0 2023-11-08T19:24:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.862+0000 7f2bcae4f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40d1e60 con 0x7f2bc40cfa00 2023-11-08T19:24:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.862+0000 7f2bcae4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40d03b0 con 0x7f2bc4161870 2023-11-08T19:24:55.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc8beb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2bc415d4a0 0x7f2bc40d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57170/0 (socket says 172.21.15.119:57170) 2023-11-08T19:24:55.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc8beb640 1 -- 172.21.15.119:0/1428655653 learned_addr learned my addr 172.21.15.119:0/1428655653 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:55.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc93ec640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 0x7f2bc40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637435747 0 0) 0x7f2bc415df50 con 0x7f2bc415d4a0 2023-11-08T19:24:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb0003520 con 0x7f2bc415d4a0 2023-11-08T19:24:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3104314298 0 0) 0x7f2bb0003520 con 0x7f2bc415d4a0 2023-11-08T19:24:55.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 msgr2=0x7f2bc40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 0x7f2bc40cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 >> v1:172.21.15.188:6789/0 conn(0x7f2bc40cfa00 legacy=0x7f2bc4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4161d40 con 0x7f2bc415d4a0 2023-11-08T19:24:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2bb40019f0 con 0x7f2bc415d4a0 2023-11-08T19:24:55.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.863+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 >> v1:172.21.15.119:6789/0 conn(0x7f2bc415d4a0 legacy=0x7f2bc40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc37fe640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00039a0 0x7f2bb0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc37fe640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb00047c0 0x7f2bb0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc37fe640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 0x7f2bb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc40d03b0 con 0x7f2bb00047c0 2023-11-08T19:24:55.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb000bab0 con 0x7f2bb00039a0 2023-11-08T19:24:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb000bc70 con 0x7f2bc4161870 2023-11-08T19:24:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb00047c0 0x7f2bb0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc8beb640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00039a0 0x7f2bb0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc93ec640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 0x7f2bb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc3fff640 1 -- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 msgr2=0x7f2bb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc4161870 0x7f2bb00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc3fff640 1 -- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00039a0 msgr2=0x7f2bb0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00039a0 0x7f2bb0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.864+0000 7f2bc3fff640 1 -- 172.21.15.119:0/1428655653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0003520 con 0x7f2bb00047c0 2023-11-08T19:24:55.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.865+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb00047c0 0x7f2bb0004c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2bac001750 tx=0x7f2bb00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e85ecb5fad841a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.865+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0052d0 con 0x7f2bb00047c0 2023-11-08T19:24:55.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac02e050 con 0x7f2bb00047c0 2023-11-08T19:24:55.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bc37fe640 1 -- 172.21.15.119:0/1428655653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0035f0 con 0x7f2bb00047c0 2023-11-08T19:24:55.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bcae4f640 1 -- 172.21.15.119:0/1428655653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb00047c0 msgr2=0x7f2bb0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/1428655653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bb00047c0 0x7f2bb0004c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2bac001750 tx=0x7f2bb00057f0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bc93ec640 1 -- 172.21.15.119:0/1428655653 reap_dead start 2023-11-08T19:24:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bcae4f640 1 -- 172.21.15.119:0/1428655653 shutdown_connections 2023-11-08T19:24:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bcae4f640 1 -- 172.21.15.119:0/1428655653 >> 172.21.15.119:0/1428655653 conn(0x7f2bc406a9f0 msgr2=0x7f2bc406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bcae4f640 1 -- 172.21.15.119:0/1428655653 shutdown_connections 2023-11-08T19:24:55.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.866+0000 7f2bcae4f640 1 -- 172.21.15.119:0/1428655653 wait complete. 2023-11-08T19:24:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.868+0000 7f2bcae4f640 1 Processor -- start 2023-11-08T19:24:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.868+0000 7f2bcae4f640 1 -- start start 2023-11-08T19:24:55.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bcae4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00047c0 0x7f2bc41f0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bcae4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 0x7f2bc4154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bcae4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc40cd010 0x7f2bc4154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bcae4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4152060 con 0x7f2bc40ccb90 2023-11-08T19:24:55.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bcae4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4152340 con 0x7f2bb00047c0 2023-11-08T19:24:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bcae4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4152620 con 0x7f2bc40cd010 2023-11-08T19:24:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bc3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 0x7f2bc4154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bc3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 0x7f2bc4154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44464/0 (socket says 172.21.15.119:44464) 2023-11-08T19:24:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.869+0000 7f2bc3fff640 1 -- 172.21.15.119:0/933655812 learned_addr learned my addr 172.21.15.119:0/933655812 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:55.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc93ec640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc40cd010 0x7f2bc4154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc8beb640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00047c0 0x7f2bc41f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc3fff640 1 -- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc40cd010 msgr2=0x7f2bc4154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc40cd010 0x7f2bc4154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc3fff640 1 -- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00047c0 msgr2=0x7f2bc41f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:55.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00047c0 0x7f2bc41f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:55.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc3fff640 1 -- 172.21.15.119:0/933655812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4152900 con 0x7f2bc40ccb90 2023-11-08T19:24:55.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc3fff640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 0x7f2bc4154020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2bac008fd0 tx=0x7f2bac001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0076c0 con 0x7f2bc40ccb90 2023-11-08T19:24:55.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac007a30 con 0x7f2bc40ccb90 2023-11-08T19:24:55.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac006be0 con 0x7f2bc40ccb90 2023-11-08T19:24:55.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.870+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc4152be0 con 0x7f2bc40ccb90 2023-11-08T19:24:55.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.871+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc4153100 con 0x7f2bc40ccb90 2023-11-08T19:24:55.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.872+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc40d10f0 con 0x7f2bc40ccb90 2023-11-08T19:24:55.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.872+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2bac006d40 con 0x7f2bc40ccb90 2023-11-08T19:24:55.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.873+0000 7f2bc17fa640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ba406ae70 0x7f2ba406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:55.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.873+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2bac0ab390 con 0x7f2bc40ccb90 2023-11-08T19:24:55.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.874+0000 7f2bc8beb640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ba406ae70 0x7f2ba406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:55.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.874+0000 7f2bc8beb640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ba406ae70 0x7f2ba406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4002010 tx=0x7f2bb4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:55.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:55.878+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bac076790 con 0x7f2bc40ccb90 2023-11-08T19:24:56.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.069+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2bc415fbd0 con 0x7f2bc40ccb90 2023-11-08T19:24:56.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.611+0000 7f2bc17fa640 1 -- 172.21.15.119:0/933655812 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2bac079db0 con 0x7f2bc40ccb90 2023-11-08T19:24:56.613 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-11-08T19:24:56.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.615+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ba406ae70 msgr2=0x7f2ba406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.615+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ba406ae70 0x7f2ba406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4002010 tx=0x7f2bb4009040 comp rx=0 tx=0).stop 2023-11-08T19:24:56.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.615+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 msgr2=0x7f2bc4154020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.615+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 0x7f2bc4154020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2bac008fd0 tx=0x7f2bac001120 comp rx=0 tx=0).stop 2023-11-08T19:24:56.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.616+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 shutdown_connections 2023-11-08T19:24:56.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.616+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ba406ae70 0x7f2ba406d330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.616+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc40cd010 0x7f2bc4154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.616+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2bc40ccb90 0x7f2bc4154020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.616+0000 7f2bcae4f640 1 --2- 172.21.15.119:0/933655812 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bb00047c0 0x7f2bc41f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.616+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 >> 172.21.15.119:0/933655812 conn(0x7f2bc406a9f0 msgr2=0x7f2bc415ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:56.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.617+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 shutdown_connections 2023-11-08T19:24:56.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.617+0000 7f2bcae4f640 1 -- 172.21.15.119:0/933655812 wait complete. 2023-11-08T19:24:56.639 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-08T19:24:56.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.845+0000 7f6627cbb640 1 Processor -- start 2023-11-08T19:24:56.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.845+0000 7f6627cbb640 1 -- start start 2023-11-08T19:24:56.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6627cbb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cb4b0 0x7f662006f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6627cbb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66200cb890 0x7f66200ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6627cbb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f662006feb0 con 0x7f66200cd020 2023-11-08T19:24:56.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6627cbb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66200c9400 con 0x7f66200cb890 2023-11-08T19:24:56.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6627cbb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200cd8c0 con 0x7f66200cb4b0 2023-11-08T19:24:56.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6626258640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66200cb890 0x7f66200ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6626258640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66200cb890 0x7f66200ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44474/0 (socket says 172.21.15.119:44474) 2023-11-08T19:24:56.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6626258640 1 -- 172.21.15.119:0/902646432 learned_addr learned my addr 172.21.15.119:0/902646432 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:56.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6625a57640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cb4b0 0x7f662006f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6626258640 1 -- 172.21.15.119:0/902646432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cb4b0 msgr2=0x7f662006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:56.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6626258640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cb4b0 0x7f662006f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.847+0000 7f6626258640 1 -- 172.21.15.119:0/902646432 >> v1:172.21.15.188:6789/0 conn(0x7f66200cd020 legacy=0x7f66200ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:56.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.848+0000 7f6626258640 1 -- 172.21.15.119:0/902646432 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200cd5e0 con 0x7f66200cb890 2023-11-08T19:24:56.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.848+0000 7f6625a57640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cb4b0 0x7f662006f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:56.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.849+0000 7f6626258640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb890 0x7f66200ca160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6618009e50 tx=0x7f661802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16a415be82c5fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:56.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.849+0000 7f6624a55640 1 -- 172.21.15.119:0/902646432 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661802c9e0 con 0x7f66200cb890 2023-11-08T19:24:56.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.849+0000 7f6624a55640 1 -- 172.21.15.119:0/902646432 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661802cb40 con 0x7f66200cb890 2023-11-08T19:24:56.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.849+0000 7f6624a55640 1 -- 172.21.15.119:0/902646432 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180343f0 con 0x7f66200cb890 2023-11-08T19:24:56.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.849+0000 7f6627cbb640 1 -- 172.21.15.119:0/902646432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb890 msgr2=0x7f66200ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.849+0000 7f6627cbb640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb890 0x7f66200ca160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6618009e50 tx=0x7f661802be40 comp rx=0 tx=0).stop 2023-11-08T19:24:56.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.850+0000 7f6627cbb640 1 -- 172.21.15.119:0/902646432 shutdown_connections 2023-11-08T19:24:56.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.850+0000 7f6627cbb640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb890 0x7f66200ca160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.850+0000 7f6627cbb640 1 --2- 172.21.15.119:0/902646432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cb4b0 0x7f662006f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.850+0000 7f6627cbb640 1 -- 172.21.15.119:0/902646432 >> 172.21.15.119:0/902646432 conn(0x7f66200b0950 msgr2=0x7f66200b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:56.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.850+0000 7f6627cbb640 1 -- 172.21.15.119:0/902646432 shutdown_connections 2023-11-08T19:24:56.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.850+0000 7f6627cbb640 1 -- 172.21.15.119:0/902646432 wait complete. 2023-11-08T19:24:56.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.851+0000 7f6627cbb640 1 Processor -- start 2023-11-08T19:24:56.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.851+0000 7f6627cbb640 1 -- start start 2023-11-08T19:24:56.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.852+0000 7f6627cbb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 0x7f66201f3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.852+0000 7f6627cbb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200cb890 0x7f66201506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.852+0000 7f6627cbb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cd020 0x7f662015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.852+0000 7f6627cbb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66200c3a80 con 0x7f66200cb4b0 2023-11-08T19:24:56.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.852+0000 7f6627cbb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66200c3d60 con 0x7f66200cb890 2023-11-08T19:24:56.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.852+0000 7f6627cbb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200c4040 con 0x7f66200cd020 2023-11-08T19:24:56.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 0x7f66201f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 0x7f66201f3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44486/0 (socket says 172.21.15.119:44486) 2023-11-08T19:24:56.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 -- 172.21.15.119:0/2795015553 learned_addr learned my addr 172.21.15.119:0/2795015553 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:56.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 -- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cd020 msgr2=0x7f662015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cd020 0x7f662015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625256640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200cb890 0x7f66201506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 -- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200cb890 msgr2=0x7f66201506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:56.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200cb890 0x7f66201506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:56.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 -- 172.21.15.119:0/2795015553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6618009b30 con 0x7f66200cb4b0 2023-11-08T19:24:56.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.853+0000 7f6625a57640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 0x7f66201f3910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f661c00c7f0 tx=0x7f661c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:56.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.854+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c007820 con 0x7f66200cb4b0 2023-11-08T19:24:56.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.854+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c007980 con 0x7f66200cb4b0 2023-11-08T19:24:56.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.854+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c007cf0 con 0x7f66200cb4b0 2023-11-08T19:24:56.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.854+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66200c4320 con 0x7f66200cb4b0 2023-11-08T19:24:56.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.854+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66201f3e50 con 0x7f66200cb4b0 2023-11-08T19:24:56.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.855+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f662004ece0 con 0x7f66200cb4b0 2023-11-08T19:24:56.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.856+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f661c0113f0 con 0x7f66200cb4b0 2023-11-08T19:24:56.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.857+0000 7f66167fc640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f660406ae70 0x7f660406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:56.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.857+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f661c0897e0 con 0x7f66200cb4b0 2023-11-08T19:24:56.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.857+0000 7f6625256640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f660406ae70 0x7f660406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:56.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.858+0000 7f6625256640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f660406ae70 0x7f660406d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6620156610 tx=0x7f6610009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:56.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:56.862+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f661c0549d0 con 0x7f66200cb4b0 2023-11-08T19:24:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.050+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f662006dc40 con 0x7f66200cb4b0 2023-11-08T19:24:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.624+0000 7f66167fc640 1 -- 172.21.15.119:0/2795015553 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7f661c057ff0 con 0x7f66200cb4b0 2023-11-08T19:24:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-11-08T19:24:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.630+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f660406ae70 msgr2=0x7f660406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:57.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.630+0000 7f6627cbb640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f660406ae70 0x7f660406d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6620156610 tx=0x7f6610009040 comp rx=0 tx=0).stop 2023-11-08T19:24:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.630+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 msgr2=0x7f66201f3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.630+0000 7f6627cbb640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 0x7f66201f3910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f661c00c7f0 tx=0x7f661c00ccb0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.631+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 shutdown_connections 2023-11-08T19:24:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.631+0000 7f6627cbb640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f660406ae70 0x7f660406d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.631+0000 7f6627cbb640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66200cd020 0x7f662015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.631+0000 7f6627cbb640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66200cb890 0x7f66201506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.631+0000 7f6627cbb640 1 --2- 172.21.15.119:0/2795015553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66200cb4b0 0x7f66201f3910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.631+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 >> 172.21.15.119:0/2795015553 conn(0x7f66200b0950 msgr2=0x7f6620071340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:57.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.634+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 shutdown_connections 2023-11-08T19:24:57.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.634+0000 7f6627cbb640 1 -- 172.21.15.119:0/2795015553 wait complete. 2023-11-08T19:24:57.671 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:24:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.859+0000 7f9974e87640 1 Processor -- start 2023-11-08T19:24:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.860+0000 7f9974e87640 1 -- start start 2023-11-08T19:24:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f9974e87640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99701596b0 0x7f997006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f9974e87640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9970070040 0x7f99700c9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f9974e87640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997006bf30 con 0x7f997006ba30 2023-11-08T19:24:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f9974e87640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997006cbb0 con 0x7f99701596b0 2023-11-08T19:24:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f9974e87640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997015e170 con 0x7f9970070040 2023-11-08T19:24:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f996e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99701596b0 0x7f997006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f996e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f99701596b0 0x7f997006cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44498/0 (socket says 172.21.15.119:44498) 2023-11-08T19:24:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.861+0000 7f996e59c640 1 -- 172.21.15.119:0/4162391099 learned_addr learned my addr 172.21.15.119:0/4162391099 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.862+0000 7f996e59c640 1 -- 172.21.15.119:0/4162391099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9970070040 msgr2=0x7f99700c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.862+0000 7f996ed9d640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9970070040 0x7f99700c9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.862+0000 7f996e59c640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9970070040 0x7f99700c9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.862+0000 7f996e59c640 1 -- 172.21.15.119:0/4162391099 >> v1:172.21.15.188:6789/0 conn(0x7f997006ba30 legacy=0x7f997006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.862+0000 7f996e59c640 1 -- 172.21.15.119:0/4162391099 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997006c210 con 0x7f99701596b0 2023-11-08T19:24:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.862+0000 7f996ed9d640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9970070040 0x7f99700c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:57.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.863+0000 7f996e59c640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99701596b0 0x7f997006cfd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99600098a0 tx=0x7f996002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6368798b326839b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.863+0000 7f996dd9b640 1 -- 172.21.15.119:0/4162391099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960031640 con 0x7f99701596b0 2023-11-08T19:24:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.863+0000 7f996dd9b640 1 -- 172.21.15.119:0/4162391099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99600317a0 con 0x7f99701596b0 2023-11-08T19:24:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.863+0000 7f996dd9b640 1 -- 172.21.15.119:0/4162391099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960031b30 con 0x7f99701596b0 2023-11-08T19:24:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 -- 172.21.15.119:0/4162391099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99701596b0 msgr2=0x7f997006cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:57.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99701596b0 0x7f997006cfd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f99600098a0 tx=0x7f996002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 -- 172.21.15.119:0/4162391099 shutdown_connections 2023-11-08T19:24:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9970070040 0x7f99700c9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 --2- 172.21.15.119:0/4162391099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f99701596b0 0x7f997006cfd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 -- 172.21.15.119:0/4162391099 >> 172.21.15.119:0/4162391099 conn(0x7f99700cb610 msgr2=0x7f99700cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 -- 172.21.15.119:0/4162391099 shutdown_connections 2023-11-08T19:24:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.864+0000 7f9974e87640 1 -- 172.21.15.119:0/4162391099 wait complete. 2023-11-08T19:24:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.866+0000 7f9974e87640 1 Processor -- start 2023-11-08T19:24:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.866+0000 7f9974e87640 1 -- start start 2023-11-08T19:24:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f9974e87640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 0x7f997015aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f9974e87640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9970070040 0x7f99701471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f9974e87640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99701596b0 0x7f997006a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f9974e87640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701f2df0 con 0x7f997006ba30 2023-11-08T19:24:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f9974e87640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99701f21e0 con 0x7f9970070040 2023-11-08T19:24:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f9974e87640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99701f24c0 con 0x7f99701596b0 2023-11-08T19:24:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 0x7f997015aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 0x7f997015aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44502/0 (socket says 172.21.15.119:44502) 2023-11-08T19:24:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 -- 172.21.15.119:0/2469552312 learned_addr learned my addr 172.21.15.119:0/2469552312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 -- 172.21.15.119:0/2469552312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99701596b0 msgr2=0x7f997006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99701596b0 0x7f997006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 -- 172.21.15.119:0/2469552312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9970070040 msgr2=0x7f99701471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9970070040 0x7f99701471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.867+0000 7f996e59c640 1 -- 172.21.15.119:0/2469552312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99600094d0 con 0x7f997006ba30 2023-11-08T19:24:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.868+0000 7f996e59c640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 0x7f997015aad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9960000c00 tx=0x7f99600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.868+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996003ac60 con 0x7f997006ba30 2023-11-08T19:24:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.868+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996003adc0 con 0x7f997006ba30 2023-11-08T19:24:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.868+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9960044610 con 0x7f997006ba30 2023-11-08T19:24:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.868+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99701f27a0 con 0x7f997006ba30 2023-11-08T19:24:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.868+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9970147730 con 0x7f997006ba30 2023-11-08T19:24:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.869+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9970074430 con 0x7f997006ba30 2023-11-08T19:24:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.870+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f996002f050 con 0x7f997006ba30 2023-11-08T19:24:57.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.871+0000 7f9966ffd640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f995406ae70 0x7f995406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.871+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f99600ab5e0 con 0x7f997006ba30 2023-11-08T19:24:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.872+0000 7f9967fff640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f995406ae70 0x7f995406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:57.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.872+0000 7f9967fff640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f995406ae70 0x7f995406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99701584a0 tx=0x7f99580079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:57.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:57.876+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99600766e0 con 0x7f997006ba30 2023-11-08T19:24:58.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.101+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f99701f2600 con 0x7f997006ba30 2023-11-08T19:24:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.641+0000 7f9966ffd640 1 -- 172.21.15.119:0/2469552312 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f99701f2600 con 0x7f997006ba30 2023-11-08T19:24:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 2 and data pool 3 2023-11-08T19:24:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.644+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f995406ae70 msgr2=0x7f995406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.644+0000 7f9974e87640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f995406ae70 0x7f995406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99701584a0 tx=0x7f99580079b0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.644+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 msgr2=0x7f997015aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.644+0000 7f9974e87640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 0x7f997015aad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9960000c00 tx=0x7f99600046c0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 shutdown_connections 2023-11-08T19:24:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f995406ae70 0x7f995406d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f99701596b0 0x7f997006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9970070040 0x7f99701471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 --2- 172.21.15.119:0/2469552312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f997006ba30 0x7f997015aad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 >> 172.21.15.119:0/2469552312 conn(0x7f99700cb610 msgr2=0x7f9970071220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 shutdown_connections 2023-11-08T19:24:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.645+0000 7f9974e87640 1 -- 172.21.15.119:0/2469552312 wait complete. 2023-11-08T19:24:58.670 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-08T19:24:58.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-08T19:24:58.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.859+0000 7fdfda656640 1 Processor -- start 2023-11-08T19:24:58.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.860+0000 7fdfda656640 1 -- start start 2023-11-08T19:24:58.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfda656640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd4144580 0x7fdfd415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfda656640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdfd4144960 0x7fdfd40cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfda656640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfd415f1d0 0x7fdfd40c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfda656640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd40c9150 con 0x7fdfd415f1d0 2023-11-08T19:24:58.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfda656640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd415ea60 con 0x7fdfd4144960 2023-11-08T19:24:58.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfda656640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40cb270 con 0x7fdfd4144580 2023-11-08T19:24:58.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfd8bf3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfd415f1d0 0x7fdfd40c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:58.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfd8bf3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfd415f1d0 0x7fdfd40c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33202/0 (socket says 172.21.15.119:33202) 2023-11-08T19:24:58.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.861+0000 7fdfd8bf3640 1 -- 172.21.15.119:0/2367885473 learned_addr learned my addr 172.21.15.119:0/2367885473 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:58.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfcbfff640 1 --2- 172.21.15.119:0/2367885473 >> v2:172.21.15.188:3300/0 conn(0x7fdfd4144960 0x7fdfd40cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:58.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd3fff640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd4144580 0x7fdfd415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:58.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd8bf3640 1 -- 172.21.15.119:0/2367885473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd4144580 msgr2=0x7fdfd415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:58.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd8bf3640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd4144580 0x7fdfd415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd8bf3640 1 -- 172.21.15.119:0/2367885473 >> v2:172.21.15.188:3300/0 conn(0x7fdfd4144960 msgr2=0x7fdfd40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:58.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd8bf3640 1 --2- 172.21.15.119:0/2367885473 >> v2:172.21.15.188:3300/0 conn(0x7fdfd4144960 0x7fdfd40cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd8bf3640 1 -- 172.21.15.119:0/2367885473 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40c9360 con 0x7fdfd415f1d0 2023-11-08T19:24:58.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.862+0000 7fdfd3fff640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd4144580 0x7fdfd415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:58.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.863+0000 7fdfd8bf3640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd415f1d0 0x7fdfd40c3a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40cc680 tx=0x7fdfcc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548afaf40e98c3d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:58.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.863+0000 7fdfd37fe640 1 -- 172.21.15.119:0/2367885473 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc02c9e0 con 0x7fdfd415f1d0 2023-11-08T19:24:58.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.863+0000 7fdfd37fe640 1 -- 172.21.15.119:0/2367885473 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc02cb40 con 0x7fdfd415f1d0 2023-11-08T19:24:58.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.863+0000 7fdfd37fe640 1 -- 172.21.15.119:0/2367885473 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc0343f0 con 0x7fdfd415f1d0 2023-11-08T19:24:58.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 -- 172.21.15.119:0/2367885473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd415f1d0 msgr2=0x7fdfd40c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:58.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd415f1d0 0x7fdfd40c3a50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40cc680 tx=0x7fdfcc02be40 comp rx=0 tx=0).stop 2023-11-08T19:24:58.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 -- 172.21.15.119:0/2367885473 shutdown_connections 2023-11-08T19:24:58.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd415f1d0 0x7fdfd40c3a50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 --2- 172.21.15.119:0/2367885473 >> v2:172.21.15.188:3300/0 conn(0x7fdfd4144960 0x7fdfd40cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 --2- 172.21.15.119:0/2367885473 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd4144580 0x7fdfd415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 -- 172.21.15.119:0/2367885473 >> 172.21.15.119:0/2367885473 conn(0x7fdfd40cb610 msgr2=0x7fdfd40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:58.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 -- 172.21.15.119:0/2367885473 shutdown_connections 2023-11-08T19:24:58.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.864+0000 7fdfda656640 1 -- 172.21.15.119:0/2367885473 wait complete. 2023-11-08T19:24:58.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.866+0000 7fdfda656640 1 Processor -- start 2023-11-08T19:24:58.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.866+0000 7fdfda656640 1 -- start start 2023-11-08T19:24:58.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfda656640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 0x7fdfd41f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfda656640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd4144960 0x7fdfd40a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfda656640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd415f1d0 0x7fdfd40a9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfda656640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd41f6ab0 con 0x7fdfd4144580 2023-11-08T19:24:58.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfda656640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40a9a30 con 0x7fdfd4144960 2023-11-08T19:24:58.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfda656640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40a9d10 con 0x7fdfd415f1d0 2023-11-08T19:24:58.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfd3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 0x7fdfd41f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:58.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfd3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 0x7fdfd41f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33218/0 (socket says 172.21.15.119:33218) 2023-11-08T19:24:58.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.867+0000 7fdfd3fff640 1 -- 172.21.15.119:0/3130744249 learned_addr learned my addr 172.21.15.119:0/3130744249 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:58.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfcbfff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd4144960 0x7fdfd40a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:58.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd3fff640 1 -- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd415f1d0 msgr2=0x7fdfd40a9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:58.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd3fff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd415f1d0 0x7fdfd40a9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd3fff640 1 -- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd4144960 msgr2=0x7fdfd40a8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:58.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd3fff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd4144960 0x7fdfd40a8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd3fff640 1 -- 172.21.15.119:0/3130744249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc009b30 con 0x7fdfd4144580 2023-11-08T19:24:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfcbfff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd4144960 0x7fdfd40a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd3fff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 0x7fdfd41f5f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c7c0 tx=0x7fdfc400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4007a30 con 0x7fdfd4144580 2023-11-08T19:24:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc4007b90 con 0x7fdfd4144580 2023-11-08T19:24:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400ce70 con 0x7fdfd4144580 2023-11-08T19:24:58.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd40a9ff0 con 0x7fdfd4144580 2023-11-08T19:24:58.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.868+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd41f6470 con 0x7fdfd4144580 2023-11-08T19:24:58.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.869+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd41f3890 con 0x7fdfd4144580 2023-11-08T19:24:58.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.870+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfc40113f0 con 0x7fdfd4144580 2023-11-08T19:24:58.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.871+0000 7fdfd17fa640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfb406ae70 0x7fdfb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.871+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfc4089710 con 0x7fdfd4144580 2023-11-08T19:24:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.871+0000 7fdfcbfff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfb406ae70 0x7fdfb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.872+0000 7fdfcbfff640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfb406ae70 0x7fdfb406d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4143430 tx=0x7fdfbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:58.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:58.876+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfd41f3890 con 0x7fdfd4144580 2023-11-08T19:24:59.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.074+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7fdfd41f3890 con 0x7fdfd4144580 2023-11-08T19:24:59.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.589+0000 7fdfd17fa640 1 -- 172.21.15.119:0/3130744249 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7fdfd41f3890 con 0x7fdfd4144580 2023-11-08T19:24:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfb406ae70 msgr2=0x7fdfb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfb406ae70 0x7fdfb406d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4143430 tx=0x7fdfbc008040 comp rx=0 tx=0).stop 2023-11-08T19:24:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 msgr2=0x7fdfd41f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 0x7fdfd41f5f30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfc400c7c0 tx=0x7fdfc400cc80 comp rx=0 tx=0).stop 2023-11-08T19:24:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 shutdown_connections 2023-11-08T19:24:59.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfb406ae70 0x7fdfb406d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfd415f1d0 0x7fdfd40a9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfd4144960 0x7fdfd40a8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 --2- 172.21.15.119:0/3130744249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfd4144580 0x7fdfd41f5f30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.592+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 >> 172.21.15.119:0/3130744249 conn(0x7fdfd40cb610 msgr2=0x7fdfd40cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:59.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.593+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 shutdown_connections 2023-11-08T19:24:59.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.593+0000 7fdfda656640 1 -- 172.21.15.119:0/3130744249 wait complete. 2023-11-08T19:24:59.614 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:24:59.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.834+0000 7ff882a2b640 1 Processor -- start 2023-11-08T19:24:59.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.835+0000 7ff882a2b640 1 -- start start 2023-11-08T19:24:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.836+0000 7ff882a2b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff87c161870 0x7ff87c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.836+0000 7ff882a2b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c15df70 con 0x7ff87c15d6c0 2023-11-08T19:24:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.836+0000 7ff882a2b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0d1e60 con 0x7ff87c0cfa00 2023-11-08T19:24:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.836+0000 7ff882a2b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c15daa0 con 0x7ff87c161870 2023-11-08T19:24:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff87c15d6c0 0x7ff87c0d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45472/0 (socket says 172.21.15.119:45472) 2023-11-08T19:24:59.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87bfff640 1 -- 172.21.15.119:0/1506045671 learned_addr learned my addr 172.21.15.119:0/1506045671 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:24:59.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff880fc8640 1 --2- 172.21.15.119:0/1506045671 >> v2:172.21.15.188:3300/0 conn(0x7ff87c161870 0x7ff87c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4261617628 0 0) 0x7ff87c15df70 con 0x7ff87c15d6c0 2023-11-08T19:24:59.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff868003520 con 0x7ff87c15d6c0 2023-11-08T19:24:59.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926638761 0 0) 0x7ff868003520 con 0x7ff87c15d6c0 2023-11-08T19:24:59.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 >> v2:172.21.15.188:3300/0 conn(0x7ff87c161870 msgr2=0x7ff87c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:59.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.837+0000 7ff87affd640 1 --2- 172.21.15.119:0/1506045671 >> v2:172.21.15.188:3300/0 conn(0x7ff87c161870 0x7ff87c0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 >> v1:172.21.15.188:6789/0 conn(0x7ff87c0cfa00 legacy=0x7ff87c161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c161d40 con 0x7ff87c15d6c0 2023-11-08T19:24:59.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7ff86c0019f0 con 0x7ff87c15d6c0 2023-11-08T19:24:59.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 >> v1:172.21.15.119:6789/0 conn(0x7ff87c15d6c0 legacy=0x7ff87c0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff868003970 0x7ff868003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8680047c0 0x7ff868004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff868005180 0x7ff86800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c15daa0 con 0x7ff868003970 2023-11-08T19:24:59.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff86800c120 con 0x7ff8680047c0 2023-11-08T19:24:59.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86800c2e0 con 0x7ff868005180 2023-11-08T19:24:59.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87bfff640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff868003970 0x7ff868003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87bfff640 1 -- 172.21.15.119:0/1506045671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff868005180 msgr2=0x7ff86800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:59.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87bfff640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff868005180 0x7ff86800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87bfff640 1 -- 172.21.15.119:0/1506045671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8680047c0 msgr2=0x7ff868004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:24:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87bfff640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff8680047c0 0x7ff868004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff87bfff640 1 -- 172.21.15.119:0/1506045671 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff868003520 con 0x7ff868003970 2023-11-08T19:24:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.838+0000 7ff880fc8640 1 -- 172.21.15.119:0/1506045671 reap_dead start 2023-11-08T19:24:59.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.839+0000 7ff87bfff640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff868003970 0x7ff868003d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00a830 tx=0x7ff8680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3df3555b6d43926 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0072d0 con 0x7ff868003970 2023-11-08T19:24:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86c02f070 con 0x7ff868003970 2023-11-08T19:24:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff87affd640 1 -- 172.21.15.119:0/1506045671 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c0076f0 con 0x7ff868003970 2023-11-08T19:24:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 -- 172.21.15.119:0/1506045671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff868003970 msgr2=0x7ff868003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff868003970 0x7ff868003d90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00a830 tx=0x7ff8680057f0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 -- 172.21.15.119:0/1506045671 shutdown_connections 2023-11-08T19:24:59.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 --2- 172.21.15.119:0/1506045671 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff868003970 0x7ff868003d90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 -- 172.21.15.119:0/1506045671 >> 172.21.15.119:0/1506045671 conn(0x7ff87c06a9f0 msgr2=0x7ff87c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:24:59.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 -- 172.21.15.119:0/1506045671 shutdown_connections 2023-11-08T19:24:59.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.840+0000 7ff882a2b640 1 -- 172.21.15.119:0/1506045671 wait complete. 2023-11-08T19:24:59.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.842+0000 7ff882a2b640 1 Processor -- start 2023-11-08T19:24:59.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.842+0000 7ff882a2b640 1 -- start start 2023-11-08T19:24:59.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.843+0000 7ff882a2b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868003970 0x7ff87c1f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.843+0000 7ff882a2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 0x7ff87c0cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.843+0000 7ff882a2b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c1572a0 0x7ff87c0ab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.843+0000 7ff882a2b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0ca0c0 con 0x7ff87c0aaf60 2023-11-08T19:24:59.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.843+0000 7ff882a2b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0ca3a0 con 0x7ff868003970 2023-11-08T19:24:59.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.843+0000 7ff882a2b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c0ca680 con 0x7ff87c1572a0 2023-11-08T19:24:59.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 0x7ff87c0cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 0x7ff87c0cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33240/0 (socket says 172.21.15.119:33240) 2023-11-08T19:24:59.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2501241785 learned_addr learned my addr 172.21.15.119:0/2501241785 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:24:59.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87bfff640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868003970 0x7ff87c1f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c1572a0 msgr2=0x7ff87c0ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff880fc8640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c1572a0 0x7ff87c0ab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c1572a0 0x7ff87c0ab9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868003970 msgr2=0x7ff87c1f3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:24:59.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868003970 0x7ff87c1f3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:24:59.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2501241785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c0ca960 con 0x7ff87c0aaf60 2023-11-08T19:24:59.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87bfff640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868003970 0x7ff87c1f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:24:59.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.844+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 0x7ff87c0cc900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff86400c500 tx=0x7ff86400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.845+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8640052d0 con 0x7ff87c0aaf60 2023-11-08T19:24:59.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.845+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff864005430 con 0x7ff87c0aaf60 2023-11-08T19:24:59.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.845+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff87c0cac40 con 0x7ff87c0aaf60 2023-11-08T19:24:59.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.845+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff864012870 con 0x7ff87c0aaf60 2023-11-08T19:24:59.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.845+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c0cce40 con 0x7ff87c0aaf60 2023-11-08T19:24:59.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.846+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c0d10f0 con 0x7ff87c0aaf60 2023-11-08T19:24:59.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.847+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8640129d0 con 0x7ff87c0aaf60 2023-11-08T19:24:59.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.848+0000 7ff878ff9640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff85c06ae70 0x7ff85c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:24:59.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.848+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5935+0+0 (secure 0 0 0) 0x7ff864089350 con 0x7ff87c0aaf60 2023-11-08T19:24:59.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.848+0000 7ff87bfff640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff85c06ae70 0x7ff85c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:24:59.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.848+0000 7ff87bfff640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff85c06ae70 0x7ff85c06d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff86c034000 tx=0x7ff86c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:24:59.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:24:59.852+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff864054310 con 0x7ff87c0aaf60 2023-11-08T19:25:00.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:00.041+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff87c0ccfd0 con 0x7ff87c0aaf60 2023-11-08T19:25:01.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.675+0000 7ff878ff9640 1 -- 172.21.15.119:0/2501241785 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff86405ce60 con 0x7ff87c0aaf60 2023-11-08T19:25:01.677 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data_ec' created 2023-11-08T19:25:01.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff85c06ae70 msgr2=0x7ff85c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff85c06ae70 0x7ff85c06d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff86c034000 tx=0x7ff86c007000 comp rx=0 tx=0).stop 2023-11-08T19:25:01.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 msgr2=0x7ff87c0cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 0x7ff87c0cc900 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff86400c500 tx=0x7ff86400c9c0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 shutdown_connections 2023-11-08T19:25:01.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff85c06ae70 0x7ff85c06d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c1572a0 0x7ff87c0ab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0aaf60 0x7ff87c0cc900 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 --2- 172.21.15.119:0/2501241785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff868003970 0x7ff87c1f3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 >> 172.21.15.119:0/2501241785 conn(0x7ff87c06a9f0 msgr2=0x7ff87c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.679+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 shutdown_connections 2023-11-08T19:25:01.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.680+0000 7ff882a2b640 1 -- 172.21.15.119:0/2501241785 wait complete. 2023-11-08T19:25:01.708 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-08T19:25:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.901+0000 7f758e75e640 1 Processor -- start 2023-11-08T19:25:01.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.902+0000 7f758e75e640 1 -- start start 2023-11-08T19:25:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758e75e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7588144580 0x7f758815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758e75e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7588144960 0x7f75880cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758e75e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f758815ea60 con 0x7f758815f1d0 2023-11-08T19:25:01.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758e75e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75880c9150 con 0x7f7588144960 2023-11-08T19:25:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758e75e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75880cb270 con 0x7f7588144580 2023-11-08T19:25:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f757ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7588144960 0x7f75880cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758ccfb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f758815f1d0 0x7f75880c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45482/0 (socket says 172.21.15.119:45482) 2023-11-08T19:25:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.903+0000 7f758ccfb640 1 -- 172.21.15.119:0/1841716546 learned_addr learned my addr 172.21.15.119:0/1841716546 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:01.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.904+0000 7f7587fff640 1 --2- 172.21.15.119:0/1841716546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7588144580 0x7f758815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.904+0000 7f757ffff640 1 -- 172.21.15.119:0/1841716546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7588144580 msgr2=0x7f758815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:01.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.904+0000 7f757ffff640 1 --2- 172.21.15.119:0/1841716546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7588144580 0x7f758815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.904+0000 7f757ffff640 1 -- 172.21.15.119:0/1841716546 >> v1:172.21.15.119:6789/0 conn(0x7f758815f1d0 legacy=0x7f75880c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.904+0000 7f757ffff640 1 -- 172.21.15.119:0/1841716546 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880c9360 con 0x7f7588144960 2023-11-08T19:25:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.904+0000 7f75877fe640 1 -- 172.21.15.119:0/1841716546 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2776637383 0 0) 0x7f758815ea60 con 0x7f758815f1d0 2023-11-08T19:25:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.905+0000 7f757ffff640 1 --2- 172.21.15.119:0/1841716546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75880cad30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75700097a0 tx=0x7f757002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46c56de97c8c4ab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.905+0000 7f75877fe640 1 -- 172.21.15.119:0/1841716546 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757002e070 con 0x7f7588144960 2023-11-08T19:25:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.905+0000 7f75877fe640 1 -- 172.21.15.119:0/1841716546 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700315a0 con 0x7f7588144960 2023-11-08T19:25:01.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.905+0000 7f75877fe640 1 -- 172.21.15.119:0/1841716546 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570031930 con 0x7f7588144960 2023-11-08T19:25:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 -- 172.21.15.119:0/1841716546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 msgr2=0x7f75880cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 --2- 172.21.15.119:0/1841716546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75880cad30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f75700097a0 tx=0x7f757002b840 comp rx=0 tx=0).stop 2023-11-08T19:25:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 -- 172.21.15.119:0/1841716546 shutdown_connections 2023-11-08T19:25:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 --2- 172.21.15.119:0/1841716546 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75880cad30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 --2- 172.21.15.119:0/1841716546 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7588144580 0x7f758815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 -- 172.21.15.119:0/1841716546 >> 172.21.15.119:0/1841716546 conn(0x7f75880cb610 msgr2=0x7f75880cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:01.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 -- 172.21.15.119:0/1841716546 shutdown_connections 2023-11-08T19:25:01.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.906+0000 7f758e75e640 1 -- 172.21.15.119:0/1841716546 wait complete. 2023-11-08T19:25:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.908+0000 7f758e75e640 1 Processor -- start 2023-11-08T19:25:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.908+0000 7f758e75e640 1 -- start start 2023-11-08T19:25:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f758e75e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7588144580 0x7f75880a8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f758e75e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75881f6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f758e75e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758815f1d0 0x7f75881f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f758e75e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c8500 con 0x7f7588144960 2023-11-08T19:25:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f758e75e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c7630 con 0x7f7588144580 2023-11-08T19:25:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f758e75e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75880c7880 con 0x7f758815f1d0 2023-11-08T19:25:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f757ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75881f6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f757ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75881f6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33252/0 (socket says 172.21.15.119:33252) 2023-11-08T19:25:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.909+0000 7f757ffff640 1 -- 172.21.15.119:0/3336715482 learned_addr learned my addr 172.21.15.119:0/3336715482 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f7587fff640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7588144580 0x7f75880a8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f758ccfb640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758815f1d0 0x7f75881f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f757ffff640 1 -- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758815f1d0 msgr2=0x7f75881f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f757ffff640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758815f1d0 0x7f75881f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f757ffff640 1 -- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7588144580 msgr2=0x7f75880a8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f757ffff640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7588144580 0x7f75880a8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f757ffff640 1 -- 172.21.15.119:0/3336715482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7570009400 con 0x7f7588144960 2023-11-08T19:25:01.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f758ccfb640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758815f1d0 0x7f75881f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f757ffff640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75881f6240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7570034040 tx=0x7f75700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570032840 con 0x7f7588144960 2023-11-08T19:25:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75700329a0 con 0x7f7588144960 2023-11-08T19:25:01.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7570032d50 con 0x7f7588144960 2023-11-08T19:25:01.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.910+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75880c7b60 con 0x7f7588144960 2023-11-08T19:25:01.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.911+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75880c8120 con 0x7f7588144960 2023-11-08T19:25:01.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.912+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75880c82b0 con 0x7f7588144960 2023-11-08T19:25:01.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.913+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f757004e430 con 0x7f7588144960 2023-11-08T19:25:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.914+0000 7f75857fa640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756806ae70 0x7f756806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:01.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.914+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f75700ab980 con 0x7f7588144960 2023-11-08T19:25:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.914+0000 7f7587fff640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756806ae70 0x7f756806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.915+0000 7f7587fff640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756806ae70 0x7f756806d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7578006fd0 tx=0x7f7578008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:01.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:01.919+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75700766c0 con 0x7f7588144960 2023-11-08T19:25:02.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.110+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f7588074430 con 0x7f7588144960 2023-11-08T19:25:02.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.688+0000 7f75857fa640 1 -- 172.21.15.119:0/3336715482 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7570079ce0 con 0x7f7588144960 2023-11-08T19:25:02.690 INFO:teuthology.orchestra.run.smithi119.stderr:set pool 4 allow_ec_overwrites to true 2023-11-08T19:25:02.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.691+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756806ae70 msgr2=0x7f756806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:02.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.691+0000 7f758e75e640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756806ae70 0x7f756806d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7578006fd0 tx=0x7f7578008040 comp rx=0 tx=0).stop 2023-11-08T19:25:02.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.691+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 msgr2=0x7f75881f6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:02.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.691+0000 7f758e75e640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75881f6240 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7570034040 tx=0x7f75700046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 shutdown_connections 2023-11-08T19:25:02.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756806ae70 0x7f756806d330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f758815f1d0 0x7f75881f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7588144960 0x7f75881f6240 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 --2- 172.21.15.119:0/3336715482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7588144580 0x7f75880a8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 >> 172.21.15.119:0/3336715482 conn(0x7f75880cb610 msgr2=0x7f75880c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:02.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 shutdown_connections 2023-11-08T19:25:02.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.692+0000 7f758e75e640 1 -- 172.21.15.119:0/3336715482 wait complete. 2023-11-08T19:25:02.727 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-08T19:25:02.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.912+0000 7f26251d7640 1 Processor -- start 2023-11-08T19:25:02.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.912+0000 7f26251d7640 1 -- start start 2023-11-08T19:25:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f26251d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f262015d9f0 0x7f262015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f26251d7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262015e440 con 0x7f262006cb40 2023-11-08T19:25:02.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f26251d7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262015e720 con 0x7f2620143600 2023-11-08T19:25:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f26251d7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26200c4f50 con 0x7f262015d9f0 2023-11-08T19:25:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f2617fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f262015d9f0 0x7f262015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:02.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f2617fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f262015d9f0 0x7f262015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33264/0 (socket says 172.21.15.119:33264) 2023-11-08T19:25:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.914+0000 7f2617fff640 1 -- 172.21.15.119:0/1744872324 learned_addr learned my addr 172.21.15.119:0/1744872324 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.913+0000 7f261f59e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f262006cb40 0x7f262006cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45488/0 (socket says 172.21.15.119:45488) 2023-11-08T19:25:02.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.914+0000 7f2617fff640 1 -- 172.21.15.119:0/1744872324 >> v1:172.21.15.188:6789/0 conn(0x7f2620143600 legacy=0x7f26200c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:02.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.914+0000 7f2617fff640 1 -- 172.21.15.119:0/1744872324 >> v1:172.21.15.119:6789/0 conn(0x7f262006cb40 legacy=0x7f262006cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:02.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.914+0000 7f2617fff640 1 -- 172.21.15.119:0/1744872324 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200c4b70 con 0x7f262015d9f0 2023-11-08T19:25:02.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.915+0000 7f2617fff640 1 --2- 172.21.15.119:0/1744872324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262015d9f0 0x7f262015de60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2608009870 tx=0x7f260802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9065af6c462b9a86 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:02.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.915+0000 7f261e59c640 1 -- 172.21.15.119:0/1744872324 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608031870 con 0x7f262015d9f0 2023-11-08T19:25:02.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.915+0000 7f261e59c640 1 -- 172.21.15.119:0/1744872324 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26080319d0 con 0x7f262015d9f0 2023-11-08T19:25:02.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.915+0000 7f261e59c640 1 -- 172.21.15.119:0/1744872324 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608031d60 con 0x7f262015d9f0 2023-11-08T19:25:02.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 -- 172.21.15.119:0/1744872324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262015d9f0 msgr2=0x7f262015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:02.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 --2- 172.21.15.119:0/1744872324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262015d9f0 0x7f262015de60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2608009870 tx=0x7f260802b910 comp rx=0 tx=0).stop 2023-11-08T19:25:02.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 -- 172.21.15.119:0/1744872324 shutdown_connections 2023-11-08T19:25:02.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 --2- 172.21.15.119:0/1744872324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f262015d9f0 0x7f262015de60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 -- 172.21.15.119:0/1744872324 >> 172.21.15.119:0/1744872324 conn(0x7f26200cb610 msgr2=0x7f26200cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:02.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 -- 172.21.15.119:0/1744872324 shutdown_connections 2023-11-08T19:25:02.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.916+0000 7f26251d7640 1 -- 172.21.15.119:0/1744872324 wait complete. 2023-11-08T19:25:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.918+0000 7f26251d7640 1 Processor -- start 2023-11-08T19:25:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.918+0000 7f26251d7640 1 -- start start 2023-11-08T19:25:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f26251d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262006cb40 0x7f2620070f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:02.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f26251d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 0x7f26201f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f26251d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262015d9f0 0x7f26200af270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f26251d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26201f37d0 con 0x7f2620143600 2023-11-08T19:25:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f26251d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262006fed0 con 0x7f262015d9f0 2023-11-08T19:25:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f26251d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26200701b0 con 0x7f262006cb40 2023-11-08T19:25:02.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f2617fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 0x7f26201f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f2617fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 0x7f26201f3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33276/0 (socket says 172.21.15.119:33276) 2023-11-08T19:25:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.919+0000 7f2617fff640 1 -- 172.21.15.119:0/496362017 learned_addr learned my addr 172.21.15.119:0/496362017 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f261ed9d640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262006cb40 0x7f2620070f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f2617fff640 1 -- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262006cb40 msgr2=0x7f2620070f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:02.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f2617fff640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262006cb40 0x7f2620070f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f2617fff640 1 -- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262015d9f0 msgr2=0x7f26200af270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:02.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f2617fff640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262015d9f0 0x7f26200af270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:02.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f2617fff640 1 -- 172.21.15.119:0/496362017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26080094d0 con 0x7f2620143600 2023-11-08T19:25:02.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f2617fff640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 0x7f26201f3290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26080090b0 tx=0x7f26080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:02.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608031800 con 0x7f2620143600 2023-11-08T19:25:02.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26080443f0 con 0x7f2620143600 2023-11-08T19:25:02.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26080447a0 con 0x7f2620143600 2023-11-08T19:25:02.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.920+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2620070490 con 0x7f2620143600 2023-11-08T19:25:02.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.921+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2620070a50 con 0x7f2620143600 2023-11-08T19:25:02.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.922+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26200702f0 con 0x7f2620143600 2023-11-08T19:25:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.923+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f260804d450 con 0x7f2620143600 2023-11-08T19:25:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.924+0000 7f26177fe640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f260006ae70 0x7f260006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.924+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6428+0+0 (secure 0 0 0) 0x7f260802d040 con 0x7f2620143600 2023-11-08T19:25:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.924+0000 7f261ed9d640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f260006ae70 0x7f260006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:02.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.925+0000 7f261ed9d640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f260006ae70 0x7f260006d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2610004ea0 tx=0x7f2610009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:02.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:02.928+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26200702f0 con 0x7f2620143600 2023-11-08T19:25:03.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.148+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f26200702f0 con 0x7f2620143600 2023-11-08T19:25:03.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.690+0000 7f26177fe640 1 -- 172.21.15.119:0/496362017 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f26200702f0 con 0x7f2620143600 2023-11-08T19:25:03.693 INFO:teuthology.orchestra.run.smithi119.stderr:added data pool 4 to fsmap 2023-11-08T19:25:03.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f260006ae70 msgr2=0x7f260006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f260006ae70 0x7f260006d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2610004ea0 tx=0x7f2610009040 comp rx=0 tx=0).stop 2023-11-08T19:25:03.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 msgr2=0x7f26201f3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 0x7f26201f3290 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f26080090b0 tx=0x7f26080046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 shutdown_connections 2023-11-08T19:25:03.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f260006ae70 0x7f260006d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f262015d9f0 0x7f26200af270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2620143600 0x7f26201f3290 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 --2- 172.21.15.119:0/496362017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f262006cb40 0x7f2620070f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 >> 172.21.15.119:0/496362017 conn(0x7f26200cb610 msgr2=0x7f2620072df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 shutdown_connections 2023-11-08T19:25:03.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.695+0000 7f26251d7640 1 -- 172.21.15.119:0/496362017 wait complete. 2023-11-08T19:25:03.718 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:03.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.914+0000 7f8bbf034640 1 Processor -- start 2023-11-08T19:25:03.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.914+0000 7f8bbf034640 1 -- start start 2023-11-08T19:25:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.915+0000 7f8bbf034640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8bb81447a0 0x7f8bb815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.915+0000 7f8bbf034640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80c3a50 con 0x7f8bb806a120 2023-11-08T19:25:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.915+0000 7f8bbf034640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb8144b80 con 0x7f8bb80cad30 2023-11-08T19:25:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.915+0000 7f8bbf034640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb8070040 con 0x7f8bb81447a0 2023-11-08T19:25:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.916+0000 7f8bbd5d1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8bb806a120 0x7f8bb806a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45504/0 (socket says 172.21.15.119:45504) 2023-11-08T19:25:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.916+0000 7f8bbd5d1640 1 -- 172.21.15.119:0/1614238125 learned_addr learned my addr 172.21.15.119:0/1614238125 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.916+0000 7f8bbcdd0640 1 --2- 172.21.15.119:0/1614238125 >> v2:172.21.15.188:3300/0 conn(0x7f8bb81447a0 0x7f8bb815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.916+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1290422128 0 0) 0x7f8bb80c3a50 con 0x7f8bb806a120 2023-11-08T19:25:03.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.916+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba0003520 con 0x7f8bb806a120 2023-11-08T19:25:03.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2571791327 0 0) 0x7f8ba0003520 con 0x7f8bb806a120 2023-11-08T19:25:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 >> v2:172.21.15.188:3300/0 conn(0x7f8bb81447a0 msgr2=0x7f8bb815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 --2- 172.21.15.119:0/1614238125 >> v2:172.21.15.188:3300/0 conn(0x7f8bb81447a0 0x7f8bb815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 >> v1:172.21.15.188:6789/0 conn(0x7f8bb80cad30 legacy=0x7f8bb806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb80c9150 con 0x7f8bb806a120 2023-11-08T19:25:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f8bb0001a10 con 0x7f8bb806a120 2023-11-08T19:25:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 >> v1:172.21.15.119:6789/0 conn(0x7f8bb806a120 legacy=0x7f8bb806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba00039c0 0x7f8ba0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba0005180 0x7f8ba000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb8070040 con 0x7f8ba00047c0 2023-11-08T19:25:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c080 con 0x7f8ba0005180 2023-11-08T19:25:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ba000c240 con 0x7f8ba00039c0 2023-11-08T19:25:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bbd5d1640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba0005180 0x7f8ba000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bbcdd0640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba00039c0 0x7f8ba0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb7fff640 1 -- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba00039c0 msgr2=0x7f8ba0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ba00039c0 0x7f8ba0003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb7fff640 1 -- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba0005180 msgr2=0x7f8ba000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ba0005180 0x7f8ba000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bb7fff640 1 -- 172.21.15.119:0/1614238125 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba0003520 con 0x7f8ba00047c0 2023-11-08T19:25:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.917+0000 7f8bbd5d1640 1 -- 172.21.15.119:0/1614238125 reap_dead start 2023-11-08T19:25:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.918+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001750 tx=0x7f8ba00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc832f819f66115 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80052d0 con 0x7f8ba00047c0 2023-11-08T19:25:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba802e050 con 0x7f8ba00047c0 2023-11-08T19:25:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bb77fe640 1 -- 172.21.15.119:0/1614238125 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8003830 con 0x7f8ba00047c0 2023-11-08T19:25:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bbf034640 1 -- 172.21.15.119:0/1614238125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 msgr2=0x7f8ba0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bbf034640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001750 tx=0x7f8ba00057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bbf034640 1 -- 172.21.15.119:0/1614238125 shutdown_connections 2023-11-08T19:25:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bbf034640 1 --2- 172.21.15.119:0/1614238125 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8ba0004c40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.919+0000 7f8bbf034640 1 -- 172.21.15.119:0/1614238125 >> 172.21.15.119:0/1614238125 conn(0x7f8bb80cb610 msgr2=0x7f8bb80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.920+0000 7f8bbf034640 1 -- 172.21.15.119:0/1614238125 shutdown_connections 2023-11-08T19:25:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.920+0000 7f8bbf034640 1 -- 172.21.15.119:0/1614238125 wait complete. 2023-11-08T19:25:03.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.921+0000 7f8bbf034640 1 Processor -- start 2023-11-08T19:25:03.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.921+0000 7f8bbf034640 1 -- start start 2023-11-08T19:25:03.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbf034640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8bb80c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbf034640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80c6d50 0x7f8bb8141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbf034640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb8141670 0x7f8bb8141ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbf034640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb81f0d50 con 0x7f8ba00047c0 2023-11-08T19:25:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbf034640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80c7900 con 0x7f8bb8141670 2023-11-08T19:25:03.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbf034640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb81f12a0 con 0x7f8bb80c6d50 2023-11-08T19:25:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.922+0000 7f8bbcdd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8bb80c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8bb80c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33298/0 (socket says 172.21.15.119:33298) 2023-11-08T19:25:03.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 -- 172.21.15.119:0/13957330 learned_addr learned my addr 172.21.15.119:0/13957330 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:03.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80c6d50 0x7f8bb8141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 -- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80c6d50 msgr2=0x7f8bb8141130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80c6d50 0x7f8bb8141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 -- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb8141670 msgr2=0x7f8bb8141ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:03.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb8141670 0x7f8bb8141ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:03.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 -- 172.21.15.119:0/13957330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8142010 con 0x7f8ba00047c0 2023-11-08T19:25:03.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbcdd0640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8bb80c6810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0107c0 tx=0x7f8bac010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac009630 con 0x7f8ba00047c0 2023-11-08T19:25:03.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac010ed0 con 0x7f8ba00047c0 2023-11-08T19:25:03.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac00ed80 con 0x7f8ba00047c0 2023-11-08T19:25:03.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.923+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb80c7340 con 0x7f8ba00047c0 2023-11-08T19:25:03.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.924+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb81f0f30 con 0x7f8ba00047c0 2023-11-08T19:25:03.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.925+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8074430 con 0x7f8ba00047c0 2023-11-08T19:25:03.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.926+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bac01c400 con 0x7f8ba00047c0 2023-11-08T19:25:03.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.926+0000 7f8bb57fa640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ba406aea0 0x7f8ba406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:03.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.927+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8bac089dc0 con 0x7f8ba00047c0 2023-11-08T19:25:03.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.927+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ba406aea0 0x7f8ba406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:03.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.927+0000 7f8bb7fff640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ba406aea0 0x7f8ba406d360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8033000 tx=0x7f8ba8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:03.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:03.931+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bac054b80 con 0x7f8ba00047c0 2023-11-08T19:25:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.144+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bb806caa0 con 0x7f8ba00047c0 2023-11-08T19:25:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.145+0000 7f8bb57fa640 1 -- 172.21.15.119:0/13957330 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f8bac0581a0 con 0x7f8ba00047c0 2023-11-08T19:25:04.147 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:04.147 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:03.150335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-08T19:25:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-11-08T19:25:04.148 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:25:04.147+0000 7f74f2178640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:25:04.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.147+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ba406aea0 msgr2=0x7f8ba406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.147+0000 7f8bbf034640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ba406aea0 0x7f8ba406d360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8033000 tx=0x7f8ba8009030 comp rx=0 tx=0).stop 2023-11-08T19:25:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.147+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 msgr2=0x7f8bb80c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.147+0000 7f8bbf034640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8bb80c6810 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0107c0 tx=0x7f8bac010c80 comp rx=0 tx=0).stop 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 shutdown_connections 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ba406aea0 0x7f8ba406d360 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8bb8141670 0x7f8bb8141ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8bb80c6d50 0x7f8bb8141130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 --2- 172.21.15.119:0/13957330 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ba00047c0 0x7f8bb80c6810 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 >> 172.21.15.119:0/13957330 conn(0x7f8bb80cb610 msgr2=0x7f8bb80c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 shutdown_connections 2023-11-08T19:25:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.148+0000 7f8bbf034640 1 -- 172.21.15.119:0/13957330 wait complete. 2023-11-08T19:25:04.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:04.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.368+0000 7f7885ce0640 1 Processor -- start 2023-11-08T19:25:04.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.368+0000 7f7885ce0640 1 -- start start 2023-11-08T19:25:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.369+0000 7f7885ce0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7880142f80 0x7f78800cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.369+0000 7f7885ce0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.369+0000 7f7885ce0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f788006cb40 0x7f788006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f7885ce0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7880143a40 con 0x7f7880142f80 2023-11-08T19:25:04.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f7885ce0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788006d510 con 0x7f788006cb40 2023-11-08T19:25:04.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f7885ce0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c4960 con 0x7f788015d9f0 2023-11-08T19:25:04.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7880142f80 0x7f78800cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7880142f80 0x7f78800cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33304/0 (socket says 172.21.15.119:33304) 2023-11-08T19:25:04.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 -- 172.21.15.119:0/2170873847 learned_addr learned my addr 172.21.15.119:0/2170873847 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 -- 172.21.15.119:0/2170873847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 msgr2=0x7f78800c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 --2- 172.21.15.119:0/2170873847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787ffff640 1 --2- 172.21.15.119:0/2170873847 >> v2:172.21.15.188:3300/0 conn(0x7f788006cb40 0x7f788006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 -- 172.21.15.119:0/2170873847 >> v2:172.21.15.188:3300/0 conn(0x7f788006cb40 msgr2=0x7f788006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 --2- 172.21.15.119:0/2170873847 >> v2:172.21.15.188:3300/0 conn(0x7f788006cb40 0x7f788006cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787f7fe640 1 -- 172.21.15.119:0/2170873847 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788015df60 con 0x7f7880142f80 2023-11-08T19:25:04.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.370+0000 7f787ffff640 1 --2- 172.21.15.119:0/2170873847 >> v2:172.21.15.188:3300/0 conn(0x7f788006cb40 0x7f788006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:04.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.371+0000 7f787f7fe640 1 --2- 172.21.15.119:0/2170873847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7880142f80 0x7f78800cc6d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78800cb550 tx=0x7f787002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61999d4fd21e2ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.372+0000 7f787effd640 1 -- 172.21.15.119:0/2170873847 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78700317e0 con 0x7f7880142f80 2023-11-08T19:25:04.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.372+0000 7f787effd640 1 -- 172.21.15.119:0/2170873847 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870031940 con 0x7f7880142f80 2023-11-08T19:25:04.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.372+0000 7f787effd640 1 -- 172.21.15.119:0/2170873847 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870031cd0 con 0x7f7880142f80 2023-11-08T19:25:04.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.372+0000 7f7885ce0640 1 -- 172.21.15.119:0/2170873847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7880142f80 msgr2=0x7f78800cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.372+0000 7f7885ce0640 1 --2- 172.21.15.119:0/2170873847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7880142f80 0x7f78800cc6d0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f78800cb550 tx=0x7f787002b880 comp rx=0 tx=0).stop 2023-11-08T19:25:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 -- 172.21.15.119:0/2170873847 shutdown_connections 2023-11-08T19:25:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 --2- 172.21.15.119:0/2170873847 >> v2:172.21.15.188:3300/0 conn(0x7f788006cb40 0x7f788006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 --2- 172.21.15.119:0/2170873847 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 --2- 172.21.15.119:0/2170873847 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7880142f80 0x7f78800cc6d0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 -- 172.21.15.119:0/2170873847 >> 172.21.15.119:0/2170873847 conn(0x7f78800cb610 msgr2=0x7f78800cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 -- 172.21.15.119:0/2170873847 shutdown_connections 2023-11-08T19:25:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.373+0000 7f7885ce0640 1 -- 172.21.15.119:0/2170873847 wait complete. 2023-11-08T19:25:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.374+0000 7f7885ce0640 1 Processor -- start 2023-11-08T19:25:04.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.374+0000 7f7885ce0640 1 -- start start 2023-11-08T19:25:04.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.375+0000 7f7885ce0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 0x7f78800affa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.375+0000 7f7885ce0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880142f80 0x7f78800c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.375+0000 7f7885ce0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.375+0000 7f7885ce0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78800c8540 con 0x7f788006cb40 2023-11-08T19:25:04.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.375+0000 7f7885ce0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78801f09b0 con 0x7f7880142f80 2023-11-08T19:25:04.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.375+0000 7f7885ce0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78800b04e0 con 0x7f788015d9f0 2023-11-08T19:25:04.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 0x7f78800affa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 0x7f78800affa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33306/0 (socket says 172.21.15.119:33306) 2023-11-08T19:25:04.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 -- 172.21.15.119:0/807622455 learned_addr learned my addr 172.21.15.119:0/807622455 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:04.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f7877fff640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880142f80 0x7f78800c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 -- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 msgr2=0x7f78800c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787ffff640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 -- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880142f80 msgr2=0x7f78800c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880142f80 0x7f78800c6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 -- 172.21.15.119:0/807622455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700094d0 con 0x7f788006cb40 2023-11-08T19:25:04.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f7877fff640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880142f80 0x7f78800c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:04.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.376+0000 7f787f7fe640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 0x7f78800affa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f787002b8c0 tx=0x7f78700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.377+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870032900 con 0x7f788006cb40 2023-11-08T19:25:04.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.377+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870032a60 con 0x7f788006cb40 2023-11-08T19:25:04.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.377+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78800c7930 con 0x7f788006cb40 2023-11-08T19:25:04.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.377+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870032e10 con 0x7f788006cb40 2023-11-08T19:25:04.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.377+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78800c7ef0 con 0x7f788006cb40 2023-11-08T19:25:04.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.378+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7880074430 con 0x7f788006cb40 2023-11-08T19:25:04.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.379+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78700453f0 con 0x7f788006cb40 2023-11-08T19:25:04.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.380+0000 7f787cff9640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f786006ae70 0x7f786006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.380+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f78700abd50 con 0x7f788006cb40 2023-11-08T19:25:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.380+0000 7f7877fff640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f786006ae70 0x7f786006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.381+0000 7f7877fff640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f786006ae70 0x7f786006d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f788015d170 tx=0x7f7868008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.384+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78700af030 con 0x7f788006cb40 2023-11-08T19:25:04.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.573+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78801421d0 con 0x7f788006cb40 2023-11-08T19:25:04.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.576+0000 7f787cff9640 1 -- 172.21.15.119:0/807622455 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18359 (secure 0 0 0) 0x7f7870076c30 con 0x7f788006cb40 2023-11-08T19:25:04.578 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:04.578 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":24,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:03.149898+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f786006ae70 msgr2=0x7f786006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f786006ae70 0x7f786006d330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f788015d170 tx=0x7f7868008040 comp rx=0 tx=0).stop 2023-11-08T19:25:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 msgr2=0x7f78800affa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 0x7f78800affa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f787002b8c0 tx=0x7f78700046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 shutdown_connections 2023-11-08T19:25:04.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f786006ae70 0x7f786006d330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f788015d9f0 0x7f78800c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7880142f80 0x7f78800c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 --2- 172.21.15.119:0/807622455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f788006cb40 0x7f78800affa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.578+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 >> 172.21.15.119:0/807622455 conn(0x7f78800cb610 msgr2=0x7f788006ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.579+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 shutdown_connections 2023-11-08T19:25:04.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.579+0000 7f7885ce0640 1 -- 172.21.15.119:0/807622455 wait complete. 2023-11-08T19:25:04.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:04.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.795+0000 7ff2a701a640 1 Processor -- start 2023-11-08T19:25:04.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.796+0000 7ff2a701a640 1 -- start start 2023-11-08T19:25:04.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a701a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2a01447a0 0x7ff2a015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a701a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a00c3a50 con 0x7ff2a006a120 2023-11-08T19:25:04.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a701a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a0144b80 con 0x7ff2a00cad30 2023-11-08T19:25:04.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a701a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a0070040 con 0x7ff2a01447a0 2023-11-08T19:25:04.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a4db6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2a01447a0 0x7ff2a015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a4db6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff2a01447a0 0x7ff2a015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33308/0 (socket says 172.21.15.119:33308) 2023-11-08T19:25:04.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.797+0000 7ff2a4db6640 1 -- 172.21.15.119:0/2308883889 learned_addr learned my addr 172.21.15.119:0/2308883889 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:04.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.798+0000 7ff2a4db6640 1 -- 172.21.15.119:0/2308883889 >> v1:172.21.15.188:6789/0 conn(0x7ff2a00cad30 legacy=0x7ff2a006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:04.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.798+0000 7ff2a4db6640 1 -- 172.21.15.119:0/2308883889 >> v1:172.21.15.119:6789/0 conn(0x7ff2a006a120 legacy=0x7ff2a006a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.798+0000 7ff2a4db6640 1 -- 172.21.15.119:0/2308883889 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a00c9150 con 0x7ff2a01447a0 2023-11-08T19:25:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.799+0000 7ff2a4db6640 1 --2- 172.21.15.119:0/2308883889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a015f4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff294009870 tx=0x7ff29402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270075a502561c00 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.799+0000 7ff29f7fe640 1 -- 172.21.15.119:0/2308883889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294030860 con 0x7ff2a01447a0 2023-11-08T19:25:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.799+0000 7ff29f7fe640 1 -- 172.21.15.119:0/2308883889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2940309c0 con 0x7ff2a01447a0 2023-11-08T19:25:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.799+0000 7ff29f7fe640 1 -- 172.21.15.119:0/2308883889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff294030d50 con 0x7ff2a01447a0 2023-11-08T19:25:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 -- 172.21.15.119:0/2308883889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 msgr2=0x7ff2a015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2308883889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a015f4c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff294009870 tx=0x7ff29402b9a0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 -- 172.21.15.119:0/2308883889 shutdown_connections 2023-11-08T19:25:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2308883889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a015f4c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 -- 172.21.15.119:0/2308883889 >> 172.21.15.119:0/2308883889 conn(0x7ff2a00cb610 msgr2=0x7ff2a00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 -- 172.21.15.119:0/2308883889 shutdown_connections 2023-11-08T19:25:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.800+0000 7ff2a701a640 1 -- 172.21.15.119:0/2308883889 wait complete. 2023-11-08T19:25:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.802+0000 7ff2a701a640 1 Processor -- start 2023-11-08T19:25:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.802+0000 7ff2a701a640 1 -- start start 2023-11-08T19:25:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a701a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2a006a120 0x7ff2a00b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a701a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2a00cad30 0x7ff2a014a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a701a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a0139560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a701a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00b4950 con 0x7ff2a01447a0 2023-11-08T19:25:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a701a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00c7580 con 0x7ff2a00cad30 2023-11-08T19:25:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a701a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a00c6cc0 con 0x7ff2a006a120 2023-11-08T19:25:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a55b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a0139560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a55b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a0139560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33322/0 (socket says 172.21.15.119:33322) 2023-11-08T19:25:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a55b7640 1 -- 172.21.15.119:0/2893853818 learned_addr learned my addr 172.21.15.119:0/2893853818 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a4db6640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2a006a120 0x7ff2a00b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff29ffff640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2a00cad30 0x7ff2a014a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.803+0000 7ff2a55b7640 1 -- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2a006a120 msgr2=0x7ff2a00b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a55b7640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2a006a120 0x7ff2a00b4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a55b7640 1 -- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2a00cad30 msgr2=0x7ff2a014a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a55b7640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2a00cad30 0x7ff2a014a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a55b7640 1 -- 172.21.15.119:0/2893853818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940094d0 con 0x7ff2a01447a0 2023-11-08T19:25:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a55b7640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a0139560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c730 tx=0x7ff29800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298003b90 con 0x7ff2a01447a0 2023-11-08T19:25:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a00c6f30 con 0x7ff2a01447a0 2023-11-08T19:25:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298003cf0 con 0x7ff2a01447a0 2023-11-08T19:25:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298005310 con 0x7ff2a01447a0 2023-11-08T19:25:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.804+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a014a660 con 0x7ff2a01447a0 2023-11-08T19:25:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.805+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a0074430 con 0x7ff2a01447a0 2023-11-08T19:25:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.806+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff298005680 con 0x7ff2a01447a0 2023-11-08T19:25:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.808+0000 7ff29d7fa640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff28406ae70 0x7ff28406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.808+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff2980898b0 con 0x7ff2a01447a0 2023-11-08T19:25:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.808+0000 7ff2a4db6640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff28406ae70 0x7ff28406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.809+0000 7ff2a4db6640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff28406ae70 0x7ff28406d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff294036000 tx=0x7ff29403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:04.812+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff298054660 con 0x7ff2a01447a0 2023-11-08T19:25:05.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.003+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2a00c76c0 con 0x7ff2a01447a0 2023-11-08T19:25:05.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.006+0000 7ff29d7fa640 1 -- 172.21.15.119:0/2893853818 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7ff298003e60 con 0x7ff2a01447a0 2023-11-08T19:25:05.008 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:05.008 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:05.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.008+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff28406ae70 msgr2=0x7ff28406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.008+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff28406ae70 0x7ff28406d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff294036000 tx=0x7ff29403b040 comp rx=0 tx=0).stop 2023-11-08T19:25:05.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.008+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 msgr2=0x7ff2a0139560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.008+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a0139560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff29800c730 tx=0x7ff29800cbf0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 shutdown_connections 2023-11-08T19:25:05.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff28406ae70 0x7ff28406d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff2a01447a0 0x7ff2a0139560 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff2a00cad30 0x7ff2a014a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 --2- 172.21.15.119:0/2893853818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff2a006a120 0x7ff2a00b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 >> 172.21.15.119:0/2893853818 conn(0x7ff2a00cb610 msgr2=0x7ff2a015b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 shutdown_connections 2023-11-08T19:25:05.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.009+0000 7ff2a701a640 1 -- 172.21.15.119:0/2893853818 wait complete. 2023-11-08T19:25:05.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:05.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.260+0000 7fc005127640 1 Processor -- start 2023-11-08T19:25:05.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.260+0000 7fc005127640 1 -- start start 2023-11-08T19:25:05.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.261+0000 7fc005127640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc0000ad7d0 0x7fc000070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.261+0000 7fc005127640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0000710a0 0x7fc0000c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.261+0000 7fc005127640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc00006cff0 con 0x7fc00015b0d0 2023-11-08T19:25:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.261+0000 7fc005127640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00006d2d0 con 0x7fc0000710a0 2023-11-08T19:25:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.261+0000 7fc005127640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc00006bd20 con 0x7fc0000ad7d0 2023-11-08T19:25:05.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0000710a0 0x7fc0000c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0000710a0 0x7fc0000c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33330/0 (socket says 172.21.15.119:33330) 2023-11-08T19:25:05.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 -- 172.21.15.119:0/3147225618 learned_addr learned my addr 172.21.15.119:0/3147225618 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:05.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fbfff7fe640 1 --2- 172.21.15.119:0/3147225618 >> v2:172.21.15.188:3300/0 conn(0x7fc0000ad7d0 0x7fc000070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 -- 172.21.15.119:0/3147225618 >> v2:172.21.15.188:3300/0 conn(0x7fc0000ad7d0 msgr2=0x7fc000070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 --2- 172.21.15.119:0/3147225618 >> v2:172.21.15.188:3300/0 conn(0x7fc0000ad7d0 0x7fc000070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 -- 172.21.15.119:0/3147225618 >> v1:172.21.15.188:6789/0 conn(0x7fc00015b0d0 legacy=0x7fc00015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.262+0000 7fc004926640 1 -- 172.21.15.119:0/3147225618 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00006d760 con 0x7fc0000710a0 2023-11-08T19:25:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.263+0000 7fc004926640 1 --2- 172.21.15.119:0/3147225618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0000710a0 0x7fc0000c98c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc0000c2570 tx=0x7fbff802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c26fb3c15d700b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fbffeffd640 1 -- 172.21.15.119:0/3147225618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff802c9f0 con 0x7fc0000710a0 2023-11-08T19:25:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fbffeffd640 1 -- 172.21.15.119:0/3147225618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff802cb50 con 0x7fc0000710a0 2023-11-08T19:25:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fbffeffd640 1 -- 172.21.15.119:0/3147225618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff80343f0 con 0x7fc0000710a0 2023-11-08T19:25:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fc005127640 1 -- 172.21.15.119:0/3147225618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0000710a0 msgr2=0x7fc0000c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fc005127640 1 --2- 172.21.15.119:0/3147225618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0000710a0 0x7fc0000c98c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc0000c2570 tx=0x7fbff802be50 comp rx=0 tx=0).stop 2023-11-08T19:25:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fc005127640 1 -- 172.21.15.119:0/3147225618 shutdown_connections 2023-11-08T19:25:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fc005127640 1 --2- 172.21.15.119:0/3147225618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0000710a0 0x7fc0000c98c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.264+0000 7fc005127640 1 --2- 172.21.15.119:0/3147225618 >> v2:172.21.15.188:3300/0 conn(0x7fc0000ad7d0 0x7fc000070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.265+0000 7fc005127640 1 -- 172.21.15.119:0/3147225618 >> 172.21.15.119:0/3147225618 conn(0x7fc0000b0560 msgr2=0x7fc0000b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.265+0000 7fc005127640 1 -- 172.21.15.119:0/3147225618 shutdown_connections 2023-11-08T19:25:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.265+0000 7fc005127640 1 -- 172.21.15.119:0/3147225618 wait complete. 2023-11-08T19:25:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.266+0000 7fc005127640 1 Processor -- start 2023-11-08T19:25:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.267+0000 7fc005127640 1 -- start start 2023-11-08T19:25:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.267+0000 7fc005127640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000710a0 0x7fc000155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc005127640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0000ad7d0 0x7fc0000acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc005127640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 0x7fc000156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc005127640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc000154330 con 0x7fc00015b0d0 2023-11-08T19:25:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc005127640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc000153d70 con 0x7fc0000710a0 2023-11-08T19:25:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc005127640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc000154580 con 0x7fc0000ad7d0 2023-11-08T19:25:05.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 0x7fc000156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 0x7fc000156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33336/0 (socket says 172.21.15.119:33336) 2023-11-08T19:25:05.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 -- 172.21.15.119:0/2189719103 learned_addr learned my addr 172.21.15.119:0/2189719103 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:05.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fbfff7fe640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0000ad7d0 0x7fc0000acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fbffffff640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000710a0 0x7fc000155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 -- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0000ad7d0 msgr2=0x7fc0000acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0000ad7d0 0x7fc0000acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 -- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000710a0 msgr2=0x7fc000155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000710a0 0x7fc000155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fc004926640 1 -- 172.21.15.119:0/2189719103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff8009b30 con 0x7fc00015b0d0 2023-11-08T19:25:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.268+0000 7fbffffff640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000710a0 0x7fc000155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.269+0000 7fc004926640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 0x7fc000156150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbff8036040 tx=0x7fbff8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.269+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8009510 con 0x7fc00015b0d0 2023-11-08T19:25:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.269+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff803e450 con 0x7fc00015b0d0 2023-11-08T19:25:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.269+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff803e7e0 con 0x7fc00015b0d0 2023-11-08T19:25:05.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.269+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc000154860 con 0x7fc00015b0d0 2023-11-08T19:25:05.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.269+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc000154e20 con 0x7fc00015b0d0 2023-11-08T19:25:05.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.270+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc00004ece0 con 0x7fc00015b0d0 2023-11-08T19:25:05.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.271+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbff803e940 con 0x7fc00015b0d0 2023-11-08T19:25:05.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.272+0000 7fbffcff9640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfec06ae70 0x7fbfec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.272+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbff80ac770 con 0x7fc00015b0d0 2023-11-08T19:25:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.272+0000 7fbffffff640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfec06ae70 0x7fbfec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.273+0000 7fbffffff640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfec06ae70 0x7fbfec06d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbff0003eb0 tx=0x7fbff0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.276+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff8077490 con 0x7fc00015b0d0 2023-11-08T19:25:05.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.497+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0000b1320 con 0x7fc00015b0d0 2023-11-08T19:25:05.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.499+0000 7fbffcff9640 1 -- 172.21.15.119:0/2189719103 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fbff807aab0 con 0x7fc00015b0d0 2023-11-08T19:25:05.500 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:05.500 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:03.150335+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-08T19:25:05.501 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 7 2023-11-08T19:25:05.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.501+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfec06ae70 msgr2=0x7fbfec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.501+0000 7fc005127640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfec06ae70 0x7fbfec06d330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbff0003eb0 tx=0x7fbff0009040 comp rx=0 tx=0).stop 2023-11-08T19:25:05.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.501+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 msgr2=0x7fc000156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.501+0000 7fc005127640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 0x7fc000156150 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbff8036040 tx=0x7fbff8004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:05.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 shutdown_connections 2023-11-08T19:25:05.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfec06ae70 0x7fbfec06d330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc00015b0d0 0x7fc000156150 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0000ad7d0 0x7fc0000acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 --2- 172.21.15.119:0/2189719103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0000710a0 0x7fc000155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 >> 172.21.15.119:0/2189719103 conn(0x7fc0000b0560 msgr2=0x7fc0000c7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 shutdown_connections 2023-11-08T19:25:05.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.502+0000 7fc005127640 1 -- 172.21.15.119:0/2189719103 wait complete. 2023-11-08T19:25:05.523 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:05.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.716+0000 7ff8827c5640 1 Processor -- start 2023-11-08T19:25:05.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.716+0000 7ff8827c5640 1 -- start start 2023-11-08T19:25:05.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff8827c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c0cc810 0x7ff87c0c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff8827c5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff87c15d180 0x7ff87c06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff8827c5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff87c0ae410 con 0x7ff87c06a390 2023-11-08T19:25:05.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff8827c5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff87c06ba30 con 0x7ff87c15d180 2023-11-08T19:25:05.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff8827c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c06bcf0 con 0x7ff87c0cc810 2023-11-08T19:25:05.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff87bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff87c06a390 0x7ff87c15ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45526/0 (socket says 172.21.15.119:45526) 2023-11-08T19:25:05.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff87bfff640 1 -- 172.21.15.119:0/4127485097 learned_addr learned my addr 172.21.15.119:0/4127485097 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:05.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/4127485097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c0cc810 0x7ff87c0c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.718+0000 7ff880d62640 1 --2- 172.21.15.119:0/4127485097 >> v2:172.21.15.188:3300/0 conn(0x7ff87c15d180 0x7ff87c06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87affd640 1 -- 172.21.15.119:0/4127485097 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2997210569 0 0) 0x7ff87c0ae410 con 0x7ff87c06a390 2023-11-08T19:25:05.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87affd640 1 -- 172.21.15.119:0/4127485097 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff864003520 con 0x7ff87c06a390 2023-11-08T19:25:05.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87b7fe640 1 -- 172.21.15.119:0/4127485097 >> v2:172.21.15.188:3300/0 conn(0x7ff87c15d180 msgr2=0x7ff87c06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:05.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/4127485097 >> v2:172.21.15.188:3300/0 conn(0x7ff87c15d180 0x7ff87c06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87b7fe640 1 -- 172.21.15.119:0/4127485097 >> v1:172.21.15.119:6789/0 conn(0x7ff87c06a390 legacy=0x7ff87c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87affd640 1 -- 172.21.15.119:0/4127485097 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3100048632 0 0) 0x7ff864003520 con 0x7ff87c06a390 2023-11-08T19:25:05.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff87b7fe640 1 -- 172.21.15.119:0/4127485097 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff87c06bfd0 con 0x7ff87c0cc810 2023-11-08T19:25:05.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.719+0000 7ff880d62640 1 --2- 172.21.15.119:0/4127485097 >> v2:172.21.15.188:3300/0 conn(0x7ff87c15d180 0x7ff87c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:05.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.720+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/4127485097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c0cc810 0x7ff87c0c4460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff8700097a0 tx=0x7ff87002b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb264f3b33a22d74 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.721+0000 7ff87affd640 1 -- 172.21.15.119:0/4127485097 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86c001670 con 0x7ff87c0cc810 2023-11-08T19:25:05.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.721+0000 7ff87affd640 1 -- 172.21.15.119:0/4127485097 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87002e050 con 0x7ff87c0cc810 2023-11-08T19:25:05.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.721+0000 7ff87affd640 1 -- 172.21.15.119:0/4127485097 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff870030730 con 0x7ff87c0cc810 2023-11-08T19:25:05.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 -- 172.21.15.119:0/4127485097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c0cc810 msgr2=0x7ff87c0c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 --2- 172.21.15.119:0/4127485097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c0cc810 0x7ff87c0c4460 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff8700097a0 tx=0x7ff87002b790 comp rx=0 tx=0).stop 2023-11-08T19:25:05.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 -- 172.21.15.119:0/4127485097 shutdown_connections 2023-11-08T19:25:05.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 --2- 172.21.15.119:0/4127485097 >> v2:172.21.15.188:3300/0 conn(0x7ff87c15d180 0x7ff87c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 --2- 172.21.15.119:0/4127485097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c0cc810 0x7ff87c0c4460 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 -- 172.21.15.119:0/4127485097 >> 172.21.15.119:0/4127485097 conn(0x7ff87c0cb610 msgr2=0x7ff87c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 -- 172.21.15.119:0/4127485097 shutdown_connections 2023-11-08T19:25:05.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.722+0000 7ff8827c5640 1 -- 172.21.15.119:0/4127485097 wait complete. 2023-11-08T19:25:05.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.723+0000 7ff8827c5640 1 Processor -- start 2023-11-08T19:25:05.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.724+0000 7ff8827c5640 1 -- start start 2023-11-08T19:25:05.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.724+0000 7ff8827c5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff87c06a390 0x7ff87c0b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.724+0000 7ff8827c5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 0x7ff87c1f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff8827c5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c15d180 0x7ff87c14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff8827c5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1f7a20 con 0x7ff87c0cc810 2023-11-08T19:25:05.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff8827c5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c14ae50 con 0x7ff87c06a390 2023-11-08T19:25:05.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff8827c5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff87c1f4710 con 0x7ff87c15d180 2023-11-08T19:25:05.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 0x7ff87c1f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 0x7ff87c1f74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33352/0 (socket says 172.21.15.119:33352) 2023-11-08T19:25:05.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2806905033 learned_addr learned my addr 172.21.15.119:0/2806905033 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:05.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2806905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c15d180 msgr2=0x7ff87c14a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c15d180 0x7ff87c14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2806905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff87c06a390 msgr2=0x7ff87c0b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff87c06a390 0x7ff87c0b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 -- 172.21.15.119:0/2806905033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff874009a70 con 0x7ff87c0cc810 2023-11-08T19:25:05.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.725+0000 7ff87b7fe640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 0x7ff87c1f74e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff870000c00 tx=0x7ff8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.726+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87003b810 con 0x7ff87c0cc810 2023-11-08T19:25:05.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.726+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff87003b970 con 0x7ff87c0cc810 2023-11-08T19:25:05.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.726+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff87003bd20 con 0x7ff87c0cc810 2023-11-08T19:25:05.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.726+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff870009400 con 0x7ff87c0cc810 2023-11-08T19:25:05.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.726+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff87c1f3950 con 0x7ff87c0cc810 2023-11-08T19:25:05.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.727+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87c074430 con 0x7ff87c0cc810 2023-11-08T19:25:05.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.728+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff870039040 con 0x7ff87c0cc810 2023-11-08T19:25:05.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.728+0000 7ff878ff9640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff86806ae70 0x7ff86806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:05.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.729+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff8700ab070 con 0x7ff87c0cc810 2023-11-08T19:25:05.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.729+0000 7ff87bfff640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff86806ae70 0x7ff86806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:05.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.729+0000 7ff87bfff640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff86806ae70 0x7ff86806d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff86c001e20 tx=0x7ff86c001250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:05.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.733+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8700ddca0 con 0x7ff87c0cc810 2023-11-08T19:25:05.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.922+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff87c06cd50 con 0x7ff87c0cc810 2023-11-08T19:25:05.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.925+0000 7ff878ff9640 1 -- 172.21.15.119:0/2806905033 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7ff87004b090 con 0x7ff87c0cc810 2023-11-08T19:25:05.927 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:05.928 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:05.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.928+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff86806ae70 msgr2=0x7ff86806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.928+0000 7ff8827c5640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff86806ae70 0x7ff86806d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff86c001e20 tx=0x7ff86c001250 comp rx=0 tx=0).stop 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.928+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 msgr2=0x7ff87c1f74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.928+0000 7ff8827c5640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 0x7ff87c1f74e0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff870000c00 tx=0x7ff8700046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 shutdown_connections 2023-11-08T19:25:05.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff86806ae70 0x7ff86806d330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff87c15d180 0x7ff87c14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff87c0cc810 0x7ff87c1f74e0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 --2- 172.21.15.119:0/2806905033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff87c06a390 0x7ff87c0b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 >> 172.21.15.119:0/2806905033 conn(0x7ff87c0cb610 msgr2=0x7ff87c15b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:05.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 shutdown_connections 2023-11-08T19:25:05.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:05.929+0000 7ff8827c5640 1 -- 172.21.15.119:0/2806905033 wait complete. 2023-11-08T19:25:06.351 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:24:58.104296+0000', 'modified': '2023-11-08T19:25:03.150335+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4254}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4254': {'gid': 4254, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/2629712656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 2629712656}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 2629712656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-08T19:25:06.352 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:25:06.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:25:06.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.573+0000 7f1324ec3640 1 Processor -- start 2023-11-08T19:25:06.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.573+0000 7f1324ec3640 1 -- start start 2023-11-08T19:25:06.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f1324ec3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f132015d310 0x7f1320071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f1324ec3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132006d6e0 con 0x7f13200cc640 2023-11-08T19:25:06.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f1324ec3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13200c90c0 con 0x7f13200ad8b0 2023-11-08T19:25:06.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f1324ec3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13200c93a0 con 0x7f132015d310 2023-11-08T19:25:06.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f131f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f132015d310 0x7f1320071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f131f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f132015d310 0x7f1320071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33366/0 (socket says 172.21.15.119:33366) 2023-11-08T19:25:06.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f131f7fe640 1 -- 172.21.15.119:0/102784692 learned_addr learned my addr 172.21.15.119:0/102784692 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f131ffff640 1 --1- 172.21.15.119:0/102784692 >> v1:172.21.15.119:6789/0 conn(0x7f13200cc640 0x7f13200c9910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45536/0 (socket says 172.21.15.119:45536) 2023-11-08T19:25:06.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.575+0000 7f131f7fe640 1 -- 172.21.15.119:0/102784692 >> v1:172.21.15.188:6789/0 conn(0x7f13200ad8b0 legacy=0x7f1320070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.576+0000 7f131f7fe640 1 -- 172.21.15.119:0/102784692 >> v1:172.21.15.119:6789/0 conn(0x7f13200cc640 legacy=0x7f13200c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.576+0000 7f131f7fe640 1 -- 172.21.15.119:0/102784692 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13200c9680 con 0x7f132015d310 2023-11-08T19:25:06.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.576+0000 7f131effd640 1 -- 172.21.15.119:0/102784692 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208921755 0 0) 0x7f132006d6e0 con 0x7f13200cc640 2023-11-08T19:25:06.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.577+0000 7f131f7fe640 1 --2- 172.21.15.119:0/102784692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f132015d310 0x7f1320071840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7c18a2bc7cf42d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.577+0000 7f131effd640 1 -- 172.21.15.119:0/102784692 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310030850 con 0x7f132015d310 2023-11-08T19:25:06.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.577+0000 7f131effd640 1 -- 172.21.15.119:0/102784692 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13100309b0 con 0x7f132015d310 2023-11-08T19:25:06.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.577+0000 7f131effd640 1 -- 172.21.15.119:0/102784692 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310030d40 con 0x7f132015d310 2023-11-08T19:25:06.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.577+0000 7f1324ec3640 1 -- 172.21.15.119:0/102784692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f132015d310 msgr2=0x7f1320071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.577+0000 7f1324ec3640 1 --2- 172.21.15.119:0/102784692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f132015d310 0x7f1320071840 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f1310009870 tx=0x7f131002b990 comp rx=0 tx=0).stop 2023-11-08T19:25:06.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.578+0000 7f1324ec3640 1 -- 172.21.15.119:0/102784692 shutdown_connections 2023-11-08T19:25:06.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.578+0000 7f1324ec3640 1 --2- 172.21.15.119:0/102784692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f132015d310 0x7f1320071840 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.578+0000 7f1324ec3640 1 -- 172.21.15.119:0/102784692 >> 172.21.15.119:0/102784692 conn(0x7f13200b0560 msgr2=0x7f13200b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.578+0000 7f1324ec3640 1 -- 172.21.15.119:0/102784692 shutdown_connections 2023-11-08T19:25:06.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.578+0000 7f1324ec3640 1 -- 172.21.15.119:0/102784692 wait complete. 2023-11-08T19:25:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.579+0000 7f1324ec3640 1 Processor -- start 2023-11-08T19:25:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.580+0000 7f1324ec3640 1 -- start start 2023-11-08T19:25:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f1324ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 0x7f13200cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f1324ec3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13200cc640 0x7f132014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f1324ec3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132015d310 0x7f132014cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f1324ec3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13201f45f0 con 0x7f13200ad8b0 2023-11-08T19:25:06.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f1324ec3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13201f3a00 con 0x7f13200cc640 2023-11-08T19:25:06.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f1324ec3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13201f3c50 con 0x7f132015d310 2023-11-08T19:25:06.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 0x7f13200cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 0x7f13200cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33376/0 (socket says 172.21.15.119:33376) 2023-11-08T19:25:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 -- 172.21.15.119:0/3874418302 learned_addr learned my addr 172.21.15.119:0/3874418302 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 -- 172.21.15.119:0/3874418302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132015d310 msgr2=0x7f132014cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132015d310 0x7f132014cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 -- 172.21.15.119:0/3874418302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13200cc640 msgr2=0x7f132014c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13200cc640 0x7f132014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.581+0000 7f131f7fe640 1 -- 172.21.15.119:0/3874418302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100094d0 con 0x7f13200ad8b0 2023-11-08T19:25:06.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.582+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 0x7f13200cceb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1310000c00 tx=0x7f1310004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.582+0000 7f131cff9640 1 -- 172.21.15.119:0/3874418302 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310039b60 con 0x7f13200ad8b0 2023-11-08T19:25:06.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.582+0000 7f131cff9640 1 -- 172.21.15.119:0/3874418302 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1310039cc0 con 0x7f13200ad8b0 2023-11-08T19:25:06.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.582+0000 7f131cff9640 1 -- 172.21.15.119:0/3874418302 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310043650 con 0x7f13200ad8b0 2023-11-08T19:25:06.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.582+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13201f3f30 con 0x7f13200ad8b0 2023-11-08T19:25:06.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.582+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13200cd3f0 con 0x7f13200ad8b0 2023-11-08T19:25:06.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.584+0000 7f131cff9640 1 -- 172.21.15.119:0/3874418302 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f131002e070 con 0x7f13200ad8b0 2023-11-08T19:25:06.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.585+0000 7f131cff9640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f130006ae70 0x7f130006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.585+0000 7f131cff9640 1 -- 172.21.15.119:0/3874418302 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f13100aba20 con 0x7f13200ad8b0 2023-11-08T19:25:06.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.585+0000 7f1317fff640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f130006ae70 0x7f130006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.586+0000 7f1328d8c740 1 Processor -- start 2023-11-08T19:25:06.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.586+0000 7f1328d8c740 1 -- start start 2023-11-08T19:25:06.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.586+0000 7f1317fff640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f130006ae70 0x7f130006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13200cdb60 tx=0x7f13080079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c306f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 0x559e9c336e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c3074c0 0x559e9c337350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1328d8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c2797e0 con 0x559e9c334460 2023-11-08T19:25:06.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c276ed0 con 0x559e9c334080 2023-11-08T19:25:06.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e9c2727c0 con 0x559e9c3074c0 2023-11-08T19:25:06.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1317fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 0x559e9c336e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1317fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 0x559e9c336e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33378/0 (socket says 172.21.15.119:33378) 2023-11-08T19:25:06.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f1317fff640 1 -- 172.21.15.119:0/3381895722 learned_addr learned my addr 172.21.15.119:0/3381895722 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f131ffff640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c3074c0 0x559e9c337350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.587+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c306f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f1317fff640 1 -- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c3074c0 msgr2=0x559e9c337350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f1317fff640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c3074c0 0x559e9c337350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f1317fff640 1 -- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 msgr2=0x559e9c306f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f1317fff640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c306f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f1317fff640 1 -- 172.21.15.119:0/3381895722 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9c245b10 con 0x559e9c334460 2023-11-08T19:25:06.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f131ffff640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c3074c0 0x559e9c337350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:06.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c306f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:06.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f1317fff640 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 0x559e9c336e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1308005510 tx=0x7f130800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600bce0e4ddc64cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f13157fa640 1 -- 172.21.15.119:0/3381895722 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308005060 con 0x559e9c334460 2023-11-08T19:25:06.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f13157fa640 1 -- 172.21.15.119:0/3381895722 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308017850 con 0x559e9c334460 2023-11-08T19:25:06.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.588+0000 7f13157fa640 1 -- 172.21.15.119:0/3381895722 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308017c00 con 0x559e9c334460 2023-11-08T19:25:06.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 -- 172.21.15.119:0/3381895722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 msgr2=0x559e9c336e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 0x559e9c336e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1308005510 tx=0x7f130800f000 comp rx=0 tx=0).stop 2023-11-08T19:25:06.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 -- 172.21.15.119:0/3381895722 shutdown_connections 2023-11-08T19:25:06.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c3074c0 0x559e9c337350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334460 0x559e9c336e10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3381895722 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c306f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 -- 172.21.15.119:0/3381895722 >> 172.21.15.119:0/3381895722 conn(0x559e9c309290 msgr2=0x559e9c351e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 -- 172.21.15.119:0/3381895722 shutdown_connections 2023-11-08T19:25:06.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.589+0000 7f1328d8c740 1 -- 172.21.15.119:0/3381895722 wait complete. 2023-11-08T19:25:06.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.590+0000 7f1328d8c740 1 -- start start 2023-11-08T19:25:06.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.590+0000 7f1328d8c740 1 Processor -- start 2023-11-08T19:25:06.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c34c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c34ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c355720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c20fc40 con 0x559e9c334080 2023-11-08T19:25:06.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c20f8e0 con 0x559e9c34bc60 2023-11-08T19:25:06.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e9bfd5f20 con 0x559e9c334460 2023-11-08T19:25:06.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f1328d8c740 10 client.0 _ll_register_callbacks cb 0x559e9c231cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:25:06.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c34c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c34c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33384/0 (socket says 172.21.15.119:33384) 2023-11-08T19:25:06.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f131f7fe640 1 -- 172.21.15.119:0/682330055 learned_addr learned my addr 172.21.15.119:0/682330055 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f131f7fe640 1 -- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 msgr2=0x559e9c34ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.591+0000 7f131f7fe640 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c34ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f131f7fe640 1 -- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 msgr2=0x559e9c355720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f131f7fe640 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c355720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f131f7fe640 1 -- 172.21.15.119:0/682330055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13080051c0 con 0x559e9c334080 2023-11-08T19:25:06.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f131f7fe640 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c34c440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f131002bfa0 tx=0x7f1310039f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c59330096e3027e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f12fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:25:06.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f12fe7fc640 1 -- 172.21.15.119:0/682330055 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131007e370 con 0x559e9c334080 2023-11-08T19:25:06.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f12fe7fc640 1 -- 172.21.15.119:0/682330055 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1310082790 con 0x559e9c334080 2023-11-08T19:25:06.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e9c246c80 con 0x559e9c334080 2023-11-08T19:25:06.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f12fe7fc640 1 -- 172.21.15.119:0/682330055 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1310079980 con 0x559e9c334080 2023-11-08T19:25:06.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f1328d8c740 10 client.4386 fetch_fsmap learned FSMap version 7 2023-11-08T19:25:06.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x559e9c270e40 con 0x559e9c334080 2023-11-08T19:25:06.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.592+0000 7f12fe7fc640 1 -- 172.21.15.119:0/682330055 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559e9c246c80 con 0x559e9c334080 2023-11-08T19:25:06.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f12fe7fc640 1 -- 172.21.15.119:0/682330055 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1310082300 con 0x559e9c334080 2023-11-08T19:25:06.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1328d8c740 10 client.4386 fetch_fsmap finished waiting for FSMap version 7 2023-11-08T19:25:06.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1328d8c740 10 client.4386 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:25:06.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1328d8c740 20 client.4386 populate_metadata read hostname 'smithi119' 2023-11-08T19:25:06.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1328d8c740 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c3396c0 0x559e9c365220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1328d8c740 4 client.4386 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:25:06.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 --> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e9c26b7a0 con 0x559e9c3396c0 2023-11-08T19:25:06.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.593+0000 7f1317fff640 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c3396c0 0x559e9c365220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.594+0000 7f1317fff640 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c3396c0 0x559e9c365220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded2ef15d954d369 server_cookie=595ca312bc3b3ec1 in_seq=0 out_seq=0 2023-11-08T19:25:06.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.594+0000 7f12fe7fc640 10 client.4386 ms_handle_connect on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:06.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f12fe7fc640 1 -- 172.21.15.119:0/682330055 <== mds.0 v2:172.21.15.188:6833/2629712656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559e9c26b7a0 con 0x559e9c3396c0 2023-11-08T19:25:06.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f12fe7fc640 10 client.4386 handle_command_reply: tid=0 2023-11-08T19:25:06.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 client.4386 shutdown 2023-11-08T19:25:06.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 msgr2=0x559e9c34c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c34c440 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f131002bfa0 tx=0x7f1310039f70 comp rx=0 tx=0).stop 2023-11-08T19:25:06.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 shutdown_connections 2023-11-08T19:25:06.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c3396c0 0x559e9c365220 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c355720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c34ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 --2- 172.21.15.119:0/682330055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c34c440 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f12fe7fc640 0 client.4386 ms_handle_reset on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:06.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 >> 172.21.15.119:0/682330055 conn(0x559e9c3074c0 msgr2=0x559e9c3066d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 shutdown_connections 2023-11-08T19:25:06.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.596+0000 7f1328d8c740 1 -- 172.21.15.119:0/682330055 wait complete. 2023-11-08T19:25:06.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.597+0000 7f1328d8c740 20 client.4386 trim_cache size 0 max 16384 2023-11-08T19:25:06.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.614+0000 7f1328d8c740 1 Processor -- start 2023-11-08T19:25:06.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.614+0000 7f1328d8c740 1 -- start start 2023-11-08T19:25:06.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c30bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c337f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c338460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f1328d8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c30cb60 con 0x559e9c334080 2023-11-08T19:25:06.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c3523c0 con 0x559e9c34bc60 2023-11-08T19:25:06.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e9c217530 con 0x559e9c334460 2023-11-08T19:25:06.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c30bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f131f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c30bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33386/0 (socket says 172.21.15.119:33386) 2023-11-08T19:25:06.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.615+0000 7f131f7fe640 1 -- 172.21.15.119:0/2686892554 learned_addr learned my addr 172.21.15.119:0/2686892554 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131ffff640 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c338460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f1317fff640 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c337f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131f7fe640 1 -- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 msgr2=0x559e9c337f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131f7fe640 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c337f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131f7fe640 1 -- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 msgr2=0x559e9c338460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131f7fe640 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c338460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131f7fe640 1 -- 172.21.15.119:0/2686892554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e9c20eb90 con 0x559e9c334080 2023-11-08T19:25:06.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f1317fff640 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c337f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:06.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f131f7fe640 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c30bd20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1310006d10 tx=0x7f131007c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52624c0a9d54b5a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f1315ffb640 1 -- 172.21.15.119:0/2686892554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1310097070 con 0x559e9c334080 2023-11-08T19:25:06.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f1315ffb640 1 -- 172.21.15.119:0/2686892554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1310082790 con 0x559e9c334080 2023-11-08T19:25:06.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.616+0000 7f1315ffb640 1 -- 172.21.15.119:0/2686892554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131007e170 con 0x559e9c334080 2023-11-08T19:25:06.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 -- 172.21.15.119:0/2686892554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 msgr2=0x559e9c30bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c30bd20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1310006d10 tx=0x7f131007c000 comp rx=0 tx=0).stop 2023-11-08T19:25:06.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 -- 172.21.15.119:0/2686892554 shutdown_connections 2023-11-08T19:25:06.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c34bc60 0x559e9c338460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c337f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 --2- 172.21.15.119:0/2686892554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c334080 0x559e9c30bd20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 -- 172.21.15.119:0/2686892554 >> 172.21.15.119:0/2686892554 conn(0x559e9c3074c0 msgr2=0x559e9c302660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 -- 172.21.15.119:0/2686892554 shutdown_connections 2023-11-08T19:25:06.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.617+0000 7f1328d8c740 1 -- 172.21.15.119:0/2686892554 wait complete. 2023-11-08T19:25:06.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.618+0000 7f1328d8c740 1 -- start start 2023-11-08T19:25:06.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.618+0000 7f1328d8c740 1 Processor -- start 2023-11-08T19:25:06.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c373950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c3711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 0x559e9c36ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c373e90 con 0x559e9c34bc60 2023-11-08T19:25:06.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e9c3742c0 con 0x559e9c334080 2023-11-08T19:25:06.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559e9c370c70 con 0x559e9c334460 2023-11-08T19:25:06.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1328d8c740 10 client.0 _ll_register_callbacks cb 0x7f1310079ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:25:06.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f131ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 0x559e9c36ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f131ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 0x559e9c36ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33394/0 (socket says 172.21.15.119:33394) 2023-11-08T19:25:06.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f131ffff640 1 -- 172.21.15.119:0/3606416308 learned_addr learned my addr 172.21.15.119:0/3606416308 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f1317fff640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c3711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f131ffff640 1 -- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 msgr2=0x559e9c3711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.620+0000 7f131ffff640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c3711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f131ffff640 1 -- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 msgr2=0x559e9c373950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c373950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f131ffff640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c373950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f131ffff640 1 -- 172.21.15.119:0/3606416308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559e9c270e40 con 0x559e9c34bc60 2023-11-08T19:25:06.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f1317fff640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c3711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:06.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f131ffff640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 0x559e9c36ae20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1318007b30 tx=0x7f1318001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53e6068af600e65 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f12fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:25:06.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f12fe7fc640 1 -- 172.21.15.119:0/3606416308 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1318012560 con 0x559e9c34bc60 2023-11-08T19:25:06.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f12fe7fc640 1 -- 172.21.15.119:0/3606416308 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f13180126c0 con 0x559e9c34bc60 2023-11-08T19:25:06.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.621+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1308017850 con 0x559e9c34bc60 2023-11-08T19:25:06.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f12fe7fc640 1 -- 172.21.15.119:0/3606416308 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1318005e00 con 0x559e9c34bc60 2023-11-08T19:25:06.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 10 client.4392 fetch_fsmap learned FSMap version 7 2023-11-08T19:25:06.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x559e9c234180 con 0x559e9c34bc60 2023-11-08T19:25:06.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f12fe7fc640 1 -- 172.21.15.119:0/3606416308 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308017850 con 0x559e9c34bc60 2023-11-08T19:25:06.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f12fe7fc640 1 -- 172.21.15.119:0/3606416308 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1318012070 con 0x559e9c34bc60 2023-11-08T19:25:06.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 10 client.4392 fetch_fsmap finished waiting for FSMap version 7 2023-11-08T19:25:06.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 10 client.4392 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:25:06.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 20 client.4392 populate_metadata read hostname 'smithi119' 2023-11-08T19:25:06.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c36b660 0x559e9c34ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 4 client.4392 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:25:06.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.622+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 --> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559e9c26b4c0 con 0x559e9c36b660 2023-11-08T19:25:06.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.623+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c36b660 0x559e9c34ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f131f7fe640 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c36b660 0x559e9c34ac20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a69045f7f984d9bb server_cookie=34a10e3920d600c2 in_seq=0 out_seq=0 2023-11-08T19:25:06.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f12fe7fc640 10 client.4392 ms_handle_connect on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:06.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f12fe7fc640 1 -- 172.21.15.119:0/3606416308 <== mds.0 v2:172.21.15.188:6833/2629712656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x559e9c26b4c0 con 0x559e9c36b660 2023-11-08T19:25:06.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f12fe7fc640 10 client.4392 handle_command_reply: tid=0 2023-11-08T19:25:06.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 client.4392 shutdown 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 msgr2=0x559e9c36ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 0x559e9c36ae20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1318007b30 tx=0x7f1318001ac0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 shutdown_connections 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x559e9c36b660 0x559e9c34ac20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559e9c34bc60 0x559e9c36ae20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559e9c334460 0x559e9c3711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f12fe7fc640 0 client.4392 ms_handle_reset on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:06.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 --2- 172.21.15.119:0/3606416308 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559e9c334080 0x559e9c373950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 >> 172.21.15.119:0/3606416308 conn(0x559e9c3074c0 msgr2=0x559e9c302560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.624+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 shutdown_connections 2023-11-08T19:25:06.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1328d8c740 1 -- 172.21.15.119:0/3606416308 wait complete. 2023-11-08T19:25:06.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1328d8c740 20 client.4392 trim_cache size 0 max 16384 2023-11-08T19:25:06.639 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4254, 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:25:06.640 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 7.9816197940000002, 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 7, 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 25, 2023-11-08T19:25:06.641 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 24, 2023-11-08T19:25:06.642 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 13.474738721 2023-11-08T19:25:06.642 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f130006ae70 msgr2=0x7f130006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f130006ae70 0x7f130006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13200cdb60 tx=0x7f13080079b0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 msgr2=0x7f13200cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 0x7f13200cceb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1310000c00 tx=0x7f1310004730 comp rx=0 tx=0).stop 2023-11-08T19:25:06.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 shutdown_connections 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f130006ae70 0x7f130006d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f132015d310 0x7f132014cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13200cc640 0x7f132014c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 --2- 172.21.15.119:0/3874418302 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13200ad8b0 0x7f13200cceb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.625+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 >> 172.21.15.119:0/3874418302 conn(0x7f13200b0560 msgr2=0x7f1320158f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.626+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 shutdown_connections 2023-11-08T19:25:06.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.626+0000 7f1324ec3640 1 -- 172.21.15.119:0/3874418302 wait complete. 2023-11-08T19:25:06.646 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:06.647 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:06.647 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:06.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.850+0000 7f9ca594c640 1 Processor -- start 2023-11-08T19:25:06.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.850+0000 7f9ca594c640 1 -- start start 2023-11-08T19:25:06.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9ca594c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9ca006d250 0x7f9ca00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9ca594c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca00c4960 con 0x7f9ca015d9f0 2023-11-08T19:25:06.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9ca594c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ca015df60 con 0x7f9ca0142f80 2023-11-08T19:25:06.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9ca594c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ca0143a40 con 0x7f9ca006d250 2023-11-08T19:25:06.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9c9e7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9ca015d9f0 0x7f9ca006cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45548/0 (socket says 172.21.15.119:45548) 2023-11-08T19:25:06.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9c9e7fc640 1 -- 172.21.15.119:0/947722202 learned_addr learned my addr 172.21.15.119:0/947722202 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:06.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9c9f7fe640 1 --2- 172.21.15.119:0/947722202 >> v2:172.21.15.119:3300/0 conn(0x7f9ca006d250 0x7f9ca00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9c9dffb640 1 -- 172.21.15.119:0/947722202 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890181740 0 0) 0x7f9ca00c4960 con 0x7f9ca015d9f0 2023-11-08T19:25:06.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.852+0000 7f9c9dffb640 1 -- 172.21.15.119:0/947722202 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c88005730 con 0x7f9ca015d9f0 2023-11-08T19:25:06.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.853+0000 7f9c9f7fe640 1 -- 172.21.15.119:0/947722202 >> v1:172.21.15.188:6789/0 conn(0x7f9ca0142f80 legacy=0x7f9ca00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:06.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.853+0000 7f9c9f7fe640 1 -- 172.21.15.119:0/947722202 >> v1:172.21.15.119:6789/0 conn(0x7f9ca015d9f0 legacy=0x7f9ca006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.853+0000 7f9c9f7fe640 1 -- 172.21.15.119:0/947722202 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca015e240 con 0x7f9ca006d250 2023-11-08T19:25:06.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.854+0000 7f9c9f7fe640 1 --2- 172.21.15.119:0/947722202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca006d250 0x7f9ca00cc6d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d90 tx=0x7f9c9802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb8b0c045d7604c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.854+0000 7f9c9dffb640 1 -- 172.21.15.119:0/947722202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c9802cc60 con 0x7f9ca006d250 2023-11-08T19:25:06.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.854+0000 7f9c9dffb640 1 -- 172.21.15.119:0/947722202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c9802cdc0 con 0x7f9ca006d250 2023-11-08T19:25:06.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.854+0000 7f9c9dffb640 1 -- 172.21.15.119:0/947722202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c98034660 con 0x7f9ca006d250 2023-11-08T19:25:06.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 -- 172.21.15.119:0/947722202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca006d250 msgr2=0x7f9ca00cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 --2- 172.21.15.119:0/947722202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca006d250 0x7f9ca00cc6d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9c98009d90 tx=0x7f9c9802beb0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 -- 172.21.15.119:0/947722202 shutdown_connections 2023-11-08T19:25:06.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 --2- 172.21.15.119:0/947722202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca006d250 0x7f9ca00cc6d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 -- 172.21.15.119:0/947722202 >> 172.21.15.119:0/947722202 conn(0x7f9ca00cb610 msgr2=0x7f9ca00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:06.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 -- 172.21.15.119:0/947722202 shutdown_connections 2023-11-08T19:25:06.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.855+0000 7f9ca594c640 1 -- 172.21.15.119:0/947722202 wait complete. 2023-11-08T19:25:06.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.856+0000 7f9ca594c640 1 Processor -- start 2023-11-08T19:25:06.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.857+0000 7f9ca594c640 1 -- start start 2023-11-08T19:25:06.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.857+0000 7f9ca594c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca006d250 0x7f9ca015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.857+0000 7f9ca594c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 0x7f9ca00b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9ca594c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca015d9f0 0x7f9ca0141d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9ca594c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00c8e60 con 0x7f9ca0142f80 2023-11-08T19:25:06.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9ca594c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00c6fc0 con 0x7f9ca006d250 2023-11-08T19:25:06.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9ca594c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ca00c72a0 con 0x7f9ca015d9f0 2023-11-08T19:25:06.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 0x7f9ca00b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 0x7f9ca00b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33406/0 (socket says 172.21.15.119:33406) 2023-11-08T19:25:06.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 -- 172.21.15.119:0/4280253747 learned_addr learned my addr 172.21.15.119:0/4280253747 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:06.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9f7fe640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca015d9f0 0x7f9ca0141d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9effd640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca006d250 0x7f9ca015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 -- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca015d9f0 msgr2=0x7f9ca0141d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca015d9f0 0x7f9ca0141d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 -- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca006d250 msgr2=0x7f9ca015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca006d250 0x7f9ca015cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.858+0000 7f9c9e7fc640 1 -- 172.21.15.119:0/4280253747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c98009a70 con 0x7f9ca0142f80 2023-11-08T19:25:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.859+0000 7f9c9e7fc640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 0x7f9ca00b0a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c500 tx=0x7f9c9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.859+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94005ce0 con 0x7f9ca0142f80 2023-11-08T19:25:06.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.859+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c94005e40 con 0x7f9ca0142f80 2023-11-08T19:25:06.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.859+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ca00c7580 con 0x7f9ca0142f80 2023-11-08T19:25:06.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.859+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c94007330 con 0x7f9ca0142f80 2023-11-08T19:25:06.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.859+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ca00c7b40 con 0x7f9ca0142f80 2023-11-08T19:25:06.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.860+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ca00c7d30 con 0x7f9ca0142f80 2023-11-08T19:25:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.861+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c94007550 con 0x7f9ca0142f80 2023-11-08T19:25:06.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.862+0000 7f9c87fff640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9c8c06ae70 0x7f9c8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:06.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.862+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9c94088fb0 con 0x7f9ca0142f80 2023-11-08T19:25:06.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.862+0000 7f9c9effd640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9c8c06ae70 0x7f9c8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:06.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.863+0000 7f9c9effd640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9c8c06ae70 0x7f9c8c06d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:06.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:06.866+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c94053e20 con 0x7f9ca0142f80 2023-11-08T19:25:07.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.094+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9ca0074430 con 0x7f9ca0142f80 2023-11-08T19:25:07.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.095+0000 7f9c87fff640 1 -- 172.21.15.119:0/4280253747 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9c94017020 con 0x7f9ca0142f80 2023-11-08T19:25:07.096 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T19:25:07.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9c8c06ae70 msgr2=0x7f9c8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9c8c06ae70 0x7f9c8c06d330 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c90006fd0 tx=0x7f9c90008040 comp rx=0 tx=0).stop 2023-11-08T19:25:07.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 msgr2=0x7f9ca00b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:07.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 0x7f9ca00b0a30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9c9400c500 tx=0x7f9c9400c9c0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 shutdown_connections 2023-11-08T19:25:07.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9c8c06ae70 0x7f9c8c06d330 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ca015d9f0 0x7f9ca0141d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9ca0142f80 0x7f9ca00b0a30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 --2- 172.21.15.119:0/4280253747 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ca006d250 0x7f9ca015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.097+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 >> 172.21.15.119:0/4280253747 conn(0x7f9ca00cb610 msgr2=0x7f9ca006c890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.098+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 shutdown_connections 2023-11-08T19:25:07.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:07.098+0000 7f9ca594c640 1 -- 172.21.15.119:0/4280253747 wait complete. 2023-11-08T19:25:07.116 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:25:07.116 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:07.116 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:25:07.117 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-08T19:25:07.117 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:07.117 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:25:07.118 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:07.118 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:07.118 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-08T19:25:07.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:07.119 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:25:07.171 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:07.171 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:07.171 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:07.172 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:07.172 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:07.172 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:07.172 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:07.172 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:07.173 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:25:07.173 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:07.173 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:07.173 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:25:07.173 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:07.174 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:07.174 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42363sec preferred_lft 42363sec 2023-11-08T19:25:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:25:07.175 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:07.176 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T19:25:07.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:07.176 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:07.177 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:07.177 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-11-08T19:25:07.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-11-08T19:25:07.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-11-08T19:25:07.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T19:25:07.179 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:07.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:07.679 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:07.756 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:25:07.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:07.760 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:25:07.815 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:25:07.815 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:07.815 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:07.815 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:07.816 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:07.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:07.817 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:07.817 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:07.817 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:07.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:07.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:07.818 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:08.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:08.556 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:25:08.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:08.618 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-11-08T19:25:08.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:08.673 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:08.674 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:08.674 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:08.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-08T19:25:08.675 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:08.813 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-08T19:25:08.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:08.814 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:08.814 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:08.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-08T19:25:08.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:08.815 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-08T19:25:08.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-08T19:25:08.816 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-08T19:25:08.817 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:09.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.101 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:09.101 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:09.101 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:25:09.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T19:25:09.102 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:09.265 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-08T19:25:09.266 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:09.266 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:09.316 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-08T19:25:09.316 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:09.317 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:09.370 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T19:25:09.423 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2023-11-08T19:25:09.424 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2023-11-08T19:25:09.424 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2023-11-08T19:25:09.424 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2023-11-08T19:25:09.424 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2023-11-08T19:25:09.424 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2023-11-08T19:25:09.425 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2023-11-08T19:25:09.425 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2023-11-08T19:25:09.425 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2023-11-08T19:25:09.425 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2023-11-08T19:25:09.426 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2023-11-08T19:25:09.426 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T19:25:09.426 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2023-11-08T19:25:09.426 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2023-11-08T19:25:09.426 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2023-11-08T19:25:09.427 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2023-11-08T19:25:09.427 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2023-11-08T19:25:09.427 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2023-11-08T19:25:09.427 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T19:25:09.427 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T19:25:09.428 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T19:25:09.428 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T19:25:09.428 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2023-11-08T19:25:09.428 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2023-11-08T19:25:09.429 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T19:25:09.429 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T19:25:09.429 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T19:25:09.429 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T19:25:09.429 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2023-11-08T19:25:09.430 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:09.430 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:09.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.432 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-11-08T19:25:09.483 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T19:25:09.483 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage stdin-killer --timeout=300 --signal=9 -- ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-11-08T19:25:09.488 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2023-11-08T19:25:09.533 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2023-11-08T19:25:09.534 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2023-11-08T19:25:09.535 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T19:25:09.536 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2023-11-08T19:25:09.536 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2023-11-08T19:25:09.536 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2023-11-08T19:25:09.536 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2023-11-08T19:25:09.537 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2023-11-08T19:25:09.537 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2023-11-08T19:25:09.537 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T19:25:09.537 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T19:25:09.537 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T19:25:09.538 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T19:25:09.538 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2023-11-08T19:25:09.538 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2023-11-08T19:25:09.539 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T19:25:09.539 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T19:25:09.539 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T19:25:09.539 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T19:25:09.539 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2023-11-08T19:25:09.540 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:09.540 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:09.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:09.541 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-11-08T19:25:09.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:2023-11-08T19:25:09.687+0000 7f8be3a10480 -1 init, newargv = 0x55df292c3cd0 newargc=15 2023-11-08T19:25:09.690 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:ceph-fuse[93976]: starting ceph client 2023-11-08T19:25:09.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:ceph-fuse[93976]: starting fuse 2023-11-08T19:25:10.728 DEBUG:teuthology.orchestra.run.smithi119:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T19:25:10.788 INFO:teuthology.orchestra.run.smithi119.stdout:/proc 2023-11-08T19:25:10.789 INFO:teuthology.orchestra.run.smithi119.stdout:/sys 2023-11-08T19:25:10.789 INFO:teuthology.orchestra.run.smithi119.stdout:/dev 2023-11-08T19:25:10.789 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/security 2023-11-08T19:25:10.789 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/shm 2023-11-08T19:25:10.789 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/pts 2023-11-08T19:25:10.790 INFO:teuthology.orchestra.run.smithi119.stdout:/run 2023-11-08T19:25:10.790 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/cgroup 2023-11-08T19:25:10.790 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/pstore 2023-11-08T19:25:10.790 INFO:teuthology.orchestra.run.smithi119.stdout:/ 2023-11-08T19:25:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/selinux 2023-11-08T19:25:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T19:25:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/mqueue 2023-11-08T19:25:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:/dev/hugepages 2023-11-08T19:25:10.791 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/debug 2023-11-08T19:25:10.792 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/tracing 2023-11-08T19:25:10.792 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/fs/fuse/connections 2023-11-08T19:25:10.792 INFO:teuthology.orchestra.run.smithi119.stdout:/sys/kernel/config 2023-11-08T19:25:10.792 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T19:25:10.793 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T19:25:10.793 INFO:teuthology.orchestra.run.smithi119.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T19:25:10.793 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T19:25:10.793 INFO:teuthology.orchestra.run.smithi119.stdout:/run/user/1000 2023-11-08T19:25:10.793 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph 2023-11-08T19:25:10.794 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T19:25:10.794 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T19:25:10.794 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T19:25:10.794 INFO:teuthology.orchestra.run.smithi119.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T19:25:10.794 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns 2023-11-08T19:25:10.795 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:10.795 INFO:teuthology.orchestra.run.smithi119.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:10.795 INFO:teuthology.orchestra.run.smithi119.stdout:/home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:10.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:10.798 DEBUG:teuthology.orchestra.run.smithi119:> ls /sys/fs/fuse/connections 2023-11-08T19:25:10.850 INFO:teuthology.orchestra.run.smithi119.stdout:40 2023-11-08T19:25:10.850 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T19:25:10.851 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer -- python3 -c ' 2023-11-08T19:25:10.851 DEBUG:teuthology.orchestra.run.smithi119:> import glob 2023-11-08T19:25:10.852 DEBUG:teuthology.orchestra.run.smithi119:> import re 2023-11-08T19:25:10.852 DEBUG:teuthology.orchestra.run.smithi119:> import os 2023-11-08T19:25:10.852 DEBUG:teuthology.orchestra.run.smithi119:> import subprocess 2023-11-08T19:25:10.853 DEBUG:teuthology.orchestra.run.smithi119:> 2023-11-08T19:25:10.853 DEBUG:teuthology.orchestra.run.smithi119:> def _find_admin_socket(client_name): 2023-11-08T19:25:10.854 DEBUG:teuthology.orchestra.run.smithi119:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-08T19:25:10.854 DEBUG:teuthology.orchestra.run.smithi119:> files = glob.glob(asok_path) 2023-11-08T19:25:10.854 DEBUG:teuthology.orchestra.run.smithi119:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-08T19:25:10.855 DEBUG:teuthology.orchestra.run.smithi119:> 2023-11-08T19:25:10.855 DEBUG:teuthology.orchestra.run.smithi119:> # Given a non-glob path, it better be there 2023-11-08T19:25:10.855 DEBUG:teuthology.orchestra.run.smithi119:> if "*" not in asok_path: 2023-11-08T19:25:10.856 DEBUG:teuthology.orchestra.run.smithi119:> assert(len(files) == 1) 2023-11-08T19:25:10.856 DEBUG:teuthology.orchestra.run.smithi119:> return files[0] 2023-11-08T19:25:10.857 DEBUG:teuthology.orchestra.run.smithi119:> 2023-11-08T19:25:10.857 DEBUG:teuthology.orchestra.run.smithi119:> for f in files: 2023-11-08T19:25:10.857 DEBUG:teuthology.orchestra.run.smithi119:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T19:25:10.858 DEBUG:teuthology.orchestra.run.smithi119:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T19:25:10.858 DEBUG:teuthology.orchestra.run.smithi119:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T19:25:10.859 DEBUG:teuthology.orchestra.run.smithi119:> contents = proc_f.read() 2023-11-08T19:25:10.859 DEBUG:teuthology.orchestra.run.smithi119:> if mountpoint in contents: 2023-11-08T19:25:10.859 DEBUG:teuthology.orchestra.run.smithi119:> return f 2023-11-08T19:25:10.860 DEBUG:teuthology.orchestra.run.smithi119:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T19:25:10.860 DEBUG:teuthology.orchestra.run.smithi119:> 2023-11-08T19:25:10.860 DEBUG:teuthology.orchestra.run.smithi119:> print(_find_admin_socket("client.admin")) 2023-11-08T19:25:10.861 DEBUG:teuthology.orchestra.run.smithi119:> ' 2023-11-08T19:25:11.043 INFO:teuthology.orchestra.run.smithi119.stdout:/var/run/ceph/ceph-client.admin.93976.asok 2023-11-08T19:25:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T19:25:11.060 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.93976.asok 2023-11-08T19:25:11.060 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-08T19:25:11.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer --timeout=300 -- ceph --admin-daemon /var/run/ceph/ceph-client.admin.93976.asok status 2023-11-08T19:25:11.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-08T19:25:11.355 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:25:11.355 INFO:teuthology.orchestra.run.smithi119.stdout: "metadata": { 2023-11-08T19:25:11.355 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_sha1": "7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4", 2023-11-08T19:25:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "ceph_version": "ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev)", 2023-11-08T19:25:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "entity_id": "admin", 2023-11-08T19:25:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "hostname": "smithi119", 2023-11-08T19:25:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-08T19:25:11.356 INFO:teuthology.orchestra.run.smithi119.stdout: "pid": "93976", 2023-11-08T19:25:11.357 INFO:teuthology.orchestra.run.smithi119.stdout: "root": "/" 2023-11-08T19:25:11.357 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:25:11.357 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_count": 0, 2023-11-08T19:25:11.357 INFO:teuthology.orchestra.run.smithi119.stdout: "dentry_pinned_count": 0, 2023-11-08T19:25:11.358 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4410, 2023-11-08T19:25:11.358 INFO:teuthology.orchestra.run.smithi119.stdout: "inst": { 2023-11-08T19:25:11.358 INFO:teuthology.orchestra.run.smithi119.stdout: "name": { 2023-11-08T19:25:11.358 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "client", 2023-11-08T19:25:11.358 INFO:teuthology.orchestra.run.smithi119.stdout: "num": 4410 2023-11-08T19:25:11.359 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:25:11.359 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-11-08T19:25:11.359 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-11-08T19:25:11.359 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-11-08T19:25:11.359 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1628210205 2023-11-08T19:25:11.360 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:25:11.360 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:25:11.360 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": { 2023-11-08T19:25:11.360 INFO:teuthology.orchestra.run.smithi119.stdout: "type": "v1", 2023-11-08T19:25:11.360 INFO:teuthology.orchestra.run.smithi119.stdout: "addr": "192.168.0.1:0", 2023-11-08T19:25:11.360 INFO:teuthology.orchestra.run.smithi119.stdout: "nonce": 1628210205 2023-11-08T19:25:11.361 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:25:11.361 INFO:teuthology.orchestra.run.smithi119.stdout: "inst_str": "client.4410 192.168.0.1:0/1628210205", 2023-11-08T19:25:11.361 INFO:teuthology.orchestra.run.smithi119.stdout: "addr_str": "192.168.0.1:0/1628210205", 2023-11-08T19:25:11.361 INFO:teuthology.orchestra.run.smithi119.stdout: "inode_count": 1, 2023-11-08T19:25:11.361 INFO:teuthology.orchestra.run.smithi119.stdout: "mds_epoch": 7, 2023-11-08T19:25:11.362 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch": 25, 2023-11-08T19:25:11.362 INFO:teuthology.orchestra.run.smithi119.stdout: "osd_epoch_barrier": 0, 2023-11-08T19:25:11.362 INFO:teuthology.orchestra.run.smithi119.stdout: "blocklisted": false, 2023-11-08T19:25:11.362 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs" 2023-11-08T19:25:11.362 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:25:11.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T19:25:11.379 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-08T19:25:11.379 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer --timeout=300 -- stat --file-system '--printf=%T 2023-11-08T19:25:11.380 DEBUG:teuthology.orchestra.run.smithi119:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:11.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-08T19:25:11.524 INFO:teuthology.orchestra.run.smithi119.stdout:fuseblk 2023-11-08T19:25:11.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T19:25:11.537 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:11.537 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-08T19:25:11.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer --timeout=300 -- chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:11.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-08T19:25:11.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T19:25:11.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:11.692 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-08T19:25:11.786 INFO:teuthology.orchestra.run.smithi119.stdout:# file: . 2023-11-08T19:25:11.787 INFO:teuthology.orchestra.run.smithi119.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-08T19:25:11.787 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:11.789 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:11.789 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:11.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:12.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.053+0000 7f7f500b0640 1 Processor -- start 2023-11-08T19:25:12.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.053+0000 7f7f500b0640 1 -- start start 2023-11-08T19:25:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f500b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7f480cc810 0x7f7f480c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f500b0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7f4815d180 0x7f7f4806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f500b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480ae410 con 0x7f7f4806a390 2023-11-08T19:25:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f500b0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4806ba30 con 0x7f7f4815d180 2023-11-08T19:25:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f500b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f4806bcf0 con 0x7f7f480cc810 2023-11-08T19:25:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f4de4c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7f4806a390 0x7f7f4815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37036/0 (socket says 172.21.15.119:37036) 2023-11-08T19:25:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.055+0000 7f7f4de4c640 1 -- 172.21.15.119:0/2924481609 learned_addr learned my addr 172.21.15.119:0/2924481609 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4e64d640 1 --2- 172.21.15.119:0/2924481609 >> v2:172.21.15.119:3300/0 conn(0x7f7f4815d180 0x7f7f4806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4d64b640 1 --2- 172.21.15.119:0/2924481609 >> v2:172.21.15.188:3300/0 conn(0x7f7f480cc810 0x7f7f480c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4ce4a640 1 -- 172.21.15.119:0/2924481609 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4061520018 0 0) 0x7f7f480ae410 con 0x7f7f4806a390 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4ce4a640 1 -- 172.21.15.119:0/2924481609 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f34003520 con 0x7f7f4806a390 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4e64d640 1 -- 172.21.15.119:0/2924481609 >> v2:172.21.15.188:3300/0 conn(0x7f7f480cc810 msgr2=0x7f7f480c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:12.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4e64d640 1 --2- 172.21.15.119:0/2924481609 >> v2:172.21.15.188:3300/0 conn(0x7f7f480cc810 0x7f7f480c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4e64d640 1 -- 172.21.15.119:0/2924481609 >> v1:172.21.15.119:6789/0 conn(0x7f7f4806a390 legacy=0x7f7f4815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.056+0000 7f7f4e64d640 1 -- 172.21.15.119:0/2924481609 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4806bfd0 con 0x7f7f4815d180 2023-11-08T19:25:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.057+0000 7f7f4e64d640 1 --2- 172.21.15.119:0/2924481609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f4806cf70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f480b1850 tx=0x7f7f4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d636e564050eee server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.057+0000 7f7f4ce4a640 1 -- 172.21.15.119:0/2924481609 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4002cc80 con 0x7f7f4815d180 2023-11-08T19:25:12.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.057+0000 7f7f4ce4a640 1 -- 172.21.15.119:0/2924481609 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4002cde0 con 0x7f7f4815d180 2023-11-08T19:25:12.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.057+0000 7f7f4ce4a640 1 -- 172.21.15.119:0/2924481609 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40034660 con 0x7f7f4815d180 2023-11-08T19:25:12.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 -- 172.21.15.119:0/2924481609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 msgr2=0x7f7f4806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 --2- 172.21.15.119:0/2924481609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f4806cf70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7f480b1850 tx=0x7f7f4002bed0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 -- 172.21.15.119:0/2924481609 shutdown_connections 2023-11-08T19:25:12.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 --2- 172.21.15.119:0/2924481609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f4806cf70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 --2- 172.21.15.119:0/2924481609 >> v2:172.21.15.188:3300/0 conn(0x7f7f480cc810 0x7f7f480c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 -- 172.21.15.119:0/2924481609 >> 172.21.15.119:0/2924481609 conn(0x7f7f480cb610 msgr2=0x7f7f480cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 -- 172.21.15.119:0/2924481609 shutdown_connections 2023-11-08T19:25:12.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.058+0000 7f7f500b0640 1 -- 172.21.15.119:0/2924481609 wait complete. 2023-11-08T19:25:12.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.060+0000 7f7f500b0640 1 Processor -- start 2023-11-08T19:25:12.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.060+0000 7f7f500b0640 1 -- start start 2023-11-08T19:25:12.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f500b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4806a390 0x7f7f4814a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f500b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480cc810 0x7f7f480adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f500b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f48070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f500b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480b1550 con 0x7f7f4815d180 2023-11-08T19:25:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f500b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480b3ee0 con 0x7f7f480cc810 2023-11-08T19:25:12.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f500b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f481f3260 con 0x7f7f4806a390 2023-11-08T19:25:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f4e64d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f48070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f4e64d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f48070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37310/0 (socket says 172.21.15.119:37310) 2023-11-08T19:25:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f4e64d640 1 -- 172.21.15.119:0/188579892 learned_addr learned my addr 172.21.15.119:0/188579892 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:12.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f4de4c640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4806a390 0x7f7f4814a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.061+0000 7f7f4d64b640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480cc810 0x7f7f480adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4e64d640 1 -- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4806a390 msgr2=0x7f7f4814a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4e64d640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4806a390 0x7f7f4814a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4e64d640 1 -- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480cc810 msgr2=0x7f7f480adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4e64d640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480cc810 0x7f7f480adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4e64d640 1 -- 172.21.15.119:0/188579892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f40009a70 con 0x7f7f4815d180 2023-11-08T19:25:12.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4de4c640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4806a390 0x7f7f4814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:12.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f4e64d640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f48070980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009e80 tx=0x7f7f40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40040650 con 0x7f7f4815d180 2023-11-08T19:25:12.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f400407b0 con 0x7f7f4815d180 2023-11-08T19:25:12.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f481f3540 con 0x7f7f4815d180 2023-11-08T19:25:12.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f40040b60 con 0x7f7f4815d180 2023-11-08T19:25:12.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.062+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f480b3b70 con 0x7f7f4815d180 2023-11-08T19:25:12.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.064+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f480b4050 con 0x7f7f4815d180 2023-11-08T19:25:12.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.064+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f40040cc0 con 0x7f7f4815d180 2023-11-08T19:25:12.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.065+0000 7f7f3e7fc640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7f2c06ae70 0x7f7f2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.065+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7f400ac680 con 0x7f7f4815d180 2023-11-08T19:25:12.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.066+0000 7f7f4de4c640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7f2c06ae70 0x7f7f2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.066+0000 7f7f4de4c640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7f2c06ae70 0x7f7f2c06d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7f440011e0 tx=0x7f7f44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.070+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f400773a0 con 0x7f7f4815d180 2023-11-08T19:25:12.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.260+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f48074430 con 0x7f7f4815d180 2023-11-08T19:25:12.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.262+0000 7f7f3e7fc640 1 -- 172.21.15.119:0/188579892 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f7f40009450 con 0x7f7f4815d180 2023-11-08T19:25:12.265 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:12.265 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:12.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.264+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7f2c06ae70 msgr2=0x7f7f2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7f2c06ae70 0x7f7f2c06d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7f440011e0 tx=0x7f7f44009040 comp rx=0 tx=0).stop 2023-11-08T19:25:12.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 msgr2=0x7f7f48070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f48070980 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7f40009e80 tx=0x7f7f40004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:12.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 shutdown_connections 2023-11-08T19:25:12.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7f2c06ae70 0x7f7f2c06d330 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7f4815d180 0x7f7f48070980 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7f480cc810 0x7f7f480adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 --2- 172.21.15.119:0/188579892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7f4806a390 0x7f7f4814a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 >> 172.21.15.119:0/188579892 conn(0x7f7f480cb610 msgr2=0x7f7f4815b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 shutdown_connections 2023-11-08T19:25:12.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.265+0000 7f7f500b0640 1 -- 172.21.15.119:0/188579892 wait complete. 2023-11-08T19:25:12.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:12.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.489+0000 7f49a5e97640 1 Processor -- start 2023-11-08T19:25:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.490+0000 7f49a5e97640 1 -- start start 2023-11-08T19:25:12.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f49a5e97640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49a00cdd70 0x7f49a00cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f49a5e97640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f49a5e97640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49a00b2710 con 0x7f49a015d340 2023-11-08T19:25:12.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f49a5e97640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a00cb4d0 con 0x7f49a00cdd70 2023-11-08T19:25:12.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f49a5e97640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49a006bbb0 con 0x7f49a015d720 2023-11-08T19:25:12.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f4997fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49a00cdd70 0x7f49a00cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f4997fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49a00cdd70 0x7f49a00cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37324/0 (socket says 172.21.15.119:37324) 2023-11-08T19:25:12.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.491+0000 7f4997fff640 1 -- 172.21.15.119:0/4227798958 learned_addr learned my addr 172.21.15.119:0/4227798958 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:12.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.492+0000 7f499ffff640 1 --2- 172.21.15.119:0/4227798958 >> v2:172.21.15.188:3300/0 conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.492+0000 7f4997fff640 1 -- 172.21.15.119:0/4227798958 >> v2:172.21.15.188:3300/0 conn(0x7f49a015d720 msgr2=0x7f49a00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:12.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.492+0000 7f4997fff640 1 --2- 172.21.15.119:0/4227798958 >> v2:172.21.15.188:3300/0 conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.492+0000 7f4997fff640 1 -- 172.21.15.119:0/4227798958 >> v1:172.21.15.188:6789/0 conn(0x7f49a015d340 legacy=0x7f49a00cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:12.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.492+0000 7f4997fff640 1 -- 172.21.15.119:0/4227798958 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a006be90 con 0x7f49a00cdd70 2023-11-08T19:25:12.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.492+0000 7f499ffff640 1 --2- 172.21.15.119:0/4227798958 >> v2:172.21.15.188:3300/0 conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:12.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.493+0000 7f4997fff640 1 --2- 172.21.15.119:0/4227798958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a00cbc90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49a006aea0 tx=0x7f498802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbc5b6bd13c6900 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.493+0000 7f499effd640 1 -- 172.21.15.119:0/4227798958 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988031870 con 0x7f49a00cdd70 2023-11-08T19:25:12.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.493+0000 7f499effd640 1 -- 172.21.15.119:0/4227798958 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49880319d0 con 0x7f49a00cdd70 2023-11-08T19:25:12.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.493+0000 7f499effd640 1 -- 172.21.15.119:0/4227798958 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988031d60 con 0x7f49a00cdd70 2023-11-08T19:25:12.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 -- 172.21.15.119:0/4227798958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 msgr2=0x7f49a00cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 --2- 172.21.15.119:0/4227798958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a00cbc90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f49a006aea0 tx=0x7f498802b910 comp rx=0 tx=0).stop 2023-11-08T19:25:12.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 -- 172.21.15.119:0/4227798958 shutdown_connections 2023-11-08T19:25:12.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 --2- 172.21.15.119:0/4227798958 >> v2:172.21.15.188:3300/0 conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 --2- 172.21.15.119:0/4227798958 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a00cbc90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 -- 172.21.15.119:0/4227798958 >> 172.21.15.119:0/4227798958 conn(0x7f49a009a110 msgr2=0x7f49a00d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 -- 172.21.15.119:0/4227798958 shutdown_connections 2023-11-08T19:25:12.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.494+0000 7f49a5e97640 1 -- 172.21.15.119:0/4227798958 wait complete. 2023-11-08T19:25:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.496+0000 7f49a5e97640 1 Processor -- start 2023-11-08T19:25:12.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.497+0000 7f49a5e97640 1 -- start start 2023-11-08T19:25:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f49a5e97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f49a5e97640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49a015d340 0x7f49a00cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f49a5e97640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f49a5e97640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a015f080 con 0x7f49a00cdd70 2023-11-08T19:25:12.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f49a5e97640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a015e400 con 0x7f49a015d720 2023-11-08T19:25:12.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f49a5e97640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49a015e6e0 con 0x7f49a015d340 2023-11-08T19:25:12.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f499f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f499f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a015db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37334/0 (socket says 172.21.15.119:37334) 2023-11-08T19:25:12.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.498+0000 7f499f7fe640 1 -- 172.21.15.119:0/3272243686 learned_addr learned my addr 172.21.15.119:0/3272243686 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:12.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f4997fff640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49a015d340 0x7f49a00cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499ffff640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499f7fe640 1 -- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49a015d340 msgr2=0x7f49a00cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499f7fe640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49a015d340 0x7f49a00cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499f7fe640 1 -- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49a015d720 msgr2=0x7f49a00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499f7fe640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499f7fe640 1 -- 172.21.15.119:0/3272243686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49880094d0 con 0x7f49a00cdd70 2023-11-08T19:25:12.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499ffff640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:12.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499f7fe640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a015db00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f499000c430 tx=0x7f499000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990011790 con 0x7f49a00cdd70 2023-11-08T19:25:12.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49900118f0 con 0x7f49a00cdd70 2023-11-08T19:25:12.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4990011ce0 con 0x7f49a00cdd70 2023-11-08T19:25:12.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a015e9c0 con 0x7f49a00cdd70 2023-11-08T19:25:12.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.499+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a015e040 con 0x7f49a00cdd70 2023-11-08T19:25:12.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.501+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a015e1d0 con 0x7f49a00cdd70 2023-11-08T19:25:12.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.502+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f499001c460 con 0x7f49a00cdd70 2023-11-08T19:25:12.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.503+0000 7f499cff9640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f498006ae70 0x7f498006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.503+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f49900898d0 con 0x7f49a00cdd70 2023-11-08T19:25:12.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.503+0000 7f4997fff640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f498006ae70 0x7f498006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.504+0000 7f4997fff640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f498006ae70 0x7f498006d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4988036000 tx=0x7f498803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.507+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4990054680 con 0x7f49a00cdd70 2023-11-08T19:25:12.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.697+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49a0074430 con 0x7f49a00cdd70 2023-11-08T19:25:12.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.698+0000 7f499cff9640 1 -- 172.21.15.119:0/3272243686 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4990057ca0 con 0x7f49a00cdd70 2023-11-08T19:25:12.701 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:12.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f498006ae70 msgr2=0x7f498006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f498006ae70 0x7f498006d330 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4988036000 tx=0x7f498803c040 comp rx=0 tx=0).stop 2023-11-08T19:25:12.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 msgr2=0x7f49a015db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a015db00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f499000c430 tx=0x7f499000c8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 shutdown_connections 2023-11-08T19:25:12.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f498006ae70 0x7f498006d330 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49a015d720 0x7f49a00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49a015d340 0x7f49a00cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 --2- 172.21.15.119:0/3272243686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49a00cdd70 0x7f49a015db00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 >> 172.21.15.119:0/3272243686 conn(0x7f49a009a110 msgr2=0x7f49a015c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 shutdown_connections 2023-11-08T19:25:12.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.701+0000 7f49a5e97640 1 -- 172.21.15.119:0/3272243686 wait complete. 2023-11-08T19:25:12.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:12.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.922+0000 7f62c3fff640 1 Processor -- start 2023-11-08T19:25:12.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.922+0000 7f62c3fff640 1 -- start start 2023-11-08T19:25:12.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.923+0000 7f62c3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c415d9b0 0x7f62c406ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.923+0000 7f62c3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62c406bf40 0x7f62c40acf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.923+0000 7f62c3fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62c406cdd0 con 0x7f62c40c9650 2023-11-08T19:25:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.923+0000 7f62c3fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62c40cb0e0 con 0x7f62c406bf40 2023-11-08T19:25:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.923+0000 7f62c3fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c406c530 con 0x7f62c415d9b0 2023-11-08T19:25:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62c406bf40 0x7f62c40acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62c406bf40 0x7f62c40acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37336/0 (socket says 172.21.15.119:37336) 2023-11-08T19:25:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 -- 172.21.15.119:0/915973787 learned_addr learned my addr 172.21.15.119:0/915973787 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62bbfff640 1 --2- 172.21.15.119:0/915973787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c415d9b0 0x7f62c406ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 -- 172.21.15.119:0/915973787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c415d9b0 msgr2=0x7f62c406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:12.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 --2- 172.21.15.119:0/915973787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c415d9b0 0x7f62c406ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 -- 172.21.15.119:0/915973787 >> v1:172.21.15.188:6789/0 conn(0x7f62c40c9650 legacy=0x7f62c40c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.924+0000 7f62c37fe640 1 -- 172.21.15.119:0/915973787 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c406c810 con 0x7f62c406bf40 2023-11-08T19:25:12.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.925+0000 7f62c37fe640 1 --2- 172.21.15.119:0/915973787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c40acf20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009e50 tx=0x7f62bc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82fec0a63a6f85e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c27fc640 1 -- 172.21.15.119:0/915973787 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc02c9e0 con 0x7f62c406bf40 2023-11-08T19:25:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c27fc640 1 -- 172.21.15.119:0/915973787 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc02cb40 con 0x7f62c406bf40 2023-11-08T19:25:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c27fc640 1 -- 172.21.15.119:0/915973787 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0343f0 con 0x7f62c406bf40 2023-11-08T19:25:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 -- 172.21.15.119:0/915973787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 msgr2=0x7f62c40acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 --2- 172.21.15.119:0/915973787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c40acf20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009e50 tx=0x7f62bc02be40 comp rx=0 tx=0).stop 2023-11-08T19:25:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 -- 172.21.15.119:0/915973787 shutdown_connections 2023-11-08T19:25:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 --2- 172.21.15.119:0/915973787 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c40acf20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 --2- 172.21.15.119:0/915973787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c415d9b0 0x7f62c406ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 -- 172.21.15.119:0/915973787 >> 172.21.15.119:0/915973787 conn(0x7f62c40ceac0 msgr2=0x7f62c40ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 -- 172.21.15.119:0/915973787 shutdown_connections 2023-11-08T19:25:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.926+0000 7f62c3fff640 1 -- 172.21.15.119:0/915973787 wait complete. 2023-11-08T19:25:12.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.928+0000 7f62c3fff640 1 Processor -- start 2023-11-08T19:25:12.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.928+0000 7f62c3fff640 1 -- start start 2023-11-08T19:25:12.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c40c9650 0x7f62c40c6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62c415d9b0 0x7f62c40c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c3fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40c5120 con 0x7f62c406bf40 2023-11-08T19:25:12.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c3fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c406f9b0 con 0x7f62c415d9b0 2023-11-08T19:25:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c3fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62c40c8270 con 0x7f62c40c9650 2023-11-08T19:25:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.929+0000 7f62c2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37348/0 (socket says 172.21.15.119:37348) 2023-11-08T19:25:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 -- 172.21.15.119:0/1868021364 learned_addr learned my addr 172.21.15.119:0/1868021364 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:12.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c37fe640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62c415d9b0 0x7f62c40c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62bbfff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c40c9650 0x7f62c40c6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 -- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c40c9650 msgr2=0x7f62c40c6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c40c9650 0x7f62c40c6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 -- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62c415d9b0 msgr2=0x7f62c40c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62c415d9b0 0x7f62c40c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 -- 172.21.15.119:0/1868021364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc009b30 con 0x7f62c406bf40 2023-11-08T19:25:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c37fe640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62c415d9b0 0x7f62c40c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62c2ffd640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c415ce90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b3f0 tx=0x7f62b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.930+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4011830 con 0x7f62c406bf40 2023-11-08T19:25:12.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.931+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b4011990 con 0x7f62c406bf40 2023-11-08T19:25:12.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.931+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c40c8550 con 0x7f62c406bf40 2023-11-08T19:25:12.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.931+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4011d40 con 0x7f62c406bf40 2023-11-08T19:25:12.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.931+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c406fbc0 con 0x7f62c406bf40 2023-11-08T19:25:12.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.932+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c404ece0 con 0x7f62c406bf40 2023-11-08T19:25:12.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.933+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62b401a400 con 0x7f62c406bf40 2023-11-08T19:25:12.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.934+0000 7f62bb7fe640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f62a406ae70 0x7f62a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:12.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.934+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f62b40897e0 con 0x7f62c406bf40 2023-11-08T19:25:12.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.934+0000 7f62bbfff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f62a406ae70 0x7f62a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:12.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.935+0000 7f62bbfff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f62a406ae70 0x7f62a406d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62c4071da0 tx=0x7f62ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:12.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:12.939+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62b4054500 con 0x7f62c406bf40 2023-11-08T19:25:13.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.130+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62c40b47d0 con 0x7f62c406bf40 2023-11-08T19:25:13.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.131+0000 7f62bb7fe640 1 -- 172.21.15.119:0/1868021364 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62b4057b20 con 0x7f62c406bf40 2023-11-08T19:25:13.134 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:13.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f62a406ae70 msgr2=0x7f62a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f62a406ae70 0x7f62a406d330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f62c4071da0 tx=0x7f62ac008040 comp rx=0 tx=0).stop 2023-11-08T19:25:13.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 msgr2=0x7f62c415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c415ce90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f62b400b3f0 tx=0x7f62b400b8b0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 shutdown_connections 2023-11-08T19:25:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f62a406ae70 0x7f62a406d330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f62c415d9b0 0x7f62c40c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62c40c9650 0x7f62c40c6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 --2- 172.21.15.119:0/1868021364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62c406bf40 0x7f62c415ce90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 >> 172.21.15.119:0/1868021364 conn(0x7f62c40ceac0 msgr2=0x7f62c40cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 shutdown_connections 2023-11-08T19:25:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.134+0000 7f62c3fff640 1 -- 172.21.15.119:0/1868021364 wait complete. 2023-11-08T19:25:13.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.366+0000 7fed1d690640 1 Processor -- start 2023-11-08T19:25:13.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.366+0000 7fed1d690640 1 -- start start 2023-11-08T19:25:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.368+0000 7fed1d690640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.368+0000 7fed1d690640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed1806d220 con 0x7fed180c3a50 2023-11-08T19:25:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.368+0000 7fed1d690640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed180aca40 con 0x7fed1815d000 2023-11-08T19:25:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.368+0000 7fed1d690640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ccd50 con 0x7fed1806a830 2023-11-08T19:25:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.368+0000 7fed177fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fed180c3a50 0x7fed18160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37042/0 (socket says 172.21.15.119:37042) 2023-11-08T19:25:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.368+0000 7fed177fe640 1 -- 172.21.15.119:0/4255646178 learned_addr learned my addr 172.21.15.119:0/4255646178 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:13.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed167fc640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500338996 0 0) 0x7fed1806d220 con 0x7fed180c3a50 2023-11-08T19:25:13.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed00003520 con 0x7fed180c3a50 2023-11-08T19:25:13.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1739754143 0 0) 0x7fed00003520 con 0x7fed180c3a50 2023-11-08T19:25:13.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 msgr2=0x7fed180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:13.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 >> v1:172.21.15.188:6789/0 conn(0x7fed1815d000 legacy=0x7fed1806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180acd20 con 0x7fed180c3a50 2023-11-08T19:25:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fed100019f0 con 0x7fed180c3a50 2023-11-08T19:25:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed167fc640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed180cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 >> v1:172.21.15.119:6789/0 conn(0x7fed180c3a50 legacy=0x7fed18160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed000039a0 0x7fed00003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed00004d00 0x7fed00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180ccd50 con 0x7fed00004d00 2023-11-08T19:25:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000bb60 con 0x7fed000039a0 2023-11-08T19:25:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000bd20 con 0x7fed1806a830 2023-11-08T19:25:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.369+0000 7fed177fe640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed00004d00 0x7fed00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed167fc640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed16ffd640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed000039a0 0x7fed00003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed177fe640 1 -- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 msgr2=0x7fed000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed177fe640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed1806a830 0x7fed000047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed177fe640 1 -- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed000039a0 msgr2=0x7fed00003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed177fe640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed000039a0 0x7fed00003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed177fe640 1 -- 172.21.15.119:0/4255646178 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00003520 con 0x7fed00004d00 2023-11-08T19:25:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.370+0000 7fed16ffd640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed000039a0 0x7fed00003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.371+0000 7fed177fe640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed00004d00 0x7fed00005180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed180c3f00 tx=0x7fed000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602f2e4af2552bf0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.371+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10034630 con 0x7fed00004d00 2023-11-08T19:25:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.371+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1002f070 con 0x7fed00004d00 2023-11-08T19:25:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.371+0000 7fed15ffb640 1 -- 172.21.15.119:0/4255646178 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10034a10 con 0x7fed00004d00 2023-11-08T19:25:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed1d690640 1 -- 172.21.15.119:0/4255646178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed00004d00 msgr2=0x7fed00005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed1d690640 1 --2- 172.21.15.119:0/4255646178 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed00004d00 0x7fed00005180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fed180c3f00 tx=0x7fed000057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed177fe640 1 -- 172.21.15.119:0/4255646178 reap_dead start 2023-11-08T19:25:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed1d690640 1 -- 172.21.15.119:0/4255646178 shutdown_connections 2023-11-08T19:25:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed1d690640 1 -- 172.21.15.119:0/4255646178 >> 172.21.15.119:0/4255646178 conn(0x7fed180cb610 msgr2=0x7fed180cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed1d690640 1 -- 172.21.15.119:0/4255646178 shutdown_connections 2023-11-08T19:25:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.372+0000 7fed1d690640 1 -- 172.21.15.119:0/4255646178 wait complete. 2023-11-08T19:25:13.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.374+0000 7fed1d690640 1 Processor -- start 2023-11-08T19:25:13.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.374+0000 7fed1d690640 1 -- start start 2023-11-08T19:25:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed1d690640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed00004d00 0x7fed181f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed1d690640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed180af430 0x7fed180ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed1d690640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 0x7fed180b4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed1d690640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180b46c0 con 0x7fed180b3cd0 2023-11-08T19:25:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed1d690640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1815ed40 con 0x7fed180af430 2023-11-08T19:25:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed1d690640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed181f3760 con 0x7fed00004d00 2023-11-08T19:25:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 0x7fed180b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 0x7fed180b4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37360/0 (socket says 172.21.15.119:37360) 2023-11-08T19:25:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.375+0000 7fed177fe640 1 -- 172.21.15.119:0/2499871081 learned_addr learned my addr 172.21.15.119:0/2499871081 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed167fc640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed180af430 0x7fed180ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed16ffd640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed00004d00 0x7fed181f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed177fe640 1 -- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed00004d00 msgr2=0x7fed181f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed177fe640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed00004d00 0x7fed181f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed177fe640 1 -- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed180af430 msgr2=0x7fed180ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed177fe640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed180af430 0x7fed180ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed177fe640 1 -- 172.21.15.119:0/2499871081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed180b0810 con 0x7fed180b3cd0 2023-11-08T19:25:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed16ffd640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed00004d00 0x7fed181f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed177fe640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 0x7fed180b4180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fed1000a850 tx=0x7fed10001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed10001250 con 0x7fed180b3cd0 2023-11-08T19:25:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10034640 con 0x7fed180b3cd0 2023-11-08T19:25:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100349b0 con 0x7fed180b3cd0 2023-11-08T19:25:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.376+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed180b0ab0 con 0x7fed180b3cd0 2023-11-08T19:25:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.377+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1815f290 con 0x7fed180b3cd0 2023-11-08T19:25:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.378+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1815eba0 con 0x7fed180b3cd0 2023-11-08T19:25:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.379+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed10031030 con 0x7fed180b3cd0 2023-11-08T19:25:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.379+0000 7fecff7fe640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed0406aea0 0x7fed0406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.380+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fed100abe40 con 0x7fed180b3cd0 2023-11-08T19:25:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.380+0000 7fed16ffd640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed0406aea0 0x7fed0406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.380+0000 7fed16ffd640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed0406aea0 0x7fed0406d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed08000c00 tx=0x7fed08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:13.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.384+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed1815eba0 con 0x7fed180b3cd0 2023-11-08T19:25:13.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.572+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fed1815eba0 con 0x7fed180b3cd0 2023-11-08T19:25:13.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.573+0000 7fecff7fe640 1 -- 172.21.15.119:0/2499871081 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fed1815eba0 con 0x7fed180b3cd0 2023-11-08T19:25:13.575 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed0406aea0 msgr2=0x7fed0406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed0406aea0 0x7fed0406d360 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fed08000c00 tx=0x7fed08009040 comp rx=0 tx=0).stop 2023-11-08T19:25:13.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 msgr2=0x7fed180b4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 0x7fed180b4180 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fed1000a850 tx=0x7fed10001cb0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 shutdown_connections 2023-11-08T19:25:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed0406aea0 0x7fed0406d360 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed180b3cd0 0x7fed180b4180 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed180af430 0x7fed180ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 --2- 172.21.15.119:0/2499871081 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed00004d00 0x7fed181f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.575+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 >> 172.21.15.119:0/2499871081 conn(0x7fed180cb610 msgr2=0x7fed180cd670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:13.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.576+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 shutdown_connections 2023-11-08T19:25:13.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.576+0000 7fed1d690640 1 -- 172.21.15.119:0/2499871081 wait complete. 2023-11-08T19:25:13.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:13.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.816+0000 7f47db59e640 1 Processor -- start 2023-11-08T19:25:13.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.816+0000 7f47db59e640 1 -- start start 2023-11-08T19:25:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.817+0000 7f47db59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f47dc0cccf0 0x7f47dc06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.817+0000 7f47db59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc071720 con 0x7f47dc15deb0 2023-11-08T19:25:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.817+0000 7f47db59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47dc0cf5e0 con 0x7f47dc15e9a0 2023-11-08T19:25:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.817+0000 7f47db59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47dc0cf8c0 con 0x7f47dc0cccf0 2023-11-08T19:25:13.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47da59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f47dc15deb0 0x7f47dc15e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37052/0 (socket says 172.21.15.119:37052) 2023-11-08T19:25:13.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47da59c640 1 -- 172.21.15.119:0/2818880043 learned_addr learned my addr 172.21.15.119:0/2818880043 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:13.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47dad9d640 1 --2- 172.21.15.119:0/2818880043 >> v2:172.21.15.188:3300/0 conn(0x7f47dc0cccf0 0x7f47dc06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825886133 0 0) 0x7f47dc071720 con 0x7f47dc15deb0 2023-11-08T19:25:13.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47c8003520 con 0x7f47dc15deb0 2023-11-08T19:25:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1150134571 0 0) 0x7f47c8003520 con 0x7f47dc15deb0 2023-11-08T19:25:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 >> v2:172.21.15.188:3300/0 conn(0x7f47dc0cccf0 msgr2=0x7f47dc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.818+0000 7f47d9d9b640 1 --2- 172.21.15.119:0/2818880043 >> v2:172.21.15.188:3300/0 conn(0x7f47dc0cccf0 0x7f47dc06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 >> v1:172.21.15.188:6789/0 conn(0x7f47dc15e9a0 legacy=0x7f47dc06b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc0710d0 con 0x7f47dc15deb0 2023-11-08T19:25:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f47cc001670 con 0x7f47dc15deb0 2023-11-08T19:25:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 >> v1:172.21.15.119:6789/0 conn(0x7f47dc15deb0 legacy=0x7f47dc15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47c80039c0 0x7f47c8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80047c0 0x7f47c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c80051d0 0x7f47c800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0cf8c0 con 0x7f47c80051d0 2023-11-08T19:25:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c800e330 con 0x7f47c80047c0 2023-11-08T19:25:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47c800e4f0 con 0x7f47c80039c0 2023-11-08T19:25:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47dad9d640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c80051d0 0x7f47c800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47d1d9b640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80047c0 0x7f47c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47da59c640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47c80039c0 0x7f47c8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47dad9d640 1 -- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47c80039c0 msgr2=0x7f47c8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47dad9d640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47c80039c0 0x7f47c8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47dad9d640 1 -- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80047c0 msgr2=0x7f47c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.819+0000 7f47dad9d640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80047c0 0x7f47c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.820+0000 7f47dad9d640 1 -- 172.21.15.119:0/2818880043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8003520 con 0x7f47c80051d0 2023-11-08T19:25:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.820+0000 7f47da59c640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47c80039c0 0x7f47c8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.820+0000 7f47dad9d640 1 -- 172.21.15.119:0/2818880043 reap_dead start 2023-11-08T19:25:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.821+0000 7f47dad9d640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c80051d0 0x7f47c800db90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009f90 tx=0x7f47c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d96116f9863241 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.821+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d402cc20 con 0x7f47c80051d0 2023-11-08T19:25:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.821+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d400f040 con 0x7f47c80051d0 2023-11-08T19:25:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.821+0000 7f47d9d9b640 1 -- 172.21.15.119:0/2818880043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d4003990 con 0x7f47c80051d0 2023-11-08T19:25:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.821+0000 7f47db59e640 1 -- 172.21.15.119:0/2818880043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c80051d0 msgr2=0x7f47c800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.821+0000 7f47db59e640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c80051d0 0x7f47c800db90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f47d4009f90 tx=0x7f47c80057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.822+0000 7f47db59e640 1 -- 172.21.15.119:0/2818880043 shutdown_connections 2023-11-08T19:25:13.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.822+0000 7f47db59e640 1 --2- 172.21.15.119:0/2818880043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47c80051d0 0x7f47c800db90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.822+0000 7f47db59e640 1 -- 172.21.15.119:0/2818880043 >> 172.21.15.119:0/2818880043 conn(0x7f47dc0ceac0 msgr2=0x7f47dc0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:13.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.822+0000 7f47db59e640 1 -- 172.21.15.119:0/2818880043 shutdown_connections 2023-11-08T19:25:13.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.822+0000 7f47db59e640 1 -- 172.21.15.119:0/2818880043 wait complete. 2023-11-08T19:25:13.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.823+0000 7f47db59e640 1 Processor -- start 2023-11-08T19:25:13.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 -- start start 2023-11-08T19:25:13.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80051d0 0x7f47dc155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47dc0ac8d0 0x7f47dc0acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 0x7f47dc1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc0ad290 con 0x7f47dc1f69d0 2023-11-08T19:25:13.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc1f31c0 con 0x7f47c80051d0 2023-11-08T19:25:13.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.824+0000 7f47db59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc1f34a0 con 0x7f47dc0ac8d0 2023-11-08T19:25:13.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 0x7f47dc1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 0x7f47dc1f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37376/0 (socket says 172.21.15.119:37376) 2023-11-08T19:25:13.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 -- 172.21.15.119:0/1088999849 learned_addr learned my addr 172.21.15.119:0/1088999849 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:13.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47da59c640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80051d0 0x7f47dc155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47d1d9b640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47dc0ac8d0 0x7f47dc0acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 -- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47dc0ac8d0 msgr2=0x7f47dc0acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47dc0ac8d0 0x7f47dc0acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 -- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80051d0 msgr2=0x7f47dc155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:13.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80051d0 0x7f47dc155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:13.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47dad9d640 1 -- 172.21.15.119:0/1088999849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc1f3780 con 0x7f47dc1f69d0 2023-11-08T19:25:13.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.825+0000 7f47d1d9b640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47dc0ac8d0 0x7f47dc0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.826+0000 7f47dad9d640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 0x7f47dc1f6e80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47d402c690 tx=0x7f47d4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.826+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d402cea0 con 0x7f47dc1f69d0 2023-11-08T19:25:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.826+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d4004230 con 0x7f47dc1f69d0 2023-11-08T19:25:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.826+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d40045e0 con 0x7f47dc1f69d0 2023-11-08T19:25:13.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.826+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc1f3a60 con 0x7f47dc1f69d0 2023-11-08T19:25:13.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.826+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc156210 con 0x7f47dc1f69d0 2023-11-08T19:25:13.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.827+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc04ece0 con 0x7f47dc1f69d0 2023-11-08T19:25:13.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.828+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47d4004740 con 0x7f47dc1f69d0 2023-11-08T19:25:13.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.829+0000 7f47d37fe640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47bc06ae70 0x7f47bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:13.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.829+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f47d40ac270 con 0x7f47dc1f69d0 2023-11-08T19:25:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.829+0000 7f47da59c640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47bc06ae70 0x7f47bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.830+0000 7f47da59c640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47bc06ae70 0x7f47bc06d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:13.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:13.834+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47d4077140 con 0x7f47dc1f69d0 2023-11-08T19:25:14.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.023+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f47dc0cb1c0 con 0x7f47dc1f69d0 2023-11-08T19:25:14.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.024+0000 7f47d37fe640 1 -- 172.21.15.119:0/1088999849 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f47d407a760 con 0x7f47dc1f69d0 2023-11-08T19:25:14.027 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:14.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47bc06ae70 msgr2=0x7f47bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47bc06ae70 0x7f47bc06d330 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47cc000c00 tx=0x7f47cc001550 comp rx=0 tx=0).stop 2023-11-08T19:25:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 msgr2=0x7f47dc1f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 0x7f47dc1f6e80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f47d402c690 tx=0x7f47d4009750 comp rx=0 tx=0).stop 2023-11-08T19:25:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 shutdown_connections 2023-11-08T19:25:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47bc06ae70 0x7f47bc06d330 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47dc1f69d0 0x7f47dc1f6e80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47dc0ac8d0 0x7f47dc0acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 --2- 172.21.15.119:0/1088999849 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47c80051d0 0x7f47dc155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.026+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 >> 172.21.15.119:0/1088999849 conn(0x7f47dc0ceac0 msgr2=0x7f47dc06bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.027+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 shutdown_connections 2023-11-08T19:25:14.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.027+0000 7f47db59e640 1 -- 172.21.15.119:0/1088999849 wait complete. 2023-11-08T19:25:14.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:25:14.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.269+0000 7f11bbada640 1 Processor -- start 2023-11-08T19:25:14.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.269+0000 7f11bbada640 1 -- start start 2023-11-08T19:25:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11bbada640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 0x7f11b40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11bbada640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b415df70 con 0x7f11b415d6c0 2023-11-08T19:25:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11bbada640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b40d1e60 con 0x7f11b40cfa00 2023-11-08T19:25:14.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11bbada640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b415daa0 con 0x7f11b4161870 2023-11-08T19:25:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b9876640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f11b415d6c0 0x7f11b40d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37058/0 (socket says 172.21.15.119:37058) 2023-11-08T19:25:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b9876640 1 -- 172.21.15.119:0/1810657368 learned_addr learned my addr 172.21.15.119:0/1810657368 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11ba077640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 0x7f11b40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834892867 0 0) 0x7f11b415df70 con 0x7f11b415d6c0 2023-11-08T19:25:14.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a0003520 con 0x7f11b415d6c0 2023-11-08T19:25:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508255161 0 0) 0x7f11a0003520 con 0x7f11b415d6c0 2023-11-08T19:25:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 msgr2=0x7f11b40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 0x7f11b40cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 >> v1:172.21.15.188:6789/0 conn(0x7f11b40cfa00 legacy=0x7f11b4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4161d40 con 0x7f11b415d6c0 2023-11-08T19:25:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f11b0001670 con 0x7f11b415d6c0 2023-11-08T19:25:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 >> v1:172.21.15.119:6789/0 conn(0x7f11b415d6c0 legacy=0x7f11b40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 0x7f11a00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a0003f20 0x7f11a00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b415daa0 con 0x7f11a0004d00 2023-11-08T19:25:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a000bb10 con 0x7f11a0003f20 2023-11-08T19:25:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.271+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a000bcd0 con 0x7f11b4161870 2023-11-08T19:25:14.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11ba077640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11b9876640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 0x7f11a00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11b9075640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a0003f20 0x7f11a00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11ba077640 1 -- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 msgr2=0x7f11a00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11ba077640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b4161870 0x7f11a00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11ba077640 1 -- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a0003f20 msgr2=0x7f11a00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11ba077640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a0003f20 0x7f11a00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11ba077640 1 -- 172.21.15.119:0/1810657368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0003520 con 0x7f11a0004d00 2023-11-08T19:25:14.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.272+0000 7f11b9075640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a0003f20 0x7f11a00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.273+0000 7f11ba077640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11a0005180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009f90 tx=0x7f11a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3846fc5b97a881f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.273+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac02cbd0 con 0x7f11a0004d00 2023-11-08T19:25:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.273+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac00f040 con 0x7f11a0004d00 2023-11-08T19:25:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.273+0000 7f11b8874640 1 -- 172.21.15.119:0/1810657368 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac034440 con 0x7f11a0004d00 2023-11-08T19:25:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11bbada640 1 -- 172.21.15.119:0/1810657368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 msgr2=0x7f11a0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11bbada640 1 --2- 172.21.15.119:0/1810657368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11a0005180 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009f90 tx=0x7f11a00057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11ba077640 1 -- 172.21.15.119:0/1810657368 reap_dead start 2023-11-08T19:25:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11bbada640 1 -- 172.21.15.119:0/1810657368 shutdown_connections 2023-11-08T19:25:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11bbada640 1 -- 172.21.15.119:0/1810657368 >> 172.21.15.119:0/1810657368 conn(0x7f11b406a9f0 msgr2=0x7f11b406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11bbada640 1 -- 172.21.15.119:0/1810657368 shutdown_connections 2023-11-08T19:25:14.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.274+0000 7f11bbada640 1 -- 172.21.15.119:0/1810657368 wait complete. 2023-11-08T19:25:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.276+0000 7f11bbada640 1 Processor -- start 2023-11-08T19:25:14.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.276+0000 7f11bbada640 1 -- start start 2023-11-08T19:25:14.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11bbada640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11b40c96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11bbada640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11b41f0260 0x7f11b41f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11bbada640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b41f6a00 0x7f11b41f6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11bbada640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4158b00 con 0x7f11a0004d00 2023-11-08T19:25:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11bbada640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c9c00 con 0x7f11b41f0260 2023-11-08T19:25:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11bbada640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40c9e50 con 0x7f11b41f6a00 2023-11-08T19:25:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11b40c96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11b40c96c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37398/0 (socket says 172.21.15.119:37398) 2023-11-08T19:25:14.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 -- 172.21.15.119:0/3052090101 learned_addr learned my addr 172.21.15.119:0/3052090101 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 -- 172.21.15.119:0/3052090101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b41f6a00 msgr2=0x7f11b41f6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b41f6a00 0x7f11b41f6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 -- 172.21.15.119:0/3052090101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11b41f0260 msgr2=0x7f11b41f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11b41f0260 0x7f11b41f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.277+0000 7f11b9876640 1 -- 172.21.15.119:0/3052090101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b41f0c00 con 0x7f11a0004d00 2023-11-08T19:25:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.278+0000 7f11b9876640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11b40c96c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e8c0 tx=0x7f11b000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.278+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b000c7f0 con 0x7f11a0004d00 2023-11-08T19:25:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.278+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b000c950 con 0x7f11a0004d00 2023-11-08T19:25:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.278+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b41552c0 con 0x7f11a0004d00 2023-11-08T19:25:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.278+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b000cd00 con 0x7f11a0004d00 2023-11-08T19:25:14.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.278+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b41547e0 con 0x7f11a0004d00 2023-11-08T19:25:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.279+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b4074430 con 0x7f11a0004d00 2023-11-08T19:25:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.280+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11b001d460 con 0x7f11a0004d00 2023-11-08T19:25:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.281+0000 7f11aa7fc640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f119806ae70 0x7f119806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.281+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f11b00899a0 con 0x7f11a0004d00 2023-11-08T19:25:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.281+0000 7f11b9075640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f119806ae70 0x7f119806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.282+0000 7f11b9075640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f119806ae70 0x7f119806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11a40016d0 tx=0x7f11a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.285+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11b0054760 con 0x7f11a0004d00 2023-11-08T19:25:14.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.488+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f11b40d10f0 con 0x7f11a0004d00 2023-11-08T19:25:14.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.489+0000 7f11aa7fc640 1 -- 172.21.15.119:0/3052090101 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f11b000ce60 con 0x7f11a0004d00 2023-11-08T19:25:14.491 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:25:14.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f119806ae70 msgr2=0x7f119806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f119806ae70 0x7f119806d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f11a40016d0 tx=0x7f11a4009040 comp rx=0 tx=0).stop 2023-11-08T19:25:14.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 msgr2=0x7f11b40c96c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11b40c96c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f11b000e8c0 tx=0x7f11b000ed80 comp rx=0 tx=0).stop 2023-11-08T19:25:14.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 shutdown_connections 2023-11-08T19:25:14.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f119806ae70 0x7f119806d330 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11b41f6a00 0x7f11b41f6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11b41f0260 0x7f11b41f06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 --2- 172.21.15.119:0/3052090101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a0004d00 0x7f11b40c96c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 >> 172.21.15.119:0/3052090101 conn(0x7f11b406a9f0 msgr2=0x7f11b40b68a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:14.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 shutdown_connections 2023-11-08T19:25:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.492+0000 7f11bbada640 1 -- 172.21.15.119:0/3052090101 wait complete. 2023-11-08T19:25:14.512 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:14.513 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:14.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:14.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.817+0000 7f0dd9103640 1 Processor -- start 2023-11-08T19:25:14.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.817+0000 7f0dd9103640 1 -- start start 2023-11-08T19:25:14.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.817+0000 7f0dd9103640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 0x7f0dd4165110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.817+0000 7f0dd9103640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd406cff0 con 0x7f0dd415b0d0 2023-11-08T19:25:14.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd9103640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd406d2d0 con 0x7f0dd40ad7d0 2023-11-08T19:25:14.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd9103640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd406bd20 con 0x7f0dd4162d20 2023-11-08T19:25:14.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0dd415b0d0 0x7f0dd415b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37068/0 (socket says 172.21.15.119:37068) 2023-11-08T19:25:14.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd3fff640 1 -- 172.21.15.119:0/1058330925 learned_addr learned my addr 172.21.15.119:0/1058330925 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:14.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315331024 0 0) 0x7f0dd406cff0 con 0x7f0dd415b0d0 2023-11-08T19:25:14.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dbc005730 con 0x7f0dd415b0d0 2023-11-08T19:25:14.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.818+0000 7f0dd8902640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 0x7f0dd4165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3552561266 0 0) 0x7f0dbc005730 con 0x7f0dd415b0d0 2023-11-08T19:25:14.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 msgr2=0x7f0dd4165110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 0x7f0dd4165110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 >> v1:172.21.15.188:6789/0 conn(0x7f0dd40ad7d0 legacy=0x7f0dd4070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd406d760 con 0x7f0dd415b0d0 2023-11-08T19:25:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f0dc4001670 con 0x7f0dd415b0d0 2023-11-08T19:25:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 >> v1:172.21.15.119:6789/0 conn(0x7f0dd415b0d0 legacy=0x7f0dd415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc005bb0 0x7f0dbc005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dbc0069d0 0x7f0dbc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 0x7f0dbc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd406bd20 con 0x7f0dbc0069d0 2023-11-08T19:25:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0076c0 con 0x7f0dbc005bb0 2023-11-08T19:25:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc007880 con 0x7f0dd4162d20 2023-11-08T19:25:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dbc0069d0 0x7f0dbc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd3fff640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc005bb0 0x7f0dbc005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd8902640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 0x7f0dbc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 msgr2=0x7f0dbc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.819+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4162d20 0x7f0dbc00db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.820+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc005bb0 msgr2=0x7f0dbc005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:14.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.820+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc005bb0 0x7f0dbc005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.820+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1058330925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc005730 con 0x7f0dbc0069d0 2023-11-08T19:25:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.820+0000 7f0dd3fff640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc005bb0 0x7f0dbc005fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dbc0069d0 0x7f0dbc006e50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001750 tx=0x7f0dbc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282efcd8a8f88c99 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80052d0 con 0x7f0dbc0069d0 2023-11-08T19:25:14.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc802e050 con 0x7f0dbc0069d0 2023-11-08T19:25:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd2ffd640 1 -- 172.21.15.119:0/1058330925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80035f0 con 0x7f0dbc0069d0 2023-11-08T19:25:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd9103640 1 -- 172.21.15.119:0/1058330925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dbc0069d0 msgr2=0x7f0dbc006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1058330925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dbc0069d0 0x7f0dbc006e50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8001750 tx=0x7f0dbc007a00 comp rx=0 tx=0).stop 2023-11-08T19:25:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd8902640 1 -- 172.21.15.119:0/1058330925 reap_dead start 2023-11-08T19:25:14.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd9103640 1 -- 172.21.15.119:0/1058330925 shutdown_connections 2023-11-08T19:25:14.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd9103640 1 -- 172.21.15.119:0/1058330925 >> 172.21.15.119:0/1058330925 conn(0x7f0dd40b0560 msgr2=0x7f0dd40b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:14.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.821+0000 7f0dd9103640 1 -- 172.21.15.119:0/1058330925 shutdown_connections 2023-11-08T19:25:14.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.822+0000 7f0dd9103640 1 -- 172.21.15.119:0/1058330925 wait complete. 2023-11-08T19:25:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.823+0000 7f0dd9103640 1 Processor -- start 2023-11-08T19:25:14.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.823+0000 7f0dd9103640 1 -- start start 2023-11-08T19:25:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.824+0000 7f0dd9103640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc0069d0 0x7f0dd40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.824+0000 7f0dd9103640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 0x7f0dd4155100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.824+0000 7f0dd9103640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4155640 0x7f0dd4155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.824+0000 7f0dd9103640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40b2050 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.824+0000 7f0dd9103640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40c4f30 con 0x7f0dbc0069d0 2023-11-08T19:25:14.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.824+0000 7f0dd9103640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd40c6c60 con 0x7f0dd4155640 2023-11-08T19:25:14.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 0x7f0dd4155100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 0x7f0dd4155100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37414/0 (socket says 172.21.15.119:37414) 2023-11-08T19:25:14.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1756614200 learned_addr learned my addr 172.21.15.119:0/1756614200 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd3fff640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc0069d0 0x7f0dd40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4155640 msgr2=0x7f0dd4155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4155640 0x7f0dd4155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc0069d0 msgr2=0x7f0dd40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc0069d0 0x7f0dd40b1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:14.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 -- 172.21.15.119:0/1756614200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd40c6330 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.825+0000 7f0dd37fe640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 0x7f0dd4155100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8008fd0 tx=0x7f0dc8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.826+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80076c0 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.826+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8007a30 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.826+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8006be0 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.826+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd40c6610 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.826+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd40b36f0 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.827+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd404ece0 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.828+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0dc8006d40 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.829+0000 7f0dd0ff9640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0dc006aea0 0x7f0dc006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:14.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.829+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f0dc80aba20 con 0x7f0dd40b1bd0 2023-11-08T19:25:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.829+0000 7f0dd3fff640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0dc006aea0 0x7f0dc006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.830+0000 7f0dd3fff640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0dc006aea0 0x7f0dc006d360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007950 tx=0x7f0dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:14.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:14.833+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0dc80768f0 con 0x7f0dd40b1bd0 2023-11-08T19:25:15.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.024+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dd40b1320 con 0x7f0dd40b1bd0 2023-11-08T19:25:15.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.026+0000 7f0dd0ff9640 1 -- 172.21.15.119:0/1756614200 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f0dc8079f10 con 0x7f0dd40b1bd0 2023-11-08T19:25:15.029 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:15.030 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:15.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0dc006aea0 msgr2=0x7f0dc006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0dc006aea0 0x7f0dc006d360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4007950 tx=0x7f0dc4009040 comp rx=0 tx=0).stop 2023-11-08T19:25:15.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 msgr2=0x7f0dd4155100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 0x7f0dd4155100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8008fd0 tx=0x7f0dc8001120 comp rx=0 tx=0).stop 2023-11-08T19:25:15.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 shutdown_connections 2023-11-08T19:25:15.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0dc006aea0 0x7f0dc006d360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0dd4155640 0x7f0dd4155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0dd40b1bd0 0x7f0dd4155100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 --2- 172.21.15.119:0/1756614200 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0dbc0069d0 0x7f0dd40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.029+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 >> 172.21.15.119:0/1756614200 conn(0x7f0dd40b0560 msgr2=0x7f0dd4156bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:15.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.030+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 shutdown_connections 2023-11-08T19:25:15.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.030+0000 7f0dd9103640 1 -- 172.21.15.119:0/1756614200 wait complete. 2023-11-08T19:25:15.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:15.051 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:15.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.256+0000 7ff5cf1f5640 1 Processor -- start 2023-11-08T19:25:15.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.256+0000 7ff5cf1f5640 1 -- start start 2023-11-08T19:25:15.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cf1f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5c8143600 0x7ff5c80c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cf1f5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5c806d250 0x7ff5c815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cf1f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c815e440 con 0x7ff5c815d9f0 2023-11-08T19:25:15.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cf1f5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80c4f50 con 0x7ff5c806d250 2023-11-08T19:25:15.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cf1f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80c4960 con 0x7ff5c8143600 2023-11-08T19:25:15.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cd792640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5c806d250 0x7ff5c815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cd792640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5c806d250 0x7ff5c815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37416/0 (socket says 172.21.15.119:37416) 2023-11-08T19:25:15.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5cd792640 1 -- 172.21.15.119:0/2139816168 learned_addr learned my addr 172.21.15.119:0/2139816168 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:15.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.258+0000 7ff5ccf91640 1 --2- 172.21.15.119:0/2139816168 >> v2:172.21.15.188:3300/0 conn(0x7ff5c8143600 0x7ff5c80c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.259+0000 7ff5cd792640 1 -- 172.21.15.119:0/2139816168 >> v2:172.21.15.188:3300/0 conn(0x7ff5c8143600 msgr2=0x7ff5c80c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:15.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.259+0000 7ff5cd792640 1 --2- 172.21.15.119:0/2139816168 >> v2:172.21.15.188:3300/0 conn(0x7ff5c8143600 0x7ff5c80c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.259+0000 7ff5cd792640 1 -- 172.21.15.119:0/2139816168 >> v1:172.21.15.188:6789/0 conn(0x7ff5c815d9f0 legacy=0x7ff5c806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:15.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.259+0000 7ff5cd792640 1 -- 172.21.15.119:0/2139816168 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c815e720 con 0x7ff5c806d250 2023-11-08T19:25:15.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.260+0000 7ff5cd792640 1 --2- 172.21.15.119:0/2139816168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c806d250 0x7ff5c815de60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c3f30 tx=0x7ff5c002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdc86b9aeb29fb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:15.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.260+0000 7ff5c77fe640 1 -- 172.21.15.119:0/2139816168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c002c9f0 con 0x7ff5c806d250 2023-11-08T19:25:15.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.260+0000 7ff5c77fe640 1 -- 172.21.15.119:0/2139816168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c002cb50 con 0x7ff5c806d250 2023-11-08T19:25:15.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.260+0000 7ff5c77fe640 1 -- 172.21.15.119:0/2139816168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c00343f0 con 0x7ff5c806d250 2023-11-08T19:25:15.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/2139816168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c806d250 msgr2=0x7ff5c815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/2139816168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c806d250 0x7ff5c815de60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff5c80c3f30 tx=0x7ff5c002be50 comp rx=0 tx=0).stop 2023-11-08T19:25:15.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/2139816168 shutdown_connections 2023-11-08T19:25:15.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/2139816168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c806d250 0x7ff5c815de60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/2139816168 >> v2:172.21.15.188:3300/0 conn(0x7ff5c8143600 0x7ff5c80c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/2139816168 >> 172.21.15.119:0/2139816168 conn(0x7ff5c80cb610 msgr2=0x7ff5c80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:15.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/2139816168 shutdown_connections 2023-11-08T19:25:15.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.261+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/2139816168 wait complete. 2023-11-08T19:25:15.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.263+0000 7ff5cf1f5640 1 Processor -- start 2023-11-08T19:25:15.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.263+0000 7ff5cf1f5640 1 -- start start 2023-11-08T19:25:15.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cf1f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5c806d250 0x7ff5c80b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cf1f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 0x7ff5c80ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cf1f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5c815d9f0 0x7ff5c8152e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cf1f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80b4820 con 0x7ff5c8143600 2023-11-08T19:25:15.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cf1f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80acf50 con 0x7ff5c806d250 2023-11-08T19:25:15.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cf1f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80ad230 con 0x7ff5c815d9f0 2023-11-08T19:25:15.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 0x7ff5c80ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 0x7ff5c80ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37428/0 (socket says 172.21.15.119:37428) 2023-11-08T19:25:15.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 -- 172.21.15.119:0/3705915799 learned_addr learned my addr 172.21.15.119:0/3705915799 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:15.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5ccf91640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5c806d250 0x7ff5c80b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5cd792640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5c815d9f0 0x7ff5c8152e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 -- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5c815d9f0 msgr2=0x7ff5c8152e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5c815d9f0 0x7ff5c8152e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 -- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5c806d250 msgr2=0x7ff5c80b0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5c806d250 0x7ff5c80b0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5c7fff640 1 -- 172.21.15.119:0/3705915799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c0009b30 con 0x7ff5c8143600 2023-11-08T19:25:15.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.264+0000 7ff5ccf91640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5c806d250 0x7ff5c80b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:15.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.265+0000 7ff5c7fff640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 0x7ff5c80ac8c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b3f0 tx=0x7ff5b400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:15.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.265+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4011830 con 0x7ff5c8143600 2023-11-08T19:25:15.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.265+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b4011990 con 0x7ff5c8143600 2023-11-08T19:25:15.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.265+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4011d40 con 0x7ff5c8143600 2023-11-08T19:25:15.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.265+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c80ad510 con 0x7ff5c8143600 2023-11-08T19:25:15.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.265+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c80adad0 con 0x7ff5c8143600 2023-11-08T19:25:15.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.266+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c8074430 con 0x7ff5c8143600 2023-11-08T19:25:15.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.267+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5b401a400 con 0x7ff5c8143600 2023-11-08T19:25:15.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.268+0000 7ff5c57fa640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5ac06ae70 0x7ff5ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.268+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff5b4089740 con 0x7ff5c8143600 2023-11-08T19:25:15.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.268+0000 7ff5ccf91640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5ac06ae70 0x7ff5ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.269+0000 7ff5ccf91640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5ac06ae70 0x7ff5ac06d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc006fd0 tx=0x7ff5bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:15.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.272+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5b4054500 con 0x7ff5c8143600 2023-11-08T19:25:15.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.460+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5c80b0ba0 con 0x7ff5c8143600 2023-11-08T19:25:15.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.461+0000 7ff5c57fa640 1 -- 172.21.15.119:0/3705915799 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5b4057b20 con 0x7ff5c8143600 2023-11-08T19:25:15.463 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:15.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.463+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5ac06ae70 msgr2=0x7ff5ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.463+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5ac06ae70 0x7ff5ac06d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc006fd0 tx=0x7ff5bc008040 comp rx=0 tx=0).stop 2023-11-08T19:25:15.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.463+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 msgr2=0x7ff5c80ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.463+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 0x7ff5c80ac8c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400b3f0 tx=0x7ff5b400b8b0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 shutdown_connections 2023-11-08T19:25:15.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5ac06ae70 0x7ff5ac06d330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5c815d9f0 0x7ff5c8152e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5c8143600 0x7ff5c80ac8c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 --2- 172.21.15.119:0/3705915799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5c806d250 0x7ff5c80b0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 >> 172.21.15.119:0/3705915799 conn(0x7ff5c80cb610 msgr2=0x7ff5c8141660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:15.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 shutdown_connections 2023-11-08T19:25:15.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.464+0000 7ff5cf1f5640 1 -- 172.21.15.119:0/3705915799 wait complete. 2023-11-08T19:25:15.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:15.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.692+0000 7ff58847c640 1 Processor -- start 2023-11-08T19:25:15.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.692+0000 7ff58847c640 1 -- start start 2023-11-08T19:25:15.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.693+0000 7ff58847c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff580144960 0x7ff58015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.693+0000 7ff58847c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58015f1d0 0x7ff5800c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.693+0000 7ff58847c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58015ea60 con 0x7ff580144580 2023-11-08T19:25:15.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.693+0000 7ff58847c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5800c9150 con 0x7ff580144960 2023-11-08T19:25:15.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.693+0000 7ff58847c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800c9360 con 0x7ff58015f1d0 2023-11-08T19:25:15.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff580144960 0x7ff58015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff580144960 0x7ff58015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37440/0 (socket says 172.21.15.119:37440) 2023-11-08T19:25:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 -- 172.21.15.119:0/846006031 learned_addr learned my addr 172.21.15.119:0/846006031 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff586a19640 1 --2- 172.21.15.119:0/846006031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58015f1d0 0x7ff5800c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 -- 172.21.15.119:0/846006031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58015f1d0 msgr2=0x7ff5800c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:15.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 --2- 172.21.15.119:0/846006031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58015f1d0 0x7ff5800c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 -- 172.21.15.119:0/846006031 >> v1:172.21.15.188:6789/0 conn(0x7ff580144580 legacy=0x7ff5800cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.694+0000 7ff585a17640 1 -- 172.21.15.119:0/846006031 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5800c9640 con 0x7ff580144960 2023-11-08T19:25:15.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.695+0000 7ff585a17640 1 --2- 172.21.15.119:0/846006031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144960 0x7ff58015bff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5800c9290 tx=0x7ff57002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9066dd49e0fb350e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:15.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff585216640 1 -- 172.21.15.119:0/846006031 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5700317c0 con 0x7ff580144960 2023-11-08T19:25:15.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff585216640 1 -- 172.21.15.119:0/846006031 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff570031920 con 0x7ff580144960 2023-11-08T19:25:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff585216640 1 -- 172.21.15.119:0/846006031 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff570031cb0 con 0x7ff580144960 2023-11-08T19:25:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 -- 172.21.15.119:0/846006031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144960 msgr2=0x7ff58015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 --2- 172.21.15.119:0/846006031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144960 0x7ff58015bff0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff5800c9290 tx=0x7ff57002b860 comp rx=0 tx=0).stop 2023-11-08T19:25:15.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 -- 172.21.15.119:0/846006031 shutdown_connections 2023-11-08T19:25:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 --2- 172.21.15.119:0/846006031 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144960 0x7ff58015bff0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 --2- 172.21.15.119:0/846006031 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff58015f1d0 0x7ff5800c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 -- 172.21.15.119:0/846006031 >> 172.21.15.119:0/846006031 conn(0x7ff5800cb610 msgr2=0x7ff5800cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 -- 172.21.15.119:0/846006031 shutdown_connections 2023-11-08T19:25:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.696+0000 7ff58847c640 1 -- 172.21.15.119:0/846006031 wait complete. 2023-11-08T19:25:15.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.698+0000 7ff58847c640 1 Processor -- start 2023-11-08T19:25:15.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.698+0000 7ff58847c640 1 -- start start 2023-11-08T19:25:15.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff58847c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 0x7ff5800b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff58847c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff580144960 0x7ff5800b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff58847c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58015f1d0 0x7ff5800b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff58847c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800b2e90 con 0x7ff580144580 2023-11-08T19:25:15.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff58847c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800b3170 con 0x7ff58015f1d0 2023-11-08T19:25:15.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff58847c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5800b3450 con 0x7ff580144960 2023-11-08T19:25:15.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff586218640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 0x7ff5800b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff586218640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 0x7ff5800b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37454/0 (socket says 172.21.15.119:37454) 2023-11-08T19:25:15.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.699+0000 7ff586218640 1 -- 172.21.15.119:0/212739042 learned_addr learned my addr 172.21.15.119:0/212739042 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:15.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff585a17640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff580144960 0x7ff5800b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586a19640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58015f1d0 0x7ff5800b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586218640 1 -- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff580144960 msgr2=0x7ff5800b4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586218640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff580144960 0x7ff5800b4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586218640 1 -- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58015f1d0 msgr2=0x7ff5800b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586218640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58015f1d0 0x7ff5800b45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586218640 1 -- 172.21.15.119:0/212739042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700094d0 con 0x7ff580144580 2023-11-08T19:25:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff585a17640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff580144960 0x7ff5800b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff586218640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 0x7ff5800b28b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c430 tx=0x7ff57c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c011790 con 0x7ff580144580 2023-11-08T19:25:15.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c0118f0 con 0x7ff580144580 2023-11-08T19:25:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c011ce0 con 0x7ff580144580 2023-11-08T19:25:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.700+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5800b3730 con 0x7ff580144580 2023-11-08T19:25:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.701+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5800adda0 con 0x7ff580144580 2023-11-08T19:25:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.702+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5800b3c50 con 0x7ff580144580 2023-11-08T19:25:15.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.703+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff57c01c460 con 0x7ff580144580 2023-11-08T19:25:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.704+0000 7ff576ffd640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff56406ae70 0x7ff56406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:15.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.704+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff57c089940 con 0x7ff580144580 2023-11-08T19:25:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.704+0000 7ff585a17640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff56406ae70 0x7ff56406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.705+0000 7ff585a17640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff56406ae70 0x7ff56406d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff570036000 tx=0x7ff57003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:15.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.708+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5800b3c50 con 0x7ff580144580 2023-11-08T19:25:15.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.896+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff5800b3c50 con 0x7ff580144580 2023-11-08T19:25:15.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.897+0000 7ff576ffd640 1 -- 172.21.15.119:0/212739042 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff5800b3c50 con 0x7ff580144580 2023-11-08T19:25:15.899 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:15.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff56406ae70 msgr2=0x7ff56406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff56406ae70 0x7ff56406d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff570036000 tx=0x7ff57003d040 comp rx=0 tx=0).stop 2023-11-08T19:25:15.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 msgr2=0x7ff5800b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:15.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 0x7ff5800b28b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00c430 tx=0x7ff57c00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 shutdown_connections 2023-11-08T19:25:15.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff56406ae70 0x7ff56406d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff58015f1d0 0x7ff5800b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff580144960 0x7ff5800b4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 --2- 172.21.15.119:0/212739042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff580144580 0x7ff5800b28b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:15.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 >> 172.21.15.119:0/212739042 conn(0x7ff5800cb610 msgr2=0x7ff58015bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:15.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 shutdown_connections 2023-11-08T19:25:15.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:15.899+0000 7ff58847c640 1 -- 172.21.15.119:0/212739042 wait complete. 2023-11-08T19:25:15.919 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:16.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.131+0000 7f92b207c640 1 Processor -- start 2023-11-08T19:25:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.132+0000 7f92b207c640 1 -- start start 2023-11-08T19:25:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.133+0000 7f92b207c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92ac06a120 0x7f92ac0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.133+0000 7f92b207c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92ac15d520 0x7f92ac15d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.133+0000 7f92b207c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92ac0cfad0 con 0x7f92ac06a500 2023-11-08T19:25:16.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.133+0000 7f92b207c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac06e790 con 0x7f92ac15d520 2023-11-08T19:25:16.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.133+0000 7f92b207c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92ac06d9d0 con 0x7f92ac06a120 2023-11-08T19:25:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92ac15d520 0x7f92ac15d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f92ac15d520 0x7f92ac15d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37458/0 (socket says 172.21.15.119:37458) 2023-11-08T19:25:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 -- 172.21.15.119:0/2583335506 learned_addr learned my addr 172.21.15.119:0/2583335506 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92ab7fe640 1 --2- 172.21.15.119:0/2583335506 >> v2:172.21.15.188:3300/0 conn(0x7f92ac06a120 0x7f92ac0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 -- 172.21.15.119:0/2583335506 >> v2:172.21.15.188:3300/0 conn(0x7f92ac06a120 msgr2=0x7f92ac0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 --2- 172.21.15.119:0/2583335506 >> v2:172.21.15.188:3300/0 conn(0x7f92ac06a120 0x7f92ac0cf410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 -- 172.21.15.119:0/2583335506 >> v1:172.21.15.188:6789/0 conn(0x7f92ac06a500 legacy=0x7f92ac0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.134+0000 7f92abfff640 1 -- 172.21.15.119:0/2583335506 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac06dcb0 con 0x7f92ac15d520 2023-11-08T19:25:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.135+0000 7f92abfff640 1 --2- 172.21.15.119:0/2583335506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac15d520 0x7f92ac15d970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92ac06c600 tx=0x7f92a402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6328432ec405e4e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92aaffd640 1 -- 172.21.15.119:0/2583335506 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a402ca80 con 0x7f92ac15d520 2023-11-08T19:25:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92aaffd640 1 -- 172.21.15.119:0/2583335506 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a402cbe0 con 0x7f92ac15d520 2023-11-08T19:25:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92aaffd640 1 -- 172.21.15.119:0/2583335506 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a40343f0 con 0x7f92ac15d520 2023-11-08T19:25:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92b207c640 1 -- 172.21.15.119:0/2583335506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac15d520 msgr2=0x7f92ac15d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92b207c640 1 --2- 172.21.15.119:0/2583335506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac15d520 0x7f92ac15d970 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f92ac06c600 tx=0x7f92a402bee0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92b207c640 1 -- 172.21.15.119:0/2583335506 shutdown_connections 2023-11-08T19:25:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92b207c640 1 --2- 172.21.15.119:0/2583335506 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac15d520 0x7f92ac15d970 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92b207c640 1 --2- 172.21.15.119:0/2583335506 >> v2:172.21.15.188:3300/0 conn(0x7f92ac06a120 0x7f92ac0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.136+0000 7f92b207c640 1 -- 172.21.15.119:0/2583335506 >> 172.21.15.119:0/2583335506 conn(0x7f92ac0cfd20 msgr2=0x7f92ac0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.137+0000 7f92b207c640 1 -- 172.21.15.119:0/2583335506 shutdown_connections 2023-11-08T19:25:16.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.137+0000 7f92b207c640 1 -- 172.21.15.119:0/2583335506 wait complete. 2023-11-08T19:25:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.138+0000 7f92b207c640 1 Processor -- start 2023-11-08T19:25:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.138+0000 7f92b207c640 1 -- start start 2023-11-08T19:25:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.139+0000 7f92b207c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 0x7f92ac14b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.139+0000 7f92b207c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac06a500 0x7f92ac14c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.139+0000 7f92b207c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac15d520 0x7f92ac0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.139+0000 7f92b207c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac1529d0 con 0x7f92ac06a120 2023-11-08T19:25:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.139+0000 7f92b207c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c4410 con 0x7f92ac15d520 2023-11-08T19:25:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.139+0000 7f92b207c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92ac0c46f0 con 0x7f92ac06a500 2023-11-08T19:25:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 0x7f92ac14b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 0x7f92ac14b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37470/0 (socket says 172.21.15.119:37470) 2023-11-08T19:25:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 -- 172.21.15.119:0/2275821445 learned_addr learned my addr 172.21.15.119:0/2275821445 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92abfff640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac15d520 0x7f92ac0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 -- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac06a500 msgr2=0x7f92ac14c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92a2ffd640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac06a500 0x7f92ac14c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac06a500 0x7f92ac14c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 -- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac15d520 msgr2=0x7f92ac0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac15d520 0x7f92ac0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92ab7fe640 1 -- 172.21.15.119:0/2275821445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4009b30 con 0x7f92ac06a120 2023-11-08T19:25:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92abfff640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac15d520 0x7f92ac0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.140+0000 7f92a2ffd640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac06a500 0x7f92ac14c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.141+0000 7f92ab7fe640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 0x7f92ac14b250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d800 tx=0x7f929c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.141+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00b6c0 con 0x7f92ac06a120 2023-11-08T19:25:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.141+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c00b820 con 0x7f92ac06a120 2023-11-08T19:25:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.141+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c00bbb0 con 0x7f92ac06a120 2023-11-08T19:25:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.141+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92ac14bc40 con 0x7f92ac06a120 2023-11-08T19:25:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.141+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92ac14c980 con 0x7f92ac06a120 2023-11-08T19:25:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.142+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92ac14baa0 con 0x7f92ac06a120 2023-11-08T19:25:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.143+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f929c00bd10 con 0x7f92ac06a120 2023-11-08T19:25:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.144+0000 7f92a8ff9640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f928c06ae70 0x7f928c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.144+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f929c089f40 con 0x7f92ac06a120 2023-11-08T19:25:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.144+0000 7f92a2ffd640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f928c06ae70 0x7f928c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.145+0000 7f92a2ffd640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f928c06ae70 0x7f928c06d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0b0f40 tx=0x7f9294009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.148+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92ac14baa0 con 0x7f92ac06a120 2023-11-08T19:25:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.336+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92ac06a970 con 0x7f92ac06a120 2023-11-08T19:25:16.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.337+0000 7f92a8ff9640 1 -- 172.21.15.119:0/2275821445 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92ac06a970 con 0x7f92ac06a120 2023-11-08T19:25:16.339 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f928c06ae70 msgr2=0x7f928c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f928c06ae70 0x7f928c06d330 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0b0f40 tx=0x7f9294009040 comp rx=0 tx=0).stop 2023-11-08T19:25:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 msgr2=0x7f92ac14b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 0x7f92ac14b250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f929c00d800 tx=0x7f929c00dcc0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 shutdown_connections 2023-11-08T19:25:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f928c06ae70 0x7f928c06d330 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92ac15d520 0x7f92ac0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92ac06a500 0x7f92ac14c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 --2- 172.21.15.119:0/2275821445 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f92ac06a120 0x7f92ac14b250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.340+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 >> 172.21.15.119:0/2275821445 conn(0x7f92ac0cfd20 msgr2=0x7f92ac0715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.341+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 shutdown_connections 2023-11-08T19:25:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.341+0000 7f92b207c640 1 -- 172.21.15.119:0/2275821445 wait complete. 2023-11-08T19:25:16.360 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:16.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.571+0000 7faeacc8a640 1 Processor -- start 2023-11-08T19:25:16.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.572+0000 7faeacc8a640 1 -- start start 2023-11-08T19:25:16.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7faeacc8a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea80cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7faeacc8a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faea815e290 0x7faea815e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7faeacc8a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faea815a8c0 0x7faea815ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7faeacc8a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea8071720 con 0x7faea815e290 2023-11-08T19:25:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7faeacc8a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea80cf5e0 con 0x7faea815a8c0 2023-11-08T19:25:16.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7faeacc8a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80cf8c0 con 0x7faea815deb0 2023-11-08T19:25:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7fae9ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faea815e290 0x7faea815e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7fae9ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faea815e290 0x7faea815e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37482/0 (socket says 172.21.15.119:37482) 2023-11-08T19:25:16.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.573+0000 7fae9ffff640 1 -- 172.21.15.119:0/1134673080 learned_addr learned my addr 172.21.15.119:0/1134673080 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7faea7fff640 1 --2- 172.21.15.119:0/1134673080 >> v2:172.21.15.188:3300/0 conn(0x7faea815a8c0 0x7faea815ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7faea77fe640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea80cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7fae9ffff640 1 -- 172.21.15.119:0/1134673080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 msgr2=0x7faea80cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7fae9ffff640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea80cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7fae9ffff640 1 -- 172.21.15.119:0/1134673080 >> v2:172.21.15.188:3300/0 conn(0x7faea815a8c0 msgr2=0x7faea815ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7fae9ffff640 1 --2- 172.21.15.119:0/1134673080 >> v2:172.21.15.188:3300/0 conn(0x7faea815a8c0 0x7faea815ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7fae9ffff640 1 -- 172.21.15.119:0/1134673080 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea80710d0 con 0x7faea815e290 2023-11-08T19:25:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7faea7fff640 1 --2- 172.21.15.119:0/1134673080 >> v2:172.21.15.188:3300/0 conn(0x7faea815a8c0 0x7faea815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.574+0000 7faea77fe640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:16.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.575+0000 7fae9ffff640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815e290 0x7faea815e6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faea815ef50 tx=0x7fae9002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80968c23b53cfc34 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.575+0000 7faea6ffd640 1 -- 172.21.15.119:0/1134673080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae900317d0 con 0x7faea815e290 2023-11-08T19:25:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.575+0000 7faea6ffd640 1 -- 172.21.15.119:0/1134673080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae90031930 con 0x7faea815e290 2023-11-08T19:25:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.575+0000 7faea6ffd640 1 -- 172.21.15.119:0/1134673080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90031cc0 con 0x7faea815e290 2023-11-08T19:25:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 -- 172.21.15.119:0/1134673080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815e290 msgr2=0x7faea815e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815e290 0x7faea815e6d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7faea815ef50 tx=0x7fae9002b870 comp rx=0 tx=0).stop 2023-11-08T19:25:16.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 -- 172.21.15.119:0/1134673080 shutdown_connections 2023-11-08T19:25:16.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1134673080 >> v2:172.21.15.188:3300/0 conn(0x7faea815a8c0 0x7faea815ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815e290 0x7faea815e6d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1134673080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea80cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 -- 172.21.15.119:0/1134673080 >> 172.21.15.119:0/1134673080 conn(0x7faea80ceac0 msgr2=0x7faea80ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 -- 172.21.15.119:0/1134673080 shutdown_connections 2023-11-08T19:25:16.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.576+0000 7faeacc8a640 1 -- 172.21.15.119:0/1134673080 wait complete. 2023-11-08T19:25:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.578+0000 7faeacc8a640 1 Processor -- start 2023-11-08T19:25:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.578+0000 7faeacc8a640 1 -- start start 2023-11-08T19:25:16.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faeacc8a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 0x7faea814b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faeacc8a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea8156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faeacc8a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faea815e290 0x7faea8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faeacc8a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea80b2bf0 con 0x7faea815a8c0 2023-11-08T19:25:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faeacc8a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea814bb80 con 0x7faea815e290 2023-11-08T19:25:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faeacc8a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea80b22c0 con 0x7faea815deb0 2023-11-08T19:25:16.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faea77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 0x7faea814b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faea77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 0x7faea814b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37496/0 (socket says 172.21.15.119:37496) 2023-11-08T19:25:16.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.579+0000 7faea77fe640 1 -- 172.21.15.119:0/1333493296 learned_addr learned my addr 172.21.15.119:0/1333493296 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:16.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea77fe640 1 -- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 msgr2=0x7faea8156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea77fe640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea8156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea7fff640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faea815e290 0x7faea8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea77fe640 1 -- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faea815e290 msgr2=0x7faea8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea77fe640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faea815e290 0x7faea8157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea77fe640 1 -- 172.21.15.119:0/1333493296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900094d0 con 0x7faea815a8c0 2023-11-08T19:25:16.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea77fe640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 0x7faea814b640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c7f0 tx=0x7fae9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98007820 con 0x7faea815a8c0 2023-11-08T19:25:16.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae98007980 con 0x7faea815a8c0 2023-11-08T19:25:16.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98007cf0 con 0x7faea815a8c0 2023-11-08T19:25:16.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea80b25a0 con 0x7faea815a8c0 2023-11-08T19:25:16.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.580+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea80b1f90 con 0x7faea815a8c0 2023-11-08T19:25:16.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.582+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea804ece0 con 0x7faea815a8c0 2023-11-08T19:25:16.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.583+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fae980113f0 con 0x7faea815a8c0 2023-11-08T19:25:16.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.583+0000 7faea4ff9640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fae8806aea0 0x7fae8806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:16.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.584+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fae98089b90 con 0x7faea815a8c0 2023-11-08T19:25:16.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.584+0000 7fae9ffff640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fae8806aea0 0x7fae8806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:16.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.584+0000 7fae9ffff640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fae8806aea0 0x7fae8806d360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae90036000 tx=0x7fae90030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:16.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.588+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fae980549e0 con 0x7faea815a8c0 2023-11-08T19:25:16.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.780+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7faea80c8c50 con 0x7faea815a8c0 2023-11-08T19:25:16.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.781+0000 7faea4ff9640 1 -- 172.21.15.119:0/1333493296 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fae9801f0a0 con 0x7faea815a8c0 2023-11-08T19:25:16.783 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:16.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.783+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fae8806aea0 msgr2=0x7fae8806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fae8806aea0 0x7fae8806d360 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae90036000 tx=0x7fae90030000 comp rx=0 tx=0).stop 2023-11-08T19:25:16.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 msgr2=0x7faea814b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:16.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 0x7faea814b640 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c7f0 tx=0x7fae9800ccb0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 shutdown_connections 2023-11-08T19:25:16.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fae8806aea0 0x7fae8806d360 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faea815e290 0x7faea8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faea815deb0 0x7faea8156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 --2- 172.21.15.119:0/1333493296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faea815a8c0 0x7faea814b640 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:16.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 >> 172.21.15.119:0/1333493296 conn(0x7faea80ceac0 msgr2=0x7faea806bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:16.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 shutdown_connections 2023-11-08T19:25:16.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:16.784+0000 7faeacc8a640 1 -- 172.21.15.119:0/1333493296 wait complete. 2023-11-08T19:25:16.805 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:25:17.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.015+0000 7f5ff6626640 1 Processor -- start 2023-11-08T19:25:17.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.016+0000 7f5ff6626640 1 -- start start 2023-11-08T19:25:17.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.017+0000 7f5ff6626640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff015d6c0 0x7f5ff00d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.017+0000 7f5ff6626640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff0161870 0x7f5ff00cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.017+0000 7f5ff6626640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ff015df70 con 0x7f5ff00cfa00 2023-11-08T19:25:17.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.017+0000 7f5ff6626640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff015daa0 con 0x7f5ff0161870 2023-11-08T19:25:17.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.017+0000 7f5ff6626640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00d1e60 con 0x7f5ff015d6c0 2023-11-08T19:25:17.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.017+0000 7f5ff4bc3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff0161870 0x7f5ff00cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5ff4bc3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ff0161870 0x7f5ff00cfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37506/0 (socket says 172.21.15.119:37506) 2023-11-08T19:25:17.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5ff4bc3640 1 -- 172.21.15.119:0/3755924366 learned_addr learned my addr 172.21.15.119:0/3755924366 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:17.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5feffff640 1 --2- 172.21.15.119:0/3755924366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff015d6c0 0x7f5ff00d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5ff4bc3640 1 -- 172.21.15.119:0/3755924366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff015d6c0 msgr2=0x7f5ff00d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:17.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5ff4bc3640 1 --2- 172.21.15.119:0/3755924366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff015d6c0 0x7f5ff00d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5ff4bc3640 1 -- 172.21.15.119:0/3755924366 >> v1:172.21.15.188:6789/0 conn(0x7f5ff00cfa00 legacy=0x7f5ff0161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:17.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.018+0000 7f5ff4bc3640 1 -- 172.21.15.119:0/3755924366 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00d03a0 con 0x7f5ff0161870 2023-11-08T19:25:17.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.019+0000 7f5ff4bc3640 1 --2- 172.21.15.119:0/3755924366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff0161870 0x7f5ff00cfe60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009870 tx=0x7f5fe802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d51afd7e6cee87 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:17.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.019+0000 7f5feeffd640 1 -- 172.21.15.119:0/3755924366 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe80317c0 con 0x7f5ff0161870 2023-11-08T19:25:17.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.019+0000 7f5feeffd640 1 -- 172.21.15.119:0/3755924366 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe8031920 con 0x7f5ff0161870 2023-11-08T19:25:17.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.019+0000 7f5feeffd640 1 -- 172.21.15.119:0/3755924366 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe8031cb0 con 0x7f5ff0161870 2023-11-08T19:25:17.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 -- 172.21.15.119:0/3755924366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff0161870 msgr2=0x7f5ff00cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 --2- 172.21.15.119:0/3755924366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff0161870 0x7f5ff00cfe60 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009870 tx=0x7f5fe802b860 comp rx=0 tx=0).stop 2023-11-08T19:25:17.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 -- 172.21.15.119:0/3755924366 shutdown_connections 2023-11-08T19:25:17.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 --2- 172.21.15.119:0/3755924366 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff0161870 0x7f5ff00cfe60 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 --2- 172.21.15.119:0/3755924366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff015d6c0 0x7f5ff00d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 -- 172.21.15.119:0/3755924366 >> 172.21.15.119:0/3755924366 conn(0x7f5ff006a9f0 msgr2=0x7f5ff006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:17.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 -- 172.21.15.119:0/3755924366 shutdown_connections 2023-11-08T19:25:17.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.020+0000 7f5ff6626640 1 -- 172.21.15.119:0/3755924366 wait complete. 2023-11-08T19:25:17.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.022+0000 7f5ff6626640 1 Processor -- start 2023-11-08T19:25:17.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.023+0000 7f5ff6626640 1 -- start start 2023-11-08T19:25:17.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff6626640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 0x7f5ff01f2e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff6626640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff015d6c0 0x7f5ff01f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff6626640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff0161870 0x7f5ff00c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff6626640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff0144400 con 0x7f5ff00cfa00 2023-11-08T19:25:17.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff6626640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff00c6e30 con 0x7f5ff015d6c0 2023-11-08T19:25:17.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff6626640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff01f3430 con 0x7f5ff0161870 2023-11-08T19:25:17.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5feffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 0x7f5ff01f2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5feffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 0x7f5ff01f2e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37516/0 (socket says 172.21.15.119:37516) 2023-11-08T19:25:17.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5feffff640 1 -- 172.21.15.119:0/1383713987 learned_addr learned my addr 172.21.15.119:0/1383713987 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:17.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5fef7fe640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff015d6c0 0x7f5ff01f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.024+0000 7f5ff4bc3640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff0161870 0x7f5ff00c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5feffff640 1 -- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff0161870 msgr2=0x7f5ff00c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5feffff640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff0161870 0x7f5ff00c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5feffff640 1 -- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff015d6c0 msgr2=0x7f5ff01f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5feffff640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff015d6c0 0x7f5ff01f6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5feffff640 1 -- 172.21.15.119:0/1383713987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80094d0 con 0x7f5ff00cfa00 2023-11-08T19:25:17.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5ff4bc3640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff0161870 0x7f5ff00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:17.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5feffff640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 0x7f5ff01f2e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0007ef0 tx=0x7f5fe000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:17.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0014620 con 0x7f5ff00cfa00 2023-11-08T19:25:17.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe0014780 con 0x7f5ff00cfa00 2023-11-08T19:25:17.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0014b30 con 0x7f5ff00cfa00 2023-11-08T19:25:17.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff01f3710 con 0x7f5ff00cfa00 2023-11-08T19:25:17.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.025+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff01f3aa0 con 0x7f5ff00cfa00 2023-11-08T19:25:17.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.026+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff0144670 con 0x7f5ff00cfa00 2023-11-08T19:25:17.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.027+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5fe0014c90 con 0x7f5ff00cfa00 2023-11-08T19:25:17.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.028+0000 7f5fecff9640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fd006ae70 0x7f5fd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.028+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5fe008a270 con 0x7f5ff00cfa00 2023-11-08T19:25:17.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.029+0000 7f5fef7fe640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fd006ae70 0x7f5fd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.029+0000 7f5fef7fe640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fd006ae70 0x7f5fd006d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0160370 tx=0x7f5fd8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:17.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.033+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5fe00550c0 con 0x7f5ff00cfa00 2023-11-08T19:25:17.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.235+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5ff01f7290 con 0x7f5ff00cfa00 2023-11-08T19:25:17.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.236+0000 7f5fecff9640 1 -- 172.21.15.119:0/1383713987 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5fe00586e0 con 0x7f5ff00cfa00 2023-11-08T19:25:17.238 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:25:17.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fd006ae70 msgr2=0x7f5fd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fd006ae70 0x7f5fd006d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0160370 tx=0x7f5fd8009040 comp rx=0 tx=0).stop 2023-11-08T19:25:17.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 msgr2=0x7f5ff01f2e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 0x7f5ff01f2e00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5fe0007ef0 tx=0x7f5fe000e4b0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 shutdown_connections 2023-11-08T19:25:17.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5fd006ae70 0x7f5fd006d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ff0161870 0x7f5ff00c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ff015d6c0 0x7f5ff01f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 --2- 172.21.15.119:0/1383713987 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ff00cfa00 0x7f5ff01f2e00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 >> 172.21.15.119:0/1383713987 conn(0x7f5ff006a9f0 msgr2=0x7f5ff00b77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:17.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 shutdown_connections 2023-11-08T19:25:17.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.239+0000 7f5ff6626640 1 -- 172.21.15.119:0/1383713987 wait complete. 2023-11-08T19:25:17.259 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi119.front.sepia.ceph.com... 2023-11-08T19:25:17.260 INFO:teuthology.orchestra.run:Running command with timeout 600 2023-11-08T19:25:17.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo stdin-killer --timeout=300 -- fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-08T19:25:17.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2023-11-08T19:25:17.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:ceph-fuse[93976]: fuse finished with error 0 and tester_r 0 2023-11-08T19:25:17.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T19:25:17.414 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi119.stderr:2023-11-08T19:25:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T19:25:17.417 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:25:17.430 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:25:17.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:17.431 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-08T19:25:17.541 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:17.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.751+0000 7facece6f640 1 Processor -- start 2023-11-08T19:25:17.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.751+0000 7facece6f640 1 -- start start 2023-11-08T19:25:17.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.752+0000 7facece6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face80b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.752+0000 7facece6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7face806a830 0x7face806e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.752+0000 7facece6f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80d0a00 con 0x7face80ca370 2023-11-08T19:25:17.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.752+0000 7facece6f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face815b500 con 0x7face806a830 2023-11-08T19:25:17.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.752+0000 7facece6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face806da50 con 0x7face815b120 2023-11-08T19:25:17.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7face806a830 0x7face806e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face5d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7face80ca370 0x7face806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37074/0 (socket says 172.21.15.119:37074) 2023-11-08T19:25:17.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face5d9b640 1 -- 172.21.15.119:0/3091576912 learned_addr learned my addr 172.21.15.119:0/3091576912 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:17.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7face806a830 0x7face806e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37528/0 (socket says 172.21.15.119:37528) 2023-11-08T19:25:17.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face6d9d640 1 -- 172.21.15.119:0/3091576912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 msgr2=0x7face80b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face659c640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face80b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face6d9d640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face80b2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face6d9d640 1 -- 172.21.15.119:0/3091576912 >> v1:172.21.15.119:6789/0 conn(0x7face80ca370 legacy=0x7face806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face6d9d640 1 -- 172.21.15.119:0/3091576912 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face806dd30 con 0x7face806a830 2023-11-08T19:25:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.753+0000 7face659c640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face80b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.755+0000 7face6d9d640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face806e810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7face0009d90 tx=0x7face002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69f06918a581941 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.755+0000 7face559a640 1 -- 172.21.15.119:0/3091576912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face002cc50 con 0x7face806a830 2023-11-08T19:25:17.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.755+0000 7face559a640 1 -- 172.21.15.119:0/3091576912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face002cdb0 con 0x7face806a830 2023-11-08T19:25:17.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.755+0000 7face559a640 1 -- 172.21.15.119:0/3091576912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0034660 con 0x7face806a830 2023-11-08T19:25:17.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 -- 172.21.15.119:0/3091576912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 msgr2=0x7face806e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face806e810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7face0009d90 tx=0x7face002bea0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 -- 172.21.15.119:0/3091576912 shutdown_connections 2023-11-08T19:25:17.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face806e810 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 --2- 172.21.15.119:0/3091576912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face80b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 -- 172.21.15.119:0/3091576912 >> 172.21.15.119:0/3091576912 conn(0x7face80cfd20 msgr2=0x7face80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:17.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 -- 172.21.15.119:0/3091576912 shutdown_connections 2023-11-08T19:25:17.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.756+0000 7facece6f640 1 -- 172.21.15.119:0/3091576912 wait complete. 2023-11-08T19:25:17.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.758+0000 7facece6f640 1 Processor -- start 2023-11-08T19:25:17.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.759+0000 7facece6f640 1 -- start start 2023-11-08T19:25:17.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7facece6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face8144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7facece6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face80ca370 0x7face815d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7facece6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face8156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7facece6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face8144640 con 0x7face806a830 2023-11-08T19:25:17.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7facece6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c67b0 con 0x7face80ca370 2023-11-08T19:25:17.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7facece6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face81553b0 con 0x7face815b120 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7face659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face8144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7face659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face8144100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37534/0 (socket says 172.21.15.119:37534) 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7face659c640 1 -- 172.21.15.119:0/3564961587 learned_addr learned my addr 172.21.15.119:0/3564961587 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.760+0000 7face5d9b640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face80ca370 0x7face815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face6d9d640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face8156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face659c640 1 -- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 msgr2=0x7face8156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face659c640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face8156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face659c640 1 -- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face80ca370 msgr2=0x7face815d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face659c640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face80ca370 0x7face815d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face659c640 1 -- 172.21.15.119:0/3564961587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0009a70 con 0x7face806a830 2023-11-08T19:25:17.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face5d9b640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face80ca370 0x7face815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:17.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7face659c640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face8144100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7facd800c820 tx=0x7facd800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:17.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8007a30 con 0x7face806a830 2023-11-08T19:25:17.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd8007b90 con 0x7face806a830 2023-11-08T19:25:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800ce10 con 0x7face806a830 2023-11-08T19:25:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face8155690 con 0x7face806a830 2023-11-08T19:25:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.761+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face80c6290 con 0x7face806a830 2023-11-08T19:25:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.763+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face8071dc0 con 0x7face806a830 2023-11-08T19:25:17.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.763+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7facd80113f0 con 0x7face806a830 2023-11-08T19:25:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.764+0000 7facd6ffd640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facc806ae70 0x7facc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:17.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.764+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7facd8089a30 con 0x7face806a830 2023-11-08T19:25:17.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.765+0000 7face5d9b640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facc806ae70 0x7facc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:17.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.765+0000 7face5d9b640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facc806ae70 0x7facc806d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7face8155280 tx=0x7facdc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:17.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.769+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facd80547e0 con 0x7face806a830 2023-11-08T19:25:17.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.952+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7face8074430 con 0x7face806a830 2023-11-08T19:25:17.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.956+0000 7facd6ffd640 1 -- 172.21.15.119:0/3564961587 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7facd801f090 con 0x7face806a830 2023-11-08T19:25:17.958 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:17.959 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:17.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facc806ae70 msgr2=0x7facc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facc806ae70 0x7facc806d330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7face8155280 tx=0x7facdc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:17.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 msgr2=0x7face8144100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:17.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face8144100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7facd800c820 tx=0x7facd800cce0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 shutdown_connections 2023-11-08T19:25:17.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facc806ae70 0x7facc806d330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face815b120 0x7face8156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face80ca370 0x7face815d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 --2- 172.21.15.119:0/3564961587 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face806a830 0x7face8144100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:17.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 >> 172.21.15.119:0/3564961587 conn(0x7face80cfd20 msgr2=0x7face8071600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:17.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 shutdown_connections 2023-11-08T19:25:17.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:17.958+0000 7facece6f640 1 -- 172.21.15.119:0/3564961587 wait complete. 2023-11-08T19:25:17.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:18.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.188+0000 7fee9eabb640 1 Processor -- start 2023-11-08T19:25:18.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.189+0000 7fee9eabb640 1 -- start start 2023-11-08T19:25:18.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.190+0000 7fee9eabb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fee98165750 0x7fee98165b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.190+0000 7fee9eabb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9806f8d0 con 0x7fee980adc20 2023-11-08T19:25:18.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.190+0000 7fee9eabb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9806b9c0 con 0x7fee980ae000 2023-11-08T19:25:18.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.190+0000 7fee9eabb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee980cac30 con 0x7fee98165750 2023-11-08T19:25:18.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9dab9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fee980adc20 0x7fee980c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37088/0 (socket says 172.21.15.119:37088) 2023-11-08T19:25:18.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9dab9640 1 -- 172.21.15.119:0/3537269057 learned_addr learned my addr 172.21.15.119:0/3537269057 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9e2ba640 1 --2- 172.21.15.119:0/3537269057 >> v2:172.21.15.188:3300/0 conn(0x7fee98165750 0x7fee98165b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825743917 0 0) 0x7fee9806f8d0 con 0x7fee980adc20 2023-11-08T19:25:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee84003520 con 0x7fee980adc20 2023-11-08T19:25:18.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1963051387 0 0) 0x7fee84003520 con 0x7fee980adc20 2023-11-08T19:25:18.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 >> v2:172.21.15.188:3300/0 conn(0x7fee98165750 msgr2=0x7fee98165b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 --2- 172.21.15.119:0/3537269057 >> v2:172.21.15.188:3300/0 conn(0x7fee98165750 0x7fee98165b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 >> v1:172.21.15.188:6789/0 conn(0x7fee980ae000 legacy=0x7fee98165040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980caed0 con 0x7fee980adc20 2023-11-08T19:25:18.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fee94001670 con 0x7fee980adc20 2023-11-08T19:25:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 >> v1:172.21.15.119:6789/0 conn(0x7fee980adc20 legacy=0x7fee980c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84003970 0x7fee84003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee840047c0 0x7fee84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.191+0000 7fee9cab7640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee840052a0 0x7fee8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee980cac30 con 0x7fee84003970 2023-11-08T19:25:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400c150 con 0x7fee840052a0 2023-11-08T19:25:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8400c310 con 0x7fee840047c0 2023-11-08T19:25:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84003970 0x7fee84003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9e2ba640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee840052a0 0x7fee8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9d2b8640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee840047c0 0x7fee84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9dab9640 1 -- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee840047c0 msgr2=0x7fee84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee840047c0 0x7fee84004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9dab9640 1 -- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee840052a0 msgr2=0x7fee8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee840052a0 0x7fee8400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9dab9640 1 -- 172.21.15.119:0/3537269057 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84003520 con 0x7fee84003970 2023-11-08T19:25:18.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.192+0000 7fee9e2ba640 1 -- 172.21.15.119:0/3537269057 reap_dead start 2023-11-08T19:25:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.193+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84003970 0x7fee84003d70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee94001eb0 tx=0x7fee840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b062f585a2f5b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94005ad0 con 0x7fee84003970 2023-11-08T19:25:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94005c30 con 0x7fee84003970 2023-11-08T19:25:18.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9cab7640 1 -- 172.21.15.119:0/3537269057 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9402c8b0 con 0x7fee84003970 2023-11-08T19:25:18.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 -- 172.21.15.119:0/3537269057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84003970 msgr2=0x7fee84003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84003970 0x7fee84003d70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee94001eb0 tx=0x7fee840057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 -- 172.21.15.119:0/3537269057 shutdown_connections 2023-11-08T19:25:18.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3537269057 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee84003970 0x7fee84003d70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 -- 172.21.15.119:0/3537269057 >> 172.21.15.119:0/3537269057 conn(0x7fee980b0560 msgr2=0x7fee980b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 -- 172.21.15.119:0/3537269057 shutdown_connections 2023-11-08T19:25:18.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.194+0000 7fee9eabb640 1 -- 172.21.15.119:0/3537269057 wait complete. 2023-11-08T19:25:18.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.196+0000 7fee9eabb640 1 Processor -- start 2023-11-08T19:25:18.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.196+0000 7fee9eabb640 1 -- start start 2023-11-08T19:25:18.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9eabb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee84003970 0x7fee981f7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9eabb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 0x7fee98070950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9eabb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee98070e90 0x7fee98162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9eabb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee98071340 con 0x7fee980c6f90 2023-11-08T19:25:18.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9eabb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee981f7a50 con 0x7fee98070e90 2023-11-08T19:25:18.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9eabb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee980c61d0 con 0x7fee84003970 2023-11-08T19:25:18.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9d2b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 0x7fee98070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9d2b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 0x7fee98070950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37548/0 (socket says 172.21.15.119:37548) 2023-11-08T19:25:18.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.197+0000 7fee9d2b8640 1 -- 172.21.15.119:0/3183827197 learned_addr learned my addr 172.21.15.119:0/3183827197 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:18.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee84003970 0x7fee981f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9e2ba640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee98070e90 0x7fee98162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9d2b8640 1 -- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee84003970 msgr2=0x7fee981f7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9d2b8640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee84003970 0x7fee981f7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9d2b8640 1 -- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee98070e90 msgr2=0x7fee98162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9d2b8640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee98070e90 0x7fee98162100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9d2b8640 1 -- 172.21.15.119:0/3183827197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee980c64b0 con 0x7fee980c6f90 2023-11-08T19:25:18.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee84003970 0x7fee981f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:18.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9d2b8640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 0x7fee98070950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fee8800e410 tx=0x7fee8800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88003a70 con 0x7fee980c6f90 2023-11-08T19:25:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee88003bd0 con 0x7fee980c6f90 2023-11-08T19:25:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee980c6790 con 0x7fee980c6f90 2023-11-08T19:25:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee88004050 con 0x7fee980c6f90 2023-11-08T19:25:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.198+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee980c73f0 con 0x7fee980c6f90 2023-11-08T19:25:18.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.200+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee980c6cb0 con 0x7fee980c6f90 2023-11-08T19:25:18.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.201+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee880041b0 con 0x7fee980c6f90 2023-11-08T19:25:18.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.202+0000 7fee8e7fc640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee7c06aea0 0x7fee7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.202+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fee880895c0 con 0x7fee980c6f90 2023-11-08T19:25:18.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.202+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee7c06aea0 0x7fee7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.203+0000 7fee9dab9640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee7c06aea0 0x7fee7c06d360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee94034000 tx=0x7fee94009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.206+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee980c6cb0 con 0x7fee980c6f90 2023-11-08T19:25:18.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.397+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee980c6cb0 con 0x7fee980c6f90 2023-11-08T19:25:18.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.400+0000 7fee8e7fc640 1 -- 172.21.15.119:0/3183827197 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fee980c6cb0 con 0x7fee980c6f90 2023-11-08T19:25:18.402 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:18.402 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:18.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee7c06aea0 msgr2=0x7fee7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee7c06aea0 0x7fee7c06d360 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fee94034000 tx=0x7fee94009030 comp rx=0 tx=0).stop 2023-11-08T19:25:18.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 msgr2=0x7fee98070950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 0x7fee98070950 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fee8800e410 tx=0x7fee8800e8d0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 shutdown_connections 2023-11-08T19:25:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee7c06aea0 0x7fee7c06d360 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee98070e90 0x7fee98162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fee980c6f90 0x7fee98070950 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 --2- 172.21.15.119:0/3183827197 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee84003970 0x7fee981f7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 >> 172.21.15.119:0/3183827197 conn(0x7fee980b0560 msgr2=0x7fee980b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 shutdown_connections 2023-11-08T19:25:18.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.403+0000 7fee9eabb640 1 -- 172.21.15.119:0/3183827197 wait complete. 2023-11-08T19:25:18.425 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:25:18.605 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:18.614 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:18.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.634+0000 7feb4f941640 1 Processor -- start 2023-11-08T19:25:18.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.634+0000 7feb4f941640 1 -- start start 2023-11-08T19:25:18.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.635+0000 7feb4f941640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feb4815d4a0 0x7feb480d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.635+0000 7feb4f941640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb480cfa00 0x7feb480cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4f941640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4815df50 con 0x7feb48161160 2023-11-08T19:25:18.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4f941640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb480d03d0 con 0x7feb480cfa00 2023-11-08T19:25:18.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4f941640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb480d1e60 con 0x7feb4815d4a0 2023-11-08T19:25:18.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb480cfa00 0x7feb480cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feb480cfa00 0x7feb480cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50866/0 (socket says 172.21.15.119:50866) 2023-11-08T19:25:18.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 -- 172.21.15.119:0/4060334147 learned_addr learned my addr 172.21.15.119:0/4060334147 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:18.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4d6dd640 1 --2- 172.21.15.119:0/4060334147 >> v2:172.21.15.188:3300/0 conn(0x7feb4815d4a0 0x7feb480d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 -- 172.21.15.119:0/4060334147 >> v2:172.21.15.188:3300/0 conn(0x7feb4815d4a0 msgr2=0x7feb480d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:18.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 --2- 172.21.15.119:0/4060334147 >> v2:172.21.15.188:3300/0 conn(0x7feb4815d4a0 0x7feb480d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 -- 172.21.15.119:0/4060334147 >> v1:172.21.15.188:6789/0 conn(0x7feb48161160 legacy=0x7feb48161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.636+0000 7feb4cedc640 1 -- 172.21.15.119:0/4060334147 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480d2830 con 0x7feb480cfa00 2023-11-08T19:25:18.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.637+0000 7feb4cedc640 1 --2- 172.21.15.119:0/4060334147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb480cfa00 0x7feb480cfe90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb48072830 tx=0x7feb3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f893c4dff87959dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb3ffff640 1 -- 172.21.15.119:0/4060334147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38031870 con 0x7feb480cfa00 2023-11-08T19:25:18.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb3ffff640 1 -- 172.21.15.119:0/4060334147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb380319d0 con 0x7feb480cfa00 2023-11-08T19:25:18.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb3ffff640 1 -- 172.21.15.119:0/4060334147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38031d60 con 0x7feb480cfa00 2023-11-08T19:25:18.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb4f941640 1 -- 172.21.15.119:0/4060334147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb480cfa00 msgr2=0x7feb480cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb4f941640 1 --2- 172.21.15.119:0/4060334147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb480cfa00 0x7feb480cfe90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb48072830 tx=0x7feb3802b910 comp rx=0 tx=0).stop 2023-11-08T19:25:18.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb4f941640 1 -- 172.21.15.119:0/4060334147 shutdown_connections 2023-11-08T19:25:18.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb4f941640 1 --2- 172.21.15.119:0/4060334147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb480cfa00 0x7feb480cfe90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.638+0000 7feb4f941640 1 --2- 172.21.15.119:0/4060334147 >> v2:172.21.15.188:3300/0 conn(0x7feb4815d4a0 0x7feb480d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.639+0000 7feb4f941640 1 -- 172.21.15.119:0/4060334147 >> 172.21.15.119:0/4060334147 conn(0x7feb4806a9f0 msgr2=0x7feb4806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:18.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.639+0000 7feb4f941640 1 -- 172.21.15.119:0/4060334147 shutdown_connections 2023-11-08T19:25:18.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.639+0000 7feb4f941640 1 -- 172.21.15.119:0/4060334147 wait complete. 2023-11-08T19:25:18.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.640+0000 7feb4f941640 1 Processor -- start 2023-11-08T19:25:18.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.641+0000 7feb4f941640 1 -- start start 2023-11-08T19:25:18.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.641+0000 7feb4f941640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb480cfa00 0x7feb4814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.641+0000 7feb4f941640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 0x7feb481f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4f941640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb48161160 0x7feb480b7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4f941640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb48155010 con 0x7feb4815d4a0 2023-11-08T19:25:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4f941640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb48155870 con 0x7feb480cfa00 2023-11-08T19:25:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4f941640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb481f3fc0 con 0x7feb48161160 2023-11-08T19:25:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 0x7feb481f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 0x7feb481f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50874/0 (socket says 172.21.15.119:50874) 2023-11-08T19:25:18.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 -- 172.21.15.119:0/2306998208 learned_addr learned my addr 172.21.15.119:0/2306998208 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:18.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4d6dd640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb480cfa00 0x7feb4814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4dede640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb48161160 0x7feb480b7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 -- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb48161160 msgr2=0x7feb480b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb48161160 0x7feb480b7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 -- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb480cfa00 msgr2=0x7feb4814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb480cfa00 0x7feb4814a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4cedc640 1 -- 172.21.15.119:0/2306998208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380094d0 con 0x7feb4815d4a0 2023-11-08T19:25:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.642+0000 7feb4dede640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb48161160 0x7feb480b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:18.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.643+0000 7feb4cedc640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 0x7feb481f4370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb3802b950 tx=0x7feb380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.643+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3803abe0 con 0x7feb4815d4a0 2023-11-08T19:25:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.643+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3803ad60 con 0x7feb4815d4a0 2023-11-08T19:25:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.643+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb48155220 con 0x7feb4815d4a0 2023-11-08T19:25:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.643+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb38044630 con 0x7feb4815d4a0 2023-11-08T19:25:18.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.643+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4814a8b0 con 0x7feb4815d4a0 2023-11-08T19:25:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.644+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb48074430 con 0x7feb4815d4a0 2023-11-08T19:25:18.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.646+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb3804d450 con 0x7feb4815d4a0 2023-11-08T19:25:18.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.647+0000 7feb3dffb640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feb2c06ae70 0x7feb2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:18.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.647+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feb3802d040 con 0x7feb4815d4a0 2023-11-08T19:25:18.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.647+0000 7feb4d6dd640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feb2c06ae70 0x7feb2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:18.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.648+0000 7feb4d6dd640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feb2c06ae70 0x7feb2c06d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb480b8500 tx=0x7feb44009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:18.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.651+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feb38076bc0 con 0x7feb4815d4a0 2023-11-08T19:25:18.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:18.877+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feb480d10f0 con 0x7feb4815d4a0 2023-11-08T19:25:19.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.757+0000 7feb3dffb640 1 -- 172.21.15.119:0/2306998208 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7feb38035070 con 0x7feb4815d4a0 2023-11-08T19:25:19.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feb2c06ae70 msgr2=0x7feb2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feb2c06ae70 0x7feb2c06d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb480b8500 tx=0x7feb44009190 comp rx=0 tx=0).stop 2023-11-08T19:25:19.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 msgr2=0x7feb481f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 0x7feb481f4370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb3802b950 tx=0x7feb380046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 shutdown_connections 2023-11-08T19:25:19.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feb2c06ae70 0x7feb2c06d330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feb48161160 0x7feb480b7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feb4815d4a0 0x7feb481f4370 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 --2- 172.21.15.119:0/2306998208 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feb480cfa00 0x7feb4814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 >> 172.21.15.119:0/2306998208 conn(0x7feb4806a9f0 msgr2=0x7feb48161570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.761+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 shutdown_connections 2023-11-08T19:25:19.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.762+0000 7feb4f941640 1 -- 172.21.15.119:0/2306998208 wait complete. 2023-11-08T19:25:19.782 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-08T19:25:19.783 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:25:19.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.980+0000 7f3c73ff5640 1 Processor -- start 2023-11-08T19:25:19.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.980+0000 7f3c73ff5640 1 -- start start 2023-11-08T19:25:19.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c73ff5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c6c06a830 0x7f3c6c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c73ff5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c06d8f0 con 0x7f3c6c0af0d0 2023-11-08T19:25:19.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c73ff5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c0cd900 con 0x7f3c6c06d080 2023-11-08T19:25:19.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c73ff5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6c0cdbe0 con 0x7f3c6c06a830 2023-11-08T19:25:19.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c71590640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c6c06a830 0x7f3c6c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c71590640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3c6c06a830 0x7f3c6c0ca2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50888/0 (socket says 172.21.15.119:50888) 2023-11-08T19:25:19.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.982+0000 7f3c71590640 1 -- 172.21.15.119:0/2954066844 learned_addr learned my addr 172.21.15.119:0/2954066844 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:19.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.983+0000 7f3c71590640 1 -- 172.21.15.119:0/2954066844 >> v1:172.21.15.188:6789/0 conn(0x7f3c6c06d080 legacy=0x7f3c6c161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:19.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.983+0000 7f3c71590640 1 -- 172.21.15.119:0/2954066844 >> v1:172.21.15.119:6789/0 conn(0x7f3c6c0af0d0 legacy=0x7f3c6c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.983+0000 7f3c71590640 1 -- 172.21.15.119:0/2954066844 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c0cf5a0 con 0x7f3c6c06a830 2023-11-08T19:25:19.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.983+0000 7f3c70d8f640 1 -- 172.21.15.119:0/2954066844 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811803500 0 0) 0x7f3c6c06d8f0 con 0x7f3c6c0af0d0 2023-11-08T19:25:19.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.984+0000 7f3c71590640 1 --2- 172.21.15.119:0/2954066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06a830 0x7f3c6c0ca2e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009870 tx=0x7f3c5c02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1100acb242c669d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.984+0000 7f3c70d8f640 1 -- 172.21.15.119:0/2954066844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c030850 con 0x7f3c6c06a830 2023-11-08T19:25:19.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.984+0000 7f3c70d8f640 1 -- 172.21.15.119:0/2954066844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c0309b0 con 0x7f3c6c06a830 2023-11-08T19:25:19.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.984+0000 7f3c70d8f640 1 -- 172.21.15.119:0/2954066844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c030d40 con 0x7f3c6c06a830 2023-11-08T19:25:19.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 -- 172.21.15.119:0/2954066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06a830 msgr2=0x7f3c6c0ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/2954066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06a830 0x7f3c6c0ca2e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009870 tx=0x7f3c5c02b990 comp rx=0 tx=0).stop 2023-11-08T19:25:19.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 -- 172.21.15.119:0/2954066844 shutdown_connections 2023-11-08T19:25:19.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/2954066844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06a830 0x7f3c6c0ca2e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 -- 172.21.15.119:0/2954066844 >> 172.21.15.119:0/2954066844 conn(0x7f3c6c06b230 msgr2=0x7f3c6c06b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:19.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 -- 172.21.15.119:0/2954066844 shutdown_connections 2023-11-08T19:25:19.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.985+0000 7f3c73ff5640 1 -- 172.21.15.119:0/2954066844 wait complete. 2023-11-08T19:25:19.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.987+0000 7f3c73ff5640 1 Processor -- start 2023-11-08T19:25:19.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.987+0000 7f3c73ff5640 1 -- start start 2023-11-08T19:25:19.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c73ff5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c6c06a830 0x7f3c6c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c73ff5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 0x7f3c6c1e6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c73ff5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c6c0af0d0 0x7f3c6c1e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c73ff5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1546f0 con 0x7f3c6c06d080 2023-11-08T19:25:19.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c73ff5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c1548e0 con 0x7f3c6c06a830 2023-11-08T19:25:19.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c73ff5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c154bc0 con 0x7f3c6c0af0d0 2023-11-08T19:25:19.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 0x7f3c6c1e6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 0x7f3c6c1e6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50902/0 (socket says 172.21.15.119:50902) 2023-11-08T19:25:19.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 -- 172.21.15.119:0/813941610 learned_addr learned my addr 172.21.15.119:0/813941610 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:19.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71d91640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c6c06a830 0x7f3c6c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c72592640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c6c0af0d0 0x7f3c6c1e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 -- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c6c0af0d0 msgr2=0x7f3c6c1e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c6c0af0d0 0x7f3c6c1e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 -- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c6c06a830 msgr2=0x7f3c6c1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:19.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c6c06a830 0x7f3c6c1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:19.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c71590640 1 -- 172.21.15.119:0/813941610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c5c0094d0 con 0x7f3c6c06d080 2023-11-08T19:25:19.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.988+0000 7f3c72592640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c6c0af0d0 0x7f3c6c1e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:19.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.989+0000 7f3c71590640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 0x7f3c6c1e6d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009870 tx=0x7f3c5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.989+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c039b60 con 0x7f3c6c06d080 2023-11-08T19:25:19.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.989+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5c039cc0 con 0x7f3c6c06d080 2023-11-08T19:25:19.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.989+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c043630 con 0x7f3c6c06d080 2023-11-08T19:25:19.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.989+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c154ea0 con 0x7f3c6c06d080 2023-11-08T19:25:19.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.989+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c155460 con 0x7f3c6c06d080 2023-11-08T19:25:19.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.991+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c0cd2b0 con 0x7f3c6c06d080 2023-11-08T19:25:19.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.991+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c5c02e070 con 0x7f3c6c06d080 2023-11-08T19:25:19.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.992+0000 7f3c627fc640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c5006ae70 0x7f3c5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:19.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.992+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f3c5c0ab9b0 con 0x7f3c6c06d080 2023-11-08T19:25:19.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.993+0000 7f3c71d91640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c5006ae70 0x7f3c5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:19.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.993+0000 7f3c71d91640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c5006ae70 0x7f3c5006d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c68000f60 tx=0x7f3c68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:19.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:19.997+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c5c076770 con 0x7f3c6c06d080 2023-11-08T19:25:20.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.216+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3c6c071b20 con 0x7f3c6c06d080 2023-11-08T19:25:20.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.774+0000 7f3c627fc640 1 -- 172.21.15.119:0/813941610 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f3c5c0480c0 con 0x7f3c6c06d080 2023-11-08T19:25:20.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.777+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c5006ae70 msgr2=0x7f3c5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.777+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c5006ae70 0x7f3c5006d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c68000f60 tx=0x7f3c68009040 comp rx=0 tx=0).stop 2023-11-08T19:25:20.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.777+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 msgr2=0x7f3c6c1e6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:20.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.777+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 0x7f3c6c1e6d80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c009870 tx=0x7f3c5c004730 comp rx=0 tx=0).stop 2023-11-08T19:25:20.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 shutdown_connections 2023-11-08T19:25:20.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c5006ae70 0x7f3c5006d330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c6c0af0d0 0x7f3c6c1e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c6c06d080 0x7f3c6c1e6d80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 --2- 172.21.15.119:0/813941610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c6c06a830 0x7f3c6c1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:20.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 >> 172.21.15.119:0/813941610 conn(0x7f3c6c06b230 msgr2=0x7f3c6c160020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:20.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 shutdown_connections 2023-11-08T19:25:20.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:20.778+0000 7f3c73ff5640 1 -- 172.21.15.119:0/813941610 wait complete. 2023-11-08T19:25:20.800 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:25:20.801 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.015+0000 7fef3ace0640 1 Processor -- start 2023-11-08T19:25:21.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.015+0000 7fef3ace0640 1 -- start start 2023-11-08T19:25:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3ace0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3415d9f0 0x7fef340cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3ace0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef3406cb40 0x7fef3406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3ace0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef340c4960 con 0x7fef34142f80 2023-11-08T19:25:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3ace0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef34143a40 con 0x7fef3406cb40 2023-11-08T19:25:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3ace0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef340ca3e0 con 0x7fef3415d9f0 2023-11-08T19:25:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3927d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef3406cb40 0x7fef3406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3927d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef3406cb40 0x7fef3406cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50904/0 (socket says 172.21.15.119:50904) 2023-11-08T19:25:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.017+0000 7fef3927d640 1 -- 172.21.15.119:0/2685559652 learned_addr learned my addr 172.21.15.119:0/2685559652 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.018+0000 7fef33fff640 1 --2- 172.21.15.119:0/2685559652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3415d9f0 0x7fef340cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.018+0000 7fef3927d640 1 -- 172.21.15.119:0/2685559652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3415d9f0 msgr2=0x7fef340cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.018+0000 7fef3927d640 1 --2- 172.21.15.119:0/2685559652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3415d9f0 0x7fef340cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.018+0000 7fef3927d640 1 -- 172.21.15.119:0/2685559652 >> v1:172.21.15.188:6789/0 conn(0x7fef34142f80 legacy=0x7fef340c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.018+0000 7fef3927d640 1 -- 172.21.15.119:0/2685559652 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3415df60 con 0x7fef3406cb40 2023-11-08T19:25:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.019+0000 7fef3927d640 1 --2- 172.21.15.119:0/2685559652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef3406cb40 0x7fef3406cfd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00d150 tx=0x7fef2c02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b66b140188f449 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.019+0000 7fef337fe640 1 -- 172.21.15.119:0/2685559652 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c02ea70 con 0x7fef3406cb40 2023-11-08T19:25:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.019+0000 7fef337fe640 1 -- 172.21.15.119:0/2685559652 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2c02ebd0 con 0x7fef3406cb40 2023-11-08T19:25:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.019+0000 7fef337fe640 1 -- 172.21.15.119:0/2685559652 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2c009600 con 0x7fef3406cb40 2023-11-08T19:25:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 -- 172.21.15.119:0/2685559652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef3406cb40 msgr2=0x7fef3406cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2685559652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef3406cb40 0x7fef3406cfd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef2c00d150 tx=0x7fef2c02e0d0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 -- 172.21.15.119:0/2685559652 shutdown_connections 2023-11-08T19:25:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2685559652 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef3406cb40 0x7fef3406cfd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2685559652 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3415d9f0 0x7fef340cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 -- 172.21.15.119:0/2685559652 >> 172.21.15.119:0/2685559652 conn(0x7fef340cb610 msgr2=0x7fef340cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 -- 172.21.15.119:0/2685559652 shutdown_connections 2023-11-08T19:25:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.020+0000 7fef3ace0640 1 -- 172.21.15.119:0/2685559652 wait complete. 2023-11-08T19:25:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.022+0000 7fef3ace0640 1 Processor -- start 2023-11-08T19:25:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.022+0000 7fef3ace0640 1 -- start start 2023-11-08T19:25:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef3ace0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3406cb40 0x7fef340c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef3ace0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 0x7fef3412ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef3ace0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3415d9f0 0x7fef34071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef3ace0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c8af0 con 0x7fef34142f80 2023-11-08T19:25:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef3ace0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef340c8f10 con 0x7fef3415d9f0 2023-11-08T19:25:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef3ace0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3412f290 con 0x7fef3406cb40 2023-11-08T19:25:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef33fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 0x7fef3412ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef33fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 0x7fef3412ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50908/0 (socket says 172.21.15.119:50908) 2023-11-08T19:25:21.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.023+0000 7fef33fff640 1 -- 172.21.15.119:0/2316273149 learned_addr learned my addr 172.21.15.119:0/2316273149 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:21.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef3927d640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3415d9f0 0x7fef34071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef38a7c640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3406cb40 0x7fef340c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef33fff640 1 -- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3406cb40 msgr2=0x7fef340c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef33fff640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3406cb40 0x7fef340c77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef33fff640 1 -- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3415d9f0 msgr2=0x7fef34071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef33fff640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3415d9f0 0x7fef34071550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef33fff640 1 -- 172.21.15.119:0/2316273149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef2c00bd40 con 0x7fef34142f80 2023-11-08T19:25:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef3927d640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3415d9f0 0x7fef34071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef38a7c640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3406cb40 0x7fef340c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef33fff640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 0x7fef3412ed50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef2000e870 tx=0x7fef2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef200094f0 con 0x7fef34142f80 2023-11-08T19:25:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2000c940 con 0x7fef34142f80 2023-11-08T19:25:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2000ccf0 con 0x7fef34142f80 2023-11-08T19:25:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.024+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef340c7e70 con 0x7fef34142f80 2023-11-08T19:25:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.025+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef340c8430 con 0x7fef34142f80 2023-11-08T19:25:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.026+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef34074430 con 0x7fef34142f80 2023-11-08T19:25:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.026+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef20013470 con 0x7fef34142f80 2023-11-08T19:25:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.027+0000 7fef317fa640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef1806ae70 0x7fef1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.027+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fef20089a00 con 0x7fef34142f80 2023-11-08T19:25:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.028+0000 7fef38a7c640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef1806ae70 0x7fef1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.028+0000 7fef38a7c640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef1806ae70 0x7fef1806d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef28000f60 tx=0x7fef28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.032+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef200547c0 con 0x7fef34142f80 2023-11-08T19:25:21.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.258+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef340c4230 con 0x7fef34142f80 2023-11-08T19:25:21.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.259+0000 7fef317fa640 1 -- 172.21.15.119:0/2316273149 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fef20017090 con 0x7fef34142f80 2023-11-08T19:25:21.261 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:21.261 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:21.262 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:25:21.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef1806ae70 msgr2=0x7fef1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef1806ae70 0x7fef1806d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fef28000f60 tx=0x7fef28009040 comp rx=0 tx=0).stop 2023-11-08T19:25:21.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 msgr2=0x7fef3412ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 0x7fef3412ed50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fef2000e870 tx=0x7fef2000ed30 comp rx=0 tx=0).stop 2023-11-08T19:25:21.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 shutdown_connections 2023-11-08T19:25:21.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef1806ae70 0x7fef1806d330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef3415d9f0 0x7fef34071550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef34142f80 0x7fef3412ed50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 --2- 172.21.15.119:0/2316273149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef3406cb40 0x7fef340c77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 >> 172.21.15.119:0/2316273149 conn(0x7fef340cb610 msgr2=0x7fef340b09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 shutdown_connections 2023-11-08T19:25:21.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.262+0000 7fef3ace0640 1 -- 172.21.15.119:0/2316273149 wait complete. 2023-11-08T19:25:21.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:21.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.496+0000 7f928b783640 1 Processor -- start 2023-11-08T19:25:21.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.496+0000 7f928b783640 1 -- start start 2023-11-08T19:25:21.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.497+0000 7f928b783640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928415f4c0 0x7f928406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.497+0000 7f928b783640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f928406a120 0x7f928406a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.497+0000 7f928b783640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f928415f900 con 0x7f92841447a0 2023-11-08T19:25:21.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f928b783640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f928406aab0 con 0x7f928406a120 2023-11-08T19:25:21.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f928b783640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92840c3a50 con 0x7f928415f4c0 2023-11-08T19:25:21.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f928406a120 0x7f928406a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f928406a120 0x7f928406a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50910/0 (socket says 172.21.15.119:50910) 2023-11-08T19:25:21.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 -- 172.21.15.119:0/2155487778 learned_addr learned my addr 172.21.15.119:0/2155487778 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:21.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9288d1e640 1 --2- 172.21.15.119:0/2155487778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928415f4c0 0x7f928406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 -- 172.21.15.119:0/2155487778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928415f4c0 msgr2=0x7f928406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 --2- 172.21.15.119:0/2155487778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928415f4c0 0x7f928406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 -- 172.21.15.119:0/2155487778 >> v1:172.21.15.188:6789/0 conn(0x7f92841447a0 legacy=0x7f92840cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.498+0000 7f9289d20640 1 -- 172.21.15.119:0/2155487778 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9284144b80 con 0x7f928406a120 2023-11-08T19:25:21.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.499+0000 7f9289d20640 1 --2- 172.21.15.119:0/2155487778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928406a570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f927c009e50 tx=0x7f927c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b119ad2b550c13ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f927bfff640 1 -- 172.21.15.119:0/2155487778 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c02ca70 con 0x7f928406a120 2023-11-08T19:25:21.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f927bfff640 1 -- 172.21.15.119:0/2155487778 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927c02cbd0 con 0x7f928406a120 2023-11-08T19:25:21.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f927bfff640 1 -- 172.21.15.119:0/2155487778 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c0343f0 con 0x7f928406a120 2023-11-08T19:25:21.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 -- 172.21.15.119:0/2155487778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 msgr2=0x7f928406a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 --2- 172.21.15.119:0/2155487778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928406a570 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f927c009e50 tx=0x7f927c02bed0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 -- 172.21.15.119:0/2155487778 shutdown_connections 2023-11-08T19:25:21.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 --2- 172.21.15.119:0/2155487778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928406a570 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 --2- 172.21.15.119:0/2155487778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f928415f4c0 0x7f928406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 -- 172.21.15.119:0/2155487778 >> 172.21.15.119:0/2155487778 conn(0x7f92840cb610 msgr2=0x7f92840cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 -- 172.21.15.119:0/2155487778 shutdown_connections 2023-11-08T19:25:21.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.500+0000 7f928b783640 1 -- 172.21.15.119:0/2155487778 wait complete. 2023-11-08T19:25:21.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.502+0000 7f928b783640 1 Processor -- start 2023-11-08T19:25:21.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.502+0000 7f928b783640 1 -- start start 2023-11-08T19:25:21.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928b783640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928b783640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92841447a0 0x7f928413e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928b783640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928415f4c0 0x7f928413eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928b783640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92841f3130 con 0x7f928406a120 2023-11-08T19:25:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928b783640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928415aeb0 con 0x7f928415f4c0 2023-11-08T19:25:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928b783640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92841f2800 con 0x7f92841447a0 2023-11-08T19:25:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928951f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928951f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928415a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50922/0 (socket says 172.21.15.119:50922) 2023-11-08T19:25:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.503+0000 7f928951f640 1 -- 172.21.15.119:0/1538307584 learned_addr learned my addr 172.21.15.119:0/1538307584 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f9288d1e640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92841447a0 0x7f928413e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f9289d20640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928415f4c0 0x7f928413eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928951f640 1 -- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92841447a0 msgr2=0x7f928413e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928951f640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92841447a0 0x7f928413e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928951f640 1 -- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928415f4c0 msgr2=0x7f928413eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928951f640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928415f4c0 0x7f928413eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928951f640 1 -- 172.21.15.119:0/1538307584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009b30 con 0x7f928406a120 2023-11-08T19:25:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928951f640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928415a970 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f928000c460 tx=0x7f928000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280012830 con 0x7f928406a120 2023-11-08T19:25:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92800129d0 con 0x7f928406a120 2023-11-08T19:25:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92841f2ae0 con 0x7f928406a120 2023-11-08T19:25:21.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.504+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9280012d80 con 0x7f928406a120 2023-11-08T19:25:21.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.505+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928413e3a0 con 0x7f928406a120 2023-11-08T19:25:21.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.506+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9284074430 con 0x7f928406a120 2023-11-08T19:25:21.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.507+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f928001a3b0 con 0x7f928406a120 2023-11-08T19:25:21.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.507+0000 7f9279ffb640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f926806ae70 0x7f926806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.508+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f92800895d0 con 0x7f928406a120 2023-11-08T19:25:21.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.508+0000 7f9288d1e640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f926806ae70 0x7f926806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.508+0000 7f9288d1e640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f926806ae70 0x7f926806d330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.512+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9280054380 con 0x7f928406a120 2023-11-08T19:25:21.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.704+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92840c3ce0 con 0x7f928406a120 2023-11-08T19:25:21.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.707+0000 7f9279ffb640 1 -- 172.21.15.119:0/1538307584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f92800579a0 con 0x7f928406a120 2023-11-08T19:25:21.709 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:21.710 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:21.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f926806ae70 msgr2=0x7f926806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f926806ae70 0x7f926806d330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274008040 comp rx=0 tx=0).stop 2023-11-08T19:25:21.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 msgr2=0x7f928415a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928415a970 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f928000c460 tx=0x7f928000c920 comp rx=0 tx=0).stop 2023-11-08T19:25:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 shutdown_connections 2023-11-08T19:25:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f926806ae70 0x7f926806d330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f928415f4c0 0x7f928413eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92841447a0 0x7f928413e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 --2- 172.21.15.119:0/1538307584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f928406a120 0x7f928415a970 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 >> 172.21.15.119:0/1538307584 conn(0x7f92840cb610 msgr2=0x7f92840ccce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 shutdown_connections 2023-11-08T19:25:21.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.710+0000 7f928b783640 1 -- 172.21.15.119:0/1538307584 wait complete. 2023-11-08T19:25:21.732 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:25:21.732 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:21.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.939+0000 7fc019535640 1 Processor -- start 2023-11-08T19:25:21.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.939+0000 7fc019535640 1 -- start start 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc019535640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc014162d20 0x7fc014165110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc019535640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01406cff0 con 0x7fc0140ad7d0 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc019535640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01406d2d0 con 0x7fc01415b0d0 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc019535640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01406bd20 con 0x7fc014162d20 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc018d34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc014162d20 0x7fc014165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc00bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc0140ad7d0 0x7fc014070b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59968/0 (socket says 172.21.15.119:59968) 2023-11-08T19:25:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc00bfff640 1 -- 172.21.15.119:0/2174519310 learned_addr learned my addr 172.21.15.119:0/2174519310 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:21.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc018d34640 1 -- 172.21.15.119:0/2174519310 >> v1:172.21.15.188:6789/0 conn(0x7fc01415b0d0 legacy=0x7fc01415b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:21.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc018d34640 1 -- 172.21.15.119:0/2174519310 >> v1:172.21.15.119:6789/0 conn(0x7fc0140ad7d0 legacy=0x7fc014070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.942+0000 7fc018d34640 1 -- 172.21.15.119:0/2174519310 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01406d760 con 0x7fc014162d20 2023-11-08T19:25:21.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.941+0000 7fc0137fe640 1 -- 172.21.15.119:0/2174519310 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4070074744 0 0) 0x7fc01406cff0 con 0x7fc0140ad7d0 2023-11-08T19:25:21.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.943+0000 7fc018d34640 1 --2- 172.21.15.119:0/2174519310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc014162d20 0x7fc014165110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009d90 tx=0x7fc00c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2eabe7b77579aa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.943+0000 7fc0137fe640 1 -- 172.21.15.119:0/2174519310 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02cbd0 con 0x7fc014162d20 2023-11-08T19:25:21.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.943+0000 7fc0137fe640 1 -- 172.21.15.119:0/2174519310 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c02cd30 con 0x7fc014162d20 2023-11-08T19:25:21.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.943+0000 7fc0137fe640 1 -- 172.21.15.119:0/2174519310 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02cea0 con 0x7fc014162d20 2023-11-08T19:25:21.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.943+0000 7fc019535640 1 -- 172.21.15.119:0/2174519310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc014162d20 msgr2=0x7fc014165110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.943+0000 7fc019535640 1 --2- 172.21.15.119:0/2174519310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc014162d20 0x7fc014165110 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009d90 tx=0x7fc00c02be20 comp rx=0 tx=0).stop 2023-11-08T19:25:21.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.944+0000 7fc019535640 1 -- 172.21.15.119:0/2174519310 shutdown_connections 2023-11-08T19:25:21.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.944+0000 7fc019535640 1 --2- 172.21.15.119:0/2174519310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc014162d20 0x7fc014165110 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.944+0000 7fc019535640 1 -- 172.21.15.119:0/2174519310 >> 172.21.15.119:0/2174519310 conn(0x7fc0140b0560 msgr2=0x7fc0140b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:21.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.944+0000 7fc019535640 1 -- 172.21.15.119:0/2174519310 shutdown_connections 2023-11-08T19:25:21.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.944+0000 7fc019535640 1 -- 172.21.15.119:0/2174519310 wait complete. 2023-11-08T19:25:21.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.945+0000 7fc019535640 1 Processor -- start 2023-11-08T19:25:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.945+0000 7fc019535640 1 -- start start 2023-11-08T19:25:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.946+0000 7fc019535640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 0x7fc0141560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.946+0000 7fc019535640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01415b0d0 0x7fc014153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.946+0000 7fc019535640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc014162d20 0x7fc0140ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.946+0000 7fc019535640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0141587c0 con 0x7fc0140ad7d0 2023-11-08T19:25:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.946+0000 7fc019535640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc014156600 con 0x7fc01415b0d0 2023-11-08T19:25:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.946+0000 7fc019535640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01406af30 con 0x7fc014162d20 2023-11-08T19:25:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 0x7fc0141560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 0x7fc0141560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50948/0 (socket says 172.21.15.119:50948) 2023-11-08T19:25:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 -- 172.21.15.119:0/635264582 learned_addr learned my addr 172.21.15.119:0/635264582 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc00bfff640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01415b0d0 0x7fc014153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc018d34640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc014162d20 0x7fc0140ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 -- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc014162d20 msgr2=0x7fc0140ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc014162d20 0x7fc0140ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 -- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01415b0d0 msgr2=0x7fc014153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01415b0d0 0x7fc014153670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 -- 172.21.15.119:0/635264582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c009a70 con 0x7fc0140ad7d0 2023-11-08T19:25:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.947+0000 7fc013fff640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 0x7fc0141560c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c460 tx=0x7fc00400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.948+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004012830 con 0x7fc0140ad7d0 2023-11-08T19:25:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.948+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0040129d0 con 0x7fc0140ad7d0 2023-11-08T19:25:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.948+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc004012d80 con 0x7fc0140ad7d0 2023-11-08T19:25:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.948+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01406a690 con 0x7fc0140ad7d0 2023-11-08T19:25:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.948+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01406ac50 con 0x7fc0140ad7d0 2023-11-08T19:25:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.949+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc014158590 con 0x7fc0140ad7d0 2023-11-08T19:25:21.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.950+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc00401a3b0 con 0x7fc0140ad7d0 2023-11-08T19:25:21.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.951+0000 7fc0117fa640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbff406ae70 0x7fbff406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:21.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.951+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc004089590 con 0x7fc0140ad7d0 2023-11-08T19:25:21.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.951+0000 7fc00bfff640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbff406ae70 0x7fbff406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:21.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.952+0000 7fc00bfff640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbff406ae70 0x7fbff406d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0140c37d0 tx=0x7fbffc007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:21.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:21.955+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc004054350 con 0x7fc0140ad7d0 2023-11-08T19:25:22.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.143+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc014072c30 con 0x7fbff406ae70 2023-11-08T19:25:22.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.157+0000 7fc0117fa640 1 -- 172.21.15.119:0/635264582 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345137 (secure 0 0 0) 0x7fc014072c30 con 0x7fbff406ae70 2023-11-08T19:25:22.164 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:22.166 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:25:22.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.164+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbff406ae70 msgr2=0x7fbff406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.164+0000 7fc019535640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbff406ae70 0x7fbff406d330 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0140c37d0 tx=0x7fbffc007b50 comp rx=0 tx=0).stop 2023-11-08T19:25:22.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.164+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 msgr2=0x7fc0141560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.164+0000 7fc019535640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 0x7fc0141560c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc00400c460 tx=0x7fc00400c920 comp rx=0 tx=0).stop 2023-11-08T19:25:22.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.164+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 shutdown_connections 2023-11-08T19:25:22.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.164+0000 7fc019535640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbff406ae70 0x7fbff406d330 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.165+0000 7fc019535640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc014162d20 0x7fc0140ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.165+0000 7fc019535640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc01415b0d0 0x7fc014153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.165+0000 7fc019535640 1 --2- 172.21.15.119:0/635264582 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0140ad7d0 0x7fc0141560c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.165+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 >> 172.21.15.119:0/635264582 conn(0x7fc0140b0560 msgr2=0x7fc0140af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.165+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 shutdown_connections 2023-11-08T19:25:22.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.165+0000 7fc019535640 1 -- 172.21.15.119:0/635264582 wait complete. 2023-11-08T19:25:22.185 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-11-08T19:25:20.653953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3375104,"data_stored":2042550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004125"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655958+0000","last_change":"2023-11-08T19:24:57.661383+0000","last_active":"2023-11-08T19:25:03.655958+0000","last_peered":"2023-11-08T19:25:03.655958+0000","last_clean":"2023-11-08T19:25:03.655958+0000","last_became_active":"2023-11-08T19:24:57.661122+0000","last_became_peered":"2023-11-08T19:24:57.661122+0000","last_unstale":"2023-11-08T19:25:03.655958+0000","last_undegraded":"2023-11-08T19:25:03.655958+0000","last_fullsized":"2023-11-08T19:25:03.655958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657959+0000","last_change":"2023-11-08T19:24:58.622972+0000","last_active":"2023-11-08T19:25:03.657959+0000","last_peered":"2023-11-08T19:25:03.657959+0000","last_clean":"2023-11-08T19:25:03.657959+0000","last_became_active":"2023-11-08T19:24:58.620907+0000","last_became_peered":"2023-11-08T19:24:58.620907+0000","last_unstale":"2023-11-08T19:25:03.657959+0000","last_undegraded":"2023-11-08T19:25:03.657959+0000","last_fullsized":"2023-11-08T19:25:03.657959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593888+0000","last_change":"2023-11-08T19:25:03.480700+0000","last_active":"2023-11-08T19:25:04.593888+0000","last_peered":"2023-11-08T19:25:04.593888+0000","last_clean":"2023-11-08T19:25:04.593888+0000","last_became_active":"2023-11-08T19:25:03.477779+0000","last_became_peered":"2023-11-08T19:25:03.477779+0000","last_unstale":"2023-11-08T19:25:04.593888+0000","last_undegraded":"2023-11-08T19:25:04.593888+0000","last_fullsized":"2023-11-08T19:25:04.593888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:47:24.018242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658227+0000","last_change":"2023-11-08T19:24:57.655981+0000","last_active":"2023-11-08T19:25:03.658227+0000","last_peered":"2023-11-08T19:25:03.658227+0000","last_clean":"2023-11-08T19:25:03.658227+0000","last_became_active":"2023-11-08T19:24:57.655420+0000","last_became_peered":"2023-11-08T19:24:57.655420+0000","last_unstale":"2023-11-08T19:25:03.658227+0000","last_undegraded":"2023-11-08T19:25:03.658227+0000","last_fullsized":"2023-11-08T19:25:03.658227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655637+0000","last_change":"2023-11-08T19:24:58.620479+0000","last_active":"2023-11-08T19:25:03.655637+0000","last_peered":"2023-11-08T19:25:03.655637+0000","last_clean":"2023-11-08T19:25:03.655637+0000","last_became_active":"2023-11-08T19:24:58.620201+0000","last_became_peered":"2023-11-08T19:24:58.620201+0000","last_unstale":"2023-11-08T19:25:03.655637+0000","last_undegraded":"2023-11-08T19:25:03.655637+0000","last_fullsized":"2023-11-08T19:25:03.655637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.598798+0000","last_change":"2023-11-08T19:25:02.722736+0000","last_active":"2023-11-08T19:25:04.598798+0000","last_peered":"2023-11-08T19:25:04.598798+0000","last_clean":"2023-11-08T19:25:04.598798+0000","last_became_active":"2023-11-08T19:25:02.722087+0000","last_became_peered":"2023-11-08T19:25:02.722087+0000","last_unstale":"2023-11-08T19:25:04.598798+0000","last_undegraded":"2023-11-08T19:25:04.598798+0000","last_fullsized":"2023-11-08T19:25:04.598798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656740+0000","last_change":"2023-11-08T19:24:57.670689+0000","last_active":"2023-11-08T19:25:03.656740+0000","last_peered":"2023-11-08T19:25:03.656740+0000","last_clean":"2023-11-08T19:25:03.656740+0000","last_became_active":"2023-11-08T19:24:57.670406+0000","last_became_peered":"2023-11-08T19:24:57.670406+0000","last_unstale":"2023-11-08T19:25:03.656740+0000","last_undegraded":"2023-11-08T19:25:03.656740+0000","last_fullsized":"2023-11-08T19:25:03.656740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600045+0000","last_change":"2023-11-08T19:24:58.629107+0000","last_active":"2023-11-08T19:25:03.600045+0000","last_peered":"2023-11-08T19:25:03.600045+0000","last_clean":"2023-11-08T19:25:03.600045+0000","last_became_active":"2023-11-08T19:24:58.628670+0000","last_became_peered":"2023-11-08T19:24:58.628670+0000","last_unstale":"2023-11-08T19:25:03.600045+0000","last_undegraded":"2023-11-08T19:25:03.600045+0000","last_fullsized":"2023-11-08T19:25:03.600045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601324+0000","last_change":"2023-11-08T19:25:02.727558+0000","last_active":"2023-11-08T19:25:04.601324+0000","last_peered":"2023-11-08T19:25:04.601324+0000","last_clean":"2023-11-08T19:25:04.601324+0000","last_became_active":"2023-11-08T19:25:02.726979+0000","last_became_peered":"2023-11-08T19:25:02.726979+0000","last_unstale":"2023-11-08T19:25:04.601324+0000","last_undegraded":"2023-11-08T19:25:04.601324+0000","last_fullsized":"2023-11-08T19:25:04.601324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655522+0000","last_change":"2023-11-08T19:24:57.673750+0000","last_active":"2023-11-08T19:25:03.655522+0000","last_peered":"2023-11-08T19:25:03.655522+0000","last_clean":"2023-11-08T19:25:03.655522+0000","last_became_active":"2023-11-08T19:24:57.673328+0000","last_became_peered":"2023-11-08T19:24:57.673328+0000","last_unstale":"2023-11-08T19:25:03.655522+0000","last_undegraded":"2023-11-08T19:25:03.655522+0000","last_fullsized":"2023-11-08T19:25:03.655522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600770+0000","last_change":"2023-11-08T19:24:58.628265+0000","last_active":"2023-11-08T19:25:03.600770+0000","last_peered":"2023-11-08T19:25:03.600770+0000","last_clean":"2023-11-08T19:25:03.600770+0000","last_became_active":"2023-11-08T19:24:58.628007+0000","last_became_peered":"2023-11-08T19:24:58.628007+0000","last_unstale":"2023-11-08T19:25:03.600770+0000","last_undegraded":"2023-11-08T19:25:03.600770+0000","last_fullsized":"2023-11-08T19:25:03.600770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601053+0000","last_change":"2023-11-08T19:25:02.726316+0000","last_active":"2023-11-08T19:25:04.601053+0000","last_peered":"2023-11-08T19:25:04.601053+0000","last_clean":"2023-11-08T19:25:04.601053+0000","last_became_active":"2023-11-08T19:25:02.725661+0000","last_became_peered":"2023-11-08T19:25:02.725661+0000","last_unstale":"2023-11-08T19:25:04.601053+0000","last_undegraded":"2023-11-08T19:25:04.601053+0000","last_fullsized":"2023-11-08T19:25:04.601053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:56.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596618+0000","last_change":"2023-11-08T19:24:58.626589+0000","last_active":"2023-11-08T19:25:03.596618+0000","last_peered":"2023-11-08T19:25:03.596618+0000","last_clean":"2023-11-08T19:25:03.596618+0000","last_became_active":"2023-11-08T19:24:58.626269+0000","last_became_peered":"2023-11-08T19:24:58.626269+0000","last_unstale":"2023-11-08T19:25:03.596618+0000","last_undegraded":"2023-11-08T19:25:03.596618+0000","last_fullsized":"2023-11-08T19:25:03.596618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:57.668738+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:57.668319+0000","last_became_peered":"2023-11-08T19:24:57.668319+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.603469+0000","last_change":"2023-11-08T19:25:02.731537+0000","last_active":"2023-11-08T19:25:04.603469+0000","last_peered":"2023-11-08T19:25:04.603469+0000","last_clean":"2023-11-08T19:25:04.603469+0000","last_became_active":"2023-11-08T19:25:02.731004+0000","last_became_peered":"2023-11-08T19:25:02.731004+0000","last_unstale":"2023-11-08T19:25:04.603469+0000","last_undegraded":"2023-11-08T19:25:04.603469+0000","last_fullsized":"2023-11-08T19:25:04.603469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:58.628177+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:58.627584+0000","last_became_peered":"2023-11-08T19:24:58.627584+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643213+0000","last_change":"2023-11-08T19:24:57.659565+0000","last_active":"2023-11-08T19:25:03.643213+0000","last_peered":"2023-11-08T19:25:03.643213+0000","last_clean":"2023-11-08T19:25:03.643213+0000","last_became_active":"2023-11-08T19:24:57.658443+0000","last_became_peered":"2023-11-08T19:24:57.658443+0000","last_unstale":"2023-11-08T19:25:03.643213+0000","last_undegraded":"2023-11-08T19:25:03.643213+0000","last_fullsized":"2023-11-08T19:25:03.643213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602536+0000","last_change":"2023-11-08T19:25:02.734167+0000","last_active":"2023-11-08T19:25:04.602536+0000","last_peered":"2023-11-08T19:25:04.602536+0000","last_clean":"2023-11-08T19:25:04.602536+0000","last_became_active":"2023-11-08T19:25:02.733732+0000","last_became_peered":"2023-11-08T19:25:02.733732+0000","last_unstale":"2023-11-08T19:25:04.602536+0000","last_undegraded":"2023-11-08T19:25:04.602536+0000","last_fullsized":"2023-11-08T19:25:04.602536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655085+0000","last_change":"2023-11-08T19:24:58.627099+0000","last_active":"2023-11-08T19:25:03.655085+0000","last_peered":"2023-11-08T19:25:03.655085+0000","last_clean":"2023-11-08T19:25:03.655085+0000","last_became_active":"2023-11-08T19:24:58.626261+0000","last_became_peered":"2023-11-08T19:24:58.626261+0000","last_unstale":"2023-11-08T19:25:03.655085+0000","last_undegraded":"2023-11-08T19:25:03.655085+0000","last_fullsized":"2023-11-08T19:25:03.655085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642570+0000","last_change":"2023-11-08T19:24:57.658778+0000","last_active":"2023-11-08T19:25:03.642570+0000","last_peered":"2023-11-08T19:25:03.642570+0000","last_clean":"2023-11-08T19:25:03.642570+0000","last_became_active":"2023-11-08T19:24:57.657370+0000","last_became_peered":"2023-11-08T19:24:57.657370+0000","last_unstale":"2023-11-08T19:25:03.642570+0000","last_undegraded":"2023-11-08T19:25:03.642570+0000","last_fullsized":"2023-11-08T19:25:03.642570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422591+0000","last_change":"2023-11-08T19:25:03.478005+0000","last_active":"2023-11-08T19:25:05.422591+0000","last_peered":"2023-11-08T19:25:05.422591+0000","last_clean":"2023-11-08T19:25:05.422591+0000","last_became_active":"2023-11-08T19:25:03.476666+0000","last_became_peered":"2023-11-08T19:25:03.476666+0000","last_unstale":"2023-11-08T19:25:05.422591+0000","last_undegraded":"2023-11-08T19:25:05.422591+0000","last_fullsized":"2023-11-08T19:25:05.422591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656312+0000","last_change":"2023-11-08T19:24:58.623061+0000","last_active":"2023-11-08T19:25:03.656312+0000","last_peered":"2023-11-08T19:25:03.656312+0000","last_clean":"2023-11-08T19:25:03.656312+0000","last_became_active":"2023-11-08T19:24:58.618367+0000","last_became_peered":"2023-11-08T19:24:58.618367+0000","last_unstale":"2023-11-08T19:25:03.656312+0000","last_undegraded":"2023-11-08T19:25:03.656312+0000","last_fullsized":"2023-11-08T19:25:03.656312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:08.538170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.653991+0000","last_change":"2023-11-08T19:24:57.666853+0000","last_active":"2023-11-08T19:25:03.653991+0000","last_peered":"2023-11-08T19:25:03.653991+0000","last_clean":"2023-11-08T19:25:03.653991+0000","last_became_active":"2023-11-08T19:24:57.666539+0000","last_became_peered":"2023-11-08T19:24:57.666539+0000","last_unstale":"2023-11-08T19:25:03.653991+0000","last_undegraded":"2023-11-08T19:25:03.653991+0000","last_fullsized":"2023-11-08T19:25:03.653991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602888+0000","last_change":"2023-11-08T19:25:02.727013+0000","last_active":"2023-11-08T19:25:04.602888+0000","last_peered":"2023-11-08T19:25:04.602888+0000","last_clean":"2023-11-08T19:25:04.602888+0000","last_became_active":"2023-11-08T19:25:02.726496+0000","last_became_peered":"2023-11-08T19:25:02.726496+0000","last_unstale":"2023-11-08T19:25:04.602888+0000","last_undegraded":"2023-11-08T19:25:04.602888+0000","last_fullsized":"2023-11-08T19:25:04.602888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655207+0000","last_change":"2023-11-08T19:24:58.620701+0000","last_active":"2023-11-08T19:25:03.655207+0000","last_peered":"2023-11-08T19:25:03.655207+0000","last_clean":"2023-11-08T19:25:03.655207+0000","last_became_active":"2023-11-08T19:24:58.620386+0000","last_became_peered":"2023-11-08T19:24:58.620386+0000","last_unstale":"2023-11-08T19:25:03.655207+0000","last_undegraded":"2023-11-08T19:25:03.655207+0000","last_fullsized":"2023-11-08T19:25:03.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597904+0000","last_change":"2023-11-08T19:24:57.667152+0000","last_active":"2023-11-08T19:25:03.597904+0000","last_peered":"2023-11-08T19:25:03.597904+0000","last_clean":"2023-11-08T19:25:03.597904+0000","last_became_active":"2023-11-08T19:24:57.666522+0000","last_became_peered":"2023-11-08T19:24:57.666522+0000","last_unstale":"2023-11-08T19:25:03.597904+0000","last_undegraded":"2023-11-08T19:25:03.597904+0000","last_fullsized":"2023-11-08T19:25:03.597904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:33.630876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599610+0000","last_change":"2023-11-08T19:25:03.469823+0000","last_active":"2023-11-08T19:25:04.599610+0000","last_peered":"2023-11-08T19:25:04.599610+0000","last_clean":"2023-11-08T19:25:04.599610+0000","last_became_active":"2023-11-08T19:25:03.468656+0000","last_became_peered":"2023-11-08T19:25:03.468656+0000","last_unstale":"2023-11-08T19:25:04.599610+0000","last_undegraded":"2023-11-08T19:25:04.599610+0000","last_fullsized":"2023-11-08T19:25:04.599610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.633451+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.633159+0000","last_became_peered":"2023-11-08T19:24:58.633159+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.644112+0000","last_change":"2023-11-08T19:24:57.663788+0000","last_active":"2023-11-08T19:25:03.644112+0000","last_peered":"2023-11-08T19:25:03.644112+0000","last_clean":"2023-11-08T19:25:03.644112+0000","last_became_active":"2023-11-08T19:24:57.663457+0000","last_became_peered":"2023-11-08T19:24:57.663457+0000","last_unstale":"2023-11-08T19:25:03.644112+0000","last_undegraded":"2023-11-08T19:25:03.644112+0000","last_fullsized":"2023-11-08T19:25:03.644112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593332+0000","last_change":"2023-11-08T19:25:03.469891+0000","last_active":"2023-11-08T19:25:04.593332+0000","last_peered":"2023-11-08T19:25:04.593332+0000","last_clean":"2023-11-08T19:25:04.593332+0000","last_became_active":"2023-11-08T19:25:03.466854+0000","last_became_peered":"2023-11-08T19:25:03.466854+0000","last_unstale":"2023-11-08T19:25:04.593332+0000","last_undegraded":"2023-11-08T19:25:04.593332+0000","last_fullsized":"2023-11-08T19:25:04.593332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:05.405858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643596+0000","last_change":"2023-11-08T19:24:58.622842+0000","last_active":"2023-11-08T19:25:03.643596+0000","last_peered":"2023-11-08T19:25:03.643596+0000","last_clean":"2023-11-08T19:25:03.643596+0000","last_became_active":"2023-11-08T19:24:58.622530+0000","last_became_peered":"2023-11-08T19:24:58.622530+0000","last_unstale":"2023-11-08T19:25:03.643596+0000","last_undegraded":"2023-11-08T19:25:03.643596+0000","last_fullsized":"2023-11-08T19:25:03.643596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656961+0000","last_change":"2023-11-08T19:24:57.669407+0000","last_active":"2023-11-08T19:25:03.656961+0000","last_peered":"2023-11-08T19:25:03.656961+0000","last_clean":"2023-11-08T19:25:03.656961+0000","last_became_active":"2023-11-08T19:24:57.669169+0000","last_became_peered":"2023-11-08T19:24:57.669169+0000","last_unstale":"2023-11-08T19:25:03.656961+0000","last_undegraded":"2023-11-08T19:25:03.656961+0000","last_fullsized":"2023-11-08T19:25:03.656961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:50.920808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530152+0000","last_change":"2023-11-08T19:25:03.469685+0000","last_active":"2023-11-08T19:25:05.530152+0000","last_peered":"2023-11-08T19:25:05.530152+0000","last_clean":"2023-11-08T19:25:05.530152+0000","last_became_active":"2023-11-08T19:25:03.468600+0000","last_became_peered":"2023-11-08T19:25:03.468600+0000","last_unstale":"2023-11-08T19:25:05.530152+0000","last_undegraded":"2023-11-08T19:25:05.530152+0000","last_fullsized":"2023-11-08T19:25:05.530152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600914+0000","last_change":"2023-11-08T19:24:58.617568+0000","last_active":"2023-11-08T19:25:03.600914+0000","last_peered":"2023-11-08T19:25:03.600914+0000","last_clean":"2023-11-08T19:25:03.600914+0000","last_became_active":"2023-11-08T19:24:58.617216+0000","last_became_peered":"2023-11-08T19:24:58.617216+0000","last_unstale":"2023-11-08T19:25:03.600914+0000","last_undegraded":"2023-11-08T19:25:03.600914+0000","last_fullsized":"2023-11-08T19:25:03.600914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643888+0000","last_change":"2023-11-08T19:24:57.653655+0000","last_active":"2023-11-08T19:25:03.643888+0000","last_peered":"2023-11-08T19:25:03.643888+0000","last_clean":"2023-11-08T19:25:03.643888+0000","last_became_active":"2023-11-08T19:24:57.653390+0000","last_became_peered":"2023-11-08T19:24:57.653390+0000","last_unstale":"2023-11-08T19:25:03.643888+0000","last_undegraded":"2023-11-08T19:25:03.643888+0000","last_fullsized":"2023-11-08T19:25:03.643888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421356+0000","last_change":"2023-11-08T19:25:02.724803+0000","last_active":"2023-11-08T19:25:05.421356+0000","last_peered":"2023-11-08T19:25:05.421356+0000","last_clean":"2023-11-08T19:25:05.421356+0000","last_became_active":"2023-11-08T19:25:02.724278+0000","last_became_peered":"2023-11-08T19:25:02.724278+0000","last_unstale":"2023-11-08T19:25:05.421356+0000","last_undegraded":"2023-11-08T19:25:05.421356+0000","last_fullsized":"2023-11-08T19:25:05.421356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655769+0000","last_change":"2023-11-08T19:24:58.617441+0000","last_active":"2023-11-08T19:25:03.655769+0000","last_peered":"2023-11-08T19:25:03.655769+0000","last_clean":"2023-11-08T19:25:03.655769+0000","last_became_active":"2023-11-08T19:24:58.617030+0000","last_became_peered":"2023-11-08T19:24:58.617030+0000","last_unstale":"2023-11-08T19:25:03.655769+0000","last_undegraded":"2023-11-08T19:25:03.655769+0000","last_fullsized":"2023-11-08T19:25:03.655769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:50.043019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599773+0000","last_change":"2023-11-08T19:24:57.662494+0000","last_active":"2023-11-08T19:25:03.599773+0000","last_peered":"2023-11-08T19:25:03.599773+0000","last_clean":"2023-11-08T19:25:03.599773+0000","last_became_active":"2023-11-08T19:24:57.662223+0000","last_became_peered":"2023-11-08T19:24:57.662223+0000","last_unstale":"2023-11-08T19:25:03.599773+0000","last_undegraded":"2023-11-08T19:25:03.599773+0000","last_fullsized":"2023-11-08T19:25:03.599773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:49.940163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421797+0000","last_change":"2023-11-08T19:25:02.729279+0000","last_active":"2023-11-08T19:25:05.421797+0000","last_peered":"2023-11-08T19:25:05.421797+0000","last_clean":"2023-11-08T19:25:05.421797+0000","last_became_active":"2023-11-08T19:25:02.728855+0000","last_became_peered":"2023-11-08T19:25:02.728855+0000","last_unstale":"2023-11-08T19:25:05.421797+0000","last_undegraded":"2023-11-08T19:25:05.421797+0000","last_fullsized":"2023-11-08T19:25:05.421797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642895+0000","last_change":"2023-11-08T19:24:58.617415+0000","last_active":"2023-11-08T19:25:03.642895+0000","last_peered":"2023-11-08T19:25:03.642895+0000","last_clean":"2023-11-08T19:25:03.642895+0000","last_became_active":"2023-11-08T19:24:58.616880+0000","last_became_peered":"2023-11-08T19:24:58.616880+0000","last_unstale":"2023-11-08T19:25:03.642895+0000","last_undegraded":"2023-11-08T19:25:03.642895+0000","last_fullsized":"2023-11-08T19:25:03.642895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592969+0000","last_change":"2023-11-08T19:24:57.650498+0000","last_active":"2023-11-08T19:25:03.592969+0000","last_peered":"2023-11-08T19:25:03.592969+0000","last_clean":"2023-11-08T19:25:03.592969+0000","last_became_active":"2023-11-08T19:24:57.650172+0000","last_became_peered":"2023-11-08T19:24:57.650172+0000","last_unstale":"2023-11-08T19:25:03.592969+0000","last_undegraded":"2023-11-08T19:25:03.592969+0000","last_fullsized":"2023-11-08T19:25:03.592969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422691+0000","last_change":"2023-11-08T19:25:02.722430+0000","last_active":"2023-11-08T19:25:05.422691+0000","last_peered":"2023-11-08T19:25:05.422691+0000","last_clean":"2023-11-08T19:25:05.422691+0000","last_became_active":"2023-11-08T19:25:02.722013+0000","last_became_peered":"2023-11-08T19:25:02.722013+0000","last_unstale":"2023-11-08T19:25:05.422691+0000","last_undegraded":"2023-11-08T19:25:05.422691+0000","last_fullsized":"2023-11-08T19:25:05.422691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654808+0000","last_change":"2023-11-08T19:24:58.629901+0000","last_active":"2023-11-08T19:25:03.654808+0000","last_peered":"2023-11-08T19:25:03.654808+0000","last_clean":"2023-11-08T19:25:03.654808+0000","last_became_active":"2023-11-08T19:24:58.629603+0000","last_became_peered":"2023-11-08T19:24:58.629603+0000","last_unstale":"2023-11-08T19:25:03.654808+0000","last_undegraded":"2023-11-08T19:25:03.654808+0000","last_fullsized":"2023-11-08T19:25:03.654808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:19.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654594+0000","last_change":"2023-11-08T19:24:57.667258+0000","last_active":"2023-11-08T19:25:03.654594+0000","last_peered":"2023-11-08T19:25:03.654594+0000","last_clean":"2023-11-08T19:25:03.654594+0000","last_became_active":"2023-11-08T19:24:57.666859+0000","last_became_peered":"2023-11-08T19:24:57.666859+0000","last_unstale":"2023-11-08T19:25:03.654594+0000","last_undegraded":"2023-11-08T19:25:03.654594+0000","last_fullsized":"2023-11-08T19:25:03.654594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530805+0000","last_change":"2023-11-08T19:25:03.480894+0000","last_active":"2023-11-08T19:25:05.530805+0000","last_peered":"2023-11-08T19:25:05.530805+0000","last_clean":"2023-11-08T19:25:05.530805+0000","last_became_active":"2023-11-08T19:25:03.479219+0000","last_became_peered":"2023-11-08T19:25:03.479219+0000","last_unstale":"2023-11-08T19:25:05.530805+0000","last_undegraded":"2023-11-08T19:25:05.530805+0000","last_fullsized":"2023-11-08T19:25:05.530805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656729+0000","last_change":"2023-11-08T19:24:58.620051+0000","last_active":"2023-11-08T19:25:03.656729+0000","last_peered":"2023-11-08T19:25:03.656729+0000","last_clean":"2023-11-08T19:25:03.656729+0000","last_became_active":"2023-11-08T19:24:58.619641+0000","last_became_peered":"2023-11-08T19:24:58.619641+0000","last_unstale":"2023-11-08T19:25:03.656729+0000","last_undegraded":"2023-11-08T19:25:03.656729+0000","last_fullsized":"2023-11-08T19:25:03.656729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599436+0000","last_change":"2023-11-08T19:24:57.656028+0000","last_active":"2023-11-08T19:25:03.599436+0000","last_peered":"2023-11-08T19:25:03.599436+0000","last_clean":"2023-11-08T19:25:03.599436+0000","last_became_active":"2023-11-08T19:24:57.651726+0000","last_became_peered":"2023-11-08T19:24:57.651726+0000","last_unstale":"2023-11-08T19:25:03.599436+0000","last_undegraded":"2023-11-08T19:25:03.599436+0000","last_fullsized":"2023-11-08T19:25:03.599436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654244+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:25:03.654244+0000","last_peered":"2023-11-08T19:25:03.654244+0000","last_clean":"2023-11-08T19:25:03.654244+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:25:03.654244+0000","last_undegraded":"2023-11-08T19:25:03.654244+0000","last_fullsized":"2023-11-08T19:25:03.654244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349456+0000","last_change":"2023-11-08T19:25:03.468384+0000","last_active":"2023-11-08T19:25:06.349456+0000","last_peered":"2023-11-08T19:25:06.349456+0000","last_clean":"2023-11-08T19:25:06.349456+0000","last_became_active":"2023-11-08T19:25:03.467219+0000","last_became_peered":"2023-11-08T19:25:03.467219+0000","last_unstale":"2023-11-08T19:25:06.349456+0000","last_undegraded":"2023-11-08T19:25:06.349456+0000","last_fullsized":"2023-11-08T19:25:06.349456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656350+0000","last_change":"2023-11-08T19:24:58.632161+0000","last_active":"2023-11-08T19:25:03.656350+0000","last_peered":"2023-11-08T19:25:03.656350+0000","last_clean":"2023-11-08T19:25:03.656350+0000","last_became_active":"2023-11-08T19:24:58.631360+0000","last_became_peered":"2023-11-08T19:24:58.631360+0000","last_unstale":"2023-11-08T19:25:03.656350+0000","last_undegraded":"2023-11-08T19:25:03.656350+0000","last_fullsized":"2023-11-08T19:25:03.656350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594245+0000","last_change":"2023-11-08T19:24:57.659462+0000","last_active":"2023-11-08T19:25:03.594245+0000","last_peered":"2023-11-08T19:25:03.594245+0000","last_clean":"2023-11-08T19:25:03.594245+0000","last_became_active":"2023-11-08T19:24:57.659200+0000","last_became_peered":"2023-11-08T19:24:57.659200+0000","last_unstale":"2023-11-08T19:25:03.594245+0000","last_undegraded":"2023-11-08T19:25:03.594245+0000","last_fullsized":"2023-11-08T19:25:03.594245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:21.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.348403+0000","last_change":"2023-11-08T19:25:03.468319+0000","last_active":"2023-11-08T19:25:06.348403+0000","last_peered":"2023-11-08T19:25:06.348403+0000","last_clean":"2023-11-08T19:25:06.348403+0000","last_became_active":"2023-11-08T19:25:03.466962+0000","last_became_peered":"2023-11-08T19:25:03.466962+0000","last_unstale":"2023-11-08T19:25:06.348403+0000","last_undegraded":"2023-11-08T19:25:06.348403+0000","last_fullsized":"2023-11-08T19:25:06.348403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600157+0000","last_change":"2023-11-08T19:24:58.620109+0000","last_active":"2023-11-08T19:25:03.600157+0000","last_peered":"2023-11-08T19:25:03.600157+0000","last_clean":"2023-11-08T19:25:03.600157+0000","last_became_active":"2023-11-08T19:24:58.619504+0000","last_became_peered":"2023-11-08T19:24:58.619504+0000","last_unstale":"2023-11-08T19:25:03.600157+0000","last_undegraded":"2023-11-08T19:25:03.600157+0000","last_fullsized":"2023-11-08T19:25:03.600157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643946+0000","last_change":"2023-11-08T19:24:57.658265+0000","last_active":"2023-11-08T19:25:03.643946+0000","last_peered":"2023-11-08T19:25:03.643946+0000","last_clean":"2023-11-08T19:25:03.643946+0000","last_became_active":"2023-11-08T19:24:57.657921+0000","last_became_peered":"2023-11-08T19:24:57.657921+0000","last_unstale":"2023-11-08T19:25:03.643946+0000","last_undegraded":"2023-11-08T19:25:03.643946+0000","last_fullsized":"2023-11-08T19:25:03.643946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530244+0000","last_change":"2023-11-08T19:25:02.727032+0000","last_active":"2023-11-08T19:25:05.530244+0000","last_peered":"2023-11-08T19:25:05.530244+0000","last_clean":"2023-11-08T19:25:05.530244+0000","last_became_active":"2023-11-08T19:25:02.726636+0000","last_became_peered":"2023-11-08T19:25:02.726636+0000","last_unstale":"2023-11-08T19:25:05.530244+0000","last_undegraded":"2023-11-08T19:25:05.530244+0000","last_fullsized":"2023-11-08T19:25:05.530244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656744+0000","last_change":"2023-11-08T19:24:58.641465+0000","last_active":"2023-11-08T19:25:03.656744+0000","last_peered":"2023-11-08T19:25:03.656744+0000","last_clean":"2023-11-08T19:25:03.656744+0000","last_became_active":"2023-11-08T19:24:58.641144+0000","last_became_peered":"2023-11-08T19:24:58.641144+0000","last_unstale":"2023-11-08T19:25:03.656744+0000","last_undegraded":"2023-11-08T19:25:03.656744+0000","last_fullsized":"2023-11-08T19:25:03.656744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:00.683350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594008+0000","last_change":"2023-11-08T19:24:57.653180+0000","last_active":"2023-11-08T19:25:03.594008+0000","last_peered":"2023-11-08T19:25:03.594008+0000","last_clean":"2023-11-08T19:25:03.594008+0000","last_became_active":"2023-11-08T19:24:57.652617+0000","last_became_peered":"2023-11-08T19:24:57.652617+0000","last_unstale":"2023-11-08T19:25:03.594008+0000","last_undegraded":"2023-11-08T19:25:03.594008+0000","last_fullsized":"2023-11-08T19:25:03.594008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:35.866163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599737+0000","last_change":"2023-11-08T19:25:03.474070+0000","last_active":"2023-11-08T19:25:04.599737+0000","last_peered":"2023-11-08T19:25:04.599737+0000","last_clean":"2023-11-08T19:25:04.599737+0000","last_became_active":"2023-11-08T19:25:03.471994+0000","last_became_peered":"2023-11-08T19:25:03.471994+0000","last_unstale":"2023-11-08T19:25:04.599737+0000","last_undegraded":"2023-11-08T19:25:04.599737+0000","last_fullsized":"2023-11-08T19:25:04.599737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655993+0000","last_change":"2023-11-08T19:24:58.640557+0000","last_active":"2023-11-08T19:25:03.655993+0000","last_peered":"2023-11-08T19:25:03.655993+0000","last_clean":"2023-11-08T19:25:03.655993+0000","last_became_active":"2023-11-08T19:24:58.640005+0000","last_became_peered":"2023-11-08T19:24:58.640005+0000","last_unstale":"2023-11-08T19:25:03.655993+0000","last_undegraded":"2023-11-08T19:25:03.655993+0000","last_fullsized":"2023-11-08T19:25:03.655993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657695+0000","last_change":"2023-11-08T19:24:57.652195+0000","last_active":"2023-11-08T19:25:03.657695+0000","last_peered":"2023-11-08T19:25:03.657695+0000","last_clean":"2023-11-08T19:25:03.657695+0000","last_became_active":"2023-11-08T19:24:57.651790+0000","last_became_peered":"2023-11-08T19:24:57.651790+0000","last_unstale":"2023-11-08T19:25:03.657695+0000","last_undegraded":"2023-11-08T19:25:03.657695+0000","last_fullsized":"2023-11-08T19:25:03.657695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:50:21.039412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599054+0000","last_change":"2023-11-08T19:25:02.722727+0000","last_active":"2023-11-08T19:25:04.599054+0000","last_peered":"2023-11-08T19:25:04.599054+0000","last_clean":"2023-11-08T19:25:04.599054+0000","last_became_active":"2023-11-08T19:25:02.722092+0000","last_became_peered":"2023-11-08T19:25:02.722092+0000","last_unstale":"2023-11-08T19:25:04.599054+0000","last_undegraded":"2023-11-08T19:25:04.599054+0000","last_fullsized":"2023-11-08T19:25:04.599054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655407+0000","last_change":"2023-11-08T19:24:58.635248+0000","last_active":"2023-11-08T19:25:03.655407+0000","last_peered":"2023-11-08T19:25:03.655407+0000","last_clean":"2023-11-08T19:25:03.655407+0000","last_became_active":"2023-11-08T19:24:58.634504+0000","last_became_peered":"2023-11-08T19:24:58.634504+0000","last_unstale":"2023-11-08T19:25:03.655407+0000","last_undegraded":"2023-11-08T19:25:03.655407+0000","last_fullsized":"2023-11-08T19:25:03.655407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592589+0000","last_change":"2023-11-08T19:24:57.651955+0000","last_active":"2023-11-08T19:25:03.592589+0000","last_peered":"2023-11-08T19:25:03.592589+0000","last_clean":"2023-11-08T19:25:03.592589+0000","last_became_active":"2023-11-08T19:24:57.651666+0000","last_became_peered":"2023-11-08T19:24:57.651666+0000","last_unstale":"2023-11-08T19:25:03.592589+0000","last_undegraded":"2023-11-08T19:25:03.592589+0000","last_fullsized":"2023-11-08T19:25:03.592589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530618+0000","last_change":"2023-11-08T19:25:02.728613+0000","last_active":"2023-11-08T19:25:05.530618+0000","last_peered":"2023-11-08T19:25:05.530618+0000","last_clean":"2023-11-08T19:25:05.530618+0000","last_became_active":"2023-11-08T19:25:02.727749+0000","last_became_peered":"2023-11-08T19:25:02.727749+0000","last_unstale":"2023-11-08T19:25:05.530618+0000","last_undegraded":"2023-11-08T19:25:05.530618+0000","last_fullsized":"2023-11-08T19:25:05.530618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595934+0000","last_change":"2023-11-08T19:24:58.616453+0000","last_active":"2023-11-08T19:25:03.595934+0000","last_peered":"2023-11-08T19:25:03.595934+0000","last_clean":"2023-11-08T19:25:03.595934+0000","last_became_active":"2023-11-08T19:24:58.615934+0000","last_became_peered":"2023-11-08T19:24:58.615934+0000","last_unstale":"2023-11-08T19:25:03.595934+0000","last_undegraded":"2023-11-08T19:25:03.595934+0000","last_fullsized":"2023-11-08T19:25:03.595934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596954+0000","last_change":"2023-11-08T19:24:57.659585+0000","last_active":"2023-11-08T19:25:03.596954+0000","last_peered":"2023-11-08T19:25:03.596954+0000","last_clean":"2023-11-08T19:25:03.596954+0000","last_became_active":"2023-11-08T19:24:57.659197+0000","last_became_peered":"2023-11-08T19:24:57.659197+0000","last_unstale":"2023-11-08T19:25:03.596954+0000","last_undegraded":"2023-11-08T19:25:03.596954+0000","last_fullsized":"2023-11-08T19:25:03.596954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595745+0000","last_change":"2023-11-08T19:25:03.468147+0000","last_active":"2023-11-08T19:25:04.595745+0000","last_peered":"2023-11-08T19:25:04.595745+0000","last_clean":"2023-11-08T19:25:04.595745+0000","last_became_active":"2023-11-08T19:25:03.466921+0000","last_became_peered":"2023-11-08T19:25:03.466921+0000","last_unstale":"2023-11-08T19:25:04.595745+0000","last_undegraded":"2023-11-08T19:25:04.595745+0000","last_fullsized":"2023-11-08T19:25:04.595745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:14:51.027517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656098+0000","last_change":"2023-11-08T19:24:58.636188+0000","last_active":"2023-11-08T19:25:03.656098+0000","last_peered":"2023-11-08T19:25:03.656098+0000","last_clean":"2023-11-08T19:25:03.656098+0000","last_became_active":"2023-11-08T19:24:58.635889+0000","last_became_peered":"2023-11-08T19:24:58.635889+0000","last_unstale":"2023-11-08T19:25:03.656098+0000","last_undegraded":"2023-11-08T19:25:03.656098+0000","last_fullsized":"2023-11-08T19:25:03.656098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599561+0000","last_change":"2023-11-08T19:24:57.655323+0000","last_active":"2023-11-08T19:25:03.599561+0000","last_peered":"2023-11-08T19:25:03.599561+0000","last_clean":"2023-11-08T19:25:03.599561+0000","last_became_active":"2023-11-08T19:24:57.654973+0000","last_became_peered":"2023-11-08T19:24:57.654973+0000","last_unstale":"2023-11-08T19:25:03.599561+0000","last_undegraded":"2023-11-08T19:25:03.599561+0000","last_fullsized":"2023-11-08T19:25:03.599561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594252+0000","last_change":"2023-11-08T19:25:03.467735+0000","last_active":"2023-11-08T19:25:04.594252+0000","last_peered":"2023-11-08T19:25:04.594252+0000","last_clean":"2023-11-08T19:25:04.594252+0000","last_became_active":"2023-11-08T19:25:03.466595+0000","last_became_peered":"2023-11-08T19:25:03.466595+0000","last_unstale":"2023-11-08T19:25:04.594252+0000","last_undegraded":"2023-11-08T19:25:04.594252+0000","last_fullsized":"2023-11-08T19:25:04.594252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655761+0000","last_change":"2023-11-08T19:24:58.626959+0000","last_active":"2023-11-08T19:25:03.655761+0000","last_peered":"2023-11-08T19:25:03.655761+0000","last_clean":"2023-11-08T19:25:03.655761+0000","last_became_active":"2023-11-08T19:24:58.626616+0000","last_became_peered":"2023-11-08T19:24:58.626616+0000","last_unstale":"2023-11-08T19:25:03.655761+0000","last_undegraded":"2023-11-08T19:25:03.655761+0000","last_fullsized":"2023-11-08T19:25:03.655761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:57.663161+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:57.660362+0000","last_became_peered":"2023-11-08T19:24:57.660362+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602355+0000","last_change":"2023-11-08T19:25:03.477682+0000","last_active":"2023-11-08T19:25:04.602355+0000","last_peered":"2023-11-08T19:25:04.602355+0000","last_clean":"2023-11-08T19:25:04.602355+0000","last_became_active":"2023-11-08T19:25:03.476098+0000","last_became_peered":"2023-11-08T19:25:03.476098+0000","last_unstale":"2023-11-08T19:25:04.602355+0000","last_undegraded":"2023-11-08T19:25:04.602355+0000","last_fullsized":"2023-11-08T19:25:04.602355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656221+0000","last_change":"2023-11-08T19:24:58.634601+0000","last_active":"2023-11-08T19:25:03.656221+0000","last_peered":"2023-11-08T19:25:03.656221+0000","last_clean":"2023-11-08T19:25:03.656221+0000","last_became_active":"2023-11-08T19:24:58.632876+0000","last_became_peered":"2023-11-08T19:24:58.632876+0000","last_unstale":"2023-11-08T19:25:03.656221+0000","last_undegraded":"2023-11-08T19:25:03.656221+0000","last_fullsized":"2023-11-08T19:25:03.656221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:09.698655+0000","last_change":"2023-11-08T19:24:57.660289+0000","last_active":"2023-11-08T19:25:09.698655+0000","last_peered":"2023-11-08T19:25:09.698655+0000","last_clean":"2023-11-08T19:25:09.698655+0000","last_became_active":"2023-11-08T19:24:57.659982+0000","last_became_peered":"2023-11-08T19:24:57.659982+0000","last_unstale":"2023-11-08T19:25:09.698655+0000","last_undegraded":"2023-11-08T19:25:09.698655+0000","last_fullsized":"2023-11-08T19:25:09.698655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421411+0000","last_change":"2023-11-08T19:25:02.718892+0000","last_active":"2023-11-08T19:25:05.421411+0000","last_peered":"2023-11-08T19:25:05.421411+0000","last_clean":"2023-11-08T19:25:05.421411+0000","last_became_active":"2023-11-08T19:25:02.717998+0000","last_became_peered":"2023-11-08T19:25:02.717998+0000","last_unstale":"2023-11-08T19:25:05.421411+0000","last_undegraded":"2023-11-08T19:25:05.421411+0000","last_fullsized":"2023-11-08T19:25:05.421411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:58.629542+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:58.627201+0000","last_became_peered":"2023-11-08T19:24:58.627201+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643852+0000","last_change":"2023-11-08T19:24:57.661152+0000","last_active":"2023-11-08T19:25:03.643852+0000","last_peered":"2023-11-08T19:25:03.643852+0000","last_clean":"2023-11-08T19:25:03.643852+0000","last_became_active":"2023-11-08T19:24:57.660740+0000","last_became_peered":"2023-11-08T19:24:57.660740+0000","last_unstale":"2023-11-08T19:25:03.643852+0000","last_undegraded":"2023-11-08T19:25:03.643852+0000","last_fullsized":"2023-11-08T19:25:03.643852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602299+0000","last_change":"2023-11-08T19:25:02.727543+0000","last_active":"2023-11-08T19:25:04.602299+0000","last_peered":"2023-11-08T19:25:04.602299+0000","last_clean":"2023-11-08T19:25:04.602299+0000","last_became_active":"2023-11-08T19:25:02.726809+0000","last_became_peered":"2023-11-08T19:25:02.726809+0000","last_unstale":"2023-11-08T19:25:04.602299+0000","last_undegraded":"2023-11-08T19:25:04.602299+0000","last_fullsized":"2023-11-08T19:25:04.602299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655948+0000","last_change":"2023-11-08T19:24:58.618188+0000","last_active":"2023-11-08T19:25:03.655948+0000","last_peered":"2023-11-08T19:25:03.655948+0000","last_clean":"2023-11-08T19:25:03.655948+0000","last_became_active":"2023-11-08T19:24:58.617893+0000","last_became_peered":"2023-11-08T19:24:58.617893+0000","last_unstale":"2023-11-08T19:25:03.655948+0000","last_undegraded":"2023-11-08T19:25:03.655948+0000","last_fullsized":"2023-11-08T19:25:03.655948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600932+0000","last_change":"2023-11-08T19:24:57.662272+0000","last_active":"2023-11-08T19:25:03.600932+0000","last_peered":"2023-11-08T19:25:03.600932+0000","last_clean":"2023-11-08T19:25:03.600932+0000","last_became_active":"2023-11-08T19:24:57.662012+0000","last_became_peered":"2023-11-08T19:24:57.662012+0000","last_unstale":"2023-11-08T19:25:03.600932+0000","last_undegraded":"2023-11-08T19:25:03.600932+0000","last_fullsized":"2023-11-08T19:25:03.600932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594552+0000","last_change":"2023-11-08T19:25:02.729257+0000","last_active":"2023-11-08T19:25:04.594552+0000","last_peered":"2023-11-08T19:25:04.594552+0000","last_clean":"2023-11-08T19:25:04.594552+0000","last_became_active":"2023-11-08T19:25:02.727346+0000","last_became_peered":"2023-11-08T19:25:02.727346+0000","last_unstale":"2023-11-08T19:25:04.594552+0000","last_undegraded":"2023-11-08T19:25:04.594552+0000","last_fullsized":"2023-11-08T19:25:04.594552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:45.742494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600772+0000","last_change":"2023-11-08T19:24:58.626572+0000","last_active":"2023-11-08T19:25:03.600772+0000","last_peered":"2023-11-08T19:25:03.600772+0000","last_clean":"2023-11-08T19:25:03.600772+0000","last_became_active":"2023-11-08T19:24:58.626191+0000","last_became_peered":"2023-11-08T19:24:58.626191+0000","last_unstale":"2023-11-08T19:25:03.600772+0000","last_undegraded":"2023-11-08T19:25:03.600772+0000","last_fullsized":"2023-11-08T19:25:03.600772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597705+0000","last_change":"2023-11-08T19:24:57.672262+0000","last_active":"2023-11-08T19:25:03.597705+0000","last_peered":"2023-11-08T19:25:03.597705+0000","last_clean":"2023-11-08T19:25:03.597705+0000","last_became_active":"2023-11-08T19:24:57.671965+0000","last_became_peered":"2023-11-08T19:24:57.671965+0000","last_unstale":"2023-11-08T19:25:03.597705+0000","last_undegraded":"2023-11-08T19:25:03.597705+0000","last_fullsized":"2023-11-08T19:25:03.597705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594345+0000","last_change":"2023-11-08T19:25:03.467509+0000","last_active":"2023-11-08T19:25:04.594345+0000","last_peered":"2023-11-08T19:25:04.594345+0000","last_clean":"2023-11-08T19:25:04.594345+0000","last_became_active":"2023-11-08T19:25:03.466319+0000","last_became_peered":"2023-11-08T19:25:03.466319+0000","last_unstale":"2023-11-08T19:25:04.594345+0000","last_undegraded":"2023-11-08T19:25:04.594345+0000","last_fullsized":"2023-11-08T19:25:04.594345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:44.481960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530577+0000","last_change":"2023-11-08T19:25:03.475886+0000","last_active":"2023-11-08T19:25:05.530577+0000","last_peered":"2023-11-08T19:25:05.530577+0000","last_clean":"2023-11-08T19:25:05.530577+0000","last_became_active":"2023-11-08T19:25:03.473511+0000","last_became_peered":"2023-11-08T19:25:03.473511+0000","last_unstale":"2023-11-08T19:25:05.530577+0000","last_undegraded":"2023-11-08T19:25:05.530577+0000","last_fullsized":"2023-11-08T19:25:05.530577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642306+0000","last_change":"2023-11-08T19:24:58.621533+0000","last_active":"2023-11-08T19:25:03.642306+0000","last_peered":"2023-11-08T19:25:03.642306+0000","last_clean":"2023-11-08T19:25:03.642306+0000","last_became_active":"2023-11-08T19:24:58.621214+0000","last_became_peered":"2023-11-08T19:24:58.621214+0000","last_unstale":"2023-11-08T19:25:03.642306+0000","last_undegraded":"2023-11-08T19:25:03.642306+0000","last_fullsized":"2023-11-08T19:25:03.642306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596137+0000","last_change":"2023-11-08T19:24:57.656298+0000","last_active":"2023-11-08T19:25:03.596137+0000","last_peered":"2023-11-08T19:25:03.596137+0000","last_clean":"2023-11-08T19:25:03.596137+0000","last_became_active":"2023-11-08T19:24:57.655961+0000","last_became_peered":"2023-11-08T19:24:57.655961+0000","last_unstale":"2023-11-08T19:25:03.596137+0000","last_undegraded":"2023-11-08T19:25:03.596137+0000","last_fullsized":"2023-11-08T19:25:03.596137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349973+0000","last_change":"2023-11-08T19:25:02.721600+0000","last_active":"2023-11-08T19:25:06.349973+0000","last_peered":"2023-11-08T19:25:06.349973+0000","last_clean":"2023-11-08T19:25:06.349973+0000","last_became_active":"2023-11-08T19:25:02.721169+0000","last_became_peered":"2023-11-08T19:25:02.721169+0000","last_unstale":"2023-11-08T19:25:06.349973+0000","last_undegraded":"2023-11-08T19:25:06.349973+0000","last_fullsized":"2023-11-08T19:25:06.349973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654845+0000","last_change":"2023-11-08T19:24:58.624276+0000","last_active":"2023-11-08T19:25:03.654845+0000","last_peered":"2023-11-08T19:25:03.654845+0000","last_clean":"2023-11-08T19:25:03.654845+0000","last_became_active":"2023-11-08T19:24:58.623432+0000","last_became_peered":"2023-11-08T19:24:58.623432+0000","last_unstale":"2023-11-08T19:25:03.654845+0000","last_undegraded":"2023-11-08T19:25:03.654845+0000","last_fullsized":"2023-11-08T19:25:03.654845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656023+0000","last_change":"2023-11-08T19:24:57.661677+0000","last_active":"2023-11-08T19:25:03.656023+0000","last_peered":"2023-11-08T19:25:03.656023+0000","last_clean":"2023-11-08T19:25:03.656023+0000","last_became_active":"2023-11-08T19:24:57.661425+0000","last_became_peered":"2023-11-08T19:24:57.661425+0000","last_unstale":"2023-11-08T19:25:03.656023+0000","last_undegraded":"2023-11-08T19:25:03.656023+0000","last_fullsized":"2023-11-08T19:25:03.656023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596849+0000","last_change":"2023-11-08T19:24:57.662729+0000","last_active":"2023-11-08T19:25:03.596849+0000","last_peered":"2023-11-08T19:25:03.596849+0000","last_clean":"2023-11-08T19:25:03.596849+0000","last_became_active":"2023-11-08T19:24:57.662390+0000","last_became_peered":"2023-11-08T19:24:57.662390+0000","last_unstale":"2023-11-08T19:25:03.596849+0000","last_undegraded":"2023-11-08T19:25:03.596849+0000","last_fullsized":"2023-11-08T19:25:03.596849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656750+0000","last_change":"2023-11-08T19:24:58.619164+0000","last_active":"2023-11-08T19:25:03.656750+0000","last_peered":"2023-11-08T19:25:03.656750+0000","last_clean":"2023-11-08T19:25:03.656750+0000","last_became_active":"2023-11-08T19:24:58.618604+0000","last_became_peered":"2023-11-08T19:24:58.618604+0000","last_unstale":"2023-11-08T19:25:03.656750+0000","last_undegraded":"2023-11-08T19:25:03.656750+0000","last_fullsized":"2023-11-08T19:25:03.656750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.514753+0000","last_change":"2023-11-08T19:25:02.720151+0000","last_active":"2023-11-08T19:25:05.514753+0000","last_peered":"2023-11-08T19:25:05.514753+0000","last_clean":"2023-11-08T19:25:05.514753+0000","last_became_active":"2023-11-08T19:25:02.719391+0000","last_became_peered":"2023-11-08T19:25:02.719391+0000","last_unstale":"2023-11-08T19:25:05.514753+0000","last_undegraded":"2023-11-08T19:25:05.514753+0000","last_fullsized":"2023-11-08T19:25:05.514753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:23.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657232+0000","last_change":"2023-11-08T19:24:57.660203+0000","last_active":"2023-11-08T19:25:03.657232+0000","last_peered":"2023-11-08T19:25:03.657232+0000","last_clean":"2023-11-08T19:25:03.657232+0000","last_became_active":"2023-11-08T19:24:57.659913+0000","last_became_peered":"2023-11-08T19:24:57.659913+0000","last_unstale":"2023-11-08T19:25:03.657232+0000","last_undegraded":"2023-11-08T19:25:03.657232+0000","last_fullsized":"2023-11-08T19:25:03.657232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657083+0000","last_change":"2023-11-08T19:24:58.620564+0000","last_active":"2023-11-08T19:25:03.657083+0000","last_peered":"2023-11-08T19:25:03.657083+0000","last_clean":"2023-11-08T19:25:03.657083+0000","last_became_active":"2023-11-08T19:24:58.620072+0000","last_became_peered":"2023-11-08T19:24:58.620072+0000","last_unstale":"2023-11-08T19:25:03.657083+0000","last_undegraded":"2023-11-08T19:25:03.657083+0000","last_fullsized":"2023-11-08T19:25:03.657083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:36.879923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595054+0000","last_change":"2023-11-08T19:25:02.729532+0000","last_active":"2023-11-08T19:25:04.595054+0000","last_peered":"2023-11-08T19:25:04.595054+0000","last_clean":"2023-11-08T19:25:04.595054+0000","last_became_active":"2023-11-08T19:25:02.728431+0000","last_became_peered":"2023-11-08T19:25:02.728431+0000","last_unstale":"2023-11-08T19:25:04.595054+0000","last_undegraded":"2023-11-08T19:25:04.595054+0000","last_fullsized":"2023-11-08T19:25:04.595054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:57.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595597+0000","last_change":"2023-11-08T19:24:58.624056+0000","last_active":"2023-11-08T19:25:03.595597+0000","last_peered":"2023-11-08T19:25:03.595597+0000","last_clean":"2023-11-08T19:25:03.595597+0000","last_became_active":"2023-11-08T19:24:58.623783+0000","last_became_peered":"2023-11-08T19:24:58.623783+0000","last_unstale":"2023-11-08T19:25:03.595597+0000","last_undegraded":"2023-11-08T19:25:03.595597+0000","last_fullsized":"2023-11-08T19:25:03.595597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593245+0000","last_change":"2023-11-08T19:24:58.617678+0000","last_active":"2023-11-08T19:25:03.593245+0000","last_peered":"2023-11-08T19:25:03.593245+0000","last_clean":"2023-11-08T19:25:03.593245+0000","last_became_active":"2023-11-08T19:24:58.616701+0000","last_became_peered":"2023-11-08T19:24:58.616701+0000","last_unstale":"2023-11-08T19:25:03.593245+0000","last_undegraded":"2023-11-08T19:25:03.593245+0000","last_fullsized":"2023-11-08T19:25:03.593245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599324+0000","last_change":"2023-11-08T19:24:58.620961+0000","last_active":"2023-11-08T19:25:03.599324+0000","last_peered":"2023-11-08T19:25:03.599324+0000","last_clean":"2023-11-08T19:25:03.599324+0000","last_became_active":"2023-11-08T19:24:58.620359+0000","last_became_peered":"2023-11-08T19:24:58.620359+0000","last_unstale":"2023-11-08T19:25:03.599324+0000","last_undegraded":"2023-11-08T19:25:03.599324+0000","last_fullsized":"2023-11-08T19:25:03.599324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656545+0000","last_change":"2023-11-08T19:24:58.622475+0000","last_active":"2023-11-08T19:25:03.656545+0000","last_peered":"2023-11-08T19:25:03.656545+0000","last_clean":"2023-11-08T19:25:03.656545+0000","last_became_active":"2023-11-08T19:24:58.622206+0000","last_became_peered":"2023-11-08T19:24:58.622206+0000","last_unstale":"2023-11-08T19:25:03.656545+0000","last_undegraded":"2023-11-08T19:25:03.656545+0000","last_fullsized":"2023-11-08T19:25:03.656545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657638+0000","last_change":"2023-11-08T19:24:58.630303+0000","last_active":"2023-11-08T19:25:03.657638+0000","last_peered":"2023-11-08T19:25:03.657638+0000","last_clean":"2023-11-08T19:25:03.657638+0000","last_became_active":"2023-11-08T19:24:58.629181+0000","last_became_peered":"2023-11-08T19:24:58.629181+0000","last_unstale":"2023-11-08T19:25:03.657638+0000","last_undegraded":"2023-11-08T19:25:03.657638+0000","last_fullsized":"2023-11-08T19:25:03.657638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643449+0000","last_change":"2023-11-08T19:24:58.622627+0000","last_active":"2023-11-08T19:25:03.643449+0000","last_peered":"2023-11-08T19:25:03.643449+0000","last_clean":"2023-11-08T19:25:03.643449+0000","last_became_active":"2023-11-08T19:24:58.622335+0000","last_became_peered":"2023-11-08T19:24:58.622335+0000","last_unstale":"2023-11-08T19:25:03.643449+0000","last_undegraded":"2023-11-08T19:25:03.643449+0000","last_fullsized":"2023-11-08T19:25:03.643449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597541+0000","last_change":"2023-11-08T19:24:58.630440+0000","last_active":"2023-11-08T19:25:03.597541+0000","last_peered":"2023-11-08T19:25:03.597541+0000","last_clean":"2023-11-08T19:25:03.597541+0000","last_became_active":"2023-11-08T19:24:58.629904+0000","last_became_peered":"2023-11-08T19:24:58.629904+0000","last_unstale":"2023-11-08T19:25:03.597541+0000","last_undegraded":"2023-11-08T19:25:03.597541+0000","last_fullsized":"2023-11-08T19:25:03.597541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.627568+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.627243+0000","last_became_peered":"2023-11-08T19:24:58.627243+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595899+0000","last_change":"2023-11-08T19:24:58.622232+0000","last_active":"2023-11-08T19:25:03.595899+0000","last_peered":"2023-11-08T19:25:03.595899+0000","last_clean":"2023-11-08T19:25:03.595899+0000","last_became_active":"2023-11-08T19:24:58.621853+0000","last_became_peered":"2023-11-08T19:24:58.621853+0000","last_unstale":"2023-11-08T19:25:03.595899+0000","last_undegraded":"2023-11-08T19:25:03.595899+0000","last_fullsized":"2023-11-08T19:25:03.595899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654133+0000","last_change":"2023-11-08T19:24:58.630530+0000","last_active":"2023-11-08T19:25:03.654133+0000","last_peered":"2023-11-08T19:25:03.654133+0000","last_clean":"2023-11-08T19:25:03.654133+0000","last_became_active":"2023-11-08T19:24:58.627752+0000","last_became_peered":"2023-11-08T19:24:58.627752+0000","last_unstale":"2023-11-08T19:25:03.654133+0000","last_undegraded":"2023-11-08T19:25:03.654133+0000","last_fullsized":"2023-11-08T19:25:03.654133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599081+0000","last_change":"2023-11-08T19:24:58.626422+0000","last_active":"2023-11-08T19:25:03.599081+0000","last_peered":"2023-11-08T19:25:03.599081+0000","last_clean":"2023-11-08T19:25:03.599081+0000","last_became_active":"2023-11-08T19:24:58.626091+0000","last_became_peered":"2023-11-08T19:24:58.626091+0000","last_unstale":"2023-11-08T19:25:03.599081+0000","last_undegraded":"2023-11-08T19:25:03.599081+0000","last_fullsized":"2023-11-08T19:25:03.599081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656191+0000","last_change":"2023-11-08T19:24:58.630415+0000","last_active":"2023-11-08T19:25:03.656191+0000","last_peered":"2023-11-08T19:25:03.656191+0000","last_clean":"2023-11-08T19:25:03.656191+0000","last_became_active":"2023-11-08T19:24:58.628526+0000","last_became_peered":"2023-11-08T19:24:58.628526+0000","last_unstale":"2023-11-08T19:25:03.656191+0000","last_undegraded":"2023-11-08T19:25:03.656191+0000","last_fullsized":"2023-11-08T19:25:03.656191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596558+0000","last_change":"2023-11-08T19:24:58.628286+0000","last_active":"2023-11-08T19:25:03.596558+0000","last_peered":"2023-11-08T19:25:03.596558+0000","last_clean":"2023-11-08T19:25:03.596558+0000","last_became_active":"2023-11-08T19:24:58.627912+0000","last_became_peered":"2023-11-08T19:24:58.627912+0000","last_unstale":"2023-11-08T19:25:03.596558+0000","last_undegraded":"2023-11-08T19:25:03.596558+0000","last_fullsized":"2023-11-08T19:25:03.596558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656975+0000","last_change":"2023-11-08T19:24:58.633630+0000","last_active":"2023-11-08T19:25:03.656975+0000","last_peered":"2023-11-08T19:25:03.656975+0000","last_clean":"2023-11-08T19:25:03.656975+0000","last_became_active":"2023-11-08T19:24:58.632441+0000","last_became_peered":"2023-11-08T19:24:58.632441+0000","last_unstale":"2023-11-08T19:25:03.656975+0000","last_undegraded":"2023-11-08T19:25:03.656975+0000","last_fullsized":"2023-11-08T19:25:03.656975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593821+0000","last_change":"2023-11-08T19:24:58.623578+0000","last_active":"2023-11-08T19:25:03.593821+0000","last_peered":"2023-11-08T19:25:03.593821+0000","last_clean":"2023-11-08T19:25:03.593821+0000","last_became_active":"2023-11-08T19:24:58.623261+0000","last_became_peered":"2023-11-08T19:24:58.623261+0000","last_unstale":"2023-11-08T19:25:03.593821+0000","last_undegraded":"2023-11-08T19:25:03.593821+0000","last_fullsized":"2023-11-08T19:25:03.593821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:36.969600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.601266+0000","last_change":"2023-11-08T19:24:58.634259+0000","last_active":"2023-11-08T19:25:03.601266+0000","last_peered":"2023-11-08T19:25:03.601266+0000","last_clean":"2023-11-08T19:25:03.601266+0000","last_became_active":"2023-11-08T19:24:58.633708+0000","last_became_peered":"2023-11-08T19:24:58.633708+0000","last_unstale":"2023-11-08T19:25:03.601266+0000","last_undegraded":"2023-11-08T19:25:03.601266+0000","last_fullsized":"2023-11-08T19:25:03.601266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595680+0000","last_change":"2023-11-08T19:24:58.615871+0000","last_active":"2023-11-08T19:25:03.595680+0000","last_peered":"2023-11-08T19:25:03.595680+0000","last_clean":"2023-11-08T19:25:03.595680+0000","last_became_active":"2023-11-08T19:24:58.615590+0000","last_became_peered":"2023-11-08T19:24:58.615590+0000","last_unstale":"2023-11-08T19:25:03.595680+0000","last_undegraded":"2023-11-08T19:25:03.595680+0000","last_fullsized":"2023-11-08T19:25:03.595680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642579+0000","last_change":"2023-11-08T19:24:58.629058+0000","last_active":"2023-11-08T19:25:03.642579+0000","last_peered":"2023-11-08T19:25:03.642579+0000","last_clean":"2023-11-08T19:25:03.642579+0000","last_became_active":"2023-11-08T19:24:58.628656+0000","last_became_peered":"2023-11-08T19:24:58.628656+0000","last_unstale":"2023-11-08T19:25:03.642579+0000","last_undegraded":"2023-11-08T19:25:03.642579+0000","last_fullsized":"2023-11-08T19:25:03.642579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655552+0000","last_change":"2023-11-08T19:24:58.622286+0000","last_active":"2023-11-08T19:25:03.655552+0000","last_peered":"2023-11-08T19:25:03.655552+0000","last_clean":"2023-11-08T19:25:03.655552+0000","last_became_active":"2023-11-08T19:24:58.621993+0000","last_became_peered":"2023-11-08T19:24:58.621993+0000","last_unstale":"2023-11-08T19:25:03.655552+0000","last_undegraded":"2023-11-08T19:25:03.655552+0000","last_fullsized":"2023-11-08T19:25:03.655552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656958+0000","last_change":"2023-11-08T19:24:58.618762+0000","last_active":"2023-11-08T19:25:03.656958+0000","last_peered":"2023-11-08T19:25:03.656958+0000","last_clean":"2023-11-08T19:25:03.656958+0000","last_became_active":"2023-11-08T19:24:58.618468+0000","last_became_peered":"2023-11-08T19:24:58.618468+0000","last_unstale":"2023-11-08T19:25:03.656958+0000","last_undegraded":"2023-11-08T19:25:03.656958+0000","last_fullsized":"2023-11-08T19:25:03.656958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655770+0000","last_change":"2023-11-08T19:24:58.626193+0000","last_active":"2023-11-08T19:25:03.655770+0000","last_peered":"2023-11-08T19:25:03.655770+0000","last_clean":"2023-11-08T19:25:03.655770+0000","last_became_active":"2023-11-08T19:24:58.625897+0000","last_became_peered":"2023-11-08T19:24:58.625897+0000","last_unstale":"2023-11-08T19:25:03.655770+0000","last_undegraded":"2023-11-08T19:25:03.655770+0000","last_fullsized":"2023-11-08T19:25:03.655770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:43.279737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596855+0000","last_change":"2023-11-08T19:24:58.621575+0000","last_active":"2023-11-08T19:25:03.596855+0000","last_peered":"2023-11-08T19:25:03.596855+0000","last_clean":"2023-11-08T19:25:03.596855+0000","last_became_active":"2023-11-08T19:24:58.621278+0000","last_became_peered":"2023-11-08T19:24:58.621278+0000","last_unstale":"2023-11-08T19:25:03.596855+0000","last_undegraded":"2023-11-08T19:25:03.596855+0000","last_fullsized":"2023-11-08T19:25:03.596855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597421+0000","last_change":"2023-11-08T19:24:58.630221+0000","last_active":"2023-11-08T19:25:03.597421+0000","last_peered":"2023-11-08T19:25:03.597421+0000","last_clean":"2023-11-08T19:25:03.597421+0000","last_became_active":"2023-11-08T19:24:58.629514+0000","last_became_peered":"2023-11-08T19:24:58.629514+0000","last_unstale":"2023-11-08T19:25:03.597421+0000","last_undegraded":"2023-11-08T19:25:03.597421+0000","last_fullsized":"2023-11-08T19:25:03.597421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656658+0000","last_change":"2023-11-08T19:24:58.637307+0000","last_active":"2023-11-08T19:25:03.656658+0000","last_peered":"2023-11-08T19:25:03.656658+0000","last_clean":"2023-11-08T19:25:03.656658+0000","last_became_active":"2023-11-08T19:24:58.636926+0000","last_became_peered":"2023-11-08T19:24:58.636926+0000","last_unstale":"2023-11-08T19:25:03.656658+0000","last_undegraded":"2023-11-08T19:25:03.656658+0000","last_fullsized":"2023-11-08T19:25:03.656658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:52.742252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654271+0000","last_change":"2023-11-08T19:24:58.619975+0000","last_active":"2023-11-08T19:25:03.654271+0000","last_peered":"2023-11-08T19:25:03.654271+0000","last_clean":"2023-11-08T19:25:03.654271+0000","last_became_active":"2023-11-08T19:24:58.619676+0000","last_became_peered":"2023-11-08T19:24:58.619676+0000","last_unstale":"2023-11-08T19:25:03.654271+0000","last_undegraded":"2023-11-08T19:25:03.654271+0000","last_fullsized":"2023-11-08T19:25:03.654271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654365+0000","last_change":"2023-11-08T19:24:58.627535+0000","last_active":"2023-11-08T19:25:03.654365+0000","last_peered":"2023-11-08T19:25:03.654365+0000","last_clean":"2023-11-08T19:25:03.654365+0000","last_became_active":"2023-11-08T19:24:58.627205+0000","last_became_peered":"2023-11-08T19:24:58.627205+0000","last_unstale":"2023-11-08T19:25:03.654365+0000","last_undegraded":"2023-11-08T19:25:03.654365+0000","last_fullsized":"2023-11-08T19:25:03.654365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599068+0000","last_change":"2023-11-08T19:24:58.620637+0000","last_active":"2023-11-08T19:25:03.599068+0000","last_peered":"2023-11-08T19:25:03.599068+0000","last_clean":"2023-11-08T19:25:03.599068+0000","last_became_active":"2023-11-08T19:24:58.619919+0000","last_became_peered":"2023-11-08T19:24:58.619919+0000","last_unstale":"2023-11-08T19:25:03.599068+0000","last_undegraded":"2023-11-08T19:25:03.599068+0000","last_fullsized":"2023-11-08T19:25:03.599068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655858+0000","last_change":"2023-11-08T19:24:58.630887+0000","last_active":"2023-11-08T19:25:03.655858+0000","last_peered":"2023-11-08T19:25:03.655858+0000","last_clean":"2023-11-08T19:25:03.655858+0000","last_became_active":"2023-11-08T19:24:58.630010+0000","last_became_peered":"2023-11-08T19:24:58.630010+0000","last_unstale":"2023-11-08T19:25:03.655858+0000","last_undegraded":"2023-11-08T19:25:03.655858+0000","last_fullsized":"2023-11-08T19:25:03.655858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599939+0000","last_change":"2023-11-08T19:24:58.625672+0000","last_active":"2023-11-08T19:25:03.599939+0000","last_peered":"2023-11-08T19:25:03.599939+0000","last_clean":"2023-11-08T19:25:03.599939+0000","last_became_active":"2023-11-08T19:24:58.619366+0000","last_became_peered":"2023-11-08T19:24:58.619366+0000","last_unstale":"2023-11-08T19:25:03.599939+0000","last_undegraded":"2023-11-08T19:25:03.599939+0000","last_fullsized":"2023-11-08T19:25:03.599939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655793+0000","last_change":"2023-11-08T19:24:58.612582+0000","last_active":"2023-11-08T19:25:03.655793+0000","last_peered":"2023-11-08T19:25:03.655793+0000","last_clean":"2023-11-08T19:25:03.655793+0000","last_became_active":"2023-11-08T19:24:58.612199+0000","last_became_peered":"2023-11-08T19:24:58.612199+0000","last_unstale":"2023-11-08T19:25:03.655793+0000","last_undegraded":"2023-11-08T19:25:03.655793+0000","last_fullsized":"2023-11-08T19:25:03.655793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643845+0000","last_change":"2023-11-08T19:24:58.616793+0000","last_active":"2023-11-08T19:25:03.643845+0000","last_peered":"2023-11-08T19:25:03.643845+0000","last_clean":"2023-11-08T19:25:03.643845+0000","last_became_active":"2023-11-08T19:24:58.616483+0000","last_became_peered":"2023-11-08T19:24:58.616483+0000","last_unstale":"2023-11-08T19:25:03.643845+0000","last_undegraded":"2023-11-08T19:25:03.643845+0000","last_fullsized":"2023-11-08T19:25:03.643845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656535+0000","last_change":"2023-11-08T19:24:58.640055+0000","last_active":"2023-11-08T19:25:03.656535+0000","last_peered":"2023-11-08T19:25:03.656535+0000","last_clean":"2023-11-08T19:25:03.656535+0000","last_became_active":"2023-11-08T19:24:58.639464+0000","last_became_peered":"2023-11-08T19:24:58.639464+0000","last_unstale":"2023-11-08T19:25:03.656535+0000","last_undegraded":"2023-11-08T19:25:03.656535+0000","last_fullsized":"2023-11-08T19:25:03.656535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:51.551899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":697807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3170000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0110000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.427}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7310000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":105838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:22.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.441+0000 7fdaedb0a640 1 Processor -- start 2023-11-08T19:25:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.442+0000 7fdaedb0a640 1 -- start start 2023-11-08T19:25:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.443+0000 7fdaedb0a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdae80adee0 0x7fdae815d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.443+0000 7fdaedb0a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae806b780 con 0x7fdae806bc10 2023-11-08T19:25:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.443+0000 7fdaedb0a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae815dd30 con 0x7fdae80ac8d0 2023-11-08T19:25:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.443+0000 7fdaedb0a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae815e010 con 0x7fdae80adee0 2023-11-08T19:25:22.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdaed309640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdae806bc10 0x7fdae806c0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59974/0 (socket says 172.21.15.119:59974) 2023-11-08T19:25:22.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdaed309640 1 -- 172.21.15.119:0/85929751 learned_addr learned my addr 172.21.15.119:0/85929751 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae7fff640 1 --2- 172.21.15.119:0/85929751 >> v2:172.21.15.188:3300/0 conn(0x7fdae80adee0 0x7fdae815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3588264371 0 0) 0x7fdae806b780 con 0x7fdae806bc10 2023-11-08T19:25:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdad8003520 con 0x7fdae806bc10 2023-11-08T19:25:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504435928 0 0) 0x7fdad8003520 con 0x7fdae806bc10 2023-11-08T19:25:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 >> v2:172.21.15.188:3300/0 conn(0x7fdae80adee0 msgr2=0x7fdae815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae77fe640 1 --2- 172.21.15.119:0/85929751 >> v2:172.21.15.188:3300/0 conn(0x7fdae80adee0 0x7fdae815d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.444+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 >> v1:172.21.15.188:6789/0 conn(0x7fdae80ac8d0 legacy=0x7fdae80accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae815e2f0 con 0x7fdae806bc10 2023-11-08T19:25:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fdae00019f0 con 0x7fdae806bc10 2023-11-08T19:25:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 >> v1:172.21.15.119:6789/0 conn(0x7fdae806bc10 legacy=0x7fdae806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad80039a0 0x7fdad8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad80047c0 0x7fdad8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad8005180 0x7fdad800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae815e010 con 0x7fdad8005180 2023-11-08T19:25:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad800c050 con 0x7fdad80039a0 2023-11-08T19:25:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad800c210 con 0x7fdad80047c0 2023-11-08T19:25:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaed309640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad8005180 0x7fdad800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaecb08640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad80039a0 0x7fdad8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaed309640 1 -- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad80047c0 msgr2=0x7fdad8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaed309640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad80047c0 0x7fdad8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaed309640 1 -- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad80039a0 msgr2=0x7fdad8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaed309640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad80039a0 0x7fdad8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.445+0000 7fdaed309640 1 -- 172.21.15.119:0/85929751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad8003520 con 0x7fdad8005180 2023-11-08T19:25:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.446+0000 7fdaed309640 1 -- 172.21.15.119:0/85929751 reap_dead start 2023-11-08T19:25:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.446+0000 7fdaed309640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad8005180 0x7fdad800b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdae000a830 tx=0x7fdad80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce5767585f426e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00072d0 con 0x7fdad8005180 2023-11-08T19:25:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae002f070 con 0x7fdad8005180 2023-11-08T19:25:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdae77fe640 1 -- 172.21.15.119:0/85929751 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00076b0 con 0x7fdad8005180 2023-11-08T19:25:22.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdaedb0a640 1 -- 172.21.15.119:0/85929751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad8005180 msgr2=0x7fdad800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad8005180 0x7fdad800b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdae000a830 tx=0x7fdad80057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdaedb0a640 1 -- 172.21.15.119:0/85929751 shutdown_connections 2023-11-08T19:25:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/85929751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdad8005180 0x7fdad800b980 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.447+0000 7fdaedb0a640 1 -- 172.21.15.119:0/85929751 >> 172.21.15.119:0/85929751 conn(0x7fdae80b0560 msgr2=0x7fdae80b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.448+0000 7fdaedb0a640 1 -- 172.21.15.119:0/85929751 shutdown_connections 2023-11-08T19:25:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.448+0000 7fdaedb0a640 1 -- 172.21.15.119:0/85929751 wait complete. 2023-11-08T19:25:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.449+0000 7fdaedb0a640 1 Processor -- start 2023-11-08T19:25:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.449+0000 7fdaedb0a640 1 -- start start 2023-11-08T19:25:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.450+0000 7fdaedb0a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad8005180 0x7fdae8156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.450+0000 7fdaedb0a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdae806ff30 0x7fdae80703b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.450+0000 7fdaedb0a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 0x7fdae80c4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.450+0000 7fdaedb0a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae80708f0 con 0x7fdae80c3ed0 2023-11-08T19:25:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.450+0000 7fdaedb0a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae815f830 con 0x7fdad8005180 2023-11-08T19:25:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.450+0000 7fdaedb0a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae815fad0 con 0x7fdae806ff30 2023-11-08T19:25:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 0x7fdae80c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 0x7fdae80c4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50968/0 (socket says 172.21.15.119:50968) 2023-11-08T19:25:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 -- 172.21.15.119:0/789468231 learned_addr learned my addr 172.21.15.119:0/789468231 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaecb08640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad8005180 0x7fdae8156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdae7fff640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdae806ff30 0x7fdae80703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 -- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdae806ff30 msgr2=0x7fdae80703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdae806ff30 0x7fdae80703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 -- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad8005180 msgr2=0x7fdae8156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad8005180 0x7fdae8156f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.451+0000 7fdaed309640 1 -- 172.21.15.119:0/789468231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae80cd4b0 con 0x7fdae80c3ed0 2023-11-08T19:25:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.452+0000 7fdaed309640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 0x7fdae80c4380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f80 tx=0x7fdae0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.452+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00019f0 con 0x7fdae80c3ed0 2023-11-08T19:25:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.452+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdae0004b60 con 0x7fdae80c3ed0 2023-11-08T19:25:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.452+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0007400 con 0x7fdae80c3ed0 2023-11-08T19:25:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.452+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae815fdb0 con 0x7fdae80c3ed0 2023-11-08T19:25:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.452+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae8160370 con 0x7fdae80c3ed0 2023-11-08T19:25:22.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.453+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae8160560 con 0x7fdae80c3ed0 2023-11-08T19:25:22.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.454+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdae0031030 con 0x7fdae80c3ed0 2023-11-08T19:25:22.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.455+0000 7fdae57fa640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdacc06ae70 0x7fdacc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.455+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fdae00ab670 con 0x7fdae80c3ed0 2023-11-08T19:25:22.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.455+0000 7fdaecb08640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdacc06ae70 0x7fdacc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.456+0000 7fdaecb08640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdacc06ae70 0x7fdacc06d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdae80b3b60 tx=0x7fdadc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.459+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdae0076390 con 0x7fdae80c3ed0 2023-11-08T19:25:22.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.651+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdae815e5d0 con 0x7fdacc06ae70 2023-11-08T19:25:22.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.668+0000 7fdae57fa640 1 -- 172.21.15.119:0/789468231 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345137 (secure 0 0 0) 0x7fdadc001350 con 0x7fdacc06ae70 2023-11-08T19:25:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:22.675 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:25:22.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdacc06ae70 msgr2=0x7fdacc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdacc06ae70 0x7fdacc06d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fdae80b3b60 tx=0x7fdadc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:22.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 msgr2=0x7fdae80c4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 0x7fdae80c4380 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdae0000f80 tx=0x7fdae0001d80 comp rx=0 tx=0).stop 2023-11-08T19:25:22.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 shutdown_connections 2023-11-08T19:25:22.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdacc06ae70 0x7fdacc06d330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdae80c3ed0 0x7fdae80c4380 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdae806ff30 0x7fdae80703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 --2- 172.21.15.119:0/789468231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad8005180 0x7fdae8156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.673+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 >> 172.21.15.119:0/789468231 conn(0x7fdae80b0560 msgr2=0x7fdae80cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.674+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 shutdown_connections 2023-11-08T19:25:22.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.674+0000 7fdaedb0a640 1 -- 172.21.15.119:0/789468231 wait complete. 2023-11-08T19:25:22.693 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2023-11-08T19:25:20.653953+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594866,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3375104,"data_stored":2042550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2088,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004125"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655958+0000","last_change":"2023-11-08T19:24:57.661383+0000","last_active":"2023-11-08T19:25:03.655958+0000","last_peered":"2023-11-08T19:25:03.655958+0000","last_clean":"2023-11-08T19:25:03.655958+0000","last_became_active":"2023-11-08T19:24:57.661122+0000","last_became_peered":"2023-11-08T19:24:57.661122+0000","last_unstale":"2023-11-08T19:25:03.655958+0000","last_undegraded":"2023-11-08T19:25:03.655958+0000","last_fullsized":"2023-11-08T19:25:03.655958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657959+0000","last_change":"2023-11-08T19:24:58.622972+0000","last_active":"2023-11-08T19:25:03.657959+0000","last_peered":"2023-11-08T19:25:03.657959+0000","last_clean":"2023-11-08T19:25:03.657959+0000","last_became_active":"2023-11-08T19:24:58.620907+0000","last_became_peered":"2023-11-08T19:24:58.620907+0000","last_unstale":"2023-11-08T19:25:03.657959+0000","last_undegraded":"2023-11-08T19:25:03.657959+0000","last_fullsized":"2023-11-08T19:25:03.657959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593888+0000","last_change":"2023-11-08T19:25:03.480700+0000","last_active":"2023-11-08T19:25:04.593888+0000","last_peered":"2023-11-08T19:25:04.593888+0000","last_clean":"2023-11-08T19:25:04.593888+0000","last_became_active":"2023-11-08T19:25:03.477779+0000","last_became_peered":"2023-11-08T19:25:03.477779+0000","last_unstale":"2023-11-08T19:25:04.593888+0000","last_undegraded":"2023-11-08T19:25:04.593888+0000","last_fullsized":"2023-11-08T19:25:04.593888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:47:24.018242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658227+0000","last_change":"2023-11-08T19:24:57.655981+0000","last_active":"2023-11-08T19:25:03.658227+0000","last_peered":"2023-11-08T19:25:03.658227+0000","last_clean":"2023-11-08T19:25:03.658227+0000","last_became_active":"2023-11-08T19:24:57.655420+0000","last_became_peered":"2023-11-08T19:24:57.655420+0000","last_unstale":"2023-11-08T19:25:03.658227+0000","last_undegraded":"2023-11-08T19:25:03.658227+0000","last_fullsized":"2023-11-08T19:25:03.658227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655637+0000","last_change":"2023-11-08T19:24:58.620479+0000","last_active":"2023-11-08T19:25:03.655637+0000","last_peered":"2023-11-08T19:25:03.655637+0000","last_clean":"2023-11-08T19:25:03.655637+0000","last_became_active":"2023-11-08T19:24:58.620201+0000","last_became_peered":"2023-11-08T19:24:58.620201+0000","last_unstale":"2023-11-08T19:25:03.655637+0000","last_undegraded":"2023-11-08T19:25:03.655637+0000","last_fullsized":"2023-11-08T19:25:03.655637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.598798+0000","last_change":"2023-11-08T19:25:02.722736+0000","last_active":"2023-11-08T19:25:04.598798+0000","last_peered":"2023-11-08T19:25:04.598798+0000","last_clean":"2023-11-08T19:25:04.598798+0000","last_became_active":"2023-11-08T19:25:02.722087+0000","last_became_peered":"2023-11-08T19:25:02.722087+0000","last_unstale":"2023-11-08T19:25:04.598798+0000","last_undegraded":"2023-11-08T19:25:04.598798+0000","last_fullsized":"2023-11-08T19:25:04.598798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656740+0000","last_change":"2023-11-08T19:24:57.670689+0000","last_active":"2023-11-08T19:25:03.656740+0000","last_peered":"2023-11-08T19:25:03.656740+0000","last_clean":"2023-11-08T19:25:03.656740+0000","last_became_active":"2023-11-08T19:24:57.670406+0000","last_became_peered":"2023-11-08T19:24:57.670406+0000","last_unstale":"2023-11-08T19:25:03.656740+0000","last_undegraded":"2023-11-08T19:25:03.656740+0000","last_fullsized":"2023-11-08T19:25:03.656740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600045+0000","last_change":"2023-11-08T19:24:58.629107+0000","last_active":"2023-11-08T19:25:03.600045+0000","last_peered":"2023-11-08T19:25:03.600045+0000","last_clean":"2023-11-08T19:25:03.600045+0000","last_became_active":"2023-11-08T19:24:58.628670+0000","last_became_peered":"2023-11-08T19:24:58.628670+0000","last_unstale":"2023-11-08T19:25:03.600045+0000","last_undegraded":"2023-11-08T19:25:03.600045+0000","last_fullsized":"2023-11-08T19:25:03.600045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601324+0000","last_change":"2023-11-08T19:25:02.727558+0000","last_active":"2023-11-08T19:25:04.601324+0000","last_peered":"2023-11-08T19:25:04.601324+0000","last_clean":"2023-11-08T19:25:04.601324+0000","last_became_active":"2023-11-08T19:25:02.726979+0000","last_became_peered":"2023-11-08T19:25:02.726979+0000","last_unstale":"2023-11-08T19:25:04.601324+0000","last_undegraded":"2023-11-08T19:25:04.601324+0000","last_fullsized":"2023-11-08T19:25:04.601324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655522+0000","last_change":"2023-11-08T19:24:57.673750+0000","last_active":"2023-11-08T19:25:03.655522+0000","last_peered":"2023-11-08T19:25:03.655522+0000","last_clean":"2023-11-08T19:25:03.655522+0000","last_became_active":"2023-11-08T19:24:57.673328+0000","last_became_peered":"2023-11-08T19:24:57.673328+0000","last_unstale":"2023-11-08T19:25:03.655522+0000","last_undegraded":"2023-11-08T19:25:03.655522+0000","last_fullsized":"2023-11-08T19:25:03.655522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600770+0000","last_change":"2023-11-08T19:24:58.628265+0000","last_active":"2023-11-08T19:25:03.600770+0000","last_peered":"2023-11-08T19:25:03.600770+0000","last_clean":"2023-11-08T19:25:03.600770+0000","last_became_active":"2023-11-08T19:24:58.628007+0000","last_became_peered":"2023-11-08T19:24:58.628007+0000","last_unstale":"2023-11-08T19:25:03.600770+0000","last_undegraded":"2023-11-08T19:25:03.600770+0000","last_fullsized":"2023-11-08T19:25:03.600770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601053+0000","last_change":"2023-11-08T19:25:02.726316+0000","last_active":"2023-11-08T19:25:04.601053+0000","last_peered":"2023-11-08T19:25:04.601053+0000","last_clean":"2023-11-08T19:25:04.601053+0000","last_became_active":"2023-11-08T19:25:02.725661+0000","last_became_peered":"2023-11-08T19:25:02.725661+0000","last_unstale":"2023-11-08T19:25:04.601053+0000","last_undegraded":"2023-11-08T19:25:04.601053+0000","last_fullsized":"2023-11-08T19:25:04.601053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:56.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596618+0000","last_change":"2023-11-08T19:24:58.626589+0000","last_active":"2023-11-08T19:25:03.596618+0000","last_peered":"2023-11-08T19:25:03.596618+0000","last_clean":"2023-11-08T19:25:03.596618+0000","last_became_active":"2023-11-08T19:24:58.626269+0000","last_became_peered":"2023-11-08T19:24:58.626269+0000","last_unstale":"2023-11-08T19:25:03.596618+0000","last_undegraded":"2023-11-08T19:25:03.596618+0000","last_fullsized":"2023-11-08T19:25:03.596618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:57.668738+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:57.668319+0000","last_became_peered":"2023-11-08T19:24:57.668319+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.603469+0000","last_change":"2023-11-08T19:25:02.731537+0000","last_active":"2023-11-08T19:25:04.603469+0000","last_peered":"2023-11-08T19:25:04.603469+0000","last_clean":"2023-11-08T19:25:04.603469+0000","last_became_active":"2023-11-08T19:25:02.731004+0000","last_became_peered":"2023-11-08T19:25:02.731004+0000","last_unstale":"2023-11-08T19:25:04.603469+0000","last_undegraded":"2023-11-08T19:25:04.603469+0000","last_fullsized":"2023-11-08T19:25:04.603469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:58.628177+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:58.627584+0000","last_became_peered":"2023-11-08T19:24:58.627584+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643213+0000","last_change":"2023-11-08T19:24:57.659565+0000","last_active":"2023-11-08T19:25:03.643213+0000","last_peered":"2023-11-08T19:25:03.643213+0000","last_clean":"2023-11-08T19:25:03.643213+0000","last_became_active":"2023-11-08T19:24:57.658443+0000","last_became_peered":"2023-11-08T19:24:57.658443+0000","last_unstale":"2023-11-08T19:25:03.643213+0000","last_undegraded":"2023-11-08T19:25:03.643213+0000","last_fullsized":"2023-11-08T19:25:03.643213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602536+0000","last_change":"2023-11-08T19:25:02.734167+0000","last_active":"2023-11-08T19:25:04.602536+0000","last_peered":"2023-11-08T19:25:04.602536+0000","last_clean":"2023-11-08T19:25:04.602536+0000","last_became_active":"2023-11-08T19:25:02.733732+0000","last_became_peered":"2023-11-08T19:25:02.733732+0000","last_unstale":"2023-11-08T19:25:04.602536+0000","last_undegraded":"2023-11-08T19:25:04.602536+0000","last_fullsized":"2023-11-08T19:25:04.602536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655085+0000","last_change":"2023-11-08T19:24:58.627099+0000","last_active":"2023-11-08T19:25:03.655085+0000","last_peered":"2023-11-08T19:25:03.655085+0000","last_clean":"2023-11-08T19:25:03.655085+0000","last_became_active":"2023-11-08T19:24:58.626261+0000","last_became_peered":"2023-11-08T19:24:58.626261+0000","last_unstale":"2023-11-08T19:25:03.655085+0000","last_undegraded":"2023-11-08T19:25:03.655085+0000","last_fullsized":"2023-11-08T19:25:03.655085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642570+0000","last_change":"2023-11-08T19:24:57.658778+0000","last_active":"2023-11-08T19:25:03.642570+0000","last_peered":"2023-11-08T19:25:03.642570+0000","last_clean":"2023-11-08T19:25:03.642570+0000","last_became_active":"2023-11-08T19:24:57.657370+0000","last_became_peered":"2023-11-08T19:24:57.657370+0000","last_unstale":"2023-11-08T19:25:03.642570+0000","last_undegraded":"2023-11-08T19:25:03.642570+0000","last_fullsized":"2023-11-08T19:25:03.642570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422591+0000","last_change":"2023-11-08T19:25:03.478005+0000","last_active":"2023-11-08T19:25:05.422591+0000","last_peered":"2023-11-08T19:25:05.422591+0000","last_clean":"2023-11-08T19:25:05.422591+0000","last_became_active":"2023-11-08T19:25:03.476666+0000","last_became_peered":"2023-11-08T19:25:03.476666+0000","last_unstale":"2023-11-08T19:25:05.422591+0000","last_undegraded":"2023-11-08T19:25:05.422591+0000","last_fullsized":"2023-11-08T19:25:05.422591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656312+0000","last_change":"2023-11-08T19:24:58.623061+0000","last_active":"2023-11-08T19:25:03.656312+0000","last_peered":"2023-11-08T19:25:03.656312+0000","last_clean":"2023-11-08T19:25:03.656312+0000","last_became_active":"2023-11-08T19:24:58.618367+0000","last_became_peered":"2023-11-08T19:24:58.618367+0000","last_unstale":"2023-11-08T19:25:03.656312+0000","last_undegraded":"2023-11-08T19:25:03.656312+0000","last_fullsized":"2023-11-08T19:25:03.656312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:08.538170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.653991+0000","last_change":"2023-11-08T19:24:57.666853+0000","last_active":"2023-11-08T19:25:03.653991+0000","last_peered":"2023-11-08T19:25:03.653991+0000","last_clean":"2023-11-08T19:25:03.653991+0000","last_became_active":"2023-11-08T19:24:57.666539+0000","last_became_peered":"2023-11-08T19:24:57.666539+0000","last_unstale":"2023-11-08T19:25:03.653991+0000","last_undegraded":"2023-11-08T19:25:03.653991+0000","last_fullsized":"2023-11-08T19:25:03.653991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602888+0000","last_change":"2023-11-08T19:25:02.727013+0000","last_active":"2023-11-08T19:25:04.602888+0000","last_peered":"2023-11-08T19:25:04.602888+0000","last_clean":"2023-11-08T19:25:04.602888+0000","last_became_active":"2023-11-08T19:25:02.726496+0000","last_became_peered":"2023-11-08T19:25:02.726496+0000","last_unstale":"2023-11-08T19:25:04.602888+0000","last_undegraded":"2023-11-08T19:25:04.602888+0000","last_fullsized":"2023-11-08T19:25:04.602888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655207+0000","last_change":"2023-11-08T19:24:58.620701+0000","last_active":"2023-11-08T19:25:03.655207+0000","last_peered":"2023-11-08T19:25:03.655207+0000","last_clean":"2023-11-08T19:25:03.655207+0000","last_became_active":"2023-11-08T19:24:58.620386+0000","last_became_peered":"2023-11-08T19:24:58.620386+0000","last_unstale":"2023-11-08T19:25:03.655207+0000","last_undegraded":"2023-11-08T19:25:03.655207+0000","last_fullsized":"2023-11-08T19:25:03.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597904+0000","last_change":"2023-11-08T19:24:57.667152+0000","last_active":"2023-11-08T19:25:03.597904+0000","last_peered":"2023-11-08T19:25:03.597904+0000","last_clean":"2023-11-08T19:25:03.597904+0000","last_became_active":"2023-11-08T19:24:57.666522+0000","last_became_peered":"2023-11-08T19:24:57.666522+0000","last_unstale":"2023-11-08T19:25:03.597904+0000","last_undegraded":"2023-11-08T19:25:03.597904+0000","last_fullsized":"2023-11-08T19:25:03.597904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:33.630876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599610+0000","last_change":"2023-11-08T19:25:03.469823+0000","last_active":"2023-11-08T19:25:04.599610+0000","last_peered":"2023-11-08T19:25:04.599610+0000","last_clean":"2023-11-08T19:25:04.599610+0000","last_became_active":"2023-11-08T19:25:03.468656+0000","last_became_peered":"2023-11-08T19:25:03.468656+0000","last_unstale":"2023-11-08T19:25:04.599610+0000","last_undegraded":"2023-11-08T19:25:04.599610+0000","last_fullsized":"2023-11-08T19:25:04.599610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.633451+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.633159+0000","last_became_peered":"2023-11-08T19:24:58.633159+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.644112+0000","last_change":"2023-11-08T19:24:57.663788+0000","last_active":"2023-11-08T19:25:03.644112+0000","last_peered":"2023-11-08T19:25:03.644112+0000","last_clean":"2023-11-08T19:25:03.644112+0000","last_became_active":"2023-11-08T19:24:57.663457+0000","last_became_peered":"2023-11-08T19:24:57.663457+0000","last_unstale":"2023-11-08T19:25:03.644112+0000","last_undegraded":"2023-11-08T19:25:03.644112+0000","last_fullsized":"2023-11-08T19:25:03.644112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593332+0000","last_change":"2023-11-08T19:25:03.469891+0000","last_active":"2023-11-08T19:25:04.593332+0000","last_peered":"2023-11-08T19:25:04.593332+0000","last_clean":"2023-11-08T19:25:04.593332+0000","last_became_active":"2023-11-08T19:25:03.466854+0000","last_became_peered":"2023-11-08T19:25:03.466854+0000","last_unstale":"2023-11-08T19:25:04.593332+0000","last_undegraded":"2023-11-08T19:25:04.593332+0000","last_fullsized":"2023-11-08T19:25:04.593332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:05.405858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643596+0000","last_change":"2023-11-08T19:24:58.622842+0000","last_active":"2023-11-08T19:25:03.643596+0000","last_peered":"2023-11-08T19:25:03.643596+0000","last_clean":"2023-11-08T19:25:03.643596+0000","last_became_active":"2023-11-08T19:24:58.622530+0000","last_became_peered":"2023-11-08T19:24:58.622530+0000","last_unstale":"2023-11-08T19:25:03.643596+0000","last_undegraded":"2023-11-08T19:25:03.643596+0000","last_fullsized":"2023-11-08T19:25:03.643596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656961+0000","last_change":"2023-11-08T19:24:57.669407+0000","last_active":"2023-11-08T19:25:03.656961+0000","last_peered":"2023-11-08T19:25:03.656961+0000","last_clean":"2023-11-08T19:25:03.656961+0000","last_became_active":"2023-11-08T19:24:57.669169+0000","last_became_peered":"2023-11-08T19:24:57.669169+0000","last_unstale":"2023-11-08T19:25:03.656961+0000","last_undegraded":"2023-11-08T19:25:03.656961+0000","last_fullsized":"2023-11-08T19:25:03.656961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:50.920808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530152+0000","last_change":"2023-11-08T19:25:03.469685+0000","last_active":"2023-11-08T19:25:05.530152+0000","last_peered":"2023-11-08T19:25:05.530152+0000","last_clean":"2023-11-08T19:25:05.530152+0000","last_became_active":"2023-11-08T19:25:03.468600+0000","last_became_peered":"2023-11-08T19:25:03.468600+0000","last_unstale":"2023-11-08T19:25:05.530152+0000","last_undegraded":"2023-11-08T19:25:05.530152+0000","last_fullsized":"2023-11-08T19:25:05.530152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600914+0000","last_change":"2023-11-08T19:24:58.617568+0000","last_active":"2023-11-08T19:25:03.600914+0000","last_peered":"2023-11-08T19:25:03.600914+0000","last_clean":"2023-11-08T19:25:03.600914+0000","last_became_active":"2023-11-08T19:24:58.617216+0000","last_became_peered":"2023-11-08T19:24:58.617216+0000","last_unstale":"2023-11-08T19:25:03.600914+0000","last_undegraded":"2023-11-08T19:25:03.600914+0000","last_fullsized":"2023-11-08T19:25:03.600914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643888+0000","last_change":"2023-11-08T19:24:57.653655+0000","last_active":"2023-11-08T19:25:03.643888+0000","last_peered":"2023-11-08T19:25:03.643888+0000","last_clean":"2023-11-08T19:25:03.643888+0000","last_became_active":"2023-11-08T19:24:57.653390+0000","last_became_peered":"2023-11-08T19:24:57.653390+0000","last_unstale":"2023-11-08T19:25:03.643888+0000","last_undegraded":"2023-11-08T19:25:03.643888+0000","last_fullsized":"2023-11-08T19:25:03.643888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421356+0000","last_change":"2023-11-08T19:25:02.724803+0000","last_active":"2023-11-08T19:25:05.421356+0000","last_peered":"2023-11-08T19:25:05.421356+0000","last_clean":"2023-11-08T19:25:05.421356+0000","last_became_active":"2023-11-08T19:25:02.724278+0000","last_became_peered":"2023-11-08T19:25:02.724278+0000","last_unstale":"2023-11-08T19:25:05.421356+0000","last_undegraded":"2023-11-08T19:25:05.421356+0000","last_fullsized":"2023-11-08T19:25:05.421356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655769+0000","last_change":"2023-11-08T19:24:58.617441+0000","last_active":"2023-11-08T19:25:03.655769+0000","last_peered":"2023-11-08T19:25:03.655769+0000","last_clean":"2023-11-08T19:25:03.655769+0000","last_became_active":"2023-11-08T19:24:58.617030+0000","last_became_peered":"2023-11-08T19:24:58.617030+0000","last_unstale":"2023-11-08T19:25:03.655769+0000","last_undegraded":"2023-11-08T19:25:03.655769+0000","last_fullsized":"2023-11-08T19:25:03.655769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:50.043019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599773+0000","last_change":"2023-11-08T19:24:57.662494+0000","last_active":"2023-11-08T19:25:03.599773+0000","last_peered":"2023-11-08T19:25:03.599773+0000","last_clean":"2023-11-08T19:25:03.599773+0000","last_became_active":"2023-11-08T19:24:57.662223+0000","last_became_peered":"2023-11-08T19:24:57.662223+0000","last_unstale":"2023-11-08T19:25:03.599773+0000","last_undegraded":"2023-11-08T19:25:03.599773+0000","last_fullsized":"2023-11-08T19:25:03.599773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:49.940163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421797+0000","last_change":"2023-11-08T19:25:02.729279+0000","last_active":"2023-11-08T19:25:05.421797+0000","last_peered":"2023-11-08T19:25:05.421797+0000","last_clean":"2023-11-08T19:25:05.421797+0000","last_became_active":"2023-11-08T19:25:02.728855+0000","last_became_peered":"2023-11-08T19:25:02.728855+0000","last_unstale":"2023-11-08T19:25:05.421797+0000","last_undegraded":"2023-11-08T19:25:05.421797+0000","last_fullsized":"2023-11-08T19:25:05.421797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642895+0000","last_change":"2023-11-08T19:24:58.617415+0000","last_active":"2023-11-08T19:25:03.642895+0000","last_peered":"2023-11-08T19:25:03.642895+0000","last_clean":"2023-11-08T19:25:03.642895+0000","last_became_active":"2023-11-08T19:24:58.616880+0000","last_became_peered":"2023-11-08T19:24:58.616880+0000","last_unstale":"2023-11-08T19:25:03.642895+0000","last_undegraded":"2023-11-08T19:25:03.642895+0000","last_fullsized":"2023-11-08T19:25:03.642895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592969+0000","last_change":"2023-11-08T19:24:57.650498+0000","last_active":"2023-11-08T19:25:03.592969+0000","last_peered":"2023-11-08T19:25:03.592969+0000","last_clean":"2023-11-08T19:25:03.592969+0000","last_became_active":"2023-11-08T19:24:57.650172+0000","last_became_peered":"2023-11-08T19:24:57.650172+0000","last_unstale":"2023-11-08T19:25:03.592969+0000","last_undegraded":"2023-11-08T19:25:03.592969+0000","last_fullsized":"2023-11-08T19:25:03.592969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422691+0000","last_change":"2023-11-08T19:25:02.722430+0000","last_active":"2023-11-08T19:25:05.422691+0000","last_peered":"2023-11-08T19:25:05.422691+0000","last_clean":"2023-11-08T19:25:05.422691+0000","last_became_active":"2023-11-08T19:25:02.722013+0000","last_became_peered":"2023-11-08T19:25:02.722013+0000","last_unstale":"2023-11-08T19:25:05.422691+0000","last_undegraded":"2023-11-08T19:25:05.422691+0000","last_fullsized":"2023-11-08T19:25:05.422691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654808+0000","last_change":"2023-11-08T19:24:58.629901+0000","last_active":"2023-11-08T19:25:03.654808+0000","last_peered":"2023-11-08T19:25:03.654808+0000","last_clean":"2023-11-08T19:25:03.654808+0000","last_became_active":"2023-11-08T19:24:58.629603+0000","last_became_peered":"2023-11-08T19:24:58.629603+0000","last_unstale":"2023-11-08T19:25:03.654808+0000","last_undegraded":"2023-11-08T19:25:03.654808+0000","last_fullsized":"2023-11-08T19:25:03.654808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:19.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654594+0000","last_change":"2023-11-08T19:24:57.667258+0000","last_active":"2023-11-08T19:25:03.654594+0000","last_peered":"2023-11-08T19:25:03.654594+0000","last_clean":"2023-11-08T19:25:03.654594+0000","last_became_active":"2023-11-08T19:24:57.666859+0000","last_became_peered":"2023-11-08T19:24:57.666859+0000","last_unstale":"2023-11-08T19:25:03.654594+0000","last_undegraded":"2023-11-08T19:25:03.654594+0000","last_fullsized":"2023-11-08T19:25:03.654594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530805+0000","last_change":"2023-11-08T19:25:03.480894+0000","last_active":"2023-11-08T19:25:05.530805+0000","last_peered":"2023-11-08T19:25:05.530805+0000","last_clean":"2023-11-08T19:25:05.530805+0000","last_became_active":"2023-11-08T19:25:03.479219+0000","last_became_peered":"2023-11-08T19:25:03.479219+0000","last_unstale":"2023-11-08T19:25:05.530805+0000","last_undegraded":"2023-11-08T19:25:05.530805+0000","last_fullsized":"2023-11-08T19:25:05.530805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656729+0000","last_change":"2023-11-08T19:24:58.620051+0000","last_active":"2023-11-08T19:25:03.656729+0000","last_peered":"2023-11-08T19:25:03.656729+0000","last_clean":"2023-11-08T19:25:03.656729+0000","last_became_active":"2023-11-08T19:24:58.619641+0000","last_became_peered":"2023-11-08T19:24:58.619641+0000","last_unstale":"2023-11-08T19:25:03.656729+0000","last_undegraded":"2023-11-08T19:25:03.656729+0000","last_fullsized":"2023-11-08T19:25:03.656729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599436+0000","last_change":"2023-11-08T19:24:57.656028+0000","last_active":"2023-11-08T19:25:03.599436+0000","last_peered":"2023-11-08T19:25:03.599436+0000","last_clean":"2023-11-08T19:25:03.599436+0000","last_became_active":"2023-11-08T19:24:57.651726+0000","last_became_peered":"2023-11-08T19:24:57.651726+0000","last_unstale":"2023-11-08T19:25:03.599436+0000","last_undegraded":"2023-11-08T19:25:03.599436+0000","last_fullsized":"2023-11-08T19:25:03.599436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654244+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:25:03.654244+0000","last_peered":"2023-11-08T19:25:03.654244+0000","last_clean":"2023-11-08T19:25:03.654244+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:25:03.654244+0000","last_undegraded":"2023-11-08T19:25:03.654244+0000","last_fullsized":"2023-11-08T19:25:03.654244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349456+0000","last_change":"2023-11-08T19:25:03.468384+0000","last_active":"2023-11-08T19:25:06.349456+0000","last_peered":"2023-11-08T19:25:06.349456+0000","last_clean":"2023-11-08T19:25:06.349456+0000","last_became_active":"2023-11-08T19:25:03.467219+0000","last_became_peered":"2023-11-08T19:25:03.467219+0000","last_unstale":"2023-11-08T19:25:06.349456+0000","last_undegraded":"2023-11-08T19:25:06.349456+0000","last_fullsized":"2023-11-08T19:25:06.349456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656350+0000","last_change":"2023-11-08T19:24:58.632161+0000","last_active":"2023-11-08T19:25:03.656350+0000","last_peered":"2023-11-08T19:25:03.656350+0000","last_clean":"2023-11-08T19:25:03.656350+0000","last_became_active":"2023-11-08T19:24:58.631360+0000","last_became_peered":"2023-11-08T19:24:58.631360+0000","last_unstale":"2023-11-08T19:25:03.656350+0000","last_undegraded":"2023-11-08T19:25:03.656350+0000","last_fullsized":"2023-11-08T19:25:03.656350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594245+0000","last_change":"2023-11-08T19:24:57.659462+0000","last_active":"2023-11-08T19:25:03.594245+0000","last_peered":"2023-11-08T19:25:03.594245+0000","last_clean":"2023-11-08T19:25:03.594245+0000","last_became_active":"2023-11-08T19:24:57.659200+0000","last_became_peered":"2023-11-08T19:24:57.659200+0000","last_unstale":"2023-11-08T19:25:03.594245+0000","last_undegraded":"2023-11-08T19:25:03.594245+0000","last_fullsized":"2023-11-08T19:25:03.594245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:21.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.348403+0000","last_change":"2023-11-08T19:25:03.468319+0000","last_active":"2023-11-08T19:25:06.348403+0000","last_peered":"2023-11-08T19:25:06.348403+0000","last_clean":"2023-11-08T19:25:06.348403+0000","last_became_active":"2023-11-08T19:25:03.466962+0000","last_became_peered":"2023-11-08T19:25:03.466962+0000","last_unstale":"2023-11-08T19:25:06.348403+0000","last_undegraded":"2023-11-08T19:25:06.348403+0000","last_fullsized":"2023-11-08T19:25:06.348403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600157+0000","last_change":"2023-11-08T19:24:58.620109+0000","last_active":"2023-11-08T19:25:03.600157+0000","last_peered":"2023-11-08T19:25:03.600157+0000","last_clean":"2023-11-08T19:25:03.600157+0000","last_became_active":"2023-11-08T19:24:58.619504+0000","last_became_peered":"2023-11-08T19:24:58.619504+0000","last_unstale":"2023-11-08T19:25:03.600157+0000","last_undegraded":"2023-11-08T19:25:03.600157+0000","last_fullsized":"2023-11-08T19:25:03.600157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643946+0000","last_change":"2023-11-08T19:24:57.658265+0000","last_active":"2023-11-08T19:25:03.643946+0000","last_peered":"2023-11-08T19:25:03.643946+0000","last_clean":"2023-11-08T19:25:03.643946+0000","last_became_active":"2023-11-08T19:24:57.657921+0000","last_became_peered":"2023-11-08T19:24:57.657921+0000","last_unstale":"2023-11-08T19:25:03.643946+0000","last_undegraded":"2023-11-08T19:25:03.643946+0000","last_fullsized":"2023-11-08T19:25:03.643946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530244+0000","last_change":"2023-11-08T19:25:02.727032+0000","last_active":"2023-11-08T19:25:05.530244+0000","last_peered":"2023-11-08T19:25:05.530244+0000","last_clean":"2023-11-08T19:25:05.530244+0000","last_became_active":"2023-11-08T19:25:02.726636+0000","last_became_peered":"2023-11-08T19:25:02.726636+0000","last_unstale":"2023-11-08T19:25:05.530244+0000","last_undegraded":"2023-11-08T19:25:05.530244+0000","last_fullsized":"2023-11-08T19:25:05.530244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656744+0000","last_change":"2023-11-08T19:24:58.641465+0000","last_active":"2023-11-08T19:25:03.656744+0000","last_peered":"2023-11-08T19:25:03.656744+0000","last_clean":"2023-11-08T19:25:03.656744+0000","last_became_active":"2023-11-08T19:24:58.641144+0000","last_became_peered":"2023-11-08T19:24:58.641144+0000","last_unstale":"2023-11-08T19:25:03.656744+0000","last_undegraded":"2023-11-08T19:25:03.656744+0000","last_fullsized":"2023-11-08T19:25:03.656744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:00.683350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594008+0000","last_change":"2023-11-08T19:24:57.653180+0000","last_active":"2023-11-08T19:25:03.594008+0000","last_peered":"2023-11-08T19:25:03.594008+0000","last_clean":"2023-11-08T19:25:03.594008+0000","last_became_active":"2023-11-08T19:24:57.652617+0000","last_became_peered":"2023-11-08T19:24:57.652617+0000","last_unstale":"2023-11-08T19:25:03.594008+0000","last_undegraded":"2023-11-08T19:25:03.594008+0000","last_fullsized":"2023-11-08T19:25:03.594008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:35.866163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599737+0000","last_change":"2023-11-08T19:25:03.474070+0000","last_active":"2023-11-08T19:25:04.599737+0000","last_peered":"2023-11-08T19:25:04.599737+0000","last_clean":"2023-11-08T19:25:04.599737+0000","last_became_active":"2023-11-08T19:25:03.471994+0000","last_became_peered":"2023-11-08T19:25:03.471994+0000","last_unstale":"2023-11-08T19:25:04.599737+0000","last_undegraded":"2023-11-08T19:25:04.599737+0000","last_fullsized":"2023-11-08T19:25:04.599737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655993+0000","last_change":"2023-11-08T19:24:58.640557+0000","last_active":"2023-11-08T19:25:03.655993+0000","last_peered":"2023-11-08T19:25:03.655993+0000","last_clean":"2023-11-08T19:25:03.655993+0000","last_became_active":"2023-11-08T19:24:58.640005+0000","last_became_peered":"2023-11-08T19:24:58.640005+0000","last_unstale":"2023-11-08T19:25:03.655993+0000","last_undegraded":"2023-11-08T19:25:03.655993+0000","last_fullsized":"2023-11-08T19:25:03.655993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657695+0000","last_change":"2023-11-08T19:24:57.652195+0000","last_active":"2023-11-08T19:25:03.657695+0000","last_peered":"2023-11-08T19:25:03.657695+0000","last_clean":"2023-11-08T19:25:03.657695+0000","last_became_active":"2023-11-08T19:24:57.651790+0000","last_became_peered":"2023-11-08T19:24:57.651790+0000","last_unstale":"2023-11-08T19:25:03.657695+0000","last_undegraded":"2023-11-08T19:25:03.657695+0000","last_fullsized":"2023-11-08T19:25:03.657695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:50:21.039412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599054+0000","last_change":"2023-11-08T19:25:02.722727+0000","last_active":"2023-11-08T19:25:04.599054+0000","last_peered":"2023-11-08T19:25:04.599054+0000","last_clean":"2023-11-08T19:25:04.599054+0000","last_became_active":"2023-11-08T19:25:02.722092+0000","last_became_peered":"2023-11-08T19:25:02.722092+0000","last_unstale":"2023-11-08T19:25:04.599054+0000","last_undegraded":"2023-11-08T19:25:04.599054+0000","last_fullsized":"2023-11-08T19:25:04.599054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655407+0000","last_change":"2023-11-08T19:24:58.635248+0000","last_active":"2023-11-08T19:25:03.655407+0000","last_peered":"2023-11-08T19:25:03.655407+0000","last_clean":"2023-11-08T19:25:03.655407+0000","last_became_active":"2023-11-08T19:24:58.634504+0000","last_became_peered":"2023-11-08T19:24:58.634504+0000","last_unstale":"2023-11-08T19:25:03.655407+0000","last_undegraded":"2023-11-08T19:25:03.655407+0000","last_fullsized":"2023-11-08T19:25:03.655407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592589+0000","last_change":"2023-11-08T19:24:57.651955+0000","last_active":"2023-11-08T19:25:03.592589+0000","last_peered":"2023-11-08T19:25:03.592589+0000","last_clean":"2023-11-08T19:25:03.592589+0000","last_became_active":"2023-11-08T19:24:57.651666+0000","last_became_peered":"2023-11-08T19:24:57.651666+0000","last_unstale":"2023-11-08T19:25:03.592589+0000","last_undegraded":"2023-11-08T19:25:03.592589+0000","last_fullsized":"2023-11-08T19:25:03.592589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530618+0000","last_change":"2023-11-08T19:25:02.728613+0000","last_active":"2023-11-08T19:25:05.530618+0000","last_peered":"2023-11-08T19:25:05.530618+0000","last_clean":"2023-11-08T19:25:05.530618+0000","last_became_active":"2023-11-08T19:25:02.727749+0000","last_became_peered":"2023-11-08T19:25:02.727749+0000","last_unstale":"2023-11-08T19:25:05.530618+0000","last_undegraded":"2023-11-08T19:25:05.530618+0000","last_fullsized":"2023-11-08T19:25:05.530618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595934+0000","last_change":"2023-11-08T19:24:58.616453+0000","last_active":"2023-11-08T19:25:03.595934+0000","last_peered":"2023-11-08T19:25:03.595934+0000","last_clean":"2023-11-08T19:25:03.595934+0000","last_became_active":"2023-11-08T19:24:58.615934+0000","last_became_peered":"2023-11-08T19:24:58.615934+0000","last_unstale":"2023-11-08T19:25:03.595934+0000","last_undegraded":"2023-11-08T19:25:03.595934+0000","last_fullsized":"2023-11-08T19:25:03.595934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596954+0000","last_change":"2023-11-08T19:24:57.659585+0000","last_active":"2023-11-08T19:25:03.596954+0000","last_peered":"2023-11-08T19:25:03.596954+0000","last_clean":"2023-11-08T19:25:03.596954+0000","last_became_active":"2023-11-08T19:24:57.659197+0000","last_became_peered":"2023-11-08T19:24:57.659197+0000","last_unstale":"2023-11-08T19:25:03.596954+0000","last_undegraded":"2023-11-08T19:25:03.596954+0000","last_fullsized":"2023-11-08T19:25:03.596954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595745+0000","last_change":"2023-11-08T19:25:03.468147+0000","last_active":"2023-11-08T19:25:04.595745+0000","last_peered":"2023-11-08T19:25:04.595745+0000","last_clean":"2023-11-08T19:25:04.595745+0000","last_became_active":"2023-11-08T19:25:03.466921+0000","last_became_peered":"2023-11-08T19:25:03.466921+0000","last_unstale":"2023-11-08T19:25:04.595745+0000","last_undegraded":"2023-11-08T19:25:04.595745+0000","last_fullsized":"2023-11-08T19:25:04.595745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:14:51.027517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656098+0000","last_change":"2023-11-08T19:24:58.636188+0000","last_active":"2023-11-08T19:25:03.656098+0000","last_peered":"2023-11-08T19:25:03.656098+0000","last_clean":"2023-11-08T19:25:03.656098+0000","last_became_active":"2023-11-08T19:24:58.635889+0000","last_became_peered":"2023-11-08T19:24:58.635889+0000","last_unstale":"2023-11-08T19:25:03.656098+0000","last_undegraded":"2023-11-08T19:25:03.656098+0000","last_fullsized":"2023-11-08T19:25:03.656098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599561+0000","last_change":"2023-11-08T19:24:57.655323+0000","last_active":"2023-11-08T19:25:03.599561+0000","last_peered":"2023-11-08T19:25:03.599561+0000","last_clean":"2023-11-08T19:25:03.599561+0000","last_became_active":"2023-11-08T19:24:57.654973+0000","last_became_peered":"2023-11-08T19:24:57.654973+0000","last_unstale":"2023-11-08T19:25:03.599561+0000","last_undegraded":"2023-11-08T19:25:03.599561+0000","last_fullsized":"2023-11-08T19:25:03.599561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594252+0000","last_change":"2023-11-08T19:25:03.467735+0000","last_active":"2023-11-08T19:25:04.594252+0000","last_peered":"2023-11-08T19:25:04.594252+0000","last_clean":"2023-11-08T19:25:04.594252+0000","last_became_active":"2023-11-08T19:25:03.466595+0000","last_became_peered":"2023-11-08T19:25:03.466595+0000","last_unstale":"2023-11-08T19:25:04.594252+0000","last_undegraded":"2023-11-08T19:25:04.594252+0000","last_fullsized":"2023-11-08T19:25:04.594252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655761+0000","last_change":"2023-11-08T19:24:58.626959+0000","last_active":"2023-11-08T19:25:03.655761+0000","last_peered":"2023-11-08T19:25:03.655761+0000","last_clean":"2023-11-08T19:25:03.655761+0000","last_became_active":"2023-11-08T19:24:58.626616+0000","last_became_peered":"2023-11-08T19:24:58.626616+0000","last_unstale":"2023-11-08T19:25:03.655761+0000","last_undegraded":"2023-11-08T19:25:03.655761+0000","last_fullsized":"2023-11-08T19:25:03.655761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:57.663161+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:57.660362+0000","last_became_peered":"2023-11-08T19:24:57.660362+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602355+0000","last_change":"2023-11-08T19:25:03.477682+0000","last_active":"2023-11-08T19:25:04.602355+0000","last_peered":"2023-11-08T19:25:04.602355+0000","last_clean":"2023-11-08T19:25:04.602355+0000","last_became_active":"2023-11-08T19:25:03.476098+0000","last_became_peered":"2023-11-08T19:25:03.476098+0000","last_unstale":"2023-11-08T19:25:04.602355+0000","last_undegraded":"2023-11-08T19:25:04.602355+0000","last_fullsized":"2023-11-08T19:25:04.602355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656221+0000","last_change":"2023-11-08T19:24:58.634601+0000","last_active":"2023-11-08T19:25:03.656221+0000","last_peered":"2023-11-08T19:25:03.656221+0000","last_clean":"2023-11-08T19:25:03.656221+0000","last_became_active":"2023-11-08T19:24:58.632876+0000","last_became_peered":"2023-11-08T19:24:58.632876+0000","last_unstale":"2023-11-08T19:25:03.656221+0000","last_undegraded":"2023-11-08T19:25:03.656221+0000","last_fullsized":"2023-11-08T19:25:03.656221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:09.698655+0000","last_change":"2023-11-08T19:24:57.660289+0000","last_active":"2023-11-08T19:25:09.698655+0000","last_peered":"2023-11-08T19:25:09.698655+0000","last_clean":"2023-11-08T19:25:09.698655+0000","last_became_active":"2023-11-08T19:24:57.659982+0000","last_became_peered":"2023-11-08T19:24:57.659982+0000","last_unstale":"2023-11-08T19:25:09.698655+0000","last_undegraded":"2023-11-08T19:25:09.698655+0000","last_fullsized":"2023-11-08T19:25:09.698655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3700,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421411+0000","last_change":"2023-11-08T19:25:02.718892+0000","last_active":"2023-11-08T19:25:05.421411+0000","last_peered":"2023-11-08T19:25:05.421411+0000","last_clean":"2023-11-08T19:25:05.421411+0000","last_became_active":"2023-11-08T19:25:02.717998+0000","last_became_peered":"2023-11-08T19:25:02.717998+0000","last_unstale":"2023-11-08T19:25:05.421411+0000","last_undegraded":"2023-11-08T19:25:05.421411+0000","last_fullsized":"2023-11-08T19:25:05.421411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:58.629542+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:58.627201+0000","last_became_peered":"2023-11-08T19:24:58.627201+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643852+0000","last_change":"2023-11-08T19:24:57.661152+0000","last_active":"2023-11-08T19:25:03.643852+0000","last_peered":"2023-11-08T19:25:03.643852+0000","last_clean":"2023-11-08T19:25:03.643852+0000","last_became_active":"2023-11-08T19:24:57.660740+0000","last_became_peered":"2023-11-08T19:24:57.660740+0000","last_unstale":"2023-11-08T19:25:03.643852+0000","last_undegraded":"2023-11-08T19:25:03.643852+0000","last_fullsized":"2023-11-08T19:25:03.643852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602299+0000","last_change":"2023-11-08T19:25:02.727543+0000","last_active":"2023-11-08T19:25:04.602299+0000","last_peered":"2023-11-08T19:25:04.602299+0000","last_clean":"2023-11-08T19:25:04.602299+0000","last_became_active":"2023-11-08T19:25:02.726809+0000","last_became_peered":"2023-11-08T19:25:02.726809+0000","last_unstale":"2023-11-08T19:25:04.602299+0000","last_undegraded":"2023-11-08T19:25:04.602299+0000","last_fullsized":"2023-11-08T19:25:04.602299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655948+0000","last_change":"2023-11-08T19:24:58.618188+0000","last_active":"2023-11-08T19:25:03.655948+0000","last_peered":"2023-11-08T19:25:03.655948+0000","last_clean":"2023-11-08T19:25:03.655948+0000","last_became_active":"2023-11-08T19:24:58.617893+0000","last_became_peered":"2023-11-08T19:24:58.617893+0000","last_unstale":"2023-11-08T19:25:03.655948+0000","last_undegraded":"2023-11-08T19:25:03.655948+0000","last_fullsized":"2023-11-08T19:25:03.655948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600932+0000","last_change":"2023-11-08T19:24:57.662272+0000","last_active":"2023-11-08T19:25:03.600932+0000","last_peered":"2023-11-08T19:25:03.600932+0000","last_clean":"2023-11-08T19:25:03.600932+0000","last_became_active":"2023-11-08T19:24:57.662012+0000","last_became_peered":"2023-11-08T19:24:57.662012+0000","last_unstale":"2023-11-08T19:25:03.600932+0000","last_undegraded":"2023-11-08T19:25:03.600932+0000","last_fullsized":"2023-11-08T19:25:03.600932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594552+0000","last_change":"2023-11-08T19:25:02.729257+0000","last_active":"2023-11-08T19:25:04.594552+0000","last_peered":"2023-11-08T19:25:04.594552+0000","last_clean":"2023-11-08T19:25:04.594552+0000","last_became_active":"2023-11-08T19:25:02.727346+0000","last_became_peered":"2023-11-08T19:25:02.727346+0000","last_unstale":"2023-11-08T19:25:04.594552+0000","last_undegraded":"2023-11-08T19:25:04.594552+0000","last_fullsized":"2023-11-08T19:25:04.594552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:45.742494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600772+0000","last_change":"2023-11-08T19:24:58.626572+0000","last_active":"2023-11-08T19:25:03.600772+0000","last_peered":"2023-11-08T19:25:03.600772+0000","last_clean":"2023-11-08T19:25:03.600772+0000","last_became_active":"2023-11-08T19:24:58.626191+0000","last_became_peered":"2023-11-08T19:24:58.626191+0000","last_unstale":"2023-11-08T19:25:03.600772+0000","last_undegraded":"2023-11-08T19:25:03.600772+0000","last_fullsized":"2023-11-08T19:25:03.600772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597705+0000","last_change":"2023-11-08T19:24:57.672262+0000","last_active":"2023-11-08T19:25:03.597705+0000","last_peered":"2023-11-08T19:25:03.597705+0000","last_clean":"2023-11-08T19:25:03.597705+0000","last_became_active":"2023-11-08T19:24:57.671965+0000","last_became_peered":"2023-11-08T19:24:57.671965+0000","last_unstale":"2023-11-08T19:25:03.597705+0000","last_undegraded":"2023-11-08T19:25:03.597705+0000","last_fullsized":"2023-11-08T19:25:03.597705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594345+0000","last_change":"2023-11-08T19:25:03.467509+0000","last_active":"2023-11-08T19:25:04.594345+0000","last_peered":"2023-11-08T19:25:04.594345+0000","last_clean":"2023-11-08T19:25:04.594345+0000","last_became_active":"2023-11-08T19:25:03.466319+0000","last_became_peered":"2023-11-08T19:25:03.466319+0000","last_unstale":"2023-11-08T19:25:04.594345+0000","last_undegraded":"2023-11-08T19:25:04.594345+0000","last_fullsized":"2023-11-08T19:25:04.594345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:44.481960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530577+0000","last_change":"2023-11-08T19:25:03.475886+0000","last_active":"2023-11-08T19:25:05.530577+0000","last_peered":"2023-11-08T19:25:05.530577+0000","last_clean":"2023-11-08T19:25:05.530577+0000","last_became_active":"2023-11-08T19:25:03.473511+0000","last_became_peered":"2023-11-08T19:25:03.473511+0000","last_unstale":"2023-11-08T19:25:05.530577+0000","last_undegraded":"2023-11-08T19:25:05.530577+0000","last_fullsized":"2023-11-08T19:25:05.530577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642306+0000","last_change":"2023-11-08T19:24:58.621533+0000","last_active":"2023-11-08T19:25:03.642306+0000","last_peered":"2023-11-08T19:25:03.642306+0000","last_clean":"2023-11-08T19:25:03.642306+0000","last_became_active":"2023-11-08T19:24:58.621214+0000","last_became_peered":"2023-11-08T19:24:58.621214+0000","last_unstale":"2023-11-08T19:25:03.642306+0000","last_undegraded":"2023-11-08T19:25:03.642306+0000","last_fullsized":"2023-11-08T19:25:03.642306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596137+0000","last_change":"2023-11-08T19:24:57.656298+0000","last_active":"2023-11-08T19:25:03.596137+0000","last_peered":"2023-11-08T19:25:03.596137+0000","last_clean":"2023-11-08T19:25:03.596137+0000","last_became_active":"2023-11-08T19:24:57.655961+0000","last_became_peered":"2023-11-08T19:24:57.655961+0000","last_unstale":"2023-11-08T19:25:03.596137+0000","last_undegraded":"2023-11-08T19:25:03.596137+0000","last_fullsized":"2023-11-08T19:25:03.596137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349973+0000","last_change":"2023-11-08T19:25:02.721600+0000","last_active":"2023-11-08T19:25:06.349973+0000","last_peered":"2023-11-08T19:25:06.349973+0000","last_clean":"2023-11-08T19:25:06.349973+0000","last_became_active":"2023-11-08T19:25:02.721169+0000","last_became_peered":"2023-11-08T19:25:02.721169+0000","last_unstale":"2023-11-08T19:25:06.349973+0000","last_undegraded":"2023-11-08T19:25:06.349973+0000","last_fullsized":"2023-11-08T19:25:06.349973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654845+0000","last_change":"2023-11-08T19:24:58.624276+0000","last_active":"2023-11-08T19:25:03.654845+0000","last_peered":"2023-11-08T19:25:03.654845+0000","last_clean":"2023-11-08T19:25:03.654845+0000","last_became_active":"2023-11-08T19:24:58.623432+0000","last_became_peered":"2023-11-08T19:24:58.623432+0000","last_unstale":"2023-11-08T19:25:03.654845+0000","last_undegraded":"2023-11-08T19:25:03.654845+0000","last_fullsized":"2023-11-08T19:25:03.654845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656023+0000","last_change":"2023-11-08T19:24:57.661677+0000","last_active":"2023-11-08T19:25:03.656023+0000","last_peered":"2023-11-08T19:25:03.656023+0000","last_clean":"2023-11-08T19:25:03.656023+0000","last_became_active":"2023-11-08T19:24:57.661425+0000","last_became_peered":"2023-11-08T19:24:57.661425+0000","last_unstale":"2023-11-08T19:25:03.656023+0000","last_undegraded":"2023-11-08T19:25:03.656023+0000","last_fullsized":"2023-11-08T19:25:03.656023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596849+0000","last_change":"2023-11-08T19:24:57.662729+0000","last_active":"2023-11-08T19:25:03.596849+0000","last_peered":"2023-11-08T19:25:03.596849+0000","last_clean":"2023-11-08T19:25:03.596849+0000","last_became_active":"2023-11-08T19:24:57.662390+0000","last_became_peered":"2023-11-08T19:24:57.662390+0000","last_unstale":"2023-11-08T19:25:03.596849+0000","last_undegraded":"2023-11-08T19:25:03.596849+0000","last_fullsized":"2023-11-08T19:25:03.596849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656750+0000","last_change":"2023-11-08T19:24:58.619164+0000","last_active":"2023-11-08T19:25:03.656750+0000","last_peered":"2023-11-08T19:25:03.656750+0000","last_clean":"2023-11-08T19:25:03.656750+0000","last_became_active":"2023-11-08T19:24:58.618604+0000","last_became_peered":"2023-11-08T19:24:58.618604+0000","last_unstale":"2023-11-08T19:25:03.656750+0000","last_undegraded":"2023-11-08T19:25:03.656750+0000","last_fullsized":"2023-11-08T19:25:03.656750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.514753+0000","last_change":"2023-11-08T19:25:02.720151+0000","last_active":"2023-11-08T19:25:05.514753+0000","last_peered":"2023-11-08T19:25:05.514753+0000","last_clean":"2023-11-08T19:25:05.514753+0000","last_became_active":"2023-11-08T19:25:02.719391+0000","last_became_peered":"2023-11-08T19:25:02.719391+0000","last_unstale":"2023-11-08T19:25:05.514753+0000","last_undegraded":"2023-11-08T19:25:05.514753+0000","last_fullsized":"2023-11-08T19:25:05.514753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:23.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657232+0000","last_change":"2023-11-08T19:24:57.660203+0000","last_active":"2023-11-08T19:25:03.657232+0000","last_peered":"2023-11-08T19:25:03.657232+0000","last_clean":"2023-11-08T19:25:03.657232+0000","last_became_active":"2023-11-08T19:24:57.659913+0000","last_became_peered":"2023-11-08T19:24:57.659913+0000","last_unstale":"2023-11-08T19:25:03.657232+0000","last_undegraded":"2023-11-08T19:25:03.657232+0000","last_fullsized":"2023-11-08T19:25:03.657232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657083+0000","last_change":"2023-11-08T19:24:58.620564+0000","last_active":"2023-11-08T19:25:03.657083+0000","last_peered":"2023-11-08T19:25:03.657083+0000","last_clean":"2023-11-08T19:25:03.657083+0000","last_became_active":"2023-11-08T19:24:58.620072+0000","last_became_peered":"2023-11-08T19:24:58.620072+0000","last_unstale":"2023-11-08T19:25:03.657083+0000","last_undegraded":"2023-11-08T19:25:03.657083+0000","last_fullsized":"2023-11-08T19:25:03.657083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:36.879923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595054+0000","last_change":"2023-11-08T19:25:02.729532+0000","last_active":"2023-11-08T19:25:04.595054+0000","last_peered":"2023-11-08T19:25:04.595054+0000","last_clean":"2023-11-08T19:25:04.595054+0000","last_became_active":"2023-11-08T19:25:02.728431+0000","last_became_peered":"2023-11-08T19:25:02.728431+0000","last_unstale":"2023-11-08T19:25:04.595054+0000","last_undegraded":"2023-11-08T19:25:04.595054+0000","last_fullsized":"2023-11-08T19:25:04.595054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:57.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595597+0000","last_change":"2023-11-08T19:24:58.624056+0000","last_active":"2023-11-08T19:25:03.595597+0000","last_peered":"2023-11-08T19:25:03.595597+0000","last_clean":"2023-11-08T19:25:03.595597+0000","last_became_active":"2023-11-08T19:24:58.623783+0000","last_became_peered":"2023-11-08T19:24:58.623783+0000","last_unstale":"2023-11-08T19:25:03.595597+0000","last_undegraded":"2023-11-08T19:25:03.595597+0000","last_fullsized":"2023-11-08T19:25:03.595597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593245+0000","last_change":"2023-11-08T19:24:58.617678+0000","last_active":"2023-11-08T19:25:03.593245+0000","last_peered":"2023-11-08T19:25:03.593245+0000","last_clean":"2023-11-08T19:25:03.593245+0000","last_became_active":"2023-11-08T19:24:58.616701+0000","last_became_peered":"2023-11-08T19:24:58.616701+0000","last_unstale":"2023-11-08T19:25:03.593245+0000","last_undegraded":"2023-11-08T19:25:03.593245+0000","last_fullsized":"2023-11-08T19:25:03.593245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599324+0000","last_change":"2023-11-08T19:24:58.620961+0000","last_active":"2023-11-08T19:25:03.599324+0000","last_peered":"2023-11-08T19:25:03.599324+0000","last_clean":"2023-11-08T19:25:03.599324+0000","last_became_active":"2023-11-08T19:24:58.620359+0000","last_became_peered":"2023-11-08T19:24:58.620359+0000","last_unstale":"2023-11-08T19:25:03.599324+0000","last_undegraded":"2023-11-08T19:25:03.599324+0000","last_fullsized":"2023-11-08T19:25:03.599324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656545+0000","last_change":"2023-11-08T19:24:58.622475+0000","last_active":"2023-11-08T19:25:03.656545+0000","last_peered":"2023-11-08T19:25:03.656545+0000","last_clean":"2023-11-08T19:25:03.656545+0000","last_became_active":"2023-11-08T19:24:58.622206+0000","last_became_peered":"2023-11-08T19:24:58.622206+0000","last_unstale":"2023-11-08T19:25:03.656545+0000","last_undegraded":"2023-11-08T19:25:03.656545+0000","last_fullsized":"2023-11-08T19:25:03.656545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657638+0000","last_change":"2023-11-08T19:24:58.630303+0000","last_active":"2023-11-08T19:25:03.657638+0000","last_peered":"2023-11-08T19:25:03.657638+0000","last_clean":"2023-11-08T19:25:03.657638+0000","last_became_active":"2023-11-08T19:24:58.629181+0000","last_became_peered":"2023-11-08T19:24:58.629181+0000","last_unstale":"2023-11-08T19:25:03.657638+0000","last_undegraded":"2023-11-08T19:25:03.657638+0000","last_fullsized":"2023-11-08T19:25:03.657638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643449+0000","last_change":"2023-11-08T19:24:58.622627+0000","last_active":"2023-11-08T19:25:03.643449+0000","last_peered":"2023-11-08T19:25:03.643449+0000","last_clean":"2023-11-08T19:25:03.643449+0000","last_became_active":"2023-11-08T19:24:58.622335+0000","last_became_peered":"2023-11-08T19:24:58.622335+0000","last_unstale":"2023-11-08T19:25:03.643449+0000","last_undegraded":"2023-11-08T19:25:03.643449+0000","last_fullsized":"2023-11-08T19:25:03.643449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597541+0000","last_change":"2023-11-08T19:24:58.630440+0000","last_active":"2023-11-08T19:25:03.597541+0000","last_peered":"2023-11-08T19:25:03.597541+0000","last_clean":"2023-11-08T19:25:03.597541+0000","last_became_active":"2023-11-08T19:24:58.629904+0000","last_became_peered":"2023-11-08T19:24:58.629904+0000","last_unstale":"2023-11-08T19:25:03.597541+0000","last_undegraded":"2023-11-08T19:25:03.597541+0000","last_fullsized":"2023-11-08T19:25:03.597541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.627568+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.627243+0000","last_became_peered":"2023-11-08T19:24:58.627243+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595899+0000","last_change":"2023-11-08T19:24:58.622232+0000","last_active":"2023-11-08T19:25:03.595899+0000","last_peered":"2023-11-08T19:25:03.595899+0000","last_clean":"2023-11-08T19:25:03.595899+0000","last_became_active":"2023-11-08T19:24:58.621853+0000","last_became_peered":"2023-11-08T19:24:58.621853+0000","last_unstale":"2023-11-08T19:25:03.595899+0000","last_undegraded":"2023-11-08T19:25:03.595899+0000","last_fullsized":"2023-11-08T19:25:03.595899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654133+0000","last_change":"2023-11-08T19:24:58.630530+0000","last_active":"2023-11-08T19:25:03.654133+0000","last_peered":"2023-11-08T19:25:03.654133+0000","last_clean":"2023-11-08T19:25:03.654133+0000","last_became_active":"2023-11-08T19:24:58.627752+0000","last_became_peered":"2023-11-08T19:24:58.627752+0000","last_unstale":"2023-11-08T19:25:03.654133+0000","last_undegraded":"2023-11-08T19:25:03.654133+0000","last_fullsized":"2023-11-08T19:25:03.654133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599081+0000","last_change":"2023-11-08T19:24:58.626422+0000","last_active":"2023-11-08T19:25:03.599081+0000","last_peered":"2023-11-08T19:25:03.599081+0000","last_clean":"2023-11-08T19:25:03.599081+0000","last_became_active":"2023-11-08T19:24:58.626091+0000","last_became_peered":"2023-11-08T19:24:58.626091+0000","last_unstale":"2023-11-08T19:25:03.599081+0000","last_undegraded":"2023-11-08T19:25:03.599081+0000","last_fullsized":"2023-11-08T19:25:03.599081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656191+0000","last_change":"2023-11-08T19:24:58.630415+0000","last_active":"2023-11-08T19:25:03.656191+0000","last_peered":"2023-11-08T19:25:03.656191+0000","last_clean":"2023-11-08T19:25:03.656191+0000","last_became_active":"2023-11-08T19:24:58.628526+0000","last_became_peered":"2023-11-08T19:24:58.628526+0000","last_unstale":"2023-11-08T19:25:03.656191+0000","last_undegraded":"2023-11-08T19:25:03.656191+0000","last_fullsized":"2023-11-08T19:25:03.656191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596558+0000","last_change":"2023-11-08T19:24:58.628286+0000","last_active":"2023-11-08T19:25:03.596558+0000","last_peered":"2023-11-08T19:25:03.596558+0000","last_clean":"2023-11-08T19:25:03.596558+0000","last_became_active":"2023-11-08T19:24:58.627912+0000","last_became_peered":"2023-11-08T19:24:58.627912+0000","last_unstale":"2023-11-08T19:25:03.596558+0000","last_undegraded":"2023-11-08T19:25:03.596558+0000","last_fullsized":"2023-11-08T19:25:03.596558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656975+0000","last_change":"2023-11-08T19:24:58.633630+0000","last_active":"2023-11-08T19:25:03.656975+0000","last_peered":"2023-11-08T19:25:03.656975+0000","last_clean":"2023-11-08T19:25:03.656975+0000","last_became_active":"2023-11-08T19:24:58.632441+0000","last_became_peered":"2023-11-08T19:24:58.632441+0000","last_unstale":"2023-11-08T19:25:03.656975+0000","last_undegraded":"2023-11-08T19:25:03.656975+0000","last_fullsized":"2023-11-08T19:25:03.656975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593821+0000","last_change":"2023-11-08T19:24:58.623578+0000","last_active":"2023-11-08T19:25:03.593821+0000","last_peered":"2023-11-08T19:25:03.593821+0000","last_clean":"2023-11-08T19:25:03.593821+0000","last_became_active":"2023-11-08T19:24:58.623261+0000","last_became_peered":"2023-11-08T19:24:58.623261+0000","last_unstale":"2023-11-08T19:25:03.593821+0000","last_undegraded":"2023-11-08T19:25:03.593821+0000","last_fullsized":"2023-11-08T19:25:03.593821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:36.969600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.601266+0000","last_change":"2023-11-08T19:24:58.634259+0000","last_active":"2023-11-08T19:25:03.601266+0000","last_peered":"2023-11-08T19:25:03.601266+0000","last_clean":"2023-11-08T19:25:03.601266+0000","last_became_active":"2023-11-08T19:24:58.633708+0000","last_became_peered":"2023-11-08T19:24:58.633708+0000","last_unstale":"2023-11-08T19:25:03.601266+0000","last_undegraded":"2023-11-08T19:25:03.601266+0000","last_fullsized":"2023-11-08T19:25:03.601266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595680+0000","last_change":"2023-11-08T19:24:58.615871+0000","last_active":"2023-11-08T19:25:03.595680+0000","last_peered":"2023-11-08T19:25:03.595680+0000","last_clean":"2023-11-08T19:25:03.595680+0000","last_became_active":"2023-11-08T19:24:58.615590+0000","last_became_peered":"2023-11-08T19:24:58.615590+0000","last_unstale":"2023-11-08T19:25:03.595680+0000","last_undegraded":"2023-11-08T19:25:03.595680+0000","last_fullsized":"2023-11-08T19:25:03.595680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642579+0000","last_change":"2023-11-08T19:24:58.629058+0000","last_active":"2023-11-08T19:25:03.642579+0000","last_peered":"2023-11-08T19:25:03.642579+0000","last_clean":"2023-11-08T19:25:03.642579+0000","last_became_active":"2023-11-08T19:24:58.628656+0000","last_became_peered":"2023-11-08T19:24:58.628656+0000","last_unstale":"2023-11-08T19:25:03.642579+0000","last_undegraded":"2023-11-08T19:25:03.642579+0000","last_fullsized":"2023-11-08T19:25:03.642579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655552+0000","last_change":"2023-11-08T19:24:58.622286+0000","last_active":"2023-11-08T19:25:03.655552+0000","last_peered":"2023-11-08T19:25:03.655552+0000","last_clean":"2023-11-08T19:25:03.655552+0000","last_became_active":"2023-11-08T19:24:58.621993+0000","last_became_peered":"2023-11-08T19:24:58.621993+0000","last_unstale":"2023-11-08T19:25:03.655552+0000","last_undegraded":"2023-11-08T19:25:03.655552+0000","last_fullsized":"2023-11-08T19:25:03.655552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656958+0000","last_change":"2023-11-08T19:24:58.618762+0000","last_active":"2023-11-08T19:25:03.656958+0000","last_peered":"2023-11-08T19:25:03.656958+0000","last_clean":"2023-11-08T19:25:03.656958+0000","last_became_active":"2023-11-08T19:24:58.618468+0000","last_became_peered":"2023-11-08T19:24:58.618468+0000","last_unstale":"2023-11-08T19:25:03.656958+0000","last_undegraded":"2023-11-08T19:25:03.656958+0000","last_fullsized":"2023-11-08T19:25:03.656958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655770+0000","last_change":"2023-11-08T19:24:58.626193+0000","last_active":"2023-11-08T19:25:03.655770+0000","last_peered":"2023-11-08T19:25:03.655770+0000","last_clean":"2023-11-08T19:25:03.655770+0000","last_became_active":"2023-11-08T19:24:58.625897+0000","last_became_peered":"2023-11-08T19:24:58.625897+0000","last_unstale":"2023-11-08T19:25:03.655770+0000","last_undegraded":"2023-11-08T19:25:03.655770+0000","last_fullsized":"2023-11-08T19:25:03.655770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:43.279737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596855+0000","last_change":"2023-11-08T19:24:58.621575+0000","last_active":"2023-11-08T19:25:03.596855+0000","last_peered":"2023-11-08T19:25:03.596855+0000","last_clean":"2023-11-08T19:25:03.596855+0000","last_became_active":"2023-11-08T19:24:58.621278+0000","last_became_peered":"2023-11-08T19:24:58.621278+0000","last_unstale":"2023-11-08T19:25:03.596855+0000","last_undegraded":"2023-11-08T19:25:03.596855+0000","last_fullsized":"2023-11-08T19:25:03.596855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597421+0000","last_change":"2023-11-08T19:24:58.630221+0000","last_active":"2023-11-08T19:25:03.597421+0000","last_peered":"2023-11-08T19:25:03.597421+0000","last_clean":"2023-11-08T19:25:03.597421+0000","last_became_active":"2023-11-08T19:24:58.629514+0000","last_became_peered":"2023-11-08T19:24:58.629514+0000","last_unstale":"2023-11-08T19:25:03.597421+0000","last_undegraded":"2023-11-08T19:25:03.597421+0000","last_fullsized":"2023-11-08T19:25:03.597421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656658+0000","last_change":"2023-11-08T19:24:58.637307+0000","last_active":"2023-11-08T19:25:03.656658+0000","last_peered":"2023-11-08T19:25:03.656658+0000","last_clean":"2023-11-08T19:25:03.656658+0000","last_became_active":"2023-11-08T19:24:58.636926+0000","last_became_peered":"2023-11-08T19:24:58.636926+0000","last_unstale":"2023-11-08T19:25:03.656658+0000","last_undegraded":"2023-11-08T19:25:03.656658+0000","last_fullsized":"2023-11-08T19:25:03.656658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:52.742252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654271+0000","last_change":"2023-11-08T19:24:58.619975+0000","last_active":"2023-11-08T19:25:03.654271+0000","last_peered":"2023-11-08T19:25:03.654271+0000","last_clean":"2023-11-08T19:25:03.654271+0000","last_became_active":"2023-11-08T19:24:58.619676+0000","last_became_peered":"2023-11-08T19:24:58.619676+0000","last_unstale":"2023-11-08T19:25:03.654271+0000","last_undegraded":"2023-11-08T19:25:03.654271+0000","last_fullsized":"2023-11-08T19:25:03.654271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654365+0000","last_change":"2023-11-08T19:24:58.627535+0000","last_active":"2023-11-08T19:25:03.654365+0000","last_peered":"2023-11-08T19:25:03.654365+0000","last_clean":"2023-11-08T19:25:03.654365+0000","last_became_active":"2023-11-08T19:24:58.627205+0000","last_became_peered":"2023-11-08T19:24:58.627205+0000","last_unstale":"2023-11-08T19:25:03.654365+0000","last_undegraded":"2023-11-08T19:25:03.654365+0000","last_fullsized":"2023-11-08T19:25:03.654365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599068+0000","last_change":"2023-11-08T19:24:58.620637+0000","last_active":"2023-11-08T19:25:03.599068+0000","last_peered":"2023-11-08T19:25:03.599068+0000","last_clean":"2023-11-08T19:25:03.599068+0000","last_became_active":"2023-11-08T19:24:58.619919+0000","last_became_peered":"2023-11-08T19:24:58.619919+0000","last_unstale":"2023-11-08T19:25:03.599068+0000","last_undegraded":"2023-11-08T19:25:03.599068+0000","last_fullsized":"2023-11-08T19:25:03.599068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655858+0000","last_change":"2023-11-08T19:24:58.630887+0000","last_active":"2023-11-08T19:25:03.655858+0000","last_peered":"2023-11-08T19:25:03.655858+0000","last_clean":"2023-11-08T19:25:03.655858+0000","last_became_active":"2023-11-08T19:24:58.630010+0000","last_became_peered":"2023-11-08T19:24:58.630010+0000","last_unstale":"2023-11-08T19:25:03.655858+0000","last_undegraded":"2023-11-08T19:25:03.655858+0000","last_fullsized":"2023-11-08T19:25:03.655858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599939+0000","last_change":"2023-11-08T19:24:58.625672+0000","last_active":"2023-11-08T19:25:03.599939+0000","last_peered":"2023-11-08T19:25:03.599939+0000","last_clean":"2023-11-08T19:25:03.599939+0000","last_became_active":"2023-11-08T19:24:58.619366+0000","last_became_peered":"2023-11-08T19:24:58.619366+0000","last_unstale":"2023-11-08T19:25:03.599939+0000","last_undegraded":"2023-11-08T19:25:03.599939+0000","last_fullsized":"2023-11-08T19:25:03.599939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655793+0000","last_change":"2023-11-08T19:24:58.612582+0000","last_active":"2023-11-08T19:25:03.655793+0000","last_peered":"2023-11-08T19:25:03.655793+0000","last_clean":"2023-11-08T19:25:03.655793+0000","last_became_active":"2023-11-08T19:24:58.612199+0000","last_became_peered":"2023-11-08T19:24:58.612199+0000","last_unstale":"2023-11-08T19:25:03.655793+0000","last_undegraded":"2023-11-08T19:25:03.655793+0000","last_fullsized":"2023-11-08T19:25:03.655793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643845+0000","last_change":"2023-11-08T19:24:58.616793+0000","last_active":"2023-11-08T19:25:03.643845+0000","last_peered":"2023-11-08T19:25:03.643845+0000","last_clean":"2023-11-08T19:25:03.643845+0000","last_became_active":"2023-11-08T19:24:58.616483+0000","last_became_peered":"2023-11-08T19:24:58.616483+0000","last_unstale":"2023-11-08T19:25:03.643845+0000","last_undegraded":"2023-11-08T19:25:03.643845+0000","last_fullsized":"2023-11-08T19:25:03.643845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656535+0000","last_change":"2023-11-08T19:24:58.640055+0000","last_active":"2023-11-08T19:25:03.656535+0000","last_peered":"2023-11-08T19:25:03.656535+0000","last_clean":"2023-11-08T19:25:03.656535+0000","last_became_active":"2023-11-08T19:24:58.639464+0000","last_became_peered":"2023-11-08T19:24:58.639464+0000","last_unstale":"2023-11-08T19:25:03.656535+0000","last_undegraded":"2023-11-08T19:25:03.656535+0000","last_fullsized":"2023-11-08T19:25:03.656535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:51.551899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4498,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":697807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.546}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.752}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3170000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0110000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.427}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3460000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.403}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5069999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1910000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7310000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5529999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3660000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":105838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3160000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2669999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4570000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1439999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.466}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.256}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2310000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.369}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3029999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.282}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:22.703 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:25:22.704 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-08T19:25:22.704 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-08T19:25:22.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-08T19:25:22.705 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-08T19:25:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.953+0000 7fb5eeed4640 1 Processor -- start 2023-11-08T19:25:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.954+0000 7fb5eeed4640 1 -- start start 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.955+0000 7fb5eeed4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5e815d6c0 0x7fb5e80d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.955+0000 7fb5eeed4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e815df70 con 0x7fb5e80cfa00 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.955+0000 7fb5eeed4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e80d1e60 con 0x7fb5e8161870 2023-11-08T19:25:22.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.955+0000 7fb5eeed4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e815daa0 con 0x7fb5e815d6c0 2023-11-08T19:25:22.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5ecc70640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb5e815d6c0 0x7fb5e80d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5e7fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb5e80cfa00 0x7fb5e8161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59978/0 (socket says 172.21.15.119:59978) 2023-11-08T19:25:22.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1125612324 learned_addr learned my addr 172.21.15.119:0/1125612324 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:22.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5e77fe640 1 -- 172.21.15.119:0/1125612324 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363619074 0 0) 0x7fb5e815df70 con 0x7fb5e80cfa00 2023-11-08T19:25:22.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5ecc70640 1 -- 172.21.15.119:0/1125612324 >> v1:172.21.15.188:6789/0 conn(0x7fb5e8161870 legacy=0x7fb5e80cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:22.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5ecc70640 1 -- 172.21.15.119:0/1125612324 >> v1:172.21.15.119:6789/0 conn(0x7fb5e80cfa00 legacy=0x7fb5e8161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.956+0000 7fb5ecc70640 1 -- 172.21.15.119:0/1125612324 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80d2830 con 0x7fb5e815d6c0 2023-11-08T19:25:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.958+0000 7fb5ecc70640 1 --2- 172.21.15.119:0/1125612324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80d22f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009d90 tx=0x7fb5d802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a81a92d830f99ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.958+0000 7fb5e77fe640 1 -- 172.21.15.119:0/1125612324 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d802cbd0 con 0x7fb5e815d6c0 2023-11-08T19:25:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.958+0000 7fb5e77fe640 1 -- 172.21.15.119:0/1125612324 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d802cd30 con 0x7fb5e815d6c0 2023-11-08T19:25:22.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.958+0000 7fb5e77fe640 1 -- 172.21.15.119:0/1125612324 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d802cea0 con 0x7fb5e815d6c0 2023-11-08T19:25:22.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 -- 172.21.15.119:0/1125612324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 msgr2=0x7fb5e80d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/1125612324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80d22f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009d90 tx=0x7fb5d802be20 comp rx=0 tx=0).stop 2023-11-08T19:25:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 -- 172.21.15.119:0/1125612324 shutdown_connections 2023-11-08T19:25:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/1125612324 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80d22f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 -- 172.21.15.119:0/1125612324 >> 172.21.15.119:0/1125612324 conn(0x7fb5e806a9f0 msgr2=0x7fb5e806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:22.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 -- 172.21.15.119:0/1125612324 shutdown_connections 2023-11-08T19:25:22.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.959+0000 7fb5eeed4640 1 -- 172.21.15.119:0/1125612324 wait complete. 2023-11-08T19:25:22.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.960+0000 7fb5eeed4640 1 Processor -- start 2023-11-08T19:25:22.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.961+0000 7fb5eeed4640 1 -- start start 2023-11-08T19:25:22.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.961+0000 7fb5eeed4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cfa00 0x7fb5e80b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5eeed4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5eeed4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e8161870 0x7fb5e80cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5eeed4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80b4730 con 0x7fb5e815d6c0 2023-11-08T19:25:22.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5eeed4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80b39a0 con 0x7fb5e8161870 2023-11-08T19:25:22.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5eeed4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8154bb0 con 0x7fb5e80cfa00 2023-11-08T19:25:22.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50986/0 (socket says 172.21.15.119:50986) 2023-11-08T19:25:22.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 -- 172.21.15.119:0/877548365 learned_addr learned my addr 172.21.15.119:0/877548365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:22.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 -- 172.21.15.119:0/877548365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cfa00 msgr2=0x7fb5e80b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cfa00 0x7fb5e80b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 -- 172.21.15.119:0/877548365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e8161870 msgr2=0x7fb5e80cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:22.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e8161870 0x7fb5e80cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:22.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.962+0000 7fb5e7fff640 1 -- 172.21.15.119:0/877548365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d8009a70 con 0x7fb5e815d6c0 2023-11-08T19:25:22.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.963+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80c94e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c4d0 tx=0x7fb5d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.963+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d00074e0 con 0x7fb5e815d6c0 2023-11-08T19:25:22.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.963+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d0007660 con 0x7fb5e815d6c0 2023-11-08T19:25:22.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.963+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e80cccb0 con 0x7fb5e815d6c0 2023-11-08T19:25:22.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.963+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0004a20 con 0x7fb5e815d6c0 2023-11-08T19:25:22.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.963+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e80b3e90 con 0x7fb5e815d6c0 2023-11-08T19:25:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.964+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e8074430 con 0x7fb5e815d6c0 2023-11-08T19:25:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.965+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5d0004b80 con 0x7fb5e815d6c0 2023-11-08T19:25:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.966+0000 7fb5e57fa640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5c806ae70 0x7fb5c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.966+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb5d0088b00 con 0x7fb5e815d6c0 2023-11-08T19:25:22.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.966+0000 7fb5ecc70640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5c806ae70 0x7fb5c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.967+0000 7fb5ecc70640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5c806ae70 0x7fb5c806d330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8036000 tx=0x7fb5d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:22.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:22.970+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5d008ae60 con 0x7fb5e815d6c0 2023-11-08T19:25:23.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.206+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb5e80d10f0 con 0x7fb5e815d6c0 2023-11-08T19:25:23.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.212+0000 7fb5e57fa640 1 -- 172.21.15.119:0/877548365 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7fb5d0018020 con 0x7fb5e815d6c0 2023-11-08T19:25:23.216 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:23.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5c806ae70 msgr2=0x7fb5c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5c806ae70 0x7fb5c806d330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8036000 tx=0x7fb5d802e000 comp rx=0 tx=0).stop 2023-11-08T19:25:23.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 msgr2=0x7fb5e80c94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80c94e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb5d000c4d0 tx=0x7fb5d000c990 comp rx=0 tx=0).stop 2023-11-08T19:25:23.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 shutdown_connections 2023-11-08T19:25:23.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5c806ae70 0x7fb5c806d330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e8161870 0x7fb5e80cbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5e815d6c0 0x7fb5e80c94e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 --2- 172.21.15.119:0/877548365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cfa00 0x7fb5e80b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.216+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 >> 172.21.15.119:0/877548365 conn(0x7fb5e806a9f0 msgr2=0x7fb5e80b66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.217+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 shutdown_connections 2023-11-08T19:25:23.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.217+0000 7fb5eeed4640 1 -- 172.21.15.119:0/877548365 wait complete. 2023-11-08T19:25:23.237 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":5,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6832","nonce":2270580327},{"type":"v1","addr":"172.21.15.119:6833","nonce":2270580327}]},"active_addr":"172.21.15.119:6833/2270580327","active_change":"2023-11-08T19:24:38.621458+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3678938740}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3656944439}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":3747458233}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.119:0","nonce":2015295089}]}]} 2023-11-08T19:25:23.242 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-08T19:25:23.242 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-08T19:25:23.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:23.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.453+0000 7f215b525640 1 Processor -- start 2023-11-08T19:25:23.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.453+0000 7f215b525640 1 -- start start 2023-11-08T19:25:23.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f215b525640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21541447a0 0x7f215415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f215b525640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21540cad30 0x7f215406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f215b525640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21540cb170 0x7f215406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f215b525640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21540c3a50 con 0x7f21540cb170 2023-11-08T19:25:23.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f215b525640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2154144b80 con 0x7f21540cad30 2023-11-08T19:25:23.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f215b525640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c9150 con 0x7f21541447a0 2023-11-08T19:25:23.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f2159ac2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21540cb170 0x7f215406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f2159ac2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f21540cb170 0x7f215406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50994/0 (socket says 172.21.15.119:50994) 2023-11-08T19:25:23.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f2159ac2640 1 -- 172.21.15.119:0/2055900257 learned_addr learned my addr 172.21.15.119:0/2055900257 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:23.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f21592c1640 1 --2- 172.21.15.119:0/2055900257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21541447a0 0x7f215415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.455+0000 7f2158ac0640 1 --2- 172.21.15.119:0/2055900257 >> v2:172.21.15.188:3300/0 conn(0x7f21540cad30 0x7f215406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.456+0000 7f2159ac2640 1 -- 172.21.15.119:0/2055900257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21541447a0 msgr2=0x7f215415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:23.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.456+0000 7f2159ac2640 1 --2- 172.21.15.119:0/2055900257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21541447a0 0x7f215415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.456+0000 7f2159ac2640 1 -- 172.21.15.119:0/2055900257 >> v2:172.21.15.188:3300/0 conn(0x7f21540cad30 msgr2=0x7f215406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:23.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.456+0000 7f2159ac2640 1 --2- 172.21.15.119:0/2055900257 >> v2:172.21.15.188:3300/0 conn(0x7f21540cad30 0x7f215406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.456+0000 7f2159ac2640 1 -- 172.21.15.119:0/2055900257 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21540c9410 con 0x7f21540cb170 2023-11-08T19:25:23.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.456+0000 7f2158ac0640 1 --2- 172.21.15.119:0/2055900257 >> v2:172.21.15.188:3300/0 conn(0x7f21540cad30 0x7f215406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:23.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.457+0000 7f2159ac2640 1 --2- 172.21.15.119:0/2055900257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21540cb170 0x7f215406a120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f215406b200 tx=0x7f214c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc208ccd8978aa10 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.457+0000 7f2153fff640 1 -- 172.21.15.119:0/2055900257 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c031860 con 0x7f21540cb170 2023-11-08T19:25:23.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.457+0000 7f2153fff640 1 -- 172.21.15.119:0/2055900257 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c0319c0 con 0x7f21540cb170 2023-11-08T19:25:23.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.457+0000 7f2153fff640 1 -- 172.21.15.119:0/2055900257 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c031d50 con 0x7f21540cb170 2023-11-08T19:25:23.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 -- 172.21.15.119:0/2055900257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21540cb170 msgr2=0x7f215406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 --2- 172.21.15.119:0/2055900257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21540cb170 0x7f215406a120 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f215406b200 tx=0x7f214c02b900 comp rx=0 tx=0).stop 2023-11-08T19:25:23.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 -- 172.21.15.119:0/2055900257 shutdown_connections 2023-11-08T19:25:23.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 --2- 172.21.15.119:0/2055900257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21540cb170 0x7f215406a120 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 --2- 172.21.15.119:0/2055900257 >> v2:172.21.15.188:3300/0 conn(0x7f21540cad30 0x7f215406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 --2- 172.21.15.119:0/2055900257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21541447a0 0x7f215415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 -- 172.21.15.119:0/2055900257 >> 172.21.15.119:0/2055900257 conn(0x7f21540cb610 msgr2=0x7f21540cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 -- 172.21.15.119:0/2055900257 shutdown_connections 2023-11-08T19:25:23.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.458+0000 7f215b525640 1 -- 172.21.15.119:0/2055900257 wait complete. 2023-11-08T19:25:23.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.460+0000 7f215b525640 1 Processor -- start 2023-11-08T19:25:23.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.460+0000 7f215b525640 1 -- start start 2023-11-08T19:25:23.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f215b525640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21540cad30 0x7f2154142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f215b525640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21540cb170 0x7f21540c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f215b525640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 0x7f215413ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f215b525640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c75b0 con 0x7f21541447a0 2023-11-08T19:25:23.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f215b525640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c69a0 con 0x7f21540cad30 2023-11-08T19:25:23.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f215b525640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21540c6c80 con 0x7f21540cb170 2023-11-08T19:25:23.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f2159ac2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 0x7f215413ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.461+0000 7f2159ac2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 0x7f215413ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51000/0 (socket says 172.21.15.119:51000) 2023-11-08T19:25:23.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 -- 172.21.15.119:0/539714711 learned_addr learned my addr 172.21.15.119:0/539714711 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:23.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2158ac0640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21540cb170 0x7f21540c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f21592c1640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21540cad30 0x7f2154142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 -- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21540cb170 msgr2=0x7f21540c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21540cb170 0x7f21540c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 -- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21540cad30 msgr2=0x7f2154142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21540cad30 0x7f2154142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 -- 172.21.15.119:0/539714711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f214c0094d0 con 0x7f21541447a0 2023-11-08T19:25:23.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.462+0000 7f2159ac2640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 0x7f215413ecb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f214c0090b0 tx=0x7f214c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.463+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c03ac60 con 0x7f21541447a0 2023-11-08T19:25:23.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.463+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c03adc0 con 0x7f21541447a0 2023-11-08T19:25:23.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.463+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c044630 con 0x7f21541447a0 2023-11-08T19:25:23.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.463+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21540c6f60 con 0x7f21541447a0 2023-11-08T19:25:23.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.463+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21540c6590 con 0x7f21541447a0 2023-11-08T19:25:23.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.464+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21540c7760 con 0x7f21541447a0 2023-11-08T19:25:23.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.465+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f214c04d450 con 0x7f21541447a0 2023-11-08T19:25:23.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.466+0000 7f2151ffb640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f214006ae70 0x7f214006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.466+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f214c0abe20 con 0x7f21541447a0 2023-11-08T19:25:23.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.466+0000 7f21592c1640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f214006ae70 0x7f214006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.467+0000 7f21592c1640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f214006ae70 0x7f214006d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21540ce7a0 tx=0x7f2144003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.470+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21540c7760 con 0x7f21541447a0 2023-11-08T19:25:23.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.659+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21540c7760 con 0x7f21541447a0 2023-11-08T19:25:23.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.663+0000 7f2151ffb640 1 -- 172.21.15.119:0/539714711 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f21540c7760 con 0x7f21541447a0 2023-11-08T19:25:23.664 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:23.665 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:23.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f214006ae70 msgr2=0x7f214006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f214006ae70 0x7f214006d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f21540ce7a0 tx=0x7f2144003990 comp rx=0 tx=0).stop 2023-11-08T19:25:23.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 msgr2=0x7f215413ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 0x7f215413ecb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f214c0090b0 tx=0x7f214c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 shutdown_connections 2023-11-08T19:25:23.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f214006ae70 0x7f214006d330 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21541447a0 0x7f215413ecb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21540cb170 0x7f21540c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 --2- 172.21.15.119:0/539714711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21540cad30 0x7f2154142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 >> 172.21.15.119:0/539714711 conn(0x7f21540cb610 msgr2=0x7f21540b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.665+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 shutdown_connections 2023-11-08T19:25:23.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.666+0000 7f215b525640 1 -- 172.21.15.119:0/539714711 wait complete. 2023-11-08T19:25:23.686 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-08T19:25:23.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:23.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.889+0000 7f97aaa4b640 1 Processor -- start 2023-11-08T19:25:23.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.889+0000 7f97aaa4b640 1 -- start start 2023-11-08T19:25:23.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97aaa4b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97a40cade0 0x7f97a40cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97aaa4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cf670 0x7f97a406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97aaa4b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a40ced30 con 0x7f97a406a660 2023-11-08T19:25:23.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97aaa4b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a406aaf0 con 0x7f97a40cade0 2023-11-08T19:25:23.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97aaa4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40ce120 con 0x7f97a40cf670 2023-11-08T19:25:23.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97a3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97a40cade0 0x7f97a40cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97a3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f97a40cade0 0x7f97a40cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51012/0 (socket says 172.21.15.119:51012) 2023-11-08T19:25:23.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97a3fff640 1 -- 172.21.15.119:0/52783997 learned_addr learned my addr 172.21.15.119:0/52783997 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:23.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.891+0000 7f97a8fe8640 1 --1- 172.21.15.119:0/52783997 >> v1:172.21.15.119:6789/0 conn(0x7f97a406a660 0x7f97a415c980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59982/0 (socket says 172.21.15.119:59982) 2023-11-08T19:25:23.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.892+0000 7f979bfff640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cf670 0x7f97a406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.892+0000 7f97a3fff640 1 -- 172.21.15.119:0/52783997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cf670 msgr2=0x7f97a406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:23.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.892+0000 7f97a3fff640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cf670 0x7f97a406a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.892+0000 7f97a3fff640 1 -- 172.21.15.119:0/52783997 >> v1:172.21.15.119:6789/0 conn(0x7f97a406a660 legacy=0x7f97a415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.892+0000 7f97a3fff640 1 -- 172.21.15.119:0/52783997 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a40ce400 con 0x7f97a40cade0 2023-11-08T19:25:23.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.892+0000 7f979bfff640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cf670 0x7f97a406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:23.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.893+0000 7f97a3fff640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40cade0 0x7f97a40cd8d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9794009870 tx=0x7f979402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f26ad8e0bfedac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97a37fe640 1 -- 172.21.15.119:0/52783997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979402f830 con 0x7f97a40cade0 2023-11-08T19:25:23.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97a37fe640 1 -- 172.21.15.119:0/52783997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979402f990 con 0x7f97a40cade0 2023-11-08T19:25:23.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97a37fe640 1 -- 172.21.15.119:0/52783997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979402fd20 con 0x7f97a40cade0 2023-11-08T19:25:23.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 -- 172.21.15.119:0/52783997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40cade0 msgr2=0x7f97a40cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40cade0 0x7f97a40cd8d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9794009870 tx=0x7f979402ba10 comp rx=0 tx=0).stop 2023-11-08T19:25:23.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 -- 172.21.15.119:0/52783997 shutdown_connections 2023-11-08T19:25:23.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cf670 0x7f97a406a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/52783997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a40cade0 0x7f97a40cd8d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 -- 172.21.15.119:0/52783997 >> 172.21.15.119:0/52783997 conn(0x7f97a40cfd00 msgr2=0x7f97a40d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:23.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 -- 172.21.15.119:0/52783997 shutdown_connections 2023-11-08T19:25:23.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.894+0000 7f97aaa4b640 1 -- 172.21.15.119:0/52783997 wait complete. 2023-11-08T19:25:23.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.896+0000 7f97aaa4b640 1 Processor -- start 2023-11-08T19:25:23.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.896+0000 7f97aaa4b640 1 -- start start 2023-11-08T19:25:23.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.897+0000 7f97aaa4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 0x7f97a40c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.897+0000 7f97aaa4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cade0 0x7f97a41f2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.897+0000 7f97aaa4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a40cf670 0x7f97a41f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.897+0000 7f97aaa4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40b4f70 con 0x7f97a406a660 2023-11-08T19:25:23.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.897+0000 7f97aaa4b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40b5250 con 0x7f97a40cf670 2023-11-08T19:25:23.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.897+0000 7f97aaa4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a40c7880 con 0x7f97a40cade0 2023-11-08T19:25:23.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 0x7f97a40c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 0x7f97a40c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51018/0 (socket says 172.21.15.119:51018) 2023-11-08T19:25:23.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 -- 172.21.15.119:0/1178358700 learned_addr learned my addr 172.21.15.119:0/1178358700 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:23.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f979bfff640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cade0 0x7f97a41f2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a8fe8640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a40cf670 0x7f97a41f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 -- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cade0 msgr2=0x7f97a41f2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cade0 0x7f97a41f2980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 -- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a40cf670 msgr2=0x7f97a41f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:23.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a40cf670 0x7f97a41f2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:23.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 -- 172.21.15.119:0/1178358700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97940094d0 con 0x7f97a406a660 2023-11-08T19:25:23.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.898+0000 7f97a3fff640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 0x7f97a40c8360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9794009870 tx=0x7f9794004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.899+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794037c90 con 0x7f97a406a660 2023-11-08T19:25:23.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.899+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9794037df0 con 0x7f97a406a660 2023-11-08T19:25:23.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.899+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9794041640 con 0x7f97a406a660 2023-11-08T19:25:23.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.899+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a40c7b60 con 0x7f97a406a660 2023-11-08T19:25:23.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.899+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a40b4c60 con 0x7f97a406a660 2023-11-08T19:25:23.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.900+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a40c8080 con 0x7f97a406a660 2023-11-08T19:25:23.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.901+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97940417a0 con 0x7f97a406a660 2023-11-08T19:25:23.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.902+0000 7f97a17fa640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f978406ae70 0x7f978406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:23.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.902+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f979403c070 con 0x7f97a406a660 2023-11-08T19:25:23.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.902+0000 7f979bfff640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f978406ae70 0x7f978406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:23.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.903+0000 7f979bfff640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f978406ae70 0x7f978406d330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97a40c3280 tx=0x7f978c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:23.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:23.906+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f97a40c8080 con 0x7f97a406a660 2023-11-08T19:25:24.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.097+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97a40c8080 con 0x7f97a406a660 2023-11-08T19:25:24.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.099+0000 7f97a17fa640 1 -- 172.21.15.119:0/1178358700 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f97a40c8080 con 0x7f97a406a660 2023-11-08T19:25:24.101 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:24.101 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:24.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f978406ae70 msgr2=0x7f978406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f978406ae70 0x7f978406d330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97a40c3280 tx=0x7f978c008040 comp rx=0 tx=0).stop 2023-11-08T19:25:24.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 msgr2=0x7f97a40c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 0x7f97a40c8360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9794009870 tx=0x7f9794004730 comp rx=0 tx=0).stop 2023-11-08T19:25:24.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 shutdown_connections 2023-11-08T19:25:24.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f978406ae70 0x7f978406d330 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97a40cf670 0x7f97a41f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97a40cade0 0x7f97a41f2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 --2- 172.21.15.119:0/1178358700 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97a406a660 0x7f97a40c8360 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 >> 172.21.15.119:0/1178358700 conn(0x7f97a40cfd00 msgr2=0x7f97a406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 shutdown_connections 2023-11-08T19:25:24.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.102+0000 7f97aaa4b640 1 -- 172.21.15.119:0/1178358700 wait complete. 2023-11-08T19:25:24.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-08T19:25:24.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-08T19:25:24.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-08T19:25:24.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-08T19:25:24.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-08T19:25:24.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-08T19:25:24.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-08T19:25:24.127 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-08T19:25:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.423+0000 7f82a5755640 1 Processor -- start 2023-11-08T19:25:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.423+0000 7f82a5755640 1 -- start start 2023-11-08T19:25:24.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f82a5755640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82a00b3de0 0x7f82a01602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f82a5755640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f82a0160830 0x7f82a00b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f82a5755640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a00d16e0 con 0x7f82a00cfb80 2023-11-08T19:25:24.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f82a5755640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00d0990 con 0x7f82a00b3de0 2023-11-08T19:25:24.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f82a5755640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00d1980 con 0x7f82a0160830 2023-11-08T19:25:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f829f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82a00b3de0 0x7f82a01602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f829f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82a00b3de0 0x7f82a01602f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51024/0 (socket says 172.21.15.119:51024) 2023-11-08T19:25:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.425+0000 7f829f7fe640 1 -- 172.21.15.119:0/1249904071 learned_addr learned my addr 172.21.15.119:0/1249904071 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.426+0000 7f82a4f54640 1 --2- 172.21.15.119:0/1249904071 >> v2:172.21.15.188:3300/0 conn(0x7f82a0160830 0x7f82a00b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.426+0000 7f829f7fe640 1 -- 172.21.15.119:0/1249904071 >> v2:172.21.15.188:3300/0 conn(0x7f82a0160830 msgr2=0x7f82a00b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.426+0000 7f829f7fe640 1 --2- 172.21.15.119:0/1249904071 >> v2:172.21.15.188:3300/0 conn(0x7f82a0160830 0x7f82a00b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.426+0000 7f829f7fe640 1 -- 172.21.15.119:0/1249904071 >> v1:172.21.15.188:6789/0 conn(0x7f82a00cfb80 legacy=0x7f82a00cff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.426+0000 7f829f7fe640 1 -- 172.21.15.119:0/1249904071 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a00d1c60 con 0x7f82a00b3de0 2023-11-08T19:25:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.428+0000 7f829f7fe640 1 --2- 172.21.15.119:0/1249904071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00b3de0 0x7f82a01602f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f82a008d420 tx=0x7f829002bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fddb41409557ba server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.428+0000 7f829effd640 1 -- 172.21.15.119:0/1249904071 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829002caa0 con 0x7f82a00b3de0 2023-11-08T19:25:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.429+0000 7f829effd640 1 -- 172.21.15.119:0/1249904071 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829002cc00 con 0x7f82a00b3de0 2023-11-08T19:25:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.429+0000 7f829effd640 1 -- 172.21.15.119:0/1249904071 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900343f0 con 0x7f82a00b3de0 2023-11-08T19:25:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.429+0000 7feeba5b8640 1 Processor -- start 2023-11-08T19:25:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 -- 172.21.15.119:0/1249904071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00b3de0 msgr2=0x7f82a01602f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 --2- 172.21.15.119:0/1249904071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00b3de0 0x7f82a01602f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f82a008d420 tx=0x7f829002bf00 comp rx=0 tx=0).stop 2023-11-08T19:25:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.429+0000 7fa331b1a640 1 Processor -- start 2023-11-08T19:25:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.429+0000 7fa331b1a640 1 -- start start 2023-11-08T19:25:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 -- 172.21.15.119:0/1249904071 shutdown_connections 2023-11-08T19:25:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 --2- 172.21.15.119:0/1249904071 >> v2:172.21.15.188:3300/0 conn(0x7f82a0160830 0x7f82a00b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 --2- 172.21.15.119:0/1249904071 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00b3de0 0x7f82a01602f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 -- 172.21.15.119:0/1249904071 >> 172.21.15.119:0/1249904071 conn(0x7f82a00d2f60 msgr2=0x7f82a00d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.429+0000 7feeba5b8640 1 -- start start 2023-11-08T19:25:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.430+0000 7f82a5755640 1 -- 172.21.15.119:0/1249904071 shutdown_connections 2023-11-08T19:25:24.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.431+0000 7f82a5755640 1 -- 172.21.15.119:0/1249904071 wait complete. 2023-11-08T19:25:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.431+0000 7fa331b1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3240e34a0 0x7fa3240543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.431+0000 7fa331b1a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3240582c0 0x7fa324056cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.431+0000 7fa331b1a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa324058dd0 con 0x7fa324057230 2023-11-08T19:25:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.431+0000 7fa331b1a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa324057b90 con 0x7fa3240582c0 2023-11-08T19:25:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.431+0000 7fa331b1a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa324057e70 con 0x7fa3240e34a0 2023-11-08T19:25:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.432+0000 7fa330b18640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3240e34a0 0x7fa3240543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.432+0000 7fa330b18640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa3240e34a0 0x7fa3240543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56678/0 (socket says 172.21.15.119:56678) 2023-11-08T19:25:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.432+0000 7fa330b18640 1 -- 172.21.15.119:0/1376447329 learned_addr learned my addr 172.21.15.119:0/1376447329 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.432+0000 7fa32bfff640 1 --2- 172.21.15.119:0/1376447329 >> v2:172.21.15.119:3300/0 conn(0x7fa3240582c0 0x7fa324056cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.432+0000 7feeba5b8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feeac0543c0 0x7feeac039140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.432+0000 7feeb95b6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feeac055400 0x7feeac0557e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60000/0 (socket says 172.21.15.119:60000) 2023-11-08T19:25:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.433+0000 7feeb95b6640 1 -- 172.21.15.119:0/1523423730 learned_addr learned my addr 172.21.15.119:0/1523423730 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.433+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0532c0 con 0x7feeac055400 2023-11-08T19:25:24.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.433+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feeac0e8e10 con 0x7feeac049ee0 2023-11-08T19:25:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feeac0e8920 con 0x7feeac0543c0 2023-11-08T19:25:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.433+0000 7fa32b7fe640 1 -- 172.21.15.119:0/1376447329 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921251679 0 0) 0x7fa324058dd0 con 0x7fa324057230 2023-11-08T19:25:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7fa32b7fe640 1 -- 172.21.15.119:0/1376447329 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa31c003520 con 0x7fa324057230 2023-11-08T19:25:24.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7fa32bfff640 1 -- 172.21.15.119:0/1376447329 >> v2:172.21.15.188:3300/0 conn(0x7fa3240e34a0 msgr2=0x7fa3240543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7fa32bfff640 1 --2- 172.21.15.119:0/1376447329 >> v2:172.21.15.188:3300/0 conn(0x7fa3240e34a0 0x7fa3240543f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7fa32bfff640 1 -- 172.21.15.119:0/1376447329 >> v1:172.21.15.119:6789/0 conn(0x7fa324057230 legacy=0x7fa32403c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7fa32bfff640 1 -- 172.21.15.119:0/1376447329 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324053810 con 0x7fa3240582c0 2023-11-08T19:25:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7feeb8db5640 1 --2- 172.21.15.119:0/1523423730 >> v2:172.21.15.119:3300/0 conn(0x7feeac0543c0 0x7feeac039140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.434+0000 7feeabfff640 1 -- 172.21.15.119:0/1523423730 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2421091550 0 0) 0x7feeac0532c0 con 0x7feeac055400 2023-11-08T19:25:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.435+0000 7feeabfff640 1 -- 172.21.15.119:0/1523423730 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea0003520 con 0x7feeac055400 2023-11-08T19:25:24.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.435+0000 7feeb8db5640 1 -- 172.21.15.119:0/1523423730 >> v1:172.21.15.188:6789/0 conn(0x7feeac049ee0 legacy=0x7feeac051930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:25:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.435+0000 7feeb8db5640 1 -- 172.21.15.119:0/1523423730 >> v1:172.21.15.119:6789/0 conn(0x7feeac055400 legacy=0x7feeac0557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.435+0000 7feeb8db5640 1 -- 172.21.15.119:0/1523423730 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0e8b30 con 0x7feeac0543c0 2023-11-08T19:25:24.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.435+0000 7fa32bfff640 1 --2- 172.21.15.119:0/1376447329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3240582c0 0x7fa324056cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097a0 tx=0x7fa32002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64ed4b489a4a1de server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.433+0000 7f82a5755640 1 Processor -- start 2023-11-08T19:25:24.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.436+0000 7fa32b7fe640 1 -- 172.21.15.119:0/1376447329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200317c0 con 0x7fa3240582c0 2023-11-08T19:25:24.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.436+0000 7fa32b7fe640 1 -- 172.21.15.119:0/1376447329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa320031920 con 0x7fa3240582c0 2023-11-08T19:25:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.436+0000 7fa32b7fe640 1 -- 172.21.15.119:0/1376447329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa320031c90 con 0x7fa3240582c0 2023-11-08T19:25:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.437+0000 7fa331b1a640 1 -- 172.21.15.119:0/1376447329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3240582c0 msgr2=0x7fa324056cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.437+0000 7fa331b1a640 1 --2- 172.21.15.119:0/1376447329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3240582c0 0x7fa324056cf0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fa3200097a0 tx=0x7fa32002b860 comp rx=0 tx=0).stop 2023-11-08T19:25:24.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.437+0000 7feeb8db5640 1 --2- 172.21.15.119:0/1523423730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac039140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009a00 tx=0x7feeb002ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5aee125e1c70312 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.437+0000 7feeabfff640 1 -- 172.21.15.119:0/1523423730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4001670 con 0x7feeac0543c0 2023-11-08T19:25:24.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7feeabfff640 1 -- 172.21.15.119:0/1523423730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb0030440 con 0x7feeac0543c0 2023-11-08T19:25:24.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7feeabfff640 1 -- 172.21.15.119:0/1523423730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4001670 con 0x7feeac0543c0 2023-11-08T19:25:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7fa331b1a640 1 -- 172.21.15.119:0/1376447329 shutdown_connections 2023-11-08T19:25:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7fa331b1a640 1 --2- 172.21.15.119:0/1376447329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3240582c0 0x7fa324056cf0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7fa331b1a640 1 --2- 172.21.15.119:0/1376447329 >> v2:172.21.15.188:3300/0 conn(0x7fa3240e34a0 0x7fa3240543f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7fa331b1a640 1 -- 172.21.15.119:0/1376447329 >> 172.21.15.119:0/1376447329 conn(0x7fa324059120 msgr2=0x7fa324059530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.436+0000 7f82a5755640 1 -- start start 2023-11-08T19:25:24.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7f82a5755640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00cfb80 0x7f82a00ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7f82a5755640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a0160830 0x7f82a01f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7f82a5755640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a01fa500 0x7f82a00b7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7f82a5755640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a01f6e40 con 0x7f82a00cfb80 2023-11-08T19:25:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7f82a5755640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00ca740 con 0x7f82a01fa500 2023-11-08T19:25:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7f82a5755640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00caa20 con 0x7f82a0160830 2023-11-08T19:25:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7fa331b1a640 1 -- 172.21.15.119:0/1376447329 shutdown_connections 2023-11-08T19:25:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.438+0000 7fa331b1a640 1 -- 172.21.15.119:0/1376447329 wait complete. 2023-11-08T19:25:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.439+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 msgr2=0x7feeac039140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.439+0000 7feeba5b8640 1 --2- 172.21.15.119:0/1523423730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac039140 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7feeb0009a00 tx=0x7feeb002ba10 comp rx=0 tx=0).stop 2023-11-08T19:25:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 shutdown_connections 2023-11-08T19:25:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7feeba5b8640 1 --2- 172.21.15.119:0/1523423730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac039140 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 >> 172.21.15.119:0/1523423730 conn(0x7feeac0538a0 msgr2=0x7feeac053cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 shutdown_connections 2023-11-08T19:25:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7feeba5b8640 1 -- 172.21.15.119:0/1523423730 wait complete. 2023-11-08T19:25:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.439+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a0160830 0x7f82a01f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.439+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a0160830 0x7f82a01f6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44558/0 (socket says 172.21.15.119:44558) 2023-11-08T19:25:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.439+0000 7f829f7fe640 1 -- 172.21.15.119:0/1652763638 learned_addr learned my addr 172.21.15.119:0/1652763638 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7f829ffff640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00cfb80 0x7f82a00ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7f829ffff640 1 -- 172.21.15.119:0/1652763638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a0160830 msgr2=0x7f82a01f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7f829ffff640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82a0160830 0x7f82a01f6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7f829ffff640 1 -- 172.21.15.119:0/1652763638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a01fa500 msgr2=0x7f82a00b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7f829ffff640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82a01fa500 0x7f82a00b7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.440+0000 7f829ffff640 1 -- 172.21.15.119:0/1652763638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009b30 con 0x7f82a00cfb80 2023-11-08T19:25:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.441+0000 7fa331b1a640 1 Processor -- start 2023-11-08T19:25:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.441+0000 7fa331b1a640 1 -- start start 2023-11-08T19:25:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.441+0000 7f829ffff640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00cfb80 0x7f82a00ca030 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f829800b3e0 tx=0x7f829800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.442+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298011830 con 0x7f82a00cfb80 2023-11-08T19:25:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.442+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298011990 con 0x7f82a00cfb80 2023-11-08T19:25:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.442+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298011d40 con 0x7f82a00cfb80 2023-11-08T19:25:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.443+0000 7fa331b1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa324057230 0x7fa3240e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.443+0000 7fa331b1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3240582c0 0x7fa3240e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.443+0000 7fa331b1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240e34a0 0x7fa32417d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7feeba5b8640 1 Processor -- start 2023-11-08T19:25:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.443+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00cad00 con 0x7f82a00cfb80 2023-11-08T19:25:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7feeba5b8640 1 -- start start 2023-11-08T19:25:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a00cb2c0 con 0x7f82a00cfb80 2023-11-08T19:25:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7fa331b1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240e5e40 con 0x7fa324057230 2023-11-08T19:25:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7fa331b1a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240e67e0 con 0x7fa3240e34a0 2023-11-08T19:25:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7fa331b1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240d9890 con 0x7fa3240582c0 2023-11-08T19:25:24.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3240582c0 0x7fa3240e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7feeba5b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeac049ee0 0x7feeac0df180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7feeba5b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac007e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeba5b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeac055400 0x7feeac0083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeb8db5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac007e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.443+0000 7f69ca75e640 1 Processor -- start 2023-11-08T19:25:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.443+0000 7f69ca75e640 1 -- start start 2023-11-08T19:25:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7f69ca75e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c416d8b0 0x7f69c40c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7f69ca75e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69c406ba30 0x7f69c40c9150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7f69ca75e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c416a210 con 0x7f69c416dc90 2023-11-08T19:25:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7f69ca75e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c40ca2e0 con 0x7f69c406ba30 2023-11-08T19:25:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7f69ca75e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40cd930 con 0x7f69c416d8b0 2023-11-08T19:25:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7f69c37fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69c406ba30 0x7f69c40c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7f69c37fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69c406ba30 0x7f69c40c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56702/0 (socket says 172.21.15.119:56702) 2023-11-08T19:25:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7f69c37fe640 1 -- 172.21.15.119:0/3223647252 learned_addr learned my addr 172.21.15.119:0/3223647252 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeb8db5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac007e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51062/0 (socket says 172.21.15.119:51062) 2023-11-08T19:25:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7f69c3fff640 1 --2- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c416d8b0 0x7f69c40c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeba5b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac03bb80 con 0x7feeac0543c0 2023-11-08T19:25:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeb8db5640 1 -- 172.21.15.119:0/3940629635 learned_addr learned my addr 172.21.15.119:0/3940629635 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeac0e41f0 con 0x7feeac055400 2023-11-08T19:25:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeac03b250 con 0x7feeac049ee0 2023-11-08T19:25:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7feeb9db7640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeac055400 0x7feeac0083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7fa32bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3240582c0 0x7fa3240e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44560/0 (socket says 172.21.15.119:44560) 2023-11-08T19:25:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7fa32bfff640 1 -- 172.21.15.119:0/481681981 learned_addr learned my addr 172.21.15.119:0/481681981 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.444+0000 7fa331319640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240e34a0 0x7fa32417d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.445+0000 7fa330b18640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa324057230 0x7fa3240e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7fa331319640 1 -- 172.21.15.119:0/481681981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3240582c0 msgr2=0x7fa3240e5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7fa331319640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3240582c0 0x7fa3240e5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7fa331319640 1 -- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa324057230 msgr2=0x7fa3240e38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7fa331319640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa324057230 0x7fa3240e38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7fa331319640 1 -- 172.21.15.119:0/481681981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa320009400 con 0x7fa3240e34a0 2023-11-08T19:25:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.446+0000 7fa32bfff640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3240582c0 0x7fa3240e5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7fa331319640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240e34a0 0x7fa32417d040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa32c04f700 tx=0x7fa32c072800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.447+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0674d0 con 0x7fa3240e34a0 2023-11-08T19:25:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82a00c9cf0 con 0x7f82a00cfb80 2023-11-08T19:25:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c067630 con 0x7fa3240e34a0 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c0738e0 con 0x7fa3240e34a0 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240d9b30 con 0x7fa3240e34a0 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240da0f0 con 0x7fa3240e34a0 2023-11-08T19:25:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7feeb95b6640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeac049ee0 0x7feeac0df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7f69c37fe640 1 -- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c416d8b0 msgr2=0x7f69c40c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7f69c37fe640 1 --2- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c416d8b0 0x7f69c40c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7f69c37fe640 1 -- 172.21.15.119:0/3223647252 >> v1:172.21.15.188:6789/0 conn(0x7f69c416dc90 legacy=0x7f69c40cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7f69c37fe640 1 -- 172.21.15.119:0/3223647252 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40c9930 con 0x7f69c406ba30 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7feeb8db5640 1 -- 172.21.15.119:0/3940629635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeac049ee0 msgr2=0x7feeac0df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7feeb8db5640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feeac049ee0 0x7feeac0df180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7feeb8db5640 1 -- 172.21.15.119:0/3940629635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeac055400 msgr2=0x7feeac0083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7feeb8db5640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feeac055400 0x7feeac0083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7feeb8db5640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0009400 con 0x7feeac0543c0 2023-11-08T19:25:24.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.448+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f829801a400 con 0x7f82a00cfb80 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7f829cff9640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f828c06aea0 0x7f828c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.449+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f8298089a00 con 0x7f82a00cfb80 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7f829cff9640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f828c0743e0 0x7f828c076820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 --> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f828c076ef0 con 0x7f828c0743e0 2023-11-08T19:25:24.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8298011ea0 con 0x7f82a00cfb80 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7f69c37fe640 1 --2- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406ba30 0x7f69c40c9150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009870 tx=0x7f69b402fd00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=445f969f2db633b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa3240078a0 con 0x7fa3240e34a0 2023-11-08T19:25:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7f82a4f54640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f828c0743e0 0x7f828c076820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.450+0000 7feeb8db5640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac007e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feeb002ba50 tx=0x7feeb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7f829f7fe640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f828c06aea0 0x7f828c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7fc400b78640 1 Processor -- start 2023-11-08T19:25:24.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7f69c2ffd640 1 -- 172.21.15.119:0/3223647252 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4030ab0 con 0x7f69c406ba30 2023-11-08T19:25:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7f69c2ffd640 1 -- 172.21.15.119:0/3223647252 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4030c10 con 0x7f69c406ba30 2023-11-08T19:25:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7f69c2ffd640 1 -- 172.21.15.119:0/3223647252 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4038440 con 0x7f69c406ba30 2023-11-08T19:25:24.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb003e070 con 0x7feeac0543c0 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb0039760 con 0x7feeac0543c0 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea4001670 con 0x7feeac0543c0 2023-11-08T19:25:24.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f82a4f54640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f828c0743e0 0x7f828c076820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa32c090020 con 0x7fa3240e34a0 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f69ca75e640 1 -- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406ba30 msgr2=0x7f69c40c9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f69ca75e640 1 --2- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406ba30 0x7f69c40c9150 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009870 tx=0x7f69b402fd00 comp rx=0 tx=0).stop 2023-11-08T19:25:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f69ca75e640 1 -- 172.21.15.119:0/3223647252 shutdown_connections 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f69ca75e640 1 --2- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c406ba30 0x7f69c40c9150 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f69ca75e640 1 --2- 172.21.15.119:0/3223647252 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c416d8b0 0x7f69c40c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7f69ca75e640 1 -- 172.21.15.119:0/3223647252 >> 172.21.15.119:0/3223647252 conn(0x7f69c40b07e0 msgr2=0x7f69c40b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7f69ca75e640 1 -- 172.21.15.119:0/3223647252 shutdown_connections 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7fc400b78640 1 -- start start 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7fc400b78640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3fc0cc6d0 0x7fc3fc0ccb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7f829f7fe640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f828c06aea0 0x7f828c06d360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82a015ff00 tx=0x7f8290030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7fc400b78640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc06ac60 con 0x7fc3fc161ad0 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7fc400b78640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3fc165160 con 0x7fc3fc144690 2023-11-08T19:25:24.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7fc400b78640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3fc165de0 con 0x7fc3fc0cc6d0 2023-11-08T19:25:24.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.451+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeac03b530 con 0x7feeac0543c0 2023-11-08T19:25:24.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.452+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeac03c010 con 0x7feeac0543c0 2023-11-08T19:25:24.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== osd.1 v2:172.21.15.119:6824/1204926209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f828c076ef0 con 0x7f828c0743e0 2023-11-08T19:25:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fa3297fa640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa31006aec0 0x7fa31006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fa32c104b50 con 0x7fa3240e34a0 2023-11-08T19:25:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fa3297fa640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa310074500 0x7fa310076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 --> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa310077010 con 0x7fa310074500 2023-11-08T19:25:24.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa32c104ff0 con 0x7fa3240e34a0 2023-11-08T19:25:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fa330b18640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa31006aec0 0x7fa31006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.453+0000 7f69ca75e640 1 -- 172.21.15.119:0/3223647252 wait complete. 2023-11-08T19:25:24.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fec3c161640 1 Processor -- start 2023-11-08T19:25:24.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fc3fa59c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fc3fc144690 0x7fc3fc06a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:60528/0 (socket says 172.21.15.119:60528) 2023-11-08T19:25:24.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fc3fa59c640 1 -- 172.21.15.119:0/2006252889 learned_addr learned my addr 172.21.15.119:0/2006252889 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:24.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fc3f9d9b640 1 --2- 172.21.15.119:0/2006252889 >> v2:172.21.15.188:3300/0 conn(0x7fc3fc0cc6d0 0x7fc3fc0ccb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.455+0000 7fc3f959a640 1 -- 172.21.15.119:0/2006252889 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526413570 0 0) 0x7fc3fc165160 con 0x7fc3fc144690 2023-11-08T19:25:24.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.455+0000 7fc3f959a640 1 -- 172.21.15.119:0/2006252889 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3e0003520 con 0x7fc3fc144690 2023-11-08T19:25:24.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.455+0000 7fa32bfff640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa310074500 0x7fa310076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.455+0000 7f69ca75e640 1 Processor -- start 2023-11-08T19:25:24.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/2006252889 >> v1:172.21.15.188:6789/0 conn(0x7fc3fc144690 legacy=0x7fc3fc06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.455+0000 7f68c7fff640 1 Processor -- start 2023-11-08T19:25:24.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/2006252889 >> v1:172.21.15.119:6789/0 conn(0x7fc3fc161ad0 legacy=0x7fc3fc161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/2006252889 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc165440 con 0x7fc3fc0cc6d0 2023-11-08T19:25:24.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7f69ca75e640 1 -- start start 2023-11-08T19:25:24.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7fa330b18640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa31006aec0 0x7fa31006d380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa318009c50 tx=0x7fa318009240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.454+0000 7fec3c161640 1 -- start start 2023-11-08T19:25:24.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7fec3c161640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec2c0e78e0 0x7fec2c04f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7f68c7fff640 1 -- start start 2023-11-08T19:25:24.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.456+0000 7fec39efd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec2c0e78e0 0x7fec2c04f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7fec39efd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec2c0e78e0 0x7fec2c04f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56728/0 (socket says 172.21.15.119:56728) 2023-11-08T19:25:24.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7fec39efd640 1 -- 172.21.15.119:0/2648356314 learned_addr learned my addr 172.21.15.119:0/2648356314 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7fec3c161640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb730 0x7fec2c053720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec2c050370 con 0x7fec2c0eb2f0 2023-11-08T19:25:24.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7f69ca75e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c406ba30 0x7f69c40c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.455+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fee94000fc0 con 0x7feeac0543c0 2023-11-08T19:25:24.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec2c0064b0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7f69ca75e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c416d8b0 0x7f69c4164ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7f69ca75e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69c416dc90 0x7f69c40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7f69ca75e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c406ff90 con 0x7f69c416dc90 2023-11-08T19:25:24.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7f69ca75e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40c61f0 con 0x7f69c416d8b0 2023-11-08T19:25:24.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7f69ca75e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c416b000 con 0x7f69c406ba30 2023-11-08T19:25:24.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.457+0000 7fec3a6fe640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb730 0x7fec2c053720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c007950 con 0x7fec2c0eb730 2023-11-08T19:25:24.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c416d8b0 0x7f69c4164ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c416d8b0 0x7f69c4164ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56740/0 (socket says 172.21.15.119:56740) 2023-11-08T19:25:24.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7f69c37fe640 1 -- 172.21.15.119:0/3152875763 learned_addr learned my addr 172.21.15.119:0/3152875763 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7fc3f9d9b640 1 --2- 172.21.15.119:0/2006252889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 0x7fc3fc0ccb40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80097a0 tx=0x7fc3e802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a87acc888a2bb2f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7f69c3fff640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c406ba30 0x7f69c40c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7f69c8cfb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69c416dc90 0x7f69c40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.458+0000 7fa32bfff640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa310074500 0x7fa310076940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fc3f959a640 1 -- 172.21.15.119:0/2006252889 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8031640 con 0x7fc3fc0cc6d0 2023-11-08T19:25:24.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fec39efd640 1 -- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb730 msgr2=0x7fec2c053720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fec39efd640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb730 0x7fec2c053720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fec38efb640 1 -- 172.21.15.119:0/2648356314 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3090394837 0 0) 0x7fec2c050370 con 0x7fec2c0eb2f0 2023-11-08T19:25:24.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f69c37fe640 1 -- 172.21.15.119:0/3152875763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c406ba30 msgr2=0x7f69c40c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f69c37fe640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69c406ba30 0x7f69c40c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f69c37fe640 1 -- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69c416dc90 msgr2=0x7f69c40c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f69c37fe640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69c416dc90 0x7f69c40c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f69c37fe640 1 -- 172.21.15.119:0/3152875763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b40094d0 con 0x7f69c416d8b0 2023-11-08T19:25:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fc3f959a640 1 -- 172.21.15.119:0/2006252889 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e80317a0 con 0x7fc3fc0cc6d0 2023-11-08T19:25:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fc3f959a640 1 -- 172.21.15.119:0/2006252889 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8031b70 con 0x7fc3fc0cc6d0 2023-11-08T19:25:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fec39efd640 1 -- 172.21.15.119:0/2648356314 >> v1:172.21.15.119:6789/0 conn(0x7fec2c0eb2f0 legacy=0x7fec2c053010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f68c7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68c81629c0 0x7f68c80b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7f68c7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68c80b1990 0x7f68c80b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c7fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c806d820 con 0x7f68c80b1990 2023-11-08T19:25:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c7fff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c80b2320 con 0x7f68c81629c0 2023-11-08T19:25:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c7fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c8161b70 con 0x7f68c8161420 2023-11-08T19:25:24.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.459+0000 7fec39efd640 1 -- 172.21.15.119:0/2648356314 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c04fdb0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c67fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f68c81629c0 0x7f68c80b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feeb002e070 con 0x7feeac0543c0 2023-11-08T19:25:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f69c37fe640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c416d8b0 0x7f69c4164ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7fc400b78640 1 -- 172.21.15.119:0/2006252889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 msgr2=0x7fc3fc0ccb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68c80b1990 0x7f68c80b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7fc400b78640 1 --2- 172.21.15.119:0/2006252889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 0x7fc3fc0ccb40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc3e80097a0 tx=0x7fc3e802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f68c80b1990 0x7f68c80b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51088/0 (socket says 172.21.15.119:51088) 2023-11-08T19:25:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.460+0000 7f68c77fe640 1 -- 172.21.15.119:0/719875016 learned_addr learned my addr 172.21.15.119:0/719875016 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fc400b78640 1 -- 172.21.15.119:0/2006252889 shutdown_connections 2023-11-08T19:25:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== osd.2 v2:172.21.15.119:6809/3756217836 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fa310077010 con 0x7fa310074500 2023-11-08T19:25:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fc400b78640 1 --2- 172.21.15.119:0/2006252889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 0x7fc3fc0ccb40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fc400b78640 1 -- 172.21.15.119:0/2006252889 >> 172.21.15.119:0/2006252889 conn(0x7fc3fc0cb610 msgr2=0x7fc3fc0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4043620 con 0x7f69c416d8b0 2023-11-08T19:25:24.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b4043780 con 0x7f69c416d8b0 2023-11-08T19:25:24.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b4043b30 con 0x7f69c416d8b0 2023-11-08T19:25:24.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c416b2e0 con 0x7f69c416d8b0 2023-11-08T19:25:24.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f68c6ffd640 1 --2- 172.21.15.119:0/719875016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f68c77fe640 1 -- 172.21.15.119:0/719875016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 msgr2=0x7f68c80ae0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f68c77fe640 1 --2- 172.21.15.119:0/719875016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c40c6be0 con 0x7f69c416d8b0 2023-11-08T19:25:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fc400b78640 1 -- 172.21.15.119:0/2006252889 shutdown_connections 2023-11-08T19:25:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fc400b78640 1 -- 172.21.15.119:0/2006252889 wait complete. 2023-11-08T19:25:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f68c77fe640 1 -- 172.21.15.119:0/719875016 >> v2:172.21.15.188:3300/0 conn(0x7f68c81629c0 msgr2=0x7f68c80b3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f68c77fe640 1 --2- 172.21.15.119:0/719875016 >> v2:172.21.15.188:3300/0 conn(0x7f68c81629c0 0x7f68c80b3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7f68c77fe640 1 -- 172.21.15.119:0/719875016 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c8161e50 con 0x7f68c80b1990 2023-11-08T19:25:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.461+0000 7fec39efd640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04f290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec24009870 tx=0x7fec2402b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b7fe750e658b4de server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.462+0000 7fec38efb640 1 -- 172.21.15.119:0/2648356314 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec2402e070 con 0x7fec2c0e78e0 2023-11-08T19:25:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.462+0000 7fec38efb640 1 -- 172.21.15.119:0/2648356314 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240315a0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.462+0000 7fec38efb640 1 -- 172.21.15.119:0/2648356314 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24031930 con 0x7fec2c0e78e0 2023-11-08T19:25:24.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.463+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f69a4000fc0 con 0x7f69c416d8b0 2023-11-08T19:25:24.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.463+0000 7f68c77fe640 1 --2- 172.21.15.119:0/719875016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80b1990 0x7f68c80b1de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f68c806b0e0 tx=0x7f68c002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edfb14ffc6569699 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.463+0000 7feea9ffb640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee9806af40 0x7fee9806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.463+0000 7f68c5ffb640 1 -- 172.21.15.119:0/719875016 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c002ca80 con 0x7f68c80b1990 2023-11-08T19:25:24.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.463+0000 7f68c5ffb640 1 -- 172.21.15.119:0/719875016 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c002cbe0 con 0x7f68c80b1990 2023-11-08T19:25:24.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.463+0000 7f68c5ffb640 1 -- 172.21.15.119:0/719875016 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c00343f0 con 0x7f68c80b1990 2023-11-08T19:25:24.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7feeb95b6640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee9806af40 0x7fee9806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7fc400b78640 1 Processor -- start 2023-11-08T19:25:24.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 -- 172.21.15.119:0/719875016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80b1990 msgr2=0x7f68c80b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7feeb00ab1c0 con 0x7feeac0543c0 2023-11-08T19:25:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7fc400b78640 1 -- start start 2023-11-08T19:25:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 --2- 172.21.15.119:0/719875016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80b1990 0x7f68c80b1de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f68c806b0e0 tx=0x7f68c002bee0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7feea9ffb640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fee98074580 0x7fee980769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fee98077090 con 0x7fee98074580 2023-11-08T19:25:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7feeb00ab570 con 0x7feeac0543c0 2023-11-08T19:25:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 -- 172.21.15.119:0/719875016 shutdown_connections 2023-11-08T19:25:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 --2- 172.21.15.119:0/719875016 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c80b1990 0x7f68c80b1de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 --2- 172.21.15.119:0/719875016 >> v2:172.21.15.188:3300/0 conn(0x7f68c81629c0 0x7f68c80b3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 --2- 172.21.15.119:0/719875016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7f68c7fff640 1 -- 172.21.15.119:0/719875016 >> 172.21.15.119:0/719875016 conn(0x7f68c80b0870 msgr2=0x7f68c80b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.465+0000 7f68c7fff640 1 -- 172.21.15.119:0/719875016 shutdown_connections 2023-11-08T19:25:24.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 msgr2=0x7fec2c04f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.464+0000 7fec3c161640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04f290 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec24009870 tx=0x7fec2402b840 comp rx=0 tx=0).stop 2023-11-08T19:25:24.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.465+0000 7f68c7fff640 1 -- 172.21.15.119:0/719875016 wait complete. 2023-11-08T19:25:24.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.465+0000 7feeb9db7640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fee98074580 0x7fee980769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.465+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f69b403f040 con 0x7f69c416d8b0 2023-11-08T19:25:24.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7feeb95b6640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee9806af40 0x7fee9806d400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feea4000f60 tx=0x7feea4001bd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 shutdown_connections 2023-11-08T19:25:24.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7feeb9db7640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fee98074580 0x7fee980769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fc400b78640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 0x7fc3fc071930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fc400b78640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3fc144690 0x7fc3fc1f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fc400b78640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc161ad0 0x7fc3fc1425d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fc400b78640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc147d40 con 0x7fc3fc161ad0 2023-11-08T19:25:24.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fc400b78640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0c7f50 con 0x7fc3fc0cc6d0 2023-11-08T19:25:24.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fc400b78640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3fc0c8230 con 0x7fc3fc144690 2023-11-08T19:25:24.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fec3c161640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb730 0x7fec2c053720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fec3c161640 1 --2- 172.21.15.119:0/2648356314 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04f290 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.466+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 >> 172.21.15.119:0/2648356314 conn(0x7fec2c054c40 msgr2=0x7fec2c055020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 shutdown_connections 2023-11-08T19:25:24.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7fc3fad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc161ad0 0x7fc3fc1425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7f69c0ff9640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f69b006aec0 0x7f69b006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f69b40b02b0 con 0x7f69c416d8b0 2023-11-08T19:25:24.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7f69c0ff9640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f69b0074500 0x7f69b0076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 --> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f69b0077010 con 0x7f69b0074500 2023-11-08T19:25:24.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b40b0750 con 0x7f69c416d8b0 2023-11-08T19:25:24.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7fc3fa59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 0x7fc3fc071930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7fc3f9d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3fc144690 0x7fc3fc1f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.468+0000 7fc3f9d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3fc144690 0x7fc3fc1f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44602/0 (socket says 172.21.15.119:44602) 2023-11-08T19:25:24.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.468+0000 7f69c3fff640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f69b006aec0 0x7f69b006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.468+0000 7fe9baefc640 1 Processor -- start 2023-11-08T19:25:24.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.468+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== osd.0 v2:172.21.15.119:6800/337690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fee98077090 con 0x7fee98074580 2023-11-08T19:25:24.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.468+0000 7f69c8cfb640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f69b0074500 0x7f69b0076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.468+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/710270066 learned_addr learned my addr 172.21.15.119:0/710270066 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.469+0000 7f69c8cfb640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f69b0074500 0x7f69b0076940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.469+0000 7f69c3fff640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f69b006aec0 0x7f69b006d380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006130 tx=0x7f69bc00f040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.469+0000 7f68c7fff640 1 Processor -- start 2023-11-08T19:25:24.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.469+0000 7f68c7fff640 1 -- start start 2023-11-08T19:25:24.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.467+0000 7fec3c161640 1 -- 172.21.15.119:0/2648356314 wait complete. 2023-11-08T19:25:24.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.470+0000 7fec3c161640 1 Processor -- start 2023-11-08T19:25:24.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.470+0000 7fec3c161640 1 -- start start 2023-11-08T19:25:24.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 msgr2=0x7fc3fc071930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7fe9baefc640 1 -- start start 2023-11-08T19:25:24.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7fc3f9d9b640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3fc0cc6d0 0x7fc3fc071930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc161ad0 msgr2=0x7fc3fc1425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68c80b1990 0x7f68c814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7fc3f9d9b640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc161ad0 0x7fc3fc1425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7f68c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c81629c0 0x7f68c80acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fc3f9d9b640 1 -- 172.21.15.119:0/710270066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e8009400 con 0x7fc3fc144690 2023-11-08T19:25:24.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c7fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c80b04d0 con 0x7f68c81629c0 2023-11-08T19:25:24.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fec3c161640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c7fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c814cef0 con 0x7f68c80b1990 2023-11-08T19:25:24.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c7fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c814dd50 con 0x7f68c8161420 2023-11-08T19:25:24.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c6ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68c80b1990 0x7f68c814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fc3fad9d640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3fc161ad0 0x7fc3fc1425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:24.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c67fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c81629c0 0x7f68c80acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c67fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44604/0 (socket says 172.21.15.119:44604) 2023-11-08T19:25:24.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7f68c67fc640 1 -- 172.21.15.119:0/296392982 learned_addr learned my addr 172.21.15.119:0/296392982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fec3c161640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb2f0 0x7fec2c0e6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fe9baefc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9b4143600 0x7fe9b40c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fe9baefc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe9b415d9f0 0x7fe9b415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec3c161640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2c0eb730 0x7fec2c0e0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec39efd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec39efd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56784/0 (socket says 172.21.15.119:56784) 2023-11-08T19:25:24.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fe9baefc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b415e440 con 0x7fe9b406cb40 2023-11-08T19:25:24.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fe9baefc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40c4f50 con 0x7fe9b4143600 2023-11-08T19:25:24.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.472+0000 7fe9baefc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b40c4960 con 0x7fe9b415d9f0 2023-11-08T19:25:24.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7f68c77fe640 1 -- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 msgr2=0x7f68c80ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7f68c77fe640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68c8161420 0x7f68c80ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7f68c77fe640 1 -- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68c80b1990 msgr2=0x7f68c814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7f68c77fe640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f68c80b1990 0x7f68c814c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec396fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb2f0 0x7fec2c0e6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec396fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb2f0 0x7fec2c0e6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44620/0 (socket says 172.21.15.119:44620) 2023-11-08T19:25:24.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fe9b8c98640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe9b4143600 0x7fe9b40c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fe9b3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe9b415d9f0 0x7fe9b415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec39efd640 1 -- 172.21.15.119:0/2896410157 learned_addr learned my addr 172.21.15.119:0/2896410157 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fe9b3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe9b415d9f0 0x7fe9b415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56792/0 (socket says 172.21.15.119:56792) 2023-11-08T19:25:24.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fe9b3fff640 1 -- 172.21.15.119:0/1615697243 learned_addr learned my addr 172.21.15.119:0/1615697243 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0e4510 con 0x7fec2c0eb730 2023-11-08T19:25:24.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c0e70a0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec2c04af10 con 0x7fec2c0eb2f0 2023-11-08T19:25:24.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.473+0000 7f68c77fe640 1 -- 172.21.15.119:0/296392982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0009b30 con 0x7f68c81629c0 2023-11-08T19:25:24.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.471+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== osd.3 v2:172.21.15.119:6808/2410755625 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f69b0077010 con 0x7f69b0074500 2023-11-08T19:25:24.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7f68c77fe640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c81629c0 0x7f68c80acfc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68c806b340 tx=0x7f68c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fec3a6fe640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2c0eb730 0x7fec2c0e0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fc3f9d9b640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3fc144690 0x7fc3fc1f9db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3e802b930 tx=0x7fc3e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fe9b3fff640 1 -- 172.21.15.119:0/1615697243 >> v2:172.21.15.119:3300/0 conn(0x7fe9b4143600 msgr2=0x7fe9b40c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fe9b3fff640 1 --2- 172.21.15.119:0/1615697243 >> v2:172.21.15.119:3300/0 conn(0x7fe9b4143600 0x7fe9b40c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fe9b3fff640 1 -- 172.21.15.119:0/1615697243 >> v1:172.21.15.119:6789/0 conn(0x7fe9b406cb40 legacy=0x7fe9b406cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e803abe0 con 0x7fc3fc144690 2023-11-08T19:25:24.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e803ad60 con 0x7fc3fc144690 2023-11-08T19:25:24.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3fc0c8510 con 0x7fc3fc144690 2023-11-08T19:25:24.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c003e450 con 0x7f68c81629c0 2023-11-08T19:25:24.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c003e5b0 con 0x7f68c81629c0 2023-11-08T19:25:24.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c003e9a0 con 0x7f68c81629c0 2023-11-08T19:25:24.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8044630 con 0x7fc3fc144690 2023-11-08T19:25:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c814d140 con 0x7f68c81629c0 2023-11-08T19:25:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.475+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c814d700 con 0x7f68c81629c0 2023-11-08T19:25:24.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.474+0000 7fe9b3fff640 1 -- 172.21.15.119:0/1615697243 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b415e720 con 0x7fe9b415d9f0 2023-11-08T19:25:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fe9b3fff640 1 --2- 172.21.15.119:0/1615697243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b415d9f0 0x7fe9b415de60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40c91e0 tx=0x7fe9a002b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb52ef1416546d5c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fec39efd640 1 -- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb2f0 msgr2=0x7fec2c0e6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fec39efd640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec2c0eb2f0 0x7fec2c0e6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fec39efd640 1 -- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2c0eb730 msgr2=0x7fec2c0e0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fec39efd640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2c0eb730 0x7fec2c0e0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fec39efd640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240094d0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fe9b37fe640 1 -- 172.21.15.119:0/1615697243 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0031880 con 0x7fe9b415d9f0 2023-11-08T19:25:24.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fe9b37fe640 1 -- 172.21.15.119:0/1615697243 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a00319e0 con 0x7fe9b415d9f0 2023-11-08T19:25:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fe9b37fe640 1 -- 172.21.15.119:0/1615697243 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0031d70 con 0x7fe9b415d9f0 2023-11-08T19:25:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.476+0000 7fec3a6fe640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec2c0eb730 0x7fec2c0e0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:24.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fec39efd640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04a880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fec240090b0 tx=0x7fec240046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3fc1fa2f0 con 0x7fc3fc144690 2023-11-08T19:25:24.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fe9baefc640 1 -- 172.21.15.119:0/1615697243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b415d9f0 msgr2=0x7fe9b415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fe9baefc640 1 --2- 172.21.15.119:0/1615697243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b415d9f0 0x7fe9b415de60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40c91e0 tx=0x7fe9a002b920 comp rx=0 tx=0).stop 2023-11-08T19:25:24.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fe9baefc640 1 -- 172.21.15.119:0/1615697243 shutdown_connections 2023-11-08T19:25:24.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fe9baefc640 1 --2- 172.21.15.119:0/1615697243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b415d9f0 0x7fe9b415de60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fe9baefc640 1 --2- 172.21.15.119:0/1615697243 >> v2:172.21.15.119:3300/0 conn(0x7fe9b4143600 0x7fe9b40c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.477+0000 7fe9baefc640 1 -- 172.21.15.119:0/1615697243 >> 172.21.15.119:0/1615697243 conn(0x7fe9b40cb610 msgr2=0x7fe9b40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.478+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24032840 con 0x7fec2c0e78e0 2023-11-08T19:25:24.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.478+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240329a0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.478+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24032d50 con 0x7fec2c0e78e0 2023-11-08T19:25:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.479+0000 7fe9baefc640 1 -- 172.21.15.119:0/1615697243 shutdown_connections 2023-11-08T19:25:24.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.479+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3e802f050 con 0x7fc3fc144690 2023-11-08T19:25:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.479+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec2c04b1f0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.479+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec2c0e1220 con 0x7fec2c0e78e0 2023-11-08T19:25:24.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.480+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3fc165720 con 0x7fc3fc144690 2023-11-08T19:25:24.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.480+0000 7fe9baefc640 1 -- 172.21.15.119:0/1615697243 wait complete. 2023-11-08T19:25:24.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.478+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f68c8162410 con 0x7f68c81629c0 2023-11-08T19:25:24.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.480+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f68c002cd50 con 0x7f68c81629c0 2023-11-08T19:25:24.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.481+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec2c04f850 con 0x7fec2c0e78e0 2023-11-08T19:25:24.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.481+0000 7f68af7fe640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f68b406af40 0x7f68b406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f68c00ac200 con 0x7f68c81629c0 2023-11-08T19:25:24.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.481+0000 7fc3f2ffd640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc3e406aea0 0x7fc3e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc3e80ab630 con 0x7fc3fc144690 2023-11-08T19:25:24.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fc3f2ffd640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7fc3e4074460 0x7fc3e40768a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 --> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3e4076f70 con 0x7fc3e4074460 2023-11-08T19:25:24.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3e80adc10 con 0x7fc3fc144690 2023-11-08T19:25:24.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fe9baefc640 1 Processor -- start 2023-11-08T19:25:24.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fc3fa59c640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc3e406aea0 0x7fc3e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7fe9baefc640 1 -- start start 2023-11-08T19:25:24.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.482+0000 7f68c6ffd640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f68b406af40 0x7f68b406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.483+0000 7f68af7fe640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7f68b4074580 0x7f68b40769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.483+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 --> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f68b4077090 con 0x7f68b4074580 2023-11-08T19:25:24.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.483+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68c00ae820 con 0x7f68c81629c0 2023-11-08T19:25:24.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.483+0000 7fc3fad9d640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7fc3e4074460 0x7fc3e40768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9baefc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b406cb40 0x7fe9b4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9baefc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b4143600 0x7fe9b40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9baefc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9b415d9f0 0x7fe9b4071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9baefc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40c8a40 con 0x7fe9b406cb40 2023-11-08T19:25:24.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9baefc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40c81a0 con 0x7fe9b4143600 2023-11-08T19:25:24.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9baefc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b40c8480 con 0x7fe9b415d9f0 2023-11-08T19:25:24.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fc3fa59c640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc3e406aea0 0x7fc3e406d360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0011e0 tx=0x7fc3ec007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7f68c67fc640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7f68b4074580 0x7f68b40769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fe9b8c98640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b406cb40 0x7fe9b4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.484+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec2404e470 con 0x7fec2c0e78e0 2023-11-08T19:25:24.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7fe9b3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b4143600 0x7fe9b40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7fe9b3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b4143600 0x7fe9b40b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56800/0 (socket says 172.21.15.119:56800) 2023-11-08T19:25:24.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7fe9b3fff640 1 -- 172.21.15.119:0/94001177 learned_addr learned my addr 172.21.15.119:0/94001177 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7fe9b9499640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9b415d9f0 0x7fe9b4071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 --> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f82a00d0670 con 0x7f828c0743e0 2023-11-08T19:25:24.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fc3fad9d640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7fc3e4074460 0x7fc3e40768a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fe9b8c98640 1 -- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9b415d9f0 msgr2=0x7fe9b4071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fe9b8c98640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9b415d9f0 0x7fe9b4071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fe9b8c98640 1 -- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b4143600 msgr2=0x7fe9b40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fe9b8c98640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe9b4143600 0x7fe9b40b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fe9b8c98640 1 -- 172.21.15.119:0/94001177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a00094d0 con 0x7fe9b406cb40 2023-11-08T19:25:24.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7f829cff9640 1 -- 172.21.15.119:0/1652763638 <== osd.1 v2:172.21.15.119:6824/1204926209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8294004040 con 0x7f828c0743e0 2023-11-08T19:25:24.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fe9b8c98640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b406cb40 0x7fe9b4158b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d5f0 tx=0x7fe9a800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7f68c6ffd640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f68b406af40 0x7f68b406d400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006fd0 tx=0x7f68b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.485+0000 7fec2a7fc640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1806af40 0x7fec1806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fec240ab5d0 con 0x7fec2c0e78e0 2023-11-08T19:25:24.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fec2a7fc640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7fec18074580 0x7fec180769c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec18077090 con 0x7fec18074580 2023-11-08T19:25:24.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.486+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec240ab980 con 0x7fec2c0e78e0 2023-11-08T19:25:24.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80125f0 con 0x7fe9b406cb40 2023-11-08T19:25:24.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8012750 con 0x7fe9b406cb40 2023-11-08T19:25:24.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8012ae0 con 0x7fe9b406cb40 2023-11-08T19:25:24.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f828c0743e0 msgr2=0x7f828c076820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a5755640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f828c0743e0 0x7f828c076820 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b40c8760 con 0x7fe9b406cb40 2023-11-08T19:25:24.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b41590c0 con 0x7fe9b406cb40 2023-11-08T19:25:24.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f828c06aea0 msgr2=0x7f828c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a5755640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f828c06aea0 0x7f828c06d360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f82a015ff00 tx=0x7f8290030000 comp rx=0 tx=0).stop 2023-11-08T19:25:24.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00cfb80 msgr2=0x7f82a00ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a5755640 1 --2- 172.21.15.119:0/1652763638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82a00cfb80 0x7f82a00ca030 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f829800b3e0 tx=0x7f829800b8a0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.487+0000 7f82a4f54640 1 -- 172.21.15.119:0/1652763638 reap_dead start 2023-11-08T19:25:24.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.488+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 shutdown_connections 2023-11-08T19:25:24.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.488+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 >> 172.21.15.119:0/1652763638 conn(0x7f82a00d2f60 msgr2=0x7f82a00b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.488+0000 7fec396fc640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1806af40 0x7fec1806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.488+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 shutdown_connections 2023-11-08T19:25:24.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.488+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== osd.5 v2:172.21.15.188:6821/704788363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fc3e4076f70 con 0x7fc3e4074460 2023-11-08T19:25:24.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.488+0000 7f82a5755640 1 -- 172.21.15.119:0/1652763638 wait complete. 2023-11-08T19:25:24.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.489+0000 7fec3a6fe640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7fec18074580 0x7fec180769c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.489+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe9b40c58b0 con 0x7fe9b406cb40 2023-11-08T19:25:24.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.490+0000 7f68c67fc640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7f68b4074580 0x7f68b40769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.490+0000 7fec396fc640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1806af40 0x7fec1806d400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec2c038a30 tx=0x7fec30009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.490+0000 7fec3a6fe640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7fec18074580 0x7fec180769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.491+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9a8012c40 con 0x7fe9b406cb40 2023-11-08T19:25:24.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.492+0000 7fe9b17fa640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe99806aec0 0x7fe99806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.492+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe9a808a830 con 0x7fe9b406cb40 2023-11-08T19:25:24.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.492+0000 7fe9b17fa640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7fe998074500 0x7fe998076940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.492+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 --> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe998077010 con 0x7fe998074500 2023-11-08T19:25:24.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.492+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9a808aba0 con 0x7fe9b406cb40 2023-11-08T19:25:24.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.493+0000 7fe9b3fff640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe99806aec0 0x7fe99806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.493+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== osd.6 v2:172.21.15.188:6808/4161471119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f68b4077090 con 0x7f68b4074580 2023-11-08T19:25:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.493+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== osd.4 v2:172.21.15.188:6800/4217594962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fec18077090 con 0x7fec18074580 2023-11-08T19:25:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.494+0000 7fe9b9499640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7fe998074500 0x7fe998076940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.494+0000 7fe9b3fff640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe99806aec0 0x7fe99806d380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9a002b960 tx=0x7fe9a003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.497+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 --> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa324053dd0 con 0x7fa310074500 2023-11-08T19:25:24.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.498+0000 7fa3297fa640 1 -- 172.21.15.119:0/481681981 <== osd.2 v2:172.21.15.119:6809/3756217836 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa320043040 con 0x7fa310074500 2023-11-08T19:25:24.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.499+0000 7fe9b9499640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7fe998074500 0x7fe998076940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa310074500 msgr2=0x7fa310076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331b1a640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa310074500 0x7fa310076940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa31006aec0 msgr2=0x7fa31006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331b1a640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa31006aec0 0x7fa31006d380 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa318009c50 tx=0x7fa318009240 comp rx=0 tx=0).stop 2023-11-08T19:25:24.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240e34a0 msgr2=0x7fa32417d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331b1a640 1 --2- 172.21.15.119:0/481681981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3240e34a0 0x7fa32417d040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa32c04f700 tx=0x7fa32c072800 comp rx=0 tx=0).stop 2023-11-08T19:25:24.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.500+0000 7fa331319640 1 -- 172.21.15.119:0/481681981 reap_dead start 2023-11-08T19:25:24.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.501+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 shutdown_connections 2023-11-08T19:25:24.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.501+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 >> 172.21.15.119:0/481681981 conn(0x7fa324059120 msgr2=0x7fa3240e0450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.501+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 shutdown_connections 2023-11-08T19:25:24.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.501+0000 7fa331b1a640 1 -- 172.21.15.119:0/481681981 wait complete. 2023-11-08T19:25:24.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.502+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== osd.7 v2:172.21.15.188:6816/2104156129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fe998077010 con 0x7fe998074500 2023-11-08T19:25:24.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.503+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 --> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f69a4002cb0 con 0x7f69b0074500 2023-11-08T19:25:24.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.504+0000 7f69c0ff9640 1 -- 172.21.15.119:0/3152875763 <== osd.3 v2:172.21.15.119:6808/2410755625 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f69a4002cb0 con 0x7f69b0074500 2023-11-08T19:25:24.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.505+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f69b0074500 msgr2=0x7f69b0076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.505+0000 7f69ca75e640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f69b0074500 0x7f69b0076940 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.505+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f69b006aec0 msgr2=0x7f69b006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.505+0000 7f69ca75e640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f69b006aec0 0x7f69b006d380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f69bc006130 tx=0x7f69bc00f040 comp rx=0 tx=0).stop 2023-11-08T19:25:24.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.506+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c416d8b0 msgr2=0x7f69c4164ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.506+0000 7f69ca75e640 1 --2- 172.21.15.119:0/3152875763 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69c416d8b0 0x7f69c4164ae0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009990 tx=0x7f69b40046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.506+0000 7f69c8cfb640 1 -- 172.21.15.119:0/3152875763 reap_dead start 2023-11-08T19:25:24.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.506+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 shutdown_connections 2023-11-08T19:25:24.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.506+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 >> 172.21.15.119:0/3152875763 conn(0x7f69c40b07e0 msgr2=0x7f69c40b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.507+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 shutdown_connections 2023-11-08T19:25:24.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.507+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 --> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fee94002cb0 con 0x7fee98074580 2023-11-08T19:25:24.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.507+0000 7f69ca75e640 1 -- 172.21.15.119:0/3152875763 wait complete. 2023-11-08T19:25:24.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.508+0000 7feea9ffb640 1 -- 172.21.15.119:0/3940629635 <== osd.0 v2:172.21.15.119:6800/337690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fee94002cb0 con 0x7fee98074580 2023-11-08T19:25:24.605 INFO:teuthology.orchestra.run.smithi119.stdout:55834574859 2023-11-08T19:25:24.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-08T19:25:24.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fee98074580 msgr2=0x7fee980769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeba5b8640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fee98074580 0x7fee980769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee9806af40 msgr2=0x7fee9806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeba5b8640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fee9806af40 0x7fee9806d400 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7feea4000f60 tx=0x7feea4001bd0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 msgr2=0x7feeac007e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeba5b8640 1 --2- 172.21.15.119:0/3940629635 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feeac0543c0 0x7feeac007e70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7feeb002ba50 tx=0x7feeb00046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.514+0000 7feeb9db7640 1 -- 172.21.15.119:0/3940629635 reap_dead start 2023-11-08T19:25:24.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.515+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 shutdown_connections 2023-11-08T19:25:24.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.515+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 >> 172.21.15.119:0/3940629635 conn(0x7feeac0538a0 msgr2=0x7feeac009290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.515+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 shutdown_connections 2023-11-08T19:25:24.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.515+0000 7feeba5b8640 1 -- 172.21.15.119:0/3940629635 wait complete. 2023-11-08T19:25:24.609 INFO:teuthology.orchestra.run.smithi119.stdout:55834574860 2023-11-08T19:25:24.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.520+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 --> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3fc0cd080 con 0x7fc3e4074460 2023-11-08T19:25:24.610 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-08T19:25:24.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.521+0000 7fc3f2ffd640 1 -- 172.21.15.119:0/710270066 <== osd.5 v2:172.21.15.188:6821/704788363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc3f4001e80 con 0x7fc3e4074460 2023-11-08T19:25:24.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.521+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 --> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f68c8162130 con 0x7f68b4074580 2023-11-08T19:25:24.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7fc3e4074460 msgr2=0x7fc3e40768a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7fc3e4074460 0x7fc3e40768a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7f68af7fe640 1 -- 172.21.15.119:0/296392982 <== osd.6 v2:172.21.15.188:6808/4161471119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f68bc0097b0 con 0x7f68b4074580 2023-11-08T19:25:24.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc3e406aea0 msgr2=0x7fc3e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc3e406aea0 0x7fc3e406d360 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec0011e0 tx=0x7fc3ec007d30 comp rx=0 tx=0).stop 2023-11-08T19:25:24.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3fc144690 msgr2=0x7fc3fc1f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 --2- 172.21.15.119:0/710270066 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3fc144690 0x7fc3fc1f9db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc3e802b930 tx=0x7fc3e80046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc3fad9d640 1 -- 172.21.15.119:0/710270066 reap_dead start 2023-11-08T19:25:24.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 shutdown_connections 2023-11-08T19:25:24.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 >> 172.21.15.119:0/710270066 conn(0x7fc3fc0cb610 msgr2=0x7fc3fc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 shutdown_connections 2023-11-08T19:25:24.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.522+0000 7fc400b78640 1 -- 172.21.15.119:0/710270066 wait complete. 2023-11-08T19:25:24.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7f68b4074580 msgr2=0x7f68b40769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c7fff640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7f68b4074580 0x7f68b40769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f68b406af40 msgr2=0x7f68b406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c7fff640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f68b406af40 0x7f68b406d400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006fd0 tx=0x7f68b8008040 comp rx=0 tx=0).stop 2023-11-08T19:25:24.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c81629c0 msgr2=0x7f68c80acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c7fff640 1 --2- 172.21.15.119:0/296392982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f68c81629c0 0x7f68c80acfc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f68c806b340 tx=0x7f68c0004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:24.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.523+0000 7f68c77fe640 1 -- 172.21.15.119:0/296392982 reap_dead start 2023-11-08T19:25:24.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.524+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 shutdown_connections 2023-11-08T19:25:24.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.524+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 >> 172.21.15.119:0/296392982 conn(0x7f68c80b0870 msgr2=0x7f68c81602b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.524+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 shutdown_connections 2023-11-08T19:25:24.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.524+0000 7f68c7fff640 1 -- 172.21.15.119:0/296392982 wait complete. 2023-11-08T19:25:24.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.525+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 --> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fec2c006140 con 0x7fec18074580 2023-11-08T19:25:24.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.526+0000 7fec2a7fc640 1 -- 172.21.15.119:0/2896410157 <== osd.4 v2:172.21.15.188:6800/4217594962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fec34090040 con 0x7fec18074580 2023-11-08T19:25:24.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.527+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7fec18074580 msgr2=0x7fec180769c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.527+0000 7fec3c161640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7fec18074580 0x7fec180769c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1806af40 msgr2=0x7fec1806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3c161640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1806af40 0x7fec1806d400 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fec2c038a30 tx=0x7fec30009040 comp rx=0 tx=0).stop 2023-11-08T19:25:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 msgr2=0x7fec2c04a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3c161640 1 --2- 172.21.15.119:0/2896410157 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec2c0e78e0 0x7fec2c04a880 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fec240090b0 tx=0x7fec240046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3a6fe640 1 -- 172.21.15.119:0/2896410157 reap_dead start 2023-11-08T19:25:24.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 shutdown_connections 2023-11-08T19:25:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.528+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 >> 172.21.15.119:0/2896410157 conn(0x7fec2c054c40 msgr2=0x7fec2c04d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.529+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 shutdown_connections 2023-11-08T19:25:24.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.529+0000 7fec3c161640 1 -- 172.21.15.119:0/2896410157 wait complete. 2023-11-08T19:25:24.623 INFO:teuthology.orchestra.run.smithi119.stdout:55834574858 2023-11-08T19:25:24.623 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-08T19:25:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.532+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 --> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe9b40c4b70 con 0x7fe998074500 2023-11-08T19:25:24.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.534+0000 7fe9b17fa640 1 -- 172.21.15.119:0/94001177 <== osd.7 v2:172.21.15.188:6816/2104156129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe9ac001eb0 con 0x7fe998074500 2023-11-08T19:25:24.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.534+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7fe998074500 msgr2=0x7fe998076940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.534+0000 7fe9baefc640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7fe998074500 0x7fe998076940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.534+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe99806aec0 msgr2=0x7fe99806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.534+0000 7fe9baefc640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe99806aec0 0x7fe99806d380 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe9a002b960 tx=0x7fe9a003c040 comp rx=0 tx=0).stop 2023-11-08T19:25:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b406cb40 msgr2=0x7fe9b4158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9baefc640 1 --2- 172.21.15.119:0/94001177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe9b406cb40 0x7fe9b4158b80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d5f0 tx=0x7fe9a800dab0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9b9499640 1 -- 172.21.15.119:0/94001177 reap_dead start 2023-11-08T19:25:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 shutdown_connections 2023-11-08T19:25:24.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 >> 172.21.15.119:0/94001177 conn(0x7fe9b40cb610 msgr2=0x7fe9b40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 shutdown_connections 2023-11-08T19:25:24.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.535+0000 7fe9baefc640 1 -- 172.21.15.119:0/94001177 wait complete. 2023-11-08T19:25:24.627 INFO:teuthology.orchestra.run.smithi119.stdout:55834574859 2023-11-08T19:25:24.628 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-08T19:25:24.629 INFO:teuthology.orchestra.run.smithi119.stdout:55834574858 2023-11-08T19:25:24.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T19:25:24.630 INFO:teuthology.orchestra.run.smithi119.stdout:55834574858 2023-11-08T19:25:24.631 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-08T19:25:24.631 INFO:teuthology.orchestra.run.smithi119.stdout:55834574860 2023-11-08T19:25:24.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-08T19:25:24.633 INFO:teuthology.orchestra.run.smithi119.stdout:55834574859 2023-11-08T19:25:24.633 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T19:25:24.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.948+0000 7fd2269f5640 1 Processor -- start 2023-11-08T19:25:24.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.948+0000 7fd2269f5640 1 -- start start 2023-11-08T19:25:24.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.950+0000 7fd2269f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2180e3060 0x7fd21803ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.950+0000 7fd2269f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd218057710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.950+0000 7fd2269f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21803c060 con 0x7fd218006b60 2023-11-08T19:25:24.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.950+0000 7fd2269f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2180070e0 con 0x7fd2180e3060 2023-11-08T19:25:24.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.950+0000 7fd2269f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21803ef70 con 0x7fd218054350 2023-11-08T19:25:24.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.951+0000 7fd21f7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fd218006b60 0x7fd218053c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:60536/0 (socket says 172.21.15.119:60536) 2023-11-08T19:25:24.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.951+0000 7fd21f7fe640 1 -- 172.21.15.119:0/545731297 learned_addr learned my addr 172.21.15.119:0/545731297 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.951+0000 7fd224f92640 1 --2- 172.21.15.119:0/545731297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd218057710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.951+0000 7fd21ffff640 1 --2- 172.21.15.119:0/545731297 >> v2:172.21.15.188:3300/0 conn(0x7fd2180e3060 0x7fd21803ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.951+0000 7fd21effd640 1 -- 172.21.15.119:0/545731297 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375546487 0 0) 0x7fd21803c060 con 0x7fd218006b60 2023-11-08T19:25:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.952+0000 7fd21effd640 1 -- 172.21.15.119:0/545731297 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd210003520 con 0x7fd218006b60 2023-11-08T19:25:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.952+0000 7fd224f92640 1 -- 172.21.15.119:0/545731297 >> v2:172.21.15.188:3300/0 conn(0x7fd2180e3060 msgr2=0x7fd21803ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.952+0000 7fd224f92640 1 --2- 172.21.15.119:0/545731297 >> v2:172.21.15.188:3300/0 conn(0x7fd2180e3060 0x7fd21803ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.952+0000 7fd21effd640 1 -- 172.21.15.119:0/545731297 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 969314322 0 0) 0x7fd210003520 con 0x7fd218006b60 2023-11-08T19:25:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.952+0000 7fd224f92640 1 -- 172.21.15.119:0/545731297 >> v1:172.21.15.188:6789/0 conn(0x7fd218006b60 legacy=0x7fd218053c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.952+0000 7fd224f92640 1 -- 172.21.15.119:0/545731297 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180073a0 con 0x7fd218054350 2023-11-08T19:25:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.954+0000 7fd224f92640 1 --2- 172.21.15.119:0/545731297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd218057710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd2200698e0 tx=0x7fd2200a6560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d495205442aa0079 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.956+0000 7fd21effd640 1 -- 172.21.15.119:0/545731297 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c001670 con 0x7fd218054350 2023-11-08T19:25:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.956+0000 7fd21effd640 1 -- 172.21.15.119:0/545731297 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2200a9050 con 0x7fd218054350 2023-11-08T19:25:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.956+0000 7fd21effd640 1 -- 172.21.15.119:0/545731297 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200ad600 con 0x7fd218054350 2023-11-08T19:25:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.956+0000 7fd2269f5640 1 -- 172.21.15.119:0/545731297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 msgr2=0x7fd218057710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.956+0000 7fd2269f5640 1 --2- 172.21.15.119:0/545731297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd218057710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd2200698e0 tx=0x7fd2200a6560 comp rx=0 tx=0).stop 2023-11-08T19:25:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.957+0000 7fd2269f5640 1 -- 172.21.15.119:0/545731297 shutdown_connections 2023-11-08T19:25:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.957+0000 7fd2269f5640 1 --2- 172.21.15.119:0/545731297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd218057710 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.957+0000 7fd2269f5640 1 --2- 172.21.15.119:0/545731297 >> v2:172.21.15.188:3300/0 conn(0x7fd2180e3060 0x7fd21803ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.957+0000 7fd2269f5640 1 -- 172.21.15.119:0/545731297 >> 172.21.15.119:0/545731297 conn(0x7fd218055a80 msgr2=0x7fd218055e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.957+0000 7fd2269f5640 1 -- 172.21.15.119:0/545731297 shutdown_connections 2023-11-08T19:25:24.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.957+0000 7fd2269f5640 1 -- 172.21.15.119:0/545731297 wait complete. 2023-11-08T19:25:24.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.961+0000 7fd2269f5640 1 Processor -- start 2023-11-08T19:25:24.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.961+0000 7fd2269f5640 1 -- start start 2023-11-08T19:25:24.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.963+0000 7fd2269f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 0x7fd21804cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.963+0000 7ff361b0f640 1 Processor -- start 2023-11-08T19:25:24.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd2269f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd2180397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7ff361b0f640 1 -- start start 2023-11-08T19:25:24.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd2269f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2180e3060 0x7fd218039d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd2269f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21803a260 con 0x7fd2180e3060 2023-11-08T19:25:24.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd2269f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21803b950 con 0x7fd218006b60 2023-11-08T19:25:24.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd2269f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd218050360 con 0x7fd218054350 2023-11-08T19:25:24.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 0x7fd21804cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd21ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 0x7fd21804cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56818/0 (socket says 172.21.15.119:56818) 2023-11-08T19:25:24.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.964+0000 7fd21ffff640 1 -- 172.21.15.119:0/1982393406 learned_addr learned my addr 172.21.15.119:0/1982393406 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.965+0000 7fd21f7fe640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd2180397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.965+0000 7fd224f92640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2180e3060 0x7fd218039d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7fd21ffff640 1 -- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 msgr2=0x7fd2180397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7fd21ffff640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd2180397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7fd21ffff640 1 -- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2180e3060 msgr2=0x7fd218039d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.965+0000 7ff361b0f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0cfb30 0x7ff35c0cff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.965+0000 7ff361b0f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff35c0ce810 0x7ff35c1612c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7ff361b0f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35c0ce380 con 0x7ff35c161800 2023-11-08T19:25:24.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7ff361b0f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35c0cda70 con 0x7ff35c0ce810 2023-11-08T19:25:24.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7ff361b0f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c162030 con 0x7ff35c0cfb30 2023-11-08T19:25:24.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7fd21ffff640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2180e3060 0x7fd218039d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7fd21ffff640 1 -- 172.21.15.119:0/1982393406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2200741f0 con 0x7fd218006b60 2023-11-08T19:25:24.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7fd224f92640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2180e3060 0x7fd218039d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:24.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7f5c48b18640 1 Processor -- start 2023-11-08T19:25:24.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.966+0000 7f5c48b18640 1 -- start start 2023-11-08T19:25:24.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7ff35bfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff35c161800 0x7ff35c0d1570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:60550/0 (socket says 172.21.15.119:60550) 2023-11-08T19:25:24.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7ff35bfff640 1 -- 172.21.15.119:0/1361822739 learned_addr learned my addr 172.21.15.119:0/1361822739 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:24.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7fd21ffff640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 0x7fd21804cbc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd21400e870 tx=0x7fd21400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7ff35b7fe640 1 --2- 172.21.15.119:0/1361822739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0cfb30 0x7ff35c0cff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7f7768b65640 1 Processor -- start 2023-11-08T19:25:24.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7ff35affd640 1 --2- 172.21.15.119:0/1361822739 >> v2:172.21.15.119:3300/0 conn(0x7ff35c0ce810 0x7ff35c1612c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2140094f0 con 0x7fd218006b60 2023-11-08T19:25:24.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7ff35a7fc640 1 -- 172.21.15.119:0/1361822739 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402750342 0 0) 0x7ff35c0ce380 con 0x7ff35c161800 2023-11-08T19:25:24.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7ff35a7fc640 1 -- 172.21.15.119:0/1361822739 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff340003520 con 0x7ff35c161800 2023-11-08T19:25:24.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.967+0000 7f7768b65640 1 -- start start 2023-11-08T19:25:24.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.968+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21400c940 con 0x7fd218006b60 2023-11-08T19:25:24.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.968+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21400ccf0 con 0x7fd218006b60 2023-11-08T19:25:24.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.968+0000 7ff35b7fe640 1 -- 172.21.15.119:0/1361822739 >> v2:172.21.15.119:3300/0 conn(0x7ff35c0ce810 msgr2=0x7ff35c1612c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.968+0000 7ff35a7fc640 1 -- 172.21.15.119:0/1361822739 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 622761 0 0) 0x7ff340003520 con 0x7ff35c161800 2023-11-08T19:25:24.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7fd2269f5640 1 -- 172.21.15.119:0/1982393406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2180d2b50 con 0x7fd218006b60 2023-11-08T19:25:24.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7fd2269f5640 1 -- 172.21.15.119:0/1982393406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21803b670 con 0x7fd218006b60 2023-11-08T19:25:24.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f5c48b18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c440cfa00 0x7f5c440cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f5c48b18640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4415df70 con 0x7f5c4415d6c0 2023-11-08T19:25:24.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f5c48b18640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c440d1e60 con 0x7f5c44161160 2023-11-08T19:25:24.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f5c48b18640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4415daa0 con 0x7f5c440cfa00 2023-11-08T19:25:24.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7ff35b7fe640 1 --2- 172.21.15.119:0/1361822739 >> v2:172.21.15.119:3300/0 conn(0x7ff35c0ce810 0x7ff35c1612c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7ff35b7fe640 1 -- 172.21.15.119:0/1361822739 >> v1:172.21.15.188:6789/0 conn(0x7ff35c161800 legacy=0x7ff35c0d1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7ff35b7fe640 1 -- 172.21.15.119:0/1361822739 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35c0cdd30 con 0x7ff35c0cfb30 2023-11-08T19:25:24.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f7768b65640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77640d1260 0x7f7764160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f7768b65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f77640d1d50 0x7f7764160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f7768b65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77640ce760 con 0x7f77640b2010 2023-11-08T19:25:24.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f7768b65640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7764161000 con 0x7f77640d1260 2023-11-08T19:25:24.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.969+0000 7f7768b65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77640d21a0 con 0x7f77640d1d50 2023-11-08T19:25:24.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7ff35affd640 1 --2- 172.21.15.119:0/1361822739 >> v2:172.21.15.119:3300/0 conn(0x7ff35c0ce810 0x7ff35c1612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:24.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f776259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77640d1260 0x7f7764160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f776259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f77640d1260 0x7f7764160580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51162/0 (socket says 172.21.15.119:51162) 2023-11-08T19:25:24.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f776259c640 1 -- 172.21.15.119:0/2554972189 learned_addr learned my addr 172.21.15.119:0/2554972189 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:24.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f5c4259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5c4415d6c0 0x7f5c440d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60044/0 (socket says 172.21.15.119:60044) 2023-11-08T19:25:24.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f7762d9d640 1 --2- 172.21.15.119:0/2554972189 >> v2:172.21.15.188:3300/0 conn(0x7f77640d1d50 0x7f7764160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f5c4259c640 1 -- 172.21.15.119:0/2424172850 learned_addr learned my addr 172.21.15.119:0/2424172850 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:24.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.970+0000 7f5c41d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c440cfa00 0x7f5c440cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.971+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f8005250 con 0x7fd218006b60 2023-11-08T19:25:24.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.972+0000 7f5c41d9b640 1 -- 172.21.15.119:0/2424172850 >> v1:172.21.15.188:6789/0 conn(0x7f5c44161160 legacy=0x7f5c44161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.972+0000 7f5c41d9b640 1 -- 172.21.15.119:0/2424172850 >> v1:172.21.15.119:6789/0 conn(0x7f5c4415d6c0 legacy=0x7f5c440d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.972+0000 7f5c41d9b640 1 -- 172.21.15.119:0/2424172850 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44161d40 con 0x7f5c440cfa00 2023-11-08T19:25:24.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.972+0000 7f5c4159a640 1 -- 172.21.15.119:0/2424172850 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518472215 0 0) 0x7f5c4415df70 con 0x7f5c4415d6c0 2023-11-08T19:25:24.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.971+0000 7f776259c640 1 -- 172.21.15.119:0/2554972189 >> v2:172.21.15.188:3300/0 conn(0x7f77640d1d50 msgr2=0x7f7764160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:24.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.971+0000 7f776259c640 1 --2- 172.21.15.119:0/2554972189 >> v2:172.21.15.188:3300/0 conn(0x7f77640d1d50 0x7f7764160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.971+0000 7f776259c640 1 -- 172.21.15.119:0/2554972189 >> v1:172.21.15.188:6789/0 conn(0x7f77640b2010 legacy=0x7f77640d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.971+0000 7f776259c640 1 -- 172.21.15.119:0/2554972189 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77640b2580 con 0x7f77640d1260 2023-11-08T19:25:24.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7ff35b7fe640 1 --2- 172.21.15.119:0/1361822739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0cfb30 0x7ff35c0cff10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0097a0 tx=0x7ff34c02b710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb43bfdfaceaf58 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.973+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd214013470 con 0x7fd218006b60 2023-11-08T19:25:24.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7ff35a7fc640 1 -- 172.21.15.119:0/1361822739 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3540019f0 con 0x7ff35c0cfb30 2023-11-08T19:25:24.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7f5c41d9b640 1 --2- 172.21.15.119:0/2424172850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440cfe70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c380097a0 tx=0x7f5c3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4ed36b82fce72e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7fd21cff9640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd20406ae70 0x7fd20406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:24.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd214089980 con 0x7fd218006b60 2023-11-08T19:25:24.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7ff35a7fc640 1 -- 172.21.15.119:0/1361822739 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c02e050 con 0x7ff35c0cfb30 2023-11-08T19:25:24.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.974+0000 7ff35a7fc640 1 -- 172.21.15.119:0/1361822739 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c030710 con 0x7ff35c0cfb30 2023-11-08T19:25:24.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.975+0000 7f5c4159a640 1 -- 172.21.15.119:0/2424172850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3802e070 con 0x7f5c440cfa00 2023-11-08T19:25:24.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.975+0000 7f5c4159a640 1 -- 172.21.15.119:0/2424172850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c380315a0 con 0x7f5c440cfa00 2023-11-08T19:25:24.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.975+0000 7fd21f7fe640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd20406ae70 0x7fd20406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:24.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.975+0000 7f5c4159a640 1 -- 172.21.15.119:0/2424172850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c38031930 con 0x7f5c440cfa00 2023-11-08T19:25:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.975+0000 7ff361b0f640 1 -- 172.21.15.119:0/1361822739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0cfb30 msgr2=0x7ff35c0cff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.975+0000 7ff361b0f640 1 --2- 172.21.15.119:0/1361822739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0cfb30 0x7ff35c0cff10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0097a0 tx=0x7ff34c02b710 comp rx=0 tx=0).stop 2023-11-08T19:25:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7ff361b0f640 1 -- 172.21.15.119:0/1361822739 shutdown_connections 2023-11-08T19:25:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7ff361b0f640 1 --2- 172.21.15.119:0/1361822739 >> v2:172.21.15.119:3300/0 conn(0x7ff35c0ce810 0x7ff35c1612c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7ff361b0f640 1 --2- 172.21.15.119:0/1361822739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0cfb30 0x7ff35c0cff10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7ff361b0f640 1 -- 172.21.15.119:0/1361822739 >> 172.21.15.119:0/1361822739 conn(0x7ff35c06a9f0 msgr2=0x7ff35c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7ff361b0f640 1 -- 172.21.15.119:0/1361822739 shutdown_connections 2023-11-08T19:25:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7f5c48b18640 1 -- 172.21.15.119:0/2424172850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 msgr2=0x7f5c440cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:24.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7fd21f7fe640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd20406ae70 0x7fd20406d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2180df210 tx=0x7fd20c001da0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7f776259c640 1 --2- 172.21.15.119:0/2554972189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1260 0x7f7764160580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f776406a870 tx=0x7f774c02b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cdf7aafb462ecbd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7f5c48b18640 1 --2- 172.21.15.119:0/2424172850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440cfe70 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c380097a0 tx=0x7f5c3802b840 comp rx=0 tx=0).stop 2023-11-08T19:25:24.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5d07fff640 1 Processor -- start 2023-11-08T19:25:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5c48b18640 1 -- 172.21.15.119:0/2424172850 shutdown_connections 2023-11-08T19:25:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5c48b18640 1 --2- 172.21.15.119:0/2424172850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440cfe70 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5c48b18640 1 -- 172.21.15.119:0/2424172850 >> 172.21.15.119:0/2424172850 conn(0x7f5c4406a9f0 msgr2=0x7f5c4406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:24.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5c48b18640 1 -- 172.21.15.119:0/2424172850 shutdown_connections 2023-11-08T19:25:24.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5c48b18640 1 -- 172.21.15.119:0/2424172850 wait complete. 2023-11-08T19:25:24.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f776159a640 1 -- 172.21.15.119:0/2554972189 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c0317f0 con 0x7f77640d1260 2023-11-08T19:25:24.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f776159a640 1 -- 172.21.15.119:0/2554972189 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774c031950 con 0x7f77640d1260 2023-11-08T19:25:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f776159a640 1 -- 172.21.15.119:0/2554972189 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774c031ce0 con 0x7f77640d1260 2023-11-08T19:25:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7fe5780c9640 1 Processor -- start 2023-11-08T19:25:24.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.978+0000 7fe5780c9640 1 -- start start 2023-11-08T19:25:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.976+0000 7ff361b0f640 1 -- 172.21.15.119:0/1361822739 wait complete. 2023-11-08T19:25:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.977+0000 7f5d07fff640 1 -- start start 2023-11-08T19:25:24.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7ff361b0f640 1 Processor -- start 2023-11-08T19:25:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f5eb714c640 1 Processor -- start 2023-11-08T19:25:25.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7ff361b0f640 1 -- start start 2023-11-08T19:25:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f7768b65640 1 -- 172.21.15.119:0/2554972189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1260 msgr2=0x7f7764160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f7768b65640 1 --2- 172.21.15.119:0/2554972189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1260 0x7f7764160580 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f776406a870 tx=0x7f774c02b890 comp rx=0 tx=0).stop 2023-11-08T19:25:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f7768b65640 1 -- 172.21.15.119:0/2554972189 shutdown_connections 2023-11-08T19:25:25.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f7768b65640 1 --2- 172.21.15.119:0/2554972189 >> v2:172.21.15.188:3300/0 conn(0x7f77640d1d50 0x7f7764160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7fe5780c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 0x7fe57015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7fe5780c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5700cad30 0x7fe57006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7fe5780c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5700cb170 0x7fe57006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7fe5780c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5700c3a50 con 0x7fe5700cad30 2023-11-08T19:25:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7fe5780c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5700ce6e0 con 0x7fe5700cb170 2023-11-08T19:25:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7fe5780c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe570144b80 con 0x7fe5701447a0 2023-11-08T19:25:25.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f7768b65640 1 --2- 172.21.15.119:0/2554972189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1260 0x7f7764160580 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f7768b65640 1 -- 172.21.15.119:0/2554972189 >> 172.21.15.119:0/2554972189 conn(0x7f77640d30f0 msgr2=0x7f77640d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f5c48b18640 1 Processor -- start 2023-11-08T19:25:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f7768b65640 1 -- 172.21.15.119:0/2554972189 shutdown_connections 2023-11-08T19:25:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f7768b65640 1 -- 172.21.15.119:0/2554972189 wait complete. 2023-11-08T19:25:25.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f5c48b18640 1 -- start start 2023-11-08T19:25:25.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7fe575e65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 0x7fe57015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7fe576666640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5700cb170 0x7fe57006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7fe575e65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 0x7fe57015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44656/0 (socket says 172.21.15.119:44656) 2023-11-08T19:25:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f5d07fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d080d0d40 0x7f5d080b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f5d07fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d080b2fb0 con 0x7f5d0815d2e0 2023-11-08T19:25:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f5d07fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d080ceeb0 con 0x7f5d080d1c50 2023-11-08T19:25:25.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7f5d07fff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d080cebd0 con 0x7f5d080d0d40 2023-11-08T19:25:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7fe576666640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5700cb170 0x7fe57006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56838/0 (socket says 172.21.15.119:56838) 2023-11-08T19:25:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.980+0000 7fe575e65640 1 -- 172.21.15.119:0/2642081650 learned_addr learned my addr 172.21.15.119:0/2642081650 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7fe575664640 1 --2- 172.21.15.119:0/2642081650 >> v2:172.21.15.119:3300/0 conn(0x7fe5700cad30 0x7fe57006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7ff361b0f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 0x7ff35c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5d06ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5d0815d2e0 0x7f5d080d1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60058/0 (socket says 172.21.15.119:60058) 2023-11-08T19:25:25.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5d06ffd640 1 -- 172.21.15.119:0/4281516454 learned_addr learned my addr 172.21.15.119:0/4281516454 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.979+0000 7f5eb714c640 1 -- start start 2023-11-08T19:25:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5eb714c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5eb006a970 0x7f5eb006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5eb714c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb00c9150 0x7f5eb00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5eb714c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eb00ccc10 0x7f5eb01653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5eb714c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb01619b0 con 0x7f5eb006a970 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5eb714c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb00cd0a0 con 0x7f5eb00ccc10 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5eb714c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb006d4b0 con 0x7f5eb00c9150 2023-11-08T19:25:25.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7ff361b0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0cfb30 0x7ff35c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7fe575e65640 1 -- 172.21.15.119:0/2642081650 >> v2:172.21.15.188:3300/0 conn(0x7fe5700cb170 msgr2=0x7fe57006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7fe575e65640 1 --2- 172.21.15.119:0/2642081650 >> v2:172.21.15.188:3300/0 conn(0x7fe5700cb170 0x7fe57006a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7fe575e65640 1 -- 172.21.15.119:0/2642081650 >> v2:172.21.15.119:3300/0 conn(0x7fe5700cad30 msgr2=0x7fe57006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7fe575e65640 1 --2- 172.21.15.119:0/2642081650 >> v2:172.21.15.119:3300/0 conn(0x7fe5700cad30 0x7fe57006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7fe575e65640 1 -- 172.21.15.119:0/2642081650 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700c9150 con 0x7fe5701447a0 2023-11-08T19:25:25.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff361b0f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff35c161800 0x7ff35c0b9190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff35b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 0x7ff35c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff361b0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c150090 con 0x7ff35c0cfb30 2023-11-08T19:25:25.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff361b0f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0b4960 con 0x7ff35c161800 2023-11-08T19:25:25.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff361b0f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35c0c9a10 con 0x7ff35c0ce810 2023-11-08T19:25:25.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff35b7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 0x7ff35c0c94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44672/0 (socket says 172.21.15.119:44672) 2023-11-08T19:25:25.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7ff35b7fe640 1 -- 172.21.15.119:0/2068900778 learned_addr learned my addr 172.21.15.119:0/2068900778 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7f5eb4ee8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5eb006a970 0x7f5eb006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7f5eb4ee8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5eb006a970 0x7f5eb006cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51174/0 (socket says 172.21.15.119:51174) 2023-11-08T19:25:25.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb00c9150 0x7f5eb00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7f5eaffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb00c9150 0x7f5eb00cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44684/0 (socket says 172.21.15.119:44684) 2023-11-08T19:25:25.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7f5eaffff640 1 -- 172.21.15.119:0/4253175138 learned_addr learned my addr 172.21.15.119:0/4253175138 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.982+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/4253175138 >> v2:172.21.15.188:3300/0 conn(0x7f5eb00ccc10 0x7f5eb01653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f7768b65640 1 Processor -- start 2023-11-08T19:25:25.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.981+0000 7f5d077fe640 1 --2- 172.21.15.119:0/4281516454 >> v2:172.21.15.188:3300/0 conn(0x7f5d080d0d40 0x7f5d080b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35bfff640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff35c161800 0x7ff35c0b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5c48b18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5c48b18640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c4415d6c0 0x7f5c440ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f7768b65640 1 -- start start 2023-11-08T19:25:25.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f7c65725640 1 Processor -- start 2023-11-08T19:25:25.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5d077fe640 1 -- 172.21.15.119:0/4281516454 >> v1:172.21.15.188:6789/0 conn(0x7f5d080d1c50 legacy=0x7f5d080b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5d077fe640 1 -- 172.21.15.119:0/4281516454 >> v1:172.21.15.119:6789/0 conn(0x7f5d0815d2e0 legacy=0x7f5d080d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5d077fe640 1 -- 172.21.15.119:0/4281516454 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d080b2960 con 0x7f5d080d0d40 2023-11-08T19:25:25.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35affd640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0cfb30 0x7ff35c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5c4259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5c4259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51192/0 (socket says 172.21.15.119:51192) 2023-11-08T19:25:25.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35b7fe640 1 -- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff35c161800 msgr2=0x7ff35c0b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35b7fe640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff35c161800 0x7ff35c0b9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35b7fe640 1 -- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0cfb30 msgr2=0x7ff35c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35b7fe640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0cfb30 0x7ff35c0cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7ff35b7fe640 1 -- 172.21.15.119:0/2068900778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34c009400 con 0x7ff35c0ce810 2023-11-08T19:25:25.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5eb56e9640 1 -- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb00c9150 msgr2=0x7f5eb00cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb00c9150 0x7f5eb00cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5eb56e9640 1 -- 172.21.15.119:0/4253175138 >> v2:172.21.15.119:3300/0 conn(0x7f5eb006a970 msgr2=0x7f5eb006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/4253175138 >> v2:172.21.15.119:3300/0 conn(0x7f5eb006a970 0x7f5eb006cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7f5eb56e9640 1 -- 172.21.15.119:0/4253175138 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0147b80 con 0x7f5eb00ccc10 2023-11-08T19:25:25.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f5c48b18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c44161160 0x7f5c440cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f5c48b18640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440b2a50 con 0x7f5c440cfa00 2023-11-08T19:25:25.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.983+0000 7fe575e65640 1 --2- 172.21.15.119:0/2642081650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 0x7fe57015f4c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe5700ca7a0 tx=0x7fe56c02b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dfbb7ac79c1c45d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f5c48b18640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440c92b0 con 0x7f5c4415d6c0 2023-11-08T19:25:25.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f5c48b18640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c440cabc0 con 0x7f5c44161160 2023-11-08T19:25:25.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f7c65725640 1 -- start start 2023-11-08T19:25:25.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f5c4259c640 1 -- 172.21.15.119:0/1819069024 learned_addr learned my addr 172.21.15.119:0/1819069024 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7fe574e63640 1 -- 172.21.15.119:0/2642081650 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c031880 con 0x7fe5701447a0 2023-11-08T19:25:25.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7ff35b7fe640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 0x7ff35c0c94d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff34c02b750 tx=0x7ff34c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7fe574e63640 1 -- 172.21.15.119:0/2642081650 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c0319e0 con 0x7fe5701447a0 2023-11-08T19:25:25.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7fe574e63640 1 -- 172.21.15.119:0/2642081650 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c031d70 con 0x7fe5701447a0 2023-11-08T19:25:25.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.984+0000 7f5c41d9b640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c4415d6c0 0x7f5c440ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c42d9d640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c44161160 0x7f5c440cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c030710 con 0x7ff35c0ce810 2023-11-08T19:25:25.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff34c030870 con 0x7ff35c0ce810 2023-11-08T19:25:25.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34c030c00 con 0x7ff35c0ce810 2023-11-08T19:25:25.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7ff361b0f640 1 -- 172.21.15.119:0/2068900778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35c0c5f30 con 0x7ff35c0ce810 2023-11-08T19:25:25.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c4259c640 1 -- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c44161160 msgr2=0x7f5c440cbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c4259c640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c44161160 0x7f5c440cbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c4259c640 1 -- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c4415d6c0 msgr2=0x7f5c440ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7ff361b0f640 1 -- 172.21.15.119:0/2068900778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35c0cb650 con 0x7ff35c0ce810 2023-11-08T19:25:25.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c4259c640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c4415d6c0 0x7f5c440ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7fe5780c9640 1 -- 172.21.15.119:0/2642081650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 msgr2=0x7fe57015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7fe5780c9640 1 --2- 172.21.15.119:0/2642081650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 0x7fe57015f4c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe5700ca7a0 tx=0x7fe56c02b920 comp rx=0 tx=0).stop 2023-11-08T19:25:25.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c4259c640 1 -- 172.21.15.119:0/1819069024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c38009400 con 0x7f5c440cfa00 2023-11-08T19:25:25.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.985+0000 7f5c42d9d640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c44161160 0x7f5c440cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:25.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 -- 172.21.15.119:0/2642081650 shutdown_connections 2023-11-08T19:25:25.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 --2- 172.21.15.119:0/2642081650 >> v2:172.21.15.188:3300/0 conn(0x7fe5700cb170 0x7fe57006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 --2- 172.21.15.119:0/2642081650 >> v2:172.21.15.119:3300/0 conn(0x7fe5700cad30 0x7fe57006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 --2- 172.21.15.119:0/2642081650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5701447a0 0x7fe57015f4c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 -- 172.21.15.119:0/2642081650 >> 172.21.15.119:0/2642081650 conn(0x7fe5700cb610 msgr2=0x7fe5700cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7c65725640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7c600cfa00 0x7f7c600cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7c65725640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c601615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7c65725640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6015df70 con 0x7f7c6015d6c0 2023-11-08T19:25:25.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7c65725640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6015daa0 con 0x7f7c600cfa00 2023-11-08T19:25:25.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7c65725640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600d03b0 con 0x7f7c60161160 2023-11-08T19:25:25.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f5c4259c640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440c8d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c8f0 tx=0x7f5c3c00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7768b65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77640b2010 0x7f77640c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7768b65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77640d1260 0x7f77640c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7768b65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 0x7f776415c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7768b65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77640c7870 con 0x7f77640d1d50 2023-11-08T19:25:25.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7768b65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77640c7b50 con 0x7f77640d1260 2023-11-08T19:25:25.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7768b65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77640c7e30 con 0x7f77640b2010 2023-11-08T19:25:25.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7762d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 0x7f776415c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7762d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 0x7f776415c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51200/0 (socket says 172.21.15.119:51200) 2023-11-08T19:25:25.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7762d9d640 1 -- 172.21.15.119:0/1052739047 learned_addr learned my addr 172.21.15.119:0/1052739047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7f7c5effd640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7c6015d6c0 0x7f7c600d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:60580/0 (socket says 172.21.15.119:60580) 2023-11-08T19:25:25.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f7c5effd640 1 -- 172.21.15.119:0/938180891 learned_addr learned my addr 172.21.15.119:0/938180891 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:25.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff33c005250 con 0x7ff35c0ce810 2023-11-08T19:25:25.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f5d077fe640 1 --2- 172.21.15.119:0/4281516454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d080d0d40 0x7f5d080b3b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009d90 tx=0x7f5d0002bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b795d523bd63d1e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f7762d9d640 1 -- 172.21.15.119:0/1052739047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77640b2010 msgr2=0x7f77640c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f7762d9d640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77640b2010 0x7f77640c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00a640 con 0x7f5c440cfa00 2023-11-08T19:25:25.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f7762d9d640 1 -- 172.21.15.119:0/1052739047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77640d1260 msgr2=0x7f77640c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f7762d9d640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77640d1260 0x7f77640c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f7762d9d640 1 -- 172.21.15.119:0/1052739047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c0094d0 con 0x7f77640d1d50 2023-11-08T19:25:25.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.988+0000 7f7c5f7fe640 1 --2- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c601615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.987+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c440caea0 con 0x7f5c440cfa00 2023-11-08T19:25:25.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.988+0000 7f7762d9d640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 0x7f776415c1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f775c00efc0 tx=0x7f775c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.988+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c440cb460 con 0x7f5c440cfa00 2023-11-08T19:25:25.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.988+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c00a7a0 con 0x7f5c440cfa00 2023-11-08T19:25:25.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.988+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c00ab50 con 0x7f5c440cfa00 2023-11-08T19:25:25.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff34c039050 con 0x7ff35c0ce810 2023-11-08T19:25:25.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c014480 con 0x7f77640d1d50 2023-11-08T19:25:25.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775c00f040 con 0x7f77640d1d50 2023-11-08T19:25:25.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c014820 con 0x7f77640d1d50 2023-11-08T19:25:25.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77640c8110 con 0x7f77640d1d50 2023-11-08T19:25:25.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 -- 172.21.15.119:0/2642081650 shutdown_connections 2023-11-08T19:25:25.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.986+0000 7fe5780c9640 1 -- 172.21.15.119:0/2642081650 wait complete. 2023-11-08T19:25:25.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00ccc10 0x7f5eb01653f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5eb006b1c0 tx=0x7f5ea802bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74c571984a359f9b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f776415c6f0 con 0x7f77640d1d50 2023-11-08T19:25:25.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7f7c5e7fc640 1 --2- 172.21.15.119:0/938180891 >> v2:172.21.15.188:3300/0 conn(0x7f7c600cfa00 0x7f7c600cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.989+0000 7fe5780c9640 1 Processor -- start 2023-11-08T19:25:25.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7ff353fff640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff34406aec0 0x7ff34406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 msgr2=0x7f7c601615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7f7c5e7fc640 1 --2- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c601615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/938180891 >> v1:172.21.15.188:6789/0 conn(0x7f7c6015d6c0 legacy=0x7f7c600d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/938180891 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c60161b10 con 0x7f7c600cfa00 2023-11-08T19:25:25.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7fe5780c9640 1 -- start start 2023-11-08T19:25:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.991+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2140547e0 con 0x7fd218006b60 2023-11-08T19:25:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c28005250 con 0x7f5c440cfa00 2023-11-08T19:25:25.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.991+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c3c00acb0 con 0x7f5c440cfa00 2023-11-08T19:25:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.990+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff34c0ab6d0 con 0x7ff35c0ce810 2023-11-08T19:25:25.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.991+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7748005250 con 0x7f77640d1d50 2023-11-08T19:25:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.991+0000 7f5eaf7fe640 1 -- 172.21.15.119:0/4253175138 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea802cb40 con 0x7f5eb00ccc10 2023-11-08T19:25:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7ff35affd640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff34406aec0 0x7ff34406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f775c014980 con 0x7f77640d1d50 2023-11-08T19:25:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f5d067fc640 1 -- 172.21.15.119:0/4281516454 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0002cd80 con 0x7f5d080d0d40 2023-11-08T19:25:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f5d067fc640 1 -- 172.21.15.119:0/4281516454 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d00034440 con 0x7f5d080d0d40 2023-11-08T19:25:25.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f5d067fc640 1 -- 172.21.15.119:0/4281516454 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d000347d0 con 0x7f5d080d0d40 2023-11-08T19:25:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.991+0000 7f5eaf7fe640 1 -- 172.21.15.119:0/4253175138 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea802cca0 con 0x7f5eb00ccc10 2023-11-08T19:25:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.991+0000 7f5eaf7fe640 1 -- 172.21.15.119:0/4253175138 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8034660 con 0x7f5eb00ccc10 2023-11-08T19:25:25.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f7c5e7fc640 1 --2- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600cfe70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c500097a0 tx=0x7f7c5002b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a5d2c2a8cbac2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7ff35affd640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff34406aec0 0x7ff34406d380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff35c143980 tx=0x7ff348008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f7c5dffb640 1 -- 172.21.15.119:0/938180891 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500317c0 con 0x7f7c600cfa00 2023-11-08T19:25:25.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f5c32ffd640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c2406aec0 0x7f5c2406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.992+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5c3c0892d0 con 0x7f5c440cfa00 2023-11-08T19:25:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f7c5dffb640 1 -- 172.21.15.119:0/938180891 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50031920 con 0x7f7c600cfa00 2023-11-08T19:25:25.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f7c5dffb640 1 -- 172.21.15.119:0/938180891 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50031cf0 con 0x7f7c600cfa00 2023-11-08T19:25:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f5c41d9b640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c2406aec0 0x7f5c2406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7f7c65725640 1 -- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 msgr2=0x7f7c600cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7f7c65725640 1 --2- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600cfe70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7c500097a0 tx=0x7f7c5002b860 comp rx=0 tx=0).stop 2023-11-08T19:25:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f5eb714c640 1 -- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00ccc10 msgr2=0x7f5eb01653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f5eb714c640 1 --2- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00ccc10 0x7f5eb01653f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5eb006b1c0 tx=0x7f5ea802bd90 comp rx=0 tx=0).stop 2023-11-08T19:25:25.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe5780c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cad30 0x7fe570202c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7f5c41d9b640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c2406aec0 0x7f5c2406d380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c38036000 tx=0x7f5c3803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe5780c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 0x7fe570138260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe5780c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5701447a0 0x7fe5700cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe5780c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57015e120 con 0x7fe5700cb170 2023-11-08T19:25:25.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe5780c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe570203140 con 0x7fe5701447a0 2023-11-08T19:25:25.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe5780c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5700b2590 con 0x7fe5700cad30 2023-11-08T19:25:25.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7f5d07fff640 1 -- 172.21.15.119:0/4281516454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d080d0d40 msgr2=0x7f5d080b3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7f5d07fff640 1 --2- 172.21.15.119:0/4281516454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d080d0d40 0x7f5d080b3b50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5d00009d90 tx=0x7f5d0002bfd0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe575664640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 0x7fe570138260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.994+0000 7fe575e65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cad30 0x7fe570202c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7fe575664640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 0x7fe570138260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51216/0 (socket says 172.21.15.119:51216) 2023-11-08T19:25:25.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7fe575664640 1 -- 172.21.15.119:0/1890221718 learned_addr learned my addr 172.21.15.119:0/1890221718 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5d07fff640 1 -- 172.21.15.119:0/4281516454 shutdown_connections 2023-11-08T19:25:25.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5d07fff640 1 --2- 172.21.15.119:0/4281516454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d080d0d40 0x7f5d080b3b50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5d07fff640 1 -- 172.21.15.119:0/4281516454 >> 172.21.15.119:0/4281516454 conn(0x7f5d080d2f60 msgr2=0x7f5d080d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 -- 172.21.15.119:0/4253175138 shutdown_connections 2023-11-08T19:25:25.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 --2- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00ccc10 0x7f5eb01653f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 --2- 172.21.15.119:0/4253175138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb00c9150 0x7f5eb00cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 --2- 172.21.15.119:0/4253175138 >> v2:172.21.15.119:3300/0 conn(0x7f5eb006a970 0x7f5eb006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 -- 172.21.15.119:0/4253175138 >> 172.21.15.119:0/4253175138 conn(0x7f5eb00cb610 msgr2=0x7f5eb00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 -- 172.21.15.119:0/4253175138 shutdown_connections 2023-11-08T19:25:25.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.993+0000 7f775affd640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f774406af70 0x7f774406d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f776259c640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f774406af70 0x7f774406d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f775c08a460 con 0x7f77640d1d50 2023-11-08T19:25:25.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7fe576666640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5701447a0 0x7fe5700cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f7c65725640 1 -- 172.21.15.119:0/938180891 shutdown_connections 2023-11-08T19:25:25.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f7c65725640 1 --2- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600cfe70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f7c65725640 1 --2- 172.21.15.119:0/938180891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c601615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f7c65725640 1 -- 172.21.15.119:0/938180891 >> 172.21.15.119:0/938180891 conn(0x7f7c6006a9f0 msgr2=0x7f7c6006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f5d07fff640 1 -- 172.21.15.119:0/4281516454 shutdown_connections 2023-11-08T19:25:25.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f7c65725640 1 -- 172.21.15.119:0/938180891 shutdown_connections 2023-11-08T19:25:25.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f776259c640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f774406af70 0x7f774406d430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f774c036000 tx=0x7f774c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7fe575664640 1 -- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cad30 msgr2=0x7fe570202c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7fe575664640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cad30 0x7fe570202c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7fe575664640 1 -- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5701447a0 msgr2=0x7fe5700cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7fe575664640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5701447a0 0x7fe5700cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7fe575664640 1 -- 172.21.15.119:0/1890221718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c0094d0 con 0x7fe5700cb170 2023-11-08T19:25:25.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.997+0000 7fe575664640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 0x7fe570138260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe56400d620 tx=0x7fe56400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.995+0000 7f5eb714c640 1 -- 172.21.15.119:0/4253175138 wait complete. 2023-11-08T19:25:25.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f7c65725640 1 -- 172.21.15.119:0/938180891 wait complete. 2023-11-08T19:25:25.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.996+0000 7f5d07fff640 1 -- 172.21.15.119:0/4281516454 wait complete. 2023-11-08T19:25:25.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.997+0000 7f5eb714c640 1 Processor -- start 2023-11-08T19:25:25.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.998+0000 7f5d07fff640 1 Processor -- start 2023-11-08T19:25:25.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f7c65725640 1 Processor -- start 2023-11-08T19:25:25.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.997+0000 7f5eb714c640 1 -- start start 2023-11-08T19:25:25.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb714c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb006a970 0x7f5eb0148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb714c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00c9150 0x7f5eb00b1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb714c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 0x7f5eb0135b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb714c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb00b1670 con 0x7f5eb00ccc10 2023-11-08T19:25:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb714c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb01350e0 con 0x7f5eb00c9150 2023-11-08T19:25:25.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb714c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb0149f00 con 0x7f5eb006a970 2023-11-08T19:25:25.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f7c65725640 1 -- start start 2023-11-08T19:25:25.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.998+0000 7f5d07fff640 1 -- start start 2023-11-08T19:25:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb56e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 0x7f5eb0135b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb56e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 0x7f5eb0135b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51230/0 (socket says 172.21.15.119:51230) 2023-11-08T19:25:25.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb56e9640 1 -- 172.21.15.119:0/2649090613 learned_addr learned my addr 172.21.15.119:0/2649090613 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:24.999+0000 7f5eb4ee8640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb006a970 0x7f5eb0148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eaffff640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00c9150 0x7f5eb00b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d080d0d40 0x7f5d081f9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 0x7f5d080b5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5d07fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d0815d2e0 0x7f5d080b60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5d07fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d08155220 con 0x7f5d080d1c50 2023-11-08T19:25:25.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5d07fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080cc0e0 con 0x7f5d0815d2e0 2023-11-08T19:25:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5d07fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080cc3c0 con 0x7f5d080d0d40 2023-11-08T19:25:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eb56e9640 1 -- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb006a970 msgr2=0x7f5eb0148c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb006a970 0x7f5eb0148c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eb56e9640 1 -- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00c9150 msgr2=0x7f5eb00b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00c9150 0x7f5eb00b1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eb56e9640 1 -- 172.21.15.119:0/2649090613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8009a70 con 0x7f5eb00ccc10 2023-11-08T19:25:25.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5eb56e9640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 0x7f5eb0135b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8037040 tx=0x7f5ea8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5cfe7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 0x7f5d080b5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5cfe7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 0x7f5d080b5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51234/0 (socket says 172.21.15.119:51234) 2023-11-08T19:25:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f5cfe7fc640 1 -- 172.21.15.119:0/899299375 learned_addr learned my addr 172.21.15.119:0/899299375 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f7c65725640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.000+0000 7f7c65725640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 0x7f7c601f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c65725640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c600b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5d077fe640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d0815d2e0 0x7f5d080b60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5d06ffd640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d080d0d40 0x7f5d081f9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 0x7f7c601f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c5e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 0x7f7c601f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51246/0 (socket says 172.21.15.119:51246) 2023-11-08T19:25:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/2498135692 learned_addr learned my addr 172.21.15.119:0/2498135692 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c5effd640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c5f7fe640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c600b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c65725640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c600b57c0 con 0x7f7c6015d6c0 2023-11-08T19:25:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c65725640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c601e7cb0 con 0x7f7c600cfa00 2023-11-08T19:25:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f7c65725640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c601e7f90 con 0x7f7c60161160 2023-11-08T19:25:25.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5cfe7fc640 1 -- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d080d0d40 msgr2=0x7f5d081f9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5cfe7fc640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d080d0d40 0x7f5d081f9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5cfe7fc640 1 -- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d0815d2e0 msgr2=0x7f5d080b60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5cfe7fc640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d0815d2e0 0x7f5d080b60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.001+0000 7f5cfe7fc640 1 -- 172.21.15.119:0/899299375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00009a70 con 0x7f5d080d1c50 2023-11-08T19:25:25.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f5d077fe640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d0815d2e0 0x7f5d080b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:25.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f5cfe7fc640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 0x7f5d080b5b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c450 tx=0x7f5cf000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 msgr2=0x7f7c600b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f7c5e7fc640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c600b4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 msgr2=0x7f7c600ca900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f7c5e7fc640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600ca900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f7c5e7fc640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009400 con 0x7f7c6015d6c0 2023-11-08T19:25:25.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f7c5e7fc640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 0x7f7c601f7060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034040 tx=0x7f7c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.002+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f775c055200 con 0x7f77640d1d50 2023-11-08T19:25:25.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032960 con 0x7f7c6015d6c0 2023-11-08T19:25:25.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56400b620 con 0x7fe5700cb170 2023-11-08T19:25:25.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00052d0 con 0x7f5d080d1c50 2023-11-08T19:25:25.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf0005430 con 0x7f5d080d1c50 2023-11-08T19:25:25.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0012870 con 0x7f5d080d1c50 2023-11-08T19:25:25.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff34c076600 con 0x7ff35c0ce810 2023-11-08T19:25:25.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8040650 con 0x7f5eb00ccc10 2023-11-08T19:25:25.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea80407b0 con 0x7f5eb00ccc10 2023-11-08T19:25:25.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8040b60 con 0x7f5eb00ccc10 2023-11-08T19:25:25.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c3c054070 con 0x7f5c440cfa00 2023-11-08T19:25:25.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50032b00 con 0x7f7c6015d6c0 2023-11-08T19:25:25.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56400b780 con 0x7fe5700cb170 2023-11-08T19:25:25.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500453f0 con 0x7f7c6015d6c0 2023-11-08T19:25:25.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7f7c65725640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c601e8270 con 0x7f7c6015d6c0 2023-11-08T19:25:25.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.007+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d080cc6a0 con 0x7f5d080d1c50 2023-11-08T19:25:25.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.008+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb014a1e0 con 0x7f5eb00ccc10 2023-11-08T19:25:25.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.008+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d081fa0a0 con 0x7f5d080d1c50 2023-11-08T19:25:25.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.008+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb0135610 con 0x7f5eb00ccc10 2023-11-08T19:25:25.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.006+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56400bb30 con 0x7fe5700cb170 2023-11-08T19:25:25.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.007+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5700b2870 con 0x7fe5700cb170 2023-11-08T19:25:25.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.007+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5700b2e30 con 0x7fe5700cb170 2023-11-08T19:25:25.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.007+0000 7f7c65725640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c600cae40 con 0x7f7c6015d6c0 2023-11-08T19:25:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.010+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d08155360 con 0x7f5d080d1c50 2023-11-08T19:25:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.010+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb0135830 con 0x7f5eb00ccc10 2023-11-08T19:25:25.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.010+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe570074430 con 0x7fe5700cb170 2023-11-08T19:25:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.011+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c40005250 con 0x7f7c6015d6c0 2023-11-08T19:25:25.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.014+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe56400bc90 con 0x7fe5700cb170 2023-11-08T19:25:25.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.015+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5cf00129d0 con 0x7f5d080d1c50 2023-11-08T19:25:25.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.014+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ea8040cc0 con 0x7f5eb00ccc10 2023-11-08T19:25:25.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.015+0000 7f5ead7fa640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e9406aec0 0x7f5e9406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.015+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5ea80ac7c0 con 0x7f5eb00ccc10 2023-11-08T19:25:25.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.015+0000 7fe5627fc640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55406af40 0x7fe55406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.016+0000 7f5eb4ee8640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e9406aec0 0x7f5e9406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.015+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fe56408a710 con 0x7fe5700cb170 2023-11-08T19:25:25.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.016+0000 7f5cfffff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ce806aec0 0x7f5ce806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.016+0000 7fe575e65640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55406af40 0x7fe55406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.017+0000 7f5eb4ee8640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e9406aec0 0x7f5e9406d380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0136400 tx=0x7f5e9c009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.016+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f5cf0089f20 con 0x7f5d080d1c50 2023-11-08T19:25:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.017+0000 7f5d06ffd640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ce806aec0 0x7f5ce806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.018+0000 7fe575e65640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55406af40 0x7fe55406d400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe56c036000 tx=0x7fe56c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.018+0000 7f5d06ffd640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ce806aec0 0x7f5ce806d380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005880 tx=0x7f5cf8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.020+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cf0054c30 con 0x7f5d080d1c50 2023-11-08T19:25:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.022+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c5004e450 con 0x7f7c6015d6c0 2023-11-08T19:25:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.023+0000 7f7c477fe640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4c06aec0 0x7f7c4c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.024+0000 7f7c5effd640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4c06aec0 0x7f7c4c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.025+0000 7f7c5effd640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4c06aec0 0x7f7c4c06d380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c54005880 tx=0x7f7c54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.029+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5640555f0 con 0x7fe5700cb170 2023-11-08T19:25:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.037+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7c5007b1a0 con 0x7f7c6015d6c0 2023-11-08T19:25:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.039+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ea8077810 con 0x7f5eb00ccc10 2023-11-08T19:25:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.047+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c50076de0 con 0x7f7c6015d6c0 2023-11-08T19:25:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.378+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd1f8005500 con 0x7fd218006b60 2023-11-08T19:25:25.383 INFO:teuthology.orchestra.run.smithi119.stdout:55834574859 2023-11-08T19:25:25.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.379+0000 7fd21cff9640 1 -- 172.21.15.119:0/1982393406 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd214057e00 con 0x7fd218006b60 2023-11-08T19:25:25.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.382+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd20406ae70 msgr2=0x7fd20406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.382+0000 7fd20a7fc640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd20406ae70 0x7fd20406d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd2180df210 tx=0x7fd20c001da0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.382+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 msgr2=0x7fd21804cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.382+0000 7fd20a7fc640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 0x7fd21804cbc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd21400e870 tx=0x7fd21400ed30 comp rx=0 tx=0).stop 2023-11-08T19:25:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 shutdown_connections 2023-11-08T19:25:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd20406ae70 0x7fd20406d330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2180e3060 0x7fd218039d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd218054350 0x7fd2180397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 --2- 172.21.15.119:0/1982393406 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd218006b60 0x7fd21804cbc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 >> 172.21.15.119:0/1982393406 conn(0x7fd218055a80 msgr2=0x7fd2180d35d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 shutdown_connections 2023-11-08T19:25:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.385+0000 7fd20a7fc640 1 -- 172.21.15.119:0/1982393406 wait complete. 2023-11-08T19:25:25.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2023-11-08T19:25:25.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.412+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7748005500 con 0x7f77640d1d50 2023-11-08T19:25:25.415 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.413+0000 7f775affd640 1 -- 172.21.15.119:0/1052739047 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f775c058820 con 0x7f77640d1d50 2023-11-08T19:25:25.416 INFO:teuthology.orchestra.run.smithi119.stdout:55834574858 2023-11-08T19:25:25.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.416+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff33c0057d0 con 0x7ff35c0ce810 2023-11-08T19:25:25.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f774406af70 msgr2=0x7f774406d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f774406af70 0x7f774406d430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f774c036000 tx=0x7f774c030000 comp rx=0 tx=0).stop 2023-11-08T19:25:25.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 msgr2=0x7f776415c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 0x7f776415c1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f775c00efc0 tx=0x7f775c00c490 comp rx=0 tx=0).stop 2023-11-08T19:25:25.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.416+0000 7ff353fff640 1 -- 172.21.15.119:0/2068900778 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff34c04c090 con 0x7ff35c0ce810 2023-11-08T19:25:25.419 INFO:teuthology.orchestra.run.smithi119.stdout:55834574858 2023-11-08T19:25:25.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5c28005500 con 0x7f5c440cfa00 2023-11-08T19:25:25.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 shutdown_connections 2023-11-08T19:25:25.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f774406af70 0x7f774406d430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77640d1d50 0x7f776415c1b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77640d1260 0x7f77640c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 --2- 172.21.15.119:0/1052739047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77640b2010 0x7f77640c9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.417+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 >> 172.21.15.119:0/1052739047 conn(0x7f77640d30f0 msgr2=0x7f776415e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.418+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 shutdown_connections 2023-11-08T19:25:25.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.418+0000 7f7768b65640 1 -- 172.21.15.119:0/1052739047 wait complete. 2023-11-08T19:25:25.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.418+0000 7f5c32ffd640 1 -- 172.21.15.119:0/1819069024 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5c3c018020 con 0x7f5c440cfa00 2023-11-08T19:25:25.423 INFO:teuthology.orchestra.run.smithi119.stdout:55834574860 2023-11-08T19:25:25.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.420+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff34406aec0 msgr2=0x7ff34406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.420+0000 7ff351ffb640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff34406aec0 0x7ff34406d380 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff35c143980 tx=0x7ff348008040 comp rx=0 tx=0).stop 2023-11-08T19:25:25.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.420+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 msgr2=0x7ff35c0c94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.420+0000 7ff351ffb640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 0x7ff35c0c94d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff34c02b750 tx=0x7ff34c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c2406aec0 msgr2=0x7f5c2406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.420+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 shutdown_connections 2023-11-08T19:25:25.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.420+0000 7ff351ffb640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff34406aec0 0x7ff34406d380 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7ff351ffb640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff35c161800 0x7ff35c0b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7ff351ffb640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff35c0cfb30 0x7ff35c0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7ff351ffb640 1 --2- 172.21.15.119:0/2068900778 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff35c0ce810 0x7ff35c0c94d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 >> 172.21.15.119:0/2068900778 conn(0x7ff35c06a9f0 msgr2=0x7ff35c0d1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c2406aec0 0x7f5c2406d380 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c38036000 tx=0x7f5c3803d040 comp rx=0 tx=0).stop 2023-11-08T19:25:25.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 msgr2=0x7f5c440c8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440c8d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00c8f0 tx=0x7f5c3c00cdb0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 shutdown_connections 2023-11-08T19:25:25.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7ff351ffb640 1 -- 172.21.15.119:0/2068900778 wait complete. 2023-11-08T19:25:25.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 shutdown_connections 2023-11-08T19:25:25.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c2406aec0 0x7f5c2406d380 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c44161160 0x7f5c440cbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c4415d6c0 0x7f5c440ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 --2- 172.21.15.119:0/1819069024 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c440cfa00 0x7f5c440c8d70 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.421+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 >> 172.21.15.119:0/1819069024 conn(0x7f5c4406a9f0 msgr2=0x7f5c4415b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.422+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 shutdown_connections 2023-11-08T19:25:25.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.422+0000 7f5c48b18640 1 -- 172.21.15.119:0/1819069024 wait complete. 2023-11-08T19:25:25.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.430+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f5d0804ece0 con 0x7f5d080d1c50 2023-11-08T19:25:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.431+0000 7f5cfffff640 1 -- 172.21.15.119:0/899299375 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5cf0058250 con 0x7f5d080d1c50 2023-11-08T19:25:25.433 INFO:teuthology.orchestra.run.smithi119.stdout:55834574860 2023-11-08T19:25:25.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.434+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ce806aec0 msgr2=0x7f5ce806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.434+0000 7f5d07fff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ce806aec0 0x7f5ce806d380 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8005880 tx=0x7f5cf8009040 comp rx=0 tx=0).stop 2023-11-08T19:25:25.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.434+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 msgr2=0x7f5d080b5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.434+0000 7f5d07fff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 0x7f5d080b5b80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000c450 tx=0x7f5cf000c910 comp rx=0 tx=0).stop 2023-11-08T19:25:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 shutdown_connections 2023-11-08T19:25:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ce806aec0 0x7f5ce806d380 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d0815d2e0 0x7f5d080b60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d080d1c50 0x7f5d080b5b80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 --2- 172.21.15.119:0/899299375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d080d0d40 0x7f5d081f9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 >> 172.21.15.119:0/899299375 conn(0x7f5d080d2f60 msgr2=0x7f5d0806b120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 shutdown_connections 2023-11-08T19:25:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.435+0000 7f5d07fff640 1 -- 172.21.15.119:0/899299375 wait complete. 2023-11-08T19:25:25.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.2 2023-11-08T19:25:25.443 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2023-11-08T19:25:25.447 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.447+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe570067260 con 0x7fe5700cb170 2023-11-08T19:25:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.447+0000 7fe5627fc640 1 -- 172.21.15.119:0/1890221718 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe564058c10 con 0x7fe5700cb170 2023-11-08T19:25:25.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2023-11-08T19:25:25.450 INFO:teuthology.orchestra.run.smithi119.stdout:55834574859 2023-11-08T19:25:25.450 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55406af40 msgr2=0x7fe55406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55406af40 0x7fe55406d400 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe56c036000 tx=0x7fe56c03c040 comp rx=0 tx=0).stop 2023-11-08T19:25:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 msgr2=0x7fe570138260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 0x7fe570138260 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe56400d620 tx=0x7fe56400dae0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 shutdown_connections 2023-11-08T19:25:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55406af40 0x7fe55406d400 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5701447a0 0x7fe5700cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5700cb170 0x7fe570138260 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 --2- 172.21.15.119:0/1890221718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5700cad30 0x7fe570202c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 >> 172.21.15.119:0/1890221718 conn(0x7fe5700cb610 msgr2=0x7fe57006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 shutdown_connections 2023-11-08T19:25:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.450+0000 7fe5780c9640 1 -- 172.21.15.119:0/1890221718 wait complete. 2023-11-08T19:25:25.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.458+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5eb0074430 con 0x7f5eb00ccc10 2023-11-08T19:25:25.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.458+0000 7f5ead7fa640 1 -- 172.21.15.119:0/2649090613 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5ea8009450 con 0x7f5eb00ccc10 2023-11-08T19:25:25.460 INFO:teuthology.orchestra.run.smithi119.stdout:55834574859 2023-11-08T19:25:25.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574860 got 55834574860 for osd.4 2023-11-08T19:25:25.461 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e9406aec0 msgr2=0x7f5e9406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e9406aec0 0x7f5e9406d380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0136400 tx=0x7f5e9c009190 comp rx=0 tx=0).stop 2023-11-08T19:25:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 msgr2=0x7f5eb0135b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 0x7f5eb0135b30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8037040 tx=0x7f5ea8004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 shutdown_connections 2023-11-08T19:25:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e9406aec0 0x7f5e9406d380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5eb00ccc10 0x7f5eb0135b30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eb00c9150 0x7f5eb00b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 --2- 172.21.15.119:0/2649090613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eb006a970 0x7f5eb0148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.461+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 >> 172.21.15.119:0/2649090613 conn(0x7f5eb00cb610 msgr2=0x7f5eb016b670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.462+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 shutdown_connections 2023-11-08T19:25:25.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.462+0000 7f5eb714c640 1 -- 172.21.15.119:0/2649090613 wait complete. 2023-11-08T19:25:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.466+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f7c400057d0 con 0x7f7c6015d6c0 2023-11-08T19:25:25.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.467+0000 7f7c477fe640 1 -- 172.21.15.119:0/2498135692 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c5007a400 con 0x7f7c6015d6c0 2023-11-08T19:25:25.468 INFO:teuthology.orchestra.run.smithi119.stdout:55834574858 2023-11-08T19:25:25.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2023-11-08T19:25:25.470 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4c06aec0 msgr2=0x7f7c4c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4c06aec0 0x7f7c4c06d380 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7c54005880 tx=0x7f7c54009040 comp rx=0 tx=0).stop 2023-11-08T19:25:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 msgr2=0x7f7c601f7060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 0x7f7c601f7060 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034040 tx=0x7f7c500046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 shutdown_connections 2023-11-08T19:25:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4c06aec0 0x7f7c4c06d380 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c60161160 0x7f7c600b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015d6c0 0x7f7c601f7060 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 --2- 172.21.15.119:0/2498135692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c600cfa00 0x7f7c600ca900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.469+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 >> 172.21.15.119:0/2498135692 conn(0x7f7c6006a9f0 msgr2=0x7f7c6015bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.470+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 shutdown_connections 2023-11-08T19:25:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.470+0000 7f7c457fa640 1 -- 172.21.15.119:0/2498135692 wait complete. 2023-11-08T19:25:25.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2023-11-08T19:25:25.481 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.487 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.5 2023-11-08T19:25:25.487 DEBUG:teuthology.parallel:result is None 2023-11-08T19:25:25.487 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T19:25:25.487 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.739+0000 7f57fe644640 1 Processor -- start 2023-11-08T19:25:25.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.740+0000 7f57fe644640 1 -- start start 2023-11-08T19:25:25.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57fe644640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57f80d1260 0x7f57f8160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57fe644640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f57f80d1d50 0x7f57f8160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57fe644640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80ce760 con 0x7f57f80b2010 2023-11-08T19:25:25.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57fe644640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f8161000 con 0x7f57f80d1260 2023-11-08T19:25:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57fe644640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f80d21a0 con 0x7f57f80d1d50 2023-11-08T19:25:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57f77fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f57f80b2010 0x7f57f80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60066/0 (socket says 172.21.15.119:60066) 2023-11-08T19:25:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57f77fe640 1 -- 172.21.15.119:0/3550952229 learned_addr learned my addr 172.21.15.119:0/3550952229 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:25.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.741+0000 7f57f7fff640 1 --2- 172.21.15.119:0/3550952229 >> v2:172.21.15.119:3300/0 conn(0x7f57f80d1260 0x7f57f8160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57fcbe1640 1 --2- 172.21.15.119:0/3550952229 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d1d50 0x7f57f8160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57f6ffd640 1 -- 172.21.15.119:0/3550952229 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937073352 0 0) 0x7f57f80ce760 con 0x7f57f80b2010 2023-11-08T19:25:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57f6ffd640 1 -- 172.21.15.119:0/3550952229 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e4003520 con 0x7f57f80b2010 2023-11-08T19:25:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57f7fff640 1 -- 172.21.15.119:0/3550952229 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d1d50 msgr2=0x7f57f8160ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57f7fff640 1 --2- 172.21.15.119:0/3550952229 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d1d50 0x7f57f8160ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57f7fff640 1 -- 172.21.15.119:0/3550952229 >> v1:172.21.15.119:6789/0 conn(0x7f57f80b2010 legacy=0x7f57f80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.742+0000 7f57f7fff640 1 -- 172.21.15.119:0/3550952229 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80b2580 con 0x7f57f80d1260 2023-11-08T19:25:25.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.743+0000 7f57f7fff640 1 --2- 172.21.15.119:0/3550952229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f8160580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57f806a870 tx=0x7f57e802bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc447284b52e7c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.743+0000 7f57f6ffd640 1 -- 172.21.15.119:0/3550952229 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e802cb50 con 0x7f57f80d1260 2023-11-08T19:25:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.743+0000 7f57f6ffd640 1 -- 172.21.15.119:0/3550952229 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e802ccb0 con 0x7f57f80d1260 2023-11-08T19:25:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.743+0000 7f57f6ffd640 1 -- 172.21.15.119:0/3550952229 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e8034660 con 0x7f57f80d1260 2023-11-08T19:25:25.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 -- 172.21.15.119:0/3550952229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 msgr2=0x7f57f8160580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 --2- 172.21.15.119:0/3550952229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f8160580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f57f806a870 tx=0x7f57e802bda0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 -- 172.21.15.119:0/3550952229 shutdown_connections 2023-11-08T19:25:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 --2- 172.21.15.119:0/3550952229 >> v2:172.21.15.188:3300/0 conn(0x7f57f80d1d50 0x7f57f8160ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 --2- 172.21.15.119:0/3550952229 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f8160580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 -- 172.21.15.119:0/3550952229 >> 172.21.15.119:0/3550952229 conn(0x7f57f80d30f0 msgr2=0x7f57f80d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 -- 172.21.15.119:0/3550952229 shutdown_connections 2023-11-08T19:25:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.744+0000 7f57fe644640 1 -- 172.21.15.119:0/3550952229 wait complete. 2023-11-08T19:25:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.745+0000 7f57fe644640 1 Processor -- start 2023-11-08T19:25:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.746+0000 7f57fe644640 1 -- start start 2023-11-08T19:25:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.746+0000 7f57fe644640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80b2010 0x7f57f80c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.746+0000 7f57fe644640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f80cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57fe644640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f80d1d50 0x7f57f80cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57fe644640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f8155980 con 0x7f57f80d1260 2023-11-08T19:25:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57fe644640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c9ac0 con 0x7f57f80d1d50 2023-11-08T19:25:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57fe644640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c76f0 con 0x7f57f80b2010 2023-11-08T19:25:25.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f80cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f80cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51262/0 (socket says 172.21.15.119:51262) 2023-11-08T19:25:25.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 -- 172.21.15.119:0/3797122952 learned_addr learned my addr 172.21.15.119:0/3797122952 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:25.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57fcbe1640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f80d1d50 0x7f57f80cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f7fff640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80b2010 0x7f57f80c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 -- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80b2010 msgr2=0x7f57f80c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80b2010 0x7f57f80c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 -- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f80d1d50 msgr2=0x7f57f80cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f80d1d50 0x7f57f80cbee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.747+0000 7f57f77fe640 1 -- 172.21.15.119:0/3797122952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e8009a70 con 0x7f57f80d1260 2023-11-08T19:25:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.748+0000 7f57f77fe640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f80cb9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c440 tx=0x7f57e000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.748+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e0005ce0 con 0x7f57f80d1260 2023-11-08T19:25:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.748+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e0005e40 con 0x7f57f80d1260 2023-11-08T19:25:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.748+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f80cc420 con 0x7f57f80d1260 2023-11-08T19:25:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.748+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e00072b0 con 0x7f57f80d1260 2023-11-08T19:25:25.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.748+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f80c70a0 con 0x7f57f80d1260 2023-11-08T19:25:25.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.749+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f80c7880 con 0x7f57f80d1260 2023-11-08T19:25:25.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.750+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57e0007410 con 0x7f57f80d1260 2023-11-08T19:25:25.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.751+0000 7f57f4ff9640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57d806aea0 0x7f57d806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:25.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.751+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f57e0088ef0 con 0x7f57f80d1260 2023-11-08T19:25:25.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.751+0000 7f57f7fff640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57d806aea0 0x7f57d806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:25.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.752+0000 7f57f7fff640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57d806aea0 0x7f57d806d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f57e8036000 tx=0x7f57e8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:25.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.755+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57e0053cd0 con 0x7f57f80d1260 2023-11-08T19:25:25.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.928+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f57f80cec90 con 0x7f57d806aea0 2023-11-08T19:25:25.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.941+0000 7f57f4ff9640 1 -- 172.21.15.119:0/3797122952 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345123 (secure 0 0 0) 0x7f57e80344c0 con 0x7f57d806aea0 2023-11-08T19:25:25.947 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:25.948 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:25:25.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57d806aea0 msgr2=0x7f57d806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57d806aea0 0x7f57d806d360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f57e8036000 tx=0x7f57e8030000 comp rx=0 tx=0).stop 2023-11-08T19:25:25.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 msgr2=0x7f57f80cb9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:25.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f80cb9a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f57e000c440 tx=0x7f57e000c900 comp rx=0 tx=0).stop 2023-11-08T19:25:25.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 shutdown_connections 2023-11-08T19:25:25.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57d806aea0 0x7f57d806d360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57f80d1d50 0x7f57f80cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57f80d1260 0x7f57f80cb9a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 --2- 172.21.15.119:0/3797122952 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57f80b2010 0x7f57f80c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:25.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.946+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 >> 172.21.15.119:0/3797122952 conn(0x7f57f80d30f0 msgr2=0x7f57f806c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:25.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.947+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 shutdown_connections 2023-11-08T19:25:25.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:25.947+0000 7f57fe644640 1 -- 172.21.15.119:0/3797122952 wait complete. 2023-11-08T19:25:25.965 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-11-08T19:25:24.667940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3375104,"data_stored":2042832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016656"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655958+0000","last_change":"2023-11-08T19:24:57.661383+0000","last_active":"2023-11-08T19:25:03.655958+0000","last_peered":"2023-11-08T19:25:03.655958+0000","last_clean":"2023-11-08T19:25:03.655958+0000","last_became_active":"2023-11-08T19:24:57.661122+0000","last_became_peered":"2023-11-08T19:24:57.661122+0000","last_unstale":"2023-11-08T19:25:03.655958+0000","last_undegraded":"2023-11-08T19:25:03.655958+0000","last_fullsized":"2023-11-08T19:25:03.655958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657959+0000","last_change":"2023-11-08T19:24:58.622972+0000","last_active":"2023-11-08T19:25:03.657959+0000","last_peered":"2023-11-08T19:25:03.657959+0000","last_clean":"2023-11-08T19:25:03.657959+0000","last_became_active":"2023-11-08T19:24:58.620907+0000","last_became_peered":"2023-11-08T19:24:58.620907+0000","last_unstale":"2023-11-08T19:25:03.657959+0000","last_undegraded":"2023-11-08T19:25:03.657959+0000","last_fullsized":"2023-11-08T19:25:03.657959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593888+0000","last_change":"2023-11-08T19:25:03.480700+0000","last_active":"2023-11-08T19:25:04.593888+0000","last_peered":"2023-11-08T19:25:04.593888+0000","last_clean":"2023-11-08T19:25:04.593888+0000","last_became_active":"2023-11-08T19:25:03.477779+0000","last_became_peered":"2023-11-08T19:25:03.477779+0000","last_unstale":"2023-11-08T19:25:04.593888+0000","last_undegraded":"2023-11-08T19:25:04.593888+0000","last_fullsized":"2023-11-08T19:25:04.593888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:47:24.018242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658227+0000","last_change":"2023-11-08T19:24:57.655981+0000","last_active":"2023-11-08T19:25:03.658227+0000","last_peered":"2023-11-08T19:25:03.658227+0000","last_clean":"2023-11-08T19:25:03.658227+0000","last_became_active":"2023-11-08T19:24:57.655420+0000","last_became_peered":"2023-11-08T19:24:57.655420+0000","last_unstale":"2023-11-08T19:25:03.658227+0000","last_undegraded":"2023-11-08T19:25:03.658227+0000","last_fullsized":"2023-11-08T19:25:03.658227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655637+0000","last_change":"2023-11-08T19:24:58.620479+0000","last_active":"2023-11-08T19:25:03.655637+0000","last_peered":"2023-11-08T19:25:03.655637+0000","last_clean":"2023-11-08T19:25:03.655637+0000","last_became_active":"2023-11-08T19:24:58.620201+0000","last_became_peered":"2023-11-08T19:24:58.620201+0000","last_unstale":"2023-11-08T19:25:03.655637+0000","last_undegraded":"2023-11-08T19:25:03.655637+0000","last_fullsized":"2023-11-08T19:25:03.655637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.598798+0000","last_change":"2023-11-08T19:25:02.722736+0000","last_active":"2023-11-08T19:25:04.598798+0000","last_peered":"2023-11-08T19:25:04.598798+0000","last_clean":"2023-11-08T19:25:04.598798+0000","last_became_active":"2023-11-08T19:25:02.722087+0000","last_became_peered":"2023-11-08T19:25:02.722087+0000","last_unstale":"2023-11-08T19:25:04.598798+0000","last_undegraded":"2023-11-08T19:25:04.598798+0000","last_fullsized":"2023-11-08T19:25:04.598798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656740+0000","last_change":"2023-11-08T19:24:57.670689+0000","last_active":"2023-11-08T19:25:03.656740+0000","last_peered":"2023-11-08T19:25:03.656740+0000","last_clean":"2023-11-08T19:25:03.656740+0000","last_became_active":"2023-11-08T19:24:57.670406+0000","last_became_peered":"2023-11-08T19:24:57.670406+0000","last_unstale":"2023-11-08T19:25:03.656740+0000","last_undegraded":"2023-11-08T19:25:03.656740+0000","last_fullsized":"2023-11-08T19:25:03.656740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600045+0000","last_change":"2023-11-08T19:24:58.629107+0000","last_active":"2023-11-08T19:25:03.600045+0000","last_peered":"2023-11-08T19:25:03.600045+0000","last_clean":"2023-11-08T19:25:03.600045+0000","last_became_active":"2023-11-08T19:24:58.628670+0000","last_became_peered":"2023-11-08T19:24:58.628670+0000","last_unstale":"2023-11-08T19:25:03.600045+0000","last_undegraded":"2023-11-08T19:25:03.600045+0000","last_fullsized":"2023-11-08T19:25:03.600045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601324+0000","last_change":"2023-11-08T19:25:02.727558+0000","last_active":"2023-11-08T19:25:04.601324+0000","last_peered":"2023-11-08T19:25:04.601324+0000","last_clean":"2023-11-08T19:25:04.601324+0000","last_became_active":"2023-11-08T19:25:02.726979+0000","last_became_peered":"2023-11-08T19:25:02.726979+0000","last_unstale":"2023-11-08T19:25:04.601324+0000","last_undegraded":"2023-11-08T19:25:04.601324+0000","last_fullsized":"2023-11-08T19:25:04.601324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655522+0000","last_change":"2023-11-08T19:24:57.673750+0000","last_active":"2023-11-08T19:25:03.655522+0000","last_peered":"2023-11-08T19:25:03.655522+0000","last_clean":"2023-11-08T19:25:03.655522+0000","last_became_active":"2023-11-08T19:24:57.673328+0000","last_became_peered":"2023-11-08T19:24:57.673328+0000","last_unstale":"2023-11-08T19:25:03.655522+0000","last_undegraded":"2023-11-08T19:25:03.655522+0000","last_fullsized":"2023-11-08T19:25:03.655522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600770+0000","last_change":"2023-11-08T19:24:58.628265+0000","last_active":"2023-11-08T19:25:03.600770+0000","last_peered":"2023-11-08T19:25:03.600770+0000","last_clean":"2023-11-08T19:25:03.600770+0000","last_became_active":"2023-11-08T19:24:58.628007+0000","last_became_peered":"2023-11-08T19:24:58.628007+0000","last_unstale":"2023-11-08T19:25:03.600770+0000","last_undegraded":"2023-11-08T19:25:03.600770+0000","last_fullsized":"2023-11-08T19:25:03.600770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601053+0000","last_change":"2023-11-08T19:25:02.726316+0000","last_active":"2023-11-08T19:25:04.601053+0000","last_peered":"2023-11-08T19:25:04.601053+0000","last_clean":"2023-11-08T19:25:04.601053+0000","last_became_active":"2023-11-08T19:25:02.725661+0000","last_became_peered":"2023-11-08T19:25:02.725661+0000","last_unstale":"2023-11-08T19:25:04.601053+0000","last_undegraded":"2023-11-08T19:25:04.601053+0000","last_fullsized":"2023-11-08T19:25:04.601053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:56.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596618+0000","last_change":"2023-11-08T19:24:58.626589+0000","last_active":"2023-11-08T19:25:03.596618+0000","last_peered":"2023-11-08T19:25:03.596618+0000","last_clean":"2023-11-08T19:25:03.596618+0000","last_became_active":"2023-11-08T19:24:58.626269+0000","last_became_peered":"2023-11-08T19:24:58.626269+0000","last_unstale":"2023-11-08T19:25:03.596618+0000","last_undegraded":"2023-11-08T19:25:03.596618+0000","last_fullsized":"2023-11-08T19:25:03.596618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:57.668738+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:57.668319+0000","last_became_peered":"2023-11-08T19:24:57.668319+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.603469+0000","last_change":"2023-11-08T19:25:02.731537+0000","last_active":"2023-11-08T19:25:04.603469+0000","last_peered":"2023-11-08T19:25:04.603469+0000","last_clean":"2023-11-08T19:25:04.603469+0000","last_became_active":"2023-11-08T19:25:02.731004+0000","last_became_peered":"2023-11-08T19:25:02.731004+0000","last_unstale":"2023-11-08T19:25:04.603469+0000","last_undegraded":"2023-11-08T19:25:04.603469+0000","last_fullsized":"2023-11-08T19:25:04.603469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:58.628177+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:58.627584+0000","last_became_peered":"2023-11-08T19:24:58.627584+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643213+0000","last_change":"2023-11-08T19:24:57.659565+0000","last_active":"2023-11-08T19:25:03.643213+0000","last_peered":"2023-11-08T19:25:03.643213+0000","last_clean":"2023-11-08T19:25:03.643213+0000","last_became_active":"2023-11-08T19:24:57.658443+0000","last_became_peered":"2023-11-08T19:24:57.658443+0000","last_unstale":"2023-11-08T19:25:03.643213+0000","last_undegraded":"2023-11-08T19:25:03.643213+0000","last_fullsized":"2023-11-08T19:25:03.643213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602536+0000","last_change":"2023-11-08T19:25:02.734167+0000","last_active":"2023-11-08T19:25:04.602536+0000","last_peered":"2023-11-08T19:25:04.602536+0000","last_clean":"2023-11-08T19:25:04.602536+0000","last_became_active":"2023-11-08T19:25:02.733732+0000","last_became_peered":"2023-11-08T19:25:02.733732+0000","last_unstale":"2023-11-08T19:25:04.602536+0000","last_undegraded":"2023-11-08T19:25:04.602536+0000","last_fullsized":"2023-11-08T19:25:04.602536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655085+0000","last_change":"2023-11-08T19:24:58.627099+0000","last_active":"2023-11-08T19:25:03.655085+0000","last_peered":"2023-11-08T19:25:03.655085+0000","last_clean":"2023-11-08T19:25:03.655085+0000","last_became_active":"2023-11-08T19:24:58.626261+0000","last_became_peered":"2023-11-08T19:24:58.626261+0000","last_unstale":"2023-11-08T19:25:03.655085+0000","last_undegraded":"2023-11-08T19:25:03.655085+0000","last_fullsized":"2023-11-08T19:25:03.655085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642570+0000","last_change":"2023-11-08T19:24:57.658778+0000","last_active":"2023-11-08T19:25:03.642570+0000","last_peered":"2023-11-08T19:25:03.642570+0000","last_clean":"2023-11-08T19:25:03.642570+0000","last_became_active":"2023-11-08T19:24:57.657370+0000","last_became_peered":"2023-11-08T19:24:57.657370+0000","last_unstale":"2023-11-08T19:25:03.642570+0000","last_undegraded":"2023-11-08T19:25:03.642570+0000","last_fullsized":"2023-11-08T19:25:03.642570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422591+0000","last_change":"2023-11-08T19:25:03.478005+0000","last_active":"2023-11-08T19:25:05.422591+0000","last_peered":"2023-11-08T19:25:05.422591+0000","last_clean":"2023-11-08T19:25:05.422591+0000","last_became_active":"2023-11-08T19:25:03.476666+0000","last_became_peered":"2023-11-08T19:25:03.476666+0000","last_unstale":"2023-11-08T19:25:05.422591+0000","last_undegraded":"2023-11-08T19:25:05.422591+0000","last_fullsized":"2023-11-08T19:25:05.422591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656312+0000","last_change":"2023-11-08T19:24:58.623061+0000","last_active":"2023-11-08T19:25:03.656312+0000","last_peered":"2023-11-08T19:25:03.656312+0000","last_clean":"2023-11-08T19:25:03.656312+0000","last_became_active":"2023-11-08T19:24:58.618367+0000","last_became_peered":"2023-11-08T19:24:58.618367+0000","last_unstale":"2023-11-08T19:25:03.656312+0000","last_undegraded":"2023-11-08T19:25:03.656312+0000","last_fullsized":"2023-11-08T19:25:03.656312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:08.538170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.653991+0000","last_change":"2023-11-08T19:24:57.666853+0000","last_active":"2023-11-08T19:25:03.653991+0000","last_peered":"2023-11-08T19:25:03.653991+0000","last_clean":"2023-11-08T19:25:03.653991+0000","last_became_active":"2023-11-08T19:24:57.666539+0000","last_became_peered":"2023-11-08T19:24:57.666539+0000","last_unstale":"2023-11-08T19:25:03.653991+0000","last_undegraded":"2023-11-08T19:25:03.653991+0000","last_fullsized":"2023-11-08T19:25:03.653991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602888+0000","last_change":"2023-11-08T19:25:02.727013+0000","last_active":"2023-11-08T19:25:04.602888+0000","last_peered":"2023-11-08T19:25:04.602888+0000","last_clean":"2023-11-08T19:25:04.602888+0000","last_became_active":"2023-11-08T19:25:02.726496+0000","last_became_peered":"2023-11-08T19:25:02.726496+0000","last_unstale":"2023-11-08T19:25:04.602888+0000","last_undegraded":"2023-11-08T19:25:04.602888+0000","last_fullsized":"2023-11-08T19:25:04.602888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655207+0000","last_change":"2023-11-08T19:24:58.620701+0000","last_active":"2023-11-08T19:25:03.655207+0000","last_peered":"2023-11-08T19:25:03.655207+0000","last_clean":"2023-11-08T19:25:03.655207+0000","last_became_active":"2023-11-08T19:24:58.620386+0000","last_became_peered":"2023-11-08T19:24:58.620386+0000","last_unstale":"2023-11-08T19:25:03.655207+0000","last_undegraded":"2023-11-08T19:25:03.655207+0000","last_fullsized":"2023-11-08T19:25:03.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597904+0000","last_change":"2023-11-08T19:24:57.667152+0000","last_active":"2023-11-08T19:25:03.597904+0000","last_peered":"2023-11-08T19:25:03.597904+0000","last_clean":"2023-11-08T19:25:03.597904+0000","last_became_active":"2023-11-08T19:24:57.666522+0000","last_became_peered":"2023-11-08T19:24:57.666522+0000","last_unstale":"2023-11-08T19:25:03.597904+0000","last_undegraded":"2023-11-08T19:25:03.597904+0000","last_fullsized":"2023-11-08T19:25:03.597904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:33.630876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599610+0000","last_change":"2023-11-08T19:25:03.469823+0000","last_active":"2023-11-08T19:25:04.599610+0000","last_peered":"2023-11-08T19:25:04.599610+0000","last_clean":"2023-11-08T19:25:04.599610+0000","last_became_active":"2023-11-08T19:25:03.468656+0000","last_became_peered":"2023-11-08T19:25:03.468656+0000","last_unstale":"2023-11-08T19:25:04.599610+0000","last_undegraded":"2023-11-08T19:25:04.599610+0000","last_fullsized":"2023-11-08T19:25:04.599610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.633451+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.633159+0000","last_became_peered":"2023-11-08T19:24:58.633159+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.644112+0000","last_change":"2023-11-08T19:24:57.663788+0000","last_active":"2023-11-08T19:25:03.644112+0000","last_peered":"2023-11-08T19:25:03.644112+0000","last_clean":"2023-11-08T19:25:03.644112+0000","last_became_active":"2023-11-08T19:24:57.663457+0000","last_became_peered":"2023-11-08T19:24:57.663457+0000","last_unstale":"2023-11-08T19:25:03.644112+0000","last_undegraded":"2023-11-08T19:25:03.644112+0000","last_fullsized":"2023-11-08T19:25:03.644112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593332+0000","last_change":"2023-11-08T19:25:03.469891+0000","last_active":"2023-11-08T19:25:04.593332+0000","last_peered":"2023-11-08T19:25:04.593332+0000","last_clean":"2023-11-08T19:25:04.593332+0000","last_became_active":"2023-11-08T19:25:03.466854+0000","last_became_peered":"2023-11-08T19:25:03.466854+0000","last_unstale":"2023-11-08T19:25:04.593332+0000","last_undegraded":"2023-11-08T19:25:04.593332+0000","last_fullsized":"2023-11-08T19:25:04.593332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:05.405858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643596+0000","last_change":"2023-11-08T19:24:58.622842+0000","last_active":"2023-11-08T19:25:03.643596+0000","last_peered":"2023-11-08T19:25:03.643596+0000","last_clean":"2023-11-08T19:25:03.643596+0000","last_became_active":"2023-11-08T19:24:58.622530+0000","last_became_peered":"2023-11-08T19:24:58.622530+0000","last_unstale":"2023-11-08T19:25:03.643596+0000","last_undegraded":"2023-11-08T19:25:03.643596+0000","last_fullsized":"2023-11-08T19:25:03.643596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656961+0000","last_change":"2023-11-08T19:24:57.669407+0000","last_active":"2023-11-08T19:25:03.656961+0000","last_peered":"2023-11-08T19:25:03.656961+0000","last_clean":"2023-11-08T19:25:03.656961+0000","last_became_active":"2023-11-08T19:24:57.669169+0000","last_became_peered":"2023-11-08T19:24:57.669169+0000","last_unstale":"2023-11-08T19:25:03.656961+0000","last_undegraded":"2023-11-08T19:25:03.656961+0000","last_fullsized":"2023-11-08T19:25:03.656961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:50.920808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530152+0000","last_change":"2023-11-08T19:25:03.469685+0000","last_active":"2023-11-08T19:25:05.530152+0000","last_peered":"2023-11-08T19:25:05.530152+0000","last_clean":"2023-11-08T19:25:05.530152+0000","last_became_active":"2023-11-08T19:25:03.468600+0000","last_became_peered":"2023-11-08T19:25:03.468600+0000","last_unstale":"2023-11-08T19:25:05.530152+0000","last_undegraded":"2023-11-08T19:25:05.530152+0000","last_fullsized":"2023-11-08T19:25:05.530152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600914+0000","last_change":"2023-11-08T19:24:58.617568+0000","last_active":"2023-11-08T19:25:03.600914+0000","last_peered":"2023-11-08T19:25:03.600914+0000","last_clean":"2023-11-08T19:25:03.600914+0000","last_became_active":"2023-11-08T19:24:58.617216+0000","last_became_peered":"2023-11-08T19:24:58.617216+0000","last_unstale":"2023-11-08T19:25:03.600914+0000","last_undegraded":"2023-11-08T19:25:03.600914+0000","last_fullsized":"2023-11-08T19:25:03.600914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643888+0000","last_change":"2023-11-08T19:24:57.653655+0000","last_active":"2023-11-08T19:25:03.643888+0000","last_peered":"2023-11-08T19:25:03.643888+0000","last_clean":"2023-11-08T19:25:03.643888+0000","last_became_active":"2023-11-08T19:24:57.653390+0000","last_became_peered":"2023-11-08T19:24:57.653390+0000","last_unstale":"2023-11-08T19:25:03.643888+0000","last_undegraded":"2023-11-08T19:25:03.643888+0000","last_fullsized":"2023-11-08T19:25:03.643888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421356+0000","last_change":"2023-11-08T19:25:02.724803+0000","last_active":"2023-11-08T19:25:05.421356+0000","last_peered":"2023-11-08T19:25:05.421356+0000","last_clean":"2023-11-08T19:25:05.421356+0000","last_became_active":"2023-11-08T19:25:02.724278+0000","last_became_peered":"2023-11-08T19:25:02.724278+0000","last_unstale":"2023-11-08T19:25:05.421356+0000","last_undegraded":"2023-11-08T19:25:05.421356+0000","last_fullsized":"2023-11-08T19:25:05.421356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655769+0000","last_change":"2023-11-08T19:24:58.617441+0000","last_active":"2023-11-08T19:25:03.655769+0000","last_peered":"2023-11-08T19:25:03.655769+0000","last_clean":"2023-11-08T19:25:03.655769+0000","last_became_active":"2023-11-08T19:24:58.617030+0000","last_became_peered":"2023-11-08T19:24:58.617030+0000","last_unstale":"2023-11-08T19:25:03.655769+0000","last_undegraded":"2023-11-08T19:25:03.655769+0000","last_fullsized":"2023-11-08T19:25:03.655769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:50.043019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599773+0000","last_change":"2023-11-08T19:24:57.662494+0000","last_active":"2023-11-08T19:25:03.599773+0000","last_peered":"2023-11-08T19:25:03.599773+0000","last_clean":"2023-11-08T19:25:03.599773+0000","last_became_active":"2023-11-08T19:24:57.662223+0000","last_became_peered":"2023-11-08T19:24:57.662223+0000","last_unstale":"2023-11-08T19:25:03.599773+0000","last_undegraded":"2023-11-08T19:25:03.599773+0000","last_fullsized":"2023-11-08T19:25:03.599773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:49.940163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421797+0000","last_change":"2023-11-08T19:25:02.729279+0000","last_active":"2023-11-08T19:25:05.421797+0000","last_peered":"2023-11-08T19:25:05.421797+0000","last_clean":"2023-11-08T19:25:05.421797+0000","last_became_active":"2023-11-08T19:25:02.728855+0000","last_became_peered":"2023-11-08T19:25:02.728855+0000","last_unstale":"2023-11-08T19:25:05.421797+0000","last_undegraded":"2023-11-08T19:25:05.421797+0000","last_fullsized":"2023-11-08T19:25:05.421797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642895+0000","last_change":"2023-11-08T19:24:58.617415+0000","last_active":"2023-11-08T19:25:03.642895+0000","last_peered":"2023-11-08T19:25:03.642895+0000","last_clean":"2023-11-08T19:25:03.642895+0000","last_became_active":"2023-11-08T19:24:58.616880+0000","last_became_peered":"2023-11-08T19:24:58.616880+0000","last_unstale":"2023-11-08T19:25:03.642895+0000","last_undegraded":"2023-11-08T19:25:03.642895+0000","last_fullsized":"2023-11-08T19:25:03.642895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592969+0000","last_change":"2023-11-08T19:24:57.650498+0000","last_active":"2023-11-08T19:25:03.592969+0000","last_peered":"2023-11-08T19:25:03.592969+0000","last_clean":"2023-11-08T19:25:03.592969+0000","last_became_active":"2023-11-08T19:24:57.650172+0000","last_became_peered":"2023-11-08T19:24:57.650172+0000","last_unstale":"2023-11-08T19:25:03.592969+0000","last_undegraded":"2023-11-08T19:25:03.592969+0000","last_fullsized":"2023-11-08T19:25:03.592969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422691+0000","last_change":"2023-11-08T19:25:02.722430+0000","last_active":"2023-11-08T19:25:05.422691+0000","last_peered":"2023-11-08T19:25:05.422691+0000","last_clean":"2023-11-08T19:25:05.422691+0000","last_became_active":"2023-11-08T19:25:02.722013+0000","last_became_peered":"2023-11-08T19:25:02.722013+0000","last_unstale":"2023-11-08T19:25:05.422691+0000","last_undegraded":"2023-11-08T19:25:05.422691+0000","last_fullsized":"2023-11-08T19:25:05.422691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654808+0000","last_change":"2023-11-08T19:24:58.629901+0000","last_active":"2023-11-08T19:25:03.654808+0000","last_peered":"2023-11-08T19:25:03.654808+0000","last_clean":"2023-11-08T19:25:03.654808+0000","last_became_active":"2023-11-08T19:24:58.629603+0000","last_became_peered":"2023-11-08T19:24:58.629603+0000","last_unstale":"2023-11-08T19:25:03.654808+0000","last_undegraded":"2023-11-08T19:25:03.654808+0000","last_fullsized":"2023-11-08T19:25:03.654808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:19.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654594+0000","last_change":"2023-11-08T19:24:57.667258+0000","last_active":"2023-11-08T19:25:03.654594+0000","last_peered":"2023-11-08T19:25:03.654594+0000","last_clean":"2023-11-08T19:25:03.654594+0000","last_became_active":"2023-11-08T19:24:57.666859+0000","last_became_peered":"2023-11-08T19:24:57.666859+0000","last_unstale":"2023-11-08T19:25:03.654594+0000","last_undegraded":"2023-11-08T19:25:03.654594+0000","last_fullsized":"2023-11-08T19:25:03.654594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530805+0000","last_change":"2023-11-08T19:25:03.480894+0000","last_active":"2023-11-08T19:25:05.530805+0000","last_peered":"2023-11-08T19:25:05.530805+0000","last_clean":"2023-11-08T19:25:05.530805+0000","last_became_active":"2023-11-08T19:25:03.479219+0000","last_became_peered":"2023-11-08T19:25:03.479219+0000","last_unstale":"2023-11-08T19:25:05.530805+0000","last_undegraded":"2023-11-08T19:25:05.530805+0000","last_fullsized":"2023-11-08T19:25:05.530805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656729+0000","last_change":"2023-11-08T19:24:58.620051+0000","last_active":"2023-11-08T19:25:03.656729+0000","last_peered":"2023-11-08T19:25:03.656729+0000","last_clean":"2023-11-08T19:25:03.656729+0000","last_became_active":"2023-11-08T19:24:58.619641+0000","last_became_peered":"2023-11-08T19:24:58.619641+0000","last_unstale":"2023-11-08T19:25:03.656729+0000","last_undegraded":"2023-11-08T19:25:03.656729+0000","last_fullsized":"2023-11-08T19:25:03.656729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599436+0000","last_change":"2023-11-08T19:24:57.656028+0000","last_active":"2023-11-08T19:25:03.599436+0000","last_peered":"2023-11-08T19:25:03.599436+0000","last_clean":"2023-11-08T19:25:03.599436+0000","last_became_active":"2023-11-08T19:24:57.651726+0000","last_became_peered":"2023-11-08T19:24:57.651726+0000","last_unstale":"2023-11-08T19:25:03.599436+0000","last_undegraded":"2023-11-08T19:25:03.599436+0000","last_fullsized":"2023-11-08T19:25:03.599436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654244+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:25:03.654244+0000","last_peered":"2023-11-08T19:25:03.654244+0000","last_clean":"2023-11-08T19:25:03.654244+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:25:03.654244+0000","last_undegraded":"2023-11-08T19:25:03.654244+0000","last_fullsized":"2023-11-08T19:25:03.654244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349456+0000","last_change":"2023-11-08T19:25:03.468384+0000","last_active":"2023-11-08T19:25:06.349456+0000","last_peered":"2023-11-08T19:25:06.349456+0000","last_clean":"2023-11-08T19:25:06.349456+0000","last_became_active":"2023-11-08T19:25:03.467219+0000","last_became_peered":"2023-11-08T19:25:03.467219+0000","last_unstale":"2023-11-08T19:25:06.349456+0000","last_undegraded":"2023-11-08T19:25:06.349456+0000","last_fullsized":"2023-11-08T19:25:06.349456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656350+0000","last_change":"2023-11-08T19:24:58.632161+0000","last_active":"2023-11-08T19:25:03.656350+0000","last_peered":"2023-11-08T19:25:03.656350+0000","last_clean":"2023-11-08T19:25:03.656350+0000","last_became_active":"2023-11-08T19:24:58.631360+0000","last_became_peered":"2023-11-08T19:24:58.631360+0000","last_unstale":"2023-11-08T19:25:03.656350+0000","last_undegraded":"2023-11-08T19:25:03.656350+0000","last_fullsized":"2023-11-08T19:25:03.656350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594245+0000","last_change":"2023-11-08T19:24:57.659462+0000","last_active":"2023-11-08T19:25:03.594245+0000","last_peered":"2023-11-08T19:25:03.594245+0000","last_clean":"2023-11-08T19:25:03.594245+0000","last_became_active":"2023-11-08T19:24:57.659200+0000","last_became_peered":"2023-11-08T19:24:57.659200+0000","last_unstale":"2023-11-08T19:25:03.594245+0000","last_undegraded":"2023-11-08T19:25:03.594245+0000","last_fullsized":"2023-11-08T19:25:03.594245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:21.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.348403+0000","last_change":"2023-11-08T19:25:03.468319+0000","last_active":"2023-11-08T19:25:06.348403+0000","last_peered":"2023-11-08T19:25:06.348403+0000","last_clean":"2023-11-08T19:25:06.348403+0000","last_became_active":"2023-11-08T19:25:03.466962+0000","last_became_peered":"2023-11-08T19:25:03.466962+0000","last_unstale":"2023-11-08T19:25:06.348403+0000","last_undegraded":"2023-11-08T19:25:06.348403+0000","last_fullsized":"2023-11-08T19:25:06.348403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600157+0000","last_change":"2023-11-08T19:24:58.620109+0000","last_active":"2023-11-08T19:25:03.600157+0000","last_peered":"2023-11-08T19:25:03.600157+0000","last_clean":"2023-11-08T19:25:03.600157+0000","last_became_active":"2023-11-08T19:24:58.619504+0000","last_became_peered":"2023-11-08T19:24:58.619504+0000","last_unstale":"2023-11-08T19:25:03.600157+0000","last_undegraded":"2023-11-08T19:25:03.600157+0000","last_fullsized":"2023-11-08T19:25:03.600157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643946+0000","last_change":"2023-11-08T19:24:57.658265+0000","last_active":"2023-11-08T19:25:03.643946+0000","last_peered":"2023-11-08T19:25:03.643946+0000","last_clean":"2023-11-08T19:25:03.643946+0000","last_became_active":"2023-11-08T19:24:57.657921+0000","last_became_peered":"2023-11-08T19:24:57.657921+0000","last_unstale":"2023-11-08T19:25:03.643946+0000","last_undegraded":"2023-11-08T19:25:03.643946+0000","last_fullsized":"2023-11-08T19:25:03.643946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530244+0000","last_change":"2023-11-08T19:25:02.727032+0000","last_active":"2023-11-08T19:25:05.530244+0000","last_peered":"2023-11-08T19:25:05.530244+0000","last_clean":"2023-11-08T19:25:05.530244+0000","last_became_active":"2023-11-08T19:25:02.726636+0000","last_became_peered":"2023-11-08T19:25:02.726636+0000","last_unstale":"2023-11-08T19:25:05.530244+0000","last_undegraded":"2023-11-08T19:25:05.530244+0000","last_fullsized":"2023-11-08T19:25:05.530244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656744+0000","last_change":"2023-11-08T19:24:58.641465+0000","last_active":"2023-11-08T19:25:03.656744+0000","last_peered":"2023-11-08T19:25:03.656744+0000","last_clean":"2023-11-08T19:25:03.656744+0000","last_became_active":"2023-11-08T19:24:58.641144+0000","last_became_peered":"2023-11-08T19:24:58.641144+0000","last_unstale":"2023-11-08T19:25:03.656744+0000","last_undegraded":"2023-11-08T19:25:03.656744+0000","last_fullsized":"2023-11-08T19:25:03.656744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:00.683350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594008+0000","last_change":"2023-11-08T19:24:57.653180+0000","last_active":"2023-11-08T19:25:03.594008+0000","last_peered":"2023-11-08T19:25:03.594008+0000","last_clean":"2023-11-08T19:25:03.594008+0000","last_became_active":"2023-11-08T19:24:57.652617+0000","last_became_peered":"2023-11-08T19:24:57.652617+0000","last_unstale":"2023-11-08T19:25:03.594008+0000","last_undegraded":"2023-11-08T19:25:03.594008+0000","last_fullsized":"2023-11-08T19:25:03.594008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:35.866163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599737+0000","last_change":"2023-11-08T19:25:03.474070+0000","last_active":"2023-11-08T19:25:04.599737+0000","last_peered":"2023-11-08T19:25:04.599737+0000","last_clean":"2023-11-08T19:25:04.599737+0000","last_became_active":"2023-11-08T19:25:03.471994+0000","last_became_peered":"2023-11-08T19:25:03.471994+0000","last_unstale":"2023-11-08T19:25:04.599737+0000","last_undegraded":"2023-11-08T19:25:04.599737+0000","last_fullsized":"2023-11-08T19:25:04.599737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655993+0000","last_change":"2023-11-08T19:24:58.640557+0000","last_active":"2023-11-08T19:25:03.655993+0000","last_peered":"2023-11-08T19:25:03.655993+0000","last_clean":"2023-11-08T19:25:03.655993+0000","last_became_active":"2023-11-08T19:24:58.640005+0000","last_became_peered":"2023-11-08T19:24:58.640005+0000","last_unstale":"2023-11-08T19:25:03.655993+0000","last_undegraded":"2023-11-08T19:25:03.655993+0000","last_fullsized":"2023-11-08T19:25:03.655993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657695+0000","last_change":"2023-11-08T19:24:57.652195+0000","last_active":"2023-11-08T19:25:03.657695+0000","last_peered":"2023-11-08T19:25:03.657695+0000","last_clean":"2023-11-08T19:25:03.657695+0000","last_became_active":"2023-11-08T19:24:57.651790+0000","last_became_peered":"2023-11-08T19:24:57.651790+0000","last_unstale":"2023-11-08T19:25:03.657695+0000","last_undegraded":"2023-11-08T19:25:03.657695+0000","last_fullsized":"2023-11-08T19:25:03.657695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:50:21.039412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599054+0000","last_change":"2023-11-08T19:25:02.722727+0000","last_active":"2023-11-08T19:25:04.599054+0000","last_peered":"2023-11-08T19:25:04.599054+0000","last_clean":"2023-11-08T19:25:04.599054+0000","last_became_active":"2023-11-08T19:25:02.722092+0000","last_became_peered":"2023-11-08T19:25:02.722092+0000","last_unstale":"2023-11-08T19:25:04.599054+0000","last_undegraded":"2023-11-08T19:25:04.599054+0000","last_fullsized":"2023-11-08T19:25:04.599054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655407+0000","last_change":"2023-11-08T19:24:58.635248+0000","last_active":"2023-11-08T19:25:03.655407+0000","last_peered":"2023-11-08T19:25:03.655407+0000","last_clean":"2023-11-08T19:25:03.655407+0000","last_became_active":"2023-11-08T19:24:58.634504+0000","last_became_peered":"2023-11-08T19:24:58.634504+0000","last_unstale":"2023-11-08T19:25:03.655407+0000","last_undegraded":"2023-11-08T19:25:03.655407+0000","last_fullsized":"2023-11-08T19:25:03.655407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592589+0000","last_change":"2023-11-08T19:24:57.651955+0000","last_active":"2023-11-08T19:25:03.592589+0000","last_peered":"2023-11-08T19:25:03.592589+0000","last_clean":"2023-11-08T19:25:03.592589+0000","last_became_active":"2023-11-08T19:24:57.651666+0000","last_became_peered":"2023-11-08T19:24:57.651666+0000","last_unstale":"2023-11-08T19:25:03.592589+0000","last_undegraded":"2023-11-08T19:25:03.592589+0000","last_fullsized":"2023-11-08T19:25:03.592589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530618+0000","last_change":"2023-11-08T19:25:02.728613+0000","last_active":"2023-11-08T19:25:05.530618+0000","last_peered":"2023-11-08T19:25:05.530618+0000","last_clean":"2023-11-08T19:25:05.530618+0000","last_became_active":"2023-11-08T19:25:02.727749+0000","last_became_peered":"2023-11-08T19:25:02.727749+0000","last_unstale":"2023-11-08T19:25:05.530618+0000","last_undegraded":"2023-11-08T19:25:05.530618+0000","last_fullsized":"2023-11-08T19:25:05.530618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595934+0000","last_change":"2023-11-08T19:24:58.616453+0000","last_active":"2023-11-08T19:25:03.595934+0000","last_peered":"2023-11-08T19:25:03.595934+0000","last_clean":"2023-11-08T19:25:03.595934+0000","last_became_active":"2023-11-08T19:24:58.615934+0000","last_became_peered":"2023-11-08T19:24:58.615934+0000","last_unstale":"2023-11-08T19:25:03.595934+0000","last_undegraded":"2023-11-08T19:25:03.595934+0000","last_fullsized":"2023-11-08T19:25:03.595934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596954+0000","last_change":"2023-11-08T19:24:57.659585+0000","last_active":"2023-11-08T19:25:03.596954+0000","last_peered":"2023-11-08T19:25:03.596954+0000","last_clean":"2023-11-08T19:25:03.596954+0000","last_became_active":"2023-11-08T19:24:57.659197+0000","last_became_peered":"2023-11-08T19:24:57.659197+0000","last_unstale":"2023-11-08T19:25:03.596954+0000","last_undegraded":"2023-11-08T19:25:03.596954+0000","last_fullsized":"2023-11-08T19:25:03.596954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595745+0000","last_change":"2023-11-08T19:25:03.468147+0000","last_active":"2023-11-08T19:25:04.595745+0000","last_peered":"2023-11-08T19:25:04.595745+0000","last_clean":"2023-11-08T19:25:04.595745+0000","last_became_active":"2023-11-08T19:25:03.466921+0000","last_became_peered":"2023-11-08T19:25:03.466921+0000","last_unstale":"2023-11-08T19:25:04.595745+0000","last_undegraded":"2023-11-08T19:25:04.595745+0000","last_fullsized":"2023-11-08T19:25:04.595745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:14:51.027517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656098+0000","last_change":"2023-11-08T19:24:58.636188+0000","last_active":"2023-11-08T19:25:03.656098+0000","last_peered":"2023-11-08T19:25:03.656098+0000","last_clean":"2023-11-08T19:25:03.656098+0000","last_became_active":"2023-11-08T19:24:58.635889+0000","last_became_peered":"2023-11-08T19:24:58.635889+0000","last_unstale":"2023-11-08T19:25:03.656098+0000","last_undegraded":"2023-11-08T19:25:03.656098+0000","last_fullsized":"2023-11-08T19:25:03.656098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599561+0000","last_change":"2023-11-08T19:24:57.655323+0000","last_active":"2023-11-08T19:25:03.599561+0000","last_peered":"2023-11-08T19:25:03.599561+0000","last_clean":"2023-11-08T19:25:03.599561+0000","last_became_active":"2023-11-08T19:24:57.654973+0000","last_became_peered":"2023-11-08T19:24:57.654973+0000","last_unstale":"2023-11-08T19:25:03.599561+0000","last_undegraded":"2023-11-08T19:25:03.599561+0000","last_fullsized":"2023-11-08T19:25:03.599561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594252+0000","last_change":"2023-11-08T19:25:03.467735+0000","last_active":"2023-11-08T19:25:04.594252+0000","last_peered":"2023-11-08T19:25:04.594252+0000","last_clean":"2023-11-08T19:25:04.594252+0000","last_became_active":"2023-11-08T19:25:03.466595+0000","last_became_peered":"2023-11-08T19:25:03.466595+0000","last_unstale":"2023-11-08T19:25:04.594252+0000","last_undegraded":"2023-11-08T19:25:04.594252+0000","last_fullsized":"2023-11-08T19:25:04.594252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655761+0000","last_change":"2023-11-08T19:24:58.626959+0000","last_active":"2023-11-08T19:25:03.655761+0000","last_peered":"2023-11-08T19:25:03.655761+0000","last_clean":"2023-11-08T19:25:03.655761+0000","last_became_active":"2023-11-08T19:24:58.626616+0000","last_became_peered":"2023-11-08T19:24:58.626616+0000","last_unstale":"2023-11-08T19:25:03.655761+0000","last_undegraded":"2023-11-08T19:25:03.655761+0000","last_fullsized":"2023-11-08T19:25:03.655761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:57.663161+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:57.660362+0000","last_became_peered":"2023-11-08T19:24:57.660362+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602355+0000","last_change":"2023-11-08T19:25:03.477682+0000","last_active":"2023-11-08T19:25:04.602355+0000","last_peered":"2023-11-08T19:25:04.602355+0000","last_clean":"2023-11-08T19:25:04.602355+0000","last_became_active":"2023-11-08T19:25:03.476098+0000","last_became_peered":"2023-11-08T19:25:03.476098+0000","last_unstale":"2023-11-08T19:25:04.602355+0000","last_undegraded":"2023-11-08T19:25:04.602355+0000","last_fullsized":"2023-11-08T19:25:04.602355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656221+0000","last_change":"2023-11-08T19:24:58.634601+0000","last_active":"2023-11-08T19:25:03.656221+0000","last_peered":"2023-11-08T19:25:03.656221+0000","last_clean":"2023-11-08T19:25:03.656221+0000","last_became_active":"2023-11-08T19:24:58.632876+0000","last_became_peered":"2023-11-08T19:24:58.632876+0000","last_unstale":"2023-11-08T19:25:03.656221+0000","last_undegraded":"2023-11-08T19:25:03.656221+0000","last_fullsized":"2023-11-08T19:25:03.656221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:17.407997+0000","last_change":"2023-11-08T19:24:57.660289+0000","last_active":"2023-11-08T19:25:17.407997+0000","last_peered":"2023-11-08T19:25:17.407997+0000","last_clean":"2023-11-08T19:25:17.407997+0000","last_became_active":"2023-11-08T19:24:57.659982+0000","last_became_peered":"2023-11-08T19:24:57.659982+0000","last_unstale":"2023-11-08T19:25:17.407997+0000","last_undegraded":"2023-11-08T19:25:17.407997+0000","last_fullsized":"2023-11-08T19:25:17.407997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421411+0000","last_change":"2023-11-08T19:25:02.718892+0000","last_active":"2023-11-08T19:25:05.421411+0000","last_peered":"2023-11-08T19:25:05.421411+0000","last_clean":"2023-11-08T19:25:05.421411+0000","last_became_active":"2023-11-08T19:25:02.717998+0000","last_became_peered":"2023-11-08T19:25:02.717998+0000","last_unstale":"2023-11-08T19:25:05.421411+0000","last_undegraded":"2023-11-08T19:25:05.421411+0000","last_fullsized":"2023-11-08T19:25:05.421411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:58.629542+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:58.627201+0000","last_became_peered":"2023-11-08T19:24:58.627201+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643852+0000","last_change":"2023-11-08T19:24:57.661152+0000","last_active":"2023-11-08T19:25:03.643852+0000","last_peered":"2023-11-08T19:25:03.643852+0000","last_clean":"2023-11-08T19:25:03.643852+0000","last_became_active":"2023-11-08T19:24:57.660740+0000","last_became_peered":"2023-11-08T19:24:57.660740+0000","last_unstale":"2023-11-08T19:25:03.643852+0000","last_undegraded":"2023-11-08T19:25:03.643852+0000","last_fullsized":"2023-11-08T19:25:03.643852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602299+0000","last_change":"2023-11-08T19:25:02.727543+0000","last_active":"2023-11-08T19:25:04.602299+0000","last_peered":"2023-11-08T19:25:04.602299+0000","last_clean":"2023-11-08T19:25:04.602299+0000","last_became_active":"2023-11-08T19:25:02.726809+0000","last_became_peered":"2023-11-08T19:25:02.726809+0000","last_unstale":"2023-11-08T19:25:04.602299+0000","last_undegraded":"2023-11-08T19:25:04.602299+0000","last_fullsized":"2023-11-08T19:25:04.602299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655948+0000","last_change":"2023-11-08T19:24:58.618188+0000","last_active":"2023-11-08T19:25:03.655948+0000","last_peered":"2023-11-08T19:25:03.655948+0000","last_clean":"2023-11-08T19:25:03.655948+0000","last_became_active":"2023-11-08T19:24:58.617893+0000","last_became_peered":"2023-11-08T19:24:58.617893+0000","last_unstale":"2023-11-08T19:25:03.655948+0000","last_undegraded":"2023-11-08T19:25:03.655948+0000","last_fullsized":"2023-11-08T19:25:03.655948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600932+0000","last_change":"2023-11-08T19:24:57.662272+0000","last_active":"2023-11-08T19:25:03.600932+0000","last_peered":"2023-11-08T19:25:03.600932+0000","last_clean":"2023-11-08T19:25:03.600932+0000","last_became_active":"2023-11-08T19:24:57.662012+0000","last_became_peered":"2023-11-08T19:24:57.662012+0000","last_unstale":"2023-11-08T19:25:03.600932+0000","last_undegraded":"2023-11-08T19:25:03.600932+0000","last_fullsized":"2023-11-08T19:25:03.600932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594552+0000","last_change":"2023-11-08T19:25:02.729257+0000","last_active":"2023-11-08T19:25:04.594552+0000","last_peered":"2023-11-08T19:25:04.594552+0000","last_clean":"2023-11-08T19:25:04.594552+0000","last_became_active":"2023-11-08T19:25:02.727346+0000","last_became_peered":"2023-11-08T19:25:02.727346+0000","last_unstale":"2023-11-08T19:25:04.594552+0000","last_undegraded":"2023-11-08T19:25:04.594552+0000","last_fullsized":"2023-11-08T19:25:04.594552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:45.742494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600772+0000","last_change":"2023-11-08T19:24:58.626572+0000","last_active":"2023-11-08T19:25:03.600772+0000","last_peered":"2023-11-08T19:25:03.600772+0000","last_clean":"2023-11-08T19:25:03.600772+0000","last_became_active":"2023-11-08T19:24:58.626191+0000","last_became_peered":"2023-11-08T19:24:58.626191+0000","last_unstale":"2023-11-08T19:25:03.600772+0000","last_undegraded":"2023-11-08T19:25:03.600772+0000","last_fullsized":"2023-11-08T19:25:03.600772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597705+0000","last_change":"2023-11-08T19:24:57.672262+0000","last_active":"2023-11-08T19:25:03.597705+0000","last_peered":"2023-11-08T19:25:03.597705+0000","last_clean":"2023-11-08T19:25:03.597705+0000","last_became_active":"2023-11-08T19:24:57.671965+0000","last_became_peered":"2023-11-08T19:24:57.671965+0000","last_unstale":"2023-11-08T19:25:03.597705+0000","last_undegraded":"2023-11-08T19:25:03.597705+0000","last_fullsized":"2023-11-08T19:25:03.597705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594345+0000","last_change":"2023-11-08T19:25:03.467509+0000","last_active":"2023-11-08T19:25:04.594345+0000","last_peered":"2023-11-08T19:25:04.594345+0000","last_clean":"2023-11-08T19:25:04.594345+0000","last_became_active":"2023-11-08T19:25:03.466319+0000","last_became_peered":"2023-11-08T19:25:03.466319+0000","last_unstale":"2023-11-08T19:25:04.594345+0000","last_undegraded":"2023-11-08T19:25:04.594345+0000","last_fullsized":"2023-11-08T19:25:04.594345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:44.481960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530577+0000","last_change":"2023-11-08T19:25:03.475886+0000","last_active":"2023-11-08T19:25:05.530577+0000","last_peered":"2023-11-08T19:25:05.530577+0000","last_clean":"2023-11-08T19:25:05.530577+0000","last_became_active":"2023-11-08T19:25:03.473511+0000","last_became_peered":"2023-11-08T19:25:03.473511+0000","last_unstale":"2023-11-08T19:25:05.530577+0000","last_undegraded":"2023-11-08T19:25:05.530577+0000","last_fullsized":"2023-11-08T19:25:05.530577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642306+0000","last_change":"2023-11-08T19:24:58.621533+0000","last_active":"2023-11-08T19:25:03.642306+0000","last_peered":"2023-11-08T19:25:03.642306+0000","last_clean":"2023-11-08T19:25:03.642306+0000","last_became_active":"2023-11-08T19:24:58.621214+0000","last_became_peered":"2023-11-08T19:24:58.621214+0000","last_unstale":"2023-11-08T19:25:03.642306+0000","last_undegraded":"2023-11-08T19:25:03.642306+0000","last_fullsized":"2023-11-08T19:25:03.642306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596137+0000","last_change":"2023-11-08T19:24:57.656298+0000","last_active":"2023-11-08T19:25:03.596137+0000","last_peered":"2023-11-08T19:25:03.596137+0000","last_clean":"2023-11-08T19:25:03.596137+0000","last_became_active":"2023-11-08T19:24:57.655961+0000","last_became_peered":"2023-11-08T19:24:57.655961+0000","last_unstale":"2023-11-08T19:25:03.596137+0000","last_undegraded":"2023-11-08T19:25:03.596137+0000","last_fullsized":"2023-11-08T19:25:03.596137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349973+0000","last_change":"2023-11-08T19:25:02.721600+0000","last_active":"2023-11-08T19:25:06.349973+0000","last_peered":"2023-11-08T19:25:06.349973+0000","last_clean":"2023-11-08T19:25:06.349973+0000","last_became_active":"2023-11-08T19:25:02.721169+0000","last_became_peered":"2023-11-08T19:25:02.721169+0000","last_unstale":"2023-11-08T19:25:06.349973+0000","last_undegraded":"2023-11-08T19:25:06.349973+0000","last_fullsized":"2023-11-08T19:25:06.349973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654845+0000","last_change":"2023-11-08T19:24:58.624276+0000","last_active":"2023-11-08T19:25:03.654845+0000","last_peered":"2023-11-08T19:25:03.654845+0000","last_clean":"2023-11-08T19:25:03.654845+0000","last_became_active":"2023-11-08T19:24:58.623432+0000","last_became_peered":"2023-11-08T19:24:58.623432+0000","last_unstale":"2023-11-08T19:25:03.654845+0000","last_undegraded":"2023-11-08T19:25:03.654845+0000","last_fullsized":"2023-11-08T19:25:03.654845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656023+0000","last_change":"2023-11-08T19:24:57.661677+0000","last_active":"2023-11-08T19:25:03.656023+0000","last_peered":"2023-11-08T19:25:03.656023+0000","last_clean":"2023-11-08T19:25:03.656023+0000","last_became_active":"2023-11-08T19:24:57.661425+0000","last_became_peered":"2023-11-08T19:24:57.661425+0000","last_unstale":"2023-11-08T19:25:03.656023+0000","last_undegraded":"2023-11-08T19:25:03.656023+0000","last_fullsized":"2023-11-08T19:25:03.656023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596849+0000","last_change":"2023-11-08T19:24:57.662729+0000","last_active":"2023-11-08T19:25:03.596849+0000","last_peered":"2023-11-08T19:25:03.596849+0000","last_clean":"2023-11-08T19:25:03.596849+0000","last_became_active":"2023-11-08T19:24:57.662390+0000","last_became_peered":"2023-11-08T19:24:57.662390+0000","last_unstale":"2023-11-08T19:25:03.596849+0000","last_undegraded":"2023-11-08T19:25:03.596849+0000","last_fullsized":"2023-11-08T19:25:03.596849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656750+0000","last_change":"2023-11-08T19:24:58.619164+0000","last_active":"2023-11-08T19:25:03.656750+0000","last_peered":"2023-11-08T19:25:03.656750+0000","last_clean":"2023-11-08T19:25:03.656750+0000","last_became_active":"2023-11-08T19:24:58.618604+0000","last_became_peered":"2023-11-08T19:24:58.618604+0000","last_unstale":"2023-11-08T19:25:03.656750+0000","last_undegraded":"2023-11-08T19:25:03.656750+0000","last_fullsized":"2023-11-08T19:25:03.656750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.514753+0000","last_change":"2023-11-08T19:25:02.720151+0000","last_active":"2023-11-08T19:25:05.514753+0000","last_peered":"2023-11-08T19:25:05.514753+0000","last_clean":"2023-11-08T19:25:05.514753+0000","last_became_active":"2023-11-08T19:25:02.719391+0000","last_became_peered":"2023-11-08T19:25:02.719391+0000","last_unstale":"2023-11-08T19:25:05.514753+0000","last_undegraded":"2023-11-08T19:25:05.514753+0000","last_fullsized":"2023-11-08T19:25:05.514753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:23.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657232+0000","last_change":"2023-11-08T19:24:57.660203+0000","last_active":"2023-11-08T19:25:03.657232+0000","last_peered":"2023-11-08T19:25:03.657232+0000","last_clean":"2023-11-08T19:25:03.657232+0000","last_became_active":"2023-11-08T19:24:57.659913+0000","last_became_peered":"2023-11-08T19:24:57.659913+0000","last_unstale":"2023-11-08T19:25:03.657232+0000","last_undegraded":"2023-11-08T19:25:03.657232+0000","last_fullsized":"2023-11-08T19:25:03.657232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657083+0000","last_change":"2023-11-08T19:24:58.620564+0000","last_active":"2023-11-08T19:25:03.657083+0000","last_peered":"2023-11-08T19:25:03.657083+0000","last_clean":"2023-11-08T19:25:03.657083+0000","last_became_active":"2023-11-08T19:24:58.620072+0000","last_became_peered":"2023-11-08T19:24:58.620072+0000","last_unstale":"2023-11-08T19:25:03.657083+0000","last_undegraded":"2023-11-08T19:25:03.657083+0000","last_fullsized":"2023-11-08T19:25:03.657083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:36.879923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595054+0000","last_change":"2023-11-08T19:25:02.729532+0000","last_active":"2023-11-08T19:25:04.595054+0000","last_peered":"2023-11-08T19:25:04.595054+0000","last_clean":"2023-11-08T19:25:04.595054+0000","last_became_active":"2023-11-08T19:25:02.728431+0000","last_became_peered":"2023-11-08T19:25:02.728431+0000","last_unstale":"2023-11-08T19:25:04.595054+0000","last_undegraded":"2023-11-08T19:25:04.595054+0000","last_fullsized":"2023-11-08T19:25:04.595054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:57.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595597+0000","last_change":"2023-11-08T19:24:58.624056+0000","last_active":"2023-11-08T19:25:03.595597+0000","last_peered":"2023-11-08T19:25:03.595597+0000","last_clean":"2023-11-08T19:25:03.595597+0000","last_became_active":"2023-11-08T19:24:58.623783+0000","last_became_peered":"2023-11-08T19:24:58.623783+0000","last_unstale":"2023-11-08T19:25:03.595597+0000","last_undegraded":"2023-11-08T19:25:03.595597+0000","last_fullsized":"2023-11-08T19:25:03.595597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593245+0000","last_change":"2023-11-08T19:24:58.617678+0000","last_active":"2023-11-08T19:25:03.593245+0000","last_peered":"2023-11-08T19:25:03.593245+0000","last_clean":"2023-11-08T19:25:03.593245+0000","last_became_active":"2023-11-08T19:24:58.616701+0000","last_became_peered":"2023-11-08T19:24:58.616701+0000","last_unstale":"2023-11-08T19:25:03.593245+0000","last_undegraded":"2023-11-08T19:25:03.593245+0000","last_fullsized":"2023-11-08T19:25:03.593245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599324+0000","last_change":"2023-11-08T19:24:58.620961+0000","last_active":"2023-11-08T19:25:03.599324+0000","last_peered":"2023-11-08T19:25:03.599324+0000","last_clean":"2023-11-08T19:25:03.599324+0000","last_became_active":"2023-11-08T19:24:58.620359+0000","last_became_peered":"2023-11-08T19:24:58.620359+0000","last_unstale":"2023-11-08T19:25:03.599324+0000","last_undegraded":"2023-11-08T19:25:03.599324+0000","last_fullsized":"2023-11-08T19:25:03.599324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656545+0000","last_change":"2023-11-08T19:24:58.622475+0000","last_active":"2023-11-08T19:25:03.656545+0000","last_peered":"2023-11-08T19:25:03.656545+0000","last_clean":"2023-11-08T19:25:03.656545+0000","last_became_active":"2023-11-08T19:24:58.622206+0000","last_became_peered":"2023-11-08T19:24:58.622206+0000","last_unstale":"2023-11-08T19:25:03.656545+0000","last_undegraded":"2023-11-08T19:25:03.656545+0000","last_fullsized":"2023-11-08T19:25:03.656545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657638+0000","last_change":"2023-11-08T19:24:58.630303+0000","last_active":"2023-11-08T19:25:03.657638+0000","last_peered":"2023-11-08T19:25:03.657638+0000","last_clean":"2023-11-08T19:25:03.657638+0000","last_became_active":"2023-11-08T19:24:58.629181+0000","last_became_peered":"2023-11-08T19:24:58.629181+0000","last_unstale":"2023-11-08T19:25:03.657638+0000","last_undegraded":"2023-11-08T19:25:03.657638+0000","last_fullsized":"2023-11-08T19:25:03.657638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643449+0000","last_change":"2023-11-08T19:24:58.622627+0000","last_active":"2023-11-08T19:25:03.643449+0000","last_peered":"2023-11-08T19:25:03.643449+0000","last_clean":"2023-11-08T19:25:03.643449+0000","last_became_active":"2023-11-08T19:24:58.622335+0000","last_became_peered":"2023-11-08T19:24:58.622335+0000","last_unstale":"2023-11-08T19:25:03.643449+0000","last_undegraded":"2023-11-08T19:25:03.643449+0000","last_fullsized":"2023-11-08T19:25:03.643449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597541+0000","last_change":"2023-11-08T19:24:58.630440+0000","last_active":"2023-11-08T19:25:03.597541+0000","last_peered":"2023-11-08T19:25:03.597541+0000","last_clean":"2023-11-08T19:25:03.597541+0000","last_became_active":"2023-11-08T19:24:58.629904+0000","last_became_peered":"2023-11-08T19:24:58.629904+0000","last_unstale":"2023-11-08T19:25:03.597541+0000","last_undegraded":"2023-11-08T19:25:03.597541+0000","last_fullsized":"2023-11-08T19:25:03.597541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.627568+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.627243+0000","last_became_peered":"2023-11-08T19:24:58.627243+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595899+0000","last_change":"2023-11-08T19:24:58.622232+0000","last_active":"2023-11-08T19:25:03.595899+0000","last_peered":"2023-11-08T19:25:03.595899+0000","last_clean":"2023-11-08T19:25:03.595899+0000","last_became_active":"2023-11-08T19:24:58.621853+0000","last_became_peered":"2023-11-08T19:24:58.621853+0000","last_unstale":"2023-11-08T19:25:03.595899+0000","last_undegraded":"2023-11-08T19:25:03.595899+0000","last_fullsized":"2023-11-08T19:25:03.595899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654133+0000","last_change":"2023-11-08T19:24:58.630530+0000","last_active":"2023-11-08T19:25:03.654133+0000","last_peered":"2023-11-08T19:25:03.654133+0000","last_clean":"2023-11-08T19:25:03.654133+0000","last_became_active":"2023-11-08T19:24:58.627752+0000","last_became_peered":"2023-11-08T19:24:58.627752+0000","last_unstale":"2023-11-08T19:25:03.654133+0000","last_undegraded":"2023-11-08T19:25:03.654133+0000","last_fullsized":"2023-11-08T19:25:03.654133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599081+0000","last_change":"2023-11-08T19:24:58.626422+0000","last_active":"2023-11-08T19:25:03.599081+0000","last_peered":"2023-11-08T19:25:03.599081+0000","last_clean":"2023-11-08T19:25:03.599081+0000","last_became_active":"2023-11-08T19:24:58.626091+0000","last_became_peered":"2023-11-08T19:24:58.626091+0000","last_unstale":"2023-11-08T19:25:03.599081+0000","last_undegraded":"2023-11-08T19:25:03.599081+0000","last_fullsized":"2023-11-08T19:25:03.599081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656191+0000","last_change":"2023-11-08T19:24:58.630415+0000","last_active":"2023-11-08T19:25:03.656191+0000","last_peered":"2023-11-08T19:25:03.656191+0000","last_clean":"2023-11-08T19:25:03.656191+0000","last_became_active":"2023-11-08T19:24:58.628526+0000","last_became_peered":"2023-11-08T19:24:58.628526+0000","last_unstale":"2023-11-08T19:25:03.656191+0000","last_undegraded":"2023-11-08T19:25:03.656191+0000","last_fullsized":"2023-11-08T19:25:03.656191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596558+0000","last_change":"2023-11-08T19:24:58.628286+0000","last_active":"2023-11-08T19:25:03.596558+0000","last_peered":"2023-11-08T19:25:03.596558+0000","last_clean":"2023-11-08T19:25:03.596558+0000","last_became_active":"2023-11-08T19:24:58.627912+0000","last_became_peered":"2023-11-08T19:24:58.627912+0000","last_unstale":"2023-11-08T19:25:03.596558+0000","last_undegraded":"2023-11-08T19:25:03.596558+0000","last_fullsized":"2023-11-08T19:25:03.596558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656975+0000","last_change":"2023-11-08T19:24:58.633630+0000","last_active":"2023-11-08T19:25:03.656975+0000","last_peered":"2023-11-08T19:25:03.656975+0000","last_clean":"2023-11-08T19:25:03.656975+0000","last_became_active":"2023-11-08T19:24:58.632441+0000","last_became_peered":"2023-11-08T19:24:58.632441+0000","last_unstale":"2023-11-08T19:25:03.656975+0000","last_undegraded":"2023-11-08T19:25:03.656975+0000","last_fullsized":"2023-11-08T19:25:03.656975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593821+0000","last_change":"2023-11-08T19:24:58.623578+0000","last_active":"2023-11-08T19:25:03.593821+0000","last_peered":"2023-11-08T19:25:03.593821+0000","last_clean":"2023-11-08T19:25:03.593821+0000","last_became_active":"2023-11-08T19:24:58.623261+0000","last_became_peered":"2023-11-08T19:24:58.623261+0000","last_unstale":"2023-11-08T19:25:03.593821+0000","last_undegraded":"2023-11-08T19:25:03.593821+0000","last_fullsized":"2023-11-08T19:25:03.593821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:36.969600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.601266+0000","last_change":"2023-11-08T19:24:58.634259+0000","last_active":"2023-11-08T19:25:03.601266+0000","last_peered":"2023-11-08T19:25:03.601266+0000","last_clean":"2023-11-08T19:25:03.601266+0000","last_became_active":"2023-11-08T19:24:58.633708+0000","last_became_peered":"2023-11-08T19:24:58.633708+0000","last_unstale":"2023-11-08T19:25:03.601266+0000","last_undegraded":"2023-11-08T19:25:03.601266+0000","last_fullsized":"2023-11-08T19:25:03.601266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595680+0000","last_change":"2023-11-08T19:24:58.615871+0000","last_active":"2023-11-08T19:25:03.595680+0000","last_peered":"2023-11-08T19:25:03.595680+0000","last_clean":"2023-11-08T19:25:03.595680+0000","last_became_active":"2023-11-08T19:24:58.615590+0000","last_became_peered":"2023-11-08T19:24:58.615590+0000","last_unstale":"2023-11-08T19:25:03.595680+0000","last_undegraded":"2023-11-08T19:25:03.595680+0000","last_fullsized":"2023-11-08T19:25:03.595680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642579+0000","last_change":"2023-11-08T19:24:58.629058+0000","last_active":"2023-11-08T19:25:03.642579+0000","last_peered":"2023-11-08T19:25:03.642579+0000","last_clean":"2023-11-08T19:25:03.642579+0000","last_became_active":"2023-11-08T19:24:58.628656+0000","last_became_peered":"2023-11-08T19:24:58.628656+0000","last_unstale":"2023-11-08T19:25:03.642579+0000","last_undegraded":"2023-11-08T19:25:03.642579+0000","last_fullsized":"2023-11-08T19:25:03.642579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655552+0000","last_change":"2023-11-08T19:24:58.622286+0000","last_active":"2023-11-08T19:25:03.655552+0000","last_peered":"2023-11-08T19:25:03.655552+0000","last_clean":"2023-11-08T19:25:03.655552+0000","last_became_active":"2023-11-08T19:24:58.621993+0000","last_became_peered":"2023-11-08T19:24:58.621993+0000","last_unstale":"2023-11-08T19:25:03.655552+0000","last_undegraded":"2023-11-08T19:25:03.655552+0000","last_fullsized":"2023-11-08T19:25:03.655552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656958+0000","last_change":"2023-11-08T19:24:58.618762+0000","last_active":"2023-11-08T19:25:03.656958+0000","last_peered":"2023-11-08T19:25:03.656958+0000","last_clean":"2023-11-08T19:25:03.656958+0000","last_became_active":"2023-11-08T19:24:58.618468+0000","last_became_peered":"2023-11-08T19:24:58.618468+0000","last_unstale":"2023-11-08T19:25:03.656958+0000","last_undegraded":"2023-11-08T19:25:03.656958+0000","last_fullsized":"2023-11-08T19:25:03.656958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655770+0000","last_change":"2023-11-08T19:24:58.626193+0000","last_active":"2023-11-08T19:25:03.655770+0000","last_peered":"2023-11-08T19:25:03.655770+0000","last_clean":"2023-11-08T19:25:03.655770+0000","last_became_active":"2023-11-08T19:24:58.625897+0000","last_became_peered":"2023-11-08T19:24:58.625897+0000","last_unstale":"2023-11-08T19:25:03.655770+0000","last_undegraded":"2023-11-08T19:25:03.655770+0000","last_fullsized":"2023-11-08T19:25:03.655770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:43.279737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596855+0000","last_change":"2023-11-08T19:24:58.621575+0000","last_active":"2023-11-08T19:25:03.596855+0000","last_peered":"2023-11-08T19:25:03.596855+0000","last_clean":"2023-11-08T19:25:03.596855+0000","last_became_active":"2023-11-08T19:24:58.621278+0000","last_became_peered":"2023-11-08T19:24:58.621278+0000","last_unstale":"2023-11-08T19:25:03.596855+0000","last_undegraded":"2023-11-08T19:25:03.596855+0000","last_fullsized":"2023-11-08T19:25:03.596855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597421+0000","last_change":"2023-11-08T19:24:58.630221+0000","last_active":"2023-11-08T19:25:03.597421+0000","last_peered":"2023-11-08T19:25:03.597421+0000","last_clean":"2023-11-08T19:25:03.597421+0000","last_became_active":"2023-11-08T19:24:58.629514+0000","last_became_peered":"2023-11-08T19:24:58.629514+0000","last_unstale":"2023-11-08T19:25:03.597421+0000","last_undegraded":"2023-11-08T19:25:03.597421+0000","last_fullsized":"2023-11-08T19:25:03.597421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656658+0000","last_change":"2023-11-08T19:24:58.637307+0000","last_active":"2023-11-08T19:25:03.656658+0000","last_peered":"2023-11-08T19:25:03.656658+0000","last_clean":"2023-11-08T19:25:03.656658+0000","last_became_active":"2023-11-08T19:24:58.636926+0000","last_became_peered":"2023-11-08T19:24:58.636926+0000","last_unstale":"2023-11-08T19:25:03.656658+0000","last_undegraded":"2023-11-08T19:25:03.656658+0000","last_fullsized":"2023-11-08T19:25:03.656658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:52.742252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654271+0000","last_change":"2023-11-08T19:24:58.619975+0000","last_active":"2023-11-08T19:25:03.654271+0000","last_peered":"2023-11-08T19:25:03.654271+0000","last_clean":"2023-11-08T19:25:03.654271+0000","last_became_active":"2023-11-08T19:24:58.619676+0000","last_became_peered":"2023-11-08T19:24:58.619676+0000","last_unstale":"2023-11-08T19:25:03.654271+0000","last_undegraded":"2023-11-08T19:25:03.654271+0000","last_fullsized":"2023-11-08T19:25:03.654271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654365+0000","last_change":"2023-11-08T19:24:58.627535+0000","last_active":"2023-11-08T19:25:03.654365+0000","last_peered":"2023-11-08T19:25:03.654365+0000","last_clean":"2023-11-08T19:25:03.654365+0000","last_became_active":"2023-11-08T19:24:58.627205+0000","last_became_peered":"2023-11-08T19:24:58.627205+0000","last_unstale":"2023-11-08T19:25:03.654365+0000","last_undegraded":"2023-11-08T19:25:03.654365+0000","last_fullsized":"2023-11-08T19:25:03.654365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599068+0000","last_change":"2023-11-08T19:24:58.620637+0000","last_active":"2023-11-08T19:25:03.599068+0000","last_peered":"2023-11-08T19:25:03.599068+0000","last_clean":"2023-11-08T19:25:03.599068+0000","last_became_active":"2023-11-08T19:24:58.619919+0000","last_became_peered":"2023-11-08T19:24:58.619919+0000","last_unstale":"2023-11-08T19:25:03.599068+0000","last_undegraded":"2023-11-08T19:25:03.599068+0000","last_fullsized":"2023-11-08T19:25:03.599068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655858+0000","last_change":"2023-11-08T19:24:58.630887+0000","last_active":"2023-11-08T19:25:03.655858+0000","last_peered":"2023-11-08T19:25:03.655858+0000","last_clean":"2023-11-08T19:25:03.655858+0000","last_became_active":"2023-11-08T19:24:58.630010+0000","last_became_peered":"2023-11-08T19:24:58.630010+0000","last_unstale":"2023-11-08T19:25:03.655858+0000","last_undegraded":"2023-11-08T19:25:03.655858+0000","last_fullsized":"2023-11-08T19:25:03.655858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599939+0000","last_change":"2023-11-08T19:24:58.625672+0000","last_active":"2023-11-08T19:25:03.599939+0000","last_peered":"2023-11-08T19:25:03.599939+0000","last_clean":"2023-11-08T19:25:03.599939+0000","last_became_active":"2023-11-08T19:24:58.619366+0000","last_became_peered":"2023-11-08T19:24:58.619366+0000","last_unstale":"2023-11-08T19:25:03.599939+0000","last_undegraded":"2023-11-08T19:25:03.599939+0000","last_fullsized":"2023-11-08T19:25:03.599939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655793+0000","last_change":"2023-11-08T19:24:58.612582+0000","last_active":"2023-11-08T19:25:03.655793+0000","last_peered":"2023-11-08T19:25:03.655793+0000","last_clean":"2023-11-08T19:25:03.655793+0000","last_became_active":"2023-11-08T19:24:58.612199+0000","last_became_peered":"2023-11-08T19:24:58.612199+0000","last_unstale":"2023-11-08T19:25:03.655793+0000","last_undegraded":"2023-11-08T19:25:03.655793+0000","last_fullsized":"2023-11-08T19:25:03.655793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643845+0000","last_change":"2023-11-08T19:24:58.616793+0000","last_active":"2023-11-08T19:25:03.643845+0000","last_peered":"2023-11-08T19:25:03.643845+0000","last_clean":"2023-11-08T19:25:03.643845+0000","last_became_active":"2023-11-08T19:24:58.616483+0000","last_became_peered":"2023-11-08T19:24:58.616483+0000","last_unstale":"2023-11-08T19:25:03.643845+0000","last_undegraded":"2023-11-08T19:25:03.643845+0000","last_fullsized":"2023-11-08T19:25:03.643845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656535+0000","last_change":"2023-11-08T19:24:58.640055+0000","last_active":"2023-11-08T19:25:03.656535+0000","last_peered":"2023-11-08T19:25:03.656535+0000","last_clean":"2023-11-08T19:25:03.656535+0000","last_became_active":"2023-11-08T19:24:58.639464+0000","last_became_peered":"2023-11-08T19:24:58.639464+0000","last_unstale":"2023-11-08T19:25:03.656535+0000","last_undegraded":"2023-11-08T19:25:03.656535+0000","last_fullsized":"2023-11-08T19:25:03.656535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:51.551899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":697807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":105838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:25.972 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:25:26.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.214+0000 7f29df9f5640 1 Processor -- start 2023-11-08T19:25:26.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.214+0000 7f29df9f5640 1 -- start start 2023-11-08T19:25:26.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29df9f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29d8142f80 0x7f29d80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29df9f5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29d806cb40 0x7f29d806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29df9f5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29d806d4d0 con 0x7f29d815d9f0 2023-11-08T19:25:26.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29df9f5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d8143a40 con 0x7f29d806cb40 2023-11-08T19:25:26.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29df9f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29d80ca3e0 con 0x7f29d8142f80 2023-11-08T19:25:26.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29ddf92640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29d806cb40 0x7f29d806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.216+0000 7f29dcf90640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f29d815d9f0 0x7f29d80c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60072/0 (socket says 172.21.15.119:60072) 2023-11-08T19:25:26.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29dcf90640 1 -- 172.21.15.119:0/2420207426 learned_addr learned my addr 172.21.15.119:0/2420207426 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:26.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29dd791640 1 --2- 172.21.15.119:0/2420207426 >> v2:172.21.15.188:3300/0 conn(0x7f29d8142f80 0x7f29d80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29ddf92640 1 -- 172.21.15.119:0/2420207426 >> v2:172.21.15.188:3300/0 conn(0x7f29d8142f80 msgr2=0x7f29d80cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:26.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29ddf92640 1 --2- 172.21.15.119:0/2420207426 >> v2:172.21.15.188:3300/0 conn(0x7f29d8142f80 0x7f29d80cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29ddf92640 1 -- 172.21.15.119:0/2420207426 >> v1:172.21.15.119:6789/0 conn(0x7f29d815d9f0 legacy=0x7f29d80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29ddf92640 1 -- 172.21.15.119:0/2420207426 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d80c4960 con 0x7f29d806cb40 2023-11-08T19:25:26.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.217+0000 7f29d7fff640 1 -- 172.21.15.119:0/2420207426 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740211079 0 0) 0x7f29d806d4d0 con 0x7f29d815d9f0 2023-11-08T19:25:26.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.218+0000 7f29ddf92640 1 --2- 172.21.15.119:0/2420207426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d806cb40 0x7f29d806cf90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29d80c3b00 tx=0x7f29d002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b8de5bcf3bebe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.218+0000 7f29d7fff640 1 -- 172.21.15.119:0/2420207426 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d002ee50 con 0x7f29d806cb40 2023-11-08T19:25:26.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.218+0000 7f29d7fff640 1 -- 172.21.15.119:0/2420207426 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0036440 con 0x7f29d806cb40 2023-11-08T19:25:26.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.218+0000 7f29d7fff640 1 -- 172.21.15.119:0/2420207426 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d00367d0 con 0x7f29d806cb40 2023-11-08T19:25:26.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 -- 172.21.15.119:0/2420207426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d806cb40 msgr2=0x7f29d806cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 --2- 172.21.15.119:0/2420207426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d806cb40 0x7f29d806cf90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29d80c3b00 tx=0x7f29d002e0a0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 -- 172.21.15.119:0/2420207426 shutdown_connections 2023-11-08T19:25:26.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 --2- 172.21.15.119:0/2420207426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d806cb40 0x7f29d806cf90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 --2- 172.21.15.119:0/2420207426 >> v2:172.21.15.188:3300/0 conn(0x7f29d8142f80 0x7f29d80cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 -- 172.21.15.119:0/2420207426 >> 172.21.15.119:0/2420207426 conn(0x7f29d80cb610 msgr2=0x7f29d80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 -- 172.21.15.119:0/2420207426 shutdown_connections 2023-11-08T19:25:26.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.219+0000 7f29df9f5640 1 -- 172.21.15.119:0/2420207426 wait complete. 2023-11-08T19:25:26.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.221+0000 7f29df9f5640 1 Processor -- start 2023-11-08T19:25:26.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.221+0000 7f29df9f5640 1 -- start start 2023-11-08T19:25:26.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29df9f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29d806cb40 0x7f29d813f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29df9f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d8142f80 0x7f29d80b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29df9f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 0x7f29d81f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29df9f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d80b0600 con 0x7f29d815d9f0 2023-11-08T19:25:26.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29df9f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d81f6000 con 0x7f29d8142f80 2023-11-08T19:25:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29df9f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29d813db10 con 0x7f29d806cb40 2023-11-08T19:25:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 0x7f29d81f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 0x7f29d81f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51280/0 (socket says 172.21.15.119:51280) 2023-11-08T19:25:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 -- 172.21.15.119:0/3462398164 learned_addr learned my addr 172.21.15.119:0/3462398164 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29dd791640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29d806cb40 0x7f29d813f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29dcf90640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d8142f80 0x7f29d80b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 -- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29d806cb40 msgr2=0x7f29d813f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29d806cb40 0x7f29d813f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 -- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d8142f80 msgr2=0x7f29d80b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d8142f80 0x7f29d80b00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.222+0000 7f29ddf92640 1 -- 172.21.15.119:0/3462398164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d000b6a0 con 0x7f29d815d9f0 2023-11-08T19:25:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.223+0000 7f29ddf92640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 0x7f29d81f28f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f29d000bb30 tx=0x7f29d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.223+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0004230 con 0x7f29d815d9f0 2023-11-08T19:25:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.223+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d0004390 con 0x7f29d815d9f0 2023-11-08T19:25:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.223+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d0003850 con 0x7f29d815d9f0 2023-11-08T19:25:26.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.223+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d813ddf0 con 0x7f29d815d9f0 2023-11-08T19:25:26.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.223+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d813e3b0 con 0x7f29d815d9f0 2023-11-08T19:25:26.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.224+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d81f61b0 con 0x7f29d815d9f0 2023-11-08T19:25:26.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.225+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29d003d040 con 0x7f29d815d9f0 2023-11-08T19:25:26.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.226+0000 7f29d5ffb640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29c406aea0 0x7f29c406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.226+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f29d00ac590 con 0x7f29d815d9f0 2023-11-08T19:25:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.226+0000 7f29dd791640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29c406aea0 0x7f29c406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.227+0000 7f29dd791640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29c406aea0 0x7f29c406d360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f29d8143410 tx=0x7f29c800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.230+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29d81f61b0 con 0x7f29d815d9f0 2023-11-08T19:25:26.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.404+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f29d815e240 con 0x7f29c406aea0 2023-11-08T19:25:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.430+0000 7f29d5ffb640 1 -- 172.21.15.119:0/3462398164 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+345123 (secure 0 0 0) 0x7f29c8007e40 con 0x7f29c406aea0 2023-11-08T19:25:26.436 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:26.438 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:25:26.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29c406aea0 msgr2=0x7f29c406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29c406aea0 0x7f29c406d360 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f29d8143410 tx=0x7f29c800b040 comp rx=0 tx=0).stop 2023-11-08T19:25:26.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 msgr2=0x7f29d81f28f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 0x7f29d81f28f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f29d000bb30 tx=0x7f29d0004730 comp rx=0 tx=0).stop 2023-11-08T19:25:26.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 shutdown_connections 2023-11-08T19:25:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29c406aea0 0x7f29c406d360 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29d815d9f0 0x7f29d81f28f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29d8142f80 0x7f29d80b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 --2- 172.21.15.119:0/3462398164 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29d806cb40 0x7f29d813f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 >> 172.21.15.119:0/3462398164 conn(0x7f29d80cb610 msgr2=0x7f29d80caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 shutdown_connections 2023-11-08T19:25:26.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.436+0000 7f29df9f5640 1 -- 172.21.15.119:0/3462398164 wait complete. 2023-11-08T19:25:26.456 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2023-11-08T19:25:24.667940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595007,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3375104,"data_stored":2042832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016656"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655958+0000","last_change":"2023-11-08T19:24:57.661383+0000","last_active":"2023-11-08T19:25:03.655958+0000","last_peered":"2023-11-08T19:25:03.655958+0000","last_clean":"2023-11-08T19:25:03.655958+0000","last_became_active":"2023-11-08T19:24:57.661122+0000","last_became_peered":"2023-11-08T19:24:57.661122+0000","last_unstale":"2023-11-08T19:25:03.655958+0000","last_undegraded":"2023-11-08T19:25:03.655958+0000","last_fullsized":"2023-11-08T19:25:03.655958+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657959+0000","last_change":"2023-11-08T19:24:58.622972+0000","last_active":"2023-11-08T19:25:03.657959+0000","last_peered":"2023-11-08T19:25:03.657959+0000","last_clean":"2023-11-08T19:25:03.657959+0000","last_became_active":"2023-11-08T19:24:58.620907+0000","last_became_peered":"2023-11-08T19:24:58.620907+0000","last_unstale":"2023-11-08T19:25:03.657959+0000","last_undegraded":"2023-11-08T19:25:03.657959+0000","last_fullsized":"2023-11-08T19:25:03.657959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593888+0000","last_change":"2023-11-08T19:25:03.480700+0000","last_active":"2023-11-08T19:25:04.593888+0000","last_peered":"2023-11-08T19:25:04.593888+0000","last_clean":"2023-11-08T19:25:04.593888+0000","last_became_active":"2023-11-08T19:25:03.477779+0000","last_became_peered":"2023-11-08T19:25:03.477779+0000","last_unstale":"2023-11-08T19:25:04.593888+0000","last_undegraded":"2023-11-08T19:25:04.593888+0000","last_fullsized":"2023-11-08T19:25:04.593888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:47:24.018242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658227+0000","last_change":"2023-11-08T19:24:57.655981+0000","last_active":"2023-11-08T19:25:03.658227+0000","last_peered":"2023-11-08T19:25:03.658227+0000","last_clean":"2023-11-08T19:25:03.658227+0000","last_became_active":"2023-11-08T19:24:57.655420+0000","last_became_peered":"2023-11-08T19:24:57.655420+0000","last_unstale":"2023-11-08T19:25:03.658227+0000","last_undegraded":"2023-11-08T19:25:03.658227+0000","last_fullsized":"2023-11-08T19:25:03.658227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655637+0000","last_change":"2023-11-08T19:24:58.620479+0000","last_active":"2023-11-08T19:25:03.655637+0000","last_peered":"2023-11-08T19:25:03.655637+0000","last_clean":"2023-11-08T19:25:03.655637+0000","last_became_active":"2023-11-08T19:24:58.620201+0000","last_became_peered":"2023-11-08T19:24:58.620201+0000","last_unstale":"2023-11-08T19:25:03.655637+0000","last_undegraded":"2023-11-08T19:25:03.655637+0000","last_fullsized":"2023-11-08T19:25:03.655637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.598798+0000","last_change":"2023-11-08T19:25:02.722736+0000","last_active":"2023-11-08T19:25:04.598798+0000","last_peered":"2023-11-08T19:25:04.598798+0000","last_clean":"2023-11-08T19:25:04.598798+0000","last_became_active":"2023-11-08T19:25:02.722087+0000","last_became_peered":"2023-11-08T19:25:02.722087+0000","last_unstale":"2023-11-08T19:25:04.598798+0000","last_undegraded":"2023-11-08T19:25:04.598798+0000","last_fullsized":"2023-11-08T19:25:04.598798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656740+0000","last_change":"2023-11-08T19:24:57.670689+0000","last_active":"2023-11-08T19:25:03.656740+0000","last_peered":"2023-11-08T19:25:03.656740+0000","last_clean":"2023-11-08T19:25:03.656740+0000","last_became_active":"2023-11-08T19:24:57.670406+0000","last_became_peered":"2023-11-08T19:24:57.670406+0000","last_unstale":"2023-11-08T19:25:03.656740+0000","last_undegraded":"2023-11-08T19:25:03.656740+0000","last_fullsized":"2023-11-08T19:25:03.656740+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600045+0000","last_change":"2023-11-08T19:24:58.629107+0000","last_active":"2023-11-08T19:25:03.600045+0000","last_peered":"2023-11-08T19:25:03.600045+0000","last_clean":"2023-11-08T19:25:03.600045+0000","last_became_active":"2023-11-08T19:24:58.628670+0000","last_became_peered":"2023-11-08T19:24:58.628670+0000","last_unstale":"2023-11-08T19:25:03.600045+0000","last_undegraded":"2023-11-08T19:25:03.600045+0000","last_fullsized":"2023-11-08T19:25:03.600045+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601324+0000","last_change":"2023-11-08T19:25:02.727558+0000","last_active":"2023-11-08T19:25:04.601324+0000","last_peered":"2023-11-08T19:25:04.601324+0000","last_clean":"2023-11-08T19:25:04.601324+0000","last_became_active":"2023-11-08T19:25:02.726979+0000","last_became_peered":"2023-11-08T19:25:02.726979+0000","last_unstale":"2023-11-08T19:25:04.601324+0000","last_undegraded":"2023-11-08T19:25:04.601324+0000","last_fullsized":"2023-11-08T19:25:04.601324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655522+0000","last_change":"2023-11-08T19:24:57.673750+0000","last_active":"2023-11-08T19:25:03.655522+0000","last_peered":"2023-11-08T19:25:03.655522+0000","last_clean":"2023-11-08T19:25:03.655522+0000","last_became_active":"2023-11-08T19:24:57.673328+0000","last_became_peered":"2023-11-08T19:24:57.673328+0000","last_unstale":"2023-11-08T19:25:03.655522+0000","last_undegraded":"2023-11-08T19:25:03.655522+0000","last_fullsized":"2023-11-08T19:25:03.655522+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600770+0000","last_change":"2023-11-08T19:24:58.628265+0000","last_active":"2023-11-08T19:25:03.600770+0000","last_peered":"2023-11-08T19:25:03.600770+0000","last_clean":"2023-11-08T19:25:03.600770+0000","last_became_active":"2023-11-08T19:24:58.628007+0000","last_became_peered":"2023-11-08T19:24:58.628007+0000","last_unstale":"2023-11-08T19:25:03.600770+0000","last_undegraded":"2023-11-08T19:25:03.600770+0000","last_fullsized":"2023-11-08T19:25:03.600770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.601053+0000","last_change":"2023-11-08T19:25:02.726316+0000","last_active":"2023-11-08T19:25:04.601053+0000","last_peered":"2023-11-08T19:25:04.601053+0000","last_clean":"2023-11-08T19:25:04.601053+0000","last_became_active":"2023-11-08T19:25:02.725661+0000","last_became_peered":"2023-11-08T19:25:02.725661+0000","last_unstale":"2023-11-08T19:25:04.601053+0000","last_undegraded":"2023-11-08T19:25:04.601053+0000","last_fullsized":"2023-11-08T19:25:04.601053+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:56.793913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596618+0000","last_change":"2023-11-08T19:24:58.626589+0000","last_active":"2023-11-08T19:25:03.596618+0000","last_peered":"2023-11-08T19:25:03.596618+0000","last_clean":"2023-11-08T19:25:03.596618+0000","last_became_active":"2023-11-08T19:24:58.626269+0000","last_became_peered":"2023-11-08T19:24:58.626269+0000","last_unstale":"2023-11-08T19:25:03.596618+0000","last_undegraded":"2023-11-08T19:25:03.596618+0000","last_fullsized":"2023-11-08T19:25:03.596618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:57.668738+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:57.668319+0000","last_became_peered":"2023-11-08T19:24:57.668319+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.603469+0000","last_change":"2023-11-08T19:25:02.731537+0000","last_active":"2023-11-08T19:25:04.603469+0000","last_peered":"2023-11-08T19:25:04.603469+0000","last_clean":"2023-11-08T19:25:04.603469+0000","last_became_active":"2023-11-08T19:25:02.731004+0000","last_became_peered":"2023-11-08T19:25:02.731004+0000","last_unstale":"2023-11-08T19:25:04.603469+0000","last_undegraded":"2023-11-08T19:25:04.603469+0000","last_fullsized":"2023-11-08T19:25:04.603469+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:58.628177+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:58.627584+0000","last_became_peered":"2023-11-08T19:24:58.627584+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643213+0000","last_change":"2023-11-08T19:24:57.659565+0000","last_active":"2023-11-08T19:25:03.643213+0000","last_peered":"2023-11-08T19:25:03.643213+0000","last_clean":"2023-11-08T19:25:03.643213+0000","last_became_active":"2023-11-08T19:24:57.658443+0000","last_became_peered":"2023-11-08T19:24:57.658443+0000","last_unstale":"2023-11-08T19:25:03.643213+0000","last_undegraded":"2023-11-08T19:25:03.643213+0000","last_fullsized":"2023-11-08T19:25:03.643213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602536+0000","last_change":"2023-11-08T19:25:02.734167+0000","last_active":"2023-11-08T19:25:04.602536+0000","last_peered":"2023-11-08T19:25:04.602536+0000","last_clean":"2023-11-08T19:25:04.602536+0000","last_became_active":"2023-11-08T19:25:02.733732+0000","last_became_peered":"2023-11-08T19:25:02.733732+0000","last_unstale":"2023-11-08T19:25:04.602536+0000","last_undegraded":"2023-11-08T19:25:04.602536+0000","last_fullsized":"2023-11-08T19:25:04.602536+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655085+0000","last_change":"2023-11-08T19:24:58.627099+0000","last_active":"2023-11-08T19:25:03.655085+0000","last_peered":"2023-11-08T19:25:03.655085+0000","last_clean":"2023-11-08T19:25:03.655085+0000","last_became_active":"2023-11-08T19:24:58.626261+0000","last_became_peered":"2023-11-08T19:24:58.626261+0000","last_unstale":"2023-11-08T19:25:03.655085+0000","last_undegraded":"2023-11-08T19:25:03.655085+0000","last_fullsized":"2023-11-08T19:25:03.655085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642570+0000","last_change":"2023-11-08T19:24:57.658778+0000","last_active":"2023-11-08T19:25:03.642570+0000","last_peered":"2023-11-08T19:25:03.642570+0000","last_clean":"2023-11-08T19:25:03.642570+0000","last_became_active":"2023-11-08T19:24:57.657370+0000","last_became_peered":"2023-11-08T19:24:57.657370+0000","last_unstale":"2023-11-08T19:25:03.642570+0000","last_undegraded":"2023-11-08T19:25:03.642570+0000","last_fullsized":"2023-11-08T19:25:03.642570+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422591+0000","last_change":"2023-11-08T19:25:03.478005+0000","last_active":"2023-11-08T19:25:05.422591+0000","last_peered":"2023-11-08T19:25:05.422591+0000","last_clean":"2023-11-08T19:25:05.422591+0000","last_became_active":"2023-11-08T19:25:03.476666+0000","last_became_peered":"2023-11-08T19:25:03.476666+0000","last_unstale":"2023-11-08T19:25:05.422591+0000","last_undegraded":"2023-11-08T19:25:05.422591+0000","last_fullsized":"2023-11-08T19:25:05.422591+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656312+0000","last_change":"2023-11-08T19:24:58.623061+0000","last_active":"2023-11-08T19:25:03.656312+0000","last_peered":"2023-11-08T19:25:03.656312+0000","last_clean":"2023-11-08T19:25:03.656312+0000","last_became_active":"2023-11-08T19:24:58.618367+0000","last_became_peered":"2023-11-08T19:24:58.618367+0000","last_unstale":"2023-11-08T19:25:03.656312+0000","last_undegraded":"2023-11-08T19:25:03.656312+0000","last_fullsized":"2023-11-08T19:25:03.656312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:08.538170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.653991+0000","last_change":"2023-11-08T19:24:57.666853+0000","last_active":"2023-11-08T19:25:03.653991+0000","last_peered":"2023-11-08T19:25:03.653991+0000","last_clean":"2023-11-08T19:25:03.653991+0000","last_became_active":"2023-11-08T19:24:57.666539+0000","last_became_peered":"2023-11-08T19:24:57.666539+0000","last_unstale":"2023-11-08T19:25:03.653991+0000","last_undegraded":"2023-11-08T19:25:03.653991+0000","last_fullsized":"2023-11-08T19:25:03.653991+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602888+0000","last_change":"2023-11-08T19:25:02.727013+0000","last_active":"2023-11-08T19:25:04.602888+0000","last_peered":"2023-11-08T19:25:04.602888+0000","last_clean":"2023-11-08T19:25:04.602888+0000","last_became_active":"2023-11-08T19:25:02.726496+0000","last_became_peered":"2023-11-08T19:25:02.726496+0000","last_unstale":"2023-11-08T19:25:04.602888+0000","last_undegraded":"2023-11-08T19:25:04.602888+0000","last_fullsized":"2023-11-08T19:25:04.602888+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655207+0000","last_change":"2023-11-08T19:24:58.620701+0000","last_active":"2023-11-08T19:25:03.655207+0000","last_peered":"2023-11-08T19:25:03.655207+0000","last_clean":"2023-11-08T19:25:03.655207+0000","last_became_active":"2023-11-08T19:24:58.620386+0000","last_became_peered":"2023-11-08T19:24:58.620386+0000","last_unstale":"2023-11-08T19:25:03.655207+0000","last_undegraded":"2023-11-08T19:25:03.655207+0000","last_fullsized":"2023-11-08T19:25:03.655207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597904+0000","last_change":"2023-11-08T19:24:57.667152+0000","last_active":"2023-11-08T19:25:03.597904+0000","last_peered":"2023-11-08T19:25:03.597904+0000","last_clean":"2023-11-08T19:25:03.597904+0000","last_became_active":"2023-11-08T19:24:57.666522+0000","last_became_peered":"2023-11-08T19:24:57.666522+0000","last_unstale":"2023-11-08T19:25:03.597904+0000","last_undegraded":"2023-11-08T19:25:03.597904+0000","last_fullsized":"2023-11-08T19:25:03.597904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:33.630876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599610+0000","last_change":"2023-11-08T19:25:03.469823+0000","last_active":"2023-11-08T19:25:04.599610+0000","last_peered":"2023-11-08T19:25:04.599610+0000","last_clean":"2023-11-08T19:25:04.599610+0000","last_became_active":"2023-11-08T19:25:03.468656+0000","last_became_peered":"2023-11-08T19:25:03.468656+0000","last_unstale":"2023-11-08T19:25:04.599610+0000","last_undegraded":"2023-11-08T19:25:04.599610+0000","last_fullsized":"2023-11-08T19:25:04.599610+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.633451+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.633159+0000","last_became_peered":"2023-11-08T19:24:58.633159+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.644112+0000","last_change":"2023-11-08T19:24:57.663788+0000","last_active":"2023-11-08T19:25:03.644112+0000","last_peered":"2023-11-08T19:25:03.644112+0000","last_clean":"2023-11-08T19:25:03.644112+0000","last_became_active":"2023-11-08T19:24:57.663457+0000","last_became_peered":"2023-11-08T19:24:57.663457+0000","last_unstale":"2023-11-08T19:25:03.644112+0000","last_undegraded":"2023-11-08T19:25:03.644112+0000","last_fullsized":"2023-11-08T19:25:03.644112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.593332+0000","last_change":"2023-11-08T19:25:03.469891+0000","last_active":"2023-11-08T19:25:04.593332+0000","last_peered":"2023-11-08T19:25:04.593332+0000","last_clean":"2023-11-08T19:25:04.593332+0000","last_became_active":"2023-11-08T19:25:03.466854+0000","last_became_peered":"2023-11-08T19:25:03.466854+0000","last_unstale":"2023-11-08T19:25:04.593332+0000","last_undegraded":"2023-11-08T19:25:04.593332+0000","last_fullsized":"2023-11-08T19:25:04.593332+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:05.405858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643596+0000","last_change":"2023-11-08T19:24:58.622842+0000","last_active":"2023-11-08T19:25:03.643596+0000","last_peered":"2023-11-08T19:25:03.643596+0000","last_clean":"2023-11-08T19:25:03.643596+0000","last_became_active":"2023-11-08T19:24:58.622530+0000","last_became_peered":"2023-11-08T19:24:58.622530+0000","last_unstale":"2023-11-08T19:25:03.643596+0000","last_undegraded":"2023-11-08T19:25:03.643596+0000","last_fullsized":"2023-11-08T19:25:03.643596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656961+0000","last_change":"2023-11-08T19:24:57.669407+0000","last_active":"2023-11-08T19:25:03.656961+0000","last_peered":"2023-11-08T19:25:03.656961+0000","last_clean":"2023-11-08T19:25:03.656961+0000","last_became_active":"2023-11-08T19:24:57.669169+0000","last_became_peered":"2023-11-08T19:24:57.669169+0000","last_unstale":"2023-11-08T19:25:03.656961+0000","last_undegraded":"2023-11-08T19:25:03.656961+0000","last_fullsized":"2023-11-08T19:25:03.656961+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:50.920808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530152+0000","last_change":"2023-11-08T19:25:03.469685+0000","last_active":"2023-11-08T19:25:05.530152+0000","last_peered":"2023-11-08T19:25:05.530152+0000","last_clean":"2023-11-08T19:25:05.530152+0000","last_became_active":"2023-11-08T19:25:03.468600+0000","last_became_peered":"2023-11-08T19:25:03.468600+0000","last_unstale":"2023-11-08T19:25:05.530152+0000","last_undegraded":"2023-11-08T19:25:05.530152+0000","last_fullsized":"2023-11-08T19:25:05.530152+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600914+0000","last_change":"2023-11-08T19:24:58.617568+0000","last_active":"2023-11-08T19:25:03.600914+0000","last_peered":"2023-11-08T19:25:03.600914+0000","last_clean":"2023-11-08T19:25:03.600914+0000","last_became_active":"2023-11-08T19:24:58.617216+0000","last_became_peered":"2023-11-08T19:24:58.617216+0000","last_unstale":"2023-11-08T19:25:03.600914+0000","last_undegraded":"2023-11-08T19:25:03.600914+0000","last_fullsized":"2023-11-08T19:25:03.600914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643888+0000","last_change":"2023-11-08T19:24:57.653655+0000","last_active":"2023-11-08T19:25:03.643888+0000","last_peered":"2023-11-08T19:25:03.643888+0000","last_clean":"2023-11-08T19:25:03.643888+0000","last_became_active":"2023-11-08T19:24:57.653390+0000","last_became_peered":"2023-11-08T19:24:57.653390+0000","last_unstale":"2023-11-08T19:25:03.643888+0000","last_undegraded":"2023-11-08T19:25:03.643888+0000","last_fullsized":"2023-11-08T19:25:03.643888+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421356+0000","last_change":"2023-11-08T19:25:02.724803+0000","last_active":"2023-11-08T19:25:05.421356+0000","last_peered":"2023-11-08T19:25:05.421356+0000","last_clean":"2023-11-08T19:25:05.421356+0000","last_became_active":"2023-11-08T19:25:02.724278+0000","last_became_peered":"2023-11-08T19:25:02.724278+0000","last_unstale":"2023-11-08T19:25:05.421356+0000","last_undegraded":"2023-11-08T19:25:05.421356+0000","last_fullsized":"2023-11-08T19:25:05.421356+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:04.735010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655769+0000","last_change":"2023-11-08T19:24:58.617441+0000","last_active":"2023-11-08T19:25:03.655769+0000","last_peered":"2023-11-08T19:25:03.655769+0000","last_clean":"2023-11-08T19:25:03.655769+0000","last_became_active":"2023-11-08T19:24:58.617030+0000","last_became_peered":"2023-11-08T19:24:58.617030+0000","last_unstale":"2023-11-08T19:25:03.655769+0000","last_undegraded":"2023-11-08T19:25:03.655769+0000","last_fullsized":"2023-11-08T19:25:03.655769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:50.043019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599773+0000","last_change":"2023-11-08T19:24:57.662494+0000","last_active":"2023-11-08T19:25:03.599773+0000","last_peered":"2023-11-08T19:25:03.599773+0000","last_clean":"2023-11-08T19:25:03.599773+0000","last_became_active":"2023-11-08T19:24:57.662223+0000","last_became_peered":"2023-11-08T19:24:57.662223+0000","last_unstale":"2023-11-08T19:25:03.599773+0000","last_undegraded":"2023-11-08T19:25:03.599773+0000","last_fullsized":"2023-11-08T19:25:03.599773+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:49.940163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421797+0000","last_change":"2023-11-08T19:25:02.729279+0000","last_active":"2023-11-08T19:25:05.421797+0000","last_peered":"2023-11-08T19:25:05.421797+0000","last_clean":"2023-11-08T19:25:05.421797+0000","last_became_active":"2023-11-08T19:25:02.728855+0000","last_became_peered":"2023-11-08T19:25:02.728855+0000","last_unstale":"2023-11-08T19:25:05.421797+0000","last_undegraded":"2023-11-08T19:25:05.421797+0000","last_fullsized":"2023-11-08T19:25:05.421797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:36.061036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642895+0000","last_change":"2023-11-08T19:24:58.617415+0000","last_active":"2023-11-08T19:25:03.642895+0000","last_peered":"2023-11-08T19:25:03.642895+0000","last_clean":"2023-11-08T19:25:03.642895+0000","last_became_active":"2023-11-08T19:24:58.616880+0000","last_became_peered":"2023-11-08T19:24:58.616880+0000","last_unstale":"2023-11-08T19:25:03.642895+0000","last_undegraded":"2023-11-08T19:25:03.642895+0000","last_fullsized":"2023-11-08T19:25:03.642895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592969+0000","last_change":"2023-11-08T19:24:57.650498+0000","last_active":"2023-11-08T19:25:03.592969+0000","last_peered":"2023-11-08T19:25:03.592969+0000","last_clean":"2023-11-08T19:25:03.592969+0000","last_became_active":"2023-11-08T19:24:57.650172+0000","last_became_peered":"2023-11-08T19:24:57.650172+0000","last_unstale":"2023-11-08T19:25:03.592969+0000","last_undegraded":"2023-11-08T19:25:03.592969+0000","last_fullsized":"2023-11-08T19:25:03.592969+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.422691+0000","last_change":"2023-11-08T19:25:02.722430+0000","last_active":"2023-11-08T19:25:05.422691+0000","last_peered":"2023-11-08T19:25:05.422691+0000","last_clean":"2023-11-08T19:25:05.422691+0000","last_became_active":"2023-11-08T19:25:02.722013+0000","last_became_peered":"2023-11-08T19:25:02.722013+0000","last_unstale":"2023-11-08T19:25:05.422691+0000","last_undegraded":"2023-11-08T19:25:05.422691+0000","last_fullsized":"2023-11-08T19:25:05.422691+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654808+0000","last_change":"2023-11-08T19:24:58.629901+0000","last_active":"2023-11-08T19:25:03.654808+0000","last_peered":"2023-11-08T19:25:03.654808+0000","last_clean":"2023-11-08T19:25:03.654808+0000","last_became_active":"2023-11-08T19:24:58.629603+0000","last_became_peered":"2023-11-08T19:24:58.629603+0000","last_unstale":"2023-11-08T19:25:03.654808+0000","last_undegraded":"2023-11-08T19:25:03.654808+0000","last_fullsized":"2023-11-08T19:25:03.654808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:19.337071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654594+0000","last_change":"2023-11-08T19:24:57.667258+0000","last_active":"2023-11-08T19:25:03.654594+0000","last_peered":"2023-11-08T19:25:03.654594+0000","last_clean":"2023-11-08T19:25:03.654594+0000","last_became_active":"2023-11-08T19:24:57.666859+0000","last_became_peered":"2023-11-08T19:24:57.666859+0000","last_unstale":"2023-11-08T19:25:03.654594+0000","last_undegraded":"2023-11-08T19:25:03.654594+0000","last_fullsized":"2023-11-08T19:25:03.654594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:49.029259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530805+0000","last_change":"2023-11-08T19:25:03.480894+0000","last_active":"2023-11-08T19:25:05.530805+0000","last_peered":"2023-11-08T19:25:05.530805+0000","last_clean":"2023-11-08T19:25:05.530805+0000","last_became_active":"2023-11-08T19:25:03.479219+0000","last_became_peered":"2023-11-08T19:25:03.479219+0000","last_unstale":"2023-11-08T19:25:05.530805+0000","last_undegraded":"2023-11-08T19:25:05.530805+0000","last_fullsized":"2023-11-08T19:25:05.530805+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656729+0000","last_change":"2023-11-08T19:24:58.620051+0000","last_active":"2023-11-08T19:25:03.656729+0000","last_peered":"2023-11-08T19:25:03.656729+0000","last_clean":"2023-11-08T19:25:03.656729+0000","last_became_active":"2023-11-08T19:24:58.619641+0000","last_became_peered":"2023-11-08T19:24:58.619641+0000","last_unstale":"2023-11-08T19:25:03.656729+0000","last_undegraded":"2023-11-08T19:25:03.656729+0000","last_fullsized":"2023-11-08T19:25:03.656729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599436+0000","last_change":"2023-11-08T19:24:57.656028+0000","last_active":"2023-11-08T19:25:03.599436+0000","last_peered":"2023-11-08T19:25:03.599436+0000","last_clean":"2023-11-08T19:25:03.599436+0000","last_became_active":"2023-11-08T19:24:57.651726+0000","last_became_peered":"2023-11-08T19:24:57.651726+0000","last_unstale":"2023-11-08T19:25:03.599436+0000","last_undegraded":"2023-11-08T19:25:03.599436+0000","last_fullsized":"2023-11-08T19:25:03.599436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":170,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654244+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:25:03.654244+0000","last_peered":"2023-11-08T19:25:03.654244+0000","last_clean":"2023-11-08T19:25:03.654244+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:25:03.654244+0000","last_undegraded":"2023-11-08T19:25:03.654244+0000","last_fullsized":"2023-11-08T19:25:03.654244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349456+0000","last_change":"2023-11-08T19:25:03.468384+0000","last_active":"2023-11-08T19:25:06.349456+0000","last_peered":"2023-11-08T19:25:06.349456+0000","last_clean":"2023-11-08T19:25:06.349456+0000","last_became_active":"2023-11-08T19:25:03.467219+0000","last_became_peered":"2023-11-08T19:25:03.467219+0000","last_unstale":"2023-11-08T19:25:06.349456+0000","last_undegraded":"2023-11-08T19:25:06.349456+0000","last_fullsized":"2023-11-08T19:25:06.349456+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656350+0000","last_change":"2023-11-08T19:24:58.632161+0000","last_active":"2023-11-08T19:25:03.656350+0000","last_peered":"2023-11-08T19:25:03.656350+0000","last_clean":"2023-11-08T19:25:03.656350+0000","last_became_active":"2023-11-08T19:24:58.631360+0000","last_became_peered":"2023-11-08T19:24:58.631360+0000","last_unstale":"2023-11-08T19:25:03.656350+0000","last_undegraded":"2023-11-08T19:25:03.656350+0000","last_fullsized":"2023-11-08T19:25:03.656350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594245+0000","last_change":"2023-11-08T19:24:57.659462+0000","last_active":"2023-11-08T19:25:03.594245+0000","last_peered":"2023-11-08T19:25:03.594245+0000","last_clean":"2023-11-08T19:25:03.594245+0000","last_became_active":"2023-11-08T19:24:57.659200+0000","last_became_peered":"2023-11-08T19:24:57.659200+0000","last_unstale":"2023-11-08T19:25:03.594245+0000","last_undegraded":"2023-11-08T19:25:03.594245+0000","last_fullsized":"2023-11-08T19:25:03.594245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:21.040122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.348403+0000","last_change":"2023-11-08T19:25:03.468319+0000","last_active":"2023-11-08T19:25:06.348403+0000","last_peered":"2023-11-08T19:25:06.348403+0000","last_clean":"2023-11-08T19:25:06.348403+0000","last_became_active":"2023-11-08T19:25:03.466962+0000","last_became_peered":"2023-11-08T19:25:03.466962+0000","last_unstale":"2023-11-08T19:25:06.348403+0000","last_undegraded":"2023-11-08T19:25:06.348403+0000","last_fullsized":"2023-11-08T19:25:06.348403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:32.352851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600157+0000","last_change":"2023-11-08T19:24:58.620109+0000","last_active":"2023-11-08T19:25:03.600157+0000","last_peered":"2023-11-08T19:25:03.600157+0000","last_clean":"2023-11-08T19:25:03.600157+0000","last_became_active":"2023-11-08T19:24:58.619504+0000","last_became_peered":"2023-11-08T19:24:58.619504+0000","last_unstale":"2023-11-08T19:25:03.600157+0000","last_undegraded":"2023-11-08T19:25:03.600157+0000","last_fullsized":"2023-11-08T19:25:03.600157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643946+0000","last_change":"2023-11-08T19:24:57.658265+0000","last_active":"2023-11-08T19:25:03.643946+0000","last_peered":"2023-11-08T19:25:03.643946+0000","last_clean":"2023-11-08T19:25:03.643946+0000","last_became_active":"2023-11-08T19:24:57.657921+0000","last_became_peered":"2023-11-08T19:24:57.657921+0000","last_unstale":"2023-11-08T19:25:03.643946+0000","last_undegraded":"2023-11-08T19:25:03.643946+0000","last_fullsized":"2023-11-08T19:25:03.643946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:36.677668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530244+0000","last_change":"2023-11-08T19:25:02.727032+0000","last_active":"2023-11-08T19:25:05.530244+0000","last_peered":"2023-11-08T19:25:05.530244+0000","last_clean":"2023-11-08T19:25:05.530244+0000","last_became_active":"2023-11-08T19:25:02.726636+0000","last_became_peered":"2023-11-08T19:25:02.726636+0000","last_unstale":"2023-11-08T19:25:05.530244+0000","last_undegraded":"2023-11-08T19:25:05.530244+0000","last_fullsized":"2023-11-08T19:25:05.530244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:57.165210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656744+0000","last_change":"2023-11-08T19:24:58.641465+0000","last_active":"2023-11-08T19:25:03.656744+0000","last_peered":"2023-11-08T19:25:03.656744+0000","last_clean":"2023-11-08T19:25:03.656744+0000","last_became_active":"2023-11-08T19:24:58.641144+0000","last_became_peered":"2023-11-08T19:24:58.641144+0000","last_unstale":"2023-11-08T19:25:03.656744+0000","last_undegraded":"2023-11-08T19:25:03.656744+0000","last_fullsized":"2023-11-08T19:25:03.656744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:00.683350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.594008+0000","last_change":"2023-11-08T19:24:57.653180+0000","last_active":"2023-11-08T19:25:03.594008+0000","last_peered":"2023-11-08T19:25:03.594008+0000","last_clean":"2023-11-08T19:25:03.594008+0000","last_became_active":"2023-11-08T19:24:57.652617+0000","last_became_peered":"2023-11-08T19:24:57.652617+0000","last_unstale":"2023-11-08T19:25:03.594008+0000","last_undegraded":"2023-11-08T19:25:03.594008+0000","last_fullsized":"2023-11-08T19:25:03.594008+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:35.866163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599737+0000","last_change":"2023-11-08T19:25:03.474070+0000","last_active":"2023-11-08T19:25:04.599737+0000","last_peered":"2023-11-08T19:25:04.599737+0000","last_clean":"2023-11-08T19:25:04.599737+0000","last_became_active":"2023-11-08T19:25:03.471994+0000","last_became_peered":"2023-11-08T19:25:03.471994+0000","last_unstale":"2023-11-08T19:25:04.599737+0000","last_undegraded":"2023-11-08T19:25:04.599737+0000","last_fullsized":"2023-11-08T19:25:04.599737+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655993+0000","last_change":"2023-11-08T19:24:58.640557+0000","last_active":"2023-11-08T19:25:03.655993+0000","last_peered":"2023-11-08T19:25:03.655993+0000","last_clean":"2023-11-08T19:25:03.655993+0000","last_became_active":"2023-11-08T19:24:58.640005+0000","last_became_peered":"2023-11-08T19:24:58.640005+0000","last_unstale":"2023-11-08T19:25:03.655993+0000","last_undegraded":"2023-11-08T19:25:03.655993+0000","last_fullsized":"2023-11-08T19:25:03.655993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657695+0000","last_change":"2023-11-08T19:24:57.652195+0000","last_active":"2023-11-08T19:25:03.657695+0000","last_peered":"2023-11-08T19:25:03.657695+0000","last_clean":"2023-11-08T19:25:03.657695+0000","last_became_active":"2023-11-08T19:24:57.651790+0000","last_became_peered":"2023-11-08T19:24:57.651790+0000","last_unstale":"2023-11-08T19:25:03.657695+0000","last_undegraded":"2023-11-08T19:25:03.657695+0000","last_fullsized":"2023-11-08T19:25:03.657695+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:50:21.039412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.599054+0000","last_change":"2023-11-08T19:25:02.722727+0000","last_active":"2023-11-08T19:25:04.599054+0000","last_peered":"2023-11-08T19:25:04.599054+0000","last_clean":"2023-11-08T19:25:04.599054+0000","last_became_active":"2023-11-08T19:25:02.722092+0000","last_became_peered":"2023-11-08T19:25:02.722092+0000","last_unstale":"2023-11-08T19:25:04.599054+0000","last_undegraded":"2023-11-08T19:25:04.599054+0000","last_fullsized":"2023-11-08T19:25:04.599054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:40.294618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655407+0000","last_change":"2023-11-08T19:24:58.635248+0000","last_active":"2023-11-08T19:25:03.655407+0000","last_peered":"2023-11-08T19:25:03.655407+0000","last_clean":"2023-11-08T19:25:03.655407+0000","last_became_active":"2023-11-08T19:24:58.634504+0000","last_became_peered":"2023-11-08T19:24:58.634504+0000","last_unstale":"2023-11-08T19:25:03.655407+0000","last_undegraded":"2023-11-08T19:25:03.655407+0000","last_fullsized":"2023-11-08T19:25:03.655407+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.592589+0000","last_change":"2023-11-08T19:24:57.651955+0000","last_active":"2023-11-08T19:25:03.592589+0000","last_peered":"2023-11-08T19:25:03.592589+0000","last_clean":"2023-11-08T19:25:03.592589+0000","last_became_active":"2023-11-08T19:24:57.651666+0000","last_became_peered":"2023-11-08T19:24:57.651666+0000","last_unstale":"2023-11-08T19:25:03.592589+0000","last_undegraded":"2023-11-08T19:25:03.592589+0000","last_fullsized":"2023-11-08T19:25:03.592589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:42.265977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530618+0000","last_change":"2023-11-08T19:25:02.728613+0000","last_active":"2023-11-08T19:25:05.530618+0000","last_peered":"2023-11-08T19:25:05.530618+0000","last_clean":"2023-11-08T19:25:05.530618+0000","last_became_active":"2023-11-08T19:25:02.727749+0000","last_became_peered":"2023-11-08T19:25:02.727749+0000","last_unstale":"2023-11-08T19:25:05.530618+0000","last_undegraded":"2023-11-08T19:25:05.530618+0000","last_fullsized":"2023-11-08T19:25:05.530618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:33:46.986406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595934+0000","last_change":"2023-11-08T19:24:58.616453+0000","last_active":"2023-11-08T19:25:03.595934+0000","last_peered":"2023-11-08T19:25:03.595934+0000","last_clean":"2023-11-08T19:25:03.595934+0000","last_became_active":"2023-11-08T19:24:58.615934+0000","last_became_peered":"2023-11-08T19:24:58.615934+0000","last_unstale":"2023-11-08T19:25:03.595934+0000","last_undegraded":"2023-11-08T19:25:03.595934+0000","last_fullsized":"2023-11-08T19:25:03.595934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596954+0000","last_change":"2023-11-08T19:24:57.659585+0000","last_active":"2023-11-08T19:25:03.596954+0000","last_peered":"2023-11-08T19:25:03.596954+0000","last_clean":"2023-11-08T19:25:03.596954+0000","last_became_active":"2023-11-08T19:24:57.659197+0000","last_became_peered":"2023-11-08T19:24:57.659197+0000","last_unstale":"2023-11-08T19:25:03.596954+0000","last_undegraded":"2023-11-08T19:25:03.596954+0000","last_fullsized":"2023-11-08T19:25:03.596954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:27.508037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595745+0000","last_change":"2023-11-08T19:25:03.468147+0000","last_active":"2023-11-08T19:25:04.595745+0000","last_peered":"2023-11-08T19:25:04.595745+0000","last_clean":"2023-11-08T19:25:04.595745+0000","last_became_active":"2023-11-08T19:25:03.466921+0000","last_became_peered":"2023-11-08T19:25:03.466921+0000","last_unstale":"2023-11-08T19:25:04.595745+0000","last_undegraded":"2023-11-08T19:25:04.595745+0000","last_fullsized":"2023-11-08T19:25:04.595745+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:14:51.027517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656098+0000","last_change":"2023-11-08T19:24:58.636188+0000","last_active":"2023-11-08T19:25:03.656098+0000","last_peered":"2023-11-08T19:25:03.656098+0000","last_clean":"2023-11-08T19:25:03.656098+0000","last_became_active":"2023-11-08T19:24:58.635889+0000","last_became_peered":"2023-11-08T19:24:58.635889+0000","last_unstale":"2023-11-08T19:25:03.656098+0000","last_undegraded":"2023-11-08T19:25:03.656098+0000","last_fullsized":"2023-11-08T19:25:03.656098+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:54:33.382812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599561+0000","last_change":"2023-11-08T19:24:57.655323+0000","last_active":"2023-11-08T19:25:03.599561+0000","last_peered":"2023-11-08T19:25:03.599561+0000","last_clean":"2023-11-08T19:25:03.599561+0000","last_became_active":"2023-11-08T19:24:57.654973+0000","last_became_peered":"2023-11-08T19:24:57.654973+0000","last_unstale":"2023-11-08T19:25:03.599561+0000","last_undegraded":"2023-11-08T19:25:03.599561+0000","last_fullsized":"2023-11-08T19:25:03.599561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:11.031981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594252+0000","last_change":"2023-11-08T19:25:03.467735+0000","last_active":"2023-11-08T19:25:04.594252+0000","last_peered":"2023-11-08T19:25:04.594252+0000","last_clean":"2023-11-08T19:25:04.594252+0000","last_became_active":"2023-11-08T19:25:03.466595+0000","last_became_peered":"2023-11-08T19:25:03.466595+0000","last_unstale":"2023-11-08T19:25:04.594252+0000","last_undegraded":"2023-11-08T19:25:04.594252+0000","last_fullsized":"2023-11-08T19:25:04.594252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:17.064402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655761+0000","last_change":"2023-11-08T19:24:58.626959+0000","last_active":"2023-11-08T19:25:03.655761+0000","last_peered":"2023-11-08T19:25:03.655761+0000","last_clean":"2023-11-08T19:25:03.655761+0000","last_became_active":"2023-11-08T19:24:58.626616+0000","last_became_peered":"2023-11-08T19:24:58.626616+0000","last_unstale":"2023-11-08T19:25:03.655761+0000","last_undegraded":"2023-11-08T19:25:03.655761+0000","last_fullsized":"2023-11-08T19:25:03.655761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657001+0000","last_change":"2023-11-08T19:24:57.663161+0000","last_active":"2023-11-08T19:25:03.657001+0000","last_peered":"2023-11-08T19:25:03.657001+0000","last_clean":"2023-11-08T19:25:03.657001+0000","last_became_active":"2023-11-08T19:24:57.660362+0000","last_became_peered":"2023-11-08T19:24:57.660362+0000","last_unstale":"2023-11-08T19:25:03.657001+0000","last_undegraded":"2023-11-08T19:25:03.657001+0000","last_fullsized":"2023-11-08T19:25:03.657001+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:03:07.524411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602355+0000","last_change":"2023-11-08T19:25:03.477682+0000","last_active":"2023-11-08T19:25:04.602355+0000","last_peered":"2023-11-08T19:25:04.602355+0000","last_clean":"2023-11-08T19:25:04.602355+0000","last_became_active":"2023-11-08T19:25:03.476098+0000","last_became_peered":"2023-11-08T19:25:03.476098+0000","last_unstale":"2023-11-08T19:25:04.602355+0000","last_undegraded":"2023-11-08T19:25:04.602355+0000","last_fullsized":"2023-11-08T19:25:04.602355+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:28.074207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656221+0000","last_change":"2023-11-08T19:24:58.634601+0000","last_active":"2023-11-08T19:25:03.656221+0000","last_peered":"2023-11-08T19:25:03.656221+0000","last_clean":"2023-11-08T19:25:03.656221+0000","last_became_active":"2023-11-08T19:24:58.632876+0000","last_became_peered":"2023-11-08T19:24:58.632876+0000","last_unstale":"2023-11-08T19:25:03.656221+0000","last_undegraded":"2023-11-08T19:25:03.656221+0000","last_fullsized":"2023-11-08T19:25:03.656221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:21.785769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'10","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:17.407997+0000","last_change":"2023-11-08T19:24:57.660289+0000","last_active":"2023-11-08T19:25:17.407997+0000","last_peered":"2023-11-08T19:25:17.407997+0000","last_clean":"2023-11-08T19:25:17.407997+0000","last_became_active":"2023-11-08T19:24:57.659982+0000","last_became_peered":"2023-11-08T19:24:57.659982+0000","last_unstale":"2023-11-08T19:25:17.407997+0000","last_undegraded":"2023-11-08T19:25:17.407997+0000","last_fullsized":"2023-11-08T19:25:17.407997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3841,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.421411+0000","last_change":"2023-11-08T19:25:02.718892+0000","last_active":"2023-11-08T19:25:05.421411+0000","last_peered":"2023-11-08T19:25:05.421411+0000","last_clean":"2023-11-08T19:25:05.421411+0000","last_became_active":"2023-11-08T19:25:02.717998+0000","last_became_peered":"2023-11-08T19:25:02.717998+0000","last_unstale":"2023-11-08T19:25:05.421411+0000","last_undegraded":"2023-11-08T19:25:05.421411+0000","last_fullsized":"2023-11-08T19:25:05.421411+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656401+0000","last_change":"2023-11-08T19:24:58.629542+0000","last_active":"2023-11-08T19:25:03.656401+0000","last_peered":"2023-11-08T19:25:03.656401+0000","last_clean":"2023-11-08T19:25:03.656401+0000","last_became_active":"2023-11-08T19:24:58.627201+0000","last_became_peered":"2023-11-08T19:24:58.627201+0000","last_unstale":"2023-11-08T19:25:03.656401+0000","last_undegraded":"2023-11-08T19:25:03.656401+0000","last_fullsized":"2023-11-08T19:25:03.656401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643852+0000","last_change":"2023-11-08T19:24:57.661152+0000","last_active":"2023-11-08T19:25:03.643852+0000","last_peered":"2023-11-08T19:25:03.643852+0000","last_clean":"2023-11-08T19:25:03.643852+0000","last_became_active":"2023-11-08T19:24:57.660740+0000","last_became_peered":"2023-11-08T19:24:57.660740+0000","last_unstale":"2023-11-08T19:25:03.643852+0000","last_undegraded":"2023-11-08T19:25:03.643852+0000","last_fullsized":"2023-11-08T19:25:03.643852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.602299+0000","last_change":"2023-11-08T19:25:02.727543+0000","last_active":"2023-11-08T19:25:04.602299+0000","last_peered":"2023-11-08T19:25:04.602299+0000","last_clean":"2023-11-08T19:25:04.602299+0000","last_became_active":"2023-11-08T19:25:02.726809+0000","last_became_peered":"2023-11-08T19:25:02.726809+0000","last_unstale":"2023-11-08T19:25:04.602299+0000","last_undegraded":"2023-11-08T19:25:04.602299+0000","last_fullsized":"2023-11-08T19:25:04.602299+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:55:40.382140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655948+0000","last_change":"2023-11-08T19:24:58.618188+0000","last_active":"2023-11-08T19:25:03.655948+0000","last_peered":"2023-11-08T19:25:03.655948+0000","last_clean":"2023-11-08T19:25:03.655948+0000","last_became_active":"2023-11-08T19:24:58.617893+0000","last_became_peered":"2023-11-08T19:24:58.617893+0000","last_unstale":"2023-11-08T19:25:03.655948+0000","last_undegraded":"2023-11-08T19:25:03.655948+0000","last_fullsized":"2023-11-08T19:25:03.655948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600932+0000","last_change":"2023-11-08T19:24:57.662272+0000","last_active":"2023-11-08T19:25:03.600932+0000","last_peered":"2023-11-08T19:25:03.600932+0000","last_clean":"2023-11-08T19:25:03.600932+0000","last_became_active":"2023-11-08T19:24:57.662012+0000","last_became_peered":"2023-11-08T19:24:57.662012+0000","last_unstale":"2023-11-08T19:25:03.600932+0000","last_undegraded":"2023-11-08T19:25:03.600932+0000","last_fullsized":"2023-11-08T19:25:03.600932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:35.675163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594552+0000","last_change":"2023-11-08T19:25:02.729257+0000","last_active":"2023-11-08T19:25:04.594552+0000","last_peered":"2023-11-08T19:25:04.594552+0000","last_clean":"2023-11-08T19:25:04.594552+0000","last_became_active":"2023-11-08T19:25:02.727346+0000","last_became_peered":"2023-11-08T19:25:02.727346+0000","last_unstale":"2023-11-08T19:25:04.594552+0000","last_undegraded":"2023-11-08T19:25:04.594552+0000","last_fullsized":"2023-11-08T19:25:04.594552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:45.742494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.600772+0000","last_change":"2023-11-08T19:24:58.626572+0000","last_active":"2023-11-08T19:25:03.600772+0000","last_peered":"2023-11-08T19:25:03.600772+0000","last_clean":"2023-11-08T19:25:03.600772+0000","last_became_active":"2023-11-08T19:24:58.626191+0000","last_became_peered":"2023-11-08T19:24:58.626191+0000","last_unstale":"2023-11-08T19:25:03.600772+0000","last_undegraded":"2023-11-08T19:25:03.600772+0000","last_fullsized":"2023-11-08T19:25:03.600772+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:40.864211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597705+0000","last_change":"2023-11-08T19:24:57.672262+0000","last_active":"2023-11-08T19:25:03.597705+0000","last_peered":"2023-11-08T19:25:03.597705+0000","last_clean":"2023-11-08T19:25:03.597705+0000","last_became_active":"2023-11-08T19:24:57.671965+0000","last_became_peered":"2023-11-08T19:24:57.671965+0000","last_unstale":"2023-11-08T19:25:03.597705+0000","last_undegraded":"2023-11-08T19:25:03.597705+0000","last_fullsized":"2023-11-08T19:25:03.597705+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.594345+0000","last_change":"2023-11-08T19:25:03.467509+0000","last_active":"2023-11-08T19:25:04.594345+0000","last_peered":"2023-11-08T19:25:04.594345+0000","last_clean":"2023-11-08T19:25:04.594345+0000","last_became_active":"2023-11-08T19:25:03.466319+0000","last_became_peered":"2023-11-08T19:25:03.466319+0000","last_unstale":"2023-11-08T19:25:04.594345+0000","last_undegraded":"2023-11-08T19:25:04.594345+0000","last_fullsized":"2023-11-08T19:25:04.594345+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:44.481960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.530577+0000","last_change":"2023-11-08T19:25:03.475886+0000","last_active":"2023-11-08T19:25:05.530577+0000","last_peered":"2023-11-08T19:25:05.530577+0000","last_clean":"2023-11-08T19:25:05.530577+0000","last_became_active":"2023-11-08T19:25:03.473511+0000","last_became_peered":"2023-11-08T19:25:03.473511+0000","last_unstale":"2023-11-08T19:25:05.530577+0000","last_undegraded":"2023-11-08T19:25:05.530577+0000","last_fullsized":"2023-11-08T19:25:05.530577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:53:26.049057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642306+0000","last_change":"2023-11-08T19:24:58.621533+0000","last_active":"2023-11-08T19:25:03.642306+0000","last_peered":"2023-11-08T19:25:03.642306+0000","last_clean":"2023-11-08T19:25:03.642306+0000","last_became_active":"2023-11-08T19:24:58.621214+0000","last_became_peered":"2023-11-08T19:24:58.621214+0000","last_unstale":"2023-11-08T19:25:03.642306+0000","last_undegraded":"2023-11-08T19:25:03.642306+0000","last_fullsized":"2023-11-08T19:25:03.642306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596137+0000","last_change":"2023-11-08T19:24:57.656298+0000","last_active":"2023-11-08T19:25:03.596137+0000","last_peered":"2023-11-08T19:25:03.596137+0000","last_clean":"2023-11-08T19:25:03.596137+0000","last_became_active":"2023-11-08T19:24:57.655961+0000","last_became_peered":"2023-11-08T19:24:57.655961+0000","last_unstale":"2023-11-08T19:25:03.596137+0000","last_undegraded":"2023-11-08T19:25:03.596137+0000","last_fullsized":"2023-11-08T19:25:03.596137+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:35.955840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:06.349973+0000","last_change":"2023-11-08T19:25:02.721600+0000","last_active":"2023-11-08T19:25:06.349973+0000","last_peered":"2023-11-08T19:25:06.349973+0000","last_clean":"2023-11-08T19:25:06.349973+0000","last_became_active":"2023-11-08T19:25:02.721169+0000","last_became_peered":"2023-11-08T19:25:02.721169+0000","last_unstale":"2023-11-08T19:25:06.349973+0000","last_undegraded":"2023-11-08T19:25:06.349973+0000","last_fullsized":"2023-11-08T19:25:06.349973+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:55.603560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654845+0000","last_change":"2023-11-08T19:24:58.624276+0000","last_active":"2023-11-08T19:25:03.654845+0000","last_peered":"2023-11-08T19:25:03.654845+0000","last_clean":"2023-11-08T19:25:03.654845+0000","last_became_active":"2023-11-08T19:24:58.623432+0000","last_became_peered":"2023-11-08T19:24:58.623432+0000","last_unstale":"2023-11-08T19:25:03.654845+0000","last_undegraded":"2023-11-08T19:25:03.654845+0000","last_fullsized":"2023-11-08T19:25:03.654845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656023+0000","last_change":"2023-11-08T19:24:57.661677+0000","last_active":"2023-11-08T19:25:03.656023+0000","last_peered":"2023-11-08T19:25:03.656023+0000","last_clean":"2023-11-08T19:25:03.656023+0000","last_became_active":"2023-11-08T19:24:57.661425+0000","last_became_peered":"2023-11-08T19:24:57.661425+0000","last_unstale":"2023-11-08T19:25:03.656023+0000","last_undegraded":"2023-11-08T19:25:03.656023+0000","last_fullsized":"2023-11-08T19:25:03.656023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596849+0000","last_change":"2023-11-08T19:24:57.662729+0000","last_active":"2023-11-08T19:25:03.596849+0000","last_peered":"2023-11-08T19:25:03.596849+0000","last_clean":"2023-11-08T19:25:03.596849+0000","last_became_active":"2023-11-08T19:24:57.662390+0000","last_became_peered":"2023-11-08T19:24:57.662390+0000","last_unstale":"2023-11-08T19:25:03.596849+0000","last_undegraded":"2023-11-08T19:25:03.596849+0000","last_fullsized":"2023-11-08T19:25:03.596849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:46:45.589255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656750+0000","last_change":"2023-11-08T19:24:58.619164+0000","last_active":"2023-11-08T19:25:03.656750+0000","last_peered":"2023-11-08T19:25:03.656750+0000","last_clean":"2023-11-08T19:25:03.656750+0000","last_became_active":"2023-11-08T19:24:58.618604+0000","last_became_peered":"2023-11-08T19:24:58.618604+0000","last_unstale":"2023-11-08T19:25:03.656750+0000","last_undegraded":"2023-11-08T19:25:03.656750+0000","last_fullsized":"2023-11-08T19:25:03.656750+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:05.514753+0000","last_change":"2023-11-08T19:25:02.720151+0000","last_active":"2023-11-08T19:25:05.514753+0000","last_peered":"2023-11-08T19:25:05.514753+0000","last_clean":"2023-11-08T19:25:05.514753+0000","last_became_active":"2023-11-08T19:25:02.719391+0000","last_became_peered":"2023-11-08T19:25:02.719391+0000","last_unstale":"2023-11-08T19:25:05.514753+0000","last_undegraded":"2023-11-08T19:25:05.514753+0000","last_fullsized":"2023-11-08T19:25:05.514753+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:23.388732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657232+0000","last_change":"2023-11-08T19:24:57.660203+0000","last_active":"2023-11-08T19:25:03.657232+0000","last_peered":"2023-11-08T19:25:03.657232+0000","last_clean":"2023-11-08T19:25:03.657232+0000","last_became_active":"2023-11-08T19:24:57.659913+0000","last_became_peered":"2023-11-08T19:24:57.659913+0000","last_unstale":"2023-11-08T19:25:03.657232+0000","last_undegraded":"2023-11-08T19:25:03.657232+0000","last_fullsized":"2023-11-08T19:25:03.657232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:56.599040+0000","last_clean_scrub_stamp":"2023-11-08T19:24:56.599040+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:18.323312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657083+0000","last_change":"2023-11-08T19:24:58.620564+0000","last_active":"2023-11-08T19:25:03.657083+0000","last_peered":"2023-11-08T19:25:03.657083+0000","last_clean":"2023-11-08T19:25:03.657083+0000","last_became_active":"2023-11-08T19:24:58.620072+0000","last_became_peered":"2023-11-08T19:24:58.620072+0000","last_unstale":"2023-11-08T19:25:03.657083+0000","last_undegraded":"2023-11-08T19:25:03.657083+0000","last_fullsized":"2023-11-08T19:25:03.657083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:36.879923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-08T19:25:04.595054+0000","last_change":"2023-11-08T19:25:02.729532+0000","last_active":"2023-11-08T19:25:04.595054+0000","last_peered":"2023-11-08T19:25:04.595054+0000","last_clean":"2023-11-08T19:25:04.595054+0000","last_became_active":"2023-11-08T19:25:02.728431+0000","last_became_peered":"2023-11-08T19:25:02.728431+0000","last_unstale":"2023-11-08T19:25:04.595054+0000","last_undegraded":"2023-11-08T19:25:04.595054+0000","last_fullsized":"2023-11-08T19:25:04.595054+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:25:01.664460+0000","last_clean_scrub_stamp":"2023-11-08T19:25:01.664460+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:57.143010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595597+0000","last_change":"2023-11-08T19:24:58.624056+0000","last_active":"2023-11-08T19:25:03.595597+0000","last_peered":"2023-11-08T19:25:03.595597+0000","last_clean":"2023-11-08T19:25:03.595597+0000","last_became_active":"2023-11-08T19:24:58.623783+0000","last_became_peered":"2023-11-08T19:24:58.623783+0000","last_unstale":"2023-11-08T19:25:03.595597+0000","last_undegraded":"2023-11-08T19:25:03.595597+0000","last_fullsized":"2023-11-08T19:25:03.595597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593245+0000","last_change":"2023-11-08T19:24:58.617678+0000","last_active":"2023-11-08T19:25:03.593245+0000","last_peered":"2023-11-08T19:25:03.593245+0000","last_clean":"2023-11-08T19:25:03.593245+0000","last_became_active":"2023-11-08T19:24:58.616701+0000","last_became_peered":"2023-11-08T19:24:58.616701+0000","last_unstale":"2023-11-08T19:25:03.593245+0000","last_undegraded":"2023-11-08T19:25:03.593245+0000","last_fullsized":"2023-11-08T19:25:03.593245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:51:28.521797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599324+0000","last_change":"2023-11-08T19:24:58.620961+0000","last_active":"2023-11-08T19:25:03.599324+0000","last_peered":"2023-11-08T19:25:03.599324+0000","last_clean":"2023-11-08T19:25:03.599324+0000","last_became_active":"2023-11-08T19:24:58.620359+0000","last_became_peered":"2023-11-08T19:24:58.620359+0000","last_unstale":"2023-11-08T19:25:03.599324+0000","last_undegraded":"2023-11-08T19:25:03.599324+0000","last_fullsized":"2023-11-08T19:25:03.599324+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656545+0000","last_change":"2023-11-08T19:24:58.622475+0000","last_active":"2023-11-08T19:25:03.656545+0000","last_peered":"2023-11-08T19:25:03.656545+0000","last_clean":"2023-11-08T19:25:03.656545+0000","last_became_active":"2023-11-08T19:24:58.622206+0000","last_became_peered":"2023-11-08T19:24:58.622206+0000","last_unstale":"2023-11-08T19:25:03.656545+0000","last_undegraded":"2023-11-08T19:25:03.656545+0000","last_fullsized":"2023-11-08T19:25:03.656545+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:08:23.000868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.657638+0000","last_change":"2023-11-08T19:24:58.630303+0000","last_active":"2023-11-08T19:25:03.657638+0000","last_peered":"2023-11-08T19:25:03.657638+0000","last_clean":"2023-11-08T19:25:03.657638+0000","last_became_active":"2023-11-08T19:24:58.629181+0000","last_became_peered":"2023-11-08T19:24:58.629181+0000","last_unstale":"2023-11-08T19:25:03.657638+0000","last_undegraded":"2023-11-08T19:25:03.657638+0000","last_fullsized":"2023-11-08T19:25:03.657638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:38:47.164726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643449+0000","last_change":"2023-11-08T19:24:58.622627+0000","last_active":"2023-11-08T19:25:03.643449+0000","last_peered":"2023-11-08T19:25:03.643449+0000","last_clean":"2023-11-08T19:25:03.643449+0000","last_became_active":"2023-11-08T19:24:58.622335+0000","last_became_peered":"2023-11-08T19:24:58.622335+0000","last_unstale":"2023-11-08T19:25:03.643449+0000","last_undegraded":"2023-11-08T19:25:03.643449+0000","last_fullsized":"2023-11-08T19:25:03.643449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597541+0000","last_change":"2023-11-08T19:24:58.630440+0000","last_active":"2023-11-08T19:25:03.597541+0000","last_peered":"2023-11-08T19:25:03.597541+0000","last_clean":"2023-11-08T19:25:03.597541+0000","last_became_active":"2023-11-08T19:24:58.629904+0000","last_became_peered":"2023-11-08T19:24:58.629904+0000","last_unstale":"2023-11-08T19:25:03.597541+0000","last_undegraded":"2023-11-08T19:25:03.597541+0000","last_fullsized":"2023-11-08T19:25:03.597541+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.658178+0000","last_change":"2023-11-08T19:24:58.627568+0000","last_active":"2023-11-08T19:25:03.658178+0000","last_peered":"2023-11-08T19:25:03.658178+0000","last_clean":"2023-11-08T19:25:03.658178+0000","last_became_active":"2023-11-08T19:24:58.627243+0000","last_became_peered":"2023-11-08T19:24:58.627243+0000","last_unstale":"2023-11-08T19:25:03.658178+0000","last_undegraded":"2023-11-08T19:25:03.658178+0000","last_fullsized":"2023-11-08T19:25:03.658178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595899+0000","last_change":"2023-11-08T19:24:58.622232+0000","last_active":"2023-11-08T19:25:03.595899+0000","last_peered":"2023-11-08T19:25:03.595899+0000","last_clean":"2023-11-08T19:25:03.595899+0000","last_became_active":"2023-11-08T19:24:58.621853+0000","last_became_peered":"2023-11-08T19:24:58.621853+0000","last_unstale":"2023-11-08T19:25:03.595899+0000","last_undegraded":"2023-11-08T19:25:03.595899+0000","last_fullsized":"2023-11-08T19:25:03.595899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:56.690358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654133+0000","last_change":"2023-11-08T19:24:58.630530+0000","last_active":"2023-11-08T19:25:03.654133+0000","last_peered":"2023-11-08T19:25:03.654133+0000","last_clean":"2023-11-08T19:25:03.654133+0000","last_became_active":"2023-11-08T19:24:58.627752+0000","last_became_peered":"2023-11-08T19:24:58.627752+0000","last_unstale":"2023-11-08T19:25:03.654133+0000","last_undegraded":"2023-11-08T19:25:03.654133+0000","last_fullsized":"2023-11-08T19:25:03.654133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599081+0000","last_change":"2023-11-08T19:24:58.626422+0000","last_active":"2023-11-08T19:25:03.599081+0000","last_peered":"2023-11-08T19:25:03.599081+0000","last_clean":"2023-11-08T19:25:03.599081+0000","last_became_active":"2023-11-08T19:24:58.626091+0000","last_became_peered":"2023-11-08T19:24:58.626091+0000","last_unstale":"2023-11-08T19:25:03.599081+0000","last_undegraded":"2023-11-08T19:25:03.599081+0000","last_fullsized":"2023-11-08T19:25:03.599081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:12:54.382416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656191+0000","last_change":"2023-11-08T19:24:58.630415+0000","last_active":"2023-11-08T19:25:03.656191+0000","last_peered":"2023-11-08T19:25:03.656191+0000","last_clean":"2023-11-08T19:25:03.656191+0000","last_became_active":"2023-11-08T19:24:58.628526+0000","last_became_peered":"2023-11-08T19:24:58.628526+0000","last_unstale":"2023-11-08T19:25:03.656191+0000","last_undegraded":"2023-11-08T19:25:03.656191+0000","last_fullsized":"2023-11-08T19:25:03.656191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596558+0000","last_change":"2023-11-08T19:24:58.628286+0000","last_active":"2023-11-08T19:25:03.596558+0000","last_peered":"2023-11-08T19:25:03.596558+0000","last_clean":"2023-11-08T19:25:03.596558+0000","last_became_active":"2023-11-08T19:24:58.627912+0000","last_became_peered":"2023-11-08T19:24:58.627912+0000","last_unstale":"2023-11-08T19:25:03.596558+0000","last_undegraded":"2023-11-08T19:25:03.596558+0000","last_fullsized":"2023-11-08T19:25:03.596558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656975+0000","last_change":"2023-11-08T19:24:58.633630+0000","last_active":"2023-11-08T19:25:03.656975+0000","last_peered":"2023-11-08T19:25:03.656975+0000","last_clean":"2023-11-08T19:25:03.656975+0000","last_became_active":"2023-11-08T19:24:58.632441+0000","last_became_peered":"2023-11-08T19:24:58.632441+0000","last_unstale":"2023-11-08T19:25:03.656975+0000","last_undegraded":"2023-11-08T19:25:03.656975+0000","last_fullsized":"2023-11-08T19:25:03.656975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:07:28.301190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.593821+0000","last_change":"2023-11-08T19:24:58.623578+0000","last_active":"2023-11-08T19:25:03.593821+0000","last_peered":"2023-11-08T19:25:03.593821+0000","last_clean":"2023-11-08T19:25:03.593821+0000","last_became_active":"2023-11-08T19:24:58.623261+0000","last_became_peered":"2023-11-08T19:24:58.623261+0000","last_unstale":"2023-11-08T19:25:03.593821+0000","last_undegraded":"2023-11-08T19:25:03.593821+0000","last_fullsized":"2023-11-08T19:25:03.593821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:43:36.969600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.601266+0000","last_change":"2023-11-08T19:24:58.634259+0000","last_active":"2023-11-08T19:25:03.601266+0000","last_peered":"2023-11-08T19:25:03.601266+0000","last_clean":"2023-11-08T19:25:03.601266+0000","last_became_active":"2023-11-08T19:24:58.633708+0000","last_became_peered":"2023-11-08T19:24:58.633708+0000","last_unstale":"2023-11-08T19:25:03.601266+0000","last_undegraded":"2023-11-08T19:25:03.601266+0000","last_fullsized":"2023-11-08T19:25:03.601266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:36.242957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.595680+0000","last_change":"2023-11-08T19:24:58.615871+0000","last_active":"2023-11-08T19:25:03.595680+0000","last_peered":"2023-11-08T19:25:03.595680+0000","last_clean":"2023-11-08T19:25:03.595680+0000","last_became_active":"2023-11-08T19:24:58.615590+0000","last_became_peered":"2023-11-08T19:24:58.615590+0000","last_unstale":"2023-11-08T19:25:03.595680+0000","last_undegraded":"2023-11-08T19:25:03.595680+0000","last_fullsized":"2023-11-08T19:25:03.595680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.642579+0000","last_change":"2023-11-08T19:24:58.629058+0000","last_active":"2023-11-08T19:25:03.642579+0000","last_peered":"2023-11-08T19:25:03.642579+0000","last_clean":"2023-11-08T19:25:03.642579+0000","last_became_active":"2023-11-08T19:24:58.628656+0000","last_became_peered":"2023-11-08T19:24:58.628656+0000","last_unstale":"2023-11-08T19:25:03.642579+0000","last_undegraded":"2023-11-08T19:25:03.642579+0000","last_fullsized":"2023-11-08T19:25:03.642579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655552+0000","last_change":"2023-11-08T19:24:58.622286+0000","last_active":"2023-11-08T19:25:03.655552+0000","last_peered":"2023-11-08T19:25:03.655552+0000","last_clean":"2023-11-08T19:25:03.655552+0000","last_became_active":"2023-11-08T19:24:58.621993+0000","last_became_peered":"2023-11-08T19:24:58.621993+0000","last_unstale":"2023-11-08T19:25:03.655552+0000","last_undegraded":"2023-11-08T19:25:03.655552+0000","last_fullsized":"2023-11-08T19:25:03.655552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:03:04.158336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656958+0000","last_change":"2023-11-08T19:24:58.618762+0000","last_active":"2023-11-08T19:25:03.656958+0000","last_peered":"2023-11-08T19:25:03.656958+0000","last_clean":"2023-11-08T19:25:03.656958+0000","last_became_active":"2023-11-08T19:24:58.618468+0000","last_became_peered":"2023-11-08T19:24:58.618468+0000","last_unstale":"2023-11-08T19:25:03.656958+0000","last_undegraded":"2023-11-08T19:25:03.656958+0000","last_fullsized":"2023-11-08T19:25:03.656958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655770+0000","last_change":"2023-11-08T19:24:58.626193+0000","last_active":"2023-11-08T19:25:03.655770+0000","last_peered":"2023-11-08T19:25:03.655770+0000","last_clean":"2023-11-08T19:25:03.655770+0000","last_became_active":"2023-11-08T19:24:58.625897+0000","last_became_peered":"2023-11-08T19:24:58.625897+0000","last_unstale":"2023-11-08T19:25:03.655770+0000","last_undegraded":"2023-11-08T19:25:03.655770+0000","last_fullsized":"2023-11-08T19:25:03.655770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:43.279737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.596855+0000","last_change":"2023-11-08T19:24:58.621575+0000","last_active":"2023-11-08T19:25:03.596855+0000","last_peered":"2023-11-08T19:25:03.596855+0000","last_clean":"2023-11-08T19:25:03.596855+0000","last_became_active":"2023-11-08T19:24:58.621278+0000","last_became_peered":"2023-11-08T19:24:58.621278+0000","last_unstale":"2023-11-08T19:25:03.596855+0000","last_undegraded":"2023-11-08T19:25:03.596855+0000","last_fullsized":"2023-11-08T19:25:03.596855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:52:12.045741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.597421+0000","last_change":"2023-11-08T19:24:58.630221+0000","last_active":"2023-11-08T19:25:03.597421+0000","last_peered":"2023-11-08T19:25:03.597421+0000","last_clean":"2023-11-08T19:25:03.597421+0000","last_became_active":"2023-11-08T19:24:58.629514+0000","last_became_peered":"2023-11-08T19:24:58.629514+0000","last_unstale":"2023-11-08T19:25:03.597421+0000","last_undegraded":"2023-11-08T19:25:03.597421+0000","last_fullsized":"2023-11-08T19:25:03.597421+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656658+0000","last_change":"2023-11-08T19:24:58.637307+0000","last_active":"2023-11-08T19:25:03.656658+0000","last_peered":"2023-11-08T19:25:03.656658+0000","last_clean":"2023-11-08T19:25:03.656658+0000","last_became_active":"2023-11-08T19:24:58.636926+0000","last_became_peered":"2023-11-08T19:24:58.636926+0000","last_unstale":"2023-11-08T19:25:03.656658+0000","last_undegraded":"2023-11-08T19:25:03.656658+0000","last_fullsized":"2023-11-08T19:25:03.656658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:52.742252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654271+0000","last_change":"2023-11-08T19:24:58.619975+0000","last_active":"2023-11-08T19:25:03.654271+0000","last_peered":"2023-11-08T19:25:03.654271+0000","last_clean":"2023-11-08T19:25:03.654271+0000","last_became_active":"2023-11-08T19:24:58.619676+0000","last_became_peered":"2023-11-08T19:24:58.619676+0000","last_unstale":"2023-11-08T19:25:03.654271+0000","last_undegraded":"2023-11-08T19:25:03.654271+0000","last_fullsized":"2023-11-08T19:25:03.654271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:43:37.691428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.654365+0000","last_change":"2023-11-08T19:24:58.627535+0000","last_active":"2023-11-08T19:25:03.654365+0000","last_peered":"2023-11-08T19:25:03.654365+0000","last_clean":"2023-11-08T19:25:03.654365+0000","last_became_active":"2023-11-08T19:24:58.627205+0000","last_became_peered":"2023-11-08T19:24:58.627205+0000","last_unstale":"2023-11-08T19:25:03.654365+0000","last_undegraded":"2023-11-08T19:25:03.654365+0000","last_fullsized":"2023-11-08T19:25:03.654365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599068+0000","last_change":"2023-11-08T19:24:58.620637+0000","last_active":"2023-11-08T19:25:03.599068+0000","last_peered":"2023-11-08T19:25:03.599068+0000","last_clean":"2023-11-08T19:25:03.599068+0000","last_became_active":"2023-11-08T19:24:58.619919+0000","last_became_peered":"2023-11-08T19:24:58.619919+0000","last_unstale":"2023-11-08T19:25:03.599068+0000","last_undegraded":"2023-11-08T19:25:03.599068+0000","last_fullsized":"2023-11-08T19:25:03.599068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:49:51.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655858+0000","last_change":"2023-11-08T19:24:58.630887+0000","last_active":"2023-11-08T19:25:03.655858+0000","last_peered":"2023-11-08T19:25:03.655858+0000","last_clean":"2023-11-08T19:25:03.655858+0000","last_became_active":"2023-11-08T19:24:58.630010+0000","last_became_peered":"2023-11-08T19:24:58.630010+0000","last_unstale":"2023-11-08T19:25:03.655858+0000","last_undegraded":"2023-11-08T19:25:03.655858+0000","last_fullsized":"2023-11-08T19:25:03.655858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:32.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.599939+0000","last_change":"2023-11-08T19:24:58.625672+0000","last_active":"2023-11-08T19:25:03.599939+0000","last_peered":"2023-11-08T19:25:03.599939+0000","last_clean":"2023-11-08T19:25:03.599939+0000","last_became_active":"2023-11-08T19:24:58.619366+0000","last_became_peered":"2023-11-08T19:24:58.619366+0000","last_unstale":"2023-11-08T19:25:03.599939+0000","last_undegraded":"2023-11-08T19:25:03.599939+0000","last_fullsized":"2023-11-08T19:25:03.599939+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:51:41.748757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.655793+0000","last_change":"2023-11-08T19:24:58.612582+0000","last_active":"2023-11-08T19:25:03.655793+0000","last_peered":"2023-11-08T19:25:03.655793+0000","last_clean":"2023-11-08T19:25:03.655793+0000","last_became_active":"2023-11-08T19:24:58.612199+0000","last_became_peered":"2023-11-08T19:24:58.612199+0000","last_unstale":"2023-11-08T19:25:03.655793+0000","last_undegraded":"2023-11-08T19:25:03.655793+0000","last_fullsized":"2023-11-08T19:25:03.655793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:37:46.763389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.643845+0000","last_change":"2023-11-08T19:24:58.616793+0000","last_active":"2023-11-08T19:25:03.643845+0000","last_peered":"2023-11-08T19:25:03.643845+0000","last_clean":"2023-11-08T19:25:03.643845+0000","last_became_active":"2023-11-08T19:24:58.616483+0000","last_became_peered":"2023-11-08T19:24:58.616483+0000","last_unstale":"2023-11-08T19:25:03.643845+0000","last_undegraded":"2023-11-08T19:25:03.643845+0000","last_fullsized":"2023-11-08T19:25:03.643845+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:50.953923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-08T19:25:03.656535+0000","last_change":"2023-11-08T19:24:58.640055+0000","last_active":"2023-11-08T19:25:03.656535+0000","last_peered":"2023-11-08T19:25:03.656535+0000","last_clean":"2023-11-08T19:25:03.656535+0000","last_became_active":"2023-11-08T19:24:58.639464+0000","last_became_peered":"2023-11-08T19:24:58.639464+0000","last_unstale":"2023-11-08T19:25:03.656535+0000","last_undegraded":"2023-11-08T19:25:03.656535+0000","last_fullsized":"2023-11-08T19:25:03.656535+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:57.612800+0000","last_clean_scrub_stamp":"2023-11-08T19:24:57.612800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:51.551899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4639,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":697807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.034}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":106947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.202}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.425}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":278528,"data_stored":106983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5580000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.101}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":110698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.593}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6639999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":262144,"data_stored":105838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75600000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.071}]}]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":274432,"data_stored":107017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86699999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27876,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343964,"statfs":{"total":96636764160,"available":96608219136,"internally_reserved":0,"allocated":868352,"data_stored":696844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2589999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.048}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4450000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.117}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:25:26.463 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T19:25:26.464 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-08T19:25:26.464 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-08T19:25:26.464 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-08T19:25:26.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.710+0000 7f35fae6f640 1 Processor -- start 2023-11-08T19:25:26.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.710+0000 7f35fae6f640 1 -- start start 2023-11-08T19:25:26.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.712+0000 7f35fae6f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f35f415f4c0 0x7f35f406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.712+0000 7f35fae6f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f40c3a50 con 0x7f35f41447a0 2023-11-08T19:25:26.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.712+0000 7f35fae6f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35f4144b80 con 0x7f35f406a120 2023-11-08T19:25:26.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.712+0000 7f35fae6f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35f415f930 con 0x7f35f415f4c0 2023-11-08T19:25:26.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.712+0000 7f35f8c0b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f35f41447a0 0x7f35f40cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60086/0 (socket says 172.21.15.119:60086) 2023-11-08T19:25:26.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.712+0000 7f35f8c0b640 1 -- 172.21.15.119:0/1114116786 learned_addr learned my addr 172.21.15.119:0/1114116786 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:26.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f3fff640 1 --2- 172.21.15.119:0/1114116786 >> v2:172.21.15.188:3300/0 conn(0x7f35f415f4c0 0x7f35f406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145232031 0 0) 0x7f35f40c3a50 con 0x7f35f41447a0 2023-11-08T19:25:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e0003520 con 0x7f35f41447a0 2023-11-08T19:25:26.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4156429395 0 0) 0x7f35e0003520 con 0x7f35f41447a0 2023-11-08T19:25:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 >> v2:172.21.15.188:3300/0 conn(0x7f35f415f4c0 msgr2=0x7f35f406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 --2- 172.21.15.119:0/1114116786 >> v2:172.21.15.188:3300/0 conn(0x7f35f415f4c0 0x7f35f406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 >> v1:172.21.15.188:6789/0 conn(0x7f35f406a120 legacy=0x7f35f406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f40c9150 con 0x7f35f41447a0 2023-11-08T19:25:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f35e4001a10 con 0x7f35f41447a0 2023-11-08T19:25:26.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 >> v1:172.21.15.119:6789/0 conn(0x7f35f41447a0 legacy=0x7f35f40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e0003970 0x7f35e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e00047c0 0x7f35e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0005180 0x7f35e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f415f930 con 0x7f35e0003970 2023-11-08T19:25:26.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e000c020 con 0x7f35e0005180 2023-11-08T19:25:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.713+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e000c1e0 con 0x7f35e00047c0 2023-11-08T19:25:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e0003970 0x7f35e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f3fff640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e00047c0 0x7f35e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f8c0b640 1 -- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e00047c0 msgr2=0x7f35e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:26.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e00047c0 0x7f35e0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f940c640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0005180 0x7f35e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f8c0b640 1 -- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0005180 msgr2=0x7f35e000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0005180 0x7f35e000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f8c0b640 1 -- 172.21.15.119:0/1114116786 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e0003520 con 0x7f35e0003970 2023-11-08T19:25:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f940c640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0005180 0x7f35e000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.714+0000 7f35f940c640 1 -- 172.21.15.119:0/1114116786 reap_dead start 2023-11-08T19:25:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.715+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e0003970 0x7f35e0003d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35f4071ea0 tx=0x7f35e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eafcbcdcc1b73069 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.715+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4034630 con 0x7f35e0003970 2023-11-08T19:25:26.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.715+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35e402f070 con 0x7f35e0003970 2023-11-08T19:25:26.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.715+0000 7f35f37fe640 1 -- 172.21.15.119:0/1114116786 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35e4034a30 con 0x7f35e0003970 2023-11-08T19:25:26.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 -- 172.21.15.119:0/1114116786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e0003970 msgr2=0x7f35e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e0003970 0x7f35e0003d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f35f4071ea0 tx=0x7f35e00057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 -- 172.21.15.119:0/1114116786 shutdown_connections 2023-11-08T19:25:26.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 --2- 172.21.15.119:0/1114116786 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e0003970 0x7f35e0003d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 -- 172.21.15.119:0/1114116786 >> 172.21.15.119:0/1114116786 conn(0x7f35f40cb610 msgr2=0x7f35f40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 -- 172.21.15.119:0/1114116786 shutdown_connections 2023-11-08T19:25:26.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.716+0000 7f35fae6f640 1 -- 172.21.15.119:0/1114116786 wait complete. 2023-11-08T19:25:26.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.718+0000 7f35fae6f640 1 Processor -- start 2023-11-08T19:25:26.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.718+0000 7f35fae6f640 1 -- start start 2023-11-08T19:25:26.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35fae6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0003970 0x7f35f40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35fae6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 0x7f35f4159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35fae6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35f40b0b10 0x7f35f40b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35fae6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40b14d0 con 0x7f35f415a730 2023-11-08T19:25:26.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35fae6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f413d810 con 0x7f35e0003970 2023-11-08T19:25:26.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35fae6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35f40cd930 con 0x7f35f40b0b10 2023-11-08T19:25:26.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 0x7f35f4159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 0x7f35f4159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51292/0 (socket says 172.21.15.119:51292) 2023-11-08T19:25:26.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.719+0000 7f35f3fff640 1 -- 172.21.15.119:0/3301142458 learned_addr learned my addr 172.21.15.119:0/3301142458 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:26.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0003970 0x7f35f40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f3fff640 1 -- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35f40b0b10 msgr2=0x7f35f40b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f3fff640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35f40b0b10 0x7f35f40b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f3fff640 1 -- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0003970 msgr2=0x7f35f40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f3fff640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0003970 0x7f35f40cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f3fff640 1 -- 172.21.15.119:0/3301142458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f40cdc10 con 0x7f35f415a730 2023-11-08T19:25:26.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f3fff640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 0x7f35f4159f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0108e0 tx=0x7f35dc010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc00eb60 con 0x7f35f415a730 2023-11-08T19:25:26.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc00ecc0 con 0x7f35f415a730 2023-11-08T19:25:26.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f40cdef0 con 0x7f35f415a730 2023-11-08T19:25:26.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.720+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc01c400 con 0x7f35f415a730 2023-11-08T19:25:26.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.721+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f40cd620 con 0x7f35f415a730 2023-11-08T19:25:26.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.722+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f4074430 con 0x7f35f415a730 2023-11-08T19:25:26.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.723+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35dc01c560 con 0x7f35f415a730 2023-11-08T19:25:26.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.724+0000 7f35f17fa640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35d406aea0 0x7f35d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.724+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f35dc089ea0 con 0x7f35f415a730 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.724+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35d406aea0 0x7f35d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:26.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.725+0000 7f35f8c0b640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35d406aea0 0x7f35d406d360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35e4034000 tx=0x7f35e402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:26.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.729+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35dc054d00 con 0x7f35f415a730 2023-11-08T19:25:26.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.959+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f35f406caa0 con 0x7f35f415a730 2023-11-08T19:25:26.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.960+0000 7f35f17fa640 1 -- 172.21.15.119:0/3301142458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f35dc058320 con 0x7f35f415a730 2023-11-08T19:25:26.962 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:26.962 INFO:teuthology.orchestra.run.smithi119.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-08T19:25:26.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.962+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35d406aea0 msgr2=0x7f35d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.962+0000 7f35fae6f640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35d406aea0 0x7f35d406d360 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f35e4034000 tx=0x7f35e402f000 comp rx=0 tx=0).stop 2023-11-08T19:25:26.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.962+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 msgr2=0x7f35f4159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:26.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.962+0000 7f35fae6f640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 0x7f35f4159f10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0108e0 tx=0x7f35dc010da0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 shutdown_connections 2023-11-08T19:25:26.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35d406aea0 0x7f35d406d360 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35f40b0b10 0x7f35f40b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35f415a730 0x7f35f4159f10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 --2- 172.21.15.119:0/3301142458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e0003970 0x7f35f40cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:26.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 >> 172.21.15.119:0/3301142458 conn(0x7f35f40cb610 msgr2=0x7f35f40c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:26.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 shutdown_connections 2023-11-08T19:25:26.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:26.963+0000 7f35fae6f640 1 -- 172.21.15.119:0/3301142458 wait complete. 2023-11-08T19:25:26.982 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-08T19:25:26.982 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:27.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.178+0000 7fb81e5b6640 1 Processor -- start 2023-11-08T19:25:27.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.178+0000 7fb81e5b6640 1 -- start start 2023-11-08T19:25:27.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.179+0000 7fb81e5b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.179+0000 7fb81e5b6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 0x7fb81806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb81e5b6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8180c4960 con 0x7fb818142f80 2023-11-08T19:25:27.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb81e5b6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb818143a40 con 0x7fb81806cb40 2023-11-08T19:25:27.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb81e5b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180ca3e0 con 0x7fb81815d9f0 2023-11-08T19:25:27.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb81cb53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 0x7fb81806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb8177fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb81cb53640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 0x7fb81806cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56992/0 (socket says 172.21.15.119:56992) 2023-11-08T19:25:27.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.180+0000 7fb81cb53640 1 -- 172.21.15.119:0/2413913169 learned_addr learned my addr 172.21.15.119:0/2413913169 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:27.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.181+0000 7fb8177fe640 1 -- 172.21.15.119:0/2413913169 >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 msgr2=0x7fb81806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:27.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.181+0000 7fb8177fe640 1 --2- 172.21.15.119:0/2413913169 >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 0x7fb81806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.181+0000 7fb8177fe640 1 -- 172.21.15.119:0/2413913169 >> v1:172.21.15.188:6789/0 conn(0x7fb818142f80 legacy=0x7fb8180c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.181+0000 7fb8177fe640 1 -- 172.21.15.119:0/2413913169 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81815df60 con 0x7fb81815d9f0 2023-11-08T19:25:27.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.181+0000 7fb81cb53640 1 --2- 172.21.15.119:0/2413913169 >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 0x7fb81806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:27.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.182+0000 7fb8177fe640 1 --2- 172.21.15.119:0/2413913169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180cc6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0097a0 tx=0x7fb80c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d81e3219ec784c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.182+0000 7fb816ffd640 1 -- 172.21.15.119:0/2413913169 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c031780 con 0x7fb81815d9f0 2023-11-08T19:25:27.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb816ffd640 1 -- 172.21.15.119:0/2413913169 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c0318e0 con 0x7fb81815d9f0 2023-11-08T19:25:27.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb816ffd640 1 -- 172.21.15.119:0/2413913169 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c031c70 con 0x7fb81815d9f0 2023-11-08T19:25:27.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 -- 172.21.15.119:0/2413913169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 msgr2=0x7fb8180cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/2413913169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180cc6d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0097a0 tx=0x7fb80c02b820 comp rx=0 tx=0).stop 2023-11-08T19:25:27.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 -- 172.21.15.119:0/2413913169 shutdown_connections 2023-11-08T19:25:27.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/2413913169 >> v2:172.21.15.188:3300/0 conn(0x7fb81806cb40 0x7fb81806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/2413913169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180cc6d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 -- 172.21.15.119:0/2413913169 >> 172.21.15.119:0/2413913169 conn(0x7fb8180cb610 msgr2=0x7fb8180cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 -- 172.21.15.119:0/2413913169 shutdown_connections 2023-11-08T19:25:27.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.183+0000 7fb81e5b6640 1 -- 172.21.15.119:0/2413913169 wait complete. 2023-11-08T19:25:27.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.185+0000 7fb81e5b6640 1 Processor -- start 2023-11-08T19:25:27.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.185+0000 7fb81e5b6640 1 -- start start 2023-11-08T19:25:27.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb81e5b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb81806cb40 0x7fb8180b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb81e5b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 0x7fb818159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb81e5b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb81e5b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180b3780 con 0x7fb818142f80 2023-11-08T19:25:27.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb81e5b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180b3a60 con 0x7fb81806cb40 2023-11-08T19:25:27.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb81e5b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8180b3d40 con 0x7fb81815d9f0 2023-11-08T19:25:27.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.186+0000 7fb8177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 0x7fb818159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 0x7fb818159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51298/0 (socket says 172.21.15.119:51298) 2023-11-08T19:25:27.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 -- 172.21.15.119:0/3308295534 learned_addr learned my addr 172.21.15.119:0/3308295534 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:27.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb81cb53640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb817fff640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb81806cb40 0x7fb8180b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 -- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 msgr2=0x7fb8180b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 -- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb81806cb40 msgr2=0x7fb8180b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb81806cb40 0x7fb8180b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 -- 172.21.15.119:0/3308295534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810009b30 con 0x7fb818142f80 2023-11-08T19:25:27.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb8177fe640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 0x7fb818159780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb80c034040 tx=0x7fb80c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.187+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c032820 con 0x7fb818142f80 2023-11-08T19:25:27.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.188+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb80c032980 con 0x7fb818142f80 2023-11-08T19:25:27.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.188+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb80c032d30 con 0x7fb818142f80 2023-11-08T19:25:27.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.188+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80c009400 con 0x7fb818142f80 2023-11-08T19:25:27.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.188+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8180b4300 con 0x7fb818142f80 2023-11-08T19:25:27.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.189+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81806d1e0 con 0x7fb818142f80 2023-11-08T19:25:27.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.190+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb80c04e440 con 0x7fb818142f80 2023-11-08T19:25:27.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.190+0000 7fb814ff9640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb80406aea0 0x7fb80406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.191+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fb80c0aba00 con 0x7fb818142f80 2023-11-08T19:25:27.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.191+0000 7fb817fff640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb80406aea0 0x7fb80406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.191+0000 7fb817fff640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb80406aea0 0x7fb80406d360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb808000f60 tx=0x7fb808009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.195+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb80c076720 con 0x7fb818142f80 2023-11-08T19:25:27.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.385+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb818074430 con 0x7fb818142f80 2023-11-08T19:25:27.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.387+0000 7fb814ff9640 1 -- 172.21.15.119:0/3308295534 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fb80c079d40 con 0x7fb818142f80 2023-11-08T19:25:27.389 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:27.389 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:27.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb80406aea0 msgr2=0x7fb80406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb80406aea0 0x7fb80406d360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb808000f60 tx=0x7fb808009040 comp rx=0 tx=0).stop 2023-11-08T19:25:27.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 msgr2=0x7fb818159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 0x7fb818159780 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb80c034040 tx=0x7fb80c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 shutdown_connections 2023-11-08T19:25:27.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb80406aea0 0x7fb80406d360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb81815d9f0 0x7fb8180b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb818142f80 0x7fb818159780 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 --2- 172.21.15.119:0/3308295534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb81806cb40 0x7fb8180b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.389+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 >> 172.21.15.119:0/3308295534 conn(0x7fb8180cb610 msgr2=0x7fb8180c4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.390+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 shutdown_connections 2023-11-08T19:25:27.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.390+0000 7fb81e5b6640 1 -- 172.21.15.119:0/3308295534 wait complete. 2023-11-08T19:25:27.410 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:27.411 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:27.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.629+0000 7fc0002a2640 1 Processor -- start 2023-11-08T19:25:27.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.629+0000 7fc0002a2640 1 -- start start 2023-11-08T19:25:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fc0002a2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbff80b0320 0x7fbff80cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fc0002a2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff80cc050 con 0x7fbff80cdd40 2023-11-08T19:25:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fc0002a2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbff815d6f0 con 0x7fbff815d310 2023-11-08T19:25:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fc0002a2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbff806de80 con 0x7fbff80b0320 2023-11-08T19:25:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fbffe83f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbff80b0320 0x7fbff80cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fbffd83d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbff80cdd40 0x7fbff80cb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60096/0 (socket says 172.21.15.119:60096) 2023-11-08T19:25:27.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.631+0000 7fbffd83d640 1 -- 172.21.15.119:0/2839023147 learned_addr learned my addr 172.21.15.119:0/2839023147 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:27.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.632+0000 7fbffe83f640 1 -- 172.21.15.119:0/2839023147 >> v1:172.21.15.188:6789/0 conn(0x7fbff815d310 legacy=0x7fbff80cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.632+0000 7fbffe83f640 1 -- 172.21.15.119:0/2839023147 >> v1:172.21.15.119:6789/0 conn(0x7fbff80cdd40 legacy=0x7fbff80cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.632+0000 7fbffe83f640 1 -- 172.21.15.119:0/2839023147 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff806e160 con 0x7fbff80b0320 2023-11-08T19:25:27.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.633+0000 7fbffe83f640 1 --2- 172.21.15.119:0/2839023147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff80b0320 0x7fbff80cbb10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009d90 tx=0x7fbff002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c8242dde5f61fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.633+0000 7fbffd03c640 1 -- 172.21.15.119:0/2839023147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff002ccf0 con 0x7fbff80b0320 2023-11-08T19:25:27.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.633+0000 7fbffd03c640 1 -- 172.21.15.119:0/2839023147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff002ce50 con 0x7fbff80b0320 2023-11-08T19:25:27.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.633+0000 7fbffd03c640 1 -- 172.21.15.119:0/2839023147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0034660 con 0x7fbff80b0320 2023-11-08T19:25:27.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 -- 172.21.15.119:0/2839023147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff80b0320 msgr2=0x7fbff80cbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 --2- 172.21.15.119:0/2839023147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff80b0320 0x7fbff80cbb10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbff0009d90 tx=0x7fbff002bf40 comp rx=0 tx=0).stop 2023-11-08T19:25:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 -- 172.21.15.119:0/2839023147 shutdown_connections 2023-11-08T19:25:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 --2- 172.21.15.119:0/2839023147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff80b0320 0x7fbff80cbb10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 -- 172.21.15.119:0/2839023147 >> 172.21.15.119:0/2839023147 conn(0x7fbff80d0db0 msgr2=0x7fbff80d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 -- 172.21.15.119:0/2839023147 shutdown_connections 2023-11-08T19:25:27.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.634+0000 7fc0002a2640 1 -- 172.21.15.119:0/2839023147 wait complete. 2023-11-08T19:25:27.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.636+0000 7fc0002a2640 1 Processor -- start 2023-11-08T19:25:27.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.636+0000 7fc0002a2640 1 -- start start 2023-11-08T19:25:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fc0002a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff80b0320 0x7fbff80c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fc0002a2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff80cdd40 0x7fbff8149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fc0002a2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 0x7fbff80b5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fc0002a2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff80c5a60 con 0x7fbff815d310 2023-11-08T19:25:27.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fc0002a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbff8156bb0 con 0x7fbff80b0320 2023-11-08T19:25:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fc0002a2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbff80c4de0 con 0x7fbff80cdd40 2023-11-08T19:25:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffe83f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 0x7fbff80b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffe83f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 0x7fbff80b5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51308/0 (socket says 172.21.15.119:51308) 2023-11-08T19:25:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffe83f640 1 -- 172.21.15.119:0/947633454 learned_addr learned my addr 172.21.15.119:0/947633454 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:27.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffd83d640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff80cdd40 0x7fbff8149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffe03e640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff80b0320 0x7fbff80c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffe83f640 1 -- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff80cdd40 msgr2=0x7fbff8149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.638+0000 7fbffe83f640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff80cdd40 0x7fbff8149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbffe83f640 1 -- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff80b0320 msgr2=0x7fbff80c5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbffe83f640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff80b0320 0x7fbff80c5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbffe83f640 1 -- 172.21.15.119:0/947633454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff0009a70 con 0x7fbff815d310 2023-11-08T19:25:27.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbffe83f640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 0x7fbff80b5650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbff0037040 tx=0x7fbff0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff003d600 con 0x7fbff815d310 2023-11-08T19:25:27.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff003d760 con 0x7fbff815d310 2023-11-08T19:25:27.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff003daf0 con 0x7fbff815d310 2023-11-08T19:25:27.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbff80c50c0 con 0x7fbff815d310 2023-11-08T19:25:27.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.639+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbff80c5680 con 0x7fbff815d310 2023-11-08T19:25:27.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.640+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbff80c5ba0 con 0x7fbff815d310 2023-11-08T19:25:27.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.641+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbff003b040 con 0x7fbff815d310 2023-11-08T19:25:27.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.643+0000 7fbfeeffd640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfdc06ae70 0x7fbfdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:27.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.643+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbff00ac0d0 con 0x7fbff815d310 2023-11-08T19:25:27.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.643+0000 7fbffe03e640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfdc06ae70 0x7fbfdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:27.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.644+0000 7fbffe03e640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfdc06ae70 0x7fbfdc06d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbff4000fb0 tx=0x7fbff4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:27.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.647+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbff80c5ba0 con 0x7fbff815d310 2023-11-08T19:25:27.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.840+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbff80c5ba0 con 0x7fbff815d310 2023-11-08T19:25:27.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.841+0000 7fbfeeffd640 1 -- 172.21.15.119:0/947633454 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbff004e020 con 0x7fbff815d310 2023-11-08T19:25:27.843 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:27.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfdc06ae70 msgr2=0x7fbfdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfdc06ae70 0x7fbfdc06d330 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbff4000fb0 tx=0x7fbff4009040 comp rx=0 tx=0).stop 2023-11-08T19:25:27.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 msgr2=0x7fbff80b5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:27.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 0x7fbff80b5650 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbff0037040 tx=0x7fbff0004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:27.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 shutdown_connections 2023-11-08T19:25:27.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbfdc06ae70 0x7fbfdc06d330 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbff815d310 0x7fbff80b5650 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbff80cdd40 0x7fbff8149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 --2- 172.21.15.119:0/947633454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbff80b0320 0x7fbff80c5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:27.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 >> 172.21.15.119:0/947633454 conn(0x7fbff80d0db0 msgr2=0x7fbff806f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:27.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 shutdown_connections 2023-11-08T19:25:27.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:27.844+0000 7fc0002a2640 1 -- 172.21.15.119:0/947633454 wait complete. 2023-11-08T19:25:27.865 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:28.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.069+0000 7f162b032640 1 Processor -- start 2023-11-08T19:25:28.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.069+0000 7f162b032640 1 -- start start 2023-11-08T19:25:28.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.070+0000 7f162b032640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f16240cdd40 0x7f162406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f162b032640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162406f460 con 0x7f16240b0320 2023-11-08T19:25:28.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f162b032640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162415d6f0 con 0x7f162415d310 2023-11-08T19:25:28.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f162b032640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162406de80 con 0x7f16240cdd40 2023-11-08T19:25:28.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f16295cf640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f16240b0320 0x7f16240cb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60110/0 (socket says 172.21.15.119:60110) 2023-11-08T19:25:28.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f16295cf640 1 -- 172.21.15.119:0/1150588343 learned_addr learned my addr 172.21.15.119:0/1150588343 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f1623fff640 1 --2- 172.21.15.119:0/1150588343 >> v2:172.21.15.188:3300/0 conn(0x7f16240cdd40 0x7f162406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015141674 0 0) 0x7f161c001490 con 0x7f16240b0320 2023-11-08T19:25:28.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.071+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1614003520 con 0x7f16240b0320 2023-11-08T19:25:28.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3596099346 0 0) 0x7f1614003520 con 0x7f16240b0320 2023-11-08T19:25:28.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 >> v2:172.21.15.188:3300/0 conn(0x7f16240cdd40 msgr2=0x7f162406ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 --2- 172.21.15.119:0/1150588343 >> v2:172.21.15.188:3300/0 conn(0x7f16240cdd40 0x7f162406ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 >> v1:172.21.15.188:6789/0 conn(0x7f162415d310 legacy=0x7f16240cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162406e160 con 0x7f16240b0320 2023-11-08T19:25:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f161c001b80 con 0x7f16240b0320 2023-11-08T19:25:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 >> v1:172.21.15.119:6789/0 conn(0x7f16240b0320 legacy=0x7f16240cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16140039c0 0x7f1614003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140047c0 0x7f1614004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614005180 0x7f161400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162406de80 con 0x7f16140039c0 2023-11-08T19:25:28.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161400c080 con 0x7f16140047c0 2023-11-08T19:25:28.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161400c240 con 0x7f1614005180 2023-11-08T19:25:28.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f1628dce640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16140039c0 0x7f1614003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.072+0000 7f1623fff640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140047c0 0x7f1614004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f16295cf640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614005180 0x7f161400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f1628dce640 1 -- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614005180 msgr2=0x7f161400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f1628dce640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1614005180 0x7f161400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f1628dce640 1 -- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140047c0 msgr2=0x7f1614004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f1628dce640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140047c0 0x7f1614004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f1628dce640 1 -- 172.21.15.119:0/1150588343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1614003520 con 0x7f16140039c0 2023-11-08T19:25:28.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.073+0000 7f16295cf640 1 -- 172.21.15.119:0/1150588343 reap_dead start 2023-11-08T19:25:28.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.074+0000 7f1628dce640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16140039c0 0x7f1614003de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1618001770 tx=0x7f16140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9c5d3614e6e6bd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.074+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16180052d0 con 0x7f16140039c0 2023-11-08T19:25:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.074+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161802e050 con 0x7f16140039c0 2023-11-08T19:25:28.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.074+0000 7f16237fe640 1 -- 172.21.15.119:0/1150588343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618003640 con 0x7f16140039c0 2023-11-08T19:25:28.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 -- 172.21.15.119:0/1150588343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16140039c0 msgr2=0x7f1614003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16140039c0 0x7f1614003de0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1618001770 tx=0x7f16140057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 -- 172.21.15.119:0/1150588343 shutdown_connections 2023-11-08T19:25:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 --2- 172.21.15.119:0/1150588343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16140039c0 0x7f1614003de0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 -- 172.21.15.119:0/1150588343 >> 172.21.15.119:0/1150588343 conn(0x7f16240d0db0 msgr2=0x7f16240d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 -- 172.21.15.119:0/1150588343 shutdown_connections 2023-11-08T19:25:28.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.075+0000 7f162b032640 1 -- 172.21.15.119:0/1150588343 wait complete. 2023-11-08T19:25:28.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.077+0000 7f162b032640 1 Processor -- start 2023-11-08T19:25:28.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.077+0000 7f162b032640 1 -- start start 2023-11-08T19:25:28.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f162b032640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140039c0 0x7f162414d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f162b032640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 0x7f16240ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f162b032640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16240b51f0 0x7f16240aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f162b032640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162415a840 con 0x7f16240b4d70 2023-11-08T19:25:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f162b032640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240b3d30 con 0x7f16140039c0 2023-11-08T19:25:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f162b032640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240b4010 con 0x7f16240b51f0 2023-11-08T19:25:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 0x7f16240ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 0x7f16240ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51328/0 (socket says 172.21.15.119:51328) 2023-11-08T19:25:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 -- 172.21.15.119:0/723975052 learned_addr learned my addr 172.21.15.119:0/723975052 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 -- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16240b51f0 msgr2=0x7f16240aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16240b51f0 0x7f16240aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1628dce640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140039c0 0x7f162414d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 -- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140039c0 msgr2=0x7f162414d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140039c0 0x7f162414d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.078+0000 7f1623fff640 1 -- 172.21.15.119:0/723975052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16240b42f0 con 0x7f16240b4d70 2023-11-08T19:25:28.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.079+0000 7f1623fff640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 0x7f16240ae740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f16100116d0 tx=0x7f1610011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.079+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000f9c0 con 0x7f16240b4d70 2023-11-08T19:25:28.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.079+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161000fb20 con 0x7f16240b4d70 2023-11-08T19:25:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.079+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16240b45d0 con 0x7f16240b4d70 2023-11-08T19:25:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.079+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161000fe90 con 0x7f16240b4d70 2023-11-08T19:25:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.079+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16240b48b0 con 0x7f16240b4d70 2023-11-08T19:25:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.080+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162409d5e0 con 0x7f16240b4d70 2023-11-08T19:25:28.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.081+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1610015460 con 0x7f16240b4d70 2023-11-08T19:25:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.082+0000 7f16217fa640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f160806ae70 0x7f160806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.082+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1610089d80 con 0x7f16240b4d70 2023-11-08T19:25:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.082+0000 7f1628dce640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f160806ae70 0x7f160806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.083+0000 7f1628dce640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f160806ae70 0x7f160806d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1618033000 tx=0x7f1618009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.086+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1610054b30 con 0x7f16240b4d70 2023-11-08T19:25:28.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.279+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f162406c990 con 0x7f16240b4d70 2023-11-08T19:25:28.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.280+0000 7f16217fa640 1 -- 172.21.15.119:0/723975052 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1610058150 con 0x7f16240b4d70 2023-11-08T19:25:28.282 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:28.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f160806ae70 msgr2=0x7f160806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f160806ae70 0x7f160806d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1618033000 tx=0x7f1618009030 comp rx=0 tx=0).stop 2023-11-08T19:25:28.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 msgr2=0x7f16240ae740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 0x7f16240ae740 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f16100116d0 tx=0x7f1610011b90 comp rx=0 tx=0).stop 2023-11-08T19:25:28.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 shutdown_connections 2023-11-08T19:25:28.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f160806ae70 0x7f160806d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f16240b51f0 0x7f16240aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f16240b4d70 0x7f16240ae740 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 --2- 172.21.15.119:0/723975052 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f16140039c0 0x7f162414d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.283+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 >> 172.21.15.119:0/723975052 conn(0x7f16240d0db0 msgr2=0x7f16240cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.284+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 shutdown_connections 2023-11-08T19:25:28.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.284+0000 7f162b032640 1 -- 172.21.15.119:0/723975052 wait complete. 2023-11-08T19:25:28.303 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:28.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.513+0000 7f7219eaf640 1 Processor -- start 2023-11-08T19:25:28.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.513+0000 7f7219eaf640 1 -- start start 2023-11-08T19:25:28.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f7219eaf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7214161870 0x7f72140cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f7219eaf640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721415df50 con 0x7f721415d4a0 2023-11-08T19:25:28.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f7219eaf640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72140d1e60 con 0x7f72140cfa00 2023-11-08T19:25:28.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f7219eaf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72140d03b0 con 0x7f7214161870 2023-11-08T19:25:28.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f72137fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f721415d4a0 0x7f72140d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60116/0 (socket says 172.21.15.119:60116) 2023-11-08T19:25:28.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f72137fe640 1 -- 172.21.15.119:0/1416722929 learned_addr learned my addr 172.21.15.119:0/1416722929 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:28.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.515+0000 7f7213fff640 1 --2- 172.21.15.119:0/1416722929 >> v2:172.21.15.119:3300/0 conn(0x7f7214161870 0x7f72140cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.516+0000 7f7212ffd640 1 -- 172.21.15.119:0/1416722929 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672965846 0 0) 0x7f721415df50 con 0x7f721415d4a0 2023-11-08T19:25:28.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.516+0000 7f7212ffd640 1 -- 172.21.15.119:0/1416722929 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7200003520 con 0x7f721415d4a0 2023-11-08T19:25:28.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.516+0000 7f7213fff640 1 -- 172.21.15.119:0/1416722929 >> v1:172.21.15.188:6789/0 conn(0x7f72140cfa00 legacy=0x7f7214161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.516+0000 7f7213fff640 1 -- 172.21.15.119:0/1416722929 >> v1:172.21.15.119:6789/0 conn(0x7f721415d4a0 legacy=0x7f72140d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.516+0000 7f7213fff640 1 -- 172.21.15.119:0/1416722929 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214161d40 con 0x7f7214161870 2023-11-08T19:25:28.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.517+0000 7f7213fff640 1 --2- 172.21.15.119:0/1416722929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7214161870 0x7f72140cfe70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f720c0097a0 tx=0x7f720c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3f56a767aa99d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.517+0000 7f7212ffd640 1 -- 172.21.15.119:0/1416722929 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c031830 con 0x7f7214161870 2023-11-08T19:25:28.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.517+0000 7f7212ffd640 1 -- 172.21.15.119:0/1416722929 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c031990 con 0x7f7214161870 2023-11-08T19:25:28.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.517+0000 7f7212ffd640 1 -- 172.21.15.119:0/1416722929 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c031d20 con 0x7f7214161870 2023-11-08T19:25:28.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 -- 172.21.15.119:0/1416722929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7214161870 msgr2=0x7f72140cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1416722929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7214161870 0x7f72140cfe70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f720c0097a0 tx=0x7f720c02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 -- 172.21.15.119:0/1416722929 shutdown_connections 2023-11-08T19:25:28.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1416722929 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7214161870 0x7f72140cfe70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 -- 172.21.15.119:0/1416722929 >> 172.21.15.119:0/1416722929 conn(0x7f721406a9f0 msgr2=0x7f721406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 -- 172.21.15.119:0/1416722929 shutdown_connections 2023-11-08T19:25:28.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.518+0000 7f7219eaf640 1 -- 172.21.15.119:0/1416722929 wait complete. 2023-11-08T19:25:28.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.520+0000 7f7219eaf640 1 Processor -- start 2023-11-08T19:25:28.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.520+0000 7f7219eaf640 1 -- start start 2023-11-08T19:25:28.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7219eaf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 0x7f721414d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7219eaf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721415d4a0 0x7f72140cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7219eaf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7214161870 0x7f72140c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7219eaf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72140cc210 con 0x7f72140cfa00 2023-11-08T19:25:28.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7219eaf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721414d840 con 0x7f721415d4a0 2023-11-08T19:25:28.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7219eaf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721415fee0 con 0x7f7214161870 2023-11-08T19:25:28.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 0x7f721414d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 0x7f721414d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51348/0 (socket says 172.21.15.119:51348) 2023-11-08T19:25:28.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 -- 172.21.15.119:0/1539585880 learned_addr learned my addr 172.21.15.119:0/1539585880 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:28.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 -- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7214161870 msgr2=0x7f72140c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f7213fff640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7214161870 0x7f72140c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f720affd640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721415d4a0 0x7f72140cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7214161870 0x7f72140c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 -- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721415d4a0 msgr2=0x7f72140cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721415d4a0 0x7f72140cc6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.521+0000 7f72137fe640 1 -- 172.21.15.119:0/1539585880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c009400 con 0x7f72140cfa00 2023-11-08T19:25:28.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.522+0000 7f72137fe640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 0x7f721414d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f720400cbc0 tx=0x7f720400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.522+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400ad40 con 0x7f72140cfa00 2023-11-08T19:25:28.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.522+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7204012400 con 0x7f72140cfa00 2023-11-08T19:25:28.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.522+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720400aea0 con 0x7f72140cfa00 2023-11-08T19:25:28.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.522+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72140cbbc0 con 0x7f72140cfa00 2023-11-08T19:25:28.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.522+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72140ccbe0 con 0x7f72140cfa00 2023-11-08T19:25:28.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.523+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72140b8360 con 0x7f72140cfa00 2023-11-08T19:25:28.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.524+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7204007070 con 0x7f72140cfa00 2023-11-08T19:25:28.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.525+0000 7f7210ff9640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71f406ae70 0x7f71f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.525+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f7204089800 con 0x7f72140cfa00 2023-11-08T19:25:28.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.525+0000 7f720affd640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71f406ae70 0x7f71f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.526+0000 7f720affd640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71f406ae70 0x7f71f406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f721414db90 tx=0x7f71fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.529+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7204054670 con 0x7f72140cfa00 2023-11-08T19:25:28.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.714+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72140d10f0 con 0x7f72140cfa00 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.715+0000 7f7210ff9640 1 -- 172.21.15.119:0/1539585880 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f72040170a0 con 0x7f72140cfa00 2023-11-08T19:25:28.716 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71f406ae70 msgr2=0x7f71f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71f406ae70 0x7f71f406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f721414db90 tx=0x7f71fc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 msgr2=0x7f721414d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 0x7f721414d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f720400cbc0 tx=0x7f720400a430 comp rx=0 tx=0).stop 2023-11-08T19:25:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 shutdown_connections 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71f406ae70 0x7f71f406d330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7214161870 0x7f72140c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f721415d4a0 0x7f72140cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 --2- 172.21.15.119:0/1539585880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f72140cfa00 0x7f721414d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 >> 172.21.15.119:0/1539585880 conn(0x7f721406a9f0 msgr2=0x7f721415dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 shutdown_connections 2023-11-08T19:25:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.717+0000 7f7219eaf640 1 -- 172.21.15.119:0/1539585880 wait complete. 2023-11-08T19:25:28.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:28.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.943+0000 7fbe74294640 1 Processor -- start 2023-11-08T19:25:28.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.943+0000 7fbe74294640 1 -- start start 2023-11-08T19:25:28.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.944+0000 7fbe74294640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c15d310 0x7fbe6c06ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe74294640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbe6c0b0320 0x7fbe6c0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe74294640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe6c06b440 0x7fbe6c0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe74294640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c06f460 con 0x7fbe6c06b440 2023-11-08T19:25:28.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe74294640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe6c15d6f0 con 0x7fbe6c0b0320 2023-11-08T19:25:28.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe74294640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c06de80 con 0x7fbe6c15d310 2023-11-08T19:25:28.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe72831640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe6c06b440 0x7fbe6c0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe72831640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe6c06b440 0x7fbe6c0cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57864/0 (socket says 172.21.15.119:57864) 2023-11-08T19:25:28.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe72831640 1 -- 172.21.15.119:0/3535142918 learned_addr learned my addr 172.21.15.119:0/3535142918 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:28.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe7182f640 1 --2- 172.21.15.119:0/3535142918 >> v2:172.21.15.188:3300/0 conn(0x7fbe6c0b0320 0x7fbe6c0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe72030640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c15d310 0x7fbe6c06ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe72831640 1 -- 172.21.15.119:0/3535142918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c15d310 msgr2=0x7fbe6c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.945+0000 7fbe72831640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c15d310 0x7fbe6c06ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.946+0000 7fbe72831640 1 -- 172.21.15.119:0/3535142918 >> v2:172.21.15.188:3300/0 conn(0x7fbe6c0b0320 msgr2=0x7fbe6c0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:28.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.946+0000 7fbe72831640 1 --2- 172.21.15.119:0/3535142918 >> v2:172.21.15.188:3300/0 conn(0x7fbe6c0b0320 0x7fbe6c0cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.946+0000 7fbe72831640 1 -- 172.21.15.119:0/3535142918 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c06e160 con 0x7fbe6c06b440 2023-11-08T19:25:28.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.946+0000 7fbe72030640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c15d310 0x7fbe6c06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:28.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.947+0000 7fbe72831640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c06b440 0x7fbe6c0cdb70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c06fb90 tx=0x7fbe6402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242265218bb2088c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.947+0000 7fbe7102e640 1 -- 172.21.15.119:0/3535142918 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64031860 con 0x7fbe6c06b440 2023-11-08T19:25:28.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.947+0000 7fbe7102e640 1 -- 172.21.15.119:0/3535142918 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe640319c0 con 0x7fbe6c06b440 2023-11-08T19:25:28.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.947+0000 7fbe7102e640 1 -- 172.21.15.119:0/3535142918 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64031d50 con 0x7fbe6c06b440 2023-11-08T19:25:28.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 -- 172.21.15.119:0/3535142918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c06b440 msgr2=0x7fbe6c0cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c06b440 0x7fbe6c0cdb70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbe6c06fb90 tx=0x7fbe6402b900 comp rx=0 tx=0).stop 2023-11-08T19:25:28.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 -- 172.21.15.119:0/3535142918 shutdown_connections 2023-11-08T19:25:28.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c06b440 0x7fbe6c0cdb70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 --2- 172.21.15.119:0/3535142918 >> v2:172.21.15.188:3300/0 conn(0x7fbe6c0b0320 0x7fbe6c0cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 --2- 172.21.15.119:0/3535142918 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c15d310 0x7fbe6c06ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 -- 172.21.15.119:0/3535142918 >> 172.21.15.119:0/3535142918 conn(0x7fbe6c0d0db0 msgr2=0x7fbe6c0d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 -- 172.21.15.119:0/3535142918 shutdown_connections 2023-11-08T19:25:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.948+0000 7fbe74294640 1 -- 172.21.15.119:0/3535142918 wait complete. 2023-11-08T19:25:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.950+0000 7fbe74294640 1 Processor -- start 2023-11-08T19:25:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.950+0000 7fbe74294640 1 -- start start 2023-11-08T19:25:28.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe74294640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c06b440 0x7fbe6c1f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe74294640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0b0320 0x7fbe6c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe74294640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 0x7fbe6c0af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe74294640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b54e0 con 0x7fbe6c15d310 2023-11-08T19:25:28.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe74294640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b48d0 con 0x7fbe6c0b0320 2023-11-08T19:25:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe74294640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c0b4bb0 con 0x7fbe6c06b440 2023-11-08T19:25:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 0x7fbe6c0af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 0x7fbe6c0af0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57878/0 (socket says 172.21.15.119:57878) 2023-11-08T19:25:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 -- 172.21.15.119:0/1060032425 learned_addr learned my addr 172.21.15.119:0/1060032425 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:28.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 -- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c06b440 msgr2=0x7fbe6c1f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c06b440 0x7fbe6c1f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe7182f640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0b0320 0x7fbe6c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 -- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0b0320 msgr2=0x7fbe6c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0b0320 0x7fbe6c158640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.952+0000 7fbe72831640 1 -- 172.21.15.119:0/1060032425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe640094d0 con 0x7fbe6c15d310 2023-11-08T19:25:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.953+0000 7fbe72831640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 0x7fbe6c0af0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbe640090b0 tx=0x7fbe640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.953+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe6403ac60 con 0x7fbe6c15d310 2023-11-08T19:25:28.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.953+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe6403adc0 con 0x7fbe6c15d310 2023-11-08T19:25:28.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.953+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe64044630 con 0x7fbe6c15d310 2023-11-08T19:25:28.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.953+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe6c0b4e90 con 0x7fbe6c15d310 2023-11-08T19:25:28.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.953+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe6c1f5d90 con 0x7fbe6c15d310 2023-11-08T19:25:28.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.954+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6c06cc30 con 0x7fbe6c15d310 2023-11-08T19:25:28.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.955+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe6404d450 con 0x7fbe6c15d310 2023-11-08T19:25:28.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.956+0000 7fbe6affd640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe5806aea0 0x7fbe5806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.956+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fbe640abeb0 con 0x7fbe6c15d310 2023-11-08T19:25:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.956+0000 7fbe72030640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe5806aea0 0x7fbe5806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:28.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.957+0000 7fbe72030640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe5806aea0 0x7fbe5806d360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c008e50 tx=0x7fbe5c008380 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:28.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:28.961+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe64076c60 con 0x7fbe6c15d310 2023-11-08T19:25:29.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.140+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbe6c06ef20 con 0x7fbe6c15d310 2023-11-08T19:25:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.141+0000 7fbe6affd640 1 -- 172.21.15.119:0/1060032425 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbe6407a280 con 0x7fbe6c15d310 2023-11-08T19:25:29.142 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe5806aea0 msgr2=0x7fbe5806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe5806aea0 0x7fbe5806d360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c008e50 tx=0x7fbe5c008380 comp rx=0 tx=0).stop 2023-11-08T19:25:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 msgr2=0x7fbe6c0af0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 0x7fbe6c0af0a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbe640090b0 tx=0x7fbe640046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 shutdown_connections 2023-11-08T19:25:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe5806aea0 0x7fbe5806d360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c15d310 0x7fbe6c0af0a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0b0320 0x7fbe6c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 --2- 172.21.15.119:0/1060032425 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe6c06b440 0x7fbe6c1f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.143+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 >> 172.21.15.119:0/1060032425 conn(0x7fbe6c0d0db0 msgr2=0x7fbe6c0d02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.144+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 shutdown_connections 2023-11-08T19:25:29.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.144+0000 7fbe74294640 1 -- 172.21.15.119:0/1060032425 wait complete. 2023-11-08T19:25:29.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:29.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.358+0000 7f663b59e640 1 Processor -- start 2023-11-08T19:25:29.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.358+0000 7f663b59e640 1 -- start start 2023-11-08T19:25:29.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663b59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f663c06d520 0x7f663c06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0cc640 0x7f663c0ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663b59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f663c15a870 con 0x7f663c06bee0 2023-11-08T19:25:29.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663b59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f663c06cbc0 con 0x7f663c06d520 2023-11-08T19:25:29.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c15cfa0 con 0x7f663c0cc640 2023-11-08T19:25:29.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663a59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f663c06bee0 0x7f663c06c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59826/0 (socket says 172.21.15.119:59826) 2023-11-08T19:25:29.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663a59c640 1 -- 172.21.15.119:0/3614681669 learned_addr learned my addr 172.21.15.119:0/3614681669 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:29.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f6631d9b640 1 --2- 172.21.15.119:0/3614681669 >> v2:172.21.15.119:3300/0 conn(0x7f663c06d520 0x7f663c06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.360+0000 7f663ad9d640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0cc640 0x7f663c0ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f6639d9b640 1 -- 172.21.15.119:0/3614681669 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3002879255 0 0) 0x7f662c001490 con 0x7f663c06bee0 2023-11-08T19:25:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f6639d9b640 1 -- 172.21.15.119:0/3614681669 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6628003520 con 0x7f663c06bee0 2023-11-08T19:25:29.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f6631d9b640 1 -- 172.21.15.119:0/3614681669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0cc640 msgr2=0x7f663c0ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f6631d9b640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0cc640 0x7f663c0ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f6631d9b640 1 -- 172.21.15.119:0/3614681669 >> v1:172.21.15.119:6789/0 conn(0x7f663c06bee0 legacy=0x7f663c06c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f6631d9b640 1 -- 172.21.15.119:0/3614681669 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0c9ad0 con 0x7f663c06d520 2023-11-08T19:25:29.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.361+0000 7f663ad9d640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0cc640 0x7f663c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.362+0000 7f6631d9b640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c06d520 0x7f663c06cfb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66240097a0 tx=0x7f662402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a74548e2d0bad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.362+0000 7f6639d9b640 1 -- 172.21.15.119:0/3614681669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624031830 con 0x7f663c06d520 2023-11-08T19:25:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.362+0000 7f6639d9b640 1 -- 172.21.15.119:0/3614681669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6624031990 con 0x7f663c06d520 2023-11-08T19:25:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.362+0000 7f6639d9b640 1 -- 172.21.15.119:0/3614681669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6624031d20 con 0x7f663c06d520 2023-11-08T19:25:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 -- 172.21.15.119:0/3614681669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c06d520 msgr2=0x7f663c06cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c06d520 0x7f663c06cfb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f66240097a0 tx=0x7f662402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 -- 172.21.15.119:0/3614681669 shutdown_connections 2023-11-08T19:25:29.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c0cc640 0x7f663c0ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 --2- 172.21.15.119:0/3614681669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c06d520 0x7f663c06cfb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 -- 172.21.15.119:0/3614681669 >> 172.21.15.119:0/3614681669 conn(0x7f663c0b0560 msgr2=0x7f663c0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 -- 172.21.15.119:0/3614681669 shutdown_connections 2023-11-08T19:25:29.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.363+0000 7f663b59e640 1 -- 172.21.15.119:0/3614681669 wait complete. 2023-11-08T19:25:29.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.364+0000 7f663b59e640 1 Processor -- start 2023-11-08T19:25:29.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.365+0000 7f663b59e640 1 -- start start 2023-11-08T19:25:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.365+0000 7f663b59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c06bee0 0x7f663c0b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.365+0000 7f663b59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c06d520 0x7f663c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663b59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 0x7f663c0aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663b59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b33c0 con 0x7f663c0cc640 2023-11-08T19:25:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663b59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c14ff20 con 0x7f663c06d520 2023-11-08T19:25:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663b59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663c0b3d20 con 0x7f663c06bee0 2023-11-08T19:25:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 0x7f663c0aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 0x7f663c0aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57896/0 (socket says 172.21.15.119:57896) 2023-11-08T19:25:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 -- 172.21.15.119:0/3920375083 learned_addr learned my addr 172.21.15.119:0/3920375083 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:29.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f6631d9b640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c06d520 0x7f663c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663a59c640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c06bee0 0x7f663c0b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 -- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c06bee0 msgr2=0x7f663c0b37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c06bee0 0x7f663c0b37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 -- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c06d520 msgr2=0x7f663c14cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c06d520 0x7f663c14cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663ad9d640 1 -- 172.21.15.119:0/3920375083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624009400 con 0x7f663c0cc640 2023-11-08T19:25:29.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f663a59c640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c06bee0 0x7f663c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:29.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.366+0000 7f6631d9b640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c06d520 0x7f663c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:29.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.367+0000 7f663ad9d640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 0x7f663c0aefe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f663400d800 tx=0x7f663400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.367+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634004030 con 0x7f663c0cc640 2023-11-08T19:25:29.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.367+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6634004190 con 0x7f663c0cc640 2023-11-08T19:25:29.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.367+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c1523e0 con 0x7f663c0cc640 2023-11-08T19:25:29.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.367+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6634004500 con 0x7f663c0cc640 2023-11-08T19:25:29.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.367+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c0b2d00 con 0x7f663c0cc640 2023-11-08T19:25:29.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.368+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c0b3500 con 0x7f663c0cc640 2023-11-08T19:25:29.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.369+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66340096a0 con 0x7f663c0cc640 2023-11-08T19:25:29.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.370+0000 7f66337fe640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f661c06ae70 0x7f661c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.371+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f6634089bf0 con 0x7f663c0cc640 2023-11-08T19:25:29.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.371+0000 7f663a59c640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f661c06ae70 0x7f661c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.371+0000 7f663a59c640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f661c06ae70 0x7f661c06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f662c0016f0 tx=0x7f662c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.375+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f663c0b3500 con 0x7f663c0cc640 2023-11-08T19:25:29.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.587+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f663c0b3500 con 0x7f663c0cc640 2023-11-08T19:25:29.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.587+0000 7f66337fe640 1 -- 172.21.15.119:0/3920375083 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f663c0b3500 con 0x7f663c0cc640 2023-11-08T19:25:29.590 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:29.590 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:29.590 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:25:29.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f661c06ae70 msgr2=0x7f661c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f661c06ae70 0x7f661c06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f662c0016f0 tx=0x7f662c009040 comp rx=0 tx=0).stop 2023-11-08T19:25:29.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 msgr2=0x7f663c0aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 0x7f663c0aefe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f663400d800 tx=0x7f663400dcc0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 shutdown_connections 2023-11-08T19:25:29.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f661c06ae70 0x7f661c06d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f663c0cc640 0x7f663c0aefe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f663c06d520 0x7f663c14cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 --2- 172.21.15.119:0/3920375083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f663c06bee0 0x7f663c0b37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 >> 172.21.15.119:0/3920375083 conn(0x7f663c0b0560 msgr2=0x7f663c06f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 shutdown_connections 2023-11-08T19:25:29.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.590+0000 7f663b59e640 1 -- 172.21.15.119:0/3920375083 wait complete. 2023-11-08T19:25:29.611 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.825+0000 7fc5f51d7640 1 Processor -- start 2023-11-08T19:25:29.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.825+0000 7fc5f51d7640 1 -- start start 2023-11-08T19:25:29.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.826+0000 7fc5f51d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.826+0000 7fc5f51d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5f00cf8c0 0x7fc5f00cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.826+0000 7fc5f51d7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5f00cda70 con 0x7fc5f0161250 2023-11-08T19:25:29.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.826+0000 7fc5f51d7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f0161da0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.826+0000 7fc5f51d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00d0280 con 0x7fc5f015b080 2023-11-08T19:25:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5f00cf8c0 0x7fc5f00cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc5f00cf8c0 0x7fc5f00cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57912/0 (socket says 172.21.15.119:57912) 2023-11-08T19:25:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5eed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 -- 172.21.15.119:0/76222219 learned_addr learned my addr 172.21.15.119:0/76222219 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:29.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 -- 172.21.15.119:0/76222219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 msgr2=0x7fc5f00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 --2- 172.21.15.119:0/76222219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 -- 172.21.15.119:0/76222219 >> v1:172.21.15.188:6789/0 conn(0x7fc5f0161250 legacy=0x7fc5f0161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.827+0000 7fc5ee59c640 1 -- 172.21.15.119:0/76222219 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0161fc0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.829+0000 7fc5ee59c640 1 --2- 172.21.15.119:0/76222219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00cfd40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009870 tx=0x7fc5e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b448fda076e53803 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.829+0000 7fc5edd9b640 1 -- 172.21.15.119:0/76222219 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4031640 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.829+0000 7fc5edd9b640 1 -- 172.21.15.119:0/76222219 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e40317a0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.829+0000 7fc5edd9b640 1 -- 172.21.15.119:0/76222219 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e4031b30 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.829+0000 7fc5f51d7640 1 -- 172.21.15.119:0/76222219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 msgr2=0x7fc5f00cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.829+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/76222219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00cfd40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4009870 tx=0x7fc5e402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.830+0000 7fc5f51d7640 1 -- 172.21.15.119:0/76222219 shutdown_connections 2023-11-08T19:25:29.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.830+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/76222219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00cfd40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.830+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/76222219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.830+0000 7fc5f51d7640 1 -- 172.21.15.119:0/76222219 >> 172.21.15.119:0/76222219 conn(0x7fc5f006a9f0 msgr2=0x7fc5f006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:29.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.830+0000 7fc5f51d7640 1 -- 172.21.15.119:0/76222219 shutdown_connections 2023-11-08T19:25:29.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.830+0000 7fc5f51d7640 1 -- 172.21.15.119:0/76222219 wait complete. 2023-11-08T19:25:29.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.831+0000 7fc5f51d7640 1 Processor -- start 2023-11-08T19:25:29.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.831+0000 7fc5f51d7640 1 -- start start 2023-11-08T19:25:29.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.832+0000 7fc5f51d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.832+0000 7fc5f51d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f014ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.832+0000 7fc5f51d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f0161250 0x7fc5f015ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.832+0000 7fc5f51d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c9a30 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.832+0000 7fc5f51d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f00c65d0 con 0x7fc5f0161250 2023-11-08T19:25:29.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.832+0000 7fc5f51d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f01505e0 con 0x7fc5f015b080 2023-11-08T19:25:29.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00c6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57926/0 (socket says 172.21.15.119:57926) 2023-11-08T19:25:29.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 -- 172.21.15.119:0/3270716808 learned_addr learned my addr 172.21.15.119:0/3270716808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:29.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 -- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 msgr2=0x7fc5f014ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f014ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 -- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f0161250 msgr2=0x7fc5f015ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:29.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5ef59e640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f0161250 0x7fc5f015ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f0161250 0x7fc5f015ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:29.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 -- 172.21.15.119:0/3270716808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40094d0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5ef59e640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f0161250 0x7fc5f015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:29.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.833+0000 7fc5eed9d640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00c6090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e9b0 tx=0x7fc5e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.834+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e0009670 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.834+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e000c9a0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.834+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e000cd50 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.834+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f014f2a0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.834+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f014f820 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.835+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f0074430 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.836+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5e0013470 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.837+0000 7fc5d77fe640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc5dc06ae70 0x7fc5dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:29.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.837+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fc5e0089bb0 con 0x7fc5f00cf8c0 2023-11-08T19:25:29.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.837+0000 7fc5ee59c640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc5dc06ae70 0x7fc5dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:29.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.838+0000 7fc5ee59c640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc5dc06ae70 0x7fc5dc06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4036000 tx=0x7fc5e4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:29.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:29.841+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5e00548d0 con 0x7fc5f00cf8c0 2023-11-08T19:25:30.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.034+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5f00d1120 con 0x7fc5f00cf8c0 2023-11-08T19:25:30.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.036+0000 7fc5d77fe640 1 -- 172.21.15.119:0/3270716808 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7fc5e0017090 con 0x7fc5f00cf8c0 2023-11-08T19:25:30.038 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:30.038 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:30.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc5dc06ae70 msgr2=0x7fc5dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc5dc06ae70 0x7fc5dc06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5e4036000 tx=0x7fc5e4030000 comp rx=0 tx=0).stop 2023-11-08T19:25:30.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 msgr2=0x7fc5f00c6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00c6090 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc5e000e9b0 tx=0x7fc5e000ee70 comp rx=0 tx=0).stop 2023-11-08T19:25:30.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 shutdown_connections 2023-11-08T19:25:30.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc5dc06ae70 0x7fc5dc06d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5f0161250 0x7fc5f015ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5f015b080 0x7fc5f014ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 --2- 172.21.15.119:0/3270716808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5f00cf8c0 0x7fc5f00c6090 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 >> 172.21.15.119:0/3270716808 conn(0x7fc5f006a9f0 msgr2=0x7fc5f00d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 shutdown_connections 2023-11-08T19:25:30.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.039+0000 7fc5f51d7640 1 -- 172.21.15.119:0/3270716808 wait complete. 2023-11-08T19:25:30.060 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:30.061 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:30.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.257+0000 7f66f4a56640 1 Processor -- start 2023-11-08T19:25:30.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.257+0000 7f66f4a56640 1 -- start start 2023-11-08T19:25:30.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66f4a56640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66f0144580 0x7f66f015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66f4a56640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66f0144960 0x7f66f00cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66f4a56640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f00c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66f4a56640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f00c9150 con 0x7f66f0144960 2023-11-08T19:25:30.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66f4a56640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f015ea60 con 0x7f66f0144580 2023-11-08T19:25:30.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66f4a56640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00cb270 con 0x7f66f015f1d0 2023-11-08T19:25:30.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66edd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66f0144960 0x7f66f00cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66edd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66f0144960 0x7f66f00cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57930/0 (socket says 172.21.15.119:57930) 2023-11-08T19:25:30.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66edd9b640 1 -- 172.21.15.119:0/4102707225 learned_addr learned my addr 172.21.15.119:0/4102707225 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:30.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66ee59c640 1 --2- 172.21.15.119:0/4102707225 >> v2:172.21.15.188:3300/0 conn(0x7f66f0144580 0x7f66f015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66eed9d640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f00c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66edd9b640 1 -- 172.21.15.119:0/4102707225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 msgr2=0x7f66f00c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.259+0000 7f66edd9b640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f00c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.260+0000 7f66edd9b640 1 -- 172.21.15.119:0/4102707225 >> v2:172.21.15.188:3300/0 conn(0x7f66f0144580 msgr2=0x7f66f015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.260+0000 7f66edd9b640 1 --2- 172.21.15.119:0/4102707225 >> v2:172.21.15.188:3300/0 conn(0x7f66f0144580 0x7f66f015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.260+0000 7f66edd9b640 1 -- 172.21.15.119:0/4102707225 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f00c9360 con 0x7f66f0144960 2023-11-08T19:25:30.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.260+0000 7f66eed9d640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f00c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:30.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.261+0000 7f66edd9b640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144960 0x7f66f00cad30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66f00ceab0 tx=0x7f66e402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2653019d91386 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.261+0000 7f66ed59a640 1 -- 172.21.15.119:0/4102707225 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e40317d0 con 0x7f66f0144960 2023-11-08T19:25:30.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.261+0000 7f66ed59a640 1 -- 172.21.15.119:0/4102707225 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4031930 con 0x7f66f0144960 2023-11-08T19:25:30.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.261+0000 7f66ed59a640 1 -- 172.21.15.119:0/4102707225 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4031cc0 con 0x7f66f0144960 2023-11-08T19:25:30.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 -- 172.21.15.119:0/4102707225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144960 msgr2=0x7f66f00cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144960 0x7f66f00cad30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66f00ceab0 tx=0x7f66e402b870 comp rx=0 tx=0).stop 2023-11-08T19:25:30.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 -- 172.21.15.119:0/4102707225 shutdown_connections 2023-11-08T19:25:30.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f00c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 --2- 172.21.15.119:0/4102707225 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144960 0x7f66f00cad30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 --2- 172.21.15.119:0/4102707225 >> v2:172.21.15.188:3300/0 conn(0x7f66f0144580 0x7f66f015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 -- 172.21.15.119:0/4102707225 >> 172.21.15.119:0/4102707225 conn(0x7f66f00cb610 msgr2=0x7f66f00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 -- 172.21.15.119:0/4102707225 shutdown_connections 2023-11-08T19:25:30.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.262+0000 7f66f4a56640 1 -- 172.21.15.119:0/4102707225 wait complete. 2023-11-08T19:25:30.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.264+0000 7f66f4a56640 1 Processor -- start 2023-11-08T19:25:30.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.264+0000 7f66f4a56640 1 -- start start 2023-11-08T19:25:30.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66f4a56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 0x7f66f013f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66f4a56640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66f0144960 0x7f66f013b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66f4a56640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f013b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66f4a56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f013d460 con 0x7f66f0144580 2023-11-08T19:25:30.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66f4a56640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f013f9f0 con 0x7f66f0144960 2023-11-08T19:25:30.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66f4a56640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00c69c0 con 0x7f66f015f1d0 2023-11-08T19:25:30.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.265+0000 7f66ee59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 0x7f66f013f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 0x7f66f013f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57946/0 (socket says 172.21.15.119:57946) 2023-11-08T19:25:30.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 -- 172.21.15.119:0/3426512878 learned_addr learned my addr 172.21.15.119:0/3426512878 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:30.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66edd9b640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66f0144960 0x7f66f013b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66eed9d640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f013b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 -- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 msgr2=0x7f66f013b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f013b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 -- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66f0144960 msgr2=0x7f66f013b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66f0144960 0x7f66f013b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 -- 172.21.15.119:0/3426512878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40094d0 con 0x7f66f0144580 2023-11-08T19:25:30.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66edd9b640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66f0144960 0x7f66f013b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:30.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66eed9d640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f013b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:30.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.266+0000 7f66ee59c640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 0x7f66f013f4b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66e000d800 tx=0x7f66e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.267+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00096a0 con 0x7f66f0144580 2023-11-08T19:25:30.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.267+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e0009800 con 0x7f66f0144580 2023-11-08T19:25:30.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.267+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f00c6ca0 con 0x7f66f0144580 2023-11-08T19:25:30.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.267+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0009b90 con 0x7f66f0144580 2023-11-08T19:25:30.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.267+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f00c7260 con 0x7f66f0144580 2023-11-08T19:25:30.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.268+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f013d700 con 0x7f66f0144580 2023-11-08T19:25:30.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.269+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66e0009cf0 con 0x7f66f0144580 2023-11-08T19:25:30.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.270+0000 7f66deffd640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66d006ae70 0x7f66d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.270+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f66e0089d90 con 0x7f66f0144580 2023-11-08T19:25:30.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.270+0000 7f66edd9b640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66d006ae70 0x7f66d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.271+0000 7f66edd9b640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66d006ae70 0x7f66d006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66e4036000 tx=0x7f66e403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.274+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66e0054b40 con 0x7f66f0144580 2023-11-08T19:25:30.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.456+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66f0074430 con 0x7f66f0144580 2023-11-08T19:25:30.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.457+0000 7f66deffd640 1 -- 172.21.15.119:0/3426512878 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f66e000b440 con 0x7f66f0144580 2023-11-08T19:25:30.459 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:30.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66d006ae70 msgr2=0x7f66d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66d006ae70 0x7f66d006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f66e4036000 tx=0x7f66e403d040 comp rx=0 tx=0).stop 2023-11-08T19:25:30.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 msgr2=0x7f66f013f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 0x7f66f013f4b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f66e000d800 tx=0x7f66e000dcc0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 shutdown_connections 2023-11-08T19:25:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66d006ae70 0x7f66d006d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66f015f1d0 0x7f66f013b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66f0144960 0x7f66f013b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 --2- 172.21.15.119:0/3426512878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66f0144580 0x7f66f013f4b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 >> 172.21.15.119:0/3426512878 conn(0x7f66f00cb610 msgr2=0x7f66f015bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 shutdown_connections 2023-11-08T19:25:30.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.459+0000 7f66f4a56640 1 -- 172.21.15.119:0/3426512878 wait complete. 2023-11-08T19:25:30.479 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.693+0000 7ff401f33640 1 Processor -- start 2023-11-08T19:25:30.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.693+0000 7ff401f33640 1 -- start start 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff401f33640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff3fc15d4a0 0x7ff3fc0d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff401f33640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3fc0cfa00 0x7ff3fc0cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff401f33640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff401f33640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc0d03d0 con 0x7ff3fc0cfa00 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff401f33640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3fc15df50 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff401f33640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0d1e60 con 0x7ff3fc161160 2023-11-08T19:25:30.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff3f3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3fc0cfa00 0x7ff3fc0cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff3f3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3fc0cfa00 0x7ff3fc0cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57950/0 (socket says 172.21.15.119:57950) 2023-11-08T19:25:30.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.695+0000 7ff3f3fff640 1 -- 172.21.15.119:0/3232568888 learned_addr learned my addr 172.21.15.119:0/3232568888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3fb7fe640 1 --2- 172.21.15.119:0/3232568888 >> v2:172.21.15.188:3300/0 conn(0x7ff3fc15d4a0 0x7ff3fc0d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3fbfff640 1 --2- 172.21.15.119:0/3232568888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3f3fff640 1 -- 172.21.15.119:0/3232568888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 msgr2=0x7ff3fc161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3f3fff640 1 --2- 172.21.15.119:0/3232568888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3f3fff640 1 -- 172.21.15.119:0/3232568888 >> v2:172.21.15.188:3300/0 conn(0x7ff3fc15d4a0 msgr2=0x7ff3fc0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:30.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3f3fff640 1 --2- 172.21.15.119:0/3232568888 >> v2:172.21.15.188:3300/0 conn(0x7ff3fc15d4a0 0x7ff3fc0d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.696+0000 7ff3f3fff640 1 -- 172.21.15.119:0/3232568888 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3fc0d2830 con 0x7ff3fc0cfa00 2023-11-08T19:25:30.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.698+0000 7ff3f3fff640 1 --2- 172.21.15.119:0/3232568888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc0cfe90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc072830 tx=0x7ff3e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3512a4d061bb0f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.698+0000 7ff3faffd640 1 -- 172.21.15.119:0/3232568888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40317e0 con 0x7ff3fc0cfa00 2023-11-08T19:25:30.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.698+0000 7ff3faffd640 1 -- 172.21.15.119:0/3232568888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4031940 con 0x7ff3fc0cfa00 2023-11-08T19:25:30.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.698+0000 7ff3faffd640 1 -- 172.21.15.119:0/3232568888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4031cd0 con 0x7ff3fc0cfa00 2023-11-08T19:25:30.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 -- 172.21.15.119:0/3232568888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc0cfa00 msgr2=0x7ff3fc0cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 --2- 172.21.15.119:0/3232568888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc0cfe90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc072830 tx=0x7ff3e402b880 comp rx=0 tx=0).stop 2023-11-08T19:25:30.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 -- 172.21.15.119:0/3232568888 shutdown_connections 2023-11-08T19:25:30.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 --2- 172.21.15.119:0/3232568888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 --2- 172.21.15.119:0/3232568888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc0cfe90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 --2- 172.21.15.119:0/3232568888 >> v2:172.21.15.188:3300/0 conn(0x7ff3fc15d4a0 0x7ff3fc0d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 -- 172.21.15.119:0/3232568888 >> 172.21.15.119:0/3232568888 conn(0x7ff3fc06a9f0 msgr2=0x7ff3fc06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 -- 172.21.15.119:0/3232568888 shutdown_connections 2023-11-08T19:25:30.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.699+0000 7ff401f33640 1 -- 172.21.15.119:0/3232568888 wait complete. 2023-11-08T19:25:30.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.701+0000 7ff401f33640 1 Processor -- start 2023-11-08T19:25:30.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.701+0000 7ff401f33640 1 -- start start 2023-11-08T19:25:30.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff401f33640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff401f33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 0x7ff3fc0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff401f33640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff401f33640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c8810 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff401f33640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c7b90 con 0x7ff3fc0cfa00 2023-11-08T19:25:30.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff401f33640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3fc0c7e70 con 0x7ff3fc161160 2023-11-08T19:25:30.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 0x7ff3fc0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 0x7ff3fc0c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57964/0 (socket says 172.21.15.119:57964) 2023-11-08T19:25:30.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff3f3fff640 1 -- 172.21.15.119:0/1243021644 learned_addr learned my addr 172.21.15.119:0/1243021644 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:30.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff3fb7fe640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.702+0000 7ff3fbfff640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f3fff640 1 -- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 msgr2=0x7ff3fc0c9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f3fff640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc0c9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f3fff640 1 -- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3fc0cfa00 msgr2=0x7ff3fc14e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f3fff640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc14e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f3fff640 1 -- 172.21.15.119:0/1243021644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e40094d0 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3fb7fe640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:30.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f3fff640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 0x7ff3fc0c9550 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3e402b8c0 tx=0x7ff3e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4032880 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e4032a00 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3fc0c8150 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4032db0 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.703+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3fc14ea80 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.705+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3fc0c8950 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.706+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3e40453f0 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.706+0000 7ff3f8ff9640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3dc06ae70 0x7ff3dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:30.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.707+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff3e40abc30 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.707+0000 7ff3fb7fe640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3dc06ae70 0x7ff3dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:30.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.707+0000 7ff3fb7fe640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3dc06ae70 0x7ff3dc06d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006fd0 tx=0x7ff3ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:30.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.713+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3fc0c8950 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.895+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff3fc0c8950 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.896+0000 7ff3f8ff9640 1 -- 172.21.15.119:0/1243021644 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff3fc0c8950 con 0x7ff3fc15d4a0 2023-11-08T19:25:30.897 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3dc06ae70 msgr2=0x7ff3dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3dc06ae70 0x7ff3dc06d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff3ec006fd0 tx=0x7ff3ec008040 comp rx=0 tx=0).stop 2023-11-08T19:25:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 msgr2=0x7ff3fc0c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:30.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 0x7ff3fc0c9550 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3e402b8c0 tx=0x7ff3e40046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 shutdown_connections 2023-11-08T19:25:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3dc06ae70 0x7ff3dc06d330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3fc161160 0x7ff3fc0c9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3fc15d4a0 0x7ff3fc0c9550 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 --2- 172.21.15.119:0/1243021644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3fc0cfa00 0x7ff3fc14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 >> 172.21.15.119:0/1243021644 conn(0x7ff3fc06a9f0 msgr2=0x7ff3fc0cde80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 shutdown_connections 2023-11-08T19:25:30.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:30.898+0000 7ff401f33640 1 -- 172.21.15.119:0/1243021644 wait complete. 2023-11-08T19:25:30.918 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:31.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.127+0000 7f1dfe3fb640 1 Processor -- start 2023-11-08T19:25:31.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.127+0000 7f1dfe3fb640 1 -- start start 2023-11-08T19:25:31.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.129+0000 7f1dfe3fb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1df815bff0 0x7f1df80c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.129+0000 7f1dfe3fb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df815ea60 con 0x7f1df8144580 2023-11-08T19:25:31.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.129+0000 7f1dfe3fb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1df815c460 con 0x7f1df8144960 2023-11-08T19:25:31.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.129+0000 7f1dfe3fb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df80c9150 con 0x7f1df815bff0 2023-11-08T19:25:31.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.129+0000 7f1df7fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1df8144580 0x7f1df80cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59834/0 (socket says 172.21.15.119:59834) 2023-11-08T19:25:31.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.129+0000 7f1df7fff640 1 -- 172.21.15.119:0/2662291735 learned_addr learned my addr 172.21.15.119:0/2662291735 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:31.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1dfc998640 1 --2- 172.21.15.119:0/2662291735 >> v2:172.21.15.188:3300/0 conn(0x7f1df815bff0 0x7f1df80c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086256413 0 0) 0x7f1df815ea60 con 0x7f1df8144580 2023-11-08T19:25:31.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de4003520 con 0x7f1df8144580 2023-11-08T19:25:31.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2575181766 0 0) 0x7f1de4003520 con 0x7f1df8144580 2023-11-08T19:25:31.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 >> v2:172.21.15.188:3300/0 conn(0x7f1df815bff0 msgr2=0x7f1df80c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 --2- 172.21.15.119:0/2662291735 >> v2:172.21.15.188:3300/0 conn(0x7f1df815bff0 0x7f1df80c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 >> v1:172.21.15.188:6789/0 conn(0x7f1df8144960 legacy=0x7f1df815f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df815f8e0 con 0x7f1df8144580 2023-11-08T19:25:31.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f1de8001670 con 0x7f1df8144580 2023-11-08T19:25:31.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 >> v1:172.21.15.119:6789/0 conn(0x7f1df8144580 legacy=0x7f1df80cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1de40039c0 0x7f1de4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.130+0000 7f1df77fe640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40047c0 0x7f1de4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df77fe640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1de40051d0 0x7f1de400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c9150 con 0x7f1de40039c0 2023-11-08T19:25:31.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de400c120 con 0x7f1de40051d0 2023-11-08T19:25:31.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de400c2e0 con 0x7f1de40047c0 2023-11-08T19:25:31.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1de40039c0 0x7f1de4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1deffff640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40047c0 0x7f1de4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df7fff640 1 -- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40047c0 msgr2=0x7f1de4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40047c0 0x7f1de4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df7fff640 1 -- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1de40051d0 msgr2=0x7f1de400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1de40051d0 0x7f1de400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1df7fff640 1 -- 172.21.15.119:0/2662291735 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de4003520 con 0x7f1de40039c0 2023-11-08T19:25:31.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.131+0000 7f1dfc998640 1 -- 172.21.15.119:0/2662291735 reap_dead start 2023-11-08T19:25:31.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.132+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1de40039c0 0x7f1de4003de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1de8001eb0 tx=0x7f1de40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22d030e899611a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de8005ad0 con 0x7f1de40039c0 2023-11-08T19:25:31.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de8005c30 con 0x7f1de40039c0 2023-11-08T19:25:31.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1df77fe640 1 -- 172.21.15.119:0/2662291735 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de802ca00 con 0x7f1de40039c0 2023-11-08T19:25:31.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2662291735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1de40039c0 msgr2=0x7f1de4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1de40039c0 0x7f1de4003de0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1de8001eb0 tx=0x7f1de40057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2662291735 shutdown_connections 2023-11-08T19:25:31.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2662291735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1de40039c0 0x7f1de4003de0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2662291735 >> 172.21.15.119:0/2662291735 conn(0x7f1df80cb610 msgr2=0x7f1df80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2662291735 shutdown_connections 2023-11-08T19:25:31.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.133+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2662291735 wait complete. 2023-11-08T19:25:31.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.135+0000 7f1dfe3fb640 1 Processor -- start 2023-11-08T19:25:31.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.135+0000 7f1dfe3fb640 1 -- start start 2023-11-08T19:25:31.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1dfe3fb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40039c0 0x7f1df80c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1dfe3fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 0x7f1df815a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1dfe3fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df813e970 0x7f1df813ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1dfe3fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df813f360 con 0x7f1df813dd90 2023-11-08T19:25:31.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1dfe3fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c6260 con 0x7f1df813e970 2023-11-08T19:25:31.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1dfe3fb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df80c6540 con 0x7f1de40039c0 2023-11-08T19:25:31.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1deffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 0x7f1df815a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1deffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 0x7f1df815a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57980/0 (socket says 172.21.15.119:57980) 2023-11-08T19:25:31.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.136+0000 7f1deffff640 1 -- 172.21.15.119:0/2436503067 learned_addr learned my addr 172.21.15.119:0/2436503067 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:31.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1dfc998640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df813e970 0x7f1df813ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40039c0 0x7f1df80c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1deffff640 1 -- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40039c0 msgr2=0x7f1df80c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1deffff640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40039c0 0x7f1df80c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1deffff640 1 -- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df813e970 msgr2=0x7f1df813ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1deffff640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df813e970 0x7f1df813ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1deffff640 1 -- 172.21.15.119:0/2436503067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80c6820 con 0x7f1df813dd90 2023-11-08T19:25:31.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1deffff640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 0x7f1df815a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1de000e410 tx=0x7f1de000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.137+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00052e0 con 0x7f1df813dd90 2023-11-08T19:25:31.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.138+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0005440 con 0x7f1df813dd90 2023-11-08T19:25:31.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.138+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0003810 con 0x7f1df813dd90 2023-11-08T19:25:31.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.138+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df80c6b00 con 0x7f1df813dd90 2023-11-08T19:25:31.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.138+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df815ace0 con 0x7f1df813dd90 2023-11-08T19:25:31.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.139+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df80c6cf0 con 0x7f1df813dd90 2023-11-08T19:25:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.140+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1de0003970 con 0x7f1df813dd90 2023-11-08T19:25:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.141+0000 7f1df57fa640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1dd806ae70 0x7f1dd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.141+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f1de00890d0 con 0x7f1df813dd90 2023-11-08T19:25:31.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.141+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1dd806ae70 0x7f1dd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.141+0000 7f1df7fff640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1dd806ae70 0x7f1dd806d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1de8034000 tx=0x7f1de8009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.145+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1de0053f30 con 0x7f1df813dd90 2023-11-08T19:25:31.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.337+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1df806cb10 con 0x7f1df813dd90 2023-11-08T19:25:31.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.337+0000 7f1df57fa640 1 -- 172.21.15.119:0/2436503067 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1de00190a0 con 0x7f1df813dd90 2023-11-08T19:25:31.339 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:31.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1dd806ae70 msgr2=0x7f1dd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1dd806ae70 0x7f1dd806d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1de8034000 tx=0x7f1de8009030 comp rx=0 tx=0).stop 2023-11-08T19:25:31.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 msgr2=0x7f1df815a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 0x7f1df815a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1de000e410 tx=0x7f1de000e8d0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 shutdown_connections 2023-11-08T19:25:31.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1dd806ae70 0x7f1dd806d330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1df813e970 0x7f1df813ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1df813dd90 0x7f1df815a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 --2- 172.21.15.119:0/2436503067 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1de40039c0 0x7f1df80c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.340+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 >> 172.21.15.119:0/2436503067 conn(0x7f1df80cb610 msgr2=0x7f1df8072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.341+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 shutdown_connections 2023-11-08T19:25:31.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.341+0000 7f1dfe3fb640 1 -- 172.21.15.119:0/2436503067 wait complete. 2023-11-08T19:25:31.361 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:31.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.569+0000 7f9a9e089640 1 Processor -- start 2023-11-08T19:25:31.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.569+0000 7f9a9e089640 1 -- start start 2023-11-08T19:25:31.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.570+0000 7f9a9e089640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9a98143600 0x7f9a980c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.570+0000 7f9a9e089640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a9806d250 0x7f9a9815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.570+0000 7f9a9e089640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9815e440 con 0x7f9a9815d9f0 2023-11-08T19:25:31.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.570+0000 7f9a9e089640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a980c4f50 con 0x7f9a98143600 2023-11-08T19:25:31.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.570+0000 7f9a9e089640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a980c4960 con 0x7f9a9806d250 2023-11-08T19:25:31.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a8ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9a9815d9f0 0x7f9a9806cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59846/0 (socket says 172.21.15.119:59846) 2023-11-08T19:25:31.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a8ffff640 1 -- 172.21.15.119:0/866574470 learned_addr learned my addr 172.21.15.119:0/866574470 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:31.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a977fe640 1 --2- 172.21.15.119:0/866574470 >> v2:172.21.15.119:3300/0 conn(0x7f9a98143600 0x7f9a980c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a97fff640 1 --2- 172.21.15.119:0/866574470 >> v2:172.21.15.188:3300/0 conn(0x7f9a9806d250 0x7f9a9815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a96ffd640 1 -- 172.21.15.119:0/866574470 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821056365 0 0) 0x7f9a9815e440 con 0x7f9a9815d9f0 2023-11-08T19:25:31.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a96ffd640 1 -- 172.21.15.119:0/866574470 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a84005730 con 0x7f9a9815d9f0 2023-11-08T19:25:31.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a977fe640 1 -- 172.21.15.119:0/866574470 >> v2:172.21.15.188:3300/0 conn(0x7f9a9806d250 msgr2=0x7f9a9815de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:31.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a977fe640 1 --2- 172.21.15.119:0/866574470 >> v2:172.21.15.188:3300/0 conn(0x7f9a9806d250 0x7f9a9815de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a977fe640 1 -- 172.21.15.119:0/866574470 >> v1:172.21.15.119:6789/0 conn(0x7f9a9815d9f0 legacy=0x7f9a9806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a977fe640 1 -- 172.21.15.119:0/866574470 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9815e720 con 0x7f9a98143600 2023-11-08T19:25:31.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.571+0000 7f9a97fff640 1 --2- 172.21.15.119:0/866574470 >> v2:172.21.15.188:3300/0 conn(0x7f9a9806d250 0x7f9a9815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:31.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.572+0000 7f9a977fe640 1 --2- 172.21.15.119:0/866574470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a98143600 0x7f9a980c5370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a980cc650 tx=0x7f9a8802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333888dc3c611d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a96ffd640 1 -- 172.21.15.119:0/866574470 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8802e070 con 0x7f9a98143600 2023-11-08T19:25:31.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a96ffd640 1 -- 172.21.15.119:0/866574470 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a880315a0 con 0x7f9a98143600 2023-11-08T19:25:31.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a96ffd640 1 -- 172.21.15.119:0/866574470 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88031930 con 0x7f9a98143600 2023-11-08T19:25:31.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a9e089640 1 -- 172.21.15.119:0/866574470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a98143600 msgr2=0x7f9a980c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a9e089640 1 --2- 172.21.15.119:0/866574470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a98143600 0x7f9a980c5370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9a980cc650 tx=0x7f9a8802b840 comp rx=0 tx=0).stop 2023-11-08T19:25:31.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a9e089640 1 -- 172.21.15.119:0/866574470 shutdown_connections 2023-11-08T19:25:31.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a9e089640 1 --2- 172.21.15.119:0/866574470 >> v2:172.21.15.188:3300/0 conn(0x7f9a9806d250 0x7f9a9815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.573+0000 7f9a9e089640 1 --2- 172.21.15.119:0/866574470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a98143600 0x7f9a980c5370 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.574+0000 7f9a9e089640 1 -- 172.21.15.119:0/866574470 >> 172.21.15.119:0/866574470 conn(0x7f9a980cb610 msgr2=0x7f9a980cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.574+0000 7f9a9e089640 1 -- 172.21.15.119:0/866574470 shutdown_connections 2023-11-08T19:25:31.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.574+0000 7f9a9e089640 1 -- 172.21.15.119:0/866574470 wait complete. 2023-11-08T19:25:31.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.575+0000 7f9a9e089640 1 Processor -- start 2023-11-08T19:25:31.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.576+0000 7f9a9e089640 1 -- start start 2023-11-08T19:25:31.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a9e089640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 0x7f9a9806cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a9e089640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a98143600 0x7f9a981edf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a9e089640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815d9f0 0x7f9a9814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a9e089640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a981393a0 con 0x7f9a9806d250 2023-11-08T19:25:31.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a9e089640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9806d0e0 con 0x7f9a98143600 2023-11-08T19:25:31.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a9e089640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a980c70b0 con 0x7f9a9815d9f0 2023-11-08T19:25:31.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 0x7f9a9806cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 0x7f9a9806cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58004/0 (socket says 172.21.15.119:58004) 2023-11-08T19:25:31.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 -- 172.21.15.119:0/164729659 learned_addr learned my addr 172.21.15.119:0/164729659 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a8ffff640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a98143600 0x7f9a981edf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 -- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815d9f0 msgr2=0x7f9a9814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815d9f0 0x7f9a9814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 -- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a98143600 msgr2=0x7f9a981edf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a98143600 0x7f9a981edf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.577+0000 7f9a977fe640 1 -- 172.21.15.119:0/164729659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88009400 con 0x7f9a9806d250 2023-11-08T19:25:31.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.578+0000 7f9a977fe640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 0x7f9a9806cba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009890 tx=0x7f9a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.578+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88032840 con 0x7f9a9806d250 2023-11-08T19:25:31.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.578+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a880329c0 con 0x7f9a9806d250 2023-11-08T19:25:31.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.578+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a88032d70 con 0x7f9a9806d250 2023-11-08T19:25:31.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.578+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a980c6780 con 0x7f9a9806d250 2023-11-08T19:25:31.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.578+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a980c6d40 con 0x7f9a9806d250 2023-11-08T19:25:31.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.579+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a98074430 con 0x7f9a9806d250 2023-11-08T19:25:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.580+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a8804d3f0 con 0x7f9a9806d250 2023-11-08T19:25:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.581+0000 7f9a94ff9640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a7806aea0 0x7f9a7806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.581+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9a880ab3e0 con 0x7f9a9806d250 2023-11-08T19:25:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.581+0000 7f9a8ffff640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a7806aea0 0x7f9a7806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:31.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.582+0000 7f9a8ffff640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a7806aea0 0x7f9a7806d360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9a98071db0 tx=0x7f9a80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:31.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.585+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a88076230 con 0x7f9a9806d250 2023-11-08T19:25:31.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.777+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9a980c4230 con 0x7f9a9806d250 2023-11-08T19:25:31.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.778+0000 7f9a94ff9640 1 -- 172.21.15.119:0/164729659 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9a88079850 con 0x7f9a9806d250 2023-11-08T19:25:31.780 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:31.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a7806aea0 msgr2=0x7f9a7806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a7806aea0 0x7f9a7806d360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9a98071db0 tx=0x7f9a80009040 comp rx=0 tx=0).stop 2023-11-08T19:25:31.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 msgr2=0x7f9a9806cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:31.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 0x7f9a9806cba0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a88009890 tx=0x7f9a880046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 shutdown_connections 2023-11-08T19:25:31.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a7806aea0 0x7f9a7806d360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a9815d9f0 0x7f9a9814eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a98143600 0x7f9a981edf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 --2- 172.21.15.119:0/164729659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a9806d250 0x7f9a9806cba0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:31.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 >> 172.21.15.119:0/164729659 conn(0x7f9a980cb610 msgr2=0x7f9a980cdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:31.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 shutdown_connections 2023-11-08T19:25:31.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:31.781+0000 7f9a9e089640 1 -- 172.21.15.119:0/164729659 wait complete. 2023-11-08T19:25:31.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.004+0000 7fd4a516e640 1 Processor -- start 2023-11-08T19:25:32.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.004+0000 7fd4a516e640 1 -- start start 2023-11-08T19:25:32.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.005+0000 7fd4a516e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4a00c9720 0x7fd4a00c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.005+0000 7fd4a516e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd4a01610c0 0x7fd4a01634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.005+0000 7fd4a516e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4a00c9360 con 0x7fd4a0159d90 2023-11-08T19:25:32.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.005+0000 7fd4a516e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a006d3c0 con 0x7fd4a00c9720 2023-11-08T19:25:32.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.005+0000 7fd4a516e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a00ca490 con 0x7fd4a01610c0 2023-11-08T19:25:32.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd497fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4a00c9720 0x7fd4a00c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd497fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd4a00c9720 0x7fd4a00c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58020/0 (socket says 172.21.15.119:58020) 2023-11-08T19:25:32.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd497fff640 1 -- 172.21.15.119:0/3057653003 learned_addr learned my addr 172.21.15.119:0/3057653003 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:32.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd49f59e640 1 --2- 172.21.15.119:0/3057653003 >> v2:172.21.15.188:3300/0 conn(0x7fd4a01610c0 0x7fd4a01634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd497fff640 1 -- 172.21.15.119:0/3057653003 >> v2:172.21.15.188:3300/0 conn(0x7fd4a01610c0 msgr2=0x7fd4a01634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd497fff640 1 --2- 172.21.15.119:0/3057653003 >> v2:172.21.15.188:3300/0 conn(0x7fd4a01610c0 0x7fd4a01634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.006+0000 7fd497fff640 1 -- 172.21.15.119:0/3057653003 >> v1:172.21.15.188:6789/0 conn(0x7fd4a0159d90 legacy=0x7fd4a015a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.007+0000 7fd497fff640 1 -- 172.21.15.119:0/3057653003 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a00ca770 con 0x7fd4a00c9720 2023-11-08T19:25:32.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.008+0000 7fd497fff640 1 --2- 172.21.15.119:0/3057653003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a00c9720 0x7fd4a00c9bd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4a006ae40 tx=0x7fd48802e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64c99f084edecea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.008+0000 7fd49e59c640 1 -- 172.21.15.119:0/3057653003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48802ee90 con 0x7fd4a00c9720 2023-11-08T19:25:32.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.008+0000 7fd49e59c640 1 -- 172.21.15.119:0/3057653003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488036440 con 0x7fd4a00c9720 2023-11-08T19:25:32.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.008+0000 7fd49e59c640 1 -- 172.21.15.119:0/3057653003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4880367d0 con 0x7fd4a00c9720 2023-11-08T19:25:32.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 -- 172.21.15.119:0/3057653003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a00c9720 msgr2=0x7fd4a00c9bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 --2- 172.21.15.119:0/3057653003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a00c9720 0x7fd4a00c9bd0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd4a006ae40 tx=0x7fd48802e0e0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 -- 172.21.15.119:0/3057653003 shutdown_connections 2023-11-08T19:25:32.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 --2- 172.21.15.119:0/3057653003 >> v2:172.21.15.188:3300/0 conn(0x7fd4a01610c0 0x7fd4a01634b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 --2- 172.21.15.119:0/3057653003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a00c9720 0x7fd4a00c9bd0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 -- 172.21.15.119:0/3057653003 >> 172.21.15.119:0/3057653003 conn(0x7fd4a00cb610 msgr2=0x7fd4a00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 -- 172.21.15.119:0/3057653003 shutdown_connections 2023-11-08T19:25:32.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.009+0000 7fd4a516e640 1 -- 172.21.15.119:0/3057653003 wait complete. 2023-11-08T19:25:32.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.011+0000 7fd4a516e640 1 Processor -- start 2023-11-08T19:25:32.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.011+0000 7fd4a516e640 1 -- start start 2023-11-08T19:25:32.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd4a516e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4a00c9720 0x7fd4a0151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd4a516e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4a0159d90 0x7fd4a00b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd4a516e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 0x7fd4a00b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd4a516e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01f3f50 con 0x7fd4a01610c0 2023-11-08T19:25:32.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd4a516e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01f3010 con 0x7fd4a0159d90 2023-11-08T19:25:32.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd4a516e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a01f32d0 con 0x7fd4a00c9720 2023-11-08T19:25:32.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd49f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 0x7fd4a00b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.012+0000 7fd49f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 0x7fd4a00b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58036/0 (socket says 172.21.15.119:58036) 2023-11-08T19:25:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 -- 172.21.15.119:0/913028517 learned_addr learned my addr 172.21.15.119:0/913028517 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49ed9d640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4a00c9720 0x7fd4a0151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd497fff640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4a0159d90 0x7fd4a00b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 -- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4a00c9720 msgr2=0x7fd4a0151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4a00c9720 0x7fd4a0151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 -- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4a0159d90 msgr2=0x7fd4a00b1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4a0159d90 0x7fd4a00b1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 -- 172.21.15.119:0/913028517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48800b770 con 0x7fd4a01610c0 2023-11-08T19:25:32.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.013+0000 7fd49f59e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 0x7fd4a00b1ec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd49800efc0 tx=0x7fd49800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.014+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498009ca0 con 0x7fd4a01610c0 2023-11-08T19:25:32.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.014+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49800f040 con 0x7fd4a01610c0 2023-11-08T19:25:32.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.014+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4980146c0 con 0x7fd4a01610c0 2023-11-08T19:25:32.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.014+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a01f35b0 con 0x7fd4a01610c0 2023-11-08T19:25:32.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.014+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a01f3b70 con 0x7fd4a01610c0 2023-11-08T19:25:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.015+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a0158c40 con 0x7fd4a01610c0 2023-11-08T19:25:32.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.016+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd498014820 con 0x7fd4a01610c0 2023-11-08T19:25:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.017+0000 7fd4977fe640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd48006aea0 0x7fd48006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.017+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fd49808a860 con 0x7fd4a01610c0 2023-11-08T19:25:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.017+0000 7fd49ed9d640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd48006aea0 0x7fd48006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.018+0000 7fd49ed9d640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd48006aea0 0x7fd48006d360 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00c8ff0 tx=0x7fd490009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.021+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd498055580 con 0x7fd4a01610c0 2023-11-08T19:25:32.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.244+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4a0074430 con 0x7fd4a01610c0 2023-11-08T19:25:32.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.245+0000 7fd4977fe640 1 -- 172.21.15.119:0/913028517 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fd498058ba0 con 0x7fd4a01610c0 2023-11-08T19:25:32.247 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:32.247 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:32.248 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:25:32.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd48006aea0 msgr2=0x7fd48006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd48006aea0 0x7fd48006d360 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00c8ff0 tx=0x7fd490009040 comp rx=0 tx=0).stop 2023-11-08T19:25:32.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 msgr2=0x7fd4a00b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 0x7fd4a00b1ec0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd49800efc0 tx=0x7fd49800c490 comp rx=0 tx=0).stop 2023-11-08T19:25:32.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 shutdown_connections 2023-11-08T19:25:32.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd48006aea0 0x7fd48006d360 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4a01610c0 0x7fd4a00b1ec0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4a0159d90 0x7fd4a00b1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 --2- 172.21.15.119:0/913028517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4a00c9720 0x7fd4a0151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 >> 172.21.15.119:0/913028517 conn(0x7fd4a00cb610 msgr2=0x7fd4a0158530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 shutdown_connections 2023-11-08T19:25:32.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.248+0000 7fd4a516e640 1 -- 172.21.15.119:0/913028517 wait complete. 2023-11-08T19:25:32.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:32.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.470+0000 7f06171d7640 1 Processor -- start 2023-11-08T19:25:32.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.471+0000 7f06171d7640 1 -- start start 2023-11-08T19:25:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06171d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06171d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f061015b900 0x7f0610070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06171d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06100710a0 0x7f061015da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06171d7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061006cff0 con 0x7f06100710a0 2023-11-08T19:25:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06171d7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061006d2d0 con 0x7f061015b900 2023-11-08T19:25:32.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06171d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061006bd20 con 0x7f061015b0d0 2023-11-08T19:25:32.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06169d6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06100710a0 0x7f061015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06169d6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f06100710a0 0x7f061015da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58052/0 (socket says 172.21.15.119:58052) 2023-11-08T19:25:32.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06169d6640 1 -- 172.21.15.119:0/2454996735 learned_addr learned my addr 172.21.15.119:0/2454996735 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:32.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06161d5640 1 --2- 172.21.15.119:0/2454996735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.472+0000 7f06159d4640 1 --2- 172.21.15.119:0/2454996735 >> v2:172.21.15.188:3300/0 conn(0x7f061015b900 0x7f0610070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.473+0000 7f06169d6640 1 -- 172.21.15.119:0/2454996735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 msgr2=0x7f06100ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.473+0000 7f06169d6640 1 --2- 172.21.15.119:0/2454996735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.473+0000 7f06169d6640 1 -- 172.21.15.119:0/2454996735 >> v2:172.21.15.188:3300/0 conn(0x7f061015b900 msgr2=0x7f0610070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.473+0000 7f06169d6640 1 --2- 172.21.15.119:0/2454996735 >> v2:172.21.15.188:3300/0 conn(0x7f061015b900 0x7f0610070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.473+0000 7f06169d6640 1 -- 172.21.15.119:0/2454996735 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061006d760 con 0x7f06100710a0 2023-11-08T19:25:32.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.474+0000 7f06169d6640 1 --2- 172.21.15.119:0/2454996735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06100710a0 0x7f061015da00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f061006b410 tx=0x7f060802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e29d41abfa13ba4e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.474+0000 7f06151d3640 1 -- 172.21.15.119:0/2454996735 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060802c9e0 con 0x7f06100710a0 2023-11-08T19:25:32.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.474+0000 7f06151d3640 1 -- 172.21.15.119:0/2454996735 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060802cb40 con 0x7f06100710a0 2023-11-08T19:25:32.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.474+0000 7f06151d3640 1 -- 172.21.15.119:0/2454996735 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06080343f0 con 0x7f06100710a0 2023-11-08T19:25:32.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 -- 172.21.15.119:0/2454996735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06100710a0 msgr2=0x7f061015da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 --2- 172.21.15.119:0/2454996735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06100710a0 0x7f061015da00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f061006b410 tx=0x7f060802be40 comp rx=0 tx=0).stop 2023-11-08T19:25:32.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 -- 172.21.15.119:0/2454996735 shutdown_connections 2023-11-08T19:25:32.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 --2- 172.21.15.119:0/2454996735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f06100710a0 0x7f061015da00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 --2- 172.21.15.119:0/2454996735 >> v2:172.21.15.188:3300/0 conn(0x7f061015b900 0x7f0610070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 --2- 172.21.15.119:0/2454996735 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 -- 172.21.15.119:0/2454996735 >> 172.21.15.119:0/2454996735 conn(0x7f06100b0560 msgr2=0x7f06100b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 -- 172.21.15.119:0/2454996735 shutdown_connections 2023-11-08T19:25:32.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.475+0000 7f06171d7640 1 -- 172.21.15.119:0/2454996735 wait complete. 2023-11-08T19:25:32.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.477+0000 7f06171d7640 1 Processor -- start 2023-11-08T19:25:32.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.477+0000 7f06171d7640 1 -- start start 2023-11-08T19:25:32.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06171d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100710a0 0x7f06101e6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06171d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06171d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 0x7f06100cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06171d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06100c84a0 con 0x7f061015b900 2023-11-08T19:25:32.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06171d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061014f7e0 con 0x7f06100710a0 2023-11-08T19:25:32.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06171d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06100c7820 con 0x7f061015b0d0 2023-11-08T19:25:32.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 0x7f06100cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 0x7f06100cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58068/0 (socket says 172.21.15.119:58068) 2023-11-08T19:25:32.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 -- 172.21.15.119:0/915298264 learned_addr learned my addr 172.21.15.119:0/915298264 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:32.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06159d4640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 -- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 msgr2=0x7f06100c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06161d5640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100710a0 0x7f06101e6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 -- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100710a0 msgr2=0x7f06101e6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100710a0 0x7f06101e6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.478+0000 7f06169d6640 1 -- 172.21.15.119:0/915298264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608009b30 con 0x7f061015b900 2023-11-08T19:25:32.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.479+0000 7f06169d6640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 0x7f06100cc620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0608009f40 tx=0x7f0608004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.479+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608009510 con 0x7f061015b900 2023-11-08T19:25:32.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.479+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f060803e450 con 0x7f061015b900 2023-11-08T19:25:32.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.479+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06100c7b00 con 0x7f061015b900 2023-11-08T19:25:32.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.479+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f060803e800 con 0x7f061015b900 2023-11-08T19:25:32.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.479+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06100c80c0 con 0x7f061015b900 2023-11-08T19:25:32.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.480+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06100c85e0 con 0x7f061015b900 2023-11-08T19:25:32.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.481+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f060803e960 con 0x7f061015b900 2023-11-08T19:25:32.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.482+0000 7f0606ffd640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05f406ae70 0x7f05f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.482+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f06080ac940 con 0x7f061015b900 2023-11-08T19:25:32.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.482+0000 7f06161d5640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05f406ae70 0x7f05f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.483+0000 7f06161d5640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05f406ae70 0x7f05f406d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f060c006fd0 tx=0x7f060c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.487+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0608077790 con 0x7f061015b900 2023-11-08T19:25:32.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.677+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f061004ece0 con 0x7f061015b900 2023-11-08T19:25:32.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.679+0000 7f0606ffd640 1 -- 172.21.15.119:0/915298264 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+18346 (secure 0 0 0) 0x7f060807adb0 con 0x7f061015b900 2023-11-08T19:25:32.680 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:32.681 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":25,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:04.587740+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:32.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05f406ae70 msgr2=0x7f05f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05f406ae70 0x7f05f406d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f060c006fd0 tx=0x7f060c008040 comp rx=0 tx=0).stop 2023-11-08T19:25:32.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 msgr2=0x7f06100cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 0x7f06100cc620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0608009f40 tx=0x7f0608004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:32.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 shutdown_connections 2023-11-08T19:25:32.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05f406ae70 0x7f05f406d330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f061015b900 0x7f06100cc620 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f061015b0d0 0x7f06100c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 --2- 172.21.15.119:0/915298264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06100710a0 0x7f06101e6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 >> 172.21.15.119:0/915298264 conn(0x7f06100b0560 msgr2=0x7f0610159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 shutdown_connections 2023-11-08T19:25:32.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.682+0000 7f06171d7640 1 -- 172.21.15.119:0/915298264 wait complete. 2023-11-08T19:25:32.702 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:25:32.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.896+0000 7fabe2294640 1 Processor -- start 2023-11-08T19:25:32.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.896+0000 7fabe2294640 1 -- start start 2023-11-08T19:25:32.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe2294640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fabdc0c9720 0x7fabdc0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe2294640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fabdc1610c0 0x7fabdc1634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe2294640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc0c9360 con 0x7fabdc159d90 2023-11-08T19:25:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe2294640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc06d3c0 con 0x7fabdc1610c0 2023-11-08T19:25:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe2294640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc0ca490 con 0x7fabdc0c9720 2023-11-08T19:25:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe0831640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fabdc1610c0 0x7fabdc1634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe0831640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fabdc1610c0 0x7fabdc1634b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58082/0 (socket says 172.21.15.119:58082) 2023-11-08T19:25:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabe0831640 1 -- 172.21.15.119:0/2113546937 learned_addr learned my addr 172.21.15.119:0/2113546937 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.898+0000 7fabdb7fe640 1 --2- 172.21.15.119:0/2113546937 >> v2:172.21.15.188:3300/0 conn(0x7fabdc0c9720 0x7fabdc0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.899+0000 7fabe0831640 1 -- 172.21.15.119:0/2113546937 >> v2:172.21.15.188:3300/0 conn(0x7fabdc0c9720 msgr2=0x7fabdc0c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.899+0000 7fabe0831640 1 --2- 172.21.15.119:0/2113546937 >> v2:172.21.15.188:3300/0 conn(0x7fabdc0c9720 0x7fabdc0c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.899+0000 7fabe0831640 1 -- 172.21.15.119:0/2113546937 >> v1:172.21.15.188:6789/0 conn(0x7fabdc159d90 legacy=0x7fabdc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.899+0000 7fabe0831640 1 -- 172.21.15.119:0/2113546937 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc0ca770 con 0x7fabdc1610c0 2023-11-08T19:25:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.900+0000 7fabe0831640 1 --2- 172.21.15.119:0/2113546937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc1610c0 0x7fabdc1634b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b18a0 tx=0x7fabd402ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04af08deec8195 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.900+0000 7fabdaffd640 1 -- 172.21.15.119:0/2113546937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4030840 con 0x7fabdc1610c0 2023-11-08T19:25:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.900+0000 7fabdaffd640 1 -- 172.21.15.119:0/2113546937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd40309a0 con 0x7fabdc1610c0 2023-11-08T19:25:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.900+0000 7fabdaffd640 1 -- 172.21.15.119:0/2113546937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4030d30 con 0x7fabdc1610c0 2023-11-08T19:25:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 -- 172.21.15.119:0/2113546937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc1610c0 msgr2=0x7fabdc1634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 --2- 172.21.15.119:0/2113546937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc1610c0 0x7fabdc1634b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fabdc0b18a0 tx=0x7fabd402ca20 comp rx=0 tx=0).stop 2023-11-08T19:25:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 -- 172.21.15.119:0/2113546937 shutdown_connections 2023-11-08T19:25:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 --2- 172.21.15.119:0/2113546937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc1610c0 0x7fabdc1634b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 --2- 172.21.15.119:0/2113546937 >> v2:172.21.15.188:3300/0 conn(0x7fabdc0c9720 0x7fabdc0c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 -- 172.21.15.119:0/2113546937 >> 172.21.15.119:0/2113546937 conn(0x7fabdc0cb610 msgr2=0x7fabdc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 -- 172.21.15.119:0/2113546937 shutdown_connections 2023-11-08T19:25:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.901+0000 7fabe2294640 1 -- 172.21.15.119:0/2113546937 wait complete. 2023-11-08T19:25:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.902+0000 7fabe2294640 1 Processor -- start 2023-11-08T19:25:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.903+0000 7fabe2294640 1 -- start start 2023-11-08T19:25:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabe2294640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabdc0c9720 0x7fabdc141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabe2294640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 0x7fabdc1501a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabe2294640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabdc1610c0 0x7fabdc15bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabe2294640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc152110 con 0x7fabdc159d90 2023-11-08T19:25:32.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabe2294640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc141b90 con 0x7fabdc1610c0 2023-11-08T19:25:32.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabe2294640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc155ff0 con 0x7fabdc0c9720 2023-11-08T19:25:32.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 0x7fabdc1501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 0x7fabdc1501a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58094/0 (socket says 172.21.15.119:58094) 2023-11-08T19:25:32.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 -- 172.21.15.119:0/3806583703 learned_addr learned my addr 172.21.15.119:0/3806583703 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:32.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 -- 172.21.15.119:0/3806583703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabdc0c9720 msgr2=0x7fabdc141650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabdc0c9720 0x7fabdc141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 -- 172.21.15.119:0/3806583703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabdc1610c0 msgr2=0x7fabdc15bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:32.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabdc1610c0 0x7fabdc15bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:32.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 -- 172.21.15.119:0/3806583703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40094d0 con 0x7fabdc159d90 2023-11-08T19:25:32.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.904+0000 7fabdb7fe640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 0x7fabdc1501a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007ef0 tx=0x7fabc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.905+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4014620 con 0x7fabdc159d90 2023-11-08T19:25:32.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.905+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc4014780 con 0x7fabdc159d90 2023-11-08T19:25:32.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.905+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc4014b30 con 0x7fabdc159d90 2023-11-08T19:25:32.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.905+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc1562d0 con 0x7fabdc159d90 2023-11-08T19:25:32.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.905+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc155ce0 con 0x7fabdc159d90 2023-11-08T19:25:32.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.906+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc074430 con 0x7fabdc159d90 2023-11-08T19:25:32.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.907+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabc4014c90 con 0x7fabdc159d90 2023-11-08T19:25:32.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.907+0000 7fabd8ff9640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fabbc06ae70 0x7fabbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:32.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.908+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7fabc408a3a0 con 0x7fabdc159d90 2023-11-08T19:25:32.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.908+0000 7fabdbfff640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fabbc06ae70 0x7fabbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:32.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.908+0000 7fabdbfff640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fabbc06ae70 0x7fabbc06d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabcc003cc0 tx=0x7fabcc0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:32.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:32.912+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabc40550c0 con 0x7fabdc159d90 2023-11-08T19:25:33.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.119+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fabdc0c8f10 con 0x7fabdc159d90 2023-11-08T19:25:33.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.121+0000 7fabd8ff9640 1 -- 172.21.15.119:0/3806583703 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fabc40586e0 con 0x7fabdc159d90 2023-11-08T19:25:33.122 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:33.123 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:25:33.123 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:25:33.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.123+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fabbc06ae70 msgr2=0x7fabbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.123+0000 7fabe2294640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fabbc06ae70 0x7fabbc06d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fabcc003cc0 tx=0x7fabcc0079b0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.123+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 msgr2=0x7fabdc1501a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.123+0000 7fabe2294640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 0x7fabdc1501a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fabc4007ef0 tx=0x7fabc400e4b0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 shutdown_connections 2023-11-08T19:25:33.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fabbc06ae70 0x7fabbc06d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fabdc1610c0 0x7fabdc15bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fabdc159d90 0x7fabdc1501a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 --2- 172.21.15.119:0/3806583703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fabdc0c9720 0x7fabdc141650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 >> 172.21.15.119:0/3806583703 conn(0x7fabdc0cb610 msgr2=0x7fabdc06a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 shutdown_connections 2023-11-08T19:25:33.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.124+0000 7fabe2294640 1 -- 172.21.15.119:0/3806583703 wait complete. 2023-11-08T19:25:33.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:24:58.104296+0000', 'modified': '2023-11-08T19:25:20.218164+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4254}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4254': {'gid': 4254, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/2629712656', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 2629712656}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 2629712656}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:25:33.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:25:33.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:25:33.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.770+0000 7f456440e640 1 Processor -- start 2023-11-08T19:25:33.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.770+0000 7f456440e640 1 -- start start 2023-11-08T19:25:33.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.771+0000 7f456440e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f455c06ebd0 0x7f455c161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.771+0000 7f456440e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f455c06efb0 0x7f455c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.771+0000 7f456440e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.771+0000 7f456440e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0cfbd0 con 0x7f455c06ebd0 2023-11-08T19:25:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.771+0000 7f456440e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0cf3e0 con 0x7f455c06efb0 2023-11-08T19:25:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.771+0000 7f456440e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c161e40 con 0x7f455c06a500 2023-11-08T19:25:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f455c06ebd0 0x7f455c161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f455c06ebd0 0x7f455c161900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58108/0 (socket says 172.21.15.119:58108) 2023-11-08T19:25:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 -- 172.21.15.119:0/2680291728 learned_addr learned my addr 172.21.15.119:0/2680291728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45619a9640 1 --2- 172.21.15.119:0/2680291728 >> v2:172.21.15.188:3300/0 conn(0x7f455c06efb0 0x7f455c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 -- 172.21.15.119:0/2680291728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 msgr2=0x7f455c06a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45629ab640 1 --2- 172.21.15.119:0/2680291728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 --2- 172.21.15.119:0/2680291728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c06a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 -- 172.21.15.119:0/2680291728 >> v2:172.21.15.188:3300/0 conn(0x7f455c06efb0 msgr2=0x7f455c070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 --2- 172.21.15.119:0/2680291728 >> v2:172.21.15.188:3300/0 conn(0x7f455c06efb0 0x7f455c070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45621aa640 1 -- 172.21.15.119:0/2680291728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0cd8f0 con 0x7f455c06ebd0 2023-11-08T19:25:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.772+0000 7f45619a9640 1 --2- 172.21.15.119:0/2680291728 >> v2:172.21.15.188:3300/0 conn(0x7f455c06efb0 0x7f455c070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:33.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.773+0000 7f45621aa640 1 --2- 172.21.15.119:0/2680291728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c161900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f455c066e00 tx=0x7f455802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd15f19502fa76ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f45611a8640 1 -- 172.21.15.119:0/2680291728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580317c0 con 0x7f455c06ebd0 2023-11-08T19:25:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f45611a8640 1 -- 172.21.15.119:0/2680291728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558031920 con 0x7f455c06ebd0 2023-11-08T19:25:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f45611a8640 1 -- 172.21.15.119:0/2680291728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558031cb0 con 0x7f455c06ebd0 2023-11-08T19:25:33.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 -- 172.21.15.119:0/2680291728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 msgr2=0x7f455c161900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 --2- 172.21.15.119:0/2680291728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c161900 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f455c066e00 tx=0x7f455802b860 comp rx=0 tx=0).stop 2023-11-08T19:25:33.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 -- 172.21.15.119:0/2680291728 shutdown_connections 2023-11-08T19:25:33.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 --2- 172.21.15.119:0/2680291728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c06a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 --2- 172.21.15.119:0/2680291728 >> v2:172.21.15.188:3300/0 conn(0x7f455c06efb0 0x7f455c070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 --2- 172.21.15.119:0/2680291728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c161900 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.774+0000 7f456440e640 1 -- 172.21.15.119:0/2680291728 >> 172.21.15.119:0/2680291728 conn(0x7f455c0cfd20 msgr2=0x7f455c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.775+0000 7f456440e640 1 -- 172.21.15.119:0/2680291728 shutdown_connections 2023-11-08T19:25:33.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.775+0000 7f456440e640 1 -- 172.21.15.119:0/2680291728 wait complete. 2023-11-08T19:25:33.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.777+0000 7f456440e640 1 Processor -- start 2023-11-08T19:25:33.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.777+0000 7f456440e640 1 -- start start 2023-11-08T19:25:33.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.778+0000 7f456440e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.778+0000 7f456440e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.778+0000 7f456440e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f455c06efb0 0x7f455c1612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.778+0000 7f456440e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0c3850 con 0x7f455c06ebd0 2023-11-08T19:25:33.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.778+0000 7f456440e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0b39d0 con 0x7f455c06efb0 2023-11-08T19:25:33.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.778+0000 7f456440e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0c3b10 con 0x7f455c06a500 2023-11-08T19:25:33.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58120/0 (socket says 172.21.15.119:58120) 2023-11-08T19:25:33.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 -- 172.21.15.119:0/1384823079 learned_addr learned my addr 172.21.15.119:0/1384823079 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:33.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45621aa640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45629ab640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f455c06efb0 0x7f455c1612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 -- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 msgr2=0x7f455c0b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c0b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 -- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f455c06efb0 msgr2=0x7f455c1612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f455c06efb0 0x7f455c1612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 -- 172.21.15.119:0/1384823079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45580094d0 con 0x7f455c06ebd0 2023-11-08T19:25:33.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.779+0000 7f45619a9640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c157bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f455000c7f0 tx=0x7f455000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.780+0000 7f454effd640 1 -- 172.21.15.119:0/1384823079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550007820 con 0x7f455c06ebd0 2023-11-08T19:25:33.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.780+0000 7f454effd640 1 -- 172.21.15.119:0/1384823079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550007980 con 0x7f455c06ebd0 2023-11-08T19:25:33.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.780+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c0c3df0 con 0x7f455c06ebd0 2023-11-08T19:25:33.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.780+0000 7f454effd640 1 -- 172.21.15.119:0/1384823079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550007cf0 con 0x7f455c06ebd0 2023-11-08T19:25:33.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.780+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c0ad0f0 con 0x7f455c06ebd0 2023-11-08T19:25:33.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.782+0000 7f454effd640 1 -- 172.21.15.119:0/1384823079 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45500113f0 con 0x7f455c06ebd0 2023-11-08T19:25:33.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.783+0000 7f454effd640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f454006ae70 0x7f454006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.783+0000 7f454effd640 1 -- 172.21.15.119:0/1384823079 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4550089c20 con 0x7f455c06ebd0 2023-11-08T19:25:33.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.783+0000 7f45621aa640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f454006ae70 0x7f454006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.784+0000 7f4566f9d740 1 Processor -- start 2023-11-08T19:25:33.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.784+0000 7f4566f9d740 1 -- start start 2023-11-08T19:25:33.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.784+0000 7f45621aa640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f454006ae70 0x7f454006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4558036000 tx=0x7f455803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbaec0 0x5650edd8ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddbdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 0x5650eddbe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f4566f9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650edd007e0 con 0x5650edd8e300 2023-11-08T19:25:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650edcfded0 con 0x5650eddbb2a0 2023-11-08T19:25:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650edcf97c0 con 0x5650eddbaec0 2023-11-08T19:25:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 0x5650eddbe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 0x5650eddbe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58132/0 (socket says 172.21.15.119:58132) 2023-11-08T19:25:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 -- 172.21.15.119:0/2988335956 learned_addr learned my addr 172.21.15.119:0/2988335956 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45621aa640 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbaec0 0x5650edd8ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 -- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbaec0 msgr2=0x5650edd8ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbaec0 0x5650edd8ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45619a9640 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddbdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 -- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 msgr2=0x5650eddbdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddbdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.785+0000 7f45629ab640 1 -- 172.21.15.119:0/2988335956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650edcccb10 con 0x5650edd8e300 2023-11-08T19:25:33.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.786+0000 7f45629ab640 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 0x5650eddbe190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4554007fd0 tx=0x7f455400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db24351cdfcd901 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.786+0000 7f453ffff640 1 -- 172.21.15.119:0/2988335956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400f6c0 con 0x5650edd8e300 2023-11-08T19:25:33.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.786+0000 7f453ffff640 1 -- 172.21.15.119:0/2988335956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554013070 con 0x5650edd8e300 2023-11-08T19:25:33.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.786+0000 7f453ffff640 1 -- 172.21.15.119:0/2988335956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400fa60 con 0x5650edd8e300 2023-11-08T19:25:33.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 -- 172.21.15.119:0/2988335956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 msgr2=0x5650eddbe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 0x5650eddbe190 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4554007fd0 tx=0x7f455400e820 comp rx=0 tx=0).stop 2023-11-08T19:25:33.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 -- 172.21.15.119:0/2988335956 shutdown_connections 2023-11-08T19:25:33.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650edd8e300 0x5650eddbe190 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddbdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 --2- 172.21.15.119:0/2988335956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbaec0 0x5650edd8ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 -- 172.21.15.119:0/2988335956 >> 172.21.15.119:0/2988335956 conn(0x5650edd900d0 msgr2=0x5650eddd8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 -- 172.21.15.119:0/2988335956 shutdown_connections 2023-11-08T19:25:33.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 -- 172.21.15.119:0/2988335956 wait complete. 2023-11-08T19:25:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.787+0000 7f4566f9d740 1 -- start start 2023-11-08T19:25:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.788+0000 7f4566f9d740 1 Processor -- start 2023-11-08T19:25:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 0x5650eddb8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddd1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 0x5650edde8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650edc96c40 con 0x5650eddbaec0 2023-11-08T19:25:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650edc968e0 con 0x5650eddbb2a0 2023-11-08T19:25:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650eda5cf20 con 0x5650eddd2100 2023-11-08T19:25:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f4566f9d740 10 client.0 _ll_register_callbacks cb 0x5650edcb8cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:25:33.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 0x5650eddb8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 0x5650eddb8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58138/0 (socket says 172.21.15.119:58138) 2023-11-08T19:25:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 -- 172.21.15.119:0/3850693246 learned_addr learned my addr 172.21.15.119:0/3850693246 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 -- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 msgr2=0x5650edde8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 0x5650edde8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 -- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 msgr2=0x5650eddd1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddd1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.789+0000 7f45621aa640 1 -- 172.21.15.119:0/3850693246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4554007bd0 con 0x5650eddbaec0 2023-11-08T19:25:33.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f45621aa640 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 0x5650eddb8ac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f455802be70 tx=0x7f455802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673fd8620430647b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f453d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:25:33.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f453d7fa640 1 -- 172.21.15.119:0/3850693246 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455804a040 con 0x5650eddbaec0 2023-11-08T19:25:33.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f453d7fa640 1 -- 172.21.15.119:0/3850693246 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4558040070 con 0x5650eddbaec0 2023-11-08T19:25:33.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650edccdc80 con 0x5650eddbaec0 2023-11-08T19:25:33.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f453d7fa640 1 -- 172.21.15.119:0/3850693246 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45580317a0 con 0x5650eddbaec0 2023-11-08T19:25:33.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f4566f9d740 10 client.4788 fetch_fsmap learned FSMap version 9 2023-11-08T19:25:33.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5650edcf7e40 con 0x5650eddbaec0 2023-11-08T19:25:33.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f453d7fa640 1 -- 172.21.15.119:0/3850693246 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5650edccdc80 con 0x5650eddbaec0 2023-11-08T19:25:33.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f453d7fa640 1 -- 172.21.15.119:0/3850693246 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f4558045a50 con 0x5650eddbaec0 2023-11-08T19:25:33.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f4566f9d740 10 client.4788 fetch_fsmap finished waiting for FSMap version 9 2023-11-08T19:25:33.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f4566f9d740 10 client.4788 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:25:33.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.790+0000 7f4566f9d740 20 client.4788 populate_metadata read hostname 'smithi119' 2023-11-08T19:25:33.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.791+0000 7f4566f9d740 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650edde9980 0x5650eddf6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.791+0000 7f4566f9d740 4 client.4788 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:25:33.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.791+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 --> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650edcf27a0 con 0x5650edde9980 2023-11-08T19:25:33.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.791+0000 7f45619a9640 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650edde9980 0x5650eddf6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.792+0000 7f453d7fa640 10 client.4788 ms_handle_connect on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:33.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.792+0000 7f45619a9640 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650edde9980 0x5650eddf6360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1ca2b0dd87e737 server_cookie=7cd28dea537dcd91 in_seq=0 out_seq=0 2023-11-08T19:25:33.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.794+0000 7f453d7fa640 1 -- 172.21.15.119:0/3850693246 <== mds.0 v2:172.21.15.188:6833/2629712656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650edcf27a0 con 0x5650edde9980 2023-11-08T19:25:33.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.794+0000 7f453d7fa640 10 client.4788 handle_command_reply: tid=0 2023-11-08T19:25:33.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.794+0000 7f4566f9d740 1 client.4788 shutdown 2023-11-08T19:25:33.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 msgr2=0x5650eddb8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 0x5650eddb8ac0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f455802be70 tx=0x7f455802e000 comp rx=0 tx=0).stop 2023-11-08T19:25:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 shutdown_connections 2023-11-08T19:25:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650edde9980 0x5650eddf6360 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 0x5650edde8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbb2a0 0x5650eddd1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 --2- 172.21.15.119:0/3850693246 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbaec0 0x5650eddb8ac0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f453d7fa640 0 client.4788 ms_handle_reset on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 >> 172.21.15.119:0/3850693246 conn(0x5650edd8e300 msgr2=0x5650edd91d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 shutdown_connections 2023-11-08T19:25:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 1 -- 172.21.15.119:0/3850693246 wait complete. 2023-11-08T19:25:33.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.795+0000 7f4566f9d740 20 client.4788 trim_cache size 0 max 16384 2023-11-08T19:25:33.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.812+0000 7f4566f9d740 1 Processor -- start 2023-11-08T19:25:33.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.812+0000 7f4566f9d740 1 -- start start 2023-11-08T19:25:33.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650edd91bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 0x5650edd92b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 0x5650edd930a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f4566f9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650edde8f50 con 0x5650eddbb2a0 2023-11-08T19:25:33.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650edddcb20 con 0x5650eddbaec0 2023-11-08T19:25:33.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650edd93e00 con 0x5650eddd2100 2023-11-08T19:25:33.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 0x5650edd92b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 0x5650edd92b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58142/0 (socket says 172.21.15.119:58142) 2023-11-08T19:25:33.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 -- 172.21.15.119:0/1660489696 learned_addr learned my addr 172.21.15.119:0/1660489696 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:33.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 -- 172.21.15.119:0/1660489696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 msgr2=0x5650edd930a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 0x5650edd930a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 -- 172.21.15.119:0/1660489696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 msgr2=0x5650edd91bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650edd91bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 -- 172.21.15.119:0/1660489696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650edc95b90 con 0x5650eddbb2a0 2023-11-08T19:25:33.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.813+0000 7f45619a9640 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 0x5650edd92b60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4550060460 tx=0x7f455000a450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd821d91fcc1a57e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f454cdff640 1 -- 172.21.15.119:0/1660489696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550087ed0 con 0x5650eddbb2a0 2023-11-08T19:25:33.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f454cdff640 1 -- 172.21.15.119:0/1660489696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550056a30 con 0x5650eddbb2a0 2023-11-08T19:25:33.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f454cdff640 1 -- 172.21.15.119:0/1660489696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4550060500 con 0x5650eddbb2a0 2023-11-08T19:25:33.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 -- 172.21.15.119:0/1660489696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 msgr2=0x5650edd92b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 0x5650edd92b60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4550060460 tx=0x7f455000a450 comp rx=0 tx=0).stop 2023-11-08T19:25:33.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 -- 172.21.15.119:0/1660489696 shutdown_connections 2023-11-08T19:25:33.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddd2100 0x5650edd930a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddbb2a0 0x5650edd92b60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 --2- 172.21.15.119:0/1660489696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650edd91bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 -- 172.21.15.119:0/1660489696 >> 172.21.15.119:0/1660489696 conn(0x5650edd8e300 msgr2=0x5650eddd9370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 -- 172.21.15.119:0/1660489696 shutdown_connections 2023-11-08T19:25:33.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.814+0000 7f4566f9d740 1 -- 172.21.15.119:0/1660489696 wait complete. 2023-11-08T19:25:33.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.815+0000 7f4566f9d740 1 -- start start 2023-11-08T19:25:33.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.815+0000 7f4566f9d740 1 Processor -- start 2023-11-08T19:25:33.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650eddce5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbb2a0 0x5650eddceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 0x5650eddf1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650ede43db0 con 0x5650eddd2100 2023-11-08T19:25:33.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650eddcf480 con 0x5650eddbaec0 2023-11-08T19:25:33.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650eddd1e50 con 0x5650eddbb2a0 2023-11-08T19:25:33.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f4566f9d740 10 client.0 _ll_register_callbacks cb 0x7f45580096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:25:33.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.816+0000 7f45629ab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 0x5650eddf1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 0x5650eddf1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58152/0 (socket says 172.21.15.119:58152) 2023-11-08T19:25:33.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 -- 172.21.15.119:0/411371012 learned_addr learned my addr 172.21.15.119:0/411371012 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:33.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45619a9640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbb2a0 0x5650eddceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45621aa640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650eddce5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 -- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbb2a0 msgr2=0x5650eddceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbb2a0 0x5650eddceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 -- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 msgr2=0x5650eddce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:33.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650eddce5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 -- 172.21.15.119:0/411371012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c0ad0f0 con 0x5650eddd2100 2023-11-08T19:25:33.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45619a9640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbb2a0 0x5650eddceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:33.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f45629ab640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 0x5650eddf1fd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4554015000 tx=0x7f4554007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d123da654d2278db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:33.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f453d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:25:33.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f453d7fa640 1 -- 172.21.15.119:0/411371012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554003c60 con 0x5650eddd2100 2023-11-08T19:25:33.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.817+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4554013070 con 0x5650eddd2100 2023-11-08T19:25:33.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f453d7fa640 1 -- 172.21.15.119:0/411371012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f4554003dc0 con 0x5650eddd2100 2023-11-08T19:25:33.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f453d7fa640 1 -- 172.21.15.119:0/411371012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4554004040 con 0x5650eddd2100 2023-11-08T19:25:33.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 10 client.4794 fetch_fsmap learned FSMap version 9 2023-11-08T19:25:33.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x5650edcbb180 con 0x5650eddd2100 2023-11-08T19:25:33.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f453d7fa640 1 -- 172.21.15.119:0/411371012 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554013070 con 0x5650eddd2100 2023-11-08T19:25:33.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f453d7fa640 1 -- 172.21.15.119:0/411371012 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f45540041a0 con 0x5650eddd2100 2023-11-08T19:25:33.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 10 client.4794 fetch_fsmap finished waiting for FSMap version 9 2023-11-08T19:25:33.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 10 client.4794 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:25:33.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 20 client.4794 populate_metadata read hostname 'smithi119' 2023-11-08T19:25:33.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650eddf25d0 0x5650eddf29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:33.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 4 client.4794 mds_command: new command op to 4254 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:25:33.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.818+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 --> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5650edcf24c0 con 0x5650eddf25d0 2023-11-08T19:25:33.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.819+0000 7f45621aa640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650eddf25d0 0x5650eddf29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:33.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.819+0000 7f453d7fa640 10 client.4794 ms_handle_connect on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:33.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.819+0000 7f45621aa640 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650eddf25d0 0x5650eddf29b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=985d7e2be299648d server_cookie=873dd44a3b8391a8 in_seq=0 out_seq=0 2023-11-08T19:25:33.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f453d7fa640 1 -- 172.21.15.119:0/411371012 <== mds.0 v2:172.21.15.188:6833/2629712656 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x5650edcf24c0 con 0x5650eddf25d0 2023-11-08T19:25:33.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f453d7fa640 10 client.4794 handle_command_reply: tid=0 2023-11-08T19:25:33.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 client.4794 shutdown 2023-11-08T19:25:33.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 msgr2=0x5650eddf1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 0x5650eddf1fd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4554015000 tx=0x7f4554007900 comp rx=0 tx=0).stop 2023-11-08T19:25:33.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 shutdown_connections 2023-11-08T19:25:33.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:6833/2629712656,v1:172.21.15.188:6835/2629712656] conn(0x5650eddf25d0 0x5650eddf29b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5650eddd2100 0x5650eddf1fd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5650eddbb2a0 0x5650eddceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f453d7fa640 0 client.4794 ms_handle_reset on v2:172.21.15.188:6833/2629712656 2023-11-08T19:25:33.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 --2- 172.21.15.119:0/411371012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5650eddbaec0 0x5650eddce5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 >> 172.21.15.119:0/411371012 conn(0x5650edd8e300 msgr2=0x5650eddfaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 shutdown_connections 2023-11-08T19:25:33.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.820+0000 7f4566f9d740 1 -- 172.21.15.119:0/411371012 wait complete. 2023-11-08T19:25:33.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f4566f9d740 20 client.4794 trim_cache size 0 max 16384 2023-11-08T19:25:33.844 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:25:33.844 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:25:33.844 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:25:33.845 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 4254, 2023-11-08T19:25:33.845 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:25:33.845 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:25:33.845 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:25:33.845 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 35.177522101000001, 2023-11-08T19:25:33.846 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 9, 2023-11-08T19:25:33.846 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 25, 2023-11-08T19:25:33.846 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 25, 2023-11-08T19:25:33.846 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 40.67064251 2023-11-08T19:25:33.846 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:25:33.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f454006ae70 msgr2=0x7f454006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f454006ae70 0x7f454006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4558036000 tx=0x7f455803d040 comp rx=0 tx=0).stop 2023-11-08T19:25:33.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 msgr2=0x7f455c157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:33.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c157bc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f455000c7f0 tx=0x7f455000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 shutdown_connections 2023-11-08T19:25:33.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f454006ae70 0x7f454006d330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f455c06efb0 0x7f455c1612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f455c06ebd0 0x7f455c157bc0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 --2- 172.21.15.119:0/1384823079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f455c06a500 0x7f455c0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:33.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 >> 172.21.15.119:0/1384823079 conn(0x7f455c0cfd20 msgr2=0x7f455c072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:33.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 shutdown_connections 2023-11-08T19:25:33.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:33.821+0000 7f456440e640 1 -- 172.21.15.119:0/1384823079 wait complete. 2023-11-08T19:25:33.850 INFO:teuthology.run_tasks:Running task kclient... 2023-11-08T19:25:33.863 INFO:tasks.kclient:Mounting kernel clients... 2023-11-08T19:25:33.864 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-08T19:25:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:33.864 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:25:33.915 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-08T19:25:33.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:33.917 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-08T19:25:34.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-11-08T19:25:34.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.143 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:25:34.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-11-08T19:25:34.274 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:34.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:34.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.277 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-08T19:25:34.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.340 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-08T19:25:34.439 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:34.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:34.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.443 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-08T19:25:34.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:34.497 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-08T19:25:34.593 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2023-11-08T19:25:34.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:25:34.597 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-08T19:25:34.597 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:34.598 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:34.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:34.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.823+0000 7ff564b2c640 1 Processor -- start 2023-11-08T19:25:34.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.823+0000 7ff564b2c640 1 -- start start 2023-11-08T19:25:34.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff564b2c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff560142f80 0x7ff5600cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff564b2c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff56006cb40 0x7ff56006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff564b2c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56006d4d0 con 0x7ff56015d9f0 2023-11-08T19:25:34.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff564b2c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff560143a40 con 0x7ff56006cb40 2023-11-08T19:25:34.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff564b2c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5600ca3e0 con 0x7ff560142f80 2023-11-08T19:25:34.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff55ed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff56006cb40 0x7ff56006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff55ed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff56006cb40 0x7ff56006cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58162/0 (socket says 172.21.15.119:58162) 2023-11-08T19:25:34.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff55ed9d640 1 -- 172.21.15.119:0/905375280 learned_addr learned my addr 172.21.15.119:0/905375280 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:34.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.825+0000 7ff55e59c640 1 --2- 172.21.15.119:0/905375280 >> v2:172.21.15.188:3300/0 conn(0x7ff560142f80 0x7ff5600cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.826+0000 7ff55ed9d640 1 -- 172.21.15.119:0/905375280 >> v2:172.21.15.188:3300/0 conn(0x7ff560142f80 msgr2=0x7ff5600cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:34.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.826+0000 7ff55ed9d640 1 --2- 172.21.15.119:0/905375280 >> v2:172.21.15.188:3300/0 conn(0x7ff560142f80 0x7ff5600cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.826+0000 7ff55ed9d640 1 -- 172.21.15.119:0/905375280 >> v1:172.21.15.188:6789/0 conn(0x7ff56015d9f0 legacy=0x7ff5600c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:34.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.826+0000 7ff55ed9d640 1 -- 172.21.15.119:0/905375280 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5600c4960 con 0x7ff56006cb40 2023-11-08T19:25:34.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.827+0000 7ff55ed9d640 1 --2- 172.21.15.119:0/905375280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56006cb40 0x7ff56006cf90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff5600c3b00 tx=0x7ff55802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db629196db0a00f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.827+0000 7ff55dd9b640 1 -- 172.21.15.119:0/905375280 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55802ca80 con 0x7ff56006cb40 2023-11-08T19:25:34.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.827+0000 7ff55dd9b640 1 -- 172.21.15.119:0/905375280 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55802cbe0 con 0x7ff56006cb40 2023-11-08T19:25:34.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.827+0000 7ff55dd9b640 1 -- 172.21.15.119:0/905375280 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5580343f0 con 0x7ff56006cb40 2023-11-08T19:25:34.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 -- 172.21.15.119:0/905375280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56006cb40 msgr2=0x7ff56006cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 --2- 172.21.15.119:0/905375280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56006cb40 0x7ff56006cf90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff5600c3b00 tx=0x7ff55802bee0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 -- 172.21.15.119:0/905375280 shutdown_connections 2023-11-08T19:25:34.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 --2- 172.21.15.119:0/905375280 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56006cb40 0x7ff56006cf90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 --2- 172.21.15.119:0/905375280 >> v2:172.21.15.188:3300/0 conn(0x7ff560142f80 0x7ff5600cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 -- 172.21.15.119:0/905375280 >> 172.21.15.119:0/905375280 conn(0x7ff5600cb610 msgr2=0x7ff5600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:34.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 -- 172.21.15.119:0/905375280 shutdown_connections 2023-11-08T19:25:34.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.828+0000 7ff564b2c640 1 -- 172.21.15.119:0/905375280 wait complete. 2023-11-08T19:25:34.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.830+0000 7ff564b2c640 1 Processor -- start 2023-11-08T19:25:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.830+0000 7ff564b2c640 1 -- start start 2023-11-08T19:25:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff564b2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56006cb40 0x7ff56015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff564b2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff560142f80 0x7ff5600b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff564b2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 0x7ff5600c7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff564b2c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5600c6530 con 0x7ff56015d9f0 2023-11-08T19:25:34.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff564b2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56015bbd0 con 0x7ff56006cb40 2023-11-08T19:25:34.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff564b2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56015beb0 con 0x7ff560142f80 2023-11-08T19:25:34.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 0x7ff5600c7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 0x7ff5600c7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58178/0 (socket says 172.21.15.119:58178) 2023-11-08T19:25:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 -- 172.21.15.119:0/1375694731 learned_addr learned my addr 172.21.15.119:0/1375694731 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55e59c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56006cb40 0x7ff56015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 -- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff560142f80 msgr2=0x7ff5600b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff560142f80 0x7ff5600b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 -- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56006cb40 msgr2=0x7ff56015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56006cb40 0x7ff56015b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.831+0000 7ff55ed9d640 1 -- 172.21.15.119:0/1375694731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff558009b30 con 0x7ff56015d9f0 2023-11-08T19:25:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.832+0000 7ff55ed9d640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 0x7ff5600c7940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff558036040 tx=0x7ff558004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.832+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558009510 con 0x7ff56015d9f0 2023-11-08T19:25:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.832+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55803e450 con 0x7ff56015d9f0 2023-11-08T19:25:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.832+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55803e7e0 con 0x7ff56015d9f0 2023-11-08T19:25:34.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.832+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56015c190 con 0x7ff56015d9f0 2023-11-08T19:25:34.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.832+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56015c750 con 0x7ff56015d9f0 2023-11-08T19:25:34.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.833+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff560074430 con 0x7ff56015d9f0 2023-11-08T19:25:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.834+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff55802cd50 con 0x7ff56015d9f0 2023-11-08T19:25:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.835+0000 7ff556ffd640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff54006ae70 0x7ff54006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.835+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7ff5580ac500 con 0x7ff56015d9f0 2023-11-08T19:25:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.835+0000 7ff55e59c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff54006ae70 0x7ff54006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:34.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.836+0000 7ff55e59c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff54006ae70 0x7ff54006d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff550006fd0 tx=0x7ff550008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:34.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:34.840+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff558077220 con 0x7ff56015d9f0 2023-11-08T19:25:35.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.068+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff56015e7f0 con 0x7ff56015d9f0 2023-11-08T19:25:35.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.069+0000 7ff556ffd640 1 -- 172.21.15.119:0/1375694731 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7ff55803e940 con 0x7ff56015d9f0 2023-11-08T19:25:35.070 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T19:25:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.071+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff54006ae70 msgr2=0x7ff54006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.071+0000 7ff564b2c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff54006ae70 0x7ff54006d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff550006fd0 tx=0x7ff550008040 comp rx=0 tx=0).stop 2023-11-08T19:25:35.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.071+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 msgr2=0x7ff5600c7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.071+0000 7ff564b2c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 0x7ff5600c7940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff558036040 tx=0x7ff558004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 shutdown_connections 2023-11-08T19:25:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff54006ae70 0x7ff54006d330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff56015d9f0 0x7ff5600c7940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff560142f80 0x7ff5600b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 --2- 172.21.15.119:0/1375694731 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff56006cb40 0x7ff56015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 >> 172.21.15.119:0/1375694731 conn(0x7ff5600cb610 msgr2=0x7ff56006b930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 shutdown_connections 2023-11-08T19:25:35.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:35.072+0000 7ff564b2c640 1 -- 172.21.15.119:0/1375694731 wait complete. 2023-11-08T19:25:35.091 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:35.091 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:35.091 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:25:35.092 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:25:35.092 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:35.092 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:35.092 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:35.093 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:35.093 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:25:35.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.094 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:25:35.148 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:35.148 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:35.148 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:35.148 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:35.149 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:35.149 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:35.149 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:35.149 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:35.149 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:25:35.150 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:35.150 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:35.150 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:25:35.150 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:35.150 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:35.151 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:35.151 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:35.151 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:35.151 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42335sec preferred_lft 42335sec 2023-11-08T19:25:35.151 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:25:35.152 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:35.152 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T19:25:35.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.153 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:35.153 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:35.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add name ceph-brx type bridge 2023-11-08T19:25:35.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr flush dev ceph-brx 2023-11-08T19:25:35.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set ceph-brx up 2023-11-08T19:25:35.154 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T19:25:35.155 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:35.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.379 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:35.456 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:25:35.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.460 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:25:35.513 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:25:35.514 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:35.514 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:35.514 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:35.514 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:35.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.515 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:35.516 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:35.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:35.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:35.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:35.517 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:35.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.701 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:25:35.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.755 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-11-08T19:25:35.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.809 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:35.809 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:35.809 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:25:35.810 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-08T19:25:35.810 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:35.950 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-08T19:25:35.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:35.951 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:35.952 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:35.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-08T19:25:35.952 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:35.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-08T19:25:35.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-08T19:25:35.953 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-08T19:25:35.954 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:36.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:36.205 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:36.205 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:36.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:25:36.206 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T19:25:36.206 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:36.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:36.366 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:25:36.414 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:36.414 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:36.513 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T19:25:36.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:36.514 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:36.657 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:36.657 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:37.128 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:25:37.129 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:25:37.131 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:25:37.132 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:25:37.136 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:37.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:37.137 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:25:37.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:37.221 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:37.276 INFO:teuthology.orchestra.run.smithi119.stdout:client4552 2023-11-08T19:25:37.277 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:37.277 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:37.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:37.324 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:37.429 INFO:teuthology.orchestra.run.smithi119.stdout:client4552 2023-11-08T19:25:37.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:37.430 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:37.489 INFO:teuthology.orchestra.run.smithi119.stdout:client4552 2023-11-08T19:25:37.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:37.490 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.client4552/status') 2023-11-08T19:25:37.587 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.4552 (3)172.21.15.119:0/2449622891 2023-11-08T19:25:37.588 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2023-11-08T19:25:37.588 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2023-11-08T19:25:37.588 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2023-11-08T19:25:37.588 INFO:teuthology.orchestra.run.smithi119.stderr:71 bytes copied, 6.6466e-05 s, 1.1 MB/s 2023-11-08T19:25:37.591 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-08T19:25:37.591 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:37.592 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:37.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:37.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.791+0000 7f9a5a864640 1 Processor -- start 2023-11-08T19:25:37.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.791+0000 7f9a5a864640 1 -- start start 2023-11-08T19:25:37.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a5a864640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 0x7f9a540cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a5a864640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a540c4960 con 0x7f9a5415d9f0 2023-11-08T19:25:37.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a5a864640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5415df60 con 0x7f9a5406cb40 2023-11-08T19:25:37.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a5a864640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54143a40 con 0x7f9a54142f80 2023-11-08T19:25:37.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a4bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9a5415d9f0 0x7f9a540c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:59860/0 (socket says 172.21.15.119:59860) 2023-11-08T19:25:37.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a4bfff640 1 -- 172.21.15.119:0/3858021808 learned_addr learned my addr 172.21.15.119:0/3858021808 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:37.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.793+0000 7f9a53fff640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 0x7f9a540cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255833976 0 0) 0x7f9a540c4960 con 0x7f9a5415d9f0 2023-11-08T19:25:37.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a40003520 con 0x7f9a5415d9f0 2023-11-08T19:25:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3210585383 0 0) 0x7f9a40003520 con 0x7f9a5415d9f0 2023-11-08T19:25:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 msgr2=0x7f9a540cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 0x7f9a540cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 >> v1:172.21.15.188:6789/0 conn(0x7f9a5406cb40 legacy=0x7f9a540c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5415e240 con 0x7f9a5415d9f0 2023-11-08T19:25:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f9a3c001670 con 0x7f9a5415d9f0 2023-11-08T19:25:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 >> v1:172.21.15.119:6789/0 conn(0x7f9a5415d9f0 legacy=0x7f9a540c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a400039a0 0x7f9a40003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 0x7f9a400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a40004d00 0x7f9a40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54143a40 con 0x7f9a400039a0 2023-11-08T19:25:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4000bb10 con 0x7f9a40004d00 2023-11-08T19:25:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4000dee0 con 0x7f9a54142f80 2023-11-08T19:25:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.794+0000 7f9a53fff640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a400039a0 0x7f9a40003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.795+0000 7f9a4bfff640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 0x7f9a400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.795+0000 7f9a53fff640 1 -- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 msgr2=0x7f9a400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.795+0000 7f9a53fff640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a54142f80 0x7f9a400047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.795+0000 7f9a53fff640 1 -- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a40004d00 msgr2=0x7f9a40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.795+0000 7f9a53fff640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a40004d00 0x7f9a40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.795+0000 7f9a53fff640 1 -- 172.21.15.119:0/3858021808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40003520 con 0x7f9a400039a0 2023-11-08T19:25:37.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.796+0000 7f9a53fff640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a400039a0 0x7f9a40003dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ac0 tx=0x7f9a400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cafe2cecc24e44 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.796+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4402c6b0 con 0x7f9a400039a0 2023-11-08T19:25:37.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.796+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4402c810 con 0x7f9a400039a0 2023-11-08T19:25:37.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.796+0000 7f9a537fe640 1 -- 172.21.15.119:0/3858021808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4402cba0 con 0x7f9a400039a0 2023-11-08T19:25:37.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a5a864640 1 -- 172.21.15.119:0/3858021808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a400039a0 msgr2=0x7f9a40003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a5a864640 1 --2- 172.21.15.119:0/3858021808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a400039a0 0x7f9a40003dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a44009ac0 tx=0x7f9a400057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a58e01640 1 -- 172.21.15.119:0/3858021808 reap_dead start 2023-11-08T19:25:37.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a5a864640 1 -- 172.21.15.119:0/3858021808 shutdown_connections 2023-11-08T19:25:37.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a5a864640 1 -- 172.21.15.119:0/3858021808 >> 172.21.15.119:0/3858021808 conn(0x7f9a540cb610 msgr2=0x7f9a540cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:37.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a5a864640 1 -- 172.21.15.119:0/3858021808 shutdown_connections 2023-11-08T19:25:37.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.797+0000 7f9a5a864640 1 -- 172.21.15.119:0/3858021808 wait complete. 2023-11-08T19:25:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.799+0000 7f9a5a864640 1 Processor -- start 2023-11-08T19:25:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.799+0000 7f9a5a864640 1 -- start start 2023-11-08T19:25:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a5a864640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a400039a0 0x7f9a5415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a5a864640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a540b1880 0x7f9a54139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a5a864640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 0x7f9a540b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a5a864640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54139dc0 con 0x7f9a5415ad40 2023-11-08T19:25:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a5a864640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5415b520 con 0x7f9a400039a0 2023-11-08T19:25:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a5a864640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a54156990 con 0x7f9a540b1880 2023-11-08T19:25:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a58e01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 0x7f9a540b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a58e01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 0x7f9a540b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58188/0 (socket says 172.21.15.119:58188) 2023-11-08T19:25:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a58e01640 1 -- 172.21.15.119:0/2677661574 learned_addr learned my addr 172.21.15.119:0/2677661574 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a4bfff640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a540b1880 0x7f9a54139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.800+0000 7f9a53fff640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a400039a0 0x7f9a5415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a58e01640 1 -- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a540b1880 msgr2=0x7f9a54139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a58e01640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a540b1880 0x7f9a54139880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a58e01640 1 -- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a400039a0 msgr2=0x7f9a5415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a58e01640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a400039a0 0x7f9a5415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a58e01640 1 -- 172.21.15.119:0/2677661574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a54156c70 con 0x7f9a5415ad40 2023-11-08T19:25:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a53fff640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a400039a0 0x7f9a5415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a58e01640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 0x7f9a540b07c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9a540c9080 tx=0x7f9a4c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c005d40 con 0x7f9a5415ad40 2023-11-08T19:25:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c00ced0 con 0x7f9a5415ad40 2023-11-08T19:25:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c00a850 con 0x7f9a5415ad40 2023-11-08T19:25:37.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a54156f50 con 0x7f9a5415ad40 2023-11-08T19:25:37.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.801+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5415b1d0 con 0x7f9a5415ad40 2023-11-08T19:25:37.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.803+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a54157140 con 0x7f9a5415ad40 2023-11-08T19:25:37.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.804+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a4c00a9b0 con 0x7f9a5415ad40 2023-11-08T19:25:37.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.804+0000 7f9a517fa640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a3406ae70 0x7f9a3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:37.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.805+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6460+0+0 (secure 0 0 0) 0x7f9a4c088840 con 0x7f9a5415ad40 2023-11-08T19:25:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.805+0000 7f9a53fff640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a3406ae70 0x7f9a3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.805+0000 7f9a53fff640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a3406ae70 0x7f9a3406d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9a44036000 tx=0x7f9a44030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:37.809+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a4c053560 con 0x7f9a5415ad40 2023-11-08T19:25:38.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.039+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9a54074430 con 0x7f9a5415ad40 2023-11-08T19:25:38.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.040+0000 7f9a517fa640 1 -- 172.21.15.119:0/2677661574 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f9a4c056b80 con 0x7f9a5415ad40 2023-11-08T19:25:38.042 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T19:25:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a3406ae70 msgr2=0x7f9a3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a3406ae70 0x7f9a3406d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9a44036000 tx=0x7f9a44030000 comp rx=0 tx=0).stop 2023-11-08T19:25:38.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 msgr2=0x7f9a540b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:38.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 0x7f9a540b07c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9a540c9080 tx=0x7f9a4c00cb90 comp rx=0 tx=0).stop 2023-11-08T19:25:38.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 shutdown_connections 2023-11-08T19:25:38.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9a3406ae70 0x7f9a3406d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9a5415ad40 0x7f9a540b07c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a540b1880 0x7f9a54139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 --2- 172.21.15.119:0/2677661574 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a400039a0 0x7f9a5415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.042+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 >> 172.21.15.119:0/2677661574 conn(0x7f9a540cb610 msgr2=0x7f9a540c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.043+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 shutdown_connections 2023-11-08T19:25:38.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:38.043+0000 7f9a5a864640 1 -- 172.21.15.119:0/2677661574 wait complete. 2023-11-08T19:25:38.063 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:38.063 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:38.064 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:25:38.064 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-08T19:25:38.064 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:38.064 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:38.065 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:38.065 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:38.065 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-08T19:25:38.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.066 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:25:38.114 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:38.114 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:38.114 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:38.115 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.115 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:38.115 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.115 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:38.115 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:38.116 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:25:38.116 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:38.116 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:38.116 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:25:38.116 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:38.117 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:38.117 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:38.117 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:38.117 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:38.117 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42332sec preferred_lft 42332sec 2023-11-08T19:25:38.118 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:25:38.118 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.118 INFO:teuthology.orchestra.run.smithi119.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:38.118 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:38.119 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:25:38.119 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.119 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::e06c:64ff:fe55:3f8b/64 scope link 2023-11-08T19:25:38.119 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.119 INFO:teuthology.orchestra.run.smithi119.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:25:38.119 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:25:38.120 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c086:39ff:fe1e:9142/64 scope link tentative 2023-11-08T19:25:38.120 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.121 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:38.230 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:25:38.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.234 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:25:38.274 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:25:38.275 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:38.275 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:38.275 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:38.275 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:38.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.276 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:38.277 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:38.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:38.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:38.278 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:38.278 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:38.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.481 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:25:38.525 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:25:38.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.526 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list-id 2023-11-08T19:25:38.617 INFO:teuthology.orchestra.run.smithi119.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-08T19:25:38.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.619 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:38.619 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:38.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:25:38.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-08T19:25:38.620 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:38.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.746 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T19:25:38.820 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:38.820 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.821 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c6:ef:8f:38:bf:1e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:38.821 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c4ef:8fff:fe38:bf1e/64 scope link 2023-11-08T19:25:38.822 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T19:25:38.905 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:38.905 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.906 INFO:teuthology.orchestra.run.smithi119.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:38.906 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c6:ef:8f:38:bf:1e brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c4ef:8fff:fe38:bf1e/64 scope link 2023-11-08T19:25:38.907 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:38.909 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-08T19:25:38.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:38.910 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:38.910 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:38.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-08T19:25:38.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:38.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-08T19:25:38.911 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-08T19:25:38.912 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-08T19:25:38.912 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:39.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:39.161 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:25:39.161 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:25:39.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.1 up 2023-11-08T19:25:39.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T19:25:39.162 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:25:39.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:39.329 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:39.375 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-08T19:25:39.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:39.376 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:39.471 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-08T19:25:39.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:39.471 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:39.613 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:39.613 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:39.650 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:39.650 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:39.651 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:25:39.651 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 2@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:25:39.651 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:25:39.652 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-11-08T19:25:39.653 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:25:39.653 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:25:39.653 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:25:39.653 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:25:39.659 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:39.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:39.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-08T19:25:39.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:39.735 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:39.790 INFO:teuthology.orchestra.run.smithi119.stdout:client4418 2023-11-08T19:25:39.791 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:39.791 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:39.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:39.840 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:39.948 INFO:teuthology.orchestra.run.smithi119.stdout:client4418 2023-11-08T19:25:39.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:39.949 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:40.018 INFO:teuthology.orchestra.run.smithi119.stdout:client4418 2023-11-08T19:25:40.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:40.019 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.client4418/status') 2023-11-08T19:25:40.116 INFO:teuthology.orchestra.run.smithi119.stdout:instance: client.4418 (3)172.21.15.119:0/1769430810 2023-11-08T19:25:40.116 INFO:teuthology.orchestra.run.smithi119.stdout:blocklisted: false 2023-11-08T19:25:40.116 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records in 2023-11-08T19:25:40.117 INFO:teuthology.orchestra.run.smithi119.stderr:0+1 records out 2023-11-08T19:25:40.117 INFO:teuthology.orchestra.run.smithi119.stderr:71 bytes copied, 6.6918e-05 s, 1.1 MB/s 2023-11-08T19:25:40.119 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-08T19:25:40.120 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:40.120 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:40.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:40.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.338+0000 7fd1e18d4640 1 Processor -- start 2023-11-08T19:25:40.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.338+0000 7fd1e18d4640 1 -- start start 2023-11-08T19:25:40.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1e18d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1dc15d9f0 0x7fd1dc0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1e18d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 0x7fd1dc06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1e18d4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc0c4960 con 0x7fd1dc142f80 2023-11-08T19:25:40.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1e18d4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc143a40 con 0x7fd1dc15d9f0 2023-11-08T19:25:40.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1e18d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0ca3e0 con 0x7fd1dc06cb40 2023-11-08T19:25:40.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1daffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd1dc142f80 0x7fd1dc0c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51668/0 (socket says 172.21.15.119:51668) 2023-11-08T19:25:40.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1daffd640 1 -- 172.21.15.119:0/2172180893 learned_addr learned my addr 172.21.15.119:0/2172180893 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:40.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1db7fe640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 0x7fd1dc06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.340+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/2172180893 >> v2:172.21.15.188:3300/0 conn(0x7fd1dc15d9f0 0x7fd1dc0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505066163 0 0) 0x7fd1dc0c4960 con 0x7fd1dc142f80 2023-11-08T19:25:40.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1c4003520 con 0x7fd1dc142f80 2023-11-08T19:25:40.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3475956053 0 0) 0x7fd1c4003520 con 0x7fd1dc142f80 2023-11-08T19:25:40.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 msgr2=0x7fd1dc06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:40.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 0x7fd1dc06cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 >> v2:172.21.15.188:3300/0 conn(0x7fd1dc15d9f0 msgr2=0x7fd1dc0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:40.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 --2- 172.21.15.119:0/2172180893 >> v2:172.21.15.188:3300/0 conn(0x7fd1dc15d9f0 0x7fd1dc0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc15df60 con 0x7fd1dc142f80 2023-11-08T19:25:40.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd1cc001670 con 0x7fd1dc142f80 2023-11-08T19:25:40.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 >> v1:172.21.15.119:6789/0 conn(0x7fd1dc142f80 legacy=0x7fd1dc0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 0x7fd1c4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1c4003f40 0x7fd1c40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4004d00 0x7fd1c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc143a40 con 0x7fd1c4003f40 2023-11-08T19:25:40.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0ca3e0 con 0x7fd1c4004d00 2023-11-08T19:25:40.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c400b980 con 0x7fd1dc06cb40 2023-11-08T19:25:40.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.341+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1c4003f40 0x7fd1c40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1daffd640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 0x7fd1c4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1db7fe640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4004d00 0x7fd1c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1da7fc640 1 -- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 msgr2=0x7fd1c4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:40.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06cb40 0x7fd1c4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1da7fc640 1 -- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4004d00 msgr2=0x7fd1c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:40.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4004d00 0x7fd1c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.342+0000 7fd1da7fc640 1 -- 172.21.15.119:0/2172180893 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4003520 con 0x7fd1c4003f40 2023-11-08T19:25:40.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.343+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1c4003f40 0x7fd1c40047c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009af0 tx=0x7fd1c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=128a524d241985ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:40.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.343+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d002c710 con 0x7fd1c4003f40 2023-11-08T19:25:40.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.343+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d002c870 con 0x7fd1c4003f40 2023-11-08T19:25:40.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.343+0000 7fd1d9ffb640 1 -- 172.21.15.119:0/2172180893 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d002cc00 con 0x7fd1c4003f40 2023-11-08T19:25:40.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1e18d4640 1 -- 172.21.15.119:0/2172180893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1c4003f40 msgr2=0x7fd1c40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/2172180893 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1c4003f40 0x7fd1c40047c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd1d0009af0 tx=0x7fd1c40057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1db7fe640 1 -- 172.21.15.119:0/2172180893 reap_dead start 2023-11-08T19:25:40.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1e18d4640 1 -- 172.21.15.119:0/2172180893 shutdown_connections 2023-11-08T19:25:40.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1e18d4640 1 -- 172.21.15.119:0/2172180893 >> 172.21.15.119:0/2172180893 conn(0x7fd1dc0cb610 msgr2=0x7fd1dc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:40.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1e18d4640 1 -- 172.21.15.119:0/2172180893 shutdown_connections 2023-11-08T19:25:40.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.344+0000 7fd1e18d4640 1 -- 172.21.15.119:0/2172180893 wait complete. 2023-11-08T19:25:40.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.346+0000 7fd1e18d4640 1 Processor -- start 2023-11-08T19:25:40.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.346+0000 7fd1e18d4640 1 -- start start 2023-11-08T19:25:40.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1e18d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4003f40 0x7fd1dc159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1e18d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06fdc0 0x7fd1dc070240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1e18d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 0x7fd1dc0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1e18d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc070780 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1e18d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1f2f10 con 0x7fd1c4003f40 2023-11-08T19:25:40.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1e18d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc1f31b0 con 0x7fd1dc06fdc0 2023-11-08T19:25:40.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1db7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 0x7fd1dc0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1db7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 0x7fd1dc0c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53910/0 (socket says 172.21.15.119:53910) 2023-11-08T19:25:40.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.347+0000 7fd1db7fe640 1 -- 172.21.15.119:0/4220034669 learned_addr learned my addr 172.21.15.119:0/4220034669 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:40.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06fdc0 0x7fd1dc070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1daffd640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4003f40 0x7fd1dc159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1db7fe640 1 -- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06fdc0 msgr2=0x7fd1dc070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1db7fe640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06fdc0 0x7fd1dc070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1db7fe640 1 -- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4003f40 msgr2=0x7fd1dc159530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1db7fe640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4003f40 0x7fd1dc159530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1db7fe640 1 -- 172.21.15.119:0/4220034669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc1f3490 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1da7fc640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06fdc0 0x7fd1dc070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:40.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1db7fe640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 0x7fd1dc0c6050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc15ddd0 tx=0x7fd1d40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:40.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400e070 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d4011e10 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc1f3770 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.348+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4003990 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.349+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc1f3c90 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.350+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc074430 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.351+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1d4003af0 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.351+0000 7fd1c3fff640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1c806ae90 0x7fd1c806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.352+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd1d4089f60 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.352+0000 7fd1daffd640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1c806ae90 0x7fd1c806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.352+0000 7fd1daffd640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1c806ae90 0x7fd1c806d350 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000fb0 tx=0x7fd1cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.356+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1d4054c30 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.583+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd1dc0b4c70 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.584+0000 7fd1c3fff640 1 -- 172.21.15.119:0/4220034669 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd1d4058250 con 0x7fd1dc0c5bd0 2023-11-08T19:25:40.586 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T19:25:40.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1c806ae90 msgr2=0x7fd1c806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1c806ae90 0x7fd1c806d350 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc000fb0 tx=0x7fd1cc009040 comp rx=0 tx=0).stop 2023-11-08T19:25:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 msgr2=0x7fd1dc0c6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 0x7fd1dc0c6050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc15ddd0 tx=0x7fd1d40115a0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 shutdown_connections 2023-11-08T19:25:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1c806ae90 0x7fd1c806d350 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1dc0c5bd0 0x7fd1dc0c6050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1dc06fdc0 0x7fd1dc070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 --2- 172.21.15.119:0/4220034669 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1c4003f40 0x7fd1dc159530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 >> 172.21.15.119:0/4220034669 conn(0x7fd1dc0cb610 msgr2=0x7fd1dc0ca520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:40.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 shutdown_connections 2023-11-08T19:25:40.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:40.587+0000 7fd1e18d4640 1 -- 172.21.15.119:0/4220034669 wait complete. 2023-11-08T19:25:40.606 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:40.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:40.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-08T19:25:40.607 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-08T19:25:40.607 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:40.608 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:40.608 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:40.608 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:40.608 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-08T19:25:40.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:40.609 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-08T19:25:40.667 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:40.668 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:40.668 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:40.668 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:40.668 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:40.669 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:40.669 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:40.669 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:40.669 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-08T19:25:40.669 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:40.670 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:40.670 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-08T19:25:40.670 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:40.670 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:40.670 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:40.671 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:40.671 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:40.671 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42356sec preferred_lft 42356sec 2023-11-08T19:25:40.671 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-08T19:25:40.671 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:40.672 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T19:25:40.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:40.672 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:40.673 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:40.673 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2023-11-08T19:25:40.673 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2023-11-08T19:25:40.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2023-11-08T19:25:40.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T19:25:40.674 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:41.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:41.126 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:41.211 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-08T19:25:41.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:41.215 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-08T19:25:41.269 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-08T19:25:41.270 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:41.270 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:41.270 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:41.270 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:41.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:41.271 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:41.271 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:41.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:41.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:41.272 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:41.272 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:42.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:42.056 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-08T19:25:42.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:42.115 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2023-11-08T19:25:42.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:42.167 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:42.168 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:42.168 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T19:25:42.168 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-08T19:25:42.168 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:42.315 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-08T19:25:42.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:42.316 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:42.317 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:42.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-08T19:25:42.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:42.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-08T19:25:42.318 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-08T19:25:42.318 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-08T19:25:42.318 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:42.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:42.596 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:42.596 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:42.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2023-11-08T19:25:42.597 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T19:25:42.597 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:42.761 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:42.762 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:42.813 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-08T19:25:42.814 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:42.814 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:42.860 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-08T19:25:42.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:42.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:43.054 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:43.054 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:43.580 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:43.580 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:43.580 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:25:43.581 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with new device syntax: 1@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:25:43.581 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:25:43.585 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-08T19:25:43.585 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:25:43.585 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:25:43.585 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:25:43.586 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:25:43.590 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:43.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:43.591 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-08T19:25:43.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:43.682 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:43.737 INFO:teuthology.orchestra.run.smithi188.stdout:client4427 2023-11-08T19:25:43.738 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:25:43.738 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:43.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:43.788 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:43.892 INFO:teuthology.orchestra.run.smithi188.stdout:client4427 2023-11-08T19:25:43.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:43.893 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:43.954 INFO:teuthology.orchestra.run.smithi188.stdout:client4427 2023-11-08T19:25:43.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:43.955 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.client4427/status') 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4427 (3)192.168.0.1:0/467098589 2023-11-08T19:25:44.047 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2023-11-08T19:25:44.048 INFO:teuthology.orchestra.run.smithi188.stderr:68 bytes copied, 7.0447e-05 s, 965 kB/s 2023-11-08T19:25:44.051 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-08T19:25:44.051 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T19:25:44.051 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:44.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:25:44.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.276+0000 7fb646365640 1 Processor -- start 2023-11-08T19:25:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.276+0000 7fb646365640 1 -- start start 2023-11-08T19:25:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb646365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb646365640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb64006cb40 0x7fb64006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb646365640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6400c4960 con 0x7fb64015d9f0 2023-11-08T19:25:44.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb646365640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb640143a40 con 0x7fb64006cb40 2023-11-08T19:25:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb646365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400ca3e0 con 0x7fb640142f80 2023-11-08T19:25:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb644902640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb64006cb40 0x7fb64006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb644902640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb64006cb40 0x7fb64006cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53926/0 (socket says 172.21.15.119:53926) 2023-11-08T19:25:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb644902640 1 -- 172.21.15.119:0/1962832028 learned_addr learned my addr 172.21.15.119:0/1962832028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb63f7fe640 1 --1- 172.21.15.119:0/1962832028 >> v1:172.21.15.119:6789/0 conn(0x7fb64015d9f0 0x7fb6400c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51680/0 (socket says 172.21.15.119:51680) 2023-11-08T19:25:44.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.278+0000 7fb63ffff640 1 --2- 172.21.15.119:0/1962832028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.279+0000 7fb644902640 1 -- 172.21.15.119:0/1962832028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 msgr2=0x7fb6400cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:44.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.279+0000 7fb644902640 1 --2- 172.21.15.119:0/1962832028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.279+0000 7fb644902640 1 -- 172.21.15.119:0/1962832028 >> v1:172.21.15.119:6789/0 conn(0x7fb64015d9f0 legacy=0x7fb6400c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.279+0000 7fb644902640 1 -- 172.21.15.119:0/1962832028 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64015df60 con 0x7fb64006cb40 2023-11-08T19:25:44.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.279+0000 7fb63effd640 1 -- 172.21.15.119:0/1962832028 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307758953 0 0) 0x7fb6400c4960 con 0x7fb64015d9f0 2023-11-08T19:25:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.280+0000 7fb644902640 1 --2- 172.21.15.119:0/1962832028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64006cb40 0x7fb64006cfd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb63800d150 tx=0x7fb63802e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df228fdc2f9ee09c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb63effd640 1 -- 172.21.15.119:0/1962832028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63802ea70 con 0x7fb64006cb40 2023-11-08T19:25:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb63effd640 1 -- 172.21.15.119:0/1962832028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63802ebd0 con 0x7fb64006cb40 2023-11-08T19:25:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb63effd640 1 -- 172.21.15.119:0/1962832028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638009600 con 0x7fb64006cb40 2023-11-08T19:25:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 -- 172.21.15.119:0/1962832028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64006cb40 msgr2=0x7fb64006cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 --2- 172.21.15.119:0/1962832028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64006cb40 0x7fb64006cfd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb63800d150 tx=0x7fb63802e0d0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 -- 172.21.15.119:0/1962832028 shutdown_connections 2023-11-08T19:25:44.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 --2- 172.21.15.119:0/1962832028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64006cb40 0x7fb64006cfd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 --2- 172.21.15.119:0/1962832028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 -- 172.21.15.119:0/1962832028 >> 172.21.15.119:0/1962832028 conn(0x7fb6400cb610 msgr2=0x7fb6400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.281+0000 7fb646365640 1 -- 172.21.15.119:0/1962832028 shutdown_connections 2023-11-08T19:25:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.282+0000 7fb646365640 1 -- 172.21.15.119:0/1962832028 wait complete. 2023-11-08T19:25:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.283+0000 7fb646365640 1 Processor -- start 2023-11-08T19:25:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.284+0000 7fb646365640 1 -- start start 2023-11-08T19:25:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.284+0000 7fb646365640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb64006cb40 0x7fb6401409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.284+0000 7fb646365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb646365640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 0x7fb6400c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb646365640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb640140f10 con 0x7fb64015d9f0 2023-11-08T19:25:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb646365640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400b29d0 con 0x7fb64006cb40 2023-11-08T19:25:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb646365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6400b2cb0 con 0x7fb640142f80 2023-11-08T19:25:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 0x7fb6400c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 0x7fb6400c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53942/0 (socket says 172.21.15.119:53942) 2023-11-08T19:25:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 -- 172.21.15.119:0/1857749795 learned_addr learned my addr 172.21.15.119:0/1857749795 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb63f7fe640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb63ffff640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb64006cb40 0x7fb6401409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 -- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 msgr2=0x7fb6400c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 -- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb64006cb40 msgr2=0x7fb6401409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb64006cb40 0x7fb6401409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.285+0000 7fb644902640 1 -- 172.21.15.119:0/1857749795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63800bd40 con 0x7fb64015d9f0 2023-11-08T19:25:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.286+0000 7fb644902640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 0x7fb6400c51f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb638038040 tx=0x7fb638004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.286+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63802ea70 con 0x7fb64015d9f0 2023-11-08T19:25:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.286+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb638036820 con 0x7fb64015d9f0 2023-11-08T19:25:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.286+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638036bd0 con 0x7fb64015d9f0 2023-11-08T19:25:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.286+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6400b2f90 con 0x7fb64015d9f0 2023-11-08T19:25:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.286+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6400b3550 con 0x7fb64015d9f0 2023-11-08T19:25:44.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.287+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb640074430 con 0x7fb64015d9f0 2023-11-08T19:25:44.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.288+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb638036d30 con 0x7fb64015d9f0 2023-11-08T19:25:44.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.289+0000 7fb63cff9640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb62c06ae70 0x7fb62c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:44.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.289+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fb6380acd60 con 0x7fb64015d9f0 2023-11-08T19:25:44.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.289+0000 7fb63ffff640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb62c06ae70 0x7fb62c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:44.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.290+0000 7fb63ffff640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb62c06ae70 0x7fb62c06d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb630006fd0 tx=0x7fb630008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:44.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.294+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb638077ac0 con 0x7fb64015d9f0 2023-11-08T19:25:44.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.520+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb640159e50 con 0x7fb64015d9f0 2023-11-08T19:25:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.520+0000 7fb63cff9640 1 -- 172.21.15.119:0/1857749795 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb63800b720 con 0x7fb64015d9f0 2023-11-08T19:25:44.522 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-08T19:25:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb62c06ae70 msgr2=0x7fb62c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb62c06ae70 0x7fb62c06d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb630006fd0 tx=0x7fb630008040 comp rx=0 tx=0).stop 2023-11-08T19:25:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 msgr2=0x7fb6400c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 0x7fb6400c51f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb638038040 tx=0x7fb638004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 shutdown_connections 2023-11-08T19:25:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb62c06ae70 0x7fb62c06d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb64015d9f0 0x7fb6400c51f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb640142f80 0x7fb6400c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 --2- 172.21.15.119:0/1857749795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb64006cb40 0x7fb6401409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 >> 172.21.15.119:0/1857749795 conn(0x7fb6400cb610 msgr2=0x7fb6400caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 shutdown_connections 2023-11-08T19:25:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:44.523+0000 7fb646365640 1 -- 172.21.15.119:0/1857749795 wait complete. 2023-11-08T19:25:44.544 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T19:25:44.545 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:25:44.545 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2023-11-08T19:25:44.545 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-08T19:25:44.546 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:44.546 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T19:25:44.546 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T19:25:44.546 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:25:44.547 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-08T19:25:44.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:44.547 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2023-11-08T19:25:44.609 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:44.609 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:44.610 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2023-11-08T19:25:44.611 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:44.612 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:44.613 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42352sec preferred_lft 42352sec 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2023-11-08T19:25:44.614 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi188.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 62:4b:ec:35:64:3b brd ff:ff:ff:ff:ff:ff 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:44.615 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::70da:c0ff:fef1:16f4/64 scope link 2023-11-08T19:25:44.616 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:44.617 INFO:teuthology.orchestra.run.smithi188.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:25:44.617 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 62:4b:ec:35:64:3b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T19:25:44.617 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::604b:ecff:fe35:643b/64 scope link 2023-11-08T19:25:44.617 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:44.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:44.618 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:25:44.712 INFO:teuthology.orchestra.run.smithi188.stdout:1 2023-11-08T19:25:44.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:44.717 DEBUG:teuthology.orchestra.run.smithi188:> route 2023-11-08T19:25:44.769 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2023-11-08T19:25:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:25:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:25:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:25:44.770 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:25:44.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:44.772 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:44.772 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:44.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:25:44.773 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:25:44.774 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:25:44.774 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:44.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:44.963 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-08T19:25:45.018 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T19:25:45.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:45.019 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2023-11-08T19:25:45.073 INFO:teuthology.orchestra.run.smithi188.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-08T19:25:45.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:45.075 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:45.075 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:45.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T19:25:45.076 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-08T19:25:45.076 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:45.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:45.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T19:25:45.312 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:45.313 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:45.313 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:45.313 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.313 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:45.314 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.314 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:45.314 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether be:17:92:f5:f0:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:45.314 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:45.314 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.315 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::bc17:92ff:fef5:f044/64 scope link 2023-11-08T19:25:45.315 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:45.317 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T19:25:45.403 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:25:45.403 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:25:45.404 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:25:45.405 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.405 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2023-11-08T19:25:45.406 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.406 INFO:teuthology.orchestra.run.smithi188.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:25:45.406 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether be:17:92:f5:f0:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T19:25:45.406 INFO:teuthology.orchestra.run.smithi188.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T19:25:45.407 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.407 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::bc17:92ff:fef5:f044/64 scope link 2023-11-08T19:25:45.407 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:25:45.409 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-08T19:25:45.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:45.409 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:45.410 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:45.410 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-08T19:25:45.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T19:25:45.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-08T19:25:45.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-08T19:25:45.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-08T19:25:45.412 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:45.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:45.695 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec bash -c ' 2023-11-08T19:25:45.695 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2023-11-08T19:25:45.695 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.1 up 2023-11-08T19:25:45.696 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T19:25:45.696 DEBUG:teuthology.orchestra.run.smithi188:> ') 2023-11-08T19:25:45.869 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:45.869 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:45.917 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-08T19:25:45.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:25:45.918 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:46.015 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-08T19:25:46.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:46.015 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:46.153 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:46.153 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:46.186 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:25:46.187 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:25:46.187 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:25:46.187 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with new device syntax: 3@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:25:46.187 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:25:46.190 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-11-08T19:25:46.191 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:25:46.191 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:25:46.191 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:25:46.192 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:25:46.195 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:25:46.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:46.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-08T19:25:46.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:46.282 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:46.340 INFO:teuthology.orchestra.run.smithi188.stdout:client4433 2023-11-08T19:25:46.341 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:25:46.341 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:46.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:46.398 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:46.455 INFO:teuthology.orchestra.run.smithi188.stdout:client4433 2023-11-08T19:25:46.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:46.456 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:25:46.512 INFO:teuthology.orchestra.run.smithi188.stdout:client4433 2023-11-08T19:25:46.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:25:46.513 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.client4433/status') 2023-11-08T19:25:46.604 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4433 (3)192.168.0.2:0/1709232121 2023-11-08T19:25:46.604 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2023-11-08T19:25:46.605 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2023-11-08T19:25:46.605 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2023-11-08T19:25:46.605 INFO:teuthology.orchestra.run.smithi188.stderr:69 bytes copied, 7.0269e-05 s, 982 kB/s 2023-11-08T19:25:46.608 INFO:teuthology.run_tasks:Running task exec... 2023-11-08T19:25:46.621 INFO:teuthology.task.exec:Executing custom commands... 2023-11-08T19:25:46.621 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:25:46.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/crash/warn_recent_interval 0' 2023-11-08T19:25:46.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.842+0000 7f2fdecea640 1 Processor -- start 2023-11-08T19:25:46.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.842+0000 7f2fdecea640 1 -- start start 2023-11-08T19:25:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdecea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142fb0 0x7f2fd806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdecea640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fd8143710 0x7f2fd80c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdecea640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd80c9760 con 0x7f2fd80ca130 2023-11-08T19:25:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdecea640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd80cac60 con 0x7f2fd8143710 2023-11-08T19:25:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdecea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd806feb0 con 0x7f2fd8142fb0 2023-11-08T19:25:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdd4e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fd8143710 0x7f2fd80c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdd4e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2fd8143710 0x7f2fd80c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53944/0 (socket says 172.21.15.119:53944) 2023-11-08T19:25:46.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdd4e7640 1 -- 172.21.15.119:0/1282638624 learned_addr learned my addr 172.21.15.119:0/1282638624 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:46.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fddce8640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142fb0 0x7f2fd806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdd4e7640 1 -- 172.21.15.119:0/1282638624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142fb0 msgr2=0x7f2fd806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.844+0000 7f2fdd4e7640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142fb0 0x7f2fd806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.845+0000 7f2fdd4e7640 1 -- 172.21.15.119:0/1282638624 >> v1:172.21.15.188:6789/0 conn(0x7f2fd80ca130 legacy=0x7f2fd80c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.845+0000 7f2fdd4e7640 1 -- 172.21.15.119:0/1282638624 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd80caf00 con 0x7f2fd8143710 2023-11-08T19:25:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.845+0000 7f2fddce8640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142fb0 0x7f2fd806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:46.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.846+0000 7f2fdd4e7640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8143710 0x7f2fd80c9bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800bb10 tx=0x7f2fc802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3b18ca12c5476b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.846+0000 7f2fdcce6640 1 -- 172.21.15.119:0/1282638624 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc802ee70 con 0x7f2fd8143710 2023-11-08T19:25:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.846+0000 7f2fdcce6640 1 -- 172.21.15.119:0/1282638624 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc8036440 con 0x7f2fd8143710 2023-11-08T19:25:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.846+0000 7f2fdcce6640 1 -- 172.21.15.119:0/1282638624 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80367d0 con 0x7f2fd8143710 2023-11-08T19:25:46.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.846+0000 7f2fdecea640 1 -- 172.21.15.119:0/1282638624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8143710 msgr2=0x7f2fd80c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.846+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8143710 0x7f2fd80c9bf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2fc800bb10 tx=0x7f2fc802e0c0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.847+0000 7f2fdecea640 1 -- 172.21.15.119:0/1282638624 shutdown_connections 2023-11-08T19:25:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.847+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8143710 0x7f2fd80c9bf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.847+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1282638624 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142fb0 0x7f2fd806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.847+0000 7f2fdecea640 1 -- 172.21.15.119:0/1282638624 >> 172.21.15.119:0/1282638624 conn(0x7f2fd80cb4a0 msgr2=0x7f2fd80cb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:46.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.847+0000 7f2fdecea640 1 -- 172.21.15.119:0/1282638624 shutdown_connections 2023-11-08T19:25:46.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.847+0000 7f2fdecea640 1 -- 172.21.15.119:0/1282638624 wait complete. 2023-11-08T19:25:46.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.848+0000 7f2fdecea640 1 Processor -- start 2023-11-08T19:25:46.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.849+0000 7f2fdecea640 1 -- start start 2023-11-08T19:25:46.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.849+0000 7f2fdecea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 0x7f2fd80b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.849+0000 7f2fdecea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd8142fb0 0x7f2fd813f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.849+0000 7f2fdecea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8143710 0x7f2fd815a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fdecea640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd81f3520 con 0x7f2fd80ca130 2023-11-08T19:25:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fdecea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd806bb10 con 0x7f2fd8142fb0 2023-11-08T19:25:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fdecea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd80b2410 con 0x7f2fd8143710 2023-11-08T19:25:46.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 0x7f2fd80b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 0x7f2fd80b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53958/0 (socket says 172.21.15.119:53958) 2023-11-08T19:25:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 -- 172.21.15.119:0/1031861839 learned_addr learned my addr 172.21.15.119:0/1031861839 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fdd4e7640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd8142fb0 0x7f2fd813f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 -- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8143710 msgr2=0x7f2fd815a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8143710 0x7f2fd815a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 -- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd8142fb0 msgr2=0x7f2fd813f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd8142fb0 0x7f2fd813f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 -- 172.21.15.119:0/1031861839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc800b770 con 0x7f2fd80ca130 2023-11-08T19:25:46.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.850+0000 7f2fddce8640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 0x7f2fd80b1ed0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c7f0 tx=0x7f2fd400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.851+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4007a30 con 0x7f2fd80ca130 2023-11-08T19:25:46.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.851+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4007b90 con 0x7f2fd80ca130 2023-11-08T19:25:46.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.851+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd813fb20 con 0x7f2fd80ca130 2023-11-08T19:25:46.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.851+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd400ce40 con 0x7f2fd80ca130 2023-11-08T19:25:46.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.851+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd806b4c0 con 0x7f2fd80ca130 2023-11-08T19:25:46.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.852+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd804ece0 con 0x7f2fd80ca130 2023-11-08T19:25:46.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.853+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fd40113f0 con 0x7f2fd80ca130 2023-11-08T19:25:46.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.854+0000 7f2fce7fc640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fbc06ae70 0x7f2fbc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:46.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.854+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2fd4089ba0 con 0x7f2fd80ca130 2023-11-08T19:25:46.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.854+0000 7f2fdd4e7640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fbc06ae70 0x7f2fbc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:46.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.855+0000 7f2fdd4e7640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fbc06ae70 0x7f2fbc06d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8038000 tx=0x7f2fc8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:46.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:46.858+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fd4054860 con 0x7f2fd80ca130 2023-11-08T19:25:47.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.048+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mgr/crash/warn_recent_interval}] v 0) v1 -- 0x7f2fd8066600 con 0x7f2fd80ca130 2023-11-08T19:25:47.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.055+0000 7f2fce7fc640 1 -- 172.21.15.119:0/1031861839 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/crash/warn_recent_interval}]=0 v10)=0 v10) v1 ==== 128+0+0 (secure 0 0 0) 0x7f2fd401f0c0 con 0x7f2fd80ca130 2023-11-08T19:25:47.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fbc06ae70 msgr2=0x7f2fbc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fbc06ae70 0x7f2fbc06d330 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8038000 tx=0x7f2fc8008040 comp rx=0 tx=0).stop 2023-11-08T19:25:47.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 msgr2=0x7f2fd80b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 0x7f2fd80b1ed0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c7f0 tx=0x7f2fd400cbf0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 shutdown_connections 2023-11-08T19:25:47.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fbc06ae70 0x7f2fbc06d330 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8143710 0x7f2fd815a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd8142fb0 0x7f2fd813f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 --2- 172.21.15.119:0/1031861839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd80ca130 0x7f2fd80b1ed0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.058+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 >> 172.21.15.119:0/1031861839 conn(0x7f2fd80cb4a0 msgr2=0x7f2fd81402c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.059+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 shutdown_connections 2023-11-08T19:25:47.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.059+0000 7f2fdecea640 1 -- 172.21.15.119:0/1031861839 wait complete. 2023-11-08T19:25:47.080 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-08T19:25:47.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.313+0000 7f13e00e0640 1 Processor -- start 2023-11-08T19:25:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.314+0000 7f13e00e0640 1 -- start start 2023-11-08T19:25:47.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.315+0000 7f13e00e0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f13d81447a0 0x7f13d815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.315+0000 7f13e00e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 0x7f13d806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.315+0000 7f13e00e0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d80c3a50 con 0x7f13d80cb170 2023-11-08T19:25:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.315+0000 7f13e00e0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d8144b80 con 0x7f13d81447a0 2023-11-08T19:25:47.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.315+0000 7f13e00e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80c9150 con 0x7f13d80cad30 2023-11-08T19:25:47.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.316+0000 7f13dd67b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 0x7f13d806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.316+0000 7f13dd67b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 0x7f13d806f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:47966/0 (socket says 172.21.15.119:47966) 2023-11-08T19:25:47.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.316+0000 7f13de67d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f13d80cb170 0x7f13d806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:53924/0 (socket says 172.21.15.119:53924) 2023-11-08T19:25:47.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.316+0000 7f13dd67b640 1 -- 172.21.15.119:0/3106477374 learned_addr learned my addr 172.21.15.119:0/3106477374 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:47.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.316+0000 7f13dde7c640 1 --2- 172.21.15.119:0/3106477374 >> v2:172.21.15.188:3300/0 conn(0x7f13d81447a0 0x7f13d815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dce7a640 1 -- 172.21.15.119:0/3106477374 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753794934 0 0) 0x7f13d80c3a50 con 0x7f13d80cb170 2023-11-08T19:25:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dce7a640 1 -- 172.21.15.119:0/3106477374 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c4003520 con 0x7f13d80cb170 2023-11-08T19:25:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dd67b640 1 -- 172.21.15.119:0/3106477374 >> v2:172.21.15.188:3300/0 conn(0x7f13d81447a0 msgr2=0x7f13d815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dd67b640 1 --2- 172.21.15.119:0/3106477374 >> v2:172.21.15.188:3300/0 conn(0x7f13d81447a0 0x7f13d815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dd67b640 1 -- 172.21.15.119:0/3106477374 >> v1:172.21.15.188:6789/0 conn(0x7f13d80cb170 legacy=0x7f13d806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dd67b640 1 -- 172.21.15.119:0/3106477374 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d80c9410 con 0x7f13d80cad30 2023-11-08T19:25:47.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.317+0000 7f13dde7c640 1 --2- 172.21.15.119:0/3106477374 >> v2:172.21.15.188:3300/0 conn(0x7f13d81447a0 0x7f13d815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:25:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.319+0000 7f13dd67b640 1 --2- 172.21.15.119:0/3106477374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 0x7f13d806f930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009870 tx=0x7f13c802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=816a1be634f512c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.319+0000 7f13dce7a640 1 -- 172.21.15.119:0/3106477374 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8030840 con 0x7f13d80cad30 2023-11-08T19:25:47.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.319+0000 7f13dce7a640 1 -- 172.21.15.119:0/3106477374 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c80309a0 con 0x7f13d80cad30 2023-11-08T19:25:47.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.319+0000 7f13dce7a640 1 -- 172.21.15.119:0/3106477374 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8030d30 con 0x7f13d80cad30 2023-11-08T19:25:47.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 -- 172.21.15.119:0/3106477374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 msgr2=0x7f13d806f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 --2- 172.21.15.119:0/3106477374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 0x7f13d806f930 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009870 tx=0x7f13c802b980 comp rx=0 tx=0).stop 2023-11-08T19:25:47.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 -- 172.21.15.119:0/3106477374 shutdown_connections 2023-11-08T19:25:47.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 --2- 172.21.15.119:0/3106477374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d80cad30 0x7f13d806f930 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 --2- 172.21.15.119:0/3106477374 >> v2:172.21.15.188:3300/0 conn(0x7f13d81447a0 0x7f13d815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 -- 172.21.15.119:0/3106477374 >> 172.21.15.119:0/3106477374 conn(0x7f13d80cb610 msgr2=0x7f13d80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 -- 172.21.15.119:0/3106477374 shutdown_connections 2023-11-08T19:25:47.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.320+0000 7f13e00e0640 1 -- 172.21.15.119:0/3106477374 wait complete. 2023-11-08T19:25:47.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.322+0000 7f13e00e0640 1 Processor -- start 2023-11-08T19:25:47.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.322+0000 7f13e00e0640 1 -- start start 2023-11-08T19:25:47.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.323+0000 7f13e00e0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cad30 0x7f13d815a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.323+0000 7f13e00e0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 0x7f13d814a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.323+0000 7f13e00e0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d81447a0 0x7f13d8136f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.323+0000 7f13e00e0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d815cef0 con 0x7f13d80cb170 2023-11-08T19:25:47.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.323+0000 7f13e00e0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d814a830 con 0x7f13d80cad30 2023-11-08T19:25:47.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.323+0000 7f13e00e0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d813a2a0 con 0x7f13d81447a0 2023-11-08T19:25:47.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 0x7f13d814a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 0x7f13d814a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53960/0 (socket says 172.21.15.119:53960) 2023-11-08T19:25:47.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 -- 172.21.15.119:0/4258815657 learned_addr learned my addr 172.21.15.119:0/4258815657 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:47.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dde7c640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cad30 0x7f13d815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13de67d640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d81447a0 0x7f13d8136f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 -- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d81447a0 msgr2=0x7f13d8136f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d81447a0 0x7f13d8136f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 -- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cad30 msgr2=0x7f13d815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cad30 0x7f13d815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 -- 172.21.15.119:0/4258815657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4009400 con 0x7f13d80cb170 2023-11-08T19:25:47.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.324+0000 7f13dd67b640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 0x7f13d814a2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000c00 tx=0x7f13c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.325+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8039c90 con 0x7f13d80cb170 2023-11-08T19:25:47.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.325+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13c8039df0 con 0x7f13d80cb170 2023-11-08T19:25:47.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.325+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8043640 con 0x7f13d80cb170 2023-11-08T19:25:47.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.325+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c80094d0 con 0x7f13d80cb170 2023-11-08T19:25:47.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.325+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d815c8a0 con 0x7f13d80cb170 2023-11-08T19:25:47.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.326+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d8074430 con 0x7f13d80cb170 2023-11-08T19:25:47.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.327+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13c80437a0 con 0x7f13d80cb170 2023-11-08T19:25:47.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.328+0000 7f13ce7fc640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f13bc06ae70 0x7f13bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.328+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f13c803e070 con 0x7f13d80cb170 2023-11-08T19:25:47.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.328+0000 7f13dde7c640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f13bc06ae70 0x7f13bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.329+0000 7f13dde7c640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f13bc06ae70 0x7f13bc06d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009bc0 tx=0x7f13d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.332+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13c8077610 con 0x7f13d80cb170 2023-11-08T19:25:47.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.522+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13d80c5180 con 0x7f13d80cb170 2023-11-08T19:25:47.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.525+0000 7f13ce7fc640 1 -- 172.21.15.119:0/4258815657 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f13c807ac30 con 0x7f13d80cb170 2023-11-08T19:25:47.527 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:47.528 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:47.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f13bc06ae70 msgr2=0x7f13bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f13bc06ae70 0x7f13bc06d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009bc0 tx=0x7f13d4008040 comp rx=0 tx=0).stop 2023-11-08T19:25:47.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 msgr2=0x7f13d814a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 0x7f13d814a2f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f13c8000c00 tx=0x7f13c8004730 comp rx=0 tx=0).stop 2023-11-08T19:25:47.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 shutdown_connections 2023-11-08T19:25:47.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f13bc06ae70 0x7f13bc06d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13d81447a0 0x7f13d8136f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13d80cb170 0x7f13d814a2f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 --2- 172.21.15.119:0/4258815657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13d80cad30 0x7f13d815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.528+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 >> 172.21.15.119:0/4258815657 conn(0x7f13d80cb610 msgr2=0x7f13d806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.529+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 shutdown_connections 2023-11-08T19:25:47.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.529+0000 7f13e00e0640 1 -- 172.21.15.119:0/4258815657 wait complete. 2023-11-08T19:25:47.550 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:47.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:47.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.769+0000 7f7478c9a640 1 Processor -- start 2023-11-08T19:25:47.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.769+0000 7f7478c9a640 1 -- start start 2023-11-08T19:25:47.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.770+0000 7f7478c9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74740b13d0 0x7f74740b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.770+0000 7f7478c9a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f747409ce90 0x7f747406bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.770+0000 7f7478c9a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7474160ab0 con 0x7f74740b2450 2023-11-08T19:25:47.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.770+0000 7f7478c9a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74740ad5d0 con 0x7f747409ce90 2023-11-08T19:25:47.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.770+0000 7f7478c9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7474160d70 con 0x7f74740b13d0 2023-11-08T19:25:47.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f74737fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f74740b2450 0x7f74740b0cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51686/0 (socket says 172.21.15.119:51686) 2023-11-08T19:25:47.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f74737fe640 1 -- 172.21.15.119:0/110377825 learned_addr learned my addr 172.21.15.119:0/110377825 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:47.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f7473fff640 1 --2- 172.21.15.119:0/110377825 >> v2:172.21.15.119:3300/0 conn(0x7f747409ce90 0x7f747406bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f7472ffd640 1 --2- 172.21.15.119:0/110377825 >> v2:172.21.15.188:3300/0 conn(0x7f74740b13d0 0x7f74740b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f7473fff640 1 -- 172.21.15.119:0/110377825 >> v2:172.21.15.188:3300/0 conn(0x7f74740b13d0 msgr2=0x7f74740b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:47.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f7473fff640 1 --2- 172.21.15.119:0/110377825 >> v2:172.21.15.188:3300/0 conn(0x7f74740b13d0 0x7f74740b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f7473fff640 1 -- 172.21.15.119:0/110377825 >> v1:172.21.15.119:6789/0 conn(0x7f74740b2450 legacy=0x7f74740b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.771+0000 7f7473fff640 1 -- 172.21.15.119:0/110377825 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7474161050 con 0x7f747409ce90 2023-11-08T19:25:47.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.772+0000 7f7473fff640 1 --2- 172.21.15.119:0/110377825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747406bd60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f747406d740 tx=0x7f746c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=803d2f5ae0ed4ebe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f74727fc640 1 -- 172.21.15.119:0/110377825 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c02e070 con 0x7f747409ce90 2023-11-08T19:25:47.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f74727fc640 1 -- 172.21.15.119:0/110377825 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c0315a0 con 0x7f747409ce90 2023-11-08T19:25:47.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f74727fc640 1 -- 172.21.15.119:0/110377825 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c031930 con 0x7f747409ce90 2023-11-08T19:25:47.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f7478c9a640 1 -- 172.21.15.119:0/110377825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 msgr2=0x7f747406bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f7478c9a640 1 --2- 172.21.15.119:0/110377825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747406bd60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f747406d740 tx=0x7f746c02b840 comp rx=0 tx=0).stop 2023-11-08T19:25:47.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f7478c9a640 1 -- 172.21.15.119:0/110377825 shutdown_connections 2023-11-08T19:25:47.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f7478c9a640 1 --2- 172.21.15.119:0/110377825 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747406bd60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f7478c9a640 1 --2- 172.21.15.119:0/110377825 >> v2:172.21.15.188:3300/0 conn(0x7f74740b13d0 0x7f74740b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.773+0000 7f7478c9a640 1 -- 172.21.15.119:0/110377825 >> 172.21.15.119:0/110377825 conn(0x7f74740cbcc0 msgr2=0x7f74740cc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.774+0000 7f7478c9a640 1 -- 172.21.15.119:0/110377825 shutdown_connections 2023-11-08T19:25:47.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.774+0000 7f7478c9a640 1 -- 172.21.15.119:0/110377825 wait complete. 2023-11-08T19:25:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.776+0000 7f7478c9a640 1 Processor -- start 2023-11-08T19:25:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.776+0000 7f7478c9a640 1 -- start start 2023-11-08T19:25:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7478c9a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747415a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7478c9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74740b13d0 0x7f747415fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7478c9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74740b2450 0x7f747415d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7478c9a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747409a940 con 0x7f747409ce90 2023-11-08T19:25:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7478c9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747415dad0 con 0x7f74740b13d0 2023-11-08T19:25:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7478c9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7474099a50 con 0x7f74740b2450 2023-11-08T19:25:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747415a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747415a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53974/0 (socket says 172.21.15.119:53974) 2023-11-08T19:25:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 -- 172.21.15.119:0/524598866 learned_addr learned my addr 172.21.15.119:0/524598866 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7472ffd640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74740b13d0 0x7f747415fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 -- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74740b2450 msgr2=0x7f747415d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f7473fff640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74740b2450 0x7f747415d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74740b2450 0x7f747415d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 -- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74740b13d0 msgr2=0x7f747415fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.778+0000 7f74737fe640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74740b13d0 0x7f747415fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f74737fe640 1 -- 172.21.15.119:0/524598866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746c009400 con 0x7f747409ce90 2023-11-08T19:25:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f7472ffd640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74740b13d0 0x7f747415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f74737fe640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747415a4d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c480 tx=0x7f745c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c005ce0 con 0x7f747409ce90 2023-11-08T19:25:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c005e40 con 0x7f747409ce90 2023-11-08T19:25:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0072f0 con 0x7f747409ce90 2023-11-08T19:25:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7474099d30 con 0x7f747409ce90 2023-11-08T19:25:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.779+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747409a2f0 con 0x7f747409ce90 2023-11-08T19:25:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.781+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747404ece0 con 0x7f747409ce90 2023-11-08T19:25:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.782+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f745c007450 con 0x7f747409ce90 2023-11-08T19:25:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.783+0000 7f746bfff640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f745406ae70 0x7f745406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:47.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.783+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f745c088df0 con 0x7f747409ce90 2023-11-08T19:25:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.783+0000 7f7472ffd640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f745406ae70 0x7f745406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.784+0000 7f7472ffd640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f745406ae70 0x7f745406d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f747415e820 tx=0x7f746400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:47.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.787+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f745c053c00 con 0x7f747409ce90 2023-11-08T19:25:47.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.976+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74741f5b20 con 0x7f747409ce90 2023-11-08T19:25:47.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.977+0000 7f746bfff640 1 -- 172.21.15.119:0/524598866 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f745c017020 con 0x7f747409ce90 2023-11-08T19:25:47.979 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:47.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f745406ae70 msgr2=0x7f745406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f745406ae70 0x7f745406d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f747415e820 tx=0x7f746400b040 comp rx=0 tx=0).stop 2023-11-08T19:25:47.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 msgr2=0x7f747415a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:47.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747415a4d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f745c00c480 tx=0x7f745c00c940 comp rx=0 tx=0).stop 2023-11-08T19:25:47.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 shutdown_connections 2023-11-08T19:25:47.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f745406ae70 0x7f745406d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74740b2450 0x7f747415d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74740b13d0 0x7f747415fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 --2- 172.21.15.119:0/524598866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f747409ce90 0x7f747415a4d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 >> 172.21.15.119:0/524598866 conn(0x7f74740cbcc0 msgr2=0x7f74740ce2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 shutdown_connections 2023-11-08T19:25:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:47.980+0000 7f7478c9a640 1 -- 172.21.15.119:0/524598866 wait complete. 2023-11-08T19:25:47.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:48.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.217+0000 7f8ac0a1a640 1 Processor -- start 2023-11-08T19:25:48.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.217+0000 7f8ac0a1a640 1 -- start start 2023-11-08T19:25:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.218+0000 7f8ac0a1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8abc144960 0x7f8abc15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.218+0000 7f8ac0a1a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8abc15f1d0 0x7f8abc0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.218+0000 7f8ac0a1a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc15ea60 con 0x7f8abc144580 2023-11-08T19:25:48.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.218+0000 7f8ac0a1a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0c9150 con 0x7f8abc15f1d0 2023-11-08T19:25:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8ac0a1a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0c9360 con 0x7f8abc144960 2023-11-08T19:25:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8abad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8abc15f1d0 0x7f8abc0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8aba59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8abc144580 0x7f8abc0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51696/0 (socket says 172.21.15.119:51696) 2023-11-08T19:25:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8aba59c640 1 -- 172.21.15.119:0/2926305894 learned_addr learned my addr 172.21.15.119:0/2926305894 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8abad9d640 1 -- 172.21.15.119:0/2926305894 >> v2:172.21.15.188:3300/0 conn(0x7f8abc144960 msgr2=0x7f8abc15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8abad9d640 1 --2- 172.21.15.119:0/2926305894 >> v2:172.21.15.188:3300/0 conn(0x7f8abc144960 0x7f8abc15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8abad9d640 1 -- 172.21.15.119:0/2926305894 >> v1:172.21.15.119:6789/0 conn(0x7f8abc144580 legacy=0x7f8abc0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8abad9d640 1 -- 172.21.15.119:0/2926305894 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0c9640 con 0x7f8abc15f1d0 2023-11-08T19:25:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.219+0000 7f8ab9d9b640 1 -- 172.21.15.119:0/2926305894 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003525070 0 0) 0x7f8abc15ea60 con 0x7f8abc144580 2023-11-08T19:25:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.220+0000 7f8abad9d640 1 --2- 172.21.15.119:0/2926305894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc15f1d0 0x7f8abc0c3a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0ce900 tx=0x7f8ab402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a047de051b7a754 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ab9d9b640 1 -- 172.21.15.119:0/2926305894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab402cbf0 con 0x7f8abc15f1d0 2023-11-08T19:25:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ab9d9b640 1 -- 172.21.15.119:0/2926305894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab402cd50 con 0x7f8abc15f1d0 2023-11-08T19:25:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ab9d9b640 1 -- 172.21.15.119:0/2926305894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab4034660 con 0x7f8abc15f1d0 2023-11-08T19:25:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/2926305894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc15f1d0 msgr2=0x7f8abc0c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/2926305894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc15f1d0 0x7f8abc0c3a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8abc0ce900 tx=0x7f8ab402be40 comp rx=0 tx=0).stop 2023-11-08T19:25:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/2926305894 shutdown_connections 2023-11-08T19:25:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/2926305894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc15f1d0 0x7f8abc0c3a50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/2926305894 >> v2:172.21.15.188:3300/0 conn(0x7f8abc144960 0x7f8abc15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/2926305894 >> 172.21.15.119:0/2926305894 conn(0x7f8abc0cb610 msgr2=0x7f8abc0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.221+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/2926305894 shutdown_connections 2023-11-08T19:25:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.222+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/2926305894 wait complete. 2023-11-08T19:25:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.223+0000 7f8ac0a1a640 1 Processor -- start 2023-11-08T19:25:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.223+0000 7f8ac0a1a640 1 -- start start 2023-11-08T19:25:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.224+0000 7f8ac0a1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc144580 0x7f8abc139110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.224+0000 7f8ac0a1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 0x7f8abc13bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.224+0000 7f8ac0a1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15f1d0 0x7f8abc139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.224+0000 7f8ac0a1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc13d670 con 0x7f8abc144960 2023-11-08T19:25:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.224+0000 7f8ac0a1a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc13d320 con 0x7f8abc144580 2023-11-08T19:25:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.224+0000 7f8ac0a1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc139650 con 0x7f8abc15f1d0 2023-11-08T19:25:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 0x7f8abc13bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 0x7f8abc13bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54002/0 (socket says 172.21.15.119:54002) 2023-11-08T19:25:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 -- 172.21.15.119:0/1096476753 learned_addr learned my addr 172.21.15.119:0/1096476753 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8aba59c640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc144580 0x7f8abc139110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 -- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15f1d0 msgr2=0x7f8abc139da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15f1d0 0x7f8abc139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 -- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc144580 msgr2=0x7f8abc139110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc144580 0x7f8abc139110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 -- 172.21.15.119:0/1096476753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4009a70 con 0x7f8abc144960 2023-11-08T19:25:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8aba59c640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc144580 0x7f8abc139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab3fff640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 0x7f8abc13bf20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b450 tx=0x7f8aa400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4011890 con 0x7f8abc144960 2023-11-08T19:25:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa40119f0 con 0x7f8abc144960 2023-11-08T19:25:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.225+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4011da0 con 0x7f8abc144960 2023-11-08T19:25:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.226+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc13c990 con 0x7f8abc144960 2023-11-08T19:25:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.226+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc13cf50 con 0x7f8abc144960 2023-11-08T19:25:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.227+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc13d7b0 con 0x7f8abc144960 2023-11-08T19:25:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.228+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8aa401a400 con 0x7f8abc144960 2023-11-08T19:25:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.229+0000 7f8ab2ffd640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a9c06ae70 0x7f8a9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.229+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f8aa4089e10 con 0x7f8abc144960 2023-11-08T19:25:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.229+0000 7f8aba59c640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a9c06ae70 0x7f8a9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.230+0000 7f8aba59c640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a9c06ae70 0x7f8a9c06d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000f60 tx=0x7f8aac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:48.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.233+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8abc13d7b0 con 0x7f8abc144960 2023-11-08T19:25:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.423+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8abc13d170 con 0x7f8abc144960 2023-11-08T19:25:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.424+0000 7f8ab2ffd640 1 -- 172.21.15.119:0/1096476753 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8aa40150a0 con 0x7f8abc144960 2023-11-08T19:25:48.426 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:48.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a9c06ae70 msgr2=0x7f8a9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a9c06ae70 0x7f8a9c06d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000f60 tx=0x7f8aac009040 comp rx=0 tx=0).stop 2023-11-08T19:25:48.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 msgr2=0x7f8abc13bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 0x7f8abc13bf20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400b450 tx=0x7f8aa400b910 comp rx=0 tx=0).stop 2023-11-08T19:25:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 shutdown_connections 2023-11-08T19:25:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a9c06ae70 0x7f8a9c06d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8abc15f1d0 0x7f8abc139da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8abc144960 0x7f8abc13bf20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 --2- 172.21.15.119:0/1096476753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8abc144580 0x7f8abc139110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.427+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 >> 172.21.15.119:0/1096476753 conn(0x7f8abc0cb610 msgr2=0x7f8abc0c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.428+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 shutdown_connections 2023-11-08T19:25:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.428+0000 7f8ac0a1a640 1 -- 172.21.15.119:0/1096476753 wait complete. 2023-11-08T19:25:48.447 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:48.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.646+0000 7f5acffff640 1 Processor -- start 2023-11-08T19:25:48.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.646+0000 7f5acffff640 1 -- start start 2023-11-08T19:25:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.647+0000 7f5acffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160910 0x7f5ad00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.647+0000 7f5acffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ad00ce5b0 0x7f5ad00b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.647+0000 7f5acffff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ad00d0830 con 0x7f5ad0160530 2023-11-08T19:25:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.647+0000 7f5acffff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ad00b27c0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.647+0000 7f5acffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad00b2aa0 con 0x7f5ad0160910 2023-11-08T19:25:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ad00ce5b0 0x7f5ad00b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ad00ce5b0 0x7f5ad00b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39960/0 (socket says 172.21.15.119:39960) 2023-11-08T19:25:48.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 -- 172.21.15.119:0/1116949079 learned_addr learned my addr 172.21.15.119:0/1116949079 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:48.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5ace7fc640 1 --2- 172.21.15.119:0/1116949079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160910 0x7f5ad00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 -- 172.21.15.119:0/1116949079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160910 msgr2=0x7f5ad00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 --2- 172.21.15.119:0/1116949079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160910 0x7f5ad00ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 -- 172.21.15.119:0/1116949079 >> v1:172.21.15.188:6789/0 conn(0x7f5ad0160530 legacy=0x7f5ad00cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.648+0000 7f5acf7fe640 1 -- 172.21.15.119:0/1116949079 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad00d1640 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.649+0000 7f5acf7fe640 1 --2- 172.21.15.119:0/1116949079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad00b2f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009e50 tx=0x7f5ac802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dc8f860734694d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acdffb640 1 -- 172.21.15.119:0/1116949079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac802ca70 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acdffb640 1 -- 172.21.15.119:0/1116949079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac802cbd0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acdffb640 1 -- 172.21.15.119:0/1116949079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac80343f0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 -- 172.21.15.119:0/1116949079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 msgr2=0x7f5ad00b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 --2- 172.21.15.119:0/1116949079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad00b2f30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009e50 tx=0x7f5ac802bed0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 -- 172.21.15.119:0/1116949079 shutdown_connections 2023-11-08T19:25:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 --2- 172.21.15.119:0/1116949079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad00b2f30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 --2- 172.21.15.119:0/1116949079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160910 0x7f5ad00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 -- 172.21.15.119:0/1116949079 >> 172.21.15.119:0/1116949079 conn(0x7f5ad00d2ec0 msgr2=0x7f5ad00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 -- 172.21.15.119:0/1116949079 shutdown_connections 2023-11-08T19:25:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.650+0000 7f5acffff640 1 -- 172.21.15.119:0/1116949079 wait complete. 2023-11-08T19:25:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.652+0000 7f5acffff640 1 Processor -- start 2023-11-08T19:25:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.652+0000 7f5acffff640 1 -- start start 2023-11-08T19:25:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5acffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad01f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5acffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160530 0x7f5ad00b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5acffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad0160910 0x7f5ad00cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5acffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad01f6470 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5acffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad01f5e60 con 0x7f5ad0160910 2023-11-08T19:25:48.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5acffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad00ccb80 con 0x7f5ad0160530 2023-11-08T19:25:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.653+0000 7f5aceffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad01f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad01f6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39964/0 (socket says 172.21.15.119:39964) 2023-11-08T19:25:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 -- 172.21.15.119:0/2008356808 learned_addr learned my addr 172.21.15.119:0/2008356808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5ace7fc640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160530 0x7f5ad00b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 -- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160530 msgr2=0x7f5ad00b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160530 0x7f5ad00b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 -- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad0160910 msgr2=0x7f5ad00cc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad0160910 0x7f5ad00cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 -- 172.21.15.119:0/2008356808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac8009b30 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5aceffd640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad01f6900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0004640 tx=0x7f5ac000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.654+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0011790 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.655+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac00118f0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.655+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad01f6100 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.655+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0011ce0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.655+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad00b41c0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.656+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad004ece0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.657+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ac0012420 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.658+0000 7f5ab77fe640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5abc06ae70 0x7f5abc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.658+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5ac0089690 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.658+0000 7f5ace7fc640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5abc06ae70 0x7f5abc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:48.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.659+0000 7f5ace7fc640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5abc06ae70 0x7f5abc06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5ad015c800 tx=0x7f5ac4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:48.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.663+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ac00542c0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.754 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:48.764 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:25:48.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.853+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ad00b3210 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.854+0000 7f5ab77fe640 1 -- 172.21.15.119:0/2008356808 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ac00578e0 con 0x7f5ad00ce5b0 2023-11-08T19:25:48.856 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5abc06ae70 msgr2=0x7f5abc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5abc06ae70 0x7f5abc06d330 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5ad015c800 tx=0x7f5ac4008040 comp rx=0 tx=0).stop 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 msgr2=0x7f5ad01f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad01f6900 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5ac0004640 tx=0x7f5ac000c8e0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 shutdown_connections 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5abc06ae70 0x7f5abc06d330 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ad0160910 0x7f5ad00cc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ad0160530 0x7f5ad00b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 --2- 172.21.15.119:0/2008356808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ad00ce5b0 0x7f5ad01f6900 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:48.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 >> 172.21.15.119:0/2008356808 conn(0x7f5ad00d2ec0 msgr2=0x7f5ad015e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:48.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 shutdown_connections 2023-11-08T19:25:48.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:48.856+0000 7f5acffff640 1 -- 172.21.15.119:0/2008356808 wait complete. 2023-11-08T19:25:48.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.079+0000 7fc66c44b640 1 Processor -- start 2023-11-08T19:25:49.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.080+0000 7fc66c44b640 1 -- start start 2023-11-08T19:25:49.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc66c44b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc664143600 0x7fc6640c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc66c44b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc66406d250 0x7fc66415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc66c44b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66415e440 con 0x7fc66415d9f0 2023-11-08T19:25:49.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc66c44b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6640c4f50 con 0x7fc664143600 2023-11-08T19:25:49.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc66c44b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6640c4960 con 0x7fc66406d250 2023-11-08T19:25:49.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc66a1e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc664143600 0x7fc6640c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc6699e6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc66415d9f0 0x7fc66406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39702/0 (socket says 172.21.15.119:39702) 2023-11-08T19:25:49.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.081+0000 7fc6699e6640 1 -- 172.21.15.119:0/1338897034 learned_addr learned my addr 172.21.15.119:0/1338897034 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc66a9e8640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc66406d250 0x7fc66415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc6691e5640 1 -- 172.21.15.119:0/1338897034 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103780144 0 0) 0x7fc66415e440 con 0x7fc66415d9f0 2023-11-08T19:25:49.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc6691e5640 1 -- 172.21.15.119:0/1338897034 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc650003520 con 0x7fc66415d9f0 2023-11-08T19:25:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc66a1e7640 1 -- 172.21.15.119:0/1338897034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc66406d250 msgr2=0x7fc66415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc66a1e7640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc66406d250 0x7fc66415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc66a1e7640 1 -- 172.21.15.119:0/1338897034 >> v1:172.21.15.119:6789/0 conn(0x7fc66415d9f0 legacy=0x7fc66406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc66a1e7640 1 -- 172.21.15.119:0/1338897034 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66415e720 con 0x7fc664143600 2023-11-08T19:25:49.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.082+0000 7fc66a9e8640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc66406d250 0x7fc66415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.083+0000 7fc66a1e7640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc664143600 0x7fc6640c5370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6600097a0 tx=0x7fc66002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359812d7cf9ab4b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc6691e5640 1 -- 172.21.15.119:0/1338897034 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660031780 con 0x7fc664143600 2023-11-08T19:25:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc6691e5640 1 -- 172.21.15.119:0/1338897034 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6600318e0 con 0x7fc664143600 2023-11-08T19:25:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc6691e5640 1 -- 172.21.15.119:0/1338897034 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660031c70 con 0x7fc664143600 2023-11-08T19:25:49.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 -- 172.21.15.119:0/1338897034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc664143600 msgr2=0x7fc6640c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc664143600 0x7fc6640c5370 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6600097a0 tx=0x7fc66002b820 comp rx=0 tx=0).stop 2023-11-08T19:25:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 -- 172.21.15.119:0/1338897034 shutdown_connections 2023-11-08T19:25:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc66406d250 0x7fc66415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 --2- 172.21.15.119:0/1338897034 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc664143600 0x7fc6640c5370 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 -- 172.21.15.119:0/1338897034 >> 172.21.15.119:0/1338897034 conn(0x7fc6640cb610 msgr2=0x7fc6640cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 -- 172.21.15.119:0/1338897034 shutdown_connections 2023-11-08T19:25:49.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.084+0000 7fc66c44b640 1 -- 172.21.15.119:0/1338897034 wait complete. 2023-11-08T19:25:49.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.086+0000 7fc66c44b640 1 Processor -- start 2023-11-08T19:25:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.086+0000 7fc66c44b640 1 -- start start 2023-11-08T19:25:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66c44b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc66406d250 0x7fc6641386e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66c44b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc664143600 0x7fc6641f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66c44b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 0x7fc6641efe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66c44b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641f6f10 con 0x7fc66415d9f0 2023-11-08T19:25:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66c44b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641f39f0 con 0x7fc66406d250 2023-11-08T19:25:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66c44b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6641f2de0 con 0x7fc664143600 2023-11-08T19:25:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66a9e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 0x7fc6641efe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.087+0000 7fc66a9e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 0x7fc6641efe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39974/0 (socket says 172.21.15.119:39974) 2023-11-08T19:25:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 -- 172.21.15.119:0/4211422584 learned_addr learned my addr 172.21.15.119:0/4211422584 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a1e7640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc66406d250 0x7fc6641386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc6699e6640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc664143600 0x7fc6641f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 -- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc664143600 msgr2=0x7fc6641f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc664143600 0x7fc6641f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 -- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc66406d250 msgr2=0x7fc6641386e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc66406d250 0x7fc6641386e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 -- 172.21.15.119:0/4211422584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc660009400 con 0x7fc66415d9f0 2023-11-08T19:25:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a1e7640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc66406d250 0x7fc6641386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66a9e8640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 0x7fc6641efe00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00ce00 tx=0x7fc65c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00d840 con 0x7fc66415d9f0 2023-11-08T19:25:49.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c00d9a0 con 0x7fc66415d9f0 2023-11-08T19:25:49.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c00dd50 con 0x7fc66415d9f0 2023-11-08T19:25:49.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.088+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6641f30c0 con 0x7fc66415d9f0 2023-11-08T19:25:49.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.089+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6641f3680 con 0x7fc66415d9f0 2023-11-08T19:25:49.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.089+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6640c5c60 con 0x7fc66415d9f0 2023-11-08T19:25:49.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.090+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc65c01e3b0 con 0x7fc66415d9f0 2023-11-08T19:25:49.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.091+0000 7fc65affd640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc64806ae70 0x7fc64806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.091+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fc65c08a340 con 0x7fc66415d9f0 2023-11-08T19:25:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.092+0000 7fc66a1e7640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc64806ae70 0x7fc64806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.092+0000 7fc66a1e7640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc64806ae70 0x7fc64806d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc660036000 tx=0x7fc66003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.095+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6640c5c60 con 0x7fc66415d9f0 2023-11-08T19:25:49.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.278+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc6640c5c60 con 0x7fc66415d9f0 2023-11-08T19:25:49.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.279+0000 7fc65affd640 1 -- 172.21.15.119:0/4211422584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc6640c5c60 con 0x7fc66415d9f0 2023-11-08T19:25:49.281 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:49.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc64806ae70 msgr2=0x7fc64806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc64806ae70 0x7fc64806d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc660036000 tx=0x7fc66003d040 comp rx=0 tx=0).stop 2023-11-08T19:25:49.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 msgr2=0x7fc6641efe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 0x7fc6641efe00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc65c00ce00 tx=0x7fc65c00a710 comp rx=0 tx=0).stop 2023-11-08T19:25:49.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 shutdown_connections 2023-11-08T19:25:49.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc64806ae70 0x7fc64806d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc66415d9f0 0x7fc6641efe00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.281+0000 7fc66c44b640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc664143600 0x7fc6641f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.282+0000 7fc66c44b640 1 --2- 172.21.15.119:0/4211422584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc66406d250 0x7fc6641386e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.282+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 >> 172.21.15.119:0/4211422584 conn(0x7fc6640cb610 msgr2=0x7fc664141970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.282+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 shutdown_connections 2023-11-08T19:25:49.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.282+0000 7fc66c44b640 1 -- 172.21.15.119:0/4211422584 wait complete. 2023-11-08T19:25:49.300 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:49.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.496+0000 7f6c9f34c640 1 Processor -- start 2023-11-08T19:25:49.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.497+0000 7f6c9f34c640 1 -- start start 2023-11-08T19:25:49.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.498+0000 7f6c9f34c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6c980cdb90 0x7f6c980ce270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.498+0000 7f6c9f34c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9806d9b0 con 0x7f6c98159e80 2023-11-08T19:25:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.498+0000 7f6c9f34c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c980ce020 con 0x7f6c9806f2c0 2023-11-08T19:25:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.498+0000 7f6c9f34c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9815e720 con 0x7f6c980cdb90 2023-11-08T19:25:49.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.498+0000 7f6c9d0e8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6c98159e80 0x7f6c980caba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39716/0 (socket says 172.21.15.119:39716) 2023-11-08T19:25:49.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.498+0000 7f6c9d0e8640 1 -- 172.21.15.119:0/1286151014 learned_addr learned my addr 172.21.15.119:0/1286151014 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:49.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 678519058 0 0) 0x7f6c9806d9b0 con 0x7f6c98159e80 2023-11-08T19:25:49.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c84003520 con 0x7f6c98159e80 2023-11-08T19:25:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/1286151014 >> v2:172.21.15.188:3300/0 conn(0x7f6c980cdb90 0x7f6c980ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 303986687 0 0) 0x7f6c84003520 con 0x7f6c98159e80 2023-11-08T19:25:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 >> v2:172.21.15.188:3300/0 conn(0x7f6c980cdb90 msgr2=0x7f6c980ce270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 --2- 172.21.15.119:0/1286151014 >> v2:172.21.15.188:3300/0 conn(0x7f6c980cdb90 0x7f6c980ce270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 >> v1:172.21.15.188:6789/0 conn(0x7f6c9806f2c0 legacy=0x7f6c9806e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9815ea00 con 0x7f6c98159e80 2023-11-08T19:25:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f6c94001670 con 0x7f6c98159e80 2023-11-08T19:25:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 >> v1:172.21.15.119:6789/0 conn(0x7f6c98159e80 legacy=0x7f6c980caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840039c0 0x7f6c84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c840047c0 0x7f6c84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c840051d0 0x7f6c8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9815e720 con 0x7f6c840047c0 2023-11-08T19:25:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8400c120 con 0x7f6c840039c0 2023-11-08T19:25:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.499+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8400c2e0 con 0x7f6c840051d0 2023-11-08T19:25:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c840047c0 0x7f6c84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9d8e9640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c840051d0 0x7f6c8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9d0e8640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840039c0 0x7f6c84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c840051d0 msgr2=0x7f6c8400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c840051d0 0x7f6c8400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840039c0 msgr2=0x7f6c84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840039c0 0x7f6c84003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/1286151014 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84003520 con 0x7f6c840047c0 2023-11-08T19:25:49.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.500+0000 7f6c9d8e9640 1 -- 172.21.15.119:0/1286151014 reap_dead start 2023-11-08T19:25:49.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.501+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c840047c0 0x7f6c84004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b5c0 tx=0x7f6c840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d1a534962603e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.501+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8802e730 con 0x7f6c840047c0 2023-11-08T19:25:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.501+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88032070 con 0x7f6c840047c0 2023-11-08T19:25:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.501+0000 7f6c8ffff640 1 -- 172.21.15.119:0/1286151014 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8802eab0 con 0x7f6c840047c0 2023-11-08T19:25:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.502+0000 7f6c9f34c640 1 -- 172.21.15.119:0/1286151014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c840047c0 msgr2=0x7f6c84004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.502+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c840047c0 0x7f6c84004c40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800b5c0 tx=0x7f6c840057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.502+0000 7f6c9f34c640 1 -- 172.21.15.119:0/1286151014 shutdown_connections 2023-11-08T19:25:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.502+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/1286151014 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c840047c0 0x7f6c84004c40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.502+0000 7f6c9f34c640 1 -- 172.21.15.119:0/1286151014 >> 172.21.15.119:0/1286151014 conn(0x7f6c980cfd20 msgr2=0x7f6c980d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.503+0000 7f6c9f34c640 1 -- 172.21.15.119:0/1286151014 shutdown_connections 2023-11-08T19:25:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.503+0000 7f6c9f34c640 1 -- 172.21.15.119:0/1286151014 wait complete. 2023-11-08T19:25:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.505+0000 7f6c9f34c640 1 Processor -- start 2023-11-08T19:25:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.505+0000 7f6c9f34c640 1 -- start start 2023-11-08T19:25:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9f34c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840047c0 0x7f6c98146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9f34c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 0x7f6c98154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9f34c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c98145020 0x7f6c981454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9f34c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981f3ef0 con 0x7f6c98144670 2023-11-08T19:25:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9f34c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c98147190 con 0x7f6c840047c0 2023-11-08T19:25:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9f34c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c981f3370 con 0x7f6c98145020 2023-11-08T19:25:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9c8e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 0x7f6c98154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9c8e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 0x7f6c98154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39988/0 (socket says 172.21.15.119:39988) 2023-11-08T19:25:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.507+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/3846801972 learned_addr learned my addr 172.21.15.119:0/3846801972 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9d0e8640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840047c0 0x7f6c98146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c98145020 msgr2=0x7f6c981454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c98145020 0x7f6c981454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840047c0 msgr2=0x7f6c98146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840047c0 0x7f6c98146c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9c8e7640 1 -- 172.21.15.119:0/3846801972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c981f3650 con 0x7f6c98144670 2023-11-08T19:25:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9c8e7640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 0x7f6c98154ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c8802e1a0 tx=0x7f6c88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88046070 con 0x7f6c98144670 2023-11-08T19:25:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88004020 con 0x7f6c98144670 2023-11-08T19:25:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88004190 con 0x7f6c98144670 2023-11-08T19:25:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.508+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c981f3930 con 0x7f6c98144670 2023-11-08T19:25:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.509+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c981459e0 con 0x7f6c98144670 2023-11-08T19:25:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.510+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c98074430 con 0x7f6c98144670 2023-11-08T19:25:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.510+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c880042f0 con 0x7f6c98144670 2023-11-08T19:25:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.512+0000 7f6c8dffb640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06ae70 0x7f6c7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.512+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6c880ab5c0 con 0x7f6c98144670 2023-11-08T19:25:49.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.512+0000 7f6c9d0e8640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06ae70 0x7f6c7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.513+0000 7f6c9d0e8640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06ae70 0x7f6c7c06d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6c94001050 tx=0x7f6c94001e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.516+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c88076290 con 0x7f6c98144670 2023-11-08T19:25:49.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.720+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c981f64d0 con 0x7f6c98144670 2023-11-08T19:25:49.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.723+0000 7f6c8dffb640 1 -- 172.21.15.119:0/3846801972 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f6c880798b0 con 0x7f6c98144670 2023-11-08T19:25:49.725 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:49.725 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06ae70 msgr2=0x7f6c7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06ae70 0x7f6c7c06d330 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6c94001050 tx=0x7f6c94001e10 comp rx=0 tx=0).stop 2023-11-08T19:25:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 msgr2=0x7f6c98154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 0x7f6c98154ff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6c8802e1a0 tx=0x7f6c88004730 comp rx=0 tx=0).stop 2023-11-08T19:25:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 shutdown_connections 2023-11-08T19:25:49.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06ae70 0x7f6c7c06d330 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c98145020 0x7f6c981454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c98144670 0x7f6c98154ff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 --2- 172.21.15.119:0/3846801972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c840047c0 0x7f6c98146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 >> 172.21.15.119:0/3846801972 conn(0x7f6c980cfd20 msgr2=0x7f6c98143df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 shutdown_connections 2023-11-08T19:25:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.726+0000 7f6c9f34c640 1 -- 172.21.15.119:0/3846801972 wait complete. 2023-11-08T19:25:49.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:49.745 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:49.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.942+0000 7f8efabfc640 1 Processor -- start 2023-11-08T19:25:49.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.942+0000 7f8efabfc640 1 -- start start 2023-11-08T19:25:49.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.943+0000 7f8efabfc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8ef415d4a0 0x7f8ef40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.943+0000 7f8efabfc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ef4161870 0x7f8ef40cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.943+0000 7f8efabfc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef415df50 con 0x7f8ef40cfa00 2023-11-08T19:25:49.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.943+0000 7f8efabfc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40d03d0 con 0x7f8ef4161870 2023-11-08T19:25:49.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.943+0000 7f8efabfc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef40d1e60 con 0x7f8ef415d4a0 2023-11-08T19:25:49.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ef4161870 0x7f8ef40cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8ef4161870 0x7f8ef40cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40004/0 (socket says 172.21.15.119:40004) 2023-11-08T19:25:49.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 -- 172.21.15.119:0/3500092675 learned_addr learned my addr 172.21.15.119:0/3500092675 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:49.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef8998640 1 --2- 172.21.15.119:0/3500092675 >> v2:172.21.15.188:3300/0 conn(0x7f8ef415d4a0 0x7f8ef40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 -- 172.21.15.119:0/3500092675 >> v2:172.21.15.188:3300/0 conn(0x7f8ef415d4a0 msgr2=0x7f8ef40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 --2- 172.21.15.119:0/3500092675 >> v2:172.21.15.188:3300/0 conn(0x7f8ef415d4a0 0x7f8ef40d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 -- 172.21.15.119:0/3500092675 >> v1:172.21.15.188:6789/0 conn(0x7f8ef40cfa00 legacy=0x7f8ef4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:49.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.944+0000 7f8ef9199640 1 -- 172.21.15.119:0/3500092675 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef40d2830 con 0x7f8ef4161870 2023-11-08T19:25:49.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.945+0000 7f8ef9199640 1 --2- 172.21.15.119:0/3500092675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef4161870 0x7f8ef40cfe90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4072830 tx=0x7f8eec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580987bbd337d13f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8ef37fe640 1 -- 172.21.15.119:0/3500092675 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec02c9f0 con 0x7f8ef4161870 2023-11-08T19:25:49.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8ef37fe640 1 -- 172.21.15.119:0/3500092675 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eec02cb50 con 0x7f8ef4161870 2023-11-08T19:25:49.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8ef37fe640 1 -- 172.21.15.119:0/3500092675 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eec0343f0 con 0x7f8ef4161870 2023-11-08T19:25:49.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 -- 172.21.15.119:0/3500092675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef4161870 msgr2=0x7f8ef40cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 --2- 172.21.15.119:0/3500092675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef4161870 0x7f8ef40cfe90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4072830 tx=0x7f8eec02be50 comp rx=0 tx=0).stop 2023-11-08T19:25:49.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 -- 172.21.15.119:0/3500092675 shutdown_connections 2023-11-08T19:25:49.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 --2- 172.21.15.119:0/3500092675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef4161870 0x7f8ef40cfe90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 --2- 172.21.15.119:0/3500092675 >> v2:172.21.15.188:3300/0 conn(0x7f8ef415d4a0 0x7f8ef40d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 -- 172.21.15.119:0/3500092675 >> 172.21.15.119:0/3500092675 conn(0x7f8ef406a9f0 msgr2=0x7f8ef406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:49.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 -- 172.21.15.119:0/3500092675 shutdown_connections 2023-11-08T19:25:49.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.946+0000 7f8efabfc640 1 -- 172.21.15.119:0/3500092675 wait complete. 2023-11-08T19:25:49.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.948+0000 7f8efabfc640 1 Processor -- start 2023-11-08T19:25:49.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.948+0000 7f8efabfc640 1 -- start start 2023-11-08T19:25:49.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8efabfc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef40cfa00 0x7f8ef41547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8efabfc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 0x7f8ef4156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8efabfc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ef4161870 0x7f8ef40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8efabfc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4154d00 con 0x7f8ef415d4a0 2023-11-08T19:25:49.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8efabfc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef4154e70 con 0x7f8ef40cfa00 2023-11-08T19:25:49.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8efabfc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef41550c0 con 0x7f8ef4161870 2023-11-08T19:25:49.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.949+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 0x7f8ef4156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 0x7f8ef4156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40016/0 (socket says 172.21.15.119:40016) 2023-11-08T19:25:49.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 -- 172.21.15.119:0/1921224942 learned_addr learned my addr 172.21.15.119:0/1921224942 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:49.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef8998640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef40cfa00 0x7f8ef41547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef9199640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ef4161870 0x7f8ef40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 -- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ef4161870 msgr2=0x7f8ef40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ef4161870 0x7f8ef40cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 -- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef40cfa00 msgr2=0x7f8ef41547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:49.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef40cfa00 0x7f8ef41547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:49.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 -- 172.21.15.119:0/1921224942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec009b30 con 0x7f8ef415d4a0 2023-11-08T19:25:49.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef9199640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ef4161870 0x7f8ef40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:49.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.950+0000 7f8ef3fff640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 0x7f8ef4156f80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c430 tx=0x7f8ee000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.951+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0011790 con 0x7f8ef415d4a0 2023-11-08T19:25:49.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.951+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00118f0 con 0x7f8ef415d4a0 2023-11-08T19:25:49.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.951+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef41553a0 con 0x7f8ef415d4a0 2023-11-08T19:25:49.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.951+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0011ce0 con 0x7f8ef415d4a0 2023-11-08T19:25:49.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.951+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef4155960 con 0x7f8ef415d4a0 2023-11-08T19:25:49.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.952+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef4074430 con 0x7f8ef415d4a0 2023-11-08T19:25:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.954+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ee001c460 con 0x7f8ef415d4a0 2023-11-08T19:25:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.955+0000 7f8ef17fa640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ed806ae70 0x7f8ed806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.955+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f8ee0089940 con 0x7f8ef415d4a0 2023-11-08T19:25:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.955+0000 7f8ef8998640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ed806ae70 0x7f8ed806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.956+0000 7f8ef8998640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ed806ae70 0x7f8ed806d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8006fd0 tx=0x7f8ee8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:49.960+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ee0054610 con 0x7f8ef415d4a0 2023-11-08T19:25:50.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.149+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ef40d10f0 con 0x7f8ef415d4a0 2023-11-08T19:25:50.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.150+0000 7f8ef17fa640 1 -- 172.21.15.119:0/1921224942 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ee0057c30 con 0x7f8ef415d4a0 2023-11-08T19:25:50.152 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:50.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ed806ae70 msgr2=0x7f8ed806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ed806ae70 0x7f8ed806d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8006fd0 tx=0x7f8ee8008040 comp rx=0 tx=0).stop 2023-11-08T19:25:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 msgr2=0x7f8ef4156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 0x7f8ef4156f80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c430 tx=0x7f8ee000c8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 shutdown_connections 2023-11-08T19:25:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8ed806ae70 0x7f8ed806d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8ef4161870 0x7f8ef40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8ef415d4a0 0x7f8ef4156f80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 --2- 172.21.15.119:0/1921224942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8ef40cfa00 0x7f8ef41547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.152+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 >> 172.21.15.119:0/1921224942 conn(0x7f8ef406a9f0 msgr2=0x7f8ef406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:50.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.153+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 shutdown_connections 2023-11-08T19:25:50.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.153+0000 7f8efabfc640 1 -- 172.21.15.119:0/1921224942 wait complete. 2023-11-08T19:25:50.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:50.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.363+0000 7fca8cb83640 1 Processor -- start 2023-11-08T19:25:50.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.363+0000 7fca8cb83640 1 -- start start 2023-11-08T19:25:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.365+0000 7fca8cb83640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca8815d9f0 0x7fca8815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.365+0000 7fca8cb83640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca8806cb40 0x7fca8806cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.365+0000 7fca8cb83640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca8806d4a0 con 0x7fca88143600 2023-11-08T19:25:50.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca8cb83640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca880c4f50 con 0x7fca8815d9f0 2023-11-08T19:25:50.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca8cb83640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca880c4960 con 0x7fca8806cb40 2023-11-08T19:25:50.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca8815d9f0 0x7fca8815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca8815d9f0 0x7fca8815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40020/0 (socket says 172.21.15.119:40020) 2023-11-08T19:25:50.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 -- 172.21.15.119:0/850063354 learned_addr learned my addr 172.21.15.119:0/850063354 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:50.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 -- 172.21.15.119:0/850063354 >> v2:172.21.15.188:3300/0 conn(0x7fca8806cb40 msgr2=0x7fca8806cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:50.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 --2- 172.21.15.119:0/850063354 >> v2:172.21.15.188:3300/0 conn(0x7fca8806cb40 0x7fca8806cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 -- 172.21.15.119:0/850063354 >> v1:172.21.15.188:6789/0 conn(0x7fca88143600 legacy=0x7fca880c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:50.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.366+0000 7fca7dd9b640 1 -- 172.21.15.119:0/850063354 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8815e440 con 0x7fca8815d9f0 2023-11-08T19:25:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.367+0000 7fca7dd9b640 1 --2- 172.21.15.119:0/850063354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8815d9f0 0x7fca8815de60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fca880cc650 tx=0x7fca7002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b8169fc8cc261e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.367+0000 7fca85d9b640 1 -- 172.21.15.119:0/850063354 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca700317e0 con 0x7fca8815d9f0 2023-11-08T19:25:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca85d9b640 1 -- 172.21.15.119:0/850063354 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca70031940 con 0x7fca8815d9f0 2023-11-08T19:25:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca85d9b640 1 -- 172.21.15.119:0/850063354 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70031cd0 con 0x7fca8815d9f0 2023-11-08T19:25:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 -- 172.21.15.119:0/850063354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8815d9f0 msgr2=0x7fca8815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 --2- 172.21.15.119:0/850063354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8815d9f0 0x7fca8815de60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fca880cc650 tx=0x7fca7002b880 comp rx=0 tx=0).stop 2023-11-08T19:25:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 -- 172.21.15.119:0/850063354 shutdown_connections 2023-11-08T19:25:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 --2- 172.21.15.119:0/850063354 >> v2:172.21.15.188:3300/0 conn(0x7fca8806cb40 0x7fca8806cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 --2- 172.21.15.119:0/850063354 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8815d9f0 0x7fca8815de60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 -- 172.21.15.119:0/850063354 >> 172.21.15.119:0/850063354 conn(0x7fca880cb610 msgr2=0x7fca880cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 -- 172.21.15.119:0/850063354 shutdown_connections 2023-11-08T19:25:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.368+0000 7fca8cb83640 1 -- 172.21.15.119:0/850063354 wait complete. 2023-11-08T19:25:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.370+0000 7fca8cb83640 1 Processor -- start 2023-11-08T19:25:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.370+0000 7fca8cb83640 1 -- start start 2023-11-08T19:25:50.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8cb83640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 0x7fca88140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8cb83640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca88143600 0x7fca880a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8cb83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8815d9f0 0x7fca880a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8cb83640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca88140b80 con 0x7fca8806cb40 2023-11-08T19:25:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8cb83640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca880aa000 con 0x7fca88143600 2023-11-08T19:25:50.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8cb83640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca880a9020 con 0x7fca8815d9f0 2023-11-08T19:25:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 0x7fca88140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 0x7fca88140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40028/0 (socket says 172.21.15.119:40028) 2023-11-08T19:25:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.371+0000 7fca8659c640 1 -- 172.21.15.119:0/3570173106 learned_addr learned my addr 172.21.15.119:0/3570173106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:50.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca7dd9b640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca88143600 0x7fca880a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca86d9d640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8815d9f0 0x7fca880a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8659c640 1 -- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8815d9f0 msgr2=0x7fca880a89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8659c640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8815d9f0 0x7fca880a89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8659c640 1 -- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca88143600 msgr2=0x7fca880a84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8659c640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca88143600 0x7fca880a84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8659c640 1 -- 172.21.15.119:0/3570173106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700094d0 con 0x7fca8806cb40 2023-11-08T19:25:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8659c640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 0x7fca88140640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fca7800c7d0 tx=0x7fca7800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78003d80 con 0x7fca8806cb40 2023-11-08T19:25:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca78005070 con 0x7fca8806cb40 2023-11-08T19:25:50.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78005420 con 0x7fca8806cb40 2023-11-08T19:25:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.372+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca880a9300 con 0x7fca8806cb40 2023-11-08T19:25:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.373+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca880aa530 con 0x7fca8806cb40 2023-11-08T19:25:50.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.374+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca880a9490 con 0x7fca8806cb40 2023-11-08T19:25:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.374+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca78005580 con 0x7fca8806cb40 2023-11-08T19:25:50.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.375+0000 7fca7f7fe640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca6806ae70 0x7fca6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.375+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fca78088ee0 con 0x7fca8806cb40 2023-11-08T19:25:50.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.376+0000 7fca7dd9b640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca6806ae70 0x7fca6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.376+0000 7fca7dd9b640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca6806ae70 0x7fca6806d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca70036000 tx=0x7fca7003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.380+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca78053ba0 con 0x7fca8806cb40 2023-11-08T19:25:50.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.605+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fca880aa750 con 0x7fca8806cb40 2023-11-08T19:25:50.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.605+0000 7fca7f7fe640 1 -- 172.21.15.119:0/3570173106 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fca780571c0 con 0x7fca8806cb40 2023-11-08T19:25:50.607 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:50.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca6806ae70 msgr2=0x7fca6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca6806ae70 0x7fca6806d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fca70036000 tx=0x7fca7003d040 comp rx=0 tx=0).stop 2023-11-08T19:25:50.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 msgr2=0x7fca88140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 0x7fca88140640 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fca7800c7d0 tx=0x7fca7800cc90 comp rx=0 tx=0).stop 2023-11-08T19:25:50.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 shutdown_connections 2023-11-08T19:25:50.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca6806ae70 0x7fca6806d330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca8815d9f0 0x7fca880a89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca88143600 0x7fca880a84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 --2- 172.21.15.119:0/3570173106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca8806cb40 0x7fca88140640 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 >> 172.21.15.119:0/3570173106 conn(0x7fca880cb610 msgr2=0x7fca880c84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:50.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 shutdown_connections 2023-11-08T19:25:50.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.608+0000 7fca8cb83640 1 -- 172.21.15.119:0/3570173106 wait complete. 2023-11-08T19:25:50.629 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:50.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.827+0000 7f0eaa3ce640 1 Processor -- start 2023-11-08T19:25:50.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.827+0000 7f0eaa3ce640 1 -- start start 2023-11-08T19:25:50.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0eaa3ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0ea4143600 0x7f0ea40c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0eaa3ce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ea406d250 0x7f0ea415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0eaa3ce640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea415e440 con 0x7f0ea415d9f0 2023-11-08T19:25:50.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0eaa3ce640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea40c4f50 con 0x7f0ea406d250 2023-11-08T19:25:50.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0eaa3ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea40c4960 con 0x7f0ea4143600 2023-11-08T19:25:50.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0ea896b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0ea406d250 0x7f0ea415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0e9bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0ea415d9f0 0x7f0ea406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39718/0 (socket says 172.21.15.119:39718) 2023-11-08T19:25:50.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0e9bfff640 1 -- 172.21.15.119:0/575165075 learned_addr learned my addr 172.21.15.119:0/575165075 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:50.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.829+0000 7f0ea3fff640 1 --2- 172.21.15.119:0/575165075 >> v2:172.21.15.188:3300/0 conn(0x7f0ea4143600 0x7f0ea40c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.830+0000 7f0ea896b640 1 -- 172.21.15.119:0/575165075 >> v2:172.21.15.188:3300/0 conn(0x7f0ea4143600 msgr2=0x7f0ea40c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:50.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.830+0000 7f0ea896b640 1 --2- 172.21.15.119:0/575165075 >> v2:172.21.15.188:3300/0 conn(0x7f0ea4143600 0x7f0ea40c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.830+0000 7f0ea896b640 1 -- 172.21.15.119:0/575165075 >> v1:172.21.15.119:6789/0 conn(0x7f0ea415d9f0 legacy=0x7f0ea406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.830+0000 7f0ea896b640 1 -- 172.21.15.119:0/575165075 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea415e720 con 0x7f0ea406d250 2023-11-08T19:25:50.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.830+0000 7f0ea37fe640 1 -- 172.21.15.119:0/575165075 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248195551 0 0) 0x7f0ea415e440 con 0x7f0ea415d9f0 2023-11-08T19:25:50.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.831+0000 7f0ea896b640 1 --2- 172.21.15.119:0/575165075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea406d250 0x7f0ea415de60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40c3f30 tx=0x7f0e9c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f1262dbc3efef2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.831+0000 7f0ea37fe640 1 -- 172.21.15.119:0/575165075 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c02cbe0 con 0x7f0ea406d250 2023-11-08T19:25:50.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.831+0000 7f0ea37fe640 1 -- 172.21.15.119:0/575165075 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9c02cd40 con 0x7f0ea406d250 2023-11-08T19:25:50.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.831+0000 7f0ea37fe640 1 -- 172.21.15.119:0/575165075 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9c034660 con 0x7f0ea406d250 2023-11-08T19:25:50.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/575165075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea406d250 msgr2=0x7f0ea415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/575165075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea406d250 0x7f0ea415de60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0ea40c3f30 tx=0x7f0e9c02be30 comp rx=0 tx=0).stop 2023-11-08T19:25:50.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/575165075 shutdown_connections 2023-11-08T19:25:50.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/575165075 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea406d250 0x7f0ea415de60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/575165075 >> v2:172.21.15.188:3300/0 conn(0x7f0ea4143600 0x7f0ea40c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/575165075 >> 172.21.15.119:0/575165075 conn(0x7f0ea40cb610 msgr2=0x7f0ea40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:50.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/575165075 shutdown_connections 2023-11-08T19:25:50.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.832+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/575165075 wait complete. 2023-11-08T19:25:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.834+0000 7f0eaa3ce640 1 Processor -- start 2023-11-08T19:25:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.834+0000 7f0eaa3ce640 1 -- start start 2023-11-08T19:25:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0eaa3ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea406d250 0x7f0ea41f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0eaa3ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 0x7f0ea4154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0eaa3ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea415d9f0 0x7f0ea406c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0eaa3ce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4151f10 con 0x7f0ea4143600 2023-11-08T19:25:50.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0eaa3ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4151c30 con 0x7f0ea406d250 2023-11-08T19:25:50.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0eaa3ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea41f5eb0 con 0x7f0ea415d9f0 2023-11-08T19:25:50.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0e9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 0x7f0ea4154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0e9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 0x7f0ea4154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40042/0 (socket says 172.21.15.119:40042) 2023-11-08T19:25:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0e9bfff640 1 -- 172.21.15.119:0/2694345150 learned_addr learned my addr 172.21.15.119:0/2694345150 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0ea3fff640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea406d250 0x7f0ea41f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0e9bfff640 1 -- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea415d9f0 msgr2=0x7f0ea406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0ea896b640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea415d9f0 0x7f0ea406c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.835+0000 7f0e9bfff640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea415d9f0 0x7f0ea406c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0e9bfff640 1 -- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea406d250 msgr2=0x7f0ea41f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0e9bfff640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea406d250 0x7f0ea41f6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0e9bfff640 1 -- 172.21.15.119:0/2694345150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c009a70 con 0x7f0ea4143600 2023-11-08T19:25:50.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0e9bfff640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 0x7f0ea4154ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c430 tx=0x7f0e8c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c0074e0 con 0x7f0ea4143600 2023-11-08T19:25:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c007660 con 0x7f0ea4143600 2023-11-08T19:25:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c004a20 con 0x7f0ea4143600 2023-11-08T19:25:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea41f6190 con 0x7f0ea4143600 2023-11-08T19:25:50.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.836+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea4155020 con 0x7f0ea4143600 2023-11-08T19:25:50.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.837+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea4074430 con 0x7f0ea4143600 2023-11-08T19:25:50.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.838+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e8c004b80 con 0x7f0ea4143600 2023-11-08T19:25:50.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.839+0000 7f0ea17fa640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e8406ae70 0x7f0e8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:50.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.839+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0e8c0898c0 con 0x7f0ea4143600 2023-11-08T19:25:50.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.839+0000 7f0ea3fff640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e8406ae70 0x7f0e8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:50.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.840+0000 7f0ea3fff640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e8406ae70 0x7f0e8406d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e94006fd0 tx=0x7f0e94008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:50.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:50.843+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e8c0544f0 con 0x7f0ea4143600 2023-11-08T19:25:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.032+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0ea413ed90 con 0x7f0ea4143600 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.033+0000 7f0ea17fa640 1 -- 172.21.15.119:0/2694345150 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0e8c057b10 con 0x7f0ea4143600 2023-11-08T19:25:51.036 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.035+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e8406ae70 msgr2=0x7f0e8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.035+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e8406ae70 0x7f0e8406d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0e94006fd0 tx=0x7f0e94008040 comp rx=0 tx=0).stop 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.035+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 msgr2=0x7f0ea4154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.035+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 0x7f0ea4154ae0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00c430 tx=0x7f0e8c00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 shutdown_connections 2023-11-08T19:25:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e8406ae70 0x7f0e8406d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0ea415d9f0 0x7f0ea406c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0ea4143600 0x7f0ea4154ae0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 --2- 172.21.15.119:0/2694345150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0ea406d250 0x7f0ea41f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 >> 172.21.15.119:0/2694345150 conn(0x7f0ea40cb610 msgr2=0x7f0ea40c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 shutdown_connections 2023-11-08T19:25:51.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.036+0000 7f0eaa3ce640 1 -- 172.21.15.119:0/2694345150 wait complete. 2023-11-08T19:25:51.057 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.272+0000 7f73db59e640 1 Processor -- start 2023-11-08T19:25:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.273+0000 7f73db59e640 1 -- start start 2023-11-08T19:25:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.274+0000 7f73db59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73dc160910 0x7f73dc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.274+0000 7f73db59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc0d0830 con 0x7f73dc160530 2023-11-08T19:25:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.274+0000 7f73db59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc0b27c0 con 0x7f73dc0ce5b0 2023-11-08T19:25:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.274+0000 7f73db59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc0b2aa0 con 0x7f73dc160910 2023-11-08T19:25:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.274+0000 7f73da59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f73dc160530 0x7f73dc0cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39728/0 (socket says 172.21.15.119:39728) 2023-11-08T19:25:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73da59c640 1 -- 172.21.15.119:0/1233936452 learned_addr learned my addr 172.21.15.119:0/1233936452 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73d9d9b640 1 --2- 172.21.15.119:0/1233936452 >> v2:172.21.15.119:3300/0 conn(0x7f73dc160910 0x7f73dc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73d959a640 1 -- 172.21.15.119:0/1233936452 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4119570787 0 0) 0x7f73dc0d0830 con 0x7f73dc160530 2023-11-08T19:25:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73d959a640 1 -- 172.21.15.119:0/1233936452 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c8003520 con 0x7f73dc160530 2023-11-08T19:25:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73d9d9b640 1 -- 172.21.15.119:0/1233936452 >> v1:172.21.15.188:6789/0 conn(0x7f73dc0ce5b0 legacy=0x7f73dc0b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73d9d9b640 1 -- 172.21.15.119:0/1233936452 >> v1:172.21.15.119:6789/0 conn(0x7f73dc160530 legacy=0x7f73dc0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.275+0000 7f73d9d9b640 1 -- 172.21.15.119:0/1233936452 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc0d1640 con 0x7f73dc160910 2023-11-08T19:25:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.276+0000 7f73d9d9b640 1 --2- 172.21.15.119:0/1233936452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160910 0x7f73dc0ce070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73c40097a0 tx=0x7f73c402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bd146fe01face6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73d959a640 1 -- 172.21.15.119:0/1233936452 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4030610 con 0x7f73dc160910 2023-11-08T19:25:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73d959a640 1 -- 172.21.15.119:0/1233936452 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c4030770 con 0x7f73dc160910 2023-11-08T19:25:51.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73d959a640 1 -- 172.21.15.119:0/1233936452 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4030b00 con 0x7f73dc160910 2023-11-08T19:25:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73db59e640 1 -- 172.21.15.119:0/1233936452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160910 msgr2=0x7f73dc0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73db59e640 1 --2- 172.21.15.119:0/1233936452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160910 0x7f73dc0ce070 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f73c40097a0 tx=0x7f73c402b960 comp rx=0 tx=0).stop 2023-11-08T19:25:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73db59e640 1 -- 172.21.15.119:0/1233936452 shutdown_connections 2023-11-08T19:25:51.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73db59e640 1 --2- 172.21.15.119:0/1233936452 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160910 0x7f73dc0ce070 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73db59e640 1 -- 172.21.15.119:0/1233936452 >> 172.21.15.119:0/1233936452 conn(0x7f73dc0d2ec0 msgr2=0x7f73dc0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.277+0000 7f73db59e640 1 -- 172.21.15.119:0/1233936452 shutdown_connections 2023-11-08T19:25:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.278+0000 7f73db59e640 1 -- 172.21.15.119:0/1233936452 wait complete. 2023-11-08T19:25:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.279+0000 7f73db59e640 1 Processor -- start 2023-11-08T19:25:51.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.279+0000 7f73db59e640 1 -- start start 2023-11-08T19:25:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.280+0000 7f73db59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73dc0ce5b0 0x7f73dc1f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.280+0000 7f73db59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 0x7f73dc1f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.280+0000 7f73db59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73dc160910 0x7f73dc1f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.280+0000 7f73db59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc0cceb0 con 0x7f73dc160530 2023-11-08T19:25:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.280+0000 7f73db59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc0b3090 con 0x7f73dc160910 2023-11-08T19:25:51.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.280+0000 7f73db59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc0b3370 con 0x7f73dc0ce5b0 2023-11-08T19:25:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 0x7f73dc1f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 0x7f73dc1f5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40064/0 (socket says 172.21.15.119:40064) 2023-11-08T19:25:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 -- 172.21.15.119:0/102169373 learned_addr learned my addr 172.21.15.119:0/102169373 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 -- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73dc0ce5b0 msgr2=0x7f73dc1f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73dc0ce5b0 0x7f73dc1f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73dad9d640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73dc160910 0x7f73dc1f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 -- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73dc160910 msgr2=0x7f73dc1f5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73dc160910 0x7f73dc1f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 -- 172.21.15.119:0/102169373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c4009400 con 0x7f73dc160530 2023-11-08T19:25:51.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.281+0000 7f73d9d9b640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 0x7f73dc1f5680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f73c40098c0 tx=0x7f73c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.282+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c403e070 con 0x7f73dc160530 2023-11-08T19:25:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.282+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c4030e10 con 0x7f73dc160530 2023-11-08T19:25:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.282+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc0b3650 con 0x7f73dc160530 2023-11-08T19:25:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.282+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4039c50 con 0x7f73dc160530 2023-11-08T19:25:51.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.282+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc1f3f40 con 0x7f73dc160530 2023-11-08T19:25:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.283+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc0b3b70 con 0x7f73dc160530 2023-11-08T19:25:51.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.284+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73c404b400 con 0x7f73dc160530 2023-11-08T19:25:51.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.285+0000 7f73d2ffd640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73bc06ae70 0x7f73bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.285+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f73c40abdb0 con 0x7f73dc160530 2023-11-08T19:25:51.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.285+0000 7f73da59c640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73bc06ae70 0x7f73bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.286+0000 7f73da59c640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73bc06ae70 0x7f73bc06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73cc007fd0 tx=0x7f73cc007b40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.289+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73dc0b3b70 con 0x7f73dc160530 2023-11-08T19:25:51.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.480+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f73dc0b3b70 con 0x7f73dc160530 2023-11-08T19:25:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.481+0000 7f73d2ffd640 1 -- 172.21.15.119:0/102169373 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f73dc0b3b70 con 0x7f73dc160530 2023-11-08T19:25:51.484 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.483+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73bc06ae70 msgr2=0x7f73bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.483+0000 7f73db59e640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73bc06ae70 0x7f73bc06d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f73cc007fd0 tx=0x7f73cc007b40 comp rx=0 tx=0).stop 2023-11-08T19:25:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.483+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 msgr2=0x7f73dc1f5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.483+0000 7f73db59e640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 0x7f73dc1f5680 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f73c40098c0 tx=0x7f73c40046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 shutdown_connections 2023-11-08T19:25:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73bc06ae70 0x7f73bc06d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73dc160910 0x7f73dc1f5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73dc160530 0x7f73dc1f5680 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 --2- 172.21.15.119:0/102169373 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73dc0ce5b0 0x7f73dc1f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 >> 172.21.15.119:0/102169373 conn(0x7f73dc0d2ec0 msgr2=0x7f73dc15e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 shutdown_connections 2023-11-08T19:25:51.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.484+0000 7f73db59e640 1 -- 172.21.15.119:0/102169373 wait complete. 2023-11-08T19:25:51.504 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:51.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.706+0000 7f7687fff640 1 Processor -- start 2023-11-08T19:25:51.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.706+0000 7f7687fff640 1 -- start start 2023-11-08T19:25:51.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f7687fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7688162d20 0x7f7688165110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f7687fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768806cff0 con 0x7f768815b0d0 2023-11-08T19:25:51.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f7687fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768806d2d0 con 0x7f76880ad7d0 2023-11-08T19:25:51.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f7687fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768806bd20 con 0x7f7688162d20 2023-11-08T19:25:51.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76877fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7688162d20 0x7f7688165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76877fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7688162d20 0x7f7688165110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40076/0 (socket says 172.21.15.119:40076) 2023-11-08T19:25:51.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76877fe640 1 -- 172.21.15.119:0/2471289243 learned_addr learned my addr 172.21.15.119:0/2471289243 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:51.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f7686ffd640 1 --1- 172.21.15.119:0/2471289243 >> v1:172.21.15.119:6789/0 conn(0x7f768815b0d0 0x7f768815b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39734/0 (socket says 172.21.15.119:39734) 2023-11-08T19:25:51.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76877fe640 1 -- 172.21.15.119:0/2471289243 >> v1:172.21.15.188:6789/0 conn(0x7f76880ad7d0 legacy=0x7f7688070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:51.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76877fe640 1 -- 172.21.15.119:0/2471289243 >> v1:172.21.15.119:6789/0 conn(0x7f768815b0d0 legacy=0x7f768815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76877fe640 1 -- 172.21.15.119:0/2471289243 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768806d760 con 0x7f7688162d20 2023-11-08T19:25:51.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.708+0000 7f76867fc640 1 -- 172.21.15.119:0/2471289243 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287481966 0 0) 0x7f768806cff0 con 0x7f768815b0d0 2023-11-08T19:25:51.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.709+0000 7f76877fe640 1 --2- 172.21.15.119:0/2471289243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f7688165110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f768000d150 tx=0x7f768002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd9fbc7ecda4ed88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f76867fc640 1 -- 172.21.15.119:0/2471289243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768002ea80 con 0x7f7688162d20 2023-11-08T19:25:51.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f76867fc640 1 -- 172.21.15.119:0/2471289243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768002ebe0 con 0x7f7688162d20 2023-11-08T19:25:51.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f76867fc640 1 -- 172.21.15.119:0/2471289243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680009600 con 0x7f7688162d20 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 -- 172.21.15.119:0/2471289243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 msgr2=0x7f7688165110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 --2- 172.21.15.119:0/2471289243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f7688165110 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f768000d150 tx=0x7f768002e0e0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 -- 172.21.15.119:0/2471289243 shutdown_connections 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 --2- 172.21.15.119:0/2471289243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f7688165110 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 -- 172.21.15.119:0/2471289243 >> 172.21.15.119:0/2471289243 conn(0x7f76880b0560 msgr2=0x7f76880b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 -- 172.21.15.119:0/2471289243 shutdown_connections 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.710+0000 7f7687fff640 1 -- 172.21.15.119:0/2471289243 wait complete. 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.712+0000 7f7687fff640 1 Processor -- start 2023-11-08T19:25:51.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.712+0000 7f7687fff640 1 -- start start 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f7687fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76880ad7d0 0x7f76880c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f7687fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768815b0d0 0x7f76880b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f7687fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f768806c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f7687fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76881f61e0 con 0x7f7688162d20 2023-11-08T19:25:51.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f7687fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76881f6480 con 0x7f76880ad7d0 2023-11-08T19:25:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f7687fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76881f6760 con 0x7f768815b0d0 2023-11-08T19:25:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.713+0000 7f76877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f768806c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f768806c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40086/0 (socket says 172.21.15.119:40086) 2023-11-08T19:25:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 -- 172.21.15.119:0/3246670102 learned_addr learned my addr 172.21.15.119:0/3246670102 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:51.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f7686ffd640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76880ad7d0 0x7f76880c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 -- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768815b0d0 msgr2=0x7f76880b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f767ffff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768815b0d0 0x7f76880b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768815b0d0 0x7f76880b1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 -- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76880ad7d0 msgr2=0x7f76880c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76880ad7d0 0x7f76880c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 -- 172.21.15.119:0/3246670102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768000bd40 con 0x7f7688162d20 2023-11-08T19:25:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f76877fe640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f768806c220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f768000d150 tx=0x7f7680004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.714+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680036800 con 0x7f7688162d20 2023-11-08T19:25:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.715+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7680036980 con 0x7f7688162d20 2023-11-08T19:25:51.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.715+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7680036d30 con 0x7f7688162d20 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.715+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76881f6a40 con 0x7f7688162d20 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.715+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768813fa80 con 0x7f7688162d20 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.716+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768804ece0 con 0x7f7688162d20 2023-11-08T19:25:51.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.717+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f768002ed50 con 0x7f7688162d20 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.718+0000 7f767f7fe640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f766806ae70 0x7f766806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.718+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f76800acbf0 con 0x7f7688162d20 2023-11-08T19:25:51.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.718+0000 7f7686ffd640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f766806ae70 0x7f766806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.719+0000 7f7686ffd640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f766806ae70 0x7f766806d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7678000f60 tx=0x7f7678009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:51.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.722+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76800778b0 con 0x7f7688162d20 2023-11-08T19:25:51.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.901+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f768804ece0 con 0x7f7688162d20 2023-11-08T19:25:51.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.903+0000 7f767f7fe640 1 -- 172.21.15.119:0/3246670102 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f768000b720 con 0x7f7688162d20 2023-11-08T19:25:51.906 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:51.906 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:51.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.905+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f766806ae70 msgr2=0x7f766806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.905+0000 7f7687fff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f766806ae70 0x7f766806d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7678000f60 tx=0x7f7678009040 comp rx=0 tx=0).stop 2023-11-08T19:25:51.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.905+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 msgr2=0x7f768806c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:51.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.905+0000 7f7687fff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f768806c220 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f768000d150 tx=0x7f7680004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:51.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.905+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 shutdown_connections 2023-11-08T19:25:51.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.905+0000 7f7687fff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f766806ae70 0x7f766806d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.906+0000 7f7687fff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7688162d20 0x7f768806c220 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.906+0000 7f7687fff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f768815b0d0 0x7f76880b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.906+0000 7f7687fff640 1 --2- 172.21.15.119:0/3246670102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f76880ad7d0 0x7f76880c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:51.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.906+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 >> 172.21.15.119:0/3246670102 conn(0x7f76880b0560 msgr2=0x7f76880c5280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:51.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.906+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 shutdown_connections 2023-11-08T19:25:51.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:51.906+0000 7f7687fff640 1 -- 172.21.15.119:0/3246670102 wait complete. 2023-11-08T19:25:51.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:51.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:52.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.125+0000 7f896b490640 1 Processor -- start 2023-11-08T19:25:52.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.125+0000 7f896b490640 1 -- start start 2023-11-08T19:25:52.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f896b490640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f896415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f896b490640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8964144960 0x7f89640cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f896b490640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f896415ea60 con 0x7f896415f1d0 2023-11-08T19:25:52.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f896b490640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89640c9150 con 0x7f8964144960 2023-11-08T19:25:52.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f896b490640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89640cb270 con 0x7f8964144580 2023-11-08T19:25:52.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f8968a2b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8964144960 0x7f89640cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f8968a2b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8964144960 0x7f89640cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40100/0 (socket says 172.21.15.119:40100) 2023-11-08T19:25:52.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.127+0000 7f8968a2b640 1 -- 172.21.15.119:0/111339866 learned_addr learned my addr 172.21.15.119:0/111339866 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:52.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.128+0000 7f8968a2b640 1 -- 172.21.15.119:0/111339866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 msgr2=0x7f896415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:52.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.128+0000 7f8968a2b640 1 --2- 172.21.15.119:0/111339866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f896415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.128+0000 7f8968a2b640 1 -- 172.21.15.119:0/111339866 >> v1:172.21.15.188:6789/0 conn(0x7f896415f1d0 legacy=0x7f89640c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:52.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.128+0000 7f8968a2b640 1 -- 172.21.15.119:0/111339866 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640c9360 con 0x7f8964144960 2023-11-08T19:25:52.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.129+0000 7f8968a2b640 1 --2- 172.21.15.119:0/111339866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f89640cad30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8954009870 tx=0x7f895402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2786d7df6da88cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:52.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.129+0000 7f895bfff640 1 -- 172.21.15.119:0/111339866 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540317c0 con 0x7f8964144960 2023-11-08T19:25:52.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.129+0000 7f895bfff640 1 -- 172.21.15.119:0/111339866 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8954031920 con 0x7f8964144960 2023-11-08T19:25:52.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.129+0000 7f895bfff640 1 -- 172.21.15.119:0/111339866 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8954031cb0 con 0x7f8964144960 2023-11-08T19:25:52.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 -- 172.21.15.119:0/111339866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 msgr2=0x7f89640cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 --2- 172.21.15.119:0/111339866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f89640cad30 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8954009870 tx=0x7f895402b860 comp rx=0 tx=0).stop 2023-11-08T19:25:52.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 -- 172.21.15.119:0/111339866 shutdown_connections 2023-11-08T19:25:52.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 --2- 172.21.15.119:0/111339866 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f89640cad30 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 --2- 172.21.15.119:0/111339866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f896415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 -- 172.21.15.119:0/111339866 >> 172.21.15.119:0/111339866 conn(0x7f89640cb610 msgr2=0x7f89640cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:52.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 -- 172.21.15.119:0/111339866 shutdown_connections 2023-11-08T19:25:52.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.130+0000 7f896b490640 1 -- 172.21.15.119:0/111339866 wait complete. 2023-11-08T19:25:52.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.132+0000 7f896b490640 1 Processor -- start 2023-11-08T19:25:52.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.132+0000 7f896b490640 1 -- start start 2023-11-08T19:25:52.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.133+0000 7f896b490640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f89640714d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.133+0000 7f896b490640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f896415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.133+0000 7f896b490640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f896415f1d0 0x7f896415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.133+0000 7f896b490640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641f4020 con 0x7f8964144960 2023-11-08T19:25:52.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.133+0000 7f896b490640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89641f4300 con 0x7f896415f1d0 2023-11-08T19:25:52.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.133+0000 7f896b490640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89641f45e0 con 0x7f8964144580 2023-11-08T19:25:52.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f896415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f896415a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40102/0 (socket says 172.21.15.119:40102) 2023-11-08T19:25:52.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 -- 172.21.15.119:0/2928995434 learned_addr learned my addr 172.21.15.119:0/2928995434 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:52.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f896922c640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f89640714d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 -- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 msgr2=0x7f89640714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8969a2d640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f896415f1d0 0x7f896415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f89640714d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 -- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f896415f1d0 msgr2=0x7f896415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f896415f1d0 0x7f896415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.134+0000 7f8968a2b640 1 -- 172.21.15.119:0/2928995434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540094d0 con 0x7f8964144960 2023-11-08T19:25:52.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.135+0000 7f8968a2b640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f896415a340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8954000c00 tx=0x7f89540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:52.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.135+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540329a0 con 0x7f8964144960 2023-11-08T19:25:52.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.135+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8954032b20 con 0x7f8964144960 2023-11-08T19:25:52.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.135+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540453f0 con 0x7f8964144960 2023-11-08T19:25:52.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.135+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89641f48c0 con 0x7f8964144960 2023-11-08T19:25:52.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.135+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896413c310 con 0x7f8964144960 2023-11-08T19:25:52.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.136+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89641f4a50 con 0x7f8964144960 2023-11-08T19:25:52.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.137+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f895404d3f0 con 0x7f8964144960 2023-11-08T19:25:52.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.139+0000 7f8959ffb640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f894806ae70 0x7f894806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.139+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f89540ab9a0 con 0x7f8964144960 2023-11-08T19:25:52.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.139+0000 7f896922c640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f894806ae70 0x7f894806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.140+0000 7f896922c640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f894806ae70 0x7f894806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89641ee580 tx=0x7f8960008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:52.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.143+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8954076660 con 0x7f8964144960 2023-11-08T19:25:52.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.332+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f89641f4de0 con 0x7f8964144960 2023-11-08T19:25:52.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.332+0000 7f8959ffb640 1 -- 172.21.15.119:0/2928995434 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f89641f4de0 con 0x7f8964144960 2023-11-08T19:25:52.335 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:52.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f894806ae70 msgr2=0x7f894806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f894806ae70 0x7f894806d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f89641ee580 tx=0x7f8960008040 comp rx=0 tx=0).stop 2023-11-08T19:25:52.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 msgr2=0x7f896415a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f896415a340 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f8954000c00 tx=0x7f89540046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 shutdown_connections 2023-11-08T19:25:52.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f894806ae70 0x7f894806d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f896415f1d0 0x7f896415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8964144960 0x7f896415a340 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 --2- 172.21.15.119:0/2928995434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8964144580 0x7f89640714d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 >> 172.21.15.119:0/2928995434 conn(0x7f89640cb610 msgr2=0x7f89640cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:52.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 shutdown_connections 2023-11-08T19:25:52.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.335+0000 7f896b490640 1 -- 172.21.15.119:0/2928995434 wait complete. 2023-11-08T19:25:52.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:52.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.566+0000 7fe354a51640 1 Processor -- start 2023-11-08T19:25:52.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.566+0000 7fe354a51640 1 -- start start 2023-11-08T19:25:52.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe354a51640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 0x7fe35015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe354a51640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35015ea60 con 0x7fe35015f1d0 2023-11-08T19:25:52.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe354a51640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3500c9360 con 0x7fe350144580 2023-11-08T19:25:52.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe354a51640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c9150 con 0x7fe350144960 2023-11-08T19:25:52.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe34ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe35015f1d0 0x7fe3500c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39746/0 (socket says 172.21.15.119:39746) 2023-11-08T19:25:52.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe34ed9d640 1 -- 172.21.15.119:0/1228552844 learned_addr learned my addr 172.21.15.119:0/1228552844 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:52.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.568+0000 7fe34dd9b640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 0x7fe35015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868949088 0 0) 0x7fe35015ea60 con 0x7fe35015f1d0 2023-11-08T19:25:52.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe338003520 con 0x7fe35015f1d0 2023-11-08T19:25:52.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1543719873 0 0) 0x7fe338003520 con 0x7fe35015f1d0 2023-11-08T19:25:52.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 msgr2=0x7fe35015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:52.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 0x7fe35015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 >> v1:172.21.15.188:6789/0 conn(0x7fe350144580 legacy=0x7fe3500cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500c9640 con 0x7fe35015f1d0 2023-11-08T19:25:52.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe3480019f0 con 0x7fe35015f1d0 2023-11-08T19:25:52.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 >> v1:172.21.15.119:6789/0 conn(0x7fe35015f1d0 legacy=0x7fe3500c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3380039a0 0x7fe338003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 0x7fe3380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe338004d00 0x7fe338005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c9150 con 0x7fe338004d00 2023-11-08T19:25:52.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33800bab0 con 0x7fe3380039a0 2023-11-08T19:25:52.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33800bc70 con 0x7fe350144960 2023-11-08T19:25:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.569+0000 7fe34ed9d640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe338004d00 0x7fe338005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34dd9b640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 0x7fe3380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34e59c640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3380039a0 0x7fe338003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34ed9d640 1 -- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 msgr2=0x7fe3380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:52.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34ed9d640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe350144960 0x7fe3380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34ed9d640 1 -- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3380039a0 msgr2=0x7fe338003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:52.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34ed9d640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3380039a0 0x7fe338003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.570+0000 7fe34ed9d640 1 -- 172.21.15.119:0/1228552844 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe338003520 con 0x7fe338004d00 2023-11-08T19:25:52.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.571+0000 7fe34ed9d640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe338004d00 0x7fe338005180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe34800a830 tx=0x7fe3380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f60b28b2d517113 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:52.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.571+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480072d0 con 0x7fe338004d00 2023-11-08T19:25:52.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.571+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34802f070 con 0x7fe338004d00 2023-11-08T19:25:52.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.571+0000 7fe34d59a640 1 -- 172.21.15.119:0/1228552844 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3480076b0 con 0x7fe338004d00 2023-11-08T19:25:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe354a51640 1 -- 172.21.15.119:0/1228552844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe338004d00 msgr2=0x7fe338005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe354a51640 1 --2- 172.21.15.119:0/1228552844 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe338004d00 0x7fe338005180 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe34800a830 tx=0x7fe3380057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe34ed9d640 1 -- 172.21.15.119:0/1228552844 reap_dead start 2023-11-08T19:25:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe354a51640 1 -- 172.21.15.119:0/1228552844 shutdown_connections 2023-11-08T19:25:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe354a51640 1 -- 172.21.15.119:0/1228552844 >> 172.21.15.119:0/1228552844 conn(0x7fe3500cb610 msgr2=0x7fe3500cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe354a51640 1 -- 172.21.15.119:0/1228552844 shutdown_connections 2023-11-08T19:25:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.572+0000 7fe354a51640 1 -- 172.21.15.119:0/1228552844 wait complete. 2023-11-08T19:25:52.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.574+0000 7fe354a51640 1 Processor -- start 2023-11-08T19:25:52.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.574+0000 7fe354a51640 1 -- start start 2023-11-08T19:25:52.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe354a51640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe338004d00 0x7fe3500c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe354a51640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 0x7fe3501f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe354a51640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3501f30c0 0x7fe3501f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe354a51640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501f3ab0 con 0x7fe3500b48e0 2023-11-08T19:25:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe354a51640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500b1ce0 con 0x7fe338004d00 2023-11-08T19:25:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe354a51640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501430e0 con 0x7fe3501f30c0 2023-11-08T19:25:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe34dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 0x7fe3501f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe34dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 0x7fe3501f2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40116/0 (socket says 172.21.15.119:40116) 2023-11-08T19:25:52.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe34dd9b640 1 -- 172.21.15.119:0/4226658364 learned_addr learned my addr 172.21.15.119:0/4226658364 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe34dd9b640 1 -- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3501f30c0 msgr2=0x7fe3501f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.575+0000 7fe34ed9d640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3501f30c0 0x7fe3501f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe34e59c640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe338004d00 0x7fe3500c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe34dd9b640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3501f30c0 0x7fe3501f3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe34dd9b640 1 -- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe338004d00 msgr2=0x7fe3500c6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe34dd9b640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe338004d00 0x7fe3500c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe34dd9b640 1 -- 172.21.15.119:0/4226658364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500c72b0 con 0x7fe3500b48e0 2023-11-08T19:25:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe34dd9b640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 0x7fe3501f2b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe344011600 tx=0x7fe344011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400f920 con 0x7fe3500b48e0 2023-11-08T19:25:52.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34400fa80 con 0x7fe3500b48e0 2023-11-08T19:25:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3500c7590 con 0x7fe3500b48e0 2023-11-08T19:25:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400fdf0 con 0x7fe3500b48e0 2023-11-08T19:25:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.576+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3500c6fa0 con 0x7fe3500b48e0 2023-11-08T19:25:52.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.577+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3500c7ab0 con 0x7fe3500b48e0 2023-11-08T19:25:52.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.578+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe344015460 con 0x7fe3500b48e0 2023-11-08T19:25:52.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.579+0000 7fe33effd640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe33006aea0 0x7fe33006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:52.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.579+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe344089f70 con 0x7fe3500b48e0 2023-11-08T19:25:52.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.580+0000 7fe34e59c640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe33006aea0 0x7fe33006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:52.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.580+0000 7fe34e59c640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe33006aea0 0x7fe33006d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe340007fd0 tx=0x7fe340009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:52.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.583+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3500c7ab0 con 0x7fe3500b48e0 2023-11-08T19:25:52.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.776+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3500c7ab0 con 0x7fe3500b48e0 2023-11-08T19:25:52.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.781+0000 7fe33effd640 1 -- 172.21.15.119:0/4226658364 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe3500c7ab0 con 0x7fe3500b48e0 2023-11-08T19:25:52.784 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:52.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe33006aea0 msgr2=0x7fe33006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe33006aea0 0x7fe33006d360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe340007fd0 tx=0x7fe340009040 comp rx=0 tx=0).stop 2023-11-08T19:25:52.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 msgr2=0x7fe3501f2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:52.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 0x7fe3501f2b80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe344011600 tx=0x7fe344011ac0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 shutdown_connections 2023-11-08T19:25:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe33006aea0 0x7fe33006d360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3501f30c0 0x7fe3501f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3500b48e0 0x7fe3501f2b80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 --2- 172.21.15.119:0/4226658364 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe338004d00 0x7fe3500c6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:52.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 >> 172.21.15.119:0/4226658364 conn(0x7fe3500cb610 msgr2=0x7fe3500cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:52.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 shutdown_connections 2023-11-08T19:25:52.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:52.784+0000 7fe354a51640 1 -- 172.21.15.119:0/4226658364 wait complete. 2023-11-08T19:25:52.806 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.018+0000 7f3998ec3640 1 Processor -- start 2023-11-08T19:25:53.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.019+0000 7f3998ec3640 1 -- start start 2023-11-08T19:25:53.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3998ec3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3994160530 0x7f39940cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3998ec3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3994160910 0x7f39940cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3998ec3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39940b27c0 con 0x7f39940d3ac0 2023-11-08T19:25:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3998ec3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940b2aa0 con 0x7f3994160910 2023-11-08T19:25:53.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3998ec3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39940d1640 con 0x7f3994160530 2023-11-08T19:25:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3992ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3994160910 0x7f39940cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3992ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3994160910 0x7f39940cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40120/0 (socket says 172.21.15.119:40120) 2023-11-08T19:25:53.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.020+0000 7f3992ffd640 1 -- 172.21.15.119:0/2781797232 learned_addr learned my addr 172.21.15.119:0/2781797232 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.021+0000 7f3992ffd640 1 -- 172.21.15.119:0/2781797232 >> v2:172.21.15.188:3300/0 conn(0x7f3994160530 msgr2=0x7f39940cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.021+0000 7f3992ffd640 1 --2- 172.21.15.119:0/2781797232 >> v2:172.21.15.188:3300/0 conn(0x7f3994160530 0x7f39940cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.021+0000 7f3992ffd640 1 -- 172.21.15.119:0/2781797232 >> v1:172.21.15.188:6789/0 conn(0x7f39940d3ac0 legacy=0x7f39940cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:53.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.021+0000 7f3992ffd640 1 -- 172.21.15.119:0/2781797232 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39940d1920 con 0x7f3994160910 2023-11-08T19:25:53.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.022+0000 7f3992ffd640 1 --2- 172.21.15.119:0/2781797232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3994160910 0x7f39940cdea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f399406a7d0 tx=0x7f397c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8818e55214cb642 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.022+0000 7f39927fc640 1 -- 172.21.15.119:0/2781797232 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c031870 con 0x7f3994160910 2023-11-08T19:25:53.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.022+0000 7f39927fc640 1 -- 172.21.15.119:0/2781797232 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c0319d0 con 0x7f3994160910 2023-11-08T19:25:53.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.022+0000 7f39927fc640 1 -- 172.21.15.119:0/2781797232 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c031d60 con 0x7f3994160910 2023-11-08T19:25:53.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.022+0000 7f3998ec3640 1 -- 172.21.15.119:0/2781797232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3994160910 msgr2=0x7f39940cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.022+0000 7f3998ec3640 1 --2- 172.21.15.119:0/2781797232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3994160910 0x7f39940cdea0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f399406a7d0 tx=0x7f397c02b910 comp rx=0 tx=0).stop 2023-11-08T19:25:53.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.023+0000 7f3998ec3640 1 -- 172.21.15.119:0/2781797232 shutdown_connections 2023-11-08T19:25:53.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.023+0000 7f3998ec3640 1 --2- 172.21.15.119:0/2781797232 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3994160910 0x7f39940cdea0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.023+0000 7f3998ec3640 1 --2- 172.21.15.119:0/2781797232 >> v2:172.21.15.188:3300/0 conn(0x7f3994160530 0x7f39940cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.023+0000 7f3998ec3640 1 -- 172.21.15.119:0/2781797232 >> 172.21.15.119:0/2781797232 conn(0x7f39940d2ec0 msgr2=0x7f39940d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.023+0000 7f3998ec3640 1 -- 172.21.15.119:0/2781797232 shutdown_connections 2023-11-08T19:25:53.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.023+0000 7f3998ec3640 1 -- 172.21.15.119:0/2781797232 wait complete. 2023-11-08T19:25:53.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.024+0000 7f3998ec3640 1 Processor -- start 2023-11-08T19:25:53.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.024+0000 7f3998ec3640 1 -- start start 2023-11-08T19:25:53.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.025+0000 7f3998ec3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 0x7f39940c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.025+0000 7f3998ec3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3994160530 0x7f3994155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.025+0000 7f3998ec3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3994160910 0x7f39940b5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.025+0000 7f3998ec3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399415c910 con 0x7f39940d3ac0 2023-11-08T19:25:53.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.025+0000 7f3998ec3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39941559c0 con 0x7f3994160530 2023-11-08T19:25:53.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.025+0000 7f3998ec3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39940b8aa0 con 0x7f3994160910 2023-11-08T19:25:53.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 0x7f39940c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 0x7f39940c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40130/0 (socket says 172.21.15.119:40130) 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 -- 172.21.15.119:0/548469368 learned_addr learned my addr 172.21.15.119:0/548469368 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f3992ffd640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3994160530 0x7f3994155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f3993fff640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3994160910 0x7f39940b5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 -- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3994160910 msgr2=0x7f39940b5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3994160910 0x7f39940b5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 -- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3994160530 msgr2=0x7f3994155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3994160530 0x7f3994155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.026+0000 7f39937fe640 1 -- 172.21.15.119:0/548469368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0094d0 con 0x7f39940d3ac0 2023-11-08T19:25:53.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.027+0000 7f39937fe640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 0x7f39940c6190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f398400ba50 tx=0x7f398400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.027+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400cbe0 con 0x7f39940d3ac0 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.027+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400cd60 con 0x7f39940d3ac0 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.027+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39940b8200 con 0x7f39940d3ac0 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.027+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984014680 con 0x7f39940d3ac0 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.027+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39940b87c0 con 0x7f39940d3ac0 2023-11-08T19:25:53.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.028+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399404ece0 con 0x7f39940d3ac0 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.029+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39840147e0 con 0x7f39940d3ac0 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.030+0000 7f398bfff640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f397406aea0 0x7f397406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.030+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3984089920 con 0x7f39940d3ac0 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.031+0000 7f3992ffd640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f397406aea0 0x7f397406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.031+0000 7f3992ffd640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f397406aea0 0x7f397406d360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f397c036000 tx=0x7f397c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.035+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39840545e0 con 0x7f39940d3ac0 2023-11-08T19:25:53.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.217+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39940b5430 con 0x7f39940d3ac0 2023-11-08T19:25:53.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.218+0000 7f398bfff640 1 -- 172.21.15.119:0/548469368 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3984019020 con 0x7f39940d3ac0 2023-11-08T19:25:53.221 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:53.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f397406aea0 msgr2=0x7f397406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f397406aea0 0x7f397406d360 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f397c036000 tx=0x7f397c03c040 comp rx=0 tx=0).stop 2023-11-08T19:25:53.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 msgr2=0x7f39940c6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 0x7f39940c6190 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f398400ba50 tx=0x7f398400bf10 comp rx=0 tx=0).stop 2023-11-08T19:25:53.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 shutdown_connections 2023-11-08T19:25:53.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f397406aea0 0x7f397406d360 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3994160910 0x7f39940b5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3994160530 0x7f3994155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 --2- 172.21.15.119:0/548469368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f39940d3ac0 0x7f39940c6190 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 >> 172.21.15.119:0/548469368 conn(0x7f39940d2ec0 msgr2=0x7f399406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 shutdown_connections 2023-11-08T19:25:53.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.221+0000 7f3998ec3640 1 -- 172.21.15.119:0/548469368 wait complete. 2023-11-08T19:25:53.241 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:53.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.445+0000 7f73191e2640 1 Processor -- start 2023-11-08T19:25:53.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.445+0000 7f73191e2640 1 -- start start 2023-11-08T19:25:53.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.446+0000 7f73191e2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f731415b2b0 0x7f7314071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.446+0000 7f73191e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f731415b690 0x7f73140ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.446+0000 7f73191e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73140cf5e0 0x7f73140ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.446+0000 7f73191e2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73140c9440 con 0x7f731415b690 2023-11-08T19:25:53.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.446+0000 7f73191e2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73140c9690 con 0x7f731415b2b0 2023-11-08T19:25:53.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.446+0000 7f73191e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c9970 con 0x7f73140cf5e0 2023-11-08T19:25:53.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f731415b690 0x7f73140ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f731415b690 0x7f73140ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40144/0 (socket says 172.21.15.119:40144) 2023-11-08T19:25:53.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 -- 172.21.15.119:0/1218430743 learned_addr learned my addr 172.21.15.119:0/1218430743 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:53.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f7313fff640 1 --2- 172.21.15.119:0/1218430743 >> v2:172.21.15.188:3300/0 conn(0x7f731415b2b0 0x7f7314071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 -- 172.21.15.119:0/1218430743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73140cf5e0 msgr2=0x7f73140ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:53.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73189e1640 1 --2- 172.21.15.119:0/1218430743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73140cf5e0 0x7f73140ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 --2- 172.21.15.119:0/1218430743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73140cf5e0 0x7f73140ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 -- 172.21.15.119:0/1218430743 >> v2:172.21.15.188:3300/0 conn(0x7f731415b2b0 msgr2=0x7f7314071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:53.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 --2- 172.21.15.119:0/1218430743 >> v2:172.21.15.188:3300/0 conn(0x7f731415b2b0 0x7f7314071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f73137fe640 1 -- 172.21.15.119:0/1218430743 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73140c9c50 con 0x7f731415b690 2023-11-08T19:25:53.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.447+0000 7f7313fff640 1 --2- 172.21.15.119:0/1218430743 >> v2:172.21.15.188:3300/0 conn(0x7f731415b2b0 0x7f7314071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:25:53.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.448+0000 7f73137fe640 1 --2- 172.21.15.119:0/1218430743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731415b690 0x7f73140ca450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f731406b3a0 tx=0x7f72fc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=105f45fc7c5ac2ef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.449+0000 7f7312ffd640 1 -- 172.21.15.119:0/1218430743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc0317d0 con 0x7f731415b690 2023-11-08T19:25:53.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.449+0000 7f7312ffd640 1 -- 172.21.15.119:0/1218430743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc031930 con 0x7f731415b690 2023-11-08T19:25:53.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.449+0000 7f7312ffd640 1 -- 172.21.15.119:0/1218430743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc031cc0 con 0x7f731415b690 2023-11-08T19:25:53.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.449+0000 7f73191e2640 1 -- 172.21.15.119:0/1218430743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731415b690 msgr2=0x7f73140ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.449+0000 7f73191e2640 1 --2- 172.21.15.119:0/1218430743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731415b690 0x7f73140ca450 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f731406b3a0 tx=0x7f72fc02b870 comp rx=0 tx=0).stop 2023-11-08T19:25:53.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 -- 172.21.15.119:0/1218430743 shutdown_connections 2023-11-08T19:25:53.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 --2- 172.21.15.119:0/1218430743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73140cf5e0 0x7f73140ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 --2- 172.21.15.119:0/1218430743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f731415b690 0x7f73140ca450 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 --2- 172.21.15.119:0/1218430743 >> v2:172.21.15.188:3300/0 conn(0x7f731415b2b0 0x7f7314071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 -- 172.21.15.119:0/1218430743 >> 172.21.15.119:0/1218430743 conn(0x7f73140ceac0 msgr2=0x7f73140ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 -- 172.21.15.119:0/1218430743 shutdown_connections 2023-11-08T19:25:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.450+0000 7f73191e2640 1 -- 172.21.15.119:0/1218430743 wait complete. 2023-11-08T19:25:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.452+0000 7f73191e2640 1 Processor -- start 2023-11-08T19:25:53.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.452+0000 7f73191e2640 1 -- start start 2023-11-08T19:25:53.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f73191e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 0x7f73140c28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f73191e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f731415b2b0 0x7f7314157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f73191e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731415b690 0x7f73141f6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f73191e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73140c34b0 con 0x7f73140cf5e0 2023-11-08T19:25:53.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f73191e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7314158170 con 0x7f731415b690 2023-11-08T19:25:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f73191e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73141f66d0 con 0x7f731415b2b0 2023-11-08T19:25:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f7313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 0x7f73140c28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f7313fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 0x7f73140c28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40158/0 (socket says 172.21.15.119:40158) 2023-11-08T19:25:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.453+0000 7f7313fff640 1 -- 172.21.15.119:0/3442881848 learned_addr learned my addr 172.21.15.119:0/3442881848 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f73137fe640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f731415b2b0 0x7f7314157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f73189e1640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731415b690 0x7f73141f6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7313fff640 1 -- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f731415b2b0 msgr2=0x7f7314157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7313fff640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f731415b2b0 0x7f7314157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7313fff640 1 -- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731415b690 msgr2=0x7f73141f6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7313fff640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731415b690 0x7f73141f6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7313fff640 1 -- 172.21.15.119:0/3442881848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc0094d0 con 0x7f73140cf5e0 2023-11-08T19:25:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7313fff640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 0x7f73140c28b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f730400e810 tx=0x7f730400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.454+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304004030 con 0x7f73140cf5e0 2023-11-08T19:25:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.455+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7304004190 con 0x7f73140cf5e0 2023-11-08T19:25:53.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.455+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7304004520 con 0x7f73140cf5e0 2023-11-08T19:25:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.455+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73141f70a0 con 0x7f73140cf5e0 2023-11-08T19:25:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.455+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73140c31d0 con 0x7f73140cf5e0 2023-11-08T19:25:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.456+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f731404ece0 con 0x7f73140cf5e0 2023-11-08T19:25:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.457+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f730400c670 con 0x7f73140cf5e0 2023-11-08T19:25:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.457+0000 7f7310ff9640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f72f406ae70 0x7f72f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.458+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f73040895d0 con 0x7f73140cf5e0 2023-11-08T19:25:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.458+0000 7f73137fe640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f72f406ae70 0x7f72f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.458+0000 7f73137fe640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f72f406ae70 0x7f72f406d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72fc036000 tx=0x7f72fc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.461+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7304054330 con 0x7f73140cf5e0 2023-11-08T19:25:53.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.651+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f731415a710 con 0x7f73140cf5e0 2023-11-08T19:25:53.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.652+0000 7f7310ff9640 1 -- 172.21.15.119:0/3442881848 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7304057950 con 0x7f73140cf5e0 2023-11-08T19:25:53.655 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:53.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f72f406ae70 msgr2=0x7f72f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f72f406ae70 0x7f72f406d330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f72fc036000 tx=0x7f72fc03d040 comp rx=0 tx=0).stop 2023-11-08T19:25:53.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 msgr2=0x7f73140c28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 0x7f73140c28b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f730400e810 tx=0x7f730400ecd0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 shutdown_connections 2023-11-08T19:25:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f72f406ae70 0x7f72f406d330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f731415b690 0x7f73141f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f731415b2b0 0x7f7314157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 --2- 172.21.15.119:0/3442881848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73140cf5e0 0x7f73140c28b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 >> 172.21.15.119:0/3442881848 conn(0x7f73140ceac0 msgr2=0x7f73141598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 shutdown_connections 2023-11-08T19:25:53.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.656+0000 7f73191e2640 1 -- 172.21.15.119:0/3442881848 wait complete. 2023-11-08T19:25:53.677 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:53.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.879+0000 7fef4c690640 1 Processor -- start 2023-11-08T19:25:53.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.880+0000 7fef4c690640 1 -- start start 2023-11-08T19:25:53.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.881+0000 7fef4c690640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef441447a0 0x7fef4415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.881+0000 7fef4c690640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef440cad30 0x7fef4406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.881+0000 7fef4c690640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef440c3a50 con 0x7fef440cb170 2023-11-08T19:25:53.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.881+0000 7fef4c690640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef44144b80 con 0x7fef440cad30 2023-11-08T19:25:53.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.881+0000 7fef4c690640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef440c9150 con 0x7fef441447a0 2023-11-08T19:25:53.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.882+0000 7fef4ac2d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fef440cb170 0x7fef4406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:45804/0 (socket says 172.21.15.119:45804) 2023-11-08T19:25:53.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.882+0000 7fef4a42c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef441447a0 0x7fef4415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.882+0000 7fef4ac2d640 1 -- 172.21.15.119:0/2660098319 learned_addr learned my addr 172.21.15.119:0/2660098319 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:53.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.882+0000 7fef49c2b640 1 --2- 172.21.15.119:0/2660098319 >> v2:172.21.15.188:3300/0 conn(0x7fef440cad30 0x7fef4406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.883+0000 7fef4a42c640 1 -- 172.21.15.119:0/2660098319 >> v2:172.21.15.188:3300/0 conn(0x7fef440cad30 msgr2=0x7fef4406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:53.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.883+0000 7fef4942a640 1 -- 172.21.15.119:0/2660098319 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257496765 0 0) 0x7fef440c3a50 con 0x7fef440cb170 2023-11-08T19:25:53.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.883+0000 7fef4a42c640 1 --2- 172.21.15.119:0/2660098319 >> v2:172.21.15.188:3300/0 conn(0x7fef440cad30 0x7fef4406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.883+0000 7fef4a42c640 1 -- 172.21.15.119:0/2660098319 >> v1:172.21.15.188:6789/0 conn(0x7fef440cb170 legacy=0x7fef4406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.883+0000 7fef4a42c640 1 -- 172.21.15.119:0/2660098319 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef440c9410 con 0x7fef441447a0 2023-11-08T19:25:53.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.884+0000 7fef4a42c640 1 --2- 172.21.15.119:0/2660098319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef441447a0 0x7fef4415f4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef400097a0 tx=0x7fef4002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=505694be072591ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4942a640 1 -- 172.21.15.119:0/2660098319 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4002d040 con 0x7fef441447a0 2023-11-08T19:25:53.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4942a640 1 -- 172.21.15.119:0/2660098319 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef40031810 con 0x7fef441447a0 2023-11-08T19:25:53.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4942a640 1 -- 172.21.15.119:0/2660098319 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef40031ba0 con 0x7fef441447a0 2023-11-08T19:25:53.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 -- 172.21.15.119:0/2660098319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef441447a0 msgr2=0x7fef4415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 --2- 172.21.15.119:0/2660098319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef441447a0 0x7fef4415f4c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fef400097a0 tx=0x7fef4002b8b0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 -- 172.21.15.119:0/2660098319 shutdown_connections 2023-11-08T19:25:53.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 --2- 172.21.15.119:0/2660098319 >> v2:172.21.15.188:3300/0 conn(0x7fef440cad30 0x7fef4406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 --2- 172.21.15.119:0/2660098319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef441447a0 0x7fef4415f4c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 -- 172.21.15.119:0/2660098319 >> 172.21.15.119:0/2660098319 conn(0x7fef440cb610 msgr2=0x7fef440cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:53.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 -- 172.21.15.119:0/2660098319 shutdown_connections 2023-11-08T19:25:53.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.885+0000 7fef4c690640 1 -- 172.21.15.119:0/2660098319 wait complete. 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.887+0000 7fef4c690640 1 Processor -- start 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.887+0000 7fef4c690640 1 -- start start 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4c690640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef440cad30 0x7fef440cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4c690640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef440cb170 0x7fef44151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4c690640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 0x7fef441523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4c690640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef440ce010 con 0x7fef441447a0 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4c690640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef440b46f0 con 0x7fef440cb170 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4c690640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef440cda50 con 0x7fef440cad30 2023-11-08T19:25:53.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.888+0000 7fef4ac2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 0x7fef441523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 0x7fef441523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40162/0 (socket says 172.21.15.119:40162) 2023-11-08T19:25:53.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 -- 172.21.15.119:0/138258114 learned_addr learned my addr 172.21.15.119:0/138258114 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:53.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef49c2b640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef440cb170 0x7fef44151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4a42c640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef440cad30 0x7fef440cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 -- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef440cad30 msgr2=0x7fef440cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef440cad30 0x7fef440cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 -- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef440cb170 msgr2=0x7fef44151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:53.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef440cb170 0x7fef44151e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:53.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 -- 172.21.15.119:0/138258114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef40009400 con 0x7fef441447a0 2023-11-08T19:25:53.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef4ac2d640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 0x7fef441523d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00c7b0 tx=0x7fef3c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00a640 con 0x7fef441447a0 2023-11-08T19:25:53.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00a7a0 con 0x7fef441447a0 2023-11-08T19:25:53.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.889+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00ab90 con 0x7fef441447a0 2023-11-08T19:25:53.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.890+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef440cdd30 con 0x7fef441447a0 2023-11-08T19:25:53.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.890+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef440b21c0 con 0x7fef441447a0 2023-11-08T19:25:53.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.891+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef440b23e0 con 0x7fef441447a0 2023-11-08T19:25:53.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.892+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef3c00acf0 con 0x7fef441447a0 2023-11-08T19:25:53.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.893+0000 7fef3affd640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef2806ae70 0x7fef2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:53.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.893+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fef3c088f60 con 0x7fef441447a0 2023-11-08T19:25:53.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.893+0000 7fef4a42c640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef2806ae70 0x7fef2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:53.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.894+0000 7fef4a42c640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef2806ae70 0x7fef2806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef40036000 tx=0x7fef4003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:53.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:53.899+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef3c053c30 con 0x7fef441447a0 2023-11-08T19:25:54.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.094+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef44074430 con 0x7fef441447a0 2023-11-08T19:25:54.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.095+0000 7fef3affd640 1 -- 172.21.15.119:0/138258114 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fef3c018070 con 0x7fef441447a0 2023-11-08T19:25:54.098 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:54.098 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:54.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef2806ae70 msgr2=0x7fef2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef2806ae70 0x7fef2806d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fef40036000 tx=0x7fef4003c040 comp rx=0 tx=0).stop 2023-11-08T19:25:54.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 msgr2=0x7fef441523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 0x7fef441523d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fef3c00c7b0 tx=0x7fef3c00cc70 comp rx=0 tx=0).stop 2023-11-08T19:25:54.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 shutdown_connections 2023-11-08T19:25:54.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef2806ae70 0x7fef2806d330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef441447a0 0x7fef441523d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef440cb170 0x7fef44151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 --2- 172.21.15.119:0/138258114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef440cad30 0x7fef440cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 >> 172.21.15.119:0/138258114 conn(0x7fef440cb610 msgr2=0x7fef4406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 shutdown_connections 2023-11-08T19:25:54.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.098+0000 7fef4c690640 1 -- 172.21.15.119:0/138258114 wait complete. 2023-11-08T19:25:54.119 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:54.120 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:54.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.329+0000 7f81a004f640 1 Processor -- start 2023-11-08T19:25:54.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.329+0000 7f81a004f640 1 -- start start 2023-11-08T19:25:54.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f81a004f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8198155550 0x7f81980702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f81a004f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f819815f220 0x7f819815c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f81a004f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819806b0f0 con 0x7f819806b580 2023-11-08T19:25:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f81a004f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819806aac0 con 0x7f819815f220 2023-11-08T19:25:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f81a004f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8198072140 con 0x7f8198155550 2023-11-08T19:25:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f819e5ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f819815f220 0x7f819815c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f819e5ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f819815f220 0x7f819815c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40176/0 (socket says 172.21.15.119:40176) 2023-11-08T19:25:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.331+0000 7f819e5ec640 1 -- 172.21.15.119:0/972409722 learned_addr learned my addr 172.21.15.119:0/972409722 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:54.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.332+0000 7f819ddeb640 1 --2- 172.21.15.119:0/972409722 >> v2:172.21.15.188:3300/0 conn(0x7f8198155550 0x7f81980702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.332+0000 7f819e5ec640 1 -- 172.21.15.119:0/972409722 >> v2:172.21.15.188:3300/0 conn(0x7f8198155550 msgr2=0x7f81980702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.332+0000 7f819e5ec640 1 --2- 172.21.15.119:0/972409722 >> v2:172.21.15.188:3300/0 conn(0x7f8198155550 0x7f81980702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.332+0000 7f819e5ec640 1 -- 172.21.15.119:0/972409722 >> v1:172.21.15.119:6789/0 conn(0x7f819806b580 legacy=0x7f819815eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.332+0000 7f819e5ec640 1 -- 172.21.15.119:0/972409722 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8198072420 con 0x7f819815f220 2023-11-08T19:25:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.333+0000 7f819e5ec640 1 --2- 172.21.15.119:0/972409722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f819815c6e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f81980d05a0 tx=0x7f819002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319b9f26796d6ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.333+0000 7f819cde9640 1 -- 172.21.15.119:0/972409722 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819002cb10 con 0x7f819815f220 2023-11-08T19:25:54.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.333+0000 7f819cde9640 1 -- 172.21.15.119:0/972409722 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819002cc70 con 0x7f819815f220 2023-11-08T19:25:54.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.333+0000 7f819cde9640 1 -- 172.21.15.119:0/972409722 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81900343f0 con 0x7f819815f220 2023-11-08T19:25:54.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 -- 172.21.15.119:0/972409722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 msgr2=0x7f819815c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 --2- 172.21.15.119:0/972409722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f819815c6e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f81980d05a0 tx=0x7f819002bf70 comp rx=0 tx=0).stop 2023-11-08T19:25:54.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 -- 172.21.15.119:0/972409722 shutdown_connections 2023-11-08T19:25:54.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 --2- 172.21.15.119:0/972409722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f819815c6e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 --2- 172.21.15.119:0/972409722 >> v2:172.21.15.188:3300/0 conn(0x7f8198155550 0x7f81980702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 -- 172.21.15.119:0/972409722 >> 172.21.15.119:0/972409722 conn(0x7f81980cf4a0 msgr2=0x7f81980cf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 -- 172.21.15.119:0/972409722 shutdown_connections 2023-11-08T19:25:54.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.334+0000 7f81a004f640 1 -- 172.21.15.119:0/972409722 wait complete. 2023-11-08T19:25:54.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.336+0000 7f81a004f640 1 Processor -- start 2023-11-08T19:25:54.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.336+0000 7f81a004f640 1 -- start start 2023-11-08T19:25:54.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f81a004f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f819806b580 0x7f81981f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f81a004f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8198155550 0x7f8198153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f81a004f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f81980b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f81a004f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81981541e0 con 0x7f819815f220 2023-11-08T19:25:54.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f81a004f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81980b4f70 con 0x7f819806b580 2023-11-08T19:25:54.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f81a004f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81980b4400 con 0x7f8198155550 2023-11-08T19:25:54.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f81980b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f81980b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40186/0 (socket says 172.21.15.119:40186) 2023-11-08T19:25:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 -- 172.21.15.119:0/1067180824 learned_addr learned my addr 172.21.15.119:0/1067180824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 -- 172.21.15.119:0/1067180824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8198155550 msgr2=0x7f8198153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8198155550 0x7f8198153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 -- 172.21.15.119:0/1067180824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f819806b580 msgr2=0x7f81981f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f819806b580 0x7f81981f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.337+0000 7f819e5ec640 1 -- 172.21.15.119:0/1067180824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8190009b30 con 0x7f819815f220 2023-11-08T19:25:54.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.338+0000 7f819e5ec640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f81980b3ec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8190036040 tx=0x7f8190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.338+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8190009510 con 0x7f819815f220 2023-11-08T19:25:54.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.338+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819003e450 con 0x7f819815f220 2023-11-08T19:25:54.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.338+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819003e800 con 0x7f819815f220 2023-11-08T19:25:54.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.338+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81980b46d0 con 0x7f819815f220 2023-11-08T19:25:54.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.338+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81980b4c90 con 0x7f819815f220 2023-11-08T19:25:54.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.339+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8198074430 con 0x7f819815f220 2023-11-08T19:25:54.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.340+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f819003e960 con 0x7f819815f220 2023-11-08T19:25:54.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.341+0000 7f818e7fc640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f817c06ae70 0x7f817c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.342+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f81900ac8f0 con 0x7f819815f220 2023-11-08T19:25:54.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.342+0000 7f819ddeb640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f817c06ae70 0x7f817c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.342+0000 7f819ddeb640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f817c06ae70 0x7f817c06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8194006fd0 tx=0x7f8194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.346+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8190077670 con 0x7f819815f220 2023-11-08T19:25:54.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.537+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f819806a2d0 con 0x7f819815f220 2023-11-08T19:25:54.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.537+0000 7f818e7fc640 1 -- 172.21.15.119:0/1067180824 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f81900aae60 con 0x7f819815f220 2023-11-08T19:25:54.540 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f817c06ae70 msgr2=0x7f817c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f817c06ae70 0x7f817c06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8194006fd0 tx=0x7f8194008040 comp rx=0 tx=0).stop 2023-11-08T19:25:54.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 msgr2=0x7f81980b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f81980b3ec0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8190036040 tx=0x7f8190004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 shutdown_connections 2023-11-08T19:25:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f817c06ae70 0x7f817c06d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f819815f220 0x7f81980b3ec0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8198155550 0x7f8198153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 --2- 172.21.15.119:0/1067180824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f819806b580 0x7f81981f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 >> 172.21.15.119:0/1067180824 conn(0x7f81980cf4a0 msgr2=0x7f81980cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 shutdown_connections 2023-11-08T19:25:54.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.540+0000 7f81a004f640 1 -- 172.21.15.119:0/1067180824 wait complete. 2023-11-08T19:25:54.562 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:54.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.776+0000 7ff9b1b2c640 1 Processor -- start 2023-11-08T19:25:54.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.776+0000 7ff9b1b2c640 1 -- start start 2023-11-08T19:25:54.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9b1b2c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff9ac0cfa00 0x7ff9ac0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9b1b2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac161160 0x7ff9ac161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9b1b2c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ac15df50 con 0x7ff9ac15d4a0 2023-11-08T19:25:54.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9b1b2c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ac0d03b0 con 0x7ff9ac0cfa00 2023-11-08T19:25:54.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9b1b2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0d1e60 con 0x7ff9ac161160 2023-11-08T19:25:54.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9ab7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff9ac15d4a0 0x7ff9ac0d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39770/0 (socket says 172.21.15.119:39770) 2023-11-08T19:25:54.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9ab7fe640 1 -- 172.21.15.119:0/3615372705 learned_addr learned my addr 172.21.15.119:0/3615372705 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:54.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.777+0000 7ff9aaffd640 1 --2- 172.21.15.119:0/3615372705 >> v2:172.21.15.119:3300/0 conn(0x7ff9ac0cfa00 0x7ff9ac0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9abfff640 1 --2- 172.21.15.119:0/3615372705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac161160 0x7ff9ac161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9aa7fc640 1 -- 172.21.15.119:0/3615372705 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816390204 0 0) 0x7ff9ac15df50 con 0x7ff9ac15d4a0 2023-11-08T19:25:54.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9aa7fc640 1 -- 172.21.15.119:0/3615372705 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff998003520 con 0x7ff9ac15d4a0 2023-11-08T19:25:54.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9aaffd640 1 -- 172.21.15.119:0/3615372705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac161160 msgr2=0x7ff9ac161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:54.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9aaffd640 1 --2- 172.21.15.119:0/3615372705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac161160 0x7ff9ac161570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9aaffd640 1 -- 172.21.15.119:0/3615372705 >> v1:172.21.15.119:6789/0 conn(0x7ff9ac15d4a0 legacy=0x7ff9ac0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.778+0000 7ff9aaffd640 1 -- 172.21.15.119:0/3615372705 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ac161ab0 con 0x7ff9ac0cfa00 2023-11-08T19:25:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.779+0000 7ff9aaffd640 1 --2- 172.21.15.119:0/3615372705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0cfa00 0x7ff9ac0cfe70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff994009d90 tx=0x7ff99402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfafbd64d6ba4002 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9aa7fc640 1 -- 172.21.15.119:0/3615372705 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99402cbd0 con 0x7ff9ac0cfa00 2023-11-08T19:25:54.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9aa7fc640 1 -- 172.21.15.119:0/3615372705 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff99402cd30 con 0x7ff9ac0cfa00 2023-11-08T19:25:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9aa7fc640 1 -- 172.21.15.119:0/3615372705 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff99402cea0 con 0x7ff9ac0cfa00 2023-11-08T19:25:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/3615372705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0cfa00 msgr2=0x7ff9ac0cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/3615372705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0cfa00 0x7ff9ac0cfe70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff994009d90 tx=0x7ff99402be20 comp rx=0 tx=0).stop 2023-11-08T19:25:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/3615372705 shutdown_connections 2023-11-08T19:25:54.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/3615372705 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac161160 0x7ff9ac161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/3615372705 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac0cfa00 0x7ff9ac0cfe70 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/3615372705 >> 172.21.15.119:0/3615372705 conn(0x7ff9ac06a9f0 msgr2=0x7ff9ac06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.780+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/3615372705 shutdown_connections 2023-11-08T19:25:54.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.781+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/3615372705 wait complete. 2023-11-08T19:25:54.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.782+0000 7ff9b1b2c640 1 Processor -- start 2023-11-08T19:25:54.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.782+0000 7ff9b1b2c640 1 -- start start 2023-11-08T19:25:54.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.783+0000 7ff9b1b2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac0cfa00 0x7ff9ac0b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.783+0000 7ff9b1b2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15d4a0 0x7ff9ac0a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.783+0000 7ff9b1b2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 0x7ff9ac0a9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.783+0000 7ff9b1b2c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0aa4f0 con 0x7ff9ac161160 2023-11-08T19:25:54.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.783+0000 7ff9b1b2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac0aa7d0 con 0x7ff9ac0cfa00 2023-11-08T19:25:54.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.783+0000 7ff9b1b2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ac1f40b0 con 0x7ff9ac15d4a0 2023-11-08T19:25:54.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 0x7ff9ac0a9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 0x7ff9ac0a9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40194/0 (socket says 172.21.15.119:40194) 2023-11-08T19:25:54.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 -- 172.21.15.119:0/1576114788 learned_addr learned my addr 172.21.15.119:0/1576114788 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:54.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 -- 172.21.15.119:0/1576114788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15d4a0 msgr2=0x7ff9ac0a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15d4a0 0x7ff9ac0a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 -- 172.21.15.119:0/1576114788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac0cfa00 msgr2=0x7ff9ac0b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac0cfa00 0x7ff9ac0b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 -- 172.21.15.119:0/1576114788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff994009a70 con 0x7ff9ac161160 2023-11-08T19:25:54.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.784+0000 7ff9abfff640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 0x7ff9ac0a9ec0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400d5d0 tx=0x7ff9a400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.785+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a40125f0 con 0x7ff9ac161160 2023-11-08T19:25:54.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.785+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a4012770 con 0x7ff9ac161160 2023-11-08T19:25:54.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.785+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ac1f4390 con 0x7ff9ac161160 2023-11-08T19:25:54.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.785+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4012b20 con 0x7ff9ac161160 2023-11-08T19:25:54.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.785+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ac0aaab0 con 0x7ff9ac161160 2023-11-08T19:25:54.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.786+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ac067f80 con 0x7ff9ac161160 2023-11-08T19:25:54.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.787+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9a4012c80 con 0x7ff9ac161160 2023-11-08T19:25:54.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.788+0000 7ff9a3fff640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff98c06ae70 0x7ff98c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:54.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.788+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7ff9a4089ec0 con 0x7ff9ac161160 2023-11-08T19:25:54.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.788+0000 7ff9ab7fe640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff98c06ae70 0x7ff98c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:54.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.789+0000 7ff9ab7fe640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff98c06ae70 0x7ff98c06d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff99c000f60 tx=0x7ff99c0078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:54.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.792+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9ac067f80 con 0x7ff9ac161160 2023-11-08T19:25:54.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.968+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9ac1f45d0 con 0x7ff9ac161160 2023-11-08T19:25:54.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.969+0000 7ff9a3fff640 1 -- 172.21.15.119:0/1576114788 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9ac1f45d0 con 0x7ff9ac161160 2023-11-08T19:25:54.972 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff98c06ae70 msgr2=0x7ff98c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff98c06ae70 0x7ff98c06d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff99c000f60 tx=0x7ff99c0078f0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 msgr2=0x7ff9ac0a9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:54.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 0x7ff9ac0a9ec0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400d5d0 tx=0x7ff9a400da90 comp rx=0 tx=0).stop 2023-11-08T19:25:54.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 shutdown_connections 2023-11-08T19:25:54.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff98c06ae70 0x7ff98c06d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff9ac161160 0x7ff9ac0a9ec0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff9ac15d4a0 0x7ff9ac0a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 --2- 172.21.15.119:0/1576114788 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff9ac0cfa00 0x7ff9ac0b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:54.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.971+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 >> 172.21.15.119:0/1576114788 conn(0x7ff9ac06a9f0 msgr2=0x7ff9ac0ce680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:54.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.972+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 shutdown_connections 2023-11-08T19:25:54.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:54.972+0000 7ff9b1b2c640 1 -- 172.21.15.119:0/1576114788 wait complete. 2023-11-08T19:25:54.992 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.184+0000 7f9e1dae0640 1 Processor -- start 2023-11-08T19:25:55.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.185+0000 7f9e1dae0640 1 -- start start 2023-11-08T19:25:55.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e1dae0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9e1815d4a0 0x7f9e180d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e1dae0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e18161870 0x7f9e180cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e1dae0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e1815df50 con 0x7f9e180cfa00 2023-11-08T19:25:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e1dae0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180d03d0 con 0x7f9e18161870 2023-11-08T19:25:55.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e1dae0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e180d1e60 con 0x7f9e1815d4a0 2023-11-08T19:25:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e17fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e18161870 0x7f9e180cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.186+0000 7f9e17fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9e18161870 0x7f9e180cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40196/0 (socket says 172.21.15.119:40196) 2023-11-08T19:25:55.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.187+0000 7f9e17fff640 1 -- 172.21.15.119:0/3697739142 learned_addr learned my addr 172.21.15.119:0/3697739142 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:55.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.187+0000 7f9e177fe640 1 --2- 172.21.15.119:0/3697739142 >> v2:172.21.15.188:3300/0 conn(0x7f9e1815d4a0 0x7f9e180d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.187+0000 7f9e17fff640 1 -- 172.21.15.119:0/3697739142 >> v2:172.21.15.188:3300/0 conn(0x7f9e1815d4a0 msgr2=0x7f9e180d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.187+0000 7f9e17fff640 1 --2- 172.21.15.119:0/3697739142 >> v2:172.21.15.188:3300/0 conn(0x7f9e1815d4a0 0x7f9e180d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.187+0000 7f9e17fff640 1 -- 172.21.15.119:0/3697739142 >> v1:172.21.15.188:6789/0 conn(0x7f9e180cfa00 legacy=0x7f9e18161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.187+0000 7f9e17fff640 1 -- 172.21.15.119:0/3697739142 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e180d2830 con 0x7f9e18161870 2023-11-08T19:25:55.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.188+0000 7f9e17fff640 1 --2- 172.21.15.119:0/3697739142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e18161870 0x7f9e180cfe90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9e18072830 tx=0x7f9e1002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80849c1902383999 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e167fc640 1 -- 172.21.15.119:0/3697739142 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1002c9f0 con 0x7f9e18161870 2023-11-08T19:25:55.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e167fc640 1 -- 172.21.15.119:0/3697739142 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1002cb50 con 0x7f9e18161870 2023-11-08T19:25:55.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e167fc640 1 -- 172.21.15.119:0/3697739142 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100343f0 con 0x7f9e18161870 2023-11-08T19:25:55.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 -- 172.21.15.119:0/3697739142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e18161870 msgr2=0x7f9e180cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/3697739142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e18161870 0x7f9e180cfe90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9e18072830 tx=0x7f9e1002be50 comp rx=0 tx=0).stop 2023-11-08T19:25:55.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 -- 172.21.15.119:0/3697739142 shutdown_connections 2023-11-08T19:25:55.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/3697739142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e18161870 0x7f9e180cfe90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/3697739142 >> v2:172.21.15.188:3300/0 conn(0x7f9e1815d4a0 0x7f9e180d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 -- 172.21.15.119:0/3697739142 >> 172.21.15.119:0/3697739142 conn(0x7f9e1806a9f0 msgr2=0x7f9e1806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 -- 172.21.15.119:0/3697739142 shutdown_connections 2023-11-08T19:25:55.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.189+0000 7f9e1dae0640 1 -- 172.21.15.119:0/3697739142 wait complete. 2023-11-08T19:25:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.191+0000 7f9e1dae0640 1 Processor -- start 2023-11-08T19:25:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.191+0000 7f9e1dae0640 1 -- start start 2023-11-08T19:25:55.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e1dae0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 0x7f9e181547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e1dae0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1815d4a0 0x7f9e18156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e1dae0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e18161870 0x7f9e180cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e1dae0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18154d00 con 0x7f9e180cfa00 2023-11-08T19:25:55.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e1dae0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18154e70 con 0x7f9e1815d4a0 2023-11-08T19:25:55.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e1dae0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e181550c0 con 0x7f9e18161870 2023-11-08T19:25:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.192+0000 7f9e177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 0x7f9e181547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 0x7f9e181547c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40200/0 (socket says 172.21.15.119:40200) 2023-11-08T19:25:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 -- 172.21.15.119:0/2063684686 learned_addr learned my addr 172.21.15.119:0/2063684686 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e16ffd640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1815d4a0 0x7f9e18156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e17fff640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e18161870 0x7f9e180cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 -- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e18161870 msgr2=0x7f9e180cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e18161870 0x7f9e180cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 -- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1815d4a0 msgr2=0x7f9e18156f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1815d4a0 0x7f9e18156f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 -- 172.21.15.119:0/2063684686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e10009b30 con 0x7f9e180cfa00 2023-11-08T19:25:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.193+0000 7f9e177fe640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 0x7f9e181547c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800d800 tx=0x7f9e0800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.194+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800b6c0 con 0x7f9e180cfa00 2023-11-08T19:25:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.194+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e0800b820 con 0x7f9e180cfa00 2023-11-08T19:25:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.194+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0800bbb0 con 0x7f9e180cfa00 2023-11-08T19:25:55.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.194+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e181553a0 con 0x7f9e180cfa00 2023-11-08T19:25:55.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.194+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e18155960 con 0x7f9e180cfa00 2023-11-08T19:25:55.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.195+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e18074430 con 0x7f9e180cfa00 2023-11-08T19:25:55.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.196+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e0800bd10 con 0x7f9e180cfa00 2023-11-08T19:25:55.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.197+0000 7f9e0ffff640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9df806ae70 0x7f9df806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.197+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f9e08089f50 con 0x7f9e180cfa00 2023-11-08T19:25:55.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.197+0000 7f9e16ffd640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9df806ae70 0x7f9df806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.198+0000 7f9e16ffd640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9df806ae70 0x7f9df806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e1815c390 tx=0x7f9e00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.201+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e08054c20 con 0x7f9e180cfa00 2023-11-08T19:25:55.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.388+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e180d10f0 con 0x7f9e180cfa00 2023-11-08T19:25:55.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.389+0000 7f9e0ffff640 1 -- 172.21.15.119:0/2063684686 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e08058240 con 0x7f9e180cfa00 2023-11-08T19:25:55.392 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.391+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9df806ae70 msgr2=0x7f9df806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.391+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9df806ae70 0x7f9df806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e1815c390 tx=0x7f9e00009040 comp rx=0 tx=0).stop 2023-11-08T19:25:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.391+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 msgr2=0x7f9e181547c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.391+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 0x7f9e181547c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800d800 tx=0x7f9e0800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 shutdown_connections 2023-11-08T19:25:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9df806ae70 0x7f9df806d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9e18161870 0x7f9e180cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e1815d4a0 0x7f9e18156f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 --2- 172.21.15.119:0/2063684686 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e180cfa00 0x7f9e181547c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 >> 172.21.15.119:0/2063684686 conn(0x7f9e1806a9f0 msgr2=0x7f9e1806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 shutdown_connections 2023-11-08T19:25:55.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.392+0000 7f9e1dae0640 1 -- 172.21.15.119:0/2063684686 wait complete. 2023-11-08T19:25:55.413 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:55.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.631+0000 7f9f77e07640 1 Processor -- start 2023-11-08T19:25:55.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.631+0000 7f9f77e07640 1 -- start start 2023-11-08T19:25:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.632+0000 7f9f77e07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 0x7f9f700d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.632+0000 7f9f77e07640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7015df50 con 0x7f9f70161870 2023-11-08T19:25:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.632+0000 7f9f77e07640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f700d1e60 con 0x7f9f700cfa00 2023-11-08T19:25:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.632+0000 7f9f77e07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700d2830 con 0x7f9f7015d4a0 2023-11-08T19:25:55.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.632+0000 7f9f763a4640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9f70161870 0x7f9f700cdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39782/0 (socket says 172.21.15.119:39782) 2023-11-08T19:25:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.632+0000 7f9f763a4640 1 -- 172.21.15.119:0/794620074 learned_addr learned my addr 172.21.15.119:0/794620074 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:55.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 0x7f9f700d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267968950 0 0) 0x7f9f7015df50 con 0x7f9f70161870 2023-11-08T19:25:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f5c003520 con 0x7f9f70161870 2023-11-08T19:25:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3743597732 0 0) 0x7f9f5c003520 con 0x7f9f70161870 2023-11-08T19:25:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 msgr2=0x7f9f700d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 0x7f9f700d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 >> v1:172.21.15.188:6789/0 conn(0x7f9f700cfa00 legacy=0x7f9f70161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f70161d40 con 0x7f9f70161870 2023-11-08T19:25:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f9f680019f0 con 0x7f9f70161870 2023-11-08T19:25:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 >> v1:172.21.15.119:6789/0 conn(0x7f9f70161870 legacy=0x7f9f700cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f5c0039a0 0x7f9f5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c0047c0 0x7f9f5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.633+0000 7f9f74ba1640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 0x7f9f5c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700d2830 con 0x7f9f5c0039a0 2023-11-08T19:25:55.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c00bb10 con 0x7f9f5c0047c0 2023-11-08T19:25:55.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c00bcd0 con 0x7f9f7015d4a0 2023-11-08T19:25:55.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f5c0039a0 0x7f9f5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f763a4640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 0x7f9f5c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f753a2640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c0047c0 0x7f9f5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f75ba3640 1 -- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 msgr2=0x7f9f5c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f7015d4a0 0x7f9f5c005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f75ba3640 1 -- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c0047c0 msgr2=0x7f9f5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:55.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c0047c0 0x7f9f5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f75ba3640 1 -- 172.21.15.119:0/794620074 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c003520 con 0x7f9f5c0039a0 2023-11-08T19:25:55.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.634+0000 7f9f753a2640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f5c0047c0 0x7f9f5c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:55.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.635+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f5c0039a0 0x7f9f5c003dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ac0 tx=0x7f9f5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb7da20d7c5b7b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.635+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c02c6b0 con 0x7f9f5c0039a0 2023-11-08T19:25:55.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.635+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c02c810 con 0x7f9f5c0039a0 2023-11-08T19:25:55.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.635+0000 7f9f74ba1640 1 -- 172.21.15.119:0/794620074 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c02cba0 con 0x7f9f5c0039a0 2023-11-08T19:25:55.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f77e07640 1 -- 172.21.15.119:0/794620074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f5c0039a0 msgr2=0x7f9f5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f77e07640 1 --2- 172.21.15.119:0/794620074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f5c0039a0 0x7f9f5c003dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ac0 tx=0x7f9f5c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f763a4640 1 -- 172.21.15.119:0/794620074 reap_dead start 2023-11-08T19:25:55.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f77e07640 1 -- 172.21.15.119:0/794620074 shutdown_connections 2023-11-08T19:25:55.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f77e07640 1 -- 172.21.15.119:0/794620074 >> 172.21.15.119:0/794620074 conn(0x7f9f7006a9f0 msgr2=0x7f9f7006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f77e07640 1 -- 172.21.15.119:0/794620074 shutdown_connections 2023-11-08T19:25:55.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.636+0000 7f9f77e07640 1 -- 172.21.15.119:0/794620074 wait complete. 2023-11-08T19:25:55.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.638+0000 7f9f77e07640 1 Processor -- start 2023-11-08T19:25:55.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.638+0000 7f9f77e07640 1 -- start start 2023-11-08T19:25:55.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f77e07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c0039a0 0x7f9f701f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f77e07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f700ca210 0x7f9f700ca690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f77e07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 0x7f9f700cb680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f77e07640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f700ccf20 con 0x7f9f700cb220 2023-11-08T19:25:55.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f77e07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70147970 con 0x7f9f700ca210 2023-11-08T19:25:55.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f77e07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f70147360 con 0x7f9f5c0039a0 2023-11-08T19:25:55.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f763a4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 0x7f9f700cb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f763a4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 0x7f9f700cb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40214/0 (socket says 172.21.15.119:40214) 2023-11-08T19:25:55.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.639+0000 7f9f763a4640 1 -- 172.21.15.119:0/2656358353 learned_addr learned my addr 172.21.15.119:0/2656358353 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:55.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c0039a0 0x7f9f701f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f763a4640 1 -- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c0039a0 msgr2=0x7f9f701f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f763a4640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c0039a0 0x7f9f701f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f763a4640 1 -- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f700ca210 msgr2=0x7f9f700ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f763a4640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f700ca210 0x7f9f700ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f763a4640 1 -- 172.21.15.119:0/2656358353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f700caf40 con 0x7f9f700cb220 2023-11-08T19:25:55.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f763a4640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 0x7f9f700cb680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f7006d3b0 tx=0x7f9f6800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68004d70 con 0x7f9f700cb220 2023-11-08T19:25:55.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6800bb80 con 0x7f9f700cb220 2023-11-08T19:25:55.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.640+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f68007070 con 0x7f9f700cb220 2023-11-08T19:25:55.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.641+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f70147600 con 0x7f9f700cb220 2023-11-08T19:25:55.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.641+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f700cabd0 con 0x7f9f700cb220 2023-11-08T19:25:55.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.642+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f701471c0 con 0x7f9f700cb220 2023-11-08T19:25:55.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.643+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f680071d0 con 0x7f9f700cb220 2023-11-08T19:25:55.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.644+0000 7f9f667fc640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9f5406ae70 0x7f9f5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:55.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.644+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f9f68089d30 con 0x7f9f700cb220 2023-11-08T19:25:55.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.644+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9f5406ae70 0x7f9f5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:55.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.645+0000 7f9f75ba3640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9f5406ae70 0x7f9f5406d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ac0 tx=0x7f9f6c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:55.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.648+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f701471c0 con 0x7f9f700cb220 2023-11-08T19:25:55.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.839+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9f701471c0 con 0x7f9f700cb220 2023-11-08T19:25:55.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.840+0000 7f9f667fc640 1 -- 172.21.15.119:0/2656358353 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9f701471c0 con 0x7f9f700cb220 2023-11-08T19:25:55.842 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:55.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.842+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9f5406ae70 msgr2=0x7f9f5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.842+0000 7f9f77e07640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9f5406ae70 0x7f9f5406d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c009ac0 tx=0x7f9f6c030000 comp rx=0 tx=0).stop 2023-11-08T19:25:55.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.842+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 msgr2=0x7f9f700cb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:55.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.842+0000 7f9f77e07640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 0x7f9f700cb680 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9f7006d3b0 tx=0x7f9f6800b4a0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 shutdown_connections 2023-11-08T19:25:55.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9f5406ae70 0x7f9f5406d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9f700cb220 0x7f9f700cb680 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f700ca210 0x7f9f700ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 --2- 172.21.15.119:0/2656358353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f5c0039a0 0x7f9f701f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:55.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 >> 172.21.15.119:0/2656358353 conn(0x7f9f7006a9f0 msgr2=0x7f9f7015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:55.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 shutdown_connections 2023-11-08T19:25:55.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:55.843+0000 7f9f77e07640 1 -- 172.21.15.119:0/2656358353 wait complete. 2023-11-08T19:25:55.875 INFO:tasks.cephfs_test_runner:Starting test: test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that attempt to attach pool with snapshots to an fs fails 2023-11-08T19:25:55.876 INFO:tasks.cephfs_test_runner:test_attaching_pools_with_snaps_to_fs_fails (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-08T19:25:55.876 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:56.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.079+0000 7f4c15a91640 1 Processor -- start 2023-11-08T19:25:56.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.080+0000 7f4c15a91640 1 -- start start 2023-11-08T19:25:56.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.081+0000 7f4c15a91640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c100ad7d0 0x7f4c10070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.081+0000 7f4c15a91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c100710a0 0x7f4c100c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.081+0000 7f4c15a91640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c1006cff0 con 0x7f4c1015b0d0 2023-11-08T19:25:56.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.081+0000 7f4c15a91640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1006d2d0 con 0x7f4c100ad7d0 2023-11-08T19:25:56.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.081+0000 7f4c15a91640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1006bd20 con 0x7f4c100710a0 2023-11-08T19:25:56.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c100ad7d0 0x7f4c10070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4c100ad7d0 0x7f4c10070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40230/0 (socket says 172.21.15.119:40230) 2023-11-08T19:25:56.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 -- 172.21.15.119:0/2089248498 learned_addr learned my addr 172.21.15.119:0/2089248498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:56.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c15290640 1 --2- 172.21.15.119:0/2089248498 >> v2:172.21.15.188:3300/0 conn(0x7f4c100710a0 0x7f4c100c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 -- 172.21.15.119:0/2089248498 >> v2:172.21.15.188:3300/0 conn(0x7f4c100710a0 msgr2=0x7f4c100c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 --2- 172.21.15.119:0/2089248498 >> v2:172.21.15.188:3300/0 conn(0x7f4c100710a0 0x7f4c100c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 -- 172.21.15.119:0/2089248498 >> v1:172.21.15.188:6789/0 conn(0x7f4c1015b0d0 legacy=0x7f4c1015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.082+0000 7f4c0ffff640 1 -- 172.21.15.119:0/2089248498 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1006d760 con 0x7f4c100ad7d0 2023-11-08T19:25:56.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.083+0000 7f4c0ffff640 1 --2- 172.21.15.119:0/2089248498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c100ad7d0 0x7f4c10070b60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c1015eaa0 tx=0x7f4bfc02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b8fa22bf783cfd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.084+0000 7f4c0f7fe640 1 -- 172.21.15.119:0/2089248498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc02f850 con 0x7f4c100ad7d0 2023-11-08T19:25:56.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.084+0000 7f4c0f7fe640 1 -- 172.21.15.119:0/2089248498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc02f9b0 con 0x7f4c100ad7d0 2023-11-08T19:25:56.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.084+0000 7f4c0f7fe640 1 -- 172.21.15.119:0/2089248498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc02fd40 con 0x7f4c100ad7d0 2023-11-08T19:25:56.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.084+0000 7f4c15a91640 1 -- 172.21.15.119:0/2089248498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c100ad7d0 msgr2=0x7f4c10070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.084+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2089248498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c100ad7d0 0x7f4c10070b60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4c1015eaa0 tx=0x7f4bfc02da90 comp rx=0 tx=0).stop 2023-11-08T19:25:56.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.085+0000 7f4c15a91640 1 -- 172.21.15.119:0/2089248498 shutdown_connections 2023-11-08T19:25:56.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.085+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2089248498 >> v2:172.21.15.188:3300/0 conn(0x7f4c100710a0 0x7f4c100c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.085+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2089248498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c100ad7d0 0x7f4c10070b60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.085+0000 7f4c15a91640 1 -- 172.21.15.119:0/2089248498 >> 172.21.15.119:0/2089248498 conn(0x7f4c100b0560 msgr2=0x7f4c100b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.085+0000 7f4c15a91640 1 -- 172.21.15.119:0/2089248498 shutdown_connections 2023-11-08T19:25:56.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.085+0000 7f4c15a91640 1 -- 172.21.15.119:0/2089248498 wait complete. 2023-11-08T19:25:56.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.086+0000 7f4c15a91640 1 Processor -- start 2023-11-08T19:25:56.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 -- start start 2023-11-08T19:25:56.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c100710a0 0x7f4c1015e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c100ad7d0 0x7f4c100cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 0x7f4c100cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c101ef780 con 0x7f4c1015b0d0 2023-11-08T19:25:56.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c100c7af0 con 0x7f4c100710a0 2023-11-08T19:25:56.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.087+0000 7f4c15a91640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c100c6d30 con 0x7f4c100ad7d0 2023-11-08T19:25:56.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 0x7f4c100cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 0x7f4c100cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40232/0 (socket says 172.21.15.119:40232) 2023-11-08T19:25:56.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 -- 172.21.15.119:0/2502077632 learned_addr learned my addr 172.21.15.119:0/2502077632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:56.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c14a8f640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c100710a0 0x7f4c1015e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c0ffff640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c100ad7d0 0x7f4c100cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 -- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c100ad7d0 msgr2=0x7f4c100cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c100ad7d0 0x7f4c100cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 -- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c100710a0 msgr2=0x7f4c1015e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c100710a0 0x7f4c1015e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.088+0000 7f4c15290640 1 -- 172.21.15.119:0/2502077632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc0094d0 con 0x7f4c1015b0d0 2023-11-08T19:25:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.089+0000 7f4c15290640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 0x7f4c100cdd70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c08007e40 tx=0x7f4c0800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.089+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c08014620 con 0x7f4c1015b0d0 2023-11-08T19:25:56.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.089+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0800ee90 con 0x7f4c1015b0d0 2023-11-08T19:25:56.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.089+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c100c7010 con 0x7f4c1015b0d0 2023-11-08T19:25:56.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.089+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c080149e0 con 0x7f4c1015b0d0 2023-11-08T19:25:56.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.089+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c100c75d0 con 0x7f4c1015b0d0 2023-11-08T19:25:56.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.090+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c100c8f10 con 0x7f4c1015b0d0 2023-11-08T19:25:56.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.091+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c08014b40 con 0x7f4c1015b0d0 2023-11-08T19:25:56.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.092+0000 7f4c0d7fa640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4bf406ae70 0x7f4bf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.092+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f4c0808a440 con 0x7f4c1015b0d0 2023-11-08T19:25:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.092+0000 7f4c14a8f640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4bf406ae70 0x7f4bf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.093+0000 7f4c14a8f640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4bf406ae70 0x7f4bf406d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c04006fd0 tx=0x7f4c04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.097+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c08055070 con 0x7f4c1015b0d0 2023-11-08T19:25:56.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.288+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c1004ece0 con 0x7f4c1015b0d0 2023-11-08T19:25:56.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.290+0000 7f4c0d7fa640 1 -- 172.21.15.119:0/2502077632 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f4c08058690 con 0x7f4c1015b0d0 2023-11-08T19:25:56.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:25:56.293 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:25:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4bf406ae70 msgr2=0x7f4bf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4bf406ae70 0x7f4bf406d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4c04006fd0 tx=0x7f4c04008040 comp rx=0 tx=0).stop 2023-11-08T19:25:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 msgr2=0x7f4c100cdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 0x7f4c100cdd70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f4c08007e40 tx=0x7f4c0800e4b0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 shutdown_connections 2023-11-08T19:25:56.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4bf406ae70 0x7f4bf406d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c1015b0d0 0x7f4c100cdd70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c100ad7d0 0x7f4c100cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 --2- 172.21.15.119:0/2502077632 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c100710a0 0x7f4c1015e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 >> 172.21.15.119:0/2502077632 conn(0x7f4c100b0560 msgr2=0x7f4c100c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 shutdown_connections 2023-11-08T19:25:56.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.293+0000 7f4c15a91640 1 -- 172.21.15.119:0/2502077632 wait complete. 2023-11-08T19:25:56.316 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:25:56.317 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:25:56.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.535+0000 7f35e8b05640 1 Processor -- start 2023-11-08T19:25:56.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.535+0000 7f35e8b05640 1 -- start start 2023-11-08T19:25:56.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.537+0000 7f35e8b05640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e4143600 0x7f35e40c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.537+0000 7f35e8b05640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35e406d250 0x7f35e415de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.537+0000 7f35e8b05640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35e415e440 con 0x7f35e415d9f0 2023-11-08T19:25:56.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.537+0000 7f35e8b05640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35e40c4f50 con 0x7f35e406d250 2023-11-08T19:25:56.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.537+0000 7f35e8b05640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40c4960 con 0x7f35e4143600 2023-11-08T19:25:56.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35e406d250 0x7f35e415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35e406d250 0x7f35e415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40234/0 (socket says 172.21.15.119:40234) 2023-11-08T19:25:56.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 -- 172.21.15.119:0/4218976459 learned_addr learned my addr 172.21.15.119:0/4218976459 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:56.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e259c640 1 --2- 172.21.15.119:0/4218976459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e4143600 0x7f35e40c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 -- 172.21.15.119:0/4218976459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e4143600 msgr2=0x7f35e40c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 --2- 172.21.15.119:0/4218976459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e4143600 0x7f35e40c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 -- 172.21.15.119:0/4218976459 >> v1:172.21.15.188:6789/0 conn(0x7f35e415d9f0 legacy=0x7f35e406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.538+0000 7f35e2d9d640 1 -- 172.21.15.119:0/4218976459 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35e415e720 con 0x7f35e406d250 2023-11-08T19:25:56.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.539+0000 7f35e2d9d640 1 --2- 172.21.15.119:0/4218976459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e406d250 0x7f35e415de30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009e50 tx=0x7f35dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7904f9dc874dd39e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.539+0000 7f35e1d9b640 1 -- 172.21.15.119:0/4218976459 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc02ca70 con 0x7f35e406d250 2023-11-08T19:25:56.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.539+0000 7f35e1d9b640 1 -- 172.21.15.119:0/4218976459 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc02cbd0 con 0x7f35e406d250 2023-11-08T19:25:56.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.539+0000 7f35e1d9b640 1 -- 172.21.15.119:0/4218976459 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc0343f0 con 0x7f35e406d250 2023-11-08T19:25:56.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 -- 172.21.15.119:0/4218976459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e406d250 msgr2=0x7f35e415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 --2- 172.21.15.119:0/4218976459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e406d250 0x7f35e415de30 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009e50 tx=0x7f35dc02bed0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 -- 172.21.15.119:0/4218976459 shutdown_connections 2023-11-08T19:25:56.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 --2- 172.21.15.119:0/4218976459 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e406d250 0x7f35e415de30 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 --2- 172.21.15.119:0/4218976459 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e4143600 0x7f35e40c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 -- 172.21.15.119:0/4218976459 >> 172.21.15.119:0/4218976459 conn(0x7f35e40cb610 msgr2=0x7f35e40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 -- 172.21.15.119:0/4218976459 shutdown_connections 2023-11-08T19:25:56.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.540+0000 7f35e8b05640 1 -- 172.21.15.119:0/4218976459 wait complete. 2023-11-08T19:25:56.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.542+0000 7f35e8b05640 1 Processor -- start 2023-11-08T19:25:56.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.542+0000 7f35e8b05640 1 -- start start 2023-11-08T19:25:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e8b05640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e406d250 0x7f35e40c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e8b05640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e4143600 0x7f35e415af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e8b05640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 0x7f35e4156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e8b05640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4140dc0 con 0x7f35e415d9f0 2023-11-08T19:25:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e8b05640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35e4142a80 con 0x7f35e4143600 2023-11-08T19:25:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e8b05640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35e40c7bc0 con 0x7f35e406d250 2023-11-08T19:25:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 0x7f35e4156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 0x7f35e4156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40246/0 (socket says 172.21.15.119:40246) 2023-11-08T19:25:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.543+0000 7f35e2d9d640 1 -- 172.21.15.119:0/1269405113 learned_addr learned my addr 172.21.15.119:0/1269405113 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e259c640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e406d250 0x7f35e40c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35d9d9b640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e4143600 0x7f35e415af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e2d9d640 1 -- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e406d250 msgr2=0x7f35e40c6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e2d9d640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e406d250 0x7f35e40c6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e2d9d640 1 -- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e4143600 msgr2=0x7f35e415af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e2d9d640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e4143600 0x7f35e415af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e2d9d640 1 -- 172.21.15.119:0/1269405113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35dc009b30 con 0x7f35e415d9f0 2023-11-08T19:25:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35d9d9b640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e4143600 0x7f35e415af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e2d9d640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 0x7f35e4156680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009e50 tx=0x7f35dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc009510 con 0x7f35e415d9f0 2023-11-08T19:25:56.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35dc03e450 con 0x7f35e415d9f0 2023-11-08T19:25:56.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35dc03e800 con 0x7f35e415d9f0 2023-11-08T19:25:56.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.544+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35e4156bc0 con 0x7f35e415d9f0 2023-11-08T19:25:56.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.545+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35e40c7600 con 0x7f35e415d9f0 2023-11-08T19:25:56.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.546+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35e4140b90 con 0x7f35e415d9f0 2023-11-08T19:25:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.547+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35dc03e960 con 0x7f35e415d9f0 2023-11-08T19:25:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.548+0000 7f35db7fe640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35c406ae70 0x7f35c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.548+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f35dc0ac780 con 0x7f35e415d9f0 2023-11-08T19:25:56.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.548+0000 7f35e259c640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35c406ae70 0x7f35c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.548+0000 7f35e259c640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35c406ae70 0x7f35c406d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35d4006fd0 tx=0x7f35d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.552+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35e4140b90 con 0x7f35e415d9f0 2023-11-08T19:25:56.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.744+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f35e4140b90 con 0x7f35e415d9f0 2023-11-08T19:25:56.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.745+0000 7f35db7fe640 1 -- 172.21.15.119:0/1269405113 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f35e4140b90 con 0x7f35e415d9f0 2023-11-08T19:25:56.747 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:25:56.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35c406ae70 msgr2=0x7f35c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35c406ae70 0x7f35c406d330 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f35d4006fd0 tx=0x7f35d4008040 comp rx=0 tx=0).stop 2023-11-08T19:25:56.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 msgr2=0x7f35e4156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 0x7f35e4156680 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f35dc009e50 tx=0x7f35dc004a40 comp rx=0 tx=0).stop 2023-11-08T19:25:56.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 shutdown_connections 2023-11-08T19:25:56.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f35c406ae70 0x7f35c406d330 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35e415d9f0 0x7f35e4156680 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35e4143600 0x7f35e415af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 --2- 172.21.15.119:0/1269405113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35e406d250 0x7f35e40c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 >> 172.21.15.119:0/1269405113 conn(0x7f35e40cb610 msgr2=0x7f35e4071480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 shutdown_connections 2023-11-08T19:25:56.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.748+0000 7f35e8b05640 1 -- 172.21.15.119:0/1269405113 wait complete. 2023-11-08T19:25:56.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:25:56.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.972+0000 7fcbb4461640 1 Processor -- start 2023-11-08T19:25:56.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.973+0000 7fcbb4461640 1 -- start start 2023-11-08T19:25:56.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.974+0000 7fcbb4461640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbac0cfa00 0x7fcbac0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.974+0000 7fcbb4461640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac15df50 con 0x7fcbac15d4a0 2023-11-08T19:25:56.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.974+0000 7fcbb4461640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac0d1e60 con 0x7fcbac161160 2023-11-08T19:25:56.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.974+0000 7fcbb4461640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac0d03b0 con 0x7fcbac0cfa00 2023-11-08T19:25:56.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.974+0000 7fcbb21fd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcbac15d4a0 0x7fcbac0d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39784/0 (socket says 172.21.15.119:39784) 2023-11-08T19:25:56.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.974+0000 7fcbb21fd640 1 -- 172.21.15.119:0/1372596312 learned_addr learned my addr 172.21.15.119:0/1372596312 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:56.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1372596312 >> v2:172.21.15.188:3300/0 conn(0x7fcbac0cfa00 0x7fcbac0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984937041 0 0) 0x7fcbac15df50 con 0x7fcbac15d4a0 2023-11-08T19:25:56.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb98003520 con 0x7fcbac15d4a0 2023-11-08T19:25:56.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3804276082 0 0) 0x7fcb98003520 con 0x7fcbac15d4a0 2023-11-08T19:25:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 >> v2:172.21.15.188:3300/0 conn(0x7fcbac0cfa00 msgr2=0x7fcbac0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 --2- 172.21.15.119:0/1372596312 >> v2:172.21.15.188:3300/0 conn(0x7fcbac0cfa00 0x7fcbac0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 >> v1:172.21.15.188:6789/0 conn(0x7fcbac161160 legacy=0x7fcbac161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac161d40 con 0x7fcbac15d4a0 2023-11-08T19:25:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.975+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fcba0001670 con 0x7fcbac15d4a0 2023-11-08T19:25:56.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 >> v1:172.21.15.119:6789/0 conn(0x7fcbac15d4a0 legacy=0x7fcbac0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb98003970 0x7fcb98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb980047c0 0x7fcb98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb98005180 0x7fcb9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0d03b0 con 0x7fcb980047c0 2023-11-08T19:25:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9800c120 con 0x7fcb98003970 2023-11-08T19:25:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9800c2e0 con 0x7fcb98005180 2023-11-08T19:25:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb980047c0 0x7fcb98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb21fd640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb98003970 0x7fcb98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb98005180 msgr2=0x7fcb9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb98005180 0x7fcb9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb98003970 msgr2=0x7fcb98003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb98003970 0x7fcb98003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb29fe640 1 -- 172.21.15.119:0/1372596312 reap_dead start 2023-11-08T19:25:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1372596312 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98003520 con 0x7fcb980047c0 2023-11-08T19:25:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.976+0000 7fcbb21fd640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb98003970 0x7fcb98003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.977+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb980047c0 0x7fcb98004c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009fc0 tx=0x7fcb980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46dad315f54695a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba802cc20 con 0x7fcb980047c0 2023-11-08T19:25:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800f040 con 0x7fcb980047c0 2023-11-08T19:25:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb11fb640 1 -- 172.21.15.119:0/1372596312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8034440 con 0x7fcb980047c0 2023-11-08T19:25:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb4461640 1 -- 172.21.15.119:0/1372596312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb980047c0 msgr2=0x7fcb98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb980047c0 0x7fcb98004c40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009fc0 tx=0x7fcb980057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb4461640 1 -- 172.21.15.119:0/1372596312 shutdown_connections 2023-11-08T19:25:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1372596312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb980047c0 0x7fcb98004c40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.978+0000 7fcbb4461640 1 -- 172.21.15.119:0/1372596312 >> 172.21.15.119:0/1372596312 conn(0x7fcbac06a9f0 msgr2=0x7fcbac06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.979+0000 7fcbb4461640 1 -- 172.21.15.119:0/1372596312 shutdown_connections 2023-11-08T19:25:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.979+0000 7fcbb4461640 1 -- 172.21.15.119:0/1372596312 wait complete. 2023-11-08T19:25:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.980+0000 7fcbb4461640 1 Processor -- start 2023-11-08T19:25:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.980+0000 7fcbb4461640 1 -- start start 2023-11-08T19:25:56.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.981+0000 7fcbb4461640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb980047c0 0x7fcbac14b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.981+0000 7fcbb4461640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 0x7fcbac0ca320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.981+0000 7fcbb4461640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbac0ca860 0x7fcbac0b7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.981+0000 7fcbb4461640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0cb980 con 0x7fcbac0cbda0 2023-11-08T19:25:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.981+0000 7fcbb4461640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac14b820 con 0x7fcbac0ca860 2023-11-08T19:25:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.981+0000 7fcbb4461640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0cad00 con 0x7fcb980047c0 2023-11-08T19:25:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 0x7fcbac0ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 0x7fcbac0ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40254/0 (socket says 172.21.15.119:40254) 2023-11-08T19:25:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1706063540 learned_addr learned my addr 172.21.15.119:0/1706063540 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb21fd640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb980047c0 0x7fcbac14b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb29fe640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbac0ca860 0x7fcbac0b7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb980047c0 msgr2=0x7fcbac14b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb980047c0 0x7fcbac14b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbac0ca860 msgr2=0x7fcbac0b7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbac0ca860 0x7fcbac0b7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 -- 172.21.15.119:0/1706063540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0cafe0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb29fe640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbac0ca860 0x7fcbac0b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.982+0000 7fcbb19fc640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 0x7fcbac0ca320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e230 tx=0x7fcba8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.983+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8034c50 con 0x7fcbac0cbda0 2023-11-08T19:25:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.983+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8034db0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.983+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8003bb0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.983+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac0cb2c0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.983+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac1556e0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.984+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac0d10f0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.985+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcba8040450 con 0x7fcbac0cbda0 2023-11-08T19:25:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.986+0000 7fcb9effd640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb9006ae70 0x7fcb9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.986+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcba80ac2c0 con 0x7fcbac0cbda0 2023-11-08T19:25:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.987+0000 7fcbb21fd640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb9006ae70 0x7fcb9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.987+0000 7fcbb21fd640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb9006ae70 0x7fcb9006d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcba0001eb0 tx=0x7fcba0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:56.991+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcba8076ef0 con 0x7fcbac0cbda0 2023-11-08T19:25:57.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.178+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbac0d0e50 con 0x7fcbac0cbda0 2023-11-08T19:25:57.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.179+0000 7fcb9effd640 1 -- 172.21.15.119:0/1706063540 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcba807a510 con 0x7fcbac0cbda0 2023-11-08T19:25:57.181 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:57.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.181+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb9006ae70 msgr2=0x7fcb9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.181+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb9006ae70 0x7fcb9006d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcba0001eb0 tx=0x7fcba0009040 comp rx=0 tx=0).stop 2023-11-08T19:25:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.181+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 msgr2=0x7fcbac0ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.181+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 0x7fcbac0ca320 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcba800e230 tx=0x7fcba8009750 comp rx=0 tx=0).stop 2023-11-08T19:25:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.181+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 shutdown_connections 2023-11-08T19:25:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb9006ae70 0x7fcb9006d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbac0ca860 0x7fcbac0b7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcbac0cbda0 0x7fcbac0ca320 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 --2- 172.21.15.119:0/1706063540 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb980047c0 0x7fcbac14b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 >> 172.21.15.119:0/1706063540 conn(0x7fcbac06a9f0 msgr2=0x7fcbac15e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 shutdown_connections 2023-11-08T19:25:57.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.182+0000 7fcbb4461640 1 -- 172.21.15.119:0/1706063540 wait complete. 2023-11-08T19:25:57.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:25:57.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.412+0000 7fe25b807640 1 Processor -- start 2023-11-08T19:25:57.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.412+0000 7fe25b807640 1 -- start start 2023-11-08T19:25:57.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.414+0000 7fe25b807640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2540c3a50 0x7fe2540cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.414+0000 7fe25b807640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25406d220 con 0x7fe25415d000 2023-11-08T19:25:57.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.414+0000 7fe25b807640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2540aca40 con 0x7fe25406a830 2023-11-08T19:25:57.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.414+0000 7fe25b807640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2540ccd50 con 0x7fe2540c3a50 2023-11-08T19:25:57.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.414+0000 7fe2595a3640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe25415d000 0x7fe25406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39792/0 (socket says 172.21.15.119:39792) 2023-11-08T19:25:57.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.414+0000 7fe2595a3640 1 -- 172.21.15.119:0/4238295029 learned_addr learned my addr 172.21.15.119:0/4238295029 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe259da4640 1 --2- 172.21.15.119:0/4238295029 >> v2:172.21.15.188:3300/0 conn(0x7fe2540c3a50 0x7fe2540cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265395010 0 0) 0x7fe25406d220 con 0x7fe25415d000 2023-11-08T19:25:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe240003520 con 0x7fe25415d000 2023-11-08T19:25:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342228403 0 0) 0x7fe2540aca40 con 0x7fe25406a830 2023-11-08T19:25:57.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe25406d220 con 0x7fe25406a830 2023-11-08T19:25:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3273530770 0 0) 0x7fe240003520 con 0x7fe25415d000 2023-11-08T19:25:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 >> v2:172.21.15.188:3300/0 conn(0x7fe2540c3a50 msgr2=0x7fe2540cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 --2- 172.21.15.119:0/4238295029 >> v2:172.21.15.188:3300/0 conn(0x7fe2540c3a50 0x7fe2540cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 >> v1:172.21.15.188:6789/0 conn(0x7fe25406a830 legacy=0x7fe254160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2540acd20 con 0x7fe25415d000 2023-11-08T19:25:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe250001690 con 0x7fe25415d000 2023-11-08T19:25:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 >> v1:172.21.15.119:6789/0 conn(0x7fe25415d000 legacy=0x7fe25406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe240003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400047c0 0x7fe240004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2400051d0 0x7fe24000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540ccd50 con 0x7fe240003b50 2023-11-08T19:25:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24000c100 con 0x7fe2400051d0 2023-11-08T19:25:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.415+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24000c2c0 con 0x7fe2400047c0 2023-11-08T19:25:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe2595a3640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe240003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe259da4640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2400051d0 0x7fe24000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe258da2640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400047c0 0x7fe240004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe2595a3640 1 -- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400047c0 msgr2=0x7fe240004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:57.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe2595a3640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2400047c0 0x7fe240004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe2595a3640 1 -- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2400051d0 msgr2=0x7fe24000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:57.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe2595a3640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2400051d0 0x7fe24000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe259da4640 1 -- 172.21.15.119:0/4238295029 reap_dead start 2023-11-08T19:25:57.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.416+0000 7fe2595a3640 1 -- 172.21.15.119:0/4238295029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe240003520 con 0x7fe240003b50 2023-11-08T19:25:57.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.418+0000 7fe2595a3640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe240003f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe25406d3b0 tx=0x7fe2400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1790b2605b1b4b38 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:57.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.418+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250005ad0 con 0x7fe240003b50 2023-11-08T19:25:57.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.418+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe250005c30 con 0x7fe240003b50 2023-11-08T19:25:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.418+0000 7fe24bfff640 1 -- 172.21.15.119:0/4238295029 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250007070 con 0x7fe240003b50 2023-11-08T19:25:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 -- 172.21.15.119:0/4238295029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 msgr2=0x7fe240003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe240003f50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe25406d3b0 tx=0x7fe2400057f0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 -- 172.21.15.119:0/4238295029 shutdown_connections 2023-11-08T19:25:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 --2- 172.21.15.119:0/4238295029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe240003f50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 -- 172.21.15.119:0/4238295029 >> 172.21.15.119:0/4238295029 conn(0x7fe2540cb610 msgr2=0x7fe2540cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 -- 172.21.15.119:0/4238295029 shutdown_connections 2023-11-08T19:25:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.419+0000 7fe25b807640 1 -- 172.21.15.119:0/4238295029 wait complete. 2023-11-08T19:25:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.421+0000 7fe25b807640 1 Processor -- start 2023-11-08T19:25:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.421+0000 7fe25b807640 1 -- start start 2023-11-08T19:25:57.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe25b807640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe25415e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe25b807640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25415d3e0 0x7fe25415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe25b807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe254157760 0x7fe254157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe25b807640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe254158120 con 0x7fe240003b50 2023-11-08T19:25:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe25b807640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25415eb90 con 0x7fe25415d3e0 2023-11-08T19:25:57.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe25b807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2540b1a30 con 0x7fe254157760 2023-11-08T19:25:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe2595a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe25415e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe2595a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe25415e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40276/0 (socket says 172.21.15.119:40276) 2023-11-08T19:25:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe2595a3640 1 -- 172.21.15.119:0/1360418332 learned_addr learned my addr 172.21.15.119:0/1360418332 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.422+0000 7fe259da4640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe254157760 0x7fe254157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe258da2640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25415d3e0 0x7fe25415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe2595a3640 1 -- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe254157760 msgr2=0x7fe254157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe2595a3640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe254157760 0x7fe254157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe2595a3640 1 -- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25415d3e0 msgr2=0x7fe25415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe2595a3640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25415d3e0 0x7fe25415f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe2595a3640 1 -- 172.21.15.119:0/1360418332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2540b1d10 con 0x7fe240003b50 2023-11-08T19:25:57.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe2595a3640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe25415e300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe250000c00 tx=0x7fe250001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:57.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25003c070 con 0x7fe240003b50 2023-11-08T19:25:57.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe250007280 con 0x7fe240003b50 2023-11-08T19:25:57.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe250007630 con 0x7fe240003b50 2023-11-08T19:25:57.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2540b1ff0 con 0x7fe240003b50 2023-11-08T19:25:57.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.423+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25415fdd0 con 0x7fe240003b50 2023-11-08T19:25:57.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.424+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25415ffc0 con 0x7fe240003b50 2023-11-08T19:25:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.425+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe250032070 con 0x7fe240003b50 2023-11-08T19:25:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.426+0000 7fe249ffb640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe23806ae70 0x7fe23806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.426+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe2500aa2f0 con 0x7fe240003b50 2023-11-08T19:25:57.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.427+0000 7fe258da2640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe23806ae70 0x7fe23806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.427+0000 7fe258da2640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe23806ae70 0x7fe23806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe244007fd0 tx=0x7fe244009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:57.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.431+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe250075170 con 0x7fe240003b50 2023-11-08T19:25:57.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.620+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe2540b1870 con 0x7fe240003b50 2023-11-08T19:25:57.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.620+0000 7fe249ffb640 1 -- 172.21.15.119:0/1360418332 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe250045020 con 0x7fe240003b50 2023-11-08T19:25:57.622 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:25:57.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe23806ae70 msgr2=0x7fe23806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe23806ae70 0x7fe23806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe244007fd0 tx=0x7fe244009040 comp rx=0 tx=0).stop 2023-11-08T19:25:57.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 msgr2=0x7fe25415e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe25415e300 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe250000c00 tx=0x7fe250001950 comp rx=0 tx=0).stop 2023-11-08T19:25:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 shutdown_connections 2023-11-08T19:25:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe23806ae70 0x7fe23806d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe254157760 0x7fe254157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25415d3e0 0x7fe25415f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 --2- 172.21.15.119:0/1360418332 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe240003b50 0x7fe25415e300 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 >> 172.21.15.119:0/1360418332 conn(0x7fe2540cb610 msgr2=0x7fe25406c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:57.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 shutdown_connections 2023-11-08T19:25:57.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.623+0000 7fe25b807640 1 -- 172.21.15.119:0/1360418332 wait complete. 2023-11-08T19:25:57.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:25:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.850+0000 7f2483b25640 1 Processor -- start 2023-11-08T19:25:57.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.850+0000 7f2483b25640 1 -- start start 2023-11-08T19:25:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f2483b25640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f2483b25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f247c144960 0x7f247c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f2483b25640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f247c15ea60 con 0x7f247c15f1d0 2023-11-08T19:25:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f2483b25640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f247c0c9150 con 0x7f247c144960 2023-11-08T19:25:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f2483b25640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247c0cb270 con 0x7f247c144580 2023-11-08T19:25:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24810c0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f247c144960 0x7f247c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24810c0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f247c144960 0x7f247c0cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40286/0 (socket says 172.21.15.119:40286) 2023-11-08T19:25:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24810c0640 1 -- 172.21.15.119:0/2113416576 learned_addr learned my addr 172.21.15.119:0/2113416576 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:57.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24820c2640 1 --1- 172.21.15.119:0/2113416576 >> v1:172.21.15.119:6789/0 conn(0x7f247c15f1d0 0x7f247c0c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39804/0 (socket says 172.21.15.119:39804) 2023-11-08T19:25:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24818c1640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24810c0640 1 -- 172.21.15.119:0/2113416576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 msgr2=0x7f247c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24810c0640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.852+0000 7f24810c0640 1 -- 172.21.15.119:0/2113416576 >> v1:172.21.15.119:6789/0 conn(0x7f247c15f1d0 legacy=0x7f247c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.853+0000 7f24810c0640 1 -- 172.21.15.119:0/2113416576 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0c9360 con 0x7f247c144960 2023-11-08T19:25:57.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.853+0000 7f24818c1640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:57.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.854+0000 7f24810c0640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c144960 0x7f247c0cad30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f246c009870 tx=0x7f246c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a24b4e77764c6e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:57.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.854+0000 7f24808bf640 1 -- 172.21.15.119:0/2113416576 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c031870 con 0x7f247c144960 2023-11-08T19:25:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.854+0000 7f24808bf640 1 -- 172.21.15.119:0/2113416576 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c0319d0 con 0x7f247c144960 2023-11-08T19:25:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.854+0000 7f24808bf640 1 -- 172.21.15.119:0/2113416576 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c031d60 con 0x7f247c144960 2023-11-08T19:25:57.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.854+0000 7f2483b25640 1 -- 172.21.15.119:0/2113416576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c144960 msgr2=0x7f247c0cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.854+0000 7f2483b25640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c144960 0x7f247c0cad30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f246c009870 tx=0x7f246c02b910 comp rx=0 tx=0).stop 2023-11-08T19:25:57.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.855+0000 7f2483b25640 1 -- 172.21.15.119:0/2113416576 shutdown_connections 2023-11-08T19:25:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.855+0000 7f2483b25640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c144960 0x7f247c0cad30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.855+0000 7f2483b25640 1 --2- 172.21.15.119:0/2113416576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.855+0000 7f2483b25640 1 -- 172.21.15.119:0/2113416576 >> 172.21.15.119:0/2113416576 conn(0x7f247c0cb610 msgr2=0x7f247c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.855+0000 7f2483b25640 1 -- 172.21.15.119:0/2113416576 shutdown_connections 2023-11-08T19:25:57.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.855+0000 7f2483b25640 1 -- 172.21.15.119:0/2113416576 wait complete. 2023-11-08T19:25:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.856+0000 7f2483b25640 1 Processor -- start 2023-11-08T19:25:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.857+0000 7f2483b25640 1 -- start start 2023-11-08T19:25:57.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.857+0000 7f2483b25640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c1f27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f2483b25640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f247c144960 0x7f247c1303f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f2483b25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 0x7f247c1427e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f2483b25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c1e7db0 con 0x7f247c15f1d0 2023-11-08T19:25:57.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f2483b25640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247c12e360 con 0x7f247c144960 2023-11-08T19:25:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f2483b25640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247c12e8e0 con 0x7f247c144580 2023-11-08T19:25:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 0x7f247c1427e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 0x7f247c1427e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40296/0 (socket says 172.21.15.119:40296) 2023-11-08T19:25:57.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 -- 172.21.15.119:0/2855279838 learned_addr learned my addr 172.21.15.119:0/2855279838 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 -- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 msgr2=0x7f247c1f27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c1f27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 -- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f247c144960 msgr2=0x7f247c1303f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24810c0640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f247c144960 0x7f247c1303f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f247c144960 0x7f247c1303f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 -- 172.21.15.119:0/2855279838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c0094d0 con 0x7f247c15f1d0 2023-11-08T19:25:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.858+0000 7f24820c2640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 0x7f247c1427e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f247400c730 tx=0x7f247400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:57.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.859+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474004d60 con 0x7f247c15f1d0 2023-11-08T19:25:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.859+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247400a640 con 0x7f247c15f1d0 2023-11-08T19:25:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.859+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c130eb0 con 0x7f247c15f1d0 2023-11-08T19:25:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.859+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247400a9f0 con 0x7f247c15f1d0 2023-11-08T19:25:57.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.859+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f247c12e570 con 0x7f247c15f1d0 2023-11-08T19:25:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.860+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f247c074430 con 0x7f247c15f1d0 2023-11-08T19:25:57.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.861+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2474007070 con 0x7f247c15f1d0 2023-11-08T19:25:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.862+0000 7f24727fc640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f246006ae70 0x7f246006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.862+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f2474089400 con 0x7f247c15f1d0 2023-11-08T19:25:57.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.862+0000 7f24818c1640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f246006ae70 0x7f246006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.863+0000 7f24818c1640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f246006ae70 0x7f246006d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2478006fd0 tx=0x7f2478008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:57.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:57.866+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2474054030 con 0x7f247c15f1d0 2023-11-08T19:25:58.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.052+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f247c12ea20 con 0x7f247c15f1d0 2023-11-08T19:25:58.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.053+0000 7f24727fc640 1 -- 172.21.15.119:0/2855279838 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2474057650 con 0x7f247c15f1d0 2023-11-08T19:25:58.055 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:25:58.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f246006ae70 msgr2=0x7f246006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f246006ae70 0x7f246006d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2478006fd0 tx=0x7f2478008040 comp rx=0 tx=0).stop 2023-11-08T19:25:58.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 msgr2=0x7f247c1427e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 0x7f247c1427e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f247400c730 tx=0x7f247400cbf0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 shutdown_connections 2023-11-08T19:25:58.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f246006ae70 0x7f246006d330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f247c15f1d0 0x7f247c1427e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f247c144960 0x7f247c1303f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 --2- 172.21.15.119:0/2855279838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f247c144580 0x7f247c1f27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 >> 172.21.15.119:0/2855279838 conn(0x7f247c0cb610 msgr2=0x7f247c06fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:58.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 shutdown_connections 2023-11-08T19:25:58.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.056+0000 7f2483b25640 1 -- 172.21.15.119:0/2855279838 wait complete. 2023-11-08T19:25:58.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2023-11-08T19:25:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.284+0000 7f3f969b8640 1 Processor -- start 2023-11-08T19:25:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.285+0000 7f3f969b8640 1 -- start start 2023-11-08T19:25:58.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f969b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f900ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:58.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f969b8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f900cea70 0x7f3f900cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f969b8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f900d1940 con 0x7f3f900cfc70 2023-11-08T19:25:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f969b8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f900d0100 con 0x7f3f900cea70 2023-11-08T19:25:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f969b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900d0d30 con 0x7f3f9015cee0 2023-11-08T19:25:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f951b5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3f900cea70 0x7f3f900cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f961b7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3f900cfc70 0x7f3f900b7ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39808/0 (socket says 172.21.15.119:39808) 2023-11-08T19:25:58.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f961b7640 1 -- 172.21.15.119:0/390515659 learned_addr learned my addr 172.21.15.119:0/390515659 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:58.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f959b6640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f900ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:58.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.286+0000 7f3f951b5640 1 -- 172.21.15.119:0/390515659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 msgr2=0x7f3f900ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:58.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.287+0000 7f3f951b5640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f900ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.287+0000 7f3f951b5640 1 -- 172.21.15.119:0/390515659 >> v1:172.21.15.119:6789/0 conn(0x7f3f900cfc70 legacy=0x7f3f900b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.287+0000 7f3f951b5640 1 -- 172.21.15.119:0/390515659 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900d1010 con 0x7f3f900cea70 2023-11-08T19:25:58.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.287+0000 7f3f959b6640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f900ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:58.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.287+0000 7f3f949b4640 1 -- 172.21.15.119:0/390515659 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955615643 0 0) 0x7f3f900d1940 con 0x7f3f900cfc70 2023-11-08T19:25:58.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.288+0000 7f3f951b5640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cea70 0x7f3f900cf730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3f800097a0 tx=0x7f3f8002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ec4385b2dad6ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:58.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.288+0000 7f3f949b4640 1 -- 172.21.15.119:0/390515659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8002d040 con 0x7f3f900cea70 2023-11-08T19:25:58.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.288+0000 7f3f949b4640 1 -- 172.21.15.119:0/390515659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80031810 con 0x7f3f900cea70 2023-11-08T19:25:58.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.288+0000 7f3f949b4640 1 -- 172.21.15.119:0/390515659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80031ba0 con 0x7f3f900cea70 2023-11-08T19:25:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.288+0000 7f3f969b8640 1 -- 172.21.15.119:0/390515659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cea70 msgr2=0x7f3f900cf730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.288+0000 7f3f969b8640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cea70 0x7f3f900cf730 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3f800097a0 tx=0x7f3f8002b8b0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.289+0000 7f3f969b8640 1 -- 172.21.15.119:0/390515659 shutdown_connections 2023-11-08T19:25:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.289+0000 7f3f969b8640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cea70 0x7f3f900cf730 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.289+0000 7f3f969b8640 1 --2- 172.21.15.119:0/390515659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f900ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.289+0000 7f3f969b8640 1 -- 172.21.15.119:0/390515659 >> 172.21.15.119:0/390515659 conn(0x7f3f9009c100 msgr2=0x7f3f900d2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.289+0000 7f3f969b8640 1 -- 172.21.15.119:0/390515659 shutdown_connections 2023-11-08T19:25:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.289+0000 7f3f969b8640 1 -- 172.21.15.119:0/390515659 wait complete. 2023-11-08T19:25:58.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.290+0000 7f3f969b8640 1 Processor -- start 2023-11-08T19:25:58.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.291+0000 7f3f969b8640 1 -- start start 2023-11-08T19:25:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.291+0000 7f3f969b8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f900cea70 0x7f3f900b6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f969b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 0x7f3f900b8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f969b8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f901f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f969b8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900b70b0 con 0x7f3f900cfc70 2023-11-08T19:25:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f969b8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900b7390 con 0x7f3f900cea70 2023-11-08T19:25:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f969b8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f900b7670 con 0x7f3f9015cee0 2023-11-08T19:25:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 0x7f3f900b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 0x7f3f900b8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40318/0 (socket says 172.21.15.119:40318) 2023-11-08T19:25:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 -- 172.21.15.119:0/449496040 learned_addr learned my addr 172.21.15.119:0/449496040 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f959b6640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f900cea70 0x7f3f900b6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f961b7640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f901f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 -- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 msgr2=0x7f3f901f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f901f60c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 -- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f900cea70 msgr2=0x7f3f900b6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f900cea70 0x7f3f900b6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f951b5640 1 -- 172.21.15.119:0/449496040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f80009400 con 0x7f3f900cfc70 2023-11-08T19:25:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.292+0000 7f3f959b6640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f900cea70 0x7f3f900b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.293+0000 7f3f951b5640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 0x7f3f900b8150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3f800097a0 tx=0x7f3f800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.293+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8003bb60 con 0x7f3f900cfc70 2023-11-08T19:25:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.293+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8003bcc0 con 0x7f3f900cfc70 2023-11-08T19:25:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.293+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80004270 con 0x7f3f900cfc70 2023-11-08T19:25:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.293+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f900b7950 con 0x7f3f900cfc70 2023-11-08T19:25:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.293+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f900b8690 con 0x7f3f900cfc70 2023-11-08T19:25:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.294+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f900b7e70 con 0x7f3f900cfc70 2023-11-08T19:25:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.295+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f80043070 con 0x7f3f900cfc70 2023-11-08T19:25:58.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.296+0000 7f3f867fc640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f7406aea0 0x7f3f7406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:58.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.296+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3f800ab9d0 con 0x7f3f900cfc70 2023-11-08T19:25:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.296+0000 7f3f959b6640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f7406aea0 0x7f3f7406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.297+0000 7f3f959b6640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f7406aea0 0x7f3f7406d360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c008870 tx=0x7f3f8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:58.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.300+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3f900b7e70 con 0x7f3f900cfc70 2023-11-08T19:25:58.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.548+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) v1 -- 0x7f3f900b7e70 con 0x7f3f900cfc70 2023-11-08T19:25:58.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.815+0000 7f3f867fc640 1 -- 172.21.15.119:0/449496040 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v80) v1 ==== 176+0+0 (secure 0 0 0) 0x7f3f900b7e70 con 0x7f3f900cfc70 2023-11-08T19:25:58.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.819+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f7406aea0 msgr2=0x7f3f7406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.819+0000 7f3f969b8640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f7406aea0 0x7f3f7406d360 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c008870 tx=0x7f3f8c008040 comp rx=0 tx=0).stop 2023-11-08T19:25:58.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.819+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 msgr2=0x7f3f900b8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:58.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.819+0000 7f3f969b8640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 0x7f3f900b8150 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3f800097a0 tx=0x7f3f800046c0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 shutdown_connections 2023-11-08T19:25:58.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f7406aea0 0x7f3f7406d360 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f9015cee0 0x7f3f901f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3f900cfc70 0x7f3f900b8150 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 --2- 172.21.15.119:0/449496040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f900cea70 0x7f3f900b6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:58.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 >> 172.21.15.119:0/449496040 conn(0x7f3f9009c100 msgr2=0x7f3f900c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:58.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 shutdown_connections 2023-11-08T19:25:58.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:58.820+0000 7f3f969b8640 1 -- 172.21.15.119:0/449496040 wait complete. 2023-11-08T19:25:58.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:25:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.044+0000 7f63db426640 1 Processor -- start 2023-11-08T19:25:59.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.044+0000 7f63db426640 1 -- start start 2023-11-08T19:25:59.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63db426640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63d4144960 0x7f63d415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63db426640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63db426640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d415ea60 con 0x7f63d4144580 2023-11-08T19:25:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63db426640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63d40c9150 con 0x7f63d4144960 2023-11-08T19:25:59.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63db426640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40c9360 con 0x7f63d415f1d0 2023-11-08T19:25:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63d89c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63d4144960 0x7f63d415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63d89c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f63d4144960 0x7f63d415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55696/0 (socket says 172.21.15.119:55696) 2023-11-08T19:25:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.046+0000 7f63d89c1640 1 -- 172.21.15.119:0/1549702080 learned_addr learned my addr 172.21.15.119:0/1549702080 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:59.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.047+0000 7f63d99c3640 1 --2- 172.21.15.119:0/1549702080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.047+0000 7f63d89c1640 1 -- 172.21.15.119:0/1549702080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 msgr2=0x7f63d40c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.047+0000 7f63d89c1640 1 --2- 172.21.15.119:0/1549702080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.047+0000 7f63d89c1640 1 -- 172.21.15.119:0/1549702080 >> v1:172.21.15.188:6789/0 conn(0x7f63d4144580 legacy=0x7f63d40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:59.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.047+0000 7f63d89c1640 1 -- 172.21.15.119:0/1549702080 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40c9640 con 0x7f63d4144960 2023-11-08T19:25:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.048+0000 7f63d89c1640 1 --2- 172.21.15.119:0/1549702080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144960 0x7f63d415bff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f63d40c9290 tx=0x7f63c402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e76f450ad9346a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.048+0000 7f63cbfff640 1 -- 172.21.15.119:0/1549702080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40317c0 con 0x7f63d4144960 2023-11-08T19:25:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.048+0000 7f63cbfff640 1 -- 172.21.15.119:0/1549702080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c4031920 con 0x7f63d4144960 2023-11-08T19:25:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.048+0000 7f63cbfff640 1 -- 172.21.15.119:0/1549702080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4031cb0 con 0x7f63d4144960 2023-11-08T19:25:59.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 -- 172.21.15.119:0/1549702080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144960 msgr2=0x7f63d415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 --2- 172.21.15.119:0/1549702080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144960 0x7f63d415bff0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f63d40c9290 tx=0x7f63c402b860 comp rx=0 tx=0).stop 2023-11-08T19:25:59.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 -- 172.21.15.119:0/1549702080 shutdown_connections 2023-11-08T19:25:59.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 --2- 172.21.15.119:0/1549702080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144960 0x7f63d415bff0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 --2- 172.21.15.119:0/1549702080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 -- 172.21.15.119:0/1549702080 >> 172.21.15.119:0/1549702080 conn(0x7f63d40cb610 msgr2=0x7f63d40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:59.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 -- 172.21.15.119:0/1549702080 shutdown_connections 2023-11-08T19:25:59.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.049+0000 7f63db426640 1 -- 172.21.15.119:0/1549702080 wait complete. 2023-11-08T19:25:59.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.051+0000 7f63db426640 1 Processor -- start 2023-11-08T19:25:59.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.051+0000 7f63db426640 1 -- start start 2023-11-08T19:25:59.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63db426640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 0x7f63d40b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63db426640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d4144960 0x7f63d4135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63db426640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63db426640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d41359d0 con 0x7f63d4144580 2023-11-08T19:25:59.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63db426640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40b1e50 con 0x7f63d4144960 2023-11-08T19:25:59.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63db426640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d40b2130 con 0x7f63d415f1d0 2023-11-08T19:25:59.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 0x7f63d40b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 0x7f63d40b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55700/0 (socket says 172.21.15.119:55700) 2023-11-08T19:25:59.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 -- 172.21.15.119:0/376831637 learned_addr learned my addr 172.21.15.119:0/376831637 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:59.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d89c1640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d4144960 0x7f63d4135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d99c3640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 -- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 msgr2=0x7f63d40c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 -- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d4144960 msgr2=0x7f63d4135490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.052+0000 7f63d91c2640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d4144960 0x7f63d4135490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63d91c2640 1 -- 172.21.15.119:0/376831637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c40094d0 con 0x7f63d4144580 2023-11-08T19:25:59.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63d91c2640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 0x7f63d40b1870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63d000c430 tx=0x7f63d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0011790 con 0x7f63d4144580 2023-11-08T19:25:59.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d00118f0 con 0x7f63d4144580 2023-11-08T19:25:59.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0011ca0 con 0x7f63d4144580 2023-11-08T19:25:59.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d40b2410 con 0x7f63d4144580 2023-11-08T19:25:59.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.053+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d40b29d0 con 0x7f63d4144580 2023-11-08T19:25:59.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.054+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d406acf0 con 0x7f63d4144580 2023-11-08T19:25:59.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.055+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63d001c460 con 0x7f63d4144580 2023-11-08T19:25:59.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.056+0000 7f63c9ffb640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f63b806ae70 0x7f63b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.056+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f63d00899d0 con 0x7f63d4144580 2023-11-08T19:25:59.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.057+0000 7f63d89c1640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f63b806ae70 0x7f63b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.057+0000 7f63d89c1640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f63b806ae70 0x7f63b806d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f63c4036000 tx=0x7f63c403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.062+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63d0054600 con 0x7f63d4144580 2023-11-08T19:25:59.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.251+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f63d4074430 con 0x7f63d4144580 2023-11-08T19:25:59.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.258+0000 7f63c9ffb640 1 -- 172.21.15.119:0/376831637 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v11)=0 v11) v1 ==== 122+0+0 (secure 0 0 0) 0x7f63d0057c20 con 0x7f63d4144580 2023-11-08T19:25:59.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f63b806ae70 msgr2=0x7f63b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f63b806ae70 0x7f63b806d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f63c4036000 tx=0x7f63c403d040 comp rx=0 tx=0).stop 2023-11-08T19:25:59.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 msgr2=0x7f63d40b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 0x7f63d40b1870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f63d000c430 tx=0x7f63d000c8f0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 shutdown_connections 2023-11-08T19:25:59.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f63b806ae70 0x7f63b806d330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63d415f1d0 0x7f63d40c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63d4144960 0x7f63d4135490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 --2- 172.21.15.119:0/376831637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f63d4144580 0x7f63d40b1870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.261+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 >> 172.21.15.119:0/376831637 conn(0x7f63d40cb610 msgr2=0x7f63d4072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:59.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.262+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 shutdown_connections 2023-11-08T19:25:59.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.262+0000 7f63db426640 1 -- 172.21.15.119:0/376831637 wait complete. 2023-11-08T19:25:59.281 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:25:59.282 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:25:59.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:25:59.516 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:25:59.516 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:59.517 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:59.575 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:59.576 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:59.632 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:25:59.633 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:25:59.683 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:25:59.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.940+0000 7fa3e9375640 1 Processor -- start 2023-11-08T19:25:59.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.940+0000 7fa3e9375640 1 -- start start 2023-11-08T19:25:59.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.941+0000 7fa3e9375640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406bee0 0x7fa3e406cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.941+0000 7fa3e9375640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3e406d520 0x7fa3e406c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.941+0000 7fa3e9375640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e40cd650 con 0x7fa3e4159860 2023-11-08T19:25:59.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.941+0000 7fa3e9375640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e406cbc0 con 0x7fa3e406d520 2023-11-08T19:25:59.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.941+0000 7fa3e9375640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e415cfa0 con 0x7fa3e406bee0 2023-11-08T19:25:59.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3e406d520 0x7fa3e406c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e8b74640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa3e4159860 0x7fa3e4159d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46852/0 (socket says 172.21.15.119:46852) 2023-11-08T19:25:59.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e8b74640 1 -- 172.21.15.119:0/1938994363 learned_addr learned my addr 172.21.15.119:0/1938994363 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:25:59.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa3e406d520 0x7fa3e406c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55706/0 (socket says 172.21.15.119:55706) 2023-11-08T19:25:59.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e3fff640 1 --2- 172.21.15.119:0/1938994363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406bee0 0x7fa3e406cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e37fe640 1 -- 172.21.15.119:0/1938994363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406bee0 msgr2=0x7fa3e406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:25:59.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e37fe640 1 --2- 172.21.15.119:0/1938994363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406bee0 0x7fa3e406cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e2ffd640 1 -- 172.21.15.119:0/1938994363 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743708540 0 0) 0x7fa3e40cd650 con 0x7fa3e4159860 2023-11-08T19:25:59.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e37fe640 1 -- 172.21.15.119:0/1938994363 >> v1:172.21.15.119:6789/0 conn(0x7fa3e4159860 legacy=0x7fa3e4159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.942+0000 7fa3e37fe640 1 -- 172.21.15.119:0/1938994363 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e406c800 con 0x7fa3e406d520 2023-11-08T19:25:59.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.943+0000 7fa3e37fe640 1 --2- 172.21.15.119:0/1938994363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e406d520 0x7fa3e406c2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400aea0 tx=0x7fa3d402cfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1100f0d1e58165cc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.944+0000 7fa3e2ffd640 1 -- 172.21.15.119:0/1938994363 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d402db40 con 0x7fa3e406d520 2023-11-08T19:25:59.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.944+0000 7fa3e2ffd640 1 -- 172.21.15.119:0/1938994363 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d402dca0 con 0x7fa3e406d520 2023-11-08T19:25:59.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.944+0000 7fa3e2ffd640 1 -- 172.21.15.119:0/1938994363 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d4008660 con 0x7fa3e406d520 2023-11-08T19:25:59.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.944+0000 7fa3e9375640 1 -- 172.21.15.119:0/1938994363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e406d520 msgr2=0x7fa3e406c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.944+0000 7fa3e9375640 1 --2- 172.21.15.119:0/1938994363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e406d520 0x7fa3e406c2c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400aea0 tx=0x7fa3d402cfa0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.945+0000 7fa3e9375640 1 -- 172.21.15.119:0/1938994363 shutdown_connections 2023-11-08T19:25:59.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.945+0000 7fa3e9375640 1 --2- 172.21.15.119:0/1938994363 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e406d520 0x7fa3e406c2c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.945+0000 7fa3e9375640 1 --2- 172.21.15.119:0/1938994363 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406bee0 0x7fa3e406cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.945+0000 7fa3e9375640 1 -- 172.21.15.119:0/1938994363 >> 172.21.15.119:0/1938994363 conn(0x7fa3e40b0560 msgr2=0x7fa3e40b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:25:59.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.945+0000 7fa3e9375640 1 -- 172.21.15.119:0/1938994363 shutdown_connections 2023-11-08T19:25:59.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.945+0000 7fa3e9375640 1 -- 172.21.15.119:0/1938994363 wait complete. 2023-11-08T19:25:59.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.946+0000 7fa3e9375640 1 Processor -- start 2023-11-08T19:25:59.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 -- start start 2023-11-08T19:25:59.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e406bee0 0x7fa3e40b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406d520 0x7fa3e40ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 0x7fa3e4157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40c4530 con 0x7fa3e4159860 2023-11-08T19:25:59.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e41f6c70 con 0x7fa3e406bee0 2023-11-08T19:25:59.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.947+0000 7fa3e9375640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40c3b90 con 0x7fa3e406d520 2023-11-08T19:25:59.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 0x7fa3e4157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 0x7fa3e4157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55720/0 (socket says 172.21.15.119:55720) 2023-11-08T19:25:59.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 -- 172.21.15.119:0/3984304338 learned_addr learned my addr 172.21.15.119:0/3984304338 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:25:59.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e37fe640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406d520 0x7fa3e40ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 -- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406d520 msgr2=0x7fa3e40ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e3fff640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e406bee0 0x7fa3e40b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406d520 0x7fa3e40ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 -- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e406bee0 msgr2=0x7fa3e40b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:25:59.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e406bee0 0x7fa3e40b1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:25:59.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 -- 172.21.15.119:0/3984304338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d400ab80 con 0x7fa3e4159860 2023-11-08T19:25:59.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e3fff640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e406bee0 0x7fa3e40b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:25:59.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.948+0000 7fa3e8b74640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 0x7fa3e4157e50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc001eb0 tx=0x7fa3dc00c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.949+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0074e0 con 0x7fa3e4159860 2023-11-08T19:25:59.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.949+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc007640 con 0x7fa3e4159860 2023-11-08T19:25:59.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.949+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc004a00 con 0x7fa3e4159860 2023-11-08T19:25:59.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.949+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e40c3e70 con 0x7fa3e4159860 2023-11-08T19:25:59.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.949+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e40b1ba0 con 0x7fa3e4159860 2023-11-08T19:25:59.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.950+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e40c4670 con 0x7fa3e4159860 2023-11-08T19:25:59.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.951+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3dc004b60 con 0x7fa3e4159860 2023-11-08T19:25:59.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.952+0000 7fa3e0ff9640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa3c406ae70 0x7fa3c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:25:59.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.952+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa3dc0897f0 con 0x7fa3e4159860 2023-11-08T19:25:59.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.952+0000 7fa3e3fff640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa3c406ae70 0x7fa3c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:25:59.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.953+0000 7fa3e3fff640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa3c406ae70 0x7fa3c406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc006fd0 tx=0x7fa3cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:25:59.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:25:59.956+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3e40c4670 con 0x7fa3e4159860 2023-11-08T19:26:00.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.145+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3e40c4670 con 0x7fa3e4159860 2023-11-08T19:26:00.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.147+0000 7fa3e0ff9640 1 -- 172.21.15.119:0/3984304338 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fa3e40c4670 con 0x7fa3e4159860 2023-11-08T19:26:00.149 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:00.149 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:00.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa3c406ae70 msgr2=0x7fa3c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa3c406ae70 0x7fa3c406d330 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc006fd0 tx=0x7fa3cc008040 comp rx=0 tx=0).stop 2023-11-08T19:26:00.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 msgr2=0x7fa3e4157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 0x7fa3e4157e50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc001eb0 tx=0x7fa3dc00c870 comp rx=0 tx=0).stop 2023-11-08T19:26:00.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 shutdown_connections 2023-11-08T19:26:00.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa3c406ae70 0x7fa3c406d330 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa3e4159860 0x7fa3e4157e50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa3e406d520 0x7fa3e40ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 --2- 172.21.15.119:0/3984304338 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa3e406bee0 0x7fa3e40b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.149+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 >> 172.21.15.119:0/3984304338 conn(0x7fa3e40b0560 msgr2=0x7fa3e40afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.150+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 shutdown_connections 2023-11-08T19:26:00.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.150+0000 7fa3e9375640 1 -- 172.21.15.119:0/3984304338 wait complete. 2023-11-08T19:26:00.171 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:26:00.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:00.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.366+0000 7f14316aa640 1 Processor -- start 2023-11-08T19:26:00.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.366+0000 7f14316aa640 1 -- start start 2023-11-08T19:26:00.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.367+0000 7f14316aa640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f142c0cccf0 0x7f142c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.367+0000 7f14316aa640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c071720 con 0x7f142c15e9a0 2023-11-08T19:26:00.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.367+0000 7f14316aa640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c0cf5e0 con 0x7f142c15deb0 2023-11-08T19:26:00.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.367+0000 7f14316aa640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142c0cf8c0 con 0x7f142c0cccf0 2023-11-08T19:26:00.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f142c15e9a0 0x7f142c06b5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46854/0 (socket says 172.21.15.119:46854) 2023-11-08T19:26:00.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142b7fe640 1 -- 172.21.15.119:0/3446139673 learned_addr learned my addr 172.21.15.119:0/3446139673 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:00.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2922931145 0 0) 0x7f142c071720 con 0x7f142c15e9a0 2023-11-08T19:26:00.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f1430ea9640 1 --2- 172.21.15.119:0/3446139673 >> v2:172.21.15.188:3300/0 conn(0x7f142c0cccf0 0x7f142c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1414005730 con 0x7f142c15e9a0 2023-11-08T19:26:00.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3363305563 0 0) 0x7f1414005730 con 0x7f142c15e9a0 2023-11-08T19:26:00.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 >> v2:172.21.15.188:3300/0 conn(0x7f142c0cccf0 msgr2=0x7f142c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 --2- 172.21.15.119:0/3446139673 >> v2:172.21.15.188:3300/0 conn(0x7f142c0cccf0 0x7f142c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 >> v1:172.21.15.188:6789/0 conn(0x7f142c15deb0 legacy=0x7f142c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0710d0 con 0x7f142c15e9a0 2023-11-08T19:26:00.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f1420001670 con 0x7f142c15e9a0 2023-11-08T19:26:00.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 >> v1:172.21.15.119:6789/0 conn(0x7f142c15e9a0 legacy=0x7f142c06b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.368+0000 7f142affd640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1414005b80 0x7f1414005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142affd640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f1414006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142affd640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14140074b0 0x7f141400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0cf8c0 con 0x7f14140069d0 2023-11-08T19:26:00.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141400e380 con 0x7f1414005b80 2023-11-08T19:26:00.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141400e540 con 0x7f14140074b0 2023-11-08T19:26:00.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142b7fe640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f1414006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142b7fe640 1 -- 172.21.15.119:0/3446139673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14140074b0 msgr2=0x7f141400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142b7fe640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14140074b0 0x7f141400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142b7fe640 1 -- 172.21.15.119:0/3446139673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1414005b80 msgr2=0x7f1414005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142b7fe640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1414005b80 0x7f1414005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f142b7fe640 1 -- 172.21.15.119:0/3446139673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1414005730 con 0x7f14140069d0 2023-11-08T19:26:00.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.369+0000 7f1430ea9640 1 -- 172.21.15.119:0/3446139673 reap_dead start 2023-11-08T19:26:00.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.370+0000 7f142b7fe640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f1414006e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f142000c210 tx=0x7f1414007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531d2217eee292a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.370+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420005ad0 con 0x7f14140069d0 2023-11-08T19:26:00.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.370+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1420005c50 con 0x7f14140069d0 2023-11-08T19:26:00.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.370+0000 7f142affd640 1 -- 172.21.15.119:0/3446139673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142002c870 con 0x7f14140069d0 2023-11-08T19:26:00.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 -- 172.21.15.119:0/3446139673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 msgr2=0x7f1414006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f1414006e50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f142000c210 tx=0x7f1414007a00 comp rx=0 tx=0).stop 2023-11-08T19:26:00.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 -- 172.21.15.119:0/3446139673 shutdown_connections 2023-11-08T19:26:00.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 --2- 172.21.15.119:0/3446139673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f1414006e50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 -- 172.21.15.119:0/3446139673 >> 172.21.15.119:0/3446139673 conn(0x7f142c0ceac0 msgr2=0x7f142c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 -- 172.21.15.119:0/3446139673 shutdown_connections 2023-11-08T19:26:00.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.371+0000 7f14316aa640 1 -- 172.21.15.119:0/3446139673 wait complete. 2023-11-08T19:26:00.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.373+0000 7f14316aa640 1 Processor -- start 2023-11-08T19:26:00.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.373+0000 7f14316aa640 1 -- start start 2023-11-08T19:26:00.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f14316aa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f142c151d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f14316aa640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f142c0cb820 0x7f142c0cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f14316aa640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f142c0c25f0 0x7f142c0c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f14316aa640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0cc1e0 con 0x7f14140069d0 2023-11-08T19:26:00.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f14316aa640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0c3320 con 0x7f142c0cb820 2023-11-08T19:26:00.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f14316aa640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c0c4670 con 0x7f142c0c25f0 2023-11-08T19:26:00.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f142c151d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f142c151d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55736/0 (socket says 172.21.15.119:55736) 2023-11-08T19:26:00.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 -- 172.21.15.119:0/3135855682 learned_addr learned my addr 172.21.15.119:0/3135855682 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:00.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 -- 172.21.15.119:0/3135855682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f142c0c25f0 msgr2=0x7f142c0c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f142c0c25f0 0x7f142c0c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 -- 172.21.15.119:0/3135855682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f142c0cb820 msgr2=0x7f142c0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f142c0cb820 0x7f142c0cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.374+0000 7f142bfff640 1 -- 172.21.15.119:0/3135855682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0c4950 con 0x7f14140069d0 2023-11-08T19:26:00.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.375+0000 7f142bfff640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f142c151d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f141c00c430 tx=0x7f141c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.375+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0052d0 con 0x7f14140069d0 2023-11-08T19:26:00.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.375+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c005470 con 0x7f14140069d0 2023-11-08T19:26:00.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.375+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c0c4c30 con 0x7f14140069d0 2023-11-08T19:26:00.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.375+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c007260 con 0x7f14140069d0 2023-11-08T19:26:00.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.375+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c0c5430 con 0x7f14140069d0 2023-11-08T19:26:00.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.376+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c04ece0 con 0x7f14140069d0 2023-11-08T19:26:00.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.377+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f141c0073c0 con 0x7f14140069d0 2023-11-08T19:26:00.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.378+0000 7f1428ff9640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f141806aea0 0x7f141806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.378+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f141c088d20 con 0x7f14140069d0 2023-11-08T19:26:00.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.378+0000 7f142b7fe640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f141806aea0 0x7f141806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.379+0000 7f142b7fe640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f141806aea0 0x7f141806d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1420034000 tx=0x7f1420039040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.382+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f141c053a80 con 0x7f14140069d0 2023-11-08T19:26:00.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.572+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f142c0cb1c0 con 0x7f14140069d0 2023-11-08T19:26:00.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.573+0000 7f1428ff9640 1 -- 172.21.15.119:0/3135855682 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f141c0570a0 con 0x7f14140069d0 2023-11-08T19:26:00.574 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:00.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f141806aea0 msgr2=0x7f141806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f141806aea0 0x7f141806d360 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1420034000 tx=0x7f1420039040 comp rx=0 tx=0).stop 2023-11-08T19:26:00.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 msgr2=0x7f142c151d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f142c151d60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f141c00c430 tx=0x7f141c00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 shutdown_connections 2023-11-08T19:26:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f141806aea0 0x7f141806d360 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f142c0c25f0 0x7f142c0c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f142c0cb820 0x7f142c0cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 --2- 172.21.15.119:0/3135855682 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14140069d0 0x7f142c151d60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 >> 172.21.15.119:0/3135855682 conn(0x7f142c0ceac0 msgr2=0x7f142c0ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 shutdown_connections 2023-11-08T19:26:00.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.576+0000 7f14316aa640 1 -- 172.21.15.119:0/3135855682 wait complete. 2023-11-08T19:26:00.596 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:00.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.814+0000 7f65b0810640 1 Processor -- start 2023-11-08T19:26:00.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.814+0000 7f65b0810640 1 -- start start 2023-11-08T19:26:00.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.815+0000 7f65b0810640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65ac0cfc20 0x7f65ac0d0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.815+0000 7f65b0810640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15e5c0 0x7f65ac15a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.815+0000 7f65b0810640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65ac15a890 0x7f65ac070ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65b0810640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac15b390 con 0x7f65ac0cfc20 2023-11-08T19:26:00.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65b0810640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac0cee80 con 0x7f65ac15a890 2023-11-08T19:26:00.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65b0810640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0d0540 con 0x7f65ac15e5c0 2023-11-08T19:26:00.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65ac0cfc20 0x7f65ac0d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65ac0cfc20 0x7f65ac0d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55740/0 (socket says 172.21.15.119:55740) 2023-11-08T19:26:00.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 -- 172.21.15.119:0/3918083741 learned_addr learned my addr 172.21.15.119:0/3918083741 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:00.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aad9d640 1 --2- 172.21.15.119:0/3918083741 >> v2:172.21.15.188:3300/0 conn(0x7f65ac15a890 0x7f65ac070ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65a9d9b640 1 --2- 172.21.15.119:0/3918083741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15e5c0 0x7f65ac15a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 -- 172.21.15.119:0/3918083741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15e5c0 msgr2=0x7f65ac15a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 --2- 172.21.15.119:0/3918083741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15e5c0 0x7f65ac15a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 -- 172.21.15.119:0/3918083741 >> v2:172.21.15.188:3300/0 conn(0x7f65ac15a890 msgr2=0x7f65ac070ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:00.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 --2- 172.21.15.119:0/3918083741 >> v2:172.21.15.188:3300/0 conn(0x7f65ac15a890 0x7f65ac070ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.816+0000 7f65aa59c640 1 -- 172.21.15.119:0/3918083741 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac15b0b0 con 0x7f65ac0cfc20 2023-11-08T19:26:00.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.818+0000 7f65aa59c640 1 --2- 172.21.15.119:0/3918083741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0d0000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65ac072360 tx=0x7f659c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14784e367accd5e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.818+0000 7f65a959a640 1 -- 172.21.15.119:0/3918083741 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c031870 con 0x7f65ac0cfc20 2023-11-08T19:26:00.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.818+0000 7f65a959a640 1 -- 172.21.15.119:0/3918083741 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c0319d0 con 0x7f65ac0cfc20 2023-11-08T19:26:00.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.818+0000 7f65a959a640 1 -- 172.21.15.119:0/3918083741 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c031d60 con 0x7f65ac0cfc20 2023-11-08T19:26:00.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 -- 172.21.15.119:0/3918083741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 msgr2=0x7f65ac0d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 --2- 172.21.15.119:0/3918083741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0d0000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f65ac072360 tx=0x7f659c02b910 comp rx=0 tx=0).stop 2023-11-08T19:26:00.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 -- 172.21.15.119:0/3918083741 shutdown_connections 2023-11-08T19:26:00.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 --2- 172.21.15.119:0/3918083741 >> v2:172.21.15.188:3300/0 conn(0x7f65ac15a890 0x7f65ac070ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 --2- 172.21.15.119:0/3918083741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15e5c0 0x7f65ac15a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 --2- 172.21.15.119:0/3918083741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0d0000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 -- 172.21.15.119:0/3918083741 >> 172.21.15.119:0/3918083741 conn(0x7f65ac06b230 msgr2=0x7f65ac06b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 -- 172.21.15.119:0/3918083741 shutdown_connections 2023-11-08T19:26:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.819+0000 7f65b0810640 1 -- 172.21.15.119:0/3918083741 wait complete. 2023-11-08T19:26:00.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.821+0000 7f65b0810640 1 Processor -- start 2023-11-08T19:26:00.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.821+0000 7f65b0810640 1 -- start start 2023-11-08T19:26:00.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65b0810640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65b0810640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15a890 0x7f65ac14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65b0810640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ac15e5c0 0x7f65ac0ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65b0810640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac14da60 con 0x7f65ac0cfc20 2023-11-08T19:26:00.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65b0810640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac14e100 con 0x7f65ac15e5c0 2023-11-08T19:26:00.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65b0810640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac14e3e0 con 0x7f65ac15a890 2023-11-08T19:26:00.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65aa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65aa59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55750/0 (socket says 172.21.15.119:55750) 2023-11-08T19:26:00.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.822+0000 7f65aa59c640 1 -- 172.21.15.119:0/2659895053 learned_addr learned my addr 172.21.15.119:0/2659895053 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:00.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65a9d9b640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15a890 0x7f65ac14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aad9d640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ac15e5c0 0x7f65ac0ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aa59c640 1 -- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15a890 msgr2=0x7f65ac14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aa59c640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15a890 0x7f65ac14d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aa59c640 1 -- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ac15e5c0 msgr2=0x7f65ac0ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:00.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aa59c640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ac15e5c0 0x7f65ac0ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:00.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aa59c640 1 -- 172.21.15.119:0/2659895053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c0094d0 con 0x7f65ac0cfc20 2023-11-08T19:26:00.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.823+0000 7f65aa59c640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0c6d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f659c0097c0 tx=0x7f659c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.824+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c03ac60 con 0x7f65ac0cfc20 2023-11-08T19:26:00.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.824+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c03adc0 con 0x7f65ac0cfc20 2023-11-08T19:26:00.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.824+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac14e6c0 con 0x7f65ac0cfc20 2023-11-08T19:26:00.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.824+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c044610 con 0x7f65ac0cfc20 2023-11-08T19:26:00.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.824+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac14ec80 con 0x7f65ac0cfc20 2023-11-08T19:26:00.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.825+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac074430 con 0x7f65ac0cfc20 2023-11-08T19:26:00.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.826+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f659c04d450 con 0x7f65ac0cfc20 2023-11-08T19:26:00.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.827+0000 7f659affd640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f658c06ae70 0x7f658c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:00.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.827+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f659c02d040 con 0x7f65ac0cfc20 2023-11-08T19:26:00.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.827+0000 7f65a9d9b640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f658c06ae70 0x7f658c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:00.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.828+0000 7f65a9d9b640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f658c06ae70 0x7f658c06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0c72a0 tx=0x7f65a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:00.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:00.832+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f659c076c10 con 0x7f65ac0cfc20 2023-11-08T19:26:01.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.020+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f65ac0ce050 con 0x7f65ac0cfc20 2023-11-08T19:26:01.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.022+0000 7f659affd640 1 -- 172.21.15.119:0/2659895053 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f659c035020 con 0x7f65ac0cfc20 2023-11-08T19:26:01.024 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:01.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f658c06ae70 msgr2=0x7f658c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f658c06ae70 0x7f658c06d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0c72a0 tx=0x7f65a0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:01.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 msgr2=0x7f65ac0c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0c6d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f659c0097c0 tx=0x7f659c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 shutdown_connections 2023-11-08T19:26:01.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f658c06ae70 0x7f658c06d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65ac15e5c0 0x7f65ac0ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65ac15a890 0x7f65ac14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 --2- 172.21.15.119:0/2659895053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65ac0cfc20 0x7f65ac0c6d60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.024+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 >> 172.21.15.119:0/2659895053 conn(0x7f65ac06b230 msgr2=0x7f65ac0b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.025+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 shutdown_connections 2023-11-08T19:26:01.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.025+0000 7f65b0810640 1 -- 172.21.15.119:0/2659895053 wait complete. 2023-11-08T19:26:01.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:01.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.265+0000 7f7790190640 1 Processor -- start 2023-11-08T19:26:01.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.265+0000 7f7790190640 1 -- start start 2023-11-08T19:26:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f7790190640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 0x7f778815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f7790190640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f778815ea60 con 0x7f7788144960 2023-11-08T19:26:01.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f7790190640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77880c9360 con 0x7f778815f1d0 2023-11-08T19:26:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f7790190640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77880c9150 con 0x7f7788144580 2023-11-08T19:26:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f778d72b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7788144960 0x7f77880cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46864/0 (socket says 172.21.15.119:46864) 2023-11-08T19:26:01.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f778d72b640 1 -- 172.21.15.119:0/3020544783 learned_addr learned my addr 172.21.15.119:0/3020544783 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:01.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.267+0000 7f778df2c640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 0x7f778815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1864799212 0 0) 0x7f778815ea60 con 0x7f7788144960 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7774003520 con 0x7f7788144960 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220260160 0 0) 0x7f7774003520 con 0x7f7788144960 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 msgr2=0x7f778815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 0x7f778815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 >> v1:172.21.15.188:6789/0 conn(0x7f778815f1d0 legacy=0x7f77880c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880c9640 con 0x7f7788144960 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f7778001670 con 0x7f7788144960 2023-11-08T19:26:01.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 >> v1:172.21.15.119:6789/0 conn(0x7f7788144960 legacy=0x7f77880cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77740039a0 0x7f7774003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 0x7f77740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7774004d00 0x7f7774005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880c9150 con 0x7f77740039a0 2023-11-08T19:26:01.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f777400bab0 con 0x7f7774004d00 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f777400bc70 con 0x7f7788144580 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.268+0000 7f778df2c640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77740039a0 0x7f7774003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778d72b640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 0x7f77740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778e72d640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7774004d00 0x7f7774005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778df2c640 1 -- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 msgr2=0x7f77740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778df2c640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7788144580 0x7f77740047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778df2c640 1 -- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7774004d00 msgr2=0x7f7774005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778df2c640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7774004d00 0x7f7774005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778df2c640 1 -- 172.21.15.119:0/3020544783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774003520 con 0x7f77740039a0 2023-11-08T19:26:01.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.269+0000 7f778e72d640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7774004d00 0x7f7774005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.270+0000 7f778df2c640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77740039a0 0x7f7774003dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7784009ac0 tx=0x7f77740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de9aabacfe929bd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.270+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778402c600 con 0x7f77740039a0 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.270+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778402c760 con 0x7f77740039a0 2023-11-08T19:26:01.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.270+0000 7f778cf2a640 1 -- 172.21.15.119:0/3020544783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778402caf0 con 0x7f77740039a0 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f7790190640 1 -- 172.21.15.119:0/3020544783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77740039a0 msgr2=0x7f7774003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f7790190640 1 --2- 172.21.15.119:0/3020544783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77740039a0 0x7f7774003dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7784009ac0 tx=0x7f77740057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f778e72d640 1 -- 172.21.15.119:0/3020544783 reap_dead start 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f7790190640 1 -- 172.21.15.119:0/3020544783 shutdown_connections 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f7790190640 1 -- 172.21.15.119:0/3020544783 >> 172.21.15.119:0/3020544783 conn(0x7f77880cb610 msgr2=0x7f77880cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f7790190640 1 -- 172.21.15.119:0/3020544783 shutdown_connections 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.271+0000 7f7790190640 1 -- 172.21.15.119:0/3020544783 wait complete. 2023-11-08T19:26:01.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.272+0000 7f7790190640 1 Processor -- start 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.273+0000 7f7790190640 1 -- start start 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.273+0000 7f7790190640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740039a0 0x7f778815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.273+0000 7f7790190640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 0x7f77880c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f7790190640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77880c6110 0x7f778815cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f7790190640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880c7210 con 0x7f778813fa40 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f7790190640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880c6590 con 0x7f77740039a0 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f7790190640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77880c6870 con 0x7f77880c6110 2023-11-08T19:26:01.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 0x7f77880c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 0x7f77880c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55766/0 (socket says 172.21.15.119:55766) 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 -- 172.21.15.119:0/826281307 learned_addr learned my addr 172.21.15.119:0/826281307 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 -- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77880c6110 msgr2=0x7f778815cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77880c6110 0x7f778815cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 -- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740039a0 msgr2=0x7f778815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778df2c640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740039a0 0x7f778815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740039a0 0x7f778815ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.274+0000 7f778d72b640 1 -- 172.21.15.119:0/826281307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880c6b50 con 0x7f778813fa40 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.275+0000 7f778d72b640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 0x7f77880c5bd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f777800d800 tx=0x7f777800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.275+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780074e0 con 0x7f778813fa40 2023-11-08T19:26:01.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.275+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778007640 con 0x7f778813fa40 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.275+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77880c6e30 con 0x7f778813fa40 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.275+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77780049e0 con 0x7f778813fa40 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.275+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778815d490 con 0x7f778813fa40 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.276+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778815d710 con 0x7f778813fa40 2023-11-08T19:26:01.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.277+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7778004b40 con 0x7f778813fa40 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.278+0000 7f777e7fc640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f776c06aea0 0x7f776c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.278+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f7778089440 con 0x7f778813fa40 2023-11-08T19:26:01.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.279+0000 7f778df2c640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f776c06aea0 0x7f776c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.279+0000 7f778df2c640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f776c06aea0 0x7f776c06d360 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7784036000 tx=0x7f7784030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.283+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7778054110 con 0x7f778813fa40 2023-11-08T19:26:01.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.475+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77880c7350 con 0x7f778813fa40 2023-11-08T19:26:01.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.476+0000 7f777e7fc640 1 -- 172.21.15.119:0/826281307 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77880c7350 con 0x7f778813fa40 2023-11-08T19:26:01.478 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f776c06aea0 msgr2=0x7f776c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f776c06aea0 0x7f776c06d360 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7784036000 tx=0x7f7784030000 comp rx=0 tx=0).stop 2023-11-08T19:26:01.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 msgr2=0x7f77880c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 0x7f77880c5bd0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f777800d800 tx=0x7f777800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 shutdown_connections 2023-11-08T19:26:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f776c06aea0 0x7f776c06d360 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77880c6110 0x7f778815cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f778813fa40 0x7f77880c5bd0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 --2- 172.21.15.119:0/826281307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f77740039a0 0x7f778815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 >> 172.21.15.119:0/826281307 conn(0x7f77880cb610 msgr2=0x7f77880c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 shutdown_connections 2023-11-08T19:26:01.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.479+0000 7f7790190640 1 -- 172.21.15.119:0/826281307 wait complete. 2023-11-08T19:26:01.498 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:26:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.717+0000 7f79db3e2640 1 Processor -- start 2023-11-08T19:26:01.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.717+0000 7f79db3e2640 1 -- start start 2023-11-08T19:26:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79db3e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79d415d4a0 0x7f79d40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79db3e2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d415df50 con 0x7f79d40cfa00 2023-11-08T19:26:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79db3e2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40d1e60 con 0x7f79d4161870 2023-11-08T19:26:01.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79db3e2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d40d2830 con 0x7f79d415d4a0 2023-11-08T19:26:01.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79d917e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79d415d4a0 0x7f79d40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79d897d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f79d40cfa00 0x7f79d4161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46874/0 (socket says 172.21.15.119:46874) 2023-11-08T19:26:01.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.719+0000 7f79d897d640 1 -- 172.21.15.119:0/4023711369 learned_addr learned my addr 172.21.15.119:0/4023711369 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:01.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.720+0000 7f79d917e640 1 -- 172.21.15.119:0/4023711369 >> v1:172.21.15.188:6789/0 conn(0x7f79d4161870 legacy=0x7f79d40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:01.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.720+0000 7f79d917e640 1 -- 172.21.15.119:0/4023711369 >> v1:172.21.15.119:6789/0 conn(0x7f79d40cfa00 legacy=0x7f79d4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.720+0000 7f79d917e640 1 -- 172.21.15.119:0/4023711369 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4161d40 con 0x7f79d415d4a0 2023-11-08T19:26:01.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.721+0000 7f79d917e640 1 --2- 172.21.15.119:0/4023711369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d415d4a0 0x7f79d40d22f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f79d00097a0 tx=0x7f79d002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6f50130e29bd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.721+0000 7f79cbfff640 1 -- 172.21.15.119:0/4023711369 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d002e070 con 0x7f79d415d4a0 2023-11-08T19:26:01.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79cbfff640 1 -- 172.21.15.119:0/4023711369 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d00315a0 con 0x7f79d415d4a0 2023-11-08T19:26:01.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79cbfff640 1 -- 172.21.15.119:0/4023711369 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0031930 con 0x7f79d415d4a0 2023-11-08T19:26:01.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 -- 172.21.15.119:0/4023711369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d415d4a0 msgr2=0x7f79d40d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4023711369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d415d4a0 0x7f79d40d22f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f79d00097a0 tx=0x7f79d002b840 comp rx=0 tx=0).stop 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 -- 172.21.15.119:0/4023711369 shutdown_connections 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4023711369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d415d4a0 0x7f79d40d22f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 -- 172.21.15.119:0/4023711369 >> 172.21.15.119:0/4023711369 conn(0x7f79d406a9f0 msgr2=0x7f79d406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 -- 172.21.15.119:0/4023711369 shutdown_connections 2023-11-08T19:26:01.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.722+0000 7f79db3e2640 1 -- 172.21.15.119:0/4023711369 wait complete. 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.724+0000 7f79db3e2640 1 Processor -- start 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.725+0000 7f79db3e2640 1 -- start start 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.725+0000 7f79db3e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 0x7f79d40c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.725+0000 7f79db3e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d415d4a0 0x7f79d40c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79db3e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d4161870 0x7f79d41f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79db3e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d414aac0 con 0x7f79d40cfa00 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79db3e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40c7dd0 con 0x7f79d4161870 2023-11-08T19:26:01.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79db3e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40c80b0 con 0x7f79d415d4a0 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 0x7f79d40c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 0x7f79d40c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55782/0 (socket says 172.21.15.119:55782) 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 -- 172.21.15.119:0/4033822685 learned_addr learned my addr 172.21.15.119:0/4033822685 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 -- 172.21.15.119:0/4033822685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d415d4a0 msgr2=0x7f79d40c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d415d4a0 0x7f79d40c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 -- 172.21.15.119:0/4033822685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d4161870 msgr2=0x7f79d41f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d4161870 0x7f79d41f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.726+0000 7f79d917e640 1 -- 172.21.15.119:0/4033822685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d0009400 con 0x7f79d40cfa00 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.727+0000 7f79d917e640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 0x7f79d40c77f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f79d0034040 tx=0x7f79d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.727+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0032900 con 0x7f79d40cfa00 2023-11-08T19:26:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.727+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0032a60 con 0x7f79d40cfa00 2023-11-08T19:26:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.727+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0032e10 con 0x7f79d40cfa00 2023-11-08T19:26:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.727+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d40c8390 con 0x7f79d40cfa00 2023-11-08T19:26:01.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.727+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d414aee0 con 0x7f79d40cfa00 2023-11-08T19:26:01.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.728+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d40cf4b0 con 0x7f79d40cfa00 2023-11-08T19:26:01.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.729+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79d003b3f0 con 0x7f79d40cfa00 2023-11-08T19:26:01.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.730+0000 7f79c9ffb640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79b806aef0 0x7f79b806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:01.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.730+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f79d00aba80 con 0x7f79d40cfa00 2023-11-08T19:26:01.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.731+0000 7f79d897d640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79b806aef0 0x7f79b806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:01.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.731+0000 7f79d897d640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79b806aef0 0x7f79b806d3b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f79d41f6fe0 tx=0x7f79c4007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:01.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.735+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79d0076730 con 0x7f79d40cfa00 2023-11-08T19:26:01.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.926+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f79d40d10f0 con 0x7f79d40cfa00 2023-11-08T19:26:01.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.927+0000 7f79c9ffb640 1 -- 172.21.15.119:0/4033822685 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f79d0079d50 con 0x7f79d40cfa00 2023-11-08T19:26:01.928 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:01.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79b806aef0 msgr2=0x7f79b806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79b806aef0 0x7f79b806d3b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f79d41f6fe0 tx=0x7f79c4007a40 comp rx=0 tx=0).stop 2023-11-08T19:26:01.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 msgr2=0x7f79d40c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:01.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 0x7f79d40c77f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f79d0034040 tx=0x7f79d00046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 shutdown_connections 2023-11-08T19:26:01.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79b806aef0 0x7f79b806d3b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79d4161870 0x7f79d41f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79d415d4a0 0x7f79d40c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 --2- 172.21.15.119:0/4033822685 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79d40cfa00 0x7f79d40c77f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:01.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 >> 172.21.15.119:0/4033822685 conn(0x7f79d406a9f0 msgr2=0x7f79d415b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:01.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 shutdown_connections 2023-11-08T19:26:01.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:01.929+0000 7f79db3e2640 1 -- 172.21.15.119:0/4033822685 wait complete. 2023-11-08T19:26:01.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.166+0000 7f3fb4b05640 1 Processor -- start 2023-11-08T19:26:02.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.166+0000 7f3fb4b05640 1 -- start start 2023-11-08T19:26:02.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.168+0000 7f3fb4b05640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3fb006a970 0x7f3fb006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.168+0000 7f3fb4b05640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3fb00c9150 0x7f3fb00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.168+0000 7f3fb4b05640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fb006d4b0 con 0x7f3fb00ccc10 2023-11-08T19:26:02.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.168+0000 7f3fb4b05640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb01619b0 con 0x7f3fb006a970 2023-11-08T19:26:02.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.168+0000 7f3fb4b05640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fb00cd080 con 0x7f3fb00c9150 2023-11-08T19:26:02.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.168+0000 7f3fae59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3fb006a970 0x7f3fb006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.169+0000 7f3fae59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3fb006a970 0x7f3fb006cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55794/0 (socket says 172.21.15.119:55794) 2023-11-08T19:26:02.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.169+0000 7f3fae59c640 1 -- 172.21.15.119:0/474082794 learned_addr learned my addr 172.21.15.119:0/474082794 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:02.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.169+0000 7f3fae59c640 1 -- 172.21.15.119:0/474082794 >> v2:172.21.15.188:3300/0 conn(0x7f3fb00c9150 msgr2=0x7f3fb00cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:02.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.169+0000 7f3fae59c640 1 --2- 172.21.15.119:0/474082794 >> v2:172.21.15.188:3300/0 conn(0x7f3fb00c9150 0x7f3fb00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.169+0000 7f3fae59c640 1 -- 172.21.15.119:0/474082794 >> v1:172.21.15.188:6789/0 conn(0x7f3fb00ccc10 legacy=0x7f3fb01653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:02.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.169+0000 7f3fae59c640 1 -- 172.21.15.119:0/474082794 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb0147b80 con 0x7f3fb006a970 2023-11-08T19:26:02.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.170+0000 7f3fae59c640 1 --2- 172.21.15.119:0/474082794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb006a970 0x7f3fb006cf70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00098a0 tx=0x7f3fa002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae989c5f9024d263 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.170+0000 7f3fadd9b640 1 -- 172.21.15.119:0/474082794 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00317c0 con 0x7f3fb006a970 2023-11-08T19:26:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.170+0000 7f3fadd9b640 1 -- 172.21.15.119:0/474082794 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0031920 con 0x7f3fb006a970 2023-11-08T19:26:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.170+0000 7f3fadd9b640 1 -- 172.21.15.119:0/474082794 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0031cb0 con 0x7f3fb006a970 2023-11-08T19:26:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 -- 172.21.15.119:0/474082794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb006a970 msgr2=0x7f3fb006cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/474082794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb006a970 0x7f3fb006cf70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3fa00098a0 tx=0x7f3fa002b860 comp rx=0 tx=0).stop 2023-11-08T19:26:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 -- 172.21.15.119:0/474082794 shutdown_connections 2023-11-08T19:26:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/474082794 >> v2:172.21.15.188:3300/0 conn(0x7f3fb00c9150 0x7f3fb00cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/474082794 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb006a970 0x7f3fb006cf70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 -- 172.21.15.119:0/474082794 >> 172.21.15.119:0/474082794 conn(0x7f3fb00cb610 msgr2=0x7f3fb00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:02.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 -- 172.21.15.119:0/474082794 shutdown_connections 2023-11-08T19:26:02.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.171+0000 7f3fb4b05640 1 -- 172.21.15.119:0/474082794 wait complete. 2023-11-08T19:26:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.173+0000 7f3fb4b05640 1 Processor -- start 2023-11-08T19:26:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.173+0000 7f3fb4b05640 1 -- start start 2023-11-08T19:26:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.174+0000 7f3fb4b05640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb006a970 0x7f3fb0160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.174+0000 7f3fb4b05640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 0x7f3fb01f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.174+0000 7f3fb4b05640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb00ccc10 0x7f3fb0163b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.174+0000 7f3fb4b05640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00c7730 con 0x7f3fb00c9150 2023-11-08T19:26:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.174+0000 7f3fb4b05640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb00c7140 con 0x7f3fb00ccc10 2023-11-08T19:26:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.174+0000 7f3fb4b05640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb01f7ce0 con 0x7f3fb006a970 2023-11-08T19:26:02.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 0x7f3fb01f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 0x7f3fb01f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55802/0 (socket says 172.21.15.119:55802) 2023-11-08T19:26:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 -- 172.21.15.119:0/39200845 learned_addr learned my addr 172.21.15.119:0/39200845 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fae59c640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb006a970 0x7f3fb0160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3faed9d640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb00ccc10 0x7f3fb0163b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 -- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb006a970 msgr2=0x7f3fb0160d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb006a970 0x7f3fb0160d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 -- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb00ccc10 msgr2=0x7f3fb0163b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb00ccc10 0x7f3fb0163b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 -- 172.21.15.119:0/39200845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00094d0 con 0x7f3fb00c9150 2023-11-08T19:26:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fae59c640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb006a970 0x7f3fb0160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.175+0000 7f3fa7fff640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 0x7f3fb01f71d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b4a0 tx=0x7f3f9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.176+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98010610 con 0x7f3fb00c9150 2023-11-08T19:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.176+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f98010770 con 0x7f3fb00c9150 2023-11-08T19:26:02.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.176+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98010b20 con 0x7f3fb00c9150 2023-11-08T19:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.176+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb01f7fc0 con 0x7f3fb00c9150 2023-11-08T19:26:02.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.176+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb00b25f0 con 0x7f3fb00c9150 2023-11-08T19:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.177+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb00c7870 con 0x7f3fb00c9150 2023-11-08T19:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.178+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3f98010c80 con 0x7f3fb00c9150 2023-11-08T19:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.179+0000 7f3fa6ffd640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f9006ae70 0x7f3f9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.179+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3f98089620 con 0x7f3fb00c9150 2023-11-08T19:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.179+0000 7f3fae59c640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f9006ae70 0x7f3f9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.180+0000 7f3fae59c640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f9006ae70 0x7f3f9006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0036000 tx=0x7f3fa003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.183+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fb00c7870 con 0x7f3fb00c9150 2023-11-08T19:26:02.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.385+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3fb00c7870 con 0x7f3fb00c9150 2023-11-08T19:26:02.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.386+0000 7f3fa6ffd640 1 -- 172.21.15.119:0/39200845 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3fb00c7870 con 0x7f3fb00c9150 2023-11-08T19:26:02.387 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:26:02.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f9006ae70 msgr2=0x7f3f9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f9006ae70 0x7f3f9006d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0036000 tx=0x7f3fa003d040 comp rx=0 tx=0).stop 2023-11-08T19:26:02.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 msgr2=0x7f3fb01f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 0x7f3fb01f71d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800b4a0 tx=0x7f3f9800b960 comp rx=0 tx=0).stop 2023-11-08T19:26:02.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 shutdown_connections 2023-11-08T19:26:02.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3f9006ae70 0x7f3f9006d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fb00ccc10 0x7f3fb0163b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fb00c9150 0x7f3fb01f71d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 --2- 172.21.15.119:0/39200845 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fb006a970 0x7f3fb0160d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 >> 172.21.15.119:0/39200845 conn(0x7f3fb00cb610 msgr2=0x7f3fb006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:02.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 shutdown_connections 2023-11-08T19:26:02.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.389+0000 7f3fb4b05640 1 -- 172.21.15.119:0/39200845 wait complete. 2023-11-08T19:26:02.407 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:26:02.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:02.408 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-08T19:26:02.501 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:26:02.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:02.502 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:26:02.552 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:02.552 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:02.612 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:02.612 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:02.665 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:02.665 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:02.715 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.937+0000 7f13ab456640 1 Processor -- start 2023-11-08T19:26:02.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.937+0000 7f13ab456640 1 -- start start 2023-11-08T19:26:02.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.938+0000 7f13ab456640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13a41447a0 0x7f13a415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.938+0000 7f13ab456640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cad30 0x7f13a406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.938+0000 7f13ab456640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a40c3a50 con 0x7f13a40cb170 2023-11-08T19:26:02.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13ab456640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a4144b80 con 0x7f13a41447a0 2023-11-08T19:26:02.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13ab456640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a40c9150 con 0x7f13a40cad30 2023-11-08T19:26:02.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13a41447a0 0x7f13a415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f13a41447a0 0x7f13a415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55806/0 (socket says 172.21.15.119:55806) 2023-11-08T19:26:02.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 -- 172.21.15.119:0/1351757737 learned_addr learned my addr 172.21.15.119:0/1351757737 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:02.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 -- 172.21.15.119:0/1351757737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cad30 msgr2=0x7f13a406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:02.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 --2- 172.21.15.119:0/1351757737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cad30 0x7f13a406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 -- 172.21.15.119:0/1351757737 >> v1:172.21.15.188:6789/0 conn(0x7f13a40cb170 legacy=0x7f13a406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:02.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.939+0000 7f13a91f2640 1 -- 172.21.15.119:0/1351757737 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40c9410 con 0x7f13a41447a0 2023-11-08T19:26:02.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.940+0000 7f13a91f2640 1 --2- 172.21.15.119:0/1351757737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a415f4c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009e50 tx=0x7f13a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78918afb5432a31e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.941+0000 7f139bfff640 1 -- 172.21.15.119:0/1351757737 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a002ca70 con 0x7f13a41447a0 2023-11-08T19:26:02.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.941+0000 7f139bfff640 1 -- 172.21.15.119:0/1351757737 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a002cbd0 con 0x7f13a41447a0 2023-11-08T19:26:02.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.941+0000 7f139bfff640 1 -- 172.21.15.119:0/1351757737 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a00343f0 con 0x7f13a41447a0 2023-11-08T19:26:02.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.941+0000 7f13ab456640 1 -- 172.21.15.119:0/1351757737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 msgr2=0x7f13a415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.941+0000 7f13ab456640 1 --2- 172.21.15.119:0/1351757737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a415f4c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009e50 tx=0x7f13a002bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.942+0000 7f13ab456640 1 -- 172.21.15.119:0/1351757737 shutdown_connections 2023-11-08T19:26:02.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.942+0000 7f13ab456640 1 --2- 172.21.15.119:0/1351757737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cad30 0x7f13a406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.942+0000 7f13ab456640 1 --2- 172.21.15.119:0/1351757737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a415f4c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.942+0000 7f13ab456640 1 -- 172.21.15.119:0/1351757737 >> 172.21.15.119:0/1351757737 conn(0x7f13a40cb610 msgr2=0x7f13a40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:02.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.942+0000 7f13ab456640 1 -- 172.21.15.119:0/1351757737 shutdown_connections 2023-11-08T19:26:02.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.942+0000 7f13ab456640 1 -- 172.21.15.119:0/1351757737 wait complete. 2023-11-08T19:26:02.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.943+0000 7f13ab456640 1 Processor -- start 2023-11-08T19:26:02.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 -- start start 2023-11-08T19:26:02.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13a40cad30 0x7f13a41f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cb170 0x7f13a4142430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a4141fa0 con 0x7f13a41447a0 2023-11-08T19:26:02.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a413a190 con 0x7f13a40cad30 2023-11-08T19:26:02.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.944+0000 7f13ab456640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a413a470 con 0x7f13a40cb170 2023-11-08T19:26:02.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a40c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55822/0 (socket says 172.21.15.119:55822) 2023-11-08T19:26:02.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 -- 172.21.15.119:0/1850877629 learned_addr learned my addr 172.21.15.119:0/1850877629 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:02.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a91f2640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13a40cad30 0x7f13a41f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 -- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cb170 msgr2=0x7f13a4142430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cb170 0x7f13a4142430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 -- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13a40cad30 msgr2=0x7f13a41f4000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:02.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13a40cad30 0x7f13a41f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:02.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.945+0000 7f13a99f3640 1 -- 172.21.15.119:0/1850877629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0009b30 con 0x7f13a41447a0 2023-11-08T19:26:02.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.946+0000 7f13a99f3640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a40c84e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c3e0 tx=0x7f139c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.946+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c011790 con 0x7f13a41447a0 2023-11-08T19:26:02.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.946+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c0118f0 con 0x7f13a41447a0 2023-11-08T19:26:02.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.946+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c011ce0 con 0x7f13a41447a0 2023-11-08T19:26:02.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.946+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a413a750 con 0x7f13a41447a0 2023-11-08T19:26:02.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.946+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a41f4540 con 0x7f13a41447a0 2023-11-08T19:26:02.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.947+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a4142150 con 0x7f13a41447a0 2023-11-08T19:26:02.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.948+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f139c01c460 con 0x7f13a41447a0 2023-11-08T19:26:02.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.949+0000 7f1399ffb640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f138806aea0 0x7f138806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:02.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.949+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f139c0897c0 con 0x7f13a41447a0 2023-11-08T19:26:02.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.949+0000 7f13a91f2640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f138806aea0 0x7f138806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:02.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.950+0000 7f13a91f2640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f138806aea0 0x7f138806d360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13a0035000 tx=0x7f13a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:02.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:02.953+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13a4142150 con 0x7f13a41447a0 2023-11-08T19:26:03.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.143+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13a41f4700 con 0x7f13a41447a0 2023-11-08T19:26:03.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.146+0000 7f1399ffb640 1 -- 172.21.15.119:0/1850877629 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f139c054520 con 0x7f13a41447a0 2023-11-08T19:26:03.147 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:03.148 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:03.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f138806aea0 msgr2=0x7f138806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f138806aea0 0x7f138806d360 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13a0035000 tx=0x7f13a0030000 comp rx=0 tx=0).stop 2023-11-08T19:26:03.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 msgr2=0x7f13a40c84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a40c84e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f139c00c3e0 tx=0x7f139c00c8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 shutdown_connections 2023-11-08T19:26:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f138806aea0 0x7f138806d360 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f13a41447a0 0x7f13a40c84e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13a40cb170 0x7f13a4142430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 --2- 172.21.15.119:0/1850877629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13a40cad30 0x7f13a41f4000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.148+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 >> 172.21.15.119:0/1850877629 conn(0x7f13a40cb610 msgr2=0x7f13a40c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:03.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.149+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 shutdown_connections 2023-11-08T19:26:03.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.149+0000 7f13ab456640 1 -- 172.21.15.119:0/1850877629 wait complete. 2023-11-08T19:26:03.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:26:03.171 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:03.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.386+0000 7f0b53ba9640 1 Processor -- start 2023-11-08T19:26:03.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.386+0000 7f0b53ba9640 1 -- start start 2023-11-08T19:26:03.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.387+0000 7f0b53ba9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b4c144960 0x7f0b4c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.387+0000 7f0b53ba9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c15ea60 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.387+0000 7f0b53ba9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b4c0c9360 con 0x7f0b4c144580 2023-11-08T19:26:03.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.387+0000 7f0b53ba9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c9150 con 0x7f0b4c144960 2023-11-08T19:26:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b51144640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b4c144960 0x7f0b4c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b52146640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0b4c15f1d0 0x7f0b4c0c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46884/0 (socket says 172.21.15.119:46884) 2023-11-08T19:26:03.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b52146640 1 -- 172.21.15.119:0/2254517524 learned_addr learned my addr 172.21.15.119:0/2254517524 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b51144640 1 -- 172.21.15.119:0/2254517524 >> v1:172.21.15.188:6789/0 conn(0x7f0b4c144580 legacy=0x7f0b4c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b51144640 1 -- 172.21.15.119:0/2254517524 >> v1:172.21.15.119:6789/0 conn(0x7f0b4c15f1d0 legacy=0x7f0b4c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b50943640 1 -- 172.21.15.119:0/2254517524 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223367475 0 0) 0x7f0b4c15ea60 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.388+0000 7f0b51144640 1 -- 172.21.15.119:0/2254517524 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0c9640 con 0x7f0b4c144960 2023-11-08T19:26:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.390+0000 7f0b51144640 1 --2- 172.21.15.119:0/2254517524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c144960 0x7f0b4c15bff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0097a0 tx=0x7f0b3c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9bec80e31923cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.390+0000 7f0b50943640 1 -- 172.21.15.119:0/2254517524 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c02e070 con 0x7f0b4c144960 2023-11-08T19:26:03.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.390+0000 7f0b50943640 1 -- 172.21.15.119:0/2254517524 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0315a0 con 0x7f0b4c144960 2023-11-08T19:26:03.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.390+0000 7f0b50943640 1 -- 172.21.15.119:0/2254517524 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c031930 con 0x7f0b4c144960 2023-11-08T19:26:03.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.390+0000 7f0b53ba9640 1 -- 172.21.15.119:0/2254517524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c144960 msgr2=0x7f0b4c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.390+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/2254517524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c144960 0x7f0b4c15bff0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c0097a0 tx=0x7f0b3c02b840 comp rx=0 tx=0).stop 2023-11-08T19:26:03.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.391+0000 7f0b53ba9640 1 -- 172.21.15.119:0/2254517524 shutdown_connections 2023-11-08T19:26:03.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.391+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/2254517524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c144960 0x7f0b4c15bff0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.391+0000 7f0b53ba9640 1 -- 172.21.15.119:0/2254517524 >> 172.21.15.119:0/2254517524 conn(0x7f0b4c0cb610 msgr2=0x7f0b4c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:03.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.391+0000 7f0b53ba9640 1 -- 172.21.15.119:0/2254517524 shutdown_connections 2023-11-08T19:26:03.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.391+0000 7f0b53ba9640 1 -- 172.21.15.119:0/2254517524 wait complete. 2023-11-08T19:26:03.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.393+0000 7f0b53ba9640 1 Processor -- start 2023-11-08T19:26:03.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.393+0000 7f0b53ba9640 1 -- start start 2023-11-08T19:26:03.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.394+0000 7f0b53ba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b4c144580 0x7f0b4c1e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.394+0000 7f0b53ba9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4c144960 0x7f0b4c0c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.394+0000 7f0b53ba9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 0x7f0b4c141650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.394+0000 7f0b53ba9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c84f0 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.394+0000 7f0b53ba9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c5fb0 con 0x7f0b4c144580 2023-11-08T19:26:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.394+0000 7f0b53ba9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4c0c6290 con 0x7f0b4c144960 2023-11-08T19:26:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 0x7f0b4c141650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 0x7f0b4c141650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55832/0 (socket says 172.21.15.119:55832) 2023-11-08T19:26:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 -- 172.21.15.119:0/1636813107 learned_addr learned my addr 172.21.15.119:0/1636813107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b51945640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b4c144580 0x7f0b4c1e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b51144640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4c144960 0x7f0b4c0c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 -- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4c144960 msgr2=0x7f0b4c0c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4c144960 0x7f0b4c0c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 -- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b4c144580 msgr2=0x7f0b4c1e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b4c144580 0x7f0b4c1e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 -- 172.21.15.119:0/1636813107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3c009400 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b51144640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4c144960 0x7f0b4c0c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.395+0000 7f0b52146640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 0x7f0b4c141650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c780 tx=0x7f0b4400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.396+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400ce30 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.396+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4400a640 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.396+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b4c0c6570 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.396+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400a9f0 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.396+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b4c0c6b30 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.397+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4c06acf0 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.398+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b4400ab50 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.399+0000 7f0b427fc640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b3006ae70 0x7f0b3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.399+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f0b44089170 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.399+0000 7f0b51945640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b3006ae70 0x7f0b3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.400+0000 7f0b51945640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b3006ae70 0x7f0b3006d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0b48004ea0 tx=0x7f0b48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.403+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b44053da0 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.592+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b4c0c6d20 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.592+0000 7f0b427fc640 1 -- 172.21.15.119:0/1636813107 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b440573c0 con 0x7f0b4c15f1d0 2023-11-08T19:26:03.594 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:03.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b3006ae70 msgr2=0x7f0b3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b3006ae70 0x7f0b3006d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0b48004ea0 tx=0x7f0b48009040 comp rx=0 tx=0).stop 2023-11-08T19:26:03.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 msgr2=0x7f0b4c141650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 0x7f0b4c141650 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400c780 tx=0x7f0b4400cc40 comp rx=0 tx=0).stop 2023-11-08T19:26:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 shutdown_connections 2023-11-08T19:26:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b3006ae70 0x7f0b3006d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b4c15f1d0 0x7f0b4c141650 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b4c144960 0x7f0b4c0c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 --2- 172.21.15.119:0/1636813107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b4c144580 0x7f0b4c1e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 >> 172.21.15.119:0/1636813107 conn(0x7f0b4c0cb610 msgr2=0x7f0b4c0c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 shutdown_connections 2023-11-08T19:26:03.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.595+0000 7f0b53ba9640 1 -- 172.21.15.119:0/1636813107 wait complete. 2023-11-08T19:26:03.615 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:03.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.828+0000 7f0398807640 1 Processor -- start 2023-11-08T19:26:03.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.828+0000 7f0398807640 1 -- start start 2023-11-08T19:26:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f0398807640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f039415d000 0x7f03940cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f0398807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 0x7f0394160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f0398807640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039406d220 con 0x7f03940c3a50 2023-11-08T19:26:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f0398807640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03940ccd50 con 0x7f039415d000 2023-11-08T19:26:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f0398807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03940aca40 con 0x7f039406a830 2023-11-08T19:26:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f038bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f03940c3a50 0x7f039406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46886/0 (socket says 172.21.15.119:46886) 2023-11-08T19:26:03.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f038bfff640 1 -- 172.21.15.119:0/4278720692 learned_addr learned my addr 172.21.15.119:0/4278720692 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f039259c640 1 --2- 172.21.15.119:0/4278720692 >> v2:172.21.15.188:3300/0 conn(0x7f039415d000 0x7f03940cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.830+0000 7f0392d9d640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 0x7f0394160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918290860 0 0) 0x7f039406d220 con 0x7f03940c3a50 2023-11-08T19:26:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0380003520 con 0x7f03940c3a50 2023-11-08T19:26:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3562271967 0 0) 0x7f0380003520 con 0x7f03940c3a50 2023-11-08T19:26:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 msgr2=0x7f0394160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 0x7f0394160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 >> v2:172.21.15.188:3300/0 conn(0x7f039415d000 msgr2=0x7f03940cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 --2- 172.21.15.119:0/4278720692 >> v2:172.21.15.188:3300/0 conn(0x7f039415d000 0x7f03940cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940acd20 con 0x7f03940c3a50 2023-11-08T19:26:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f037c001690 con 0x7f03940c3a50 2023-11-08T19:26:03.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 >> v1:172.21.15.119:6789/0 conn(0x7f03940c3a50 legacy=0x7f039406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0380003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03800047c0 0x7f0380004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 0x7f0380005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940ccd50 con 0x7f0380003970 2023-11-08T19:26:03.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940aca40 con 0x7f03800047c0 2023-11-08T19:26:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f038000ba00 con 0x7f039406a830 2023-11-08T19:26:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.831+0000 7f039259c640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0380003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.832+0000 7f038bfff640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03800047c0 0x7f0380004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.832+0000 7f039259c640 1 -- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 msgr2=0x7f0380005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.832+0000 7f039259c640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f039406a830 0x7f0380005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.832+0000 7f039259c640 1 -- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03800047c0 msgr2=0x7f0380004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.832+0000 7f039259c640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03800047c0 0x7f0380004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.832+0000 7f039259c640 1 -- 172.21.15.119:0/4278720692 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0380003520 con 0x7f0380003970 2023-11-08T19:26:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.833+0000 7f039259c640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0380003d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0384009ac0 tx=0x7f03800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2e01e02446956 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.833+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038402c710 con 0x7f0380003970 2023-11-08T19:26:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.833+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038402c870 con 0x7f0380003970 2023-11-08T19:26:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.833+0000 7f0391d9b640 1 -- 172.21.15.119:0/4278720692 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038402cc00 con 0x7f0380003970 2023-11-08T19:26:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0398807640 1 -- 172.21.15.119:0/4278720692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 msgr2=0x7f0380003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0398807640 1 --2- 172.21.15.119:0/4278720692 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0380003d90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0384009ac0 tx=0x7f03800057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0392d9d640 1 -- 172.21.15.119:0/4278720692 reap_dead start 2023-11-08T19:26:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0398807640 1 -- 172.21.15.119:0/4278720692 shutdown_connections 2023-11-08T19:26:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0398807640 1 -- 172.21.15.119:0/4278720692 >> 172.21.15.119:0/4278720692 conn(0x7f03940cb610 msgr2=0x7f03940cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0398807640 1 -- 172.21.15.119:0/4278720692 shutdown_connections 2023-11-08T19:26:03.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.834+0000 7f0398807640 1 -- 172.21.15.119:0/4278720692 wait complete. 2023-11-08T19:26:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.836+0000 7f0398807640 1 Processor -- start 2023-11-08T19:26:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.836+0000 7f0398807640 1 -- start start 2023-11-08T19:26:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f0398807640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0394070420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f0398807640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0394151570 0x7f03941519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f0398807640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0394157700 0x7f0394157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f0398807640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0394151f30 con 0x7f0380003970 2023-11-08T19:26:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f0398807640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0394158470 con 0x7f0394157700 2023-11-08T19:26:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f0398807640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0394158750 con 0x7f0394151570 2023-11-08T19:26:03.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f039259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0394070420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f039259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0394070420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55862/0 (socket says 172.21.15.119:55862) 2023-11-08T19:26:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f039259c640 1 -- 172.21.15.119:0/83310006 learned_addr learned my addr 172.21.15.119:0/83310006 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.837+0000 7f039259c640 1 -- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0394151570 msgr2=0x7f03941519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f039259c640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0394151570 0x7f03941519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f039259c640 1 -- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0394157700 msgr2=0x7f0394157b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f0392d9d640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0394157700 0x7f0394157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f039259c640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0394157700 0x7f0394157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f039259c640 1 -- 172.21.15.119:0/83310006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394158a30 con 0x7f0380003970 2023-11-08T19:26:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f039259c640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0394070420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0384037040 tx=0x7f0384004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384003810 con 0x7f0380003970 2023-11-08T19:26:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0384003970 con 0x7f0380003970 2023-11-08T19:26:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0394158d10 con 0x7f0380003970 2023-11-08T19:26:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0384003d20 con 0x7f0380003970 2023-11-08T19:26:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.838+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03941580b0 con 0x7f0380003970 2023-11-08T19:26:03.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.839+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039406cd50 con 0x7f0380003970 2023-11-08T19:26:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.840+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0384040450 con 0x7f0380003970 2023-11-08T19:26:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.841+0000 7f038affd640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f037406aea0 0x7f037406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.841+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f03840ab850 con 0x7f0380003970 2023-11-08T19:26:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.842+0000 7f038bfff640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f037406aea0 0x7f037406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:03.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.842+0000 7f038bfff640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f037406aea0 0x7f037406d360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f037c0017c0 tx=0x7f037c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:03.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:03.846+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0384076480 con 0x7f0380003970 2023-11-08T19:26:04.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.035+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03940b0240 con 0x7f0380003970 2023-11-08T19:26:04.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.036+0000 7f038affd640 1 -- 172.21.15.119:0/83310006 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0384079aa0 con 0x7f0380003970 2023-11-08T19:26:04.038 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:04.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f037406aea0 msgr2=0x7f037406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f037406aea0 0x7f037406d360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f037c0017c0 tx=0x7f037c009040 comp rx=0 tx=0).stop 2023-11-08T19:26:04.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 msgr2=0x7f0394070420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0394070420 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0384037040 tx=0x7f0384004730 comp rx=0 tx=0).stop 2023-11-08T19:26:04.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 shutdown_connections 2023-11-08T19:26:04.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f037406aea0 0x7f037406d360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0394157700 0x7f0394157b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0394151570 0x7f03941519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 --2- 172.21.15.119:0/83310006 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0380003970 0x7f0394070420 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 >> 172.21.15.119:0/83310006 conn(0x7f03940cb610 msgr2=0x7f039415a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 shutdown_connections 2023-11-08T19:26:04.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.039+0000 7f0398807640 1 -- 172.21.15.119:0/83310006 wait complete. 2023-11-08T19:26:04.060 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:04.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.279+0000 7f49ca973640 1 Processor -- start 2023-11-08T19:26:04.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.280+0000 7f49ca973640 1 -- start start 2023-11-08T19:26:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.281+0000 7f49ca973640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f49c415d9f0 0x7f49c415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.281+0000 7f49ca973640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c415e440 con 0x7f49c4143600 2023-11-08T19:26:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49ca973640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c415e720 con 0x7f49c406cb40 2023-11-08T19:26:04.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49ca973640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40c4f50 con 0x7f49c415d9f0 2023-11-08T19:26:04.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49c3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f49c4143600 0x7f49c40c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46902/0 (socket says 172.21.15.119:46902) 2023-11-08T19:26:04.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49c3fff640 1 -- 172.21.15.119:0/2928041709 learned_addr learned my addr 172.21.15.119:0/2928041709 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:04.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49c37fe640 1 --2- 172.21.15.119:0/2928041709 >> v2:172.21.15.188:3300/0 conn(0x7f49c415d9f0 0x7f49c415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115901494 0 0) 0x7f49c415e440 con 0x7f49c4143600 2023-11-08T19:26:04.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.282+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac003520 con 0x7f49c4143600 2023-11-08T19:26:04.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2010509603 0 0) 0x7f49ac003520 con 0x7f49c4143600 2023-11-08T19:26:04.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 >> v2:172.21.15.188:3300/0 conn(0x7f49c415d9f0 msgr2=0x7f49c415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 --2- 172.21.15.119:0/2928041709 >> v2:172.21.15.188:3300/0 conn(0x7f49c415d9f0 0x7f49c415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 >> v1:172.21.15.188:6789/0 conn(0x7f49c406cb40 legacy=0x7f49c406cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40c4b70 con 0x7f49c4143600 2023-11-08T19:26:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f49b4001670 con 0x7f49c4143600 2023-11-08T19:26:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 >> v1:172.21.15.119:6789/0 conn(0x7f49c4143600 legacy=0x7f49c40c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac003970 0x7f49ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac0047c0 0x7f49ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40c4f50 con 0x7f49ac005180 2023-11-08T19:26:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00c120 con 0x7f49ac0047c0 2023-11-08T19:26:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00e4f0 con 0x7f49ac003970 2023-11-08T19:26:04.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c8f10640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.283+0000 7f49c37fe640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac0047c0 0x7f49ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.284+0000 7f49c3fff640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac003970 0x7f49ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.284+0000 7f49c8f10640 1 -- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac003970 msgr2=0x7f49ac003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.284+0000 7f49c8f10640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac003970 0x7f49ac003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.284+0000 7f49c8f10640 1 -- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac0047c0 msgr2=0x7f49ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.284+0000 7f49c8f10640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac0047c0 0x7f49ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.284+0000 7f49c8f10640 1 -- 172.21.15.119:0/2928041709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac003520 con 0x7f49ac005180 2023-11-08T19:26:04.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.285+0000 7f49c8f10640 1 -- 172.21.15.119:0/2928041709 reap_dead start 2023-11-08T19:26:04.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.285+0000 7f49c8f10640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f49bc001750 tx=0x7f49ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d0c701ee31f2d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.285+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0052d0 con 0x7f49ac005180 2023-11-08T19:26:04.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.285+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc02e050 con 0x7f49ac005180 2023-11-08T19:26:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.285+0000 7f49c2ffd640 1 -- 172.21.15.119:0/2928041709 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc003830 con 0x7f49ac005180 2023-11-08T19:26:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 -- 172.21.15.119:0/2928041709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac005180 msgr2=0x7f49ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f49bc001750 tx=0x7f49ac0057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 -- 172.21.15.119:0/2928041709 shutdown_connections 2023-11-08T19:26:04.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 --2- 172.21.15.119:0/2928041709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 -- 172.21.15.119:0/2928041709 >> 172.21.15.119:0/2928041709 conn(0x7f49c40cb610 msgr2=0x7f49c40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 -- 172.21.15.119:0/2928041709 shutdown_connections 2023-11-08T19:26:04.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.286+0000 7f49ca973640 1 -- 172.21.15.119:0/2928041709 wait complete. 2023-11-08T19:26:04.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.288+0000 7f49ca973640 1 Processor -- start 2023-11-08T19:26:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.288+0000 7f49ca973640 1 -- start start 2023-11-08T19:26:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49ca973640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac005180 0x7f49c412ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49ca973640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c413f120 0x7f49c41400c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49ca973640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 0x7f49c40b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49ca973640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4159210 con 0x7f49c40b3960 2023-11-08T19:26:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49ca973640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4130e90 con 0x7f49c413f120 2023-11-08T19:26:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49ca973640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c41308d0 con 0x7f49ac005180 2023-11-08T19:26:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 0x7f49c40b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 0x7f49c40b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55886/0 (socket says 172.21.15.119:55886) 2023-11-08T19:26:04.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 -- 172.21.15.119:0/3426011422 learned_addr learned my addr 172.21.15.119:0/3426011422 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:04.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c37fe640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c413f120 0x7f49c41400c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 -- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac005180 msgr2=0x7f49c412ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac005180 0x7f49c412ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 -- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c413f120 msgr2=0x7f49c41400c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c413f120 0x7f49c41400c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.289+0000 7f49c8f10640 1 -- 172.21.15.119:0/3426011422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c4130bb0 con 0x7f49c40b3960 2023-11-08T19:26:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.290+0000 7f49c8f10640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 0x7f49c40b3df0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f49bc001870 tx=0x7f49bc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.290+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc039070 con 0x7f49c40b3960 2023-11-08T19:26:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.290+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc005450 con 0x7f49c40b3960 2023-11-08T19:26:04.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.290+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc007230 con 0x7f49c40b3960 2023-11-08T19:26:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.290+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c412df90 con 0x7f49c40b3960 2023-11-08T19:26:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.290+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c40b4330 con 0x7f49c40b3960 2023-11-08T19:26:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.291+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c412e4b0 con 0x7f49c40b3960 2023-11-08T19:26:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.292+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49bc03c020 con 0x7f49c40b3960 2023-11-08T19:26:04.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.293+0000 7f49c0ff9640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f49b006ae70 0x7f49b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.293+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f49bc0a9d50 con 0x7f49c40b3960 2023-11-08T19:26:04.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.293+0000 7f49c3fff640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f49b006ae70 0x7f49b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.294+0000 7f49c3fff640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f49b006ae70 0x7f49b006d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49b40017a0 tx=0x7f49b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.297+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49c412e4b0 con 0x7f49c40b3960 2023-11-08T19:26:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.491+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49c412e4b0 con 0x7f49c40b3960 2023-11-08T19:26:04.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.492+0000 7f49c0ff9640 1 -- 172.21.15.119:0/3426011422 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49c412e4b0 con 0x7f49c40b3960 2023-11-08T19:26:04.494 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f49b006ae70 msgr2=0x7f49b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f49b006ae70 0x7f49b006d330 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f49b40017a0 tx=0x7f49b4009040 comp rx=0 tx=0).stop 2023-11-08T19:26:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 msgr2=0x7f49c40b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 0x7f49c40b3df0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f49bc001870 tx=0x7f49bc001120 comp rx=0 tx=0).stop 2023-11-08T19:26:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 shutdown_connections 2023-11-08T19:26:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f49b006ae70 0x7f49b006d330 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49c40b3960 0x7f49c40b3df0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49c413f120 0x7f49c41400c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 --2- 172.21.15.119:0/3426011422 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac005180 0x7f49c412ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 >> 172.21.15.119:0/3426011422 conn(0x7f49c40cb610 msgr2=0x7f49c406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 shutdown_connections 2023-11-08T19:26:04.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.496+0000 7f49ca973640 1 -- 172.21.15.119:0/3426011422 wait complete. 2023-11-08T19:26:04.517 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:26:04.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.723+0000 7fecb344b640 1 Processor -- start 2023-11-08T19:26:04.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.723+0000 7fecb344b640 1 -- start start 2023-11-08T19:26:04.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.725+0000 7fecb344b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 0x7fecac0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.725+0000 7fecb344b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac0cd050 con 0x7fecac144690 2023-11-08T19:26:04.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.725+0000 7fecb344b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac06ac60 con 0x7fecac161ad0 2023-11-08T19:26:04.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.725+0000 7fecb344b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac165de0 con 0x7fecac0cc6d0 2023-11-08T19:26:04.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.725+0000 7fecb11e7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fecac144690 0x7fecac06a550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46904/0 (socket says 172.21.15.119:46904) 2023-11-08T19:26:04.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.725+0000 7fecb11e7640 1 -- 172.21.15.119:0/2239501384 learned_addr learned my addr 172.21.15.119:0/2239501384 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:04.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140624741 0 0) 0x7fecac0cd050 con 0x7fecac144690 2023-11-08T19:26:04.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec98003520 con 0x7fecac144690 2023-11-08T19:26:04.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7fecb09e6640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 0x7fecac0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 417463563 0 0) 0x7fec98003520 con 0x7fecac144690 2023-11-08T19:26:04.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 msgr2=0x7fecac0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 0x7fecac0ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 >> v1:172.21.15.188:6789/0 conn(0x7fecac161ad0 legacy=0x7fecac161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac165160 con 0x7fecac144690 2023-11-08T19:26:04.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7feca8001670 con 0x7fecac144690 2023-11-08T19:26:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 >> v1:172.21.15.119:6789/0 conn(0x7fecac144690 legacy=0x7fecac06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec980039a0 0x7fec98003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec980047c0 0x7fec98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 0x7fec980051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac165de0 con 0x7fec980047c0 2023-11-08T19:26:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9800ba90 con 0x7fec980039a0 2023-11-08T19:26:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.726+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9800bc50 con 0x7fecac0cc6d0 2023-11-08T19:26:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb09e6640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec980047c0 0x7fec98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb11e7640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec980039a0 0x7fec98003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb19e8640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 0x7fec980051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb09e6640 1 -- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 msgr2=0x7fec980051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb09e6640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecac0cc6d0 0x7fec980051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb09e6640 1 -- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec980039a0 msgr2=0x7fec98003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:04.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb09e6640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec980039a0 0x7fec98003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb09e6640 1 -- 172.21.15.119:0/2239501384 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec98003520 con 0x7fec980047c0 2023-11-08T19:26:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.727+0000 7fecb11e7640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec980039a0 0x7fec98003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.728+0000 7fecb09e6640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec980047c0 0x7fec98004c40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00ba30 tx=0x7fec980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec5bc637490b6a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.728+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c032640 con 0x7fec980047c0 2023-11-08T19:26:04.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.728+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c0327a0 con 0x7fec980047c0 2023-11-08T19:26:04.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.728+0000 7feca3fff640 1 -- 172.21.15.119:0/2239501384 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c032b30 con 0x7fec980047c0 2023-11-08T19:26:04.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb344b640 1 -- 172.21.15.119:0/2239501384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec980047c0 msgr2=0x7fec98004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb344b640 1 --2- 172.21.15.119:0/2239501384 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec980047c0 0x7fec98004c40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00ba30 tx=0x7fec980057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb19e8640 1 -- 172.21.15.119:0/2239501384 reap_dead start 2023-11-08T19:26:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb344b640 1 -- 172.21.15.119:0/2239501384 shutdown_connections 2023-11-08T19:26:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb344b640 1 -- 172.21.15.119:0/2239501384 >> 172.21.15.119:0/2239501384 conn(0x7fecac0cb610 msgr2=0x7fecac0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb344b640 1 -- 172.21.15.119:0/2239501384 shutdown_connections 2023-11-08T19:26:04.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.729+0000 7fecb344b640 1 -- 172.21.15.119:0/2239501384 wait complete. 2023-11-08T19:26:04.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.731+0000 7fecb344b640 1 Processor -- start 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.731+0000 7fecb344b640 1 -- start start 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb344b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980047c0 0x7fecac145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb344b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 0x7fecac144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb344b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecac13f8b0 0x7fecac070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb344b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac145290 con 0x7fecac158020 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb344b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac145f40 con 0x7fecac13f8b0 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb344b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac146220 con 0x7fec980047c0 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb09e6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 0x7fecac144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb09e6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 0x7fecac144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55912/0 (socket says 172.21.15.119:55912) 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.732+0000 7fecb09e6640 1 -- 172.21.15.119:0/552954737 learned_addr learned my addr 172.21.15.119:0/552954737 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb11e7640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980047c0 0x7fecac145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb19e8640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecac13f8b0 0x7fecac070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb09e6640 1 -- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980047c0 msgr2=0x7fecac145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb09e6640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980047c0 0x7fecac145960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb09e6640 1 -- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecac13f8b0 msgr2=0x7fecac070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb09e6640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecac13f8b0 0x7fecac070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb09e6640 1 -- 172.21.15.119:0/552954737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0b24c0 con 0x7fecac158020 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb09e6640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 0x7fecac144d50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00bb50 tx=0x7fec9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c03ca80 con 0x7fecac158020 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c03cbe0 con 0x7fecac158020 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c030410 con 0x7fecac158020 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac0b27a0 con 0x7fecac158020 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.733+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac13fd30 con 0x7fecac158020 2023-11-08T19:26:04.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.734+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac0b2930 con 0x7fecac158020 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.735+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec9c030570 con 0x7fecac158020 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.737+0000 7feca1ffb640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec9006ae70 0x7fec9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.737+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fec9c0ac300 con 0x7fecac158020 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.737+0000 7fecb11e7640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec9006ae70 0x7fec9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:04.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.737+0000 7fecb11e7640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec9006ae70 0x7fec9006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feca80017a0 tx=0x7feca8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:04.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.741+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec9c076fc0 con 0x7fecac158020 2023-11-08T19:26:04.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.920+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fecac1584b0 con 0x7fecac158020 2023-11-08T19:26:04.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.921+0000 7feca1ffb640 1 -- 172.21.15.119:0/552954737 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fec9c07a5e0 con 0x7fecac158020 2023-11-08T19:26:04.924 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec9006ae70 msgr2=0x7fec9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec9006ae70 0x7fec9006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7feca80017a0 tx=0x7feca8009040 comp rx=0 tx=0).stop 2023-11-08T19:26:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 msgr2=0x7fecac144d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:04.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 0x7fecac144d50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00bb50 tx=0x7fec9c004730 comp rx=0 tx=0).stop 2023-11-08T19:26:04.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 shutdown_connections 2023-11-08T19:26:04.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec9006ae70 0x7fec9006d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecac13f8b0 0x7fecac070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fecac158020 0x7fecac144d50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 --2- 172.21.15.119:0/552954737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec980047c0 0x7fecac145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:04.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 >> 172.21.15.119:0/552954737 conn(0x7fecac0cb610 msgr2=0x7fecac0c4260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:04.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 shutdown_connections 2023-11-08T19:26:04.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:04.923+0000 7fecb344b640 1 -- 172.21.15.119:0/552954737 wait complete. 2023-11-08T19:26:04.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:05.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.147+0000 7fea9c075640 1 Processor -- start 2023-11-08T19:26:05.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.147+0000 7fea9c075640 1 -- start start 2023-11-08T19:26:05.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.149+0000 7fea9c075640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 0x7fea940cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.149+0000 7fea9c075640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fea9415d9f0 0x7fea940c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.149+0000 7fea9c075640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea940c4960 con 0x7fea9406cb40 2023-11-08T19:26:05.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.149+0000 7fea9c075640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea94143a40 con 0x7fea9415d9f0 2023-11-08T19:26:05.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.149+0000 7fea9c075640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9415df60 con 0x7fea94142f80 2023-11-08T19:26:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 0x7fea940cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 0x7fea940cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:50024/0 (socket says 172.21.15.119:50024) 2023-11-08T19:26:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99610640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fea9415d9f0 0x7fea940c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 -- 172.21.15.119:0/2658686380 learned_addr learned my addr 172.21.15.119:0/2658686380 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 -- 172.21.15.119:0/2658686380 >> v2:172.21.15.188:3300/0 conn(0x7fea9415d9f0 msgr2=0x7fea940c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 --2- 172.21.15.119:0/2658686380 >> v2:172.21.15.188:3300/0 conn(0x7fea9415d9f0 0x7fea940c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 -- 172.21.15.119:0/2658686380 >> v1:172.21.15.188:6789/0 conn(0x7fea9406cb40 legacy=0x7fea940c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.150+0000 7fea99e11640 1 -- 172.21.15.119:0/2658686380 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9415e240 con 0x7fea94142f80 2023-11-08T19:26:05.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.152+0000 7fea99e11640 1 --2- 172.21.15.119:0/2658686380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 0x7fea940cc6d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fea90009870 tx=0x7fea9002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d9b0ae5a5659afa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.152+0000 7fea98e0f640 1 -- 172.21.15.119:0/2658686380 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90031640 con 0x7fea94142f80 2023-11-08T19:26:05.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.152+0000 7fea98e0f640 1 -- 172.21.15.119:0/2658686380 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea900317a0 con 0x7fea94142f80 2023-11-08T19:26:05.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.152+0000 7fea98e0f640 1 -- 172.21.15.119:0/2658686380 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea90031b30 con 0x7fea94142f80 2023-11-08T19:26:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 -- 172.21.15.119:0/2658686380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 msgr2=0x7fea940cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 --2- 172.21.15.119:0/2658686380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 0x7fea940cc6d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fea90009870 tx=0x7fea9002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 -- 172.21.15.119:0/2658686380 shutdown_connections 2023-11-08T19:26:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 --2- 172.21.15.119:0/2658686380 >> v2:172.21.15.188:3300/0 conn(0x7fea9415d9f0 0x7fea940c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 --2- 172.21.15.119:0/2658686380 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea94142f80 0x7fea940cc6d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 -- 172.21.15.119:0/2658686380 >> 172.21.15.119:0/2658686380 conn(0x7fea940cb610 msgr2=0x7fea940cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 -- 172.21.15.119:0/2658686380 shutdown_connections 2023-11-08T19:26:05.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.153+0000 7fea9c075640 1 -- 172.21.15.119:0/2658686380 wait complete. 2023-11-08T19:26:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.155+0000 7fea9c075640 1 Processor -- start 2023-11-08T19:26:05.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.155+0000 7fea9c075640 1 -- start start 2023-11-08T19:26:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea9c075640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9406cb40 0x7fea941595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea9c075640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 0x7fea940ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea9c075640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9415d9f0 0x7fea940cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea9c075640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9415ade0 con 0x7fea94142f80 2023-11-08T19:26:05.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea9c075640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9415b080 con 0x7fea9415d9f0 2023-11-08T19:26:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea9c075640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9415b360 con 0x7fea9406cb40 2023-11-08T19:26:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea99610640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 0x7fea940ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea99610640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 0x7fea940ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55922/0 (socket says 172.21.15.119:55922) 2023-11-08T19:26:05.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea99610640 1 -- 172.21.15.119:0/2714925925 learned_addr learned my addr 172.21.15.119:0/2714925925 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:05.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.156+0000 7fea99e11640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9406cb40 0x7fea941595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea9a612640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9415d9f0 0x7fea940cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea99610640 1 -- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9406cb40 msgr2=0x7fea941595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea99610640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9406cb40 0x7fea941595f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea99610640 1 -- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9415d9f0 msgr2=0x7fea940cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea99610640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9415d9f0 0x7fea940cb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea99610640 1 -- 172.21.15.119:0/2714925925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea900094d0 con 0x7fea94142f80 2023-11-08T19:26:05.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea99610640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 0x7fea940ac8c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e870 tx=0x7fea8400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea840094f0 con 0x7fea94142f80 2023-11-08T19:26:05.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8400c940 con 0x7fea94142f80 2023-11-08T19:26:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8400ccf0 con 0x7fea94142f80 2023-11-08T19:26:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea9415b640 con 0x7fea94142f80 2023-11-08T19:26:05.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.157+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea940b4270 con 0x7fea94142f80 2023-11-08T19:26:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.159+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea94074430 con 0x7fea94142f80 2023-11-08T19:26:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.159+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea84013470 con 0x7fea94142f80 2023-11-08T19:26:05.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.160+0000 7fea8a7fc640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea7806aec0 0x7fea7806d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.160+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fea84089a80 con 0x7fea94142f80 2023-11-08T19:26:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.160+0000 7fea99e11640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea7806aec0 0x7fea7806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.161+0000 7fea99e11640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea7806aec0 0x7fea7806d380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea90036000 tx=0x7fea9003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.165+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea84054840 con 0x7fea94142f80 2023-11-08T19:26:05.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.366+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fea940b3f00 con 0x7fea94142f80 2023-11-08T19:26:05.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.367+0000 7fea8a7fc640 1 -- 172.21.15.119:0/2714925925 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fea84057e60 con 0x7fea94142f80 2023-11-08T19:26:05.369 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:26:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea7806aec0 msgr2=0x7fea7806d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea7806aec0 0x7fea7806d380 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea90036000 tx=0x7fea9003c040 comp rx=0 tx=0).stop 2023-11-08T19:26:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 msgr2=0x7fea940ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 0x7fea940ac8c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e870 tx=0x7fea8400ed30 comp rx=0 tx=0).stop 2023-11-08T19:26:05.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 shutdown_connections 2023-11-08T19:26:05.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea7806aec0 0x7fea7806d380 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fea9415d9f0 0x7fea940cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fea94142f80 0x7fea940ac8c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 --2- 172.21.15.119:0/2714925925 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fea9406cb40 0x7fea941595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 >> 172.21.15.119:0/2714925925 conn(0x7fea940cb610 msgr2=0x7fea94070bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 shutdown_connections 2023-11-08T19:26:05.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.370+0000 7fea9c075640 1 -- 172.21.15.119:0/2714925925 wait complete. 2023-11-08T19:26:05.389 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-08T19:26:05.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:05.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-08T19:26:05.492 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:26:05.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:05.493 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T19:26:05.551 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:05.551 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:05.611 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:05.612 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:05.665 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:05.666 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:05.719 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:05.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.933+0000 7f681ad90640 1 Processor -- start 2023-11-08T19:26:05.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.933+0000 7f681ad90640 1 -- start start 2023-11-08T19:26:05.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.934+0000 7f681ad90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f681415f4c0 0x7f681406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.934+0000 7f681ad90640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140c3a50 con 0x7f681406a120 2023-11-08T19:26:05.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.934+0000 7f681ad90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6814144b80 con 0x7f68141447a0 2023-11-08T19:26:05.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.934+0000 7f681ad90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f681415f930 con 0x7f681415f4c0 2023-11-08T19:26:05.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.935+0000 7f681932d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f681406a120 0x7f681406a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46914/0 (socket says 172.21.15.119:46914) 2023-11-08T19:26:05.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.935+0000 7f681932d640 1 -- 172.21.15.119:0/3116376005 learned_addr learned my addr 172.21.15.119:0/3116376005 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:05.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.935+0000 7f680bfff640 1 --2- 172.21.15.119:0/3116376005 >> v2:172.21.15.188:3300/0 conn(0x7f681415f4c0 0x7f681406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.935+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864414143 0 0) 0x7f68140c3a50 con 0x7f681406a120 2023-11-08T19:26:05.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.935+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67fc003520 con 0x7f681406a120 2023-11-08T19:26:05.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2649057315 0 0) 0x7f67fc003520 con 0x7f681406a120 2023-11-08T19:26:05.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 >> v2:172.21.15.188:3300/0 conn(0x7f681415f4c0 msgr2=0x7f681406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 --2- 172.21.15.119:0/3116376005 >> v2:172.21.15.188:3300/0 conn(0x7f681415f4c0 0x7f681406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 >> v1:172.21.15.188:6789/0 conn(0x7f68141447a0 legacy=0x7f68140cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140c9150 con 0x7f681406a120 2023-11-08T19:26:05.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f680c001a10 con 0x7f681406a120 2023-11-08T19:26:05.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 >> v1:172.21.15.119:6789/0 conn(0x7f681406a120 legacy=0x7f681406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67fc0039c0 0x7f67fc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67fc0047c0 0x7f67fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc005180 0x7f67fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681415f930 con 0x7f67fc0039c0 2023-11-08T19:26:05.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc00c080 con 0x7f67fc005180 2023-11-08T19:26:05.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc00c240 con 0x7f67fc0047c0 2023-11-08T19:26:05.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f6818b2c640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67fc0039c0 0x7f67fc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f681932d640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc005180 0x7f67fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f6818b2c640 1 -- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67fc0047c0 msgr2=0x7f67fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f6818b2c640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67fc0047c0 0x7f67fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f6818b2c640 1 -- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc005180 msgr2=0x7f67fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:05.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f6818b2c640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc005180 0x7f67fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f6818b2c640 1 -- 172.21.15.119:0/3116376005 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc003520 con 0x7f67fc0039c0 2023-11-08T19:26:05.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.936+0000 7f681932d640 1 -- 172.21.15.119:0/3116376005 reap_dead start 2023-11-08T19:26:05.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f6818b2c640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67fc0039c0 0x7f67fc003de0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6810001330 tx=0x7f67fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd339c369a18526 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68100052d0 con 0x7f67fc0039c0 2023-11-08T19:26:05.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6810005430 con 0x7f67fc0039c0 2023-11-08T19:26:05.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f680b7fe640 1 -- 172.21.15.119:0/3116376005 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6810003830 con 0x7f67fc0039c0 2023-11-08T19:26:05.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f681ad90640 1 -- 172.21.15.119:0/3116376005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67fc0039c0 msgr2=0x7f67fc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f681ad90640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67fc0039c0 0x7f67fc003de0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6810001330 tx=0x7f67fc0057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f681ad90640 1 -- 172.21.15.119:0/3116376005 shutdown_connections 2023-11-08T19:26:05.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f681ad90640 1 --2- 172.21.15.119:0/3116376005 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f67fc0039c0 0x7f67fc003de0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.938+0000 7f681ad90640 1 -- 172.21.15.119:0/3116376005 >> 172.21.15.119:0/3116376005 conn(0x7f68140cb610 msgr2=0x7f68140cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:05.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.939+0000 7f681ad90640 1 -- 172.21.15.119:0/3116376005 shutdown_connections 2023-11-08T19:26:05.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.939+0000 7f681ad90640 1 -- 172.21.15.119:0/3116376005 wait complete. 2023-11-08T19:26:05.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.940+0000 7f681ad90640 1 Processor -- start 2023-11-08T19:26:05.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.941+0000 7f681ad90640 1 -- start start 2023-11-08T19:26:05.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.941+0000 7f681ad90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc0039c0 0x7f681413c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.941+0000 7f681ad90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 0x7f681415a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.941+0000 7f681ad90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68140b3a00 0x7f68140b3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f681ad90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140b47c0 con 0x7f681415a360 2023-11-08T19:26:05.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f681ad90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814139b20 con 0x7f67fc0039c0 2023-11-08T19:26:05.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f681ad90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f681415ad20 con 0x7f68140b3a00 2023-11-08T19:26:05.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 0x7f681415a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 0x7f681415a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55936/0 (socket says 172.21.15.119:55936) 2023-11-08T19:26:05.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 -- 172.21.15.119:0/106640230 learned_addr learned my addr 172.21.15.119:0/106640230 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:05.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f6818b2c640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc0039c0 0x7f681413c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 -- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68140b3a00 msgr2=0x7f68140b3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68140b3a00 0x7f68140b3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 -- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc0039c0 msgr2=0x7f681413c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:05.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc0039c0 0x7f681413c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:05.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 -- 172.21.15.119:0/106640230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68141391f0 con 0x7f681415a360 2023-11-08T19:26:05.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.942+0000 7f680bfff640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 0x7f681415a7e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f680400e7a0 tx=0x7f680400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.943+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400ee50 con 0x7f681415a360 2023-11-08T19:26:05.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.943+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680400c940 con 0x7f681415a360 2023-11-08T19:26:05.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.943+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68141394d0 con 0x7f681415a360 2023-11-08T19:26:05.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.943+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680400ccf0 con 0x7f681415a360 2023-11-08T19:26:05.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.943+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681413cdf0 con 0x7f681415a360 2023-11-08T19:26:05.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.944+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6814139c90 con 0x7f681415a360 2023-11-08T19:26:05.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.945+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6804013470 con 0x7f681415a360 2023-11-08T19:26:05.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.946+0000 7f68097fa640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f680006aea0 0x7f680006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:05.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.946+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6804089850 con 0x7f681415a360 2023-11-08T19:26:05.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.946+0000 7f6818b2c640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f680006aea0 0x7f680006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:05.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.947+0000 7f6818b2c640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f680006aea0 0x7f680006d360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6810038000 tx=0x7f6810009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:05.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:05.950+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f68040545b0 con 0x7f681415a360 2023-11-08T19:26:06.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.143+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6814074430 con 0x7f681415a360 2023-11-08T19:26:06.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.145+0000 7f68097fa640 1 -- 172.21.15.119:0/106640230 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f6804057bd0 con 0x7f681415a360 2023-11-08T19:26:06.147 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:06.147 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:06.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f680006aea0 msgr2=0x7f680006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f680006aea0 0x7f680006d360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6810038000 tx=0x7f6810009030 comp rx=0 tx=0).stop 2023-11-08T19:26:06.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 msgr2=0x7f681415a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 0x7f681415a7e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f680400e7a0 tx=0x7f680400ec60 comp rx=0 tx=0).stop 2023-11-08T19:26:06.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 shutdown_connections 2023-11-08T19:26:06.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f680006aea0 0x7f680006d360 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f68140b3a00 0x7f68140b3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f681415a360 0x7f681415a7e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 --2- 172.21.15.119:0/106640230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67fc0039c0 0x7f681413c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 >> 172.21.15.119:0/106640230 conn(0x7f68140cb610 msgr2=0x7f68140ce0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 shutdown_connections 2023-11-08T19:26:06.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.148+0000 7f681ad90640 1 -- 172.21.15.119:0/106640230 wait complete. 2023-11-08T19:26:06.169 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:26:06.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:06.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.377+0000 7f152a728640 1 Processor -- start 2023-11-08T19:26:06.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.377+0000 7f152a728640 1 -- start start 2023-11-08T19:26:06.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f152a728640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f1524071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f152a728640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f152415b690 0x7f15240ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f152a728640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f15240cf5e0 0x7f15240ca990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f152a728640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15240c9440 con 0x7f152415b690 2023-11-08T19:26:06.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f152a728640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15240c9690 con 0x7f15240cf5e0 2023-11-08T19:26:06.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f152a728640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15240c9970 con 0x7f152415b2b0 2023-11-08T19:26:06.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f1528f25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f152415b690 0x7f15240ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f1528f25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f152415b690 0x7f15240ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55938/0 (socket says 172.21.15.119:55938) 2023-11-08T19:26:06.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.379+0000 7f1528f25640 1 -- 172.21.15.119:0/2951611988 learned_addr learned my addr 172.21.15.119:0/2951611988 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:06.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1529f27640 1 --2- 172.21.15.119:0/2951611988 >> v2:172.21.15.188:3300/0 conn(0x7f15240cf5e0 0x7f15240ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1529726640 1 --2- 172.21.15.119:0/2951611988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f1524071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1528f25640 1 -- 172.21.15.119:0/2951611988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 msgr2=0x7f1524071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1528f25640 1 --2- 172.21.15.119:0/2951611988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f1524071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1528f25640 1 -- 172.21.15.119:0/2951611988 >> v2:172.21.15.188:3300/0 conn(0x7f15240cf5e0 msgr2=0x7f15240ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1528f25640 1 --2- 172.21.15.119:0/2951611988 >> v2:172.21.15.188:3300/0 conn(0x7f15240cf5e0 0x7f15240ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1528f25640 1 -- 172.21.15.119:0/2951611988 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15240c9c50 con 0x7f152415b690 2023-11-08T19:26:06.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.380+0000 7f1529f27640 1 --2- 172.21.15.119:0/2951611988 >> v2:172.21.15.188:3300/0 conn(0x7f15240cf5e0 0x7f15240ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:06.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.381+0000 7f1528f25640 1 --2- 172.21.15.119:0/2951611988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f152415b690 0x7f15240ca450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f15240b1630 tx=0x7f151402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e4f59c33e0791 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.381+0000 7f151bfff640 1 -- 172.21.15.119:0/2951611988 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140317e0 con 0x7f152415b690 2023-11-08T19:26:06.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.381+0000 7f151bfff640 1 -- 172.21.15.119:0/2951611988 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514031940 con 0x7f152415b690 2023-11-08T19:26:06.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.381+0000 7f151bfff640 1 -- 172.21.15.119:0/2951611988 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514031cd0 con 0x7f152415b690 2023-11-08T19:26:06.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 -- 172.21.15.119:0/2951611988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f152415b690 msgr2=0x7f15240ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 --2- 172.21.15.119:0/2951611988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f152415b690 0x7f15240ca450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f15240b1630 tx=0x7f151402b880 comp rx=0 tx=0).stop 2023-11-08T19:26:06.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 -- 172.21.15.119:0/2951611988 shutdown_connections 2023-11-08T19:26:06.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 --2- 172.21.15.119:0/2951611988 >> v2:172.21.15.188:3300/0 conn(0x7f15240cf5e0 0x7f15240ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 --2- 172.21.15.119:0/2951611988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f152415b690 0x7f15240ca450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 --2- 172.21.15.119:0/2951611988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f1524071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 -- 172.21.15.119:0/2951611988 >> 172.21.15.119:0/2951611988 conn(0x7f15240ceac0 msgr2=0x7f15240ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 -- 172.21.15.119:0/2951611988 shutdown_connections 2023-11-08T19:26:06.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.382+0000 7f152a728640 1 -- 172.21.15.119:0/2951611988 wait complete. 2023-11-08T19:26:06.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.384+0000 7f152a728640 1 Processor -- start 2023-11-08T19:26:06.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.384+0000 7f152a728640 1 -- start start 2023-11-08T19:26:06.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f152a728640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 0x7f1524149620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f152a728640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f15240c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f152a728640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f152415b690 0x7f152414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f152a728640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152415e890 con 0x7f15240cf5e0 2023-11-08T19:26:06.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f152a728640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152414fe40 con 0x7f152415b690 2023-11-08T19:26:06.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f152a728640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1524150120 con 0x7f152415b2b0 2023-11-08T19:26:06.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 0x7f1524149620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 0x7f1524149620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55942/0 (socket says 172.21.15.119:55942) 2023-11-08T19:26:06.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 -- 172.21.15.119:0/258124703 learned_addr learned my addr 172.21.15.119:0/258124703 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:06.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 -- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 msgr2=0x7f15240c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f15240c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529f27640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f152415b690 0x7f152414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 -- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f152415b690 msgr2=0x7f152414bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f152415b690 0x7f152414bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.385+0000 7f1529726640 1 -- 172.21.15.119:0/258124703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15140094d0 con 0x7f15240cf5e0 2023-11-08T19:26:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.386+0000 7f1529726640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 0x7f1524149620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f152000c7f0 tx=0x7f152000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.386+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520004220 con 0x7f15240cf5e0 2023-11-08T19:26:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.386+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15200043a0 con 0x7f15240cf5e0 2023-11-08T19:26:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.386+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1520007860 con 0x7f15240cf5e0 2023-11-08T19:26:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.386+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15240c4eb0 con 0x7f15240cf5e0 2023-11-08T19:26:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.386+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1524149b60 con 0x7f15240cf5e0 2023-11-08T19:26:06.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.387+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15240c53d0 con 0x7f15240cf5e0 2023-11-08T19:26:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.388+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1520007bd0 con 0x7f15240cf5e0 2023-11-08T19:26:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.389+0000 7f1519ffb640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f150806ae70 0x7f150806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.389+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f15200898e0 con 0x7f15240cf5e0 2023-11-08T19:26:06.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.389+0000 7f1528f25640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f150806ae70 0x7f150806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.390+0000 7f1528f25640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f150806ae70 0x7f150806d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1514036000 tx=0x7f1514030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.394+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1520054640 con 0x7f15240cf5e0 2023-11-08T19:26:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.585+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f152404ece0 con 0x7f15240cf5e0 2023-11-08T19:26:06.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.586+0000 7f1519ffb640 1 -- 172.21.15.119:0/258124703 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f152001e0a0 con 0x7f15240cf5e0 2023-11-08T19:26:06.588 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:06.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f150806ae70 msgr2=0x7f150806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f150806ae70 0x7f150806d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1514036000 tx=0x7f1514030000 comp rx=0 tx=0).stop 2023-11-08T19:26:06.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 msgr2=0x7f1524149620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 0x7f1524149620 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f152000c7f0 tx=0x7f152000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 shutdown_connections 2023-11-08T19:26:06.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f150806ae70 0x7f150806d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f152415b690 0x7f152414bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f152415b2b0 0x7f15240c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 --2- 172.21.15.119:0/258124703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15240cf5e0 0x7f1524149620 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 >> 172.21.15.119:0/258124703 conn(0x7f15240ceac0 msgr2=0x7f15240ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 shutdown_connections 2023-11-08T19:26:06.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.589+0000 7f152a728640 1 -- 172.21.15.119:0/258124703 wait complete. 2023-11-08T19:26:06.611 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:06.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.824+0000 7f5818234640 1 Processor -- start 2023-11-08T19:26:06.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.824+0000 7f5818234640 1 -- start start 2023-11-08T19:26:06.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.825+0000 7f5818234640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f581006d250 0x7f581015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.825+0000 7f5818234640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581015e440 con 0x7f5810143600 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.825+0000 7f5818234640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581015e720 con 0x7f581015d9f0 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.825+0000 7f5818234640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58100c4f50 con 0x7f581006d250 2023-11-08T19:26:06.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5815fd0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5810143600 0x7f58100c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46930/0 (socket says 172.21.15.119:46930) 2023-11-08T19:26:06.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5815fd0640 1 -- 172.21.15.119:0/462906690 learned_addr learned my addr 172.21.15.119:0/462906690 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:06.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f58167d1640 1 --2- 172.21.15.119:0/462906690 >> v2:172.21.15.188:3300/0 conn(0x7f581006d250 0x7f581015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376610726 0 0) 0x7f581015e440 con 0x7f5810143600 2023-11-08T19:26:06.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57fc003520 con 0x7f5810143600 2023-11-08T19:26:06.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463732956 0 0) 0x7f57fc003520 con 0x7f5810143600 2023-11-08T19:26:06.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 >> v2:172.21.15.188:3300/0 conn(0x7f581006d250 msgr2=0x7f581015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 --2- 172.21.15.119:0/462906690 >> v2:172.21.15.188:3300/0 conn(0x7f581006d250 0x7f581015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 >> v1:172.21.15.188:6789/0 conn(0x7f581015d9f0 legacy=0x7f581006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100c4b70 con 0x7f5810143600 2023-11-08T19:26:06.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f580c001670 con 0x7f5810143600 2023-11-08T19:26:06.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 >> v1:172.21.15.119:6789/0 conn(0x7f5810143600 legacy=0x7f58100c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc0039c0 0x7f57fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.826+0000 7f5814fce640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57fc0047c0 0x7f57fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f5814fce640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f57fc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100c4f50 con 0x7f57fc005250 2023-11-08T19:26:06.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc00e3b0 con 0x7f57fc0039c0 2023-11-08T19:26:06.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc00e570 con 0x7f57fc0047c0 2023-11-08T19:26:06.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58167d1640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f57fc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f5815fd0640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc0039c0 0x7f57fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58157cf640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57fc0047c0 0x7f57fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58167d1640 1 -- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57fc0047c0 msgr2=0x7f57fc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58167d1640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57fc0047c0 0x7f57fc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58167d1640 1 -- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc0039c0 msgr2=0x7f57fc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:06.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58167d1640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57fc0039c0 0x7f57fc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.827+0000 7f58167d1640 1 -- 172.21.15.119:0/462906690 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc003520 con 0x7f57fc005250 2023-11-08T19:26:06.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.828+0000 7f58167d1640 1 -- 172.21.15.119:0/462906690 reap_dead start 2023-11-08T19:26:06.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.828+0000 7f58167d1640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f57fc00db90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5808007330 tx=0x7f57fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f08d7b49ef06b38 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580802ccd0 con 0x7f57fc005250 2023-11-08T19:26:06.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580802ce30 con 0x7f57fc005250 2023-11-08T19:26:06.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5814fce640 1 -- 172.21.15.119:0/462906690 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5808003bd0 con 0x7f57fc005250 2023-11-08T19:26:06.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 -- 172.21.15.119:0/462906690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 msgr2=0x7f57fc00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f57fc00db90 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f5808007330 tx=0x7f57fc0057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 -- 172.21.15.119:0/462906690 shutdown_connections 2023-11-08T19:26:06.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 --2- 172.21.15.119:0/462906690 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f57fc00db90 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 -- 172.21.15.119:0/462906690 >> 172.21.15.119:0/462906690 conn(0x7f58100cb610 msgr2=0x7f58100cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:06.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 -- 172.21.15.119:0/462906690 shutdown_connections 2023-11-08T19:26:06.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.829+0000 7f5818234640 1 -- 172.21.15.119:0/462906690 wait complete. 2023-11-08T19:26:06.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.831+0000 7f5818234640 1 Processor -- start 2023-11-08T19:26:06.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.831+0000 7f5818234640 1 -- start start 2023-11-08T19:26:06.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5818234640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f58101f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5818234640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100acd90 0x7f58100b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5818234640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100c85c0 0x7f58100aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5818234640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100ad220 con 0x7f57fc005250 2023-11-08T19:26:06.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5818234640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58100ad500 con 0x7f58100c85c0 2023-11-08T19:26:06.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5818234640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58100ad7e0 con 0x7f58100acd90 2023-11-08T19:26:06.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5815fd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f58101f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5815fd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f58101f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55948/0 (socket says 172.21.15.119:55948) 2023-11-08T19:26:06.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.832+0000 7f5815fd0640 1 -- 172.21.15.119:0/2383823751 learned_addr learned my addr 172.21.15.119:0/2383823751 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:06.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5815fd0640 1 -- 172.21.15.119:0/2383823751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100acd90 msgr2=0x7f58100b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5815fd0640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100acd90 0x7f58100b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5815fd0640 1 -- 172.21.15.119:0/2383823751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100c85c0 msgr2=0x7f58100aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:06.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5815fd0640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100c85c0 0x7f58100aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:06.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5815fd0640 1 -- 172.21.15.119:0/2383823751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100adac0 con 0x7f57fc005250 2023-11-08T19:26:06.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5815fd0640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f58101f2970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f580c00e730 tx=0x7f580c00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00c6c0 con 0x7f57fc005250 2023-11-08T19:26:06.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580c00c820 con 0x7f57fc005250 2023-11-08T19:26:06.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580c00cbb0 con 0x7f57fc005250 2023-11-08T19:26:06.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.833+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58100af060 con 0x7f57fc005250 2023-11-08T19:26:06.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.834+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58101f5f60 con 0x7f57fc005250 2023-11-08T19:26:06.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.834+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58100c89e0 con 0x7f57fc005250 2023-11-08T19:26:06.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.836+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f580c00cd10 con 0x7f57fc005250 2023-11-08T19:26:06.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.837+0000 7f58067fc640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57f406ae70 0x7f57f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:06.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.837+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f580c0898a0 con 0x7f57fc005250 2023-11-08T19:26:06.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.837+0000 7f58157cf640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57f406ae70 0x7f57f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:06.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.837+0000 7f58157cf640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57f406ae70 0x7f57f406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f581015e030 tx=0x7f5800009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:06.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:06.841+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f580c0544d0 con 0x7f57fc005250 2023-11-08T19:26:07.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.032+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58100c4230 con 0x7f57fc005250 2023-11-08T19:26:07.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.033+0000 7f58067fc640 1 -- 172.21.15.119:0/2383823751 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f580c057af0 con 0x7f57fc005250 2023-11-08T19:26:07.034 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:07.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57f406ae70 msgr2=0x7f57f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57f406ae70 0x7f57f406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f581015e030 tx=0x7f5800009040 comp rx=0 tx=0).stop 2023-11-08T19:26:07.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 msgr2=0x7f58101f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f58101f2970 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f580c00e730 tx=0x7f580c00ebf0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 shutdown_connections 2023-11-08T19:26:07.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f57f406ae70 0x7f57f406d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58100c85c0 0x7f58100aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.035+0000 7f5818234640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58100acd90 0x7f58100b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.036+0000 7f5818234640 1 --2- 172.21.15.119:0/2383823751 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57fc005250 0x7f58101f2970 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.036+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 >> 172.21.15.119:0/2383823751 conn(0x7f58100cb610 msgr2=0x7f581006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.036+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 shutdown_connections 2023-11-08T19:26:07.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.036+0000 7f5818234640 1 -- 172.21.15.119:0/2383823751 wait complete. 2023-11-08T19:26:07.056 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:07.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.272+0000 7f80f66b4640 1 Processor -- start 2023-11-08T19:26:07.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.272+0000 7f80f66b4640 1 -- start start 2023-11-08T19:26:07.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.273+0000 7f80f66b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f66b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80f00ce5b0 0x7f80f00b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f66b4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f00d0830 con 0x7f80f0160530 2023-11-08T19:26:07.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f66b4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f00b27c0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f66b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00b2aa0 con 0x7f80f0160910 2023-11-08T19:26:07.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80f00ce5b0 0x7f80f00b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80f00ce5b0 0x7f80f00b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55964/0 (socket says 172.21.15.119:55964) 2023-11-08T19:26:07.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 -- 172.21.15.119:0/3965268303 learned_addr learned my addr 172.21.15.119:0/3965268303 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:07.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f4eb1640 1 --2- 172.21.15.119:0/3965268303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 -- 172.21.15.119:0/3965268303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 msgr2=0x7f80f00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 --2- 172.21.15.119:0/3965268303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 -- 172.21.15.119:0/3965268303 >> v1:172.21.15.188:6789/0 conn(0x7f80f0160530 legacy=0x7f80f00cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.274+0000 7f80f5eb3640 1 -- 172.21.15.119:0/3965268303 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f00d1640 con 0x7f80f00ce5b0 2023-11-08T19:26:07.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.275+0000 7f80f5eb3640 1 --2- 172.21.15.119:0/3965268303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00b2f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009e50 tx=0x7f80e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5b588b26ba6c52 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.276+0000 7f80e7fff640 1 -- 172.21.15.119:0/3965268303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e802ca70 con 0x7f80f00ce5b0 2023-11-08T19:26:07.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.276+0000 7f80e7fff640 1 -- 172.21.15.119:0/3965268303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e802cbd0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.276+0000 7f80e7fff640 1 -- 172.21.15.119:0/3965268303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e80343f0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.276+0000 7f80f66b4640 1 -- 172.21.15.119:0/3965268303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 msgr2=0x7f80f00b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.276+0000 7f80f66b4640 1 --2- 172.21.15.119:0/3965268303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00b2f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009e50 tx=0x7f80e802bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.277+0000 7f80f66b4640 1 -- 172.21.15.119:0/3965268303 shutdown_connections 2023-11-08T19:26:07.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.277+0000 7f80f66b4640 1 --2- 172.21.15.119:0/3965268303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00b2f30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.277+0000 7f80f66b4640 1 --2- 172.21.15.119:0/3965268303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.277+0000 7f80f66b4640 1 -- 172.21.15.119:0/3965268303 >> 172.21.15.119:0/3965268303 conn(0x7f80f00d2ec0 msgr2=0x7f80f00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.277+0000 7f80f66b4640 1 -- 172.21.15.119:0/3965268303 shutdown_connections 2023-11-08T19:26:07.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.277+0000 7f80f66b4640 1 -- 172.21.15.119:0/3965268303 wait complete. 2023-11-08T19:26:07.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.278+0000 7f80f66b4640 1 Processor -- start 2023-11-08T19:26:07.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.278+0000 7f80f66b4640 1 -- start start 2023-11-08T19:26:07.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.279+0000 7f80f66b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.279+0000 7f80f66b4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f0160530 0x7f80f00b6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.279+0000 7f80f66b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.279+0000 7f80f66b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00ccf70 con 0x7f80f00ce5b0 2023-11-08T19:26:07.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.279+0000 7f80f66b4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00c94e0 con 0x7f80f0160530 2023-11-08T19:26:07.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.279+0000 7f80f66b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f00b6860 con 0x7f80f0160910 2023-11-08T19:26:07.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00c8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55976/0 (socket says 172.21.15.119:55976) 2023-11-08T19:26:07.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 -- 172.21.15.119:0/2886191839 learned_addr learned my addr 172.21.15.119:0/2886191839 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:07.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 -- 172.21.15.119:0/2886191839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 msgr2=0x7f80f00cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 -- 172.21.15.119:0/2886191839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f0160530 msgr2=0x7f80f00b6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f0160530 0x7f80f00b6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 -- 172.21.15.119:0/2886191839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e8009b30 con 0x7f80f00ce5b0 2023-11-08T19:26:07.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.280+0000 7f80f56b2640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00c8fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f80ec004640 tx=0x7f80ec00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.281+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec011790 con 0x7f80f00ce5b0 2023-11-08T19:26:07.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.281+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec0118f0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.281+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec011ce0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.281+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f00cc480 con 0x7f80f00ce5b0 2023-11-08T19:26:07.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.281+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f00cc9b0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.282+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f004ece0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.283+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80ec012420 con 0x7f80f00ce5b0 2023-11-08T19:26:07.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.284+0000 7f80e5ffb640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.284+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f80ec089500 con 0x7f80f00ce5b0 2023-11-08T19:26:07.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.284+0000 7f80f4eb1640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.285+0000 7f80f4eb1640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80f00c9870 tx=0x7f80e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.288+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80ec054260 con 0x7f80f00ce5b0 2023-11-08T19:26:07.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.482+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80f00b33b0 con 0x7f80f00ce5b0 2023-11-08T19:26:07.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.483+0000 7f80e5ffb640 1 -- 172.21.15.119:0/2886191839 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f80ec057880 con 0x7f80f00ce5b0 2023-11-08T19:26:07.484 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:07.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.485+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 msgr2=0x7f80d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.485+0000 7f80f66b4640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80f00c9870 tx=0x7f80e0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.485+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 msgr2=0x7f80f00c8fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.485+0000 7f80f66b4640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00c8fa0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f80ec004640 tx=0x7f80ec00c8e0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 shutdown_connections 2023-11-08T19:26:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f0160910 0x7f80f00cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f0160530 0x7f80f00b6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 --2- 172.21.15.119:0/2886191839 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f00ce5b0 0x7f80f00c8fa0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 >> 172.21.15.119:0/2886191839 conn(0x7f80f00d2ec0 msgr2=0x7f80f00cd960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 shutdown_connections 2023-11-08T19:26:07.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.486+0000 7f80f66b4640 1 -- 172.21.15.119:0/2886191839 wait complete. 2023-11-08T19:26:07.506 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:26:07.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.710+0000 7f1067aa9640 1 Processor -- start 2023-11-08T19:26:07.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.710+0000 7f1067aa9640 1 -- start start 2023-11-08T19:26:07.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1067aa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f106015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1067aa9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f106015f1d0 0x7f10600c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1067aa9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f106015ea60 con 0x7f1060144960 2023-11-08T19:26:07.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1067aa9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10600c9150 con 0x7f106015f1d0 2023-11-08T19:26:07.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1067aa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10600c9360 con 0x7f1060144580 2023-11-08T19:26:07.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1065044640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1060144960 0x7f10600cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46934/0 (socket says 172.21.15.119:46934) 2023-11-08T19:26:07.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1065044640 1 -- 172.21.15.119:0/4120729306 learned_addr learned my addr 172.21.15.119:0/4120729306 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:07.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1066046640 1 --2- 172.21.15.119:0/4120729306 >> v2:172.21.15.119:3300/0 conn(0x7f106015f1d0 0x7f10600c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.712+0000 7f1065845640 1 --2- 172.21.15.119:0/4120729306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f106015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.713+0000 7f1064843640 1 -- 172.21.15.119:0/4120729306 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428815806 0 0) 0x7f106015ea60 con 0x7f1060144960 2023-11-08T19:26:07.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.713+0000 7f1064843640 1 -- 172.21.15.119:0/4120729306 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c003520 con 0x7f1060144960 2023-11-08T19:26:07.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.713+0000 7f1066046640 1 -- 172.21.15.119:0/4120729306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 msgr2=0x7f106015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:07.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.713+0000 7f1066046640 1 --2- 172.21.15.119:0/4120729306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f106015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.713+0000 7f1066046640 1 -- 172.21.15.119:0/4120729306 >> v1:172.21.15.119:6789/0 conn(0x7f1060144960 legacy=0x7f10600cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.713+0000 7f1066046640 1 -- 172.21.15.119:0/4120729306 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600c9640 con 0x7f106015f1d0 2023-11-08T19:26:07.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.714+0000 7f1066046640 1 --2- 172.21.15.119:0/4120729306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f106015f1d0 0x7f10600c3a50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f10600c9290 tx=0x7f105802be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d070f8560fe294b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.714+0000 7f1064843640 1 -- 172.21.15.119:0/4120729306 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105802cbb0 con 0x7f106015f1d0 2023-11-08T19:26:07.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.714+0000 7f1064843640 1 -- 172.21.15.119:0/4120729306 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f105802cd10 con 0x7f106015f1d0 2023-11-08T19:26:07.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.714+0000 7f1064843640 1 -- 172.21.15.119:0/4120729306 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1058034660 con 0x7f106015f1d0 2023-11-08T19:26:07.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 -- 172.21.15.119:0/4120729306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f106015f1d0 msgr2=0x7f10600c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4120729306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f106015f1d0 0x7f10600c3a50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f10600c9290 tx=0x7f105802be00 comp rx=0 tx=0).stop 2023-11-08T19:26:07.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 -- 172.21.15.119:0/4120729306 shutdown_connections 2023-11-08T19:26:07.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4120729306 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f106015f1d0 0x7f10600c3a50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4120729306 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f106015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 -- 172.21.15.119:0/4120729306 >> 172.21.15.119:0/4120729306 conn(0x7f10600cb610 msgr2=0x7f10600cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 -- 172.21.15.119:0/4120729306 shutdown_connections 2023-11-08T19:26:07.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.715+0000 7f1067aa9640 1 -- 172.21.15.119:0/4120729306 wait complete. 2023-11-08T19:26:07.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.717+0000 7f1067aa9640 1 Processor -- start 2023-11-08T19:26:07.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.717+0000 7f1067aa9640 1 -- start start 2023-11-08T19:26:07.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1067aa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f10600cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1067aa9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 0x7f10600b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1067aa9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f106015f1d0 0x7f10600addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1067aa9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10601f2c70 con 0x7f1060144960 2023-11-08T19:26:07.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1067aa9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10601f2f50 con 0x7f106015f1d0 2023-11-08T19:26:07.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1067aa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10601f3230 con 0x7f1060144580 2023-11-08T19:26:07.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1065044640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 0x7f10600b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1065044640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 0x7f10600b3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55994/0 (socket says 172.21.15.119:55994) 2023-11-08T19:26:07.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.718+0000 7f1065044640 1 -- 172.21.15.119:0/4018761821 learned_addr learned my addr 172.21.15.119:0/4018761821 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:07.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1066046640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f106015f1d0 0x7f10600addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065845640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f10600cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065044640 1 -- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 msgr2=0x7f10600cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065044640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f10600cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065044640 1 -- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f106015f1d0 msgr2=0x7f10600addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065044640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f106015f1d0 0x7f10600addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065044640 1 -- 172.21.15.119:0/4018761821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058009a70 con 0x7f1060144960 2023-11-08T19:26:07.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1065044640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 0x7f10600b3f40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f105000c430 tx=0x7f105000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050005ce0 con 0x7f1060144960 2023-11-08T19:26:07.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1050005e40 con 0x7f1060144960 2023-11-08T19:26:07.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10601f3510 con 0x7f1060144960 2023-11-08T19:26:07.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10500072f0 con 0x7f1060144960 2023-11-08T19:26:07.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.719+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10601f2900 con 0x7f1060144960 2023-11-08T19:26:07.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.720+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10600ac8c0 con 0x7f1060144960 2023-11-08T19:26:07.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.722+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1050007450 con 0x7f1060144960 2023-11-08T19:26:07.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.723+0000 7f10567fc640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f104406ae70 0x7f104406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:07.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.724+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1050089070 con 0x7f1060144960 2023-11-08T19:26:07.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.724+0000 7f1065845640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f104406ae70 0x7f104406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:07.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.724+0000 7f1065845640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f104406ae70 0x7f104406d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10600c8bd0 tx=0x7f105c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:07.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.730+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10600ac8c0 con 0x7f1060144960 2023-11-08T19:26:07.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.920+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f10600ac8c0 con 0x7f1060144960 2023-11-08T19:26:07.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.921+0000 7f10567fc640 1 -- 172.21.15.119:0/4018761821 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f10600ac8c0 con 0x7f1060144960 2023-11-08T19:26:07.922 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:07.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f104406ae70 msgr2=0x7f104406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f104406ae70 0x7f104406d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f10600c8bd0 tx=0x7f105c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:07.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 msgr2=0x7f10600b3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:07.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 0x7f10600b3f40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f105000c430 tx=0x7f105000c8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 shutdown_connections 2023-11-08T19:26:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f104406ae70 0x7f104406d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f106015f1d0 0x7f10600addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1060144960 0x7f10600b3f40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 --2- 172.21.15.119:0/4018761821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1060144580 0x7f10600cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 >> 172.21.15.119:0/4018761821 conn(0x7f10600cb610 msgr2=0x7f10600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:07.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 shutdown_connections 2023-11-08T19:26:07.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:07.924+0000 7f1067aa9640 1 -- 172.21.15.119:0/4018761821 wait complete. 2023-11-08T19:26:07.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:08.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.146+0000 7fdf218c2640 1 Processor -- start 2023-11-08T19:26:08.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.147+0000 7fdf218c2640 1 -- start start 2023-11-08T19:26:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.148+0000 7fdf218c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf1c142f80 0x7fdf1c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf218c2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf1c15d9f0 0x7fdf1c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf218c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf218c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c143a40 con 0x7fdf1c142f80 2023-11-08T19:26:08.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf218c2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c06d510 con 0x7fdf1c15d9f0 2023-11-08T19:26:08.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf218c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0c4960 con 0x7fdf1c06cb40 2023-11-08T19:26:08.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf1c142f80 0x7fdf1c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf1c142f80 0x7fdf1c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56002/0 (socket says 172.21.15.119:56002) 2023-11-08T19:26:08.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 -- 172.21.15.119:0/3312122766 learned_addr learned my addr 172.21.15.119:0/3312122766 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:08.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf127fc640 1 --2- 172.21.15.119:0/3312122766 >> v2:172.21.15.188:3300/0 conn(0x7fdf1c15d9f0 0x7fdf1c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1b7fe640 1 --2- 172.21.15.119:0/3312122766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 -- 172.21.15.119:0/3312122766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 msgr2=0x7fdf1c06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 --2- 172.21.15.119:0/3312122766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c06cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 -- 172.21.15.119:0/3312122766 >> v2:172.21.15.188:3300/0 conn(0x7fdf1c15d9f0 msgr2=0x7fdf1c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.149+0000 7fdf1affd640 1 --2- 172.21.15.119:0/3312122766 >> v2:172.21.15.188:3300/0 conn(0x7fdf1c15d9f0 0x7fdf1c0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.150+0000 7fdf1affd640 1 -- 172.21.15.119:0/3312122766 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c15df60 con 0x7fdf1c142f80 2023-11-08T19:26:08.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.151+0000 7fdf1affd640 1 --2- 172.21.15.119:0/3312122766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0cc6d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c06df40 tx=0x7fdf0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6afbeb5a56430da5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.151+0000 7fdf1a7fc640 1 -- 172.21.15.119:0/3312122766 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c031870 con 0x7fdf1c142f80 2023-11-08T19:26:08.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.151+0000 7fdf1a7fc640 1 -- 172.21.15.119:0/3312122766 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0c0319d0 con 0x7fdf1c142f80 2023-11-08T19:26:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.151+0000 7fdf1a7fc640 1 -- 172.21.15.119:0/3312122766 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c031d60 con 0x7fdf1c142f80 2023-11-08T19:26:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 -- 172.21.15.119:0/3312122766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 msgr2=0x7fdf1c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 --2- 172.21.15.119:0/3312122766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0cc6d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c06df40 tx=0x7fdf0c02b910 comp rx=0 tx=0).stop 2023-11-08T19:26:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 -- 172.21.15.119:0/3312122766 shutdown_connections 2023-11-08T19:26:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 --2- 172.21.15.119:0/3312122766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c06cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 --2- 172.21.15.119:0/3312122766 >> v2:172.21.15.188:3300/0 conn(0x7fdf1c15d9f0 0x7fdf1c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 --2- 172.21.15.119:0/3312122766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0cc6d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 -- 172.21.15.119:0/3312122766 >> 172.21.15.119:0/3312122766 conn(0x7fdf1c0cb610 msgr2=0x7fdf1c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 -- 172.21.15.119:0/3312122766 shutdown_connections 2023-11-08T19:26:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.152+0000 7fdf218c2640 1 -- 172.21.15.119:0/3312122766 wait complete. 2023-11-08T19:26:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.154+0000 7fdf218c2640 1 Processor -- start 2023-11-08T19:26:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.154+0000 7fdf218c2640 1 -- start start 2023-11-08T19:26:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf218c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf218c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf218c2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c15d9f0 0x7fdf1c0b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf218c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0b2ed0 con 0x7fdf1c142f80 2023-11-08T19:26:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf218c2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0acf50 con 0x7fdf1c15d9f0 2023-11-08T19:26:08.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf218c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c0ad230 con 0x7fdf1c06cb40 2023-11-08T19:26:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf127fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf127fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56004/0 (socket says 172.21.15.119:56004) 2023-11-08T19:26:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.155+0000 7fdf127fc640 1 -- 172.21.15.119:0/1520616143 learned_addr learned my addr 172.21.15.119:0/1520616143 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf1b7fe640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c15d9f0 0x7fdf1c0b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf127fc640 1 -- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 msgr2=0x7fdf1c0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf1affd640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf127fc640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c0ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf127fc640 1 -- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c15d9f0 msgr2=0x7fdf1c0b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf127fc640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c15d9f0 0x7fdf1c0b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf127fc640 1 -- 172.21.15.119:0/1520616143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0c0094d0 con 0x7fdf1c142f80 2023-11-08T19:26:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf1b7fe640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c15d9f0 0x7fdf1c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf127fc640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0adff0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c7f0 tx=0x7fdf0400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04004220 con 0x7fdf1c142f80 2023-11-08T19:26:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf040043a0 con 0x7fdf1c142f80 2023-11-08T19:26:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04007860 con 0x7fdf1c142f80 2023-11-08T19:26:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.156+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c0ad510 con 0x7fdf1c142f80 2023-11-08T19:26:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.157+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c0adad0 con 0x7fdf1c142f80 2023-11-08T19:26:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.158+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c074430 con 0x7fdf1c142f80 2023-11-08T19:26:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.159+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf04007bd0 con 0x7fdf1c142f80 2023-11-08T19:26:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.159+0000 7fdf18ff9640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdefc06aea0 0x7fdefc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.160+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdf04089990 con 0x7fdf1c142f80 2023-11-08T19:26:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.160+0000 7fdf1affd640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdefc06aea0 0x7fdefc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.160+0000 7fdf1affd640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdefc06aea0 0x7fdefc06d360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c036000 tx=0x7fdf0c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.164+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf04054650 con 0x7fdf1c142f80 2023-11-08T19:26:08.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.366+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdf1c0cacf0 con 0x7fdf1c142f80 2023-11-08T19:26:08.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.367+0000 7fdf18ff9640 1 -- 172.21.15.119:0/1520616143 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdf0401e080 con 0x7fdf1c142f80 2023-11-08T19:26:08.369 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:26:08.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdefc06aea0 msgr2=0x7fdefc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdefc06aea0 0x7fdefc06d360 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c036000 tx=0x7fdf0c03c040 comp rx=0 tx=0).stop 2023-11-08T19:26:08.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 msgr2=0x7fdf1c0adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0adff0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c7f0 tx=0x7fdf0400ccb0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 shutdown_connections 2023-11-08T19:26:08.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdefc06aea0 0x7fdefc06d360 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf1c15d9f0 0x7fdf1c0b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf1c142f80 0x7fdf1c0adff0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 --2- 172.21.15.119:0/1520616143 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1c06cb40 0x7fdf1c0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 >> 172.21.15.119:0/1520616143 conn(0x7fdf1c0cb610 msgr2=0x7fdf1c0cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.370+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 shutdown_connections 2023-11-08T19:26:08.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.371+0000 7fdf218c2640 1 -- 172.21.15.119:0/1520616143 wait complete. 2023-11-08T19:26:08.390 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-08T19:26:08.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:08.391 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-08T19:26:08.484 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:26:08.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:08.485 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T19:26:08.537 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:08.537 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:08.599 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:08.599 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:08.652 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:08.653 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:08.706 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.929+0000 7f9943887640 1 Processor -- start 2023-11-08T19:26:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.930+0000 7f9943887640 1 -- start start 2023-11-08T19:26:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9943887640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f993c1447a0 0x7f993c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9943887640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f993c0cad30 0x7f993c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9943887640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c06a120 0x7f993c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9943887640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c06aa90 con 0x7f993c0cad30 2023-11-08T19:26:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9943887640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993c0c3a50 con 0x7f993c1447a0 2023-11-08T19:26:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9943887640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c144b80 con 0x7f993c06a120 2023-11-08T19:26:08.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9940e22640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f993c0cad30 0x7f993c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9940e22640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f993c0cad30 0x7f993c06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44652/0 (socket says 172.21.15.119:44652) 2023-11-08T19:26:08.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.931+0000 7f9940e22640 1 -- 172.21.15.119:0/1740894439 learned_addr learned my addr 172.21.15.119:0/1740894439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9941623640 1 --2- 172.21.15.119:0/1740894439 >> v2:172.21.15.188:3300/0 conn(0x7f993c1447a0 0x7f993c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9940e22640 1 -- 172.21.15.119:0/1740894439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c06a120 msgr2=0x7f993c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9940e22640 1 --2- 172.21.15.119:0/1740894439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c06a120 0x7f993c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9940e22640 1 -- 172.21.15.119:0/1740894439 >> v2:172.21.15.188:3300/0 conn(0x7f993c1447a0 msgr2=0x7f993c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9940e22640 1 --2- 172.21.15.119:0/1740894439 >> v2:172.21.15.188:3300/0 conn(0x7f993c1447a0 0x7f993c15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9940e22640 1 -- 172.21.15.119:0/1740894439 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0c9150 con 0x7f993c0cad30 2023-11-08T19:26:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.932+0000 7f9941623640 1 --2- 172.21.15.119:0/1740894439 >> v2:172.21.15.188:3300/0 conn(0x7f993c1447a0 0x7f993c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.933+0000 7f9940e22640 1 --2- 172.21.15.119:0/1740894439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c0cad30 0x7f993c06f930 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f993c0cfbd0 tx=0x7f992c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15efee759e61669e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.933+0000 7f9933fff640 1 -- 172.21.15.119:0/1740894439 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c031870 con 0x7f993c0cad30 2023-11-08T19:26:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.933+0000 7f9933fff640 1 -- 172.21.15.119:0/1740894439 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992c0319d0 con 0x7f993c0cad30 2023-11-08T19:26:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.933+0000 7f9933fff640 1 -- 172.21.15.119:0/1740894439 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992c031d60 con 0x7f993c0cad30 2023-11-08T19:26:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 -- 172.21.15.119:0/1740894439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c0cad30 msgr2=0x7f993c06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 --2- 172.21.15.119:0/1740894439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c0cad30 0x7f993c06f930 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f993c0cfbd0 tx=0x7f992c02b910 comp rx=0 tx=0).stop 2023-11-08T19:26:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 -- 172.21.15.119:0/1740894439 shutdown_connections 2023-11-08T19:26:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 --2- 172.21.15.119:0/1740894439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c06a120 0x7f993c06a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 --2- 172.21.15.119:0/1740894439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c0cad30 0x7f993c06f930 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 --2- 172.21.15.119:0/1740894439 >> v2:172.21.15.188:3300/0 conn(0x7f993c1447a0 0x7f993c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 -- 172.21.15.119:0/1740894439 >> 172.21.15.119:0/1740894439 conn(0x7f993c0cb610 msgr2=0x7f993c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 -- 172.21.15.119:0/1740894439 shutdown_connections 2023-11-08T19:26:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.934+0000 7f9943887640 1 -- 172.21.15.119:0/1740894439 wait complete. 2023-11-08T19:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.936+0000 7f9943887640 1 Processor -- start 2023-11-08T19:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.936+0000 7f9943887640 1 -- start start 2023-11-08T19:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9943887640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 0x7f993c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9943887640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c0cad30 0x7f993c0c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9943887640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f993c1447a0 0x7f993c0c7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9943887640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c15d270 con 0x7f993c06a120 2023-11-08T19:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9943887640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993c15c660 con 0x7f993c1447a0 2023-11-08T19:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9943887640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c15c940 con 0x7f993c0cad30 2023-11-08T19:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 0x7f993c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 0x7f993c0c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44660/0 (socket says 172.21.15.119:44660) 2023-11-08T19:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 -- 172.21.15.119:0/2001633855 learned_addr learned my addr 172.21.15.119:0/2001633855 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 -- 172.21.15.119:0/2001633855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c0cad30 msgr2=0x7f993c0c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c0cad30 0x7f993c0c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 -- 172.21.15.119:0/2001633855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f993c1447a0 msgr2=0x7f993c0c7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:08.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f993c1447a0 0x7f993c0c7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.937+0000 7f9941623640 1 -- 172.21.15.119:0/2001633855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c0094d0 con 0x7f993c06a120 2023-11-08T19:26:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.938+0000 7f9941623640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 0x7f993c0c5bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f993800c7f0 tx=0x7f993800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.938+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938007a30 con 0x7f993c06a120 2023-11-08T19:26:08.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.938+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9938007b90 con 0x7f993c06a120 2023-11-08T19:26:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.938+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993800ce40 con 0x7f993c06a120 2023-11-08T19:26:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.938+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993c15cc20 con 0x7f993c06a120 2023-11-08T19:26:08.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.938+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993c15ac60 con 0x7f993c06a120 2023-11-08T19:26:08.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.939+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993c15d3e0 con 0x7f993c06a120 2023-11-08T19:26:08.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.940+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f99380113f0 con 0x7f993c06a120 2023-11-08T19:26:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.941+0000 7f9931ffb640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f992006ae70 0x7f992006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.941+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f9938089cb0 con 0x7f993c06a120 2023-11-08T19:26:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.941+0000 7f9940e22640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f992006ae70 0x7f992006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:08.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.942+0000 7f9940e22640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f992006ae70 0x7f992006d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f992c036000 tx=0x7f992c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:08.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:08.946+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f99380548e0 con 0x7f993c06a120 2023-11-08T19:26:09.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.138+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f993c074430 con 0x7f993c06a120 2023-11-08T19:26:09.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.139+0000 7f9931ffb640 1 -- 172.21.15.119:0/2001633855 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f993801f090 con 0x7f993c06a120 2023-11-08T19:26:09.141 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:09.142 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:09.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f992006ae70 msgr2=0x7f992006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f992006ae70 0x7f992006d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f992c036000 tx=0x7f992c030000 comp rx=0 tx=0).stop 2023-11-08T19:26:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 msgr2=0x7f993c0c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 0x7f993c0c5bd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f993800c7f0 tx=0x7f993800cbf0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 shutdown_connections 2023-11-08T19:26:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f992006ae70 0x7f992006d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f993c1447a0 0x7f993c0c7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f993c0cad30 0x7f993c0c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 --2- 172.21.15.119:0/2001633855 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f993c06a120 0x7f993c0c5bd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 >> 172.21.15.119:0/2001633855 conn(0x7f993c0cb610 msgr2=0x7f993c0c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 shutdown_connections 2023-11-08T19:26:09.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.143+0000 7f9943887640 1 -- 172.21.15.119:0/2001633855 wait complete. 2023-11-08T19:26:09.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:26:09.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:09.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.370+0000 7f29c8864640 1 Processor -- start 2023-11-08T19:26:09.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.370+0000 7f29c8864640 1 -- start start 2023-11-08T19:26:09.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c8864640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29c4144580 0x7f29c415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c8864640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f29c415f1d0 0x7f29c40c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c8864640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c415ea60 con 0x7f29c4144960 2023-11-08T19:26:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c8864640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40c9150 con 0x7f29c4144580 2023-11-08T19:26:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c8864640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c40c9360 con 0x7f29c415f1d0 2023-11-08T19:26:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29c4144580 0x7f29c415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f29c4144580 0x7f29c415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44672/0 (socket says 172.21.15.119:44672) 2023-11-08T19:26:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c259c640 1 -- 172.21.15.119:0/964577312 learned_addr learned my addr 172.21.15.119:0/964577312 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.372+0000 7f29c2d9d640 1 --2- 172.21.15.119:0/964577312 >> v2:172.21.15.188:3300/0 conn(0x7f29c415f1d0 0x7f29c40c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.373+0000 7f29c259c640 1 -- 172.21.15.119:0/964577312 >> v2:172.21.15.188:3300/0 conn(0x7f29c415f1d0 msgr2=0x7f29c40c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.373+0000 7f29c259c640 1 --2- 172.21.15.119:0/964577312 >> v2:172.21.15.188:3300/0 conn(0x7f29c415f1d0 0x7f29c40c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.373+0000 7f29c259c640 1 -- 172.21.15.119:0/964577312 >> v1:172.21.15.188:6789/0 conn(0x7f29c4144960 legacy=0x7f29c40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.373+0000 7f29c259c640 1 -- 172.21.15.119:0/964577312 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c40c9640 con 0x7f29c4144580 2023-11-08T19:26:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.374+0000 7f29c259c640 1 --2- 172.21.15.119:0/964577312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c4144580 0x7f29c415bff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f29c406d420 tx=0x7f29b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def4d4513b6e893f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.374+0000 7f29c1d9b640 1 -- 172.21.15.119:0/964577312 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b40317e0 con 0x7f29c4144580 2023-11-08T19:26:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.374+0000 7f29c1d9b640 1 -- 172.21.15.119:0/964577312 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b4031940 con 0x7f29c4144580 2023-11-08T19:26:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.374+0000 7f29c1d9b640 1 -- 172.21.15.119:0/964577312 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b4031cd0 con 0x7f29c4144580 2023-11-08T19:26:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 -- 172.21.15.119:0/964577312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c4144580 msgr2=0x7f29c415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 --2- 172.21.15.119:0/964577312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c4144580 0x7f29c415bff0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f29c406d420 tx=0x7f29b402b880 comp rx=0 tx=0).stop 2023-11-08T19:26:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 -- 172.21.15.119:0/964577312 shutdown_connections 2023-11-08T19:26:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 --2- 172.21.15.119:0/964577312 >> v2:172.21.15.188:3300/0 conn(0x7f29c415f1d0 0x7f29c40c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 --2- 172.21.15.119:0/964577312 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c4144580 0x7f29c415bff0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 -- 172.21.15.119:0/964577312 >> 172.21.15.119:0/964577312 conn(0x7f29c40cb610 msgr2=0x7f29c40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 -- 172.21.15.119:0/964577312 shutdown_connections 2023-11-08T19:26:09.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.375+0000 7f29c8864640 1 -- 172.21.15.119:0/964577312 wait complete. 2023-11-08T19:26:09.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.377+0000 7f29c8864640 1 Processor -- start 2023-11-08T19:26:09.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.377+0000 7f29c8864640 1 -- start start 2023-11-08T19:26:09.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c8864640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4144580 0x7f29c40ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c8864640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c4144960 0x7f29c40c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c8864640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 0x7f29c413d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c8864640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40c6c30 con 0x7f29c415f1d0 2023-11-08T19:26:09.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c8864640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40c7f00 con 0x7f29c4144580 2023-11-08T19:26:09.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c8864640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c40c81e0 con 0x7f29c4144960 2023-11-08T19:26:09.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 0x7f29c413d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 0x7f29c413d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44686/0 (socket says 172.21.15.119:44686) 2023-11-08T19:26:09.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 -- 172.21.15.119:0/653093912 learned_addr learned my addr 172.21.15.119:0/653093912 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:09.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 -- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c4144960 msgr2=0x7f29c40c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c4144960 0x7f29c40c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c259c640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4144580 0x7f29c40ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 -- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4144580 msgr2=0x7f29c40ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4144580 0x7f29c40ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.378+0000 7f29c2d9d640 1 -- 172.21.15.119:0/653093912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b40094d0 con 0x7f29c415f1d0 2023-11-08T19:26:09.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.379+0000 7f29c2d9d640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 0x7f29c413d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f29bc007d90 tx=0x7f29bc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.379+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc014420 con 0x7f29c415f1d0 2023-11-08T19:26:09.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.379+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc0145a0 con 0x7f29c415f1d0 2023-11-08T19:26:09.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.379+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc014950 con 0x7f29c415f1d0 2023-11-08T19:26:09.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.379+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c40c65e0 con 0x7f29c415f1d0 2023-11-08T19:26:09.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.379+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c413df00 con 0x7f29c415f1d0 2023-11-08T19:26:09.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.380+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c40c6de0 con 0x7f29c415f1d0 2023-11-08T19:26:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.381+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29bc014ab0 con 0x7f29c415f1d0 2023-11-08T19:26:09.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.382+0000 7f29baffd640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29a406ae70 0x7f29a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.382+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f29bc08a3e0 con 0x7f29c415f1d0 2023-11-08T19:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.382+0000 7f29c259c640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29a406ae70 0x7f29a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.383+0000 7f29c259c640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29a406ae70 0x7f29a406d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29b4036000 tx=0x7f29b403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.386+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f29c40c6de0 con 0x7f29c415f1d0 2023-11-08T19:26:09.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.574+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29c4074430 con 0x7f29c415f1d0 2023-11-08T19:26:09.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.575+0000 7f29baffd640 1 -- 172.21.15.119:0/653093912 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29bc058aa0 con 0x7f29c415f1d0 2023-11-08T19:26:09.576 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:09.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29a406ae70 msgr2=0x7f29a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29a406ae70 0x7f29a406d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f29b4036000 tx=0x7f29b403d040 comp rx=0 tx=0).stop 2023-11-08T19:26:09.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 msgr2=0x7f29c413d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 0x7f29c413d9c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f29bc007d90 tx=0x7f29bc00e4b0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 shutdown_connections 2023-11-08T19:26:09.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f29a406ae70 0x7f29a406d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29c415f1d0 0x7f29c413d9c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29c4144960 0x7f29c40c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 --2- 172.21.15.119:0/653093912 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29c4144580 0x7f29c40ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 >> 172.21.15.119:0/653093912 conn(0x7f29c40cb610 msgr2=0x7f29c40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 shutdown_connections 2023-11-08T19:26:09.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.577+0000 7f29c8864640 1 -- 172.21.15.119:0/653093912 wait complete. 2023-11-08T19:26:09.598 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:09.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.815+0000 7f6f4fff5640 1 Processor -- start 2023-11-08T19:26:09.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.815+0000 7f6f4fff5640 1 -- start start 2023-11-08T19:26:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.816+0000 7f6f4fff5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 0x7f6f480c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.816+0000 7f6f4fff5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4815e440 con 0x7f6f4806d250 2023-11-08T19:26:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4fff5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4815e720 con 0x7f6f4815d9f0 2023-11-08T19:26:09.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4fff5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c4f50 con 0x7f6f48143600 2023-11-08T19:26:09.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4e592640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6f4806d250 0x7f6f480c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58754/0 (socket says 172.21.15.119:58754) 2023-11-08T19:26:09.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4e592640 1 -- 172.21.15.119:0/2278464101 learned_addr learned my addr 172.21.15.119:0/2278464101 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4dd91640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 0x7f6f480c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234307989 0 0) 0x7f6f4815e440 con 0x7f6f4806d250 2023-11-08T19:26:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.817+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f34003520 con 0x7f6f4806d250 2023-11-08T19:26:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432842802 0 0) 0x7f6f34003520 con 0x7f6f4806d250 2023-11-08T19:26:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 msgr2=0x7f6f480c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 0x7f6f480c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 >> v1:172.21.15.188:6789/0 conn(0x7f6f4815d9f0 legacy=0x7f6f4806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480c4b70 con 0x7f6f4806d250 2023-11-08T19:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f6f400019f0 con 0x7f6f4806d250 2023-11-08T19:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 >> v1:172.21.15.119:6789/0 conn(0x7f6f4806d250 legacy=0x7f6f480c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f340039a0 0x7f6f34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f340047c0 0x7f6f34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 0x7f6f34005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c4f50 con 0x7f6f340047c0 2023-11-08T19:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3400bb10 con 0x7f6f340039a0 2023-11-08T19:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3400dee0 con 0x7f6f48143600 2023-11-08T19:26:09.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4d590640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f340047c0 0x7f6f34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.818+0000 7f6f4dd91640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f340039a0 0x7f6f34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.819+0000 7f6f4e592640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 0x7f6f34005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.819+0000 7f6f4d590640 1 -- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 msgr2=0x7f6f34005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.819+0000 7f6f4d590640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f48143600 0x7f6f34005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.819+0000 7f6f4d590640 1 -- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f340039a0 msgr2=0x7f6f34003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.819+0000 7f6f4d590640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f340039a0 0x7f6f34003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.819+0000 7f6f4d590640 1 -- 172.21.15.119:0/2278464101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f34003520 con 0x7f6f340047c0 2023-11-08T19:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.820+0000 7f6f4d590640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f340047c0 0x7f6f34004c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f38001750 tx=0x7f6f340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1d5e1ed1a867a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.820+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380052d0 con 0x7f6f340047c0 2023-11-08T19:26:09.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.820+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3802e050 con 0x7f6f340047c0 2023-11-08T19:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.820+0000 7f6f4cd8f640 1 -- 172.21.15.119:0/2278464101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380035f0 con 0x7f6f340047c0 2023-11-08T19:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.820+0000 7f6f4fff5640 1 -- 172.21.15.119:0/2278464101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f340047c0 msgr2=0x7f6f34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.820+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/2278464101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f340047c0 0x7f6f34004c40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6f38001750 tx=0x7f6f340057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.821+0000 7f6f4e592640 1 -- 172.21.15.119:0/2278464101 reap_dead start 2023-11-08T19:26:09.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.821+0000 7f6f4fff5640 1 -- 172.21.15.119:0/2278464101 shutdown_connections 2023-11-08T19:26:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.821+0000 7f6f4fff5640 1 -- 172.21.15.119:0/2278464101 >> 172.21.15.119:0/2278464101 conn(0x7f6f480cb610 msgr2=0x7f6f480cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.821+0000 7f6f4fff5640 1 -- 172.21.15.119:0/2278464101 shutdown_connections 2023-11-08T19:26:09.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.821+0000 7f6f4fff5640 1 -- 172.21.15.119:0/2278464101 wait complete. 2023-11-08T19:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.822+0000 7f6f4fff5640 1 Processor -- start 2023-11-08T19:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.823+0000 7f6f4fff5640 1 -- start start 2023-11-08T19:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4fff5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f340047c0 0x7f6f4815ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4fff5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 0x7f6f4813e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4fff5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f480b3c60 0x7f6f480b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4fff5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b4620 con 0x7f6f4813de90 2023-11-08T19:26:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4fff5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480c7dd0 con 0x7f6f480b3c60 2023-11-08T19:26:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4fff5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f480b23c0 con 0x7f6f340047c0 2023-11-08T19:26:09.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 0x7f6f4813e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 0x7f6f4813e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44718/0 (socket says 172.21.15.119:44718) 2023-11-08T19:26:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 -- 172.21.15.119:0/1363031688 learned_addr learned my addr 172.21.15.119:0/1363031688 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4e592640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f480b3c60 0x7f6f480b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4dd91640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f340047c0 0x7f6f4815ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 -- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f340047c0 msgr2=0x7f6f4815ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f340047c0 0x7f6f4815ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 -- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f480b3c60 msgr2=0x7f6f480b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f480b3c60 0x7f6f480b40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4d590640 1 -- 172.21.15.119:0/1363031688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f480b26a0 con 0x7f6f4813de90 2023-11-08T19:26:09.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.824+0000 7f6f4e592640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f480b3c60 0x7f6f480b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.825+0000 7f6f4d590640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 0x7f6f4813e310 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f38008fd0 tx=0x7f6f38001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.825+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f380076c0 con 0x7f6f4813de90 2023-11-08T19:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.825+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f38007a30 con 0x7f6f4813de90 2023-11-08T19:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.825+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f38006be0 con 0x7f6f4813de90 2023-11-08T19:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.825+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f480b2980 con 0x7f6f4813de90 2023-11-08T19:26:09.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.825+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f480b2f40 con 0x7f6f4813de90 2023-11-08T19:26:09.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.826+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f480c4230 con 0x7f6f4813de90 2023-11-08T19:26:09.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.827+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f38006d40 con 0x7f6f4813de90 2023-11-08T19:26:09.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.828+0000 7f6f3e7fc640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6f2c06aea0 0x7f6f2c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:09.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.828+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f6f380ab750 con 0x7f6f4813de90 2023-11-08T19:26:09.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.828+0000 7f6f4dd91640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6f2c06aea0 0x7f6f2c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:09.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.829+0000 7f6f4dd91640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6f2c06aea0 0x7f6f2c06d360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f480c8640 tx=0x7f6f44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:09.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:09.833+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f38076380 con 0x7f6f4813de90 2023-11-08T19:26:10.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.023+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f480ccdf0 con 0x7f6f4813de90 2023-11-08T19:26:10.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.024+0000 7f6f3e7fc640 1 -- 172.21.15.119:0/1363031688 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f380799a0 con 0x7f6f4813de90 2023-11-08T19:26:10.026 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:10.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6f2c06aea0 msgr2=0x7f6f2c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6f2c06aea0 0x7f6f2c06d360 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6f480c8640 tx=0x7f6f44008040 comp rx=0 tx=0).stop 2023-11-08T19:26:10.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 msgr2=0x7f6f4813e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 0x7f6f4813e310 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6f38008fd0 tx=0x7f6f38001120 comp rx=0 tx=0).stop 2023-11-08T19:26:10.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 shutdown_connections 2023-11-08T19:26:10.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6f2c06aea0 0x7f6f2c06d360 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f480b3c60 0x7f6f480b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6f4813de90 0x7f6f4813e310 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 --2- 172.21.15.119:0/1363031688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f340047c0 0x7f6f4815ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 >> 172.21.15.119:0/1363031688 conn(0x7f6f480cb610 msgr2=0x7f6f480c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 shutdown_connections 2023-11-08T19:26:10.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.027+0000 7f6f4fff5640 1 -- 172.21.15.119:0/1363031688 wait complete. 2023-11-08T19:26:10.047 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:10.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.265+0000 7f376d4d4640 1 Processor -- start 2023-11-08T19:26:10.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.265+0000 7f376d4d4640 1 -- start start 2023-11-08T19:26:10.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f376d4d4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3768144580 0x7f376815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f376d4d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f376815f1d0 0x7f37680c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f376d4d4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f376815ea60 con 0x7f3768144960 2023-11-08T19:26:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f376d4d4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37680c9150 con 0x7f3768144580 2023-11-08T19:26:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f376d4d4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37680c9360 con 0x7f376815f1d0 2023-11-08T19:26:10.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f3766ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3768144580 0x7f376815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f37667fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3768144960 0x7f37680cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58760/0 (socket says 172.21.15.119:58760) 2023-11-08T19:26:10.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.267+0000 7f37667fc640 1 -- 172.21.15.119:0/3300456099 learned_addr learned my addr 172.21.15.119:0/3300456099 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.268+0000 7f37677fe640 1 --2- 172.21.15.119:0/3300456099 >> v2:172.21.15.188:3300/0 conn(0x7f376815f1d0 0x7f37680c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.268+0000 7f3766ffd640 1 -- 172.21.15.119:0/3300456099 >> v2:172.21.15.188:3300/0 conn(0x7f376815f1d0 msgr2=0x7f37680c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.268+0000 7f3766ffd640 1 --2- 172.21.15.119:0/3300456099 >> v2:172.21.15.188:3300/0 conn(0x7f376815f1d0 0x7f37680c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.268+0000 7f3766ffd640 1 -- 172.21.15.119:0/3300456099 >> v1:172.21.15.119:6789/0 conn(0x7f3768144960 legacy=0x7f37680cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.268+0000 7f3766ffd640 1 -- 172.21.15.119:0/3300456099 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37680c9640 con 0x7f3768144580 2023-11-08T19:26:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.268+0000 7f3765ffb640 1 -- 172.21.15.119:0/3300456099 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274966507 0 0) 0x7f376815ea60 con 0x7f3768144960 2023-11-08T19:26:10.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.269+0000 7f3766ffd640 1 --2- 172.21.15.119:0/3300456099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3768144580 0x7f376815bff0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f376806d420 tx=0x7f375802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c373491c7f6a03 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.269+0000 7f3765ffb640 1 -- 172.21.15.119:0/3300456099 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375802e070 con 0x7f3768144580 2023-11-08T19:26:10.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.269+0000 7f3765ffb640 1 -- 172.21.15.119:0/3300456099 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37580315a0 con 0x7f3768144580 2023-11-08T19:26:10.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.269+0000 7f3765ffb640 1 -- 172.21.15.119:0/3300456099 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758031930 con 0x7f3768144580 2023-11-08T19:26:10.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 -- 172.21.15.119:0/3300456099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3768144580 msgr2=0x7f376815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 --2- 172.21.15.119:0/3300456099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3768144580 0x7f376815bff0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f376806d420 tx=0x7f375802b840 comp rx=0 tx=0).stop 2023-11-08T19:26:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 -- 172.21.15.119:0/3300456099 shutdown_connections 2023-11-08T19:26:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 --2- 172.21.15.119:0/3300456099 >> v2:172.21.15.188:3300/0 conn(0x7f376815f1d0 0x7f37680c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 --2- 172.21.15.119:0/3300456099 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3768144580 0x7f376815bff0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 -- 172.21.15.119:0/3300456099 >> 172.21.15.119:0/3300456099 conn(0x7f37680cb610 msgr2=0x7f37680cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 -- 172.21.15.119:0/3300456099 shutdown_connections 2023-11-08T19:26:10.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.270+0000 7f376d4d4640 1 -- 172.21.15.119:0/3300456099 wait complete. 2023-11-08T19:26:10.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.272+0000 7f376d4d4640 1 Processor -- start 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.272+0000 7f376d4d4640 1 -- start start 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f376d4d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3768144580 0x7f376815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f376d4d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3768144960 0x7f37680a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f376d4d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 0x7f37680a9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f376d4d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376815ad70 con 0x7f376815f1d0 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f376d4d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37680c5f10 con 0x7f3768144580 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f376d4d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37681f2a10 con 0x7f3768144960 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 0x7f37680a9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 0x7f37680a9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44742/0 (socket says 172.21.15.119:44742) 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 -- 172.21.15.119:0/1688395095 learned_addr learned my addr 172.21.15.119:0/1688395095 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f3766ffd640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3768144580 0x7f376815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37667fc640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3768144960 0x7f37680a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 -- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3768144960 msgr2=0x7f37680a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3768144960 0x7f37680a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 -- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3768144580 msgr2=0x7f376815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.273+0000 7f37677fe640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3768144580 0x7f376815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f37677fe640 1 -- 172.21.15.119:0/1688395095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3758009400 con 0x7f376815f1d0 2023-11-08T19:26:10.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f3766ffd640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3768144580 0x7f376815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:10.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f37677fe640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 0x7f37680a9ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f376000cd20 tx=0x7f376000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003b90 con 0x7f376815f1d0 2023-11-08T19:26:10.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760003cf0 con 0x7f376815f1d0 2023-11-08T19:26:10.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760004240 con 0x7f376815f1d0 2023-11-08T19:26:10.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37680c76d0 con 0x7f376815f1d0 2023-11-08T19:26:10.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.274+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37680c7c90 con 0x7f376815f1d0 2023-11-08T19:26:10.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.275+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37681f2bd0 con 0x7f376815f1d0 2023-11-08T19:26:10.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.276+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37600043a0 con 0x7f376815f1d0 2023-11-08T19:26:10.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.277+0000 7f374f7fe640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375406ae70 0x7f375406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.277+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3760012070 con 0x7f376815f1d0 2023-11-08T19:26:10.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.277+0000 7f3766ffd640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375406ae70 0x7f375406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.278+0000 7f3766ffd640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375406ae70 0x7f375406d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3758036000 tx=0x7f375803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.281+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3760054fe0 con 0x7f376815f1d0 2023-11-08T19:26:10.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.470+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3768074430 con 0x7f376815f1d0 2023-11-08T19:26:10.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.471+0000 7f374f7fe640 1 -- 172.21.15.119:0/1688395095 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3760058600 con 0x7f376815f1d0 2023-11-08T19:26:10.472 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:10.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375406ae70 msgr2=0x7f375406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375406ae70 0x7f375406d330 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3758036000 tx=0x7f375803d040 comp rx=0 tx=0).stop 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 msgr2=0x7f37680a9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 0x7f37680a9ae0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f376000cd20 tx=0x7f376000a660 comp rx=0 tx=0).stop 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 shutdown_connections 2023-11-08T19:26:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375406ae70 0x7f375406d330 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f376815f1d0 0x7f37680a9ae0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3768144960 0x7f37680a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 --2- 172.21.15.119:0/1688395095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3768144580 0x7f376815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 >> 172.21.15.119:0/1688395095 conn(0x7f37680cb610 msgr2=0x7f37680cdb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.473+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 shutdown_connections 2023-11-08T19:26:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.474+0000 7f376d4d4640 1 -- 172.21.15.119:0/1688395095 wait complete. 2023-11-08T19:26:10.492 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:26:10.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.715+0000 7f8041b9e640 1 Processor -- start 2023-11-08T19:26:10.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.715+0000 7f8041b9e640 1 -- start start 2023-11-08T19:26:10.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.716+0000 7f8041b9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f803c06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.716+0000 7f8041b9e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c071720 con 0x7f803c15deb0 2023-11-08T19:26:10.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.716+0000 7f8041b9e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c0cf5e0 con 0x7f803c15e9a0 2023-11-08T19:26:10.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.716+0000 7f8041b9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0cf8c0 con 0x7f803c0cccf0 2023-11-08T19:26:10.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.716+0000 7f8040b9c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f803c15deb0 0x7f803c15e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58776/0 (socket says 172.21.15.119:58776) 2023-11-08T19:26:10.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.716+0000 7f8040b9c640 1 -- 172.21.15.119:0/3607137848 learned_addr learned my addr 172.21.15.119:0/3607137848 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:10.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f804139d640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f803c06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751174706 0 0) 0x7f803c071720 con 0x7f803c15deb0 2023-11-08T19:26:10.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8024003520 con 0x7f803c15deb0 2023-11-08T19:26:10.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3574788993 0 0) 0x7f8024003520 con 0x7f803c15deb0 2023-11-08T19:26:10.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 msgr2=0x7f803c06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f803c06bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 >> v1:172.21.15.188:6789/0 conn(0x7f803c15e9a0 legacy=0x7f803c06b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0710d0 con 0x7f803c15deb0 2023-11-08T19:26:10.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f8030001670 con 0x7f803c15deb0 2023-11-08T19:26:10.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 >> v1:172.21.15.119:6789/0 conn(0x7f803c15deb0 legacy=0x7f803c15e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80240039a0 0x7f8024003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f80240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.717+0000 7f803b7fe640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024004d00 0x7f8024005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0cf8c0 con 0x7f80240039a0 2023-11-08T19:26:10.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802400bc00 con 0x7f8024004d00 2023-11-08T19:26:10.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802400bdc0 con 0x7f803c0cccf0 2023-11-08T19:26:10.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80240039a0 0x7f8024003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f8040b9c640 1 -- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 msgr2=0x7f80240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f803bfff640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f80240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f804139d640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024004d00 0x7f8024005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f80240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f8040b9c640 1 -- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024004d00 msgr2=0x7f8024005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:10.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8024004d00 0x7f8024005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f8040b9c640 1 -- 172.21.15.119:0/3607137848 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8024003520 con 0x7f80240039a0 2023-11-08T19:26:10.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.718+0000 7f803bfff640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0cccf0 0x7f80240047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:10.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.719+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80240039a0 0x7f8024003dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f803000c0e0 tx=0x7f80240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddddfdfe3b281b23 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8030005ad0 con 0x7f80240039a0 2023-11-08T19:26:10.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8030005c30 con 0x7f80240039a0 2023-11-08T19:26:10.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f803b7fe640 1 -- 172.21.15.119:0/3607137848 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803002c810 con 0x7f80240039a0 2023-11-08T19:26:10.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f8041b9e640 1 -- 172.21.15.119:0/3607137848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80240039a0 msgr2=0x7f8024003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3607137848 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80240039a0 0x7f8024003dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f803000c0e0 tx=0x7f80240057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f804139d640 1 -- 172.21.15.119:0/3607137848 reap_dead start 2023-11-08T19:26:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f8041b9e640 1 -- 172.21.15.119:0/3607137848 shutdown_connections 2023-11-08T19:26:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f8041b9e640 1 -- 172.21.15.119:0/3607137848 >> 172.21.15.119:0/3607137848 conn(0x7f803c0ceac0 msgr2=0x7f803c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f8041b9e640 1 -- 172.21.15.119:0/3607137848 shutdown_connections 2023-11-08T19:26:10.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.720+0000 7f8041b9e640 1 -- 172.21.15.119:0/3607137848 wait complete. 2023-11-08T19:26:10.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.722+0000 7f8041b9e640 1 Processor -- start 2023-11-08T19:26:10.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.722+0000 7f8041b9e640 1 -- start start 2023-11-08T19:26:10.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.723+0000 7f8041b9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80240039a0 0x7f803c14fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.723+0000 7f8041b9e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 0x7f803c0c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.723+0000 7f8041b9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0c3480 0x7f803c0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.723+0000 7f8041b9e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0c76a0 con 0x7f803c0c3000 2023-11-08T19:26:10.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.723+0000 7f8041b9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0c6780 con 0x7f80240039a0 2023-11-08T19:26:10.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.723+0000 7f8041b9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0c6a20 con 0x7f803c0c3480 2023-11-08T19:26:10.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 0x7f803c0c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 0x7f803c0c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44756/0 (socket says 172.21.15.119:44756) 2023-11-08T19:26:10.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 -- 172.21.15.119:0/3330106198 learned_addr learned my addr 172.21.15.119:0/3330106198 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:10.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80240039a0 0x7f803c14fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 -- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0c3480 msgr2=0x7f803c0c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f804139d640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0c3480 0x7f803c0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0c3480 0x7f803c0c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 -- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80240039a0 msgr2=0x7f803c14fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80240039a0 0x7f803c14fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 -- 172.21.15.119:0/3330106198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0c6d00 con 0x7f803c0c3000 2023-11-08T19:26:10.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.724+0000 7f803bfff640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 0x7f803c0c7f60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d7a0 tx=0x7f802c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.725+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c017070 con 0x7f803c0c3000 2023-11-08T19:26:10.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.725+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c005070 con 0x7f803c0c3000 2023-11-08T19:26:10.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.725+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c0c6fe0 con 0x7f803c0c3000 2023-11-08T19:26:10.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.725+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c005420 con 0x7f803c0c3000 2023-11-08T19:26:10.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.725+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c0c7ac0 con 0x7f803c0c3000 2023-11-08T19:26:10.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.726+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c04ece0 con 0x7f803c0c3000 2023-11-08T19:26:10.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.727+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f802c007030 con 0x7f803c0c3000 2023-11-08T19:26:10.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.728+0000 7f80397fa640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f802806ae70 0x7f802806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:10.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.728+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f802c089320 con 0x7f803c0c3000 2023-11-08T19:26:10.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.728+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f802806ae70 0x7f802806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:10.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.729+0000 7f8040b9c640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f802806ae70 0x7f802806d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f8030009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:10.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.732+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f802c053fe0 con 0x7f803c0c3000 2023-11-08T19:26:10.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.925+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f803c0c2040 con 0x7f803c0c3000 2023-11-08T19:26:10.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.926+0000 7f80397fa640 1 -- 172.21.15.119:0/3330106198 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f802c057600 con 0x7f803c0c3000 2023-11-08T19:26:10.928 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:10.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f802806ae70 msgr2=0x7f802806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f802806ae70 0x7f802806d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8030034000 tx=0x7f8030009030 comp rx=0 tx=0).stop 2023-11-08T19:26:10.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 msgr2=0x7f803c0c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:10.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 0x7f803c0c7f60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d7a0 tx=0x7f802c00dc60 comp rx=0 tx=0).stop 2023-11-08T19:26:10.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 shutdown_connections 2023-11-08T19:26:10.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f802806ae70 0x7f802806d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f803c0c3480 0x7f803c0c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f803c0c3000 0x7f803c0c7f60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 --2- 172.21.15.119:0/3330106198 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80240039a0 0x7f803c14fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:10.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 >> 172.21.15.119:0/3330106198 conn(0x7f803c0ceac0 msgr2=0x7f803c070580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:10.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 shutdown_connections 2023-11-08T19:26:10.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:10.930+0000 7f8041b9e640 1 -- 172.21.15.119:0/3330106198 wait complete. 2023-11-08T19:26:10.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:11.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.169+0000 7f1a74ae5640 1 Processor -- start 2023-11-08T19:26:11.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.170+0000 7f1a74ae5640 1 -- start start 2023-11-08T19:26:11.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a74ae5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a7015b080 0x7f1a700ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a74ae5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a700cf8c0 0x7f1a700cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a74ae5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a700cda70 con 0x7f1a70161250 2023-11-08T19:26:11.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a74ae5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a70161dd0 con 0x7f1a7015b080 2023-11-08T19:26:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a74ae5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700d02b0 con 0x7f1a700cf8c0 2023-11-08T19:26:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a6e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a7015b080 0x7f1a700ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a6e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a7015b080 0x7f1a700ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44758/0 (socket says 172.21.15.119:44758) 2023-11-08T19:26:11.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.171+0000 7f1a6e59c640 1 -- 172.21.15.119:0/1802435522 learned_addr learned my addr 172.21.15.119:0/1802435522 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:11.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.172+0000 7f1a6dd9b640 1 --2- 172.21.15.119:0/1802435522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a700cf8c0 0x7f1a700cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.172+0000 7f1a6e59c640 1 -- 172.21.15.119:0/1802435522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a700cf8c0 msgr2=0x7f1a700cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:11.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.172+0000 7f1a6e59c640 1 --2- 172.21.15.119:0/1802435522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a700cf8c0 0x7f1a700cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.172+0000 7f1a6e59c640 1 -- 172.21.15.119:0/1802435522 >> v1:172.21.15.188:6789/0 conn(0x7f1a70161250 legacy=0x7f1a701616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.172+0000 7f1a6e59c640 1 -- 172.21.15.119:0/1802435522 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a70161fc0 con 0x7f1a7015b080 2023-11-08T19:26:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.173+0000 7f1a6e59c640 1 --2- 172.21.15.119:0/1802435522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a700ce7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009870 tx=0x7f1a6002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dacc38813ff60adb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.173+0000 7f1a6d59a640 1 -- 172.21.15.119:0/1802435522 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a600317c0 con 0x7f1a7015b080 2023-11-08T19:26:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.173+0000 7f1a6d59a640 1 -- 172.21.15.119:0/1802435522 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60031920 con 0x7f1a7015b080 2023-11-08T19:26:11.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.173+0000 7f1a6d59a640 1 -- 172.21.15.119:0/1802435522 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60031cb0 con 0x7f1a7015b080 2023-11-08T19:26:11.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 -- 172.21.15.119:0/1802435522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 msgr2=0x7f1a700ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/1802435522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a700ce7a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009870 tx=0x7f1a6002b860 comp rx=0 tx=0).stop 2023-11-08T19:26:11.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 -- 172.21.15.119:0/1802435522 shutdown_connections 2023-11-08T19:26:11.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/1802435522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a700cf8c0 0x7f1a700cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/1802435522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a700ce7a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 -- 172.21.15.119:0/1802435522 >> 172.21.15.119:0/1802435522 conn(0x7f1a7006a9f0 msgr2=0x7f1a7006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 -- 172.21.15.119:0/1802435522 shutdown_connections 2023-11-08T19:26:11.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.174+0000 7f1a74ae5640 1 -- 172.21.15.119:0/1802435522 wait complete. 2023-11-08T19:26:11.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.176+0000 7f1a74ae5640 1 Processor -- start 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.176+0000 7f1a74ae5640 1 -- start start 2023-11-08T19:26:11.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a74ae5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a700cf8c0 0x7f1a70149460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a74ae5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a7015e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a74ae5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a70161250 0x7f1a700ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a74ae5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7015e220 con 0x7f1a7015b080 2023-11-08T19:26:11.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a74ae5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a7015dc30 con 0x7f1a700cf8c0 2023-11-08T19:26:11.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a74ae5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a700cab10 con 0x7f1a70161250 2023-11-08T19:26:11.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.177+0000 7f1a6dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a7015e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a7015e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44766/0 (socket says 172.21.15.119:44766) 2023-11-08T19:26:11.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 -- 172.21.15.119:0/2881555374 learned_addr learned my addr 172.21.15.119:0/2881555374 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:11.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6e59c640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a700cf8c0 0x7f1a70149460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 -- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a70161250 msgr2=0x7f1a700ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6ed9d640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a70161250 0x7f1a700ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a70161250 0x7f1a700ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 -- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a700cf8c0 msgr2=0x7f1a70149460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a700cf8c0 0x7f1a70149460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 -- 172.21.15.119:0/2881555374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a600094d0 con 0x7f1a7015b080 2023-11-08T19:26:11.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6e59c640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a700cf8c0 0x7f1a70149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a6dd9b640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a7015e640 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7f0 tx=0x7f1a5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.178+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58004220 con 0x7f1a7015b080 2023-11-08T19:26:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.179+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a58004380 con 0x7f1a7015b080 2023-11-08T19:26:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.179+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58007840 con 0x7f1a7015b080 2023-11-08T19:26:11.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.179+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a700cadf0 con 0x7f1a7015b080 2023-11-08T19:26:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.179+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a701499a0 con 0x7f1a7015b080 2023-11-08T19:26:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.180+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a700cafe0 con 0x7f1a7015b080 2023-11-08T19:26:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.181+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a58007bb0 con 0x7f1a7015b080 2023-11-08T19:26:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.182+0000 7f1a66ffd640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a5006ae70 0x7f1a5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.182+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f1a58089990 con 0x7f1a7015b080 2023-11-08T19:26:11.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.182+0000 7f1a6e59c640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a5006ae70 0x7f1a5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.183+0000 7f1a6e59c640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a5006ae70 0x7f1a5006d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a60036000 tx=0x7f1a6003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.186+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a700cafe0 con 0x7f1a7015b080 2023-11-08T19:26:11.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.388+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1a700cafe0 con 0x7f1a7015b080 2023-11-08T19:26:11.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.389+0000 7f1a66ffd640 1 -- 172.21.15.119:0/2881555374 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1a700cafe0 con 0x7f1a7015b080 2023-11-08T19:26:11.391 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:26:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a5006ae70 msgr2=0x7f1a5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a5006ae70 0x7f1a5006d330 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1a60036000 tx=0x7f1a6003d040 comp rx=0 tx=0).stop 2023-11-08T19:26:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 msgr2=0x7f1a7015e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a7015e640 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7f0 tx=0x7f1a5800ccb0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 shutdown_connections 2023-11-08T19:26:11.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a5006ae70 0x7f1a5006d330 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a70161250 0x7f1a700ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a7015b080 0x7f1a7015e640 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 --2- 172.21.15.119:0/2881555374 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a700cf8c0 0x7f1a70149460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 >> 172.21.15.119:0/2881555374 conn(0x7f1a7006a9f0 msgr2=0x7f1a700b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 shutdown_connections 2023-11-08T19:26:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.392+0000 7f1a74ae5640 1 -- 172.21.15.119:0/2881555374 wait complete. 2023-11-08T19:26:11.413 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-08T19:26:11.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:11.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-08T19:26:11.521 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:26:11.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:11.522 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T19:26:11.577 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:26:11.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:11.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.795+0000 7f191ed4c640 1 Processor -- start 2023-11-08T19:26:11.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.795+0000 7f191ed4c640 1 -- start start 2023-11-08T19:26:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.796+0000 7f191ed4c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f191815f4c0 0x7f191806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.796+0000 7f191ed4c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f191806a120 0x7f191806a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191ed4c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191806aa70 con 0x7f19181447a0 2023-11-08T19:26:11.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191ed4c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191815f930 con 0x7f191806a120 2023-11-08T19:26:11.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191ed4c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180c3a50 con 0x7f191815f4c0 2023-11-08T19:26:11.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191d2e9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f191806a120 0x7f191806a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191cae8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f19181447a0 0x7f19180cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58790/0 (socket says 172.21.15.119:58790) 2023-11-08T19:26:11.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191cae8640 1 -- 172.21.15.119:0/1314662148 learned_addr learned my addr 172.21.15.119:0/1314662148 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191d2e9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f191806a120 0x7f191806a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44768/0 (socket says 172.21.15.119:44768) 2023-11-08T19:26:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f1917fff640 1 --2- 172.21.15.119:0/1314662148 >> v2:172.21.15.188:3300/0 conn(0x7f191815f4c0 0x7f191806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191d2e9640 1 -- 172.21.15.119:0/1314662148 >> v2:172.21.15.188:3300/0 conn(0x7f191815f4c0 msgr2=0x7f191806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191d2e9640 1 --2- 172.21.15.119:0/1314662148 >> v2:172.21.15.188:3300/0 conn(0x7f191815f4c0 0x7f191806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191d2e9640 1 -- 172.21.15.119:0/1314662148 >> v1:172.21.15.119:6789/0 conn(0x7f19181447a0 legacy=0x7f19180cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f191d2e9640 1 -- 172.21.15.119:0/1314662148 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1918144b80 con 0x7f191806a120 2023-11-08T19:26:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.797+0000 7f1917fff640 1 --2- 172.21.15.119:0/1314662148 >> v2:172.21.15.188:3300/0 conn(0x7f191815f4c0 0x7f191806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.799+0000 7f191d2e9640 1 --2- 172.21.15.119:0/1314662148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191806a530 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f191806df20 tx=0x7f191002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54efc310be2173a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.799+0000 7f19177fe640 1 -- 172.21.15.119:0/1314662148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191002cc70 con 0x7f191806a120 2023-11-08T19:26:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.799+0000 7f19177fe640 1 -- 172.21.15.119:0/1314662148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191002cdd0 con 0x7f191806a120 2023-11-08T19:26:11.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.799+0000 7f19177fe640 1 -- 172.21.15.119:0/1314662148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910034660 con 0x7f191806a120 2023-11-08T19:26:11.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.799+0000 7f191ed4c640 1 -- 172.21.15.119:0/1314662148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 msgr2=0x7f191806a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.799+0000 7f191ed4c640 1 --2- 172.21.15.119:0/1314662148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191806a530 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f191806df20 tx=0x7f191002bec0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.800+0000 7f191ed4c640 1 -- 172.21.15.119:0/1314662148 shutdown_connections 2023-11-08T19:26:11.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.800+0000 7f191ed4c640 1 --2- 172.21.15.119:0/1314662148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191806a530 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.800+0000 7f191ed4c640 1 --2- 172.21.15.119:0/1314662148 >> v2:172.21.15.188:3300/0 conn(0x7f191815f4c0 0x7f191806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.800+0000 7f191ed4c640 1 -- 172.21.15.119:0/1314662148 >> 172.21.15.119:0/1314662148 conn(0x7f19180cb610 msgr2=0x7f19180cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:11.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.800+0000 7f191ed4c640 1 -- 172.21.15.119:0/1314662148 shutdown_connections 2023-11-08T19:26:11.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.800+0000 7f191ed4c640 1 -- 172.21.15.119:0/1314662148 wait complete. 2023-11-08T19:26:11.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.801+0000 7f191ed4c640 1 Processor -- start 2023-11-08T19:26:11.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 -- start start 2023-11-08T19:26:11.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191815a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181447a0 0x7f19180c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815f4c0 0x7f19180c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c7150 con 0x7f191806a120 2023-11-08T19:26:11.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c64d0 con 0x7f19181447a0 2023-11-08T19:26:11.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.802+0000 7f191ed4c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c67b0 con 0x7f191815f4c0 2023-11-08T19:26:11.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191815a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191815a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44774/0 (socket says 172.21.15.119:44774) 2023-11-08T19:26:11.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 -- 172.21.15.119:0/346924135 learned_addr learned my addr 172.21.15.119:0/346924135 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:11.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 -- 172.21.15.119:0/346924135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815f4c0 msgr2=0x7f19180c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815f4c0 0x7f19180c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 -- 172.21.15.119:0/346924135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181447a0 msgr2=0x7f19180c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:11.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181447a0 0x7f19180c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:11.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 -- 172.21.15.119:0/346924135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1910009a70 con 0x7f191806a120 2023-11-08T19:26:11.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.803+0000 7f191cae8640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191815a7c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c450 tx=0x7f190c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.804+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c005ce0 con 0x7f191806a120 2023-11-08T19:26:11.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.804+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c005e40 con 0x7f191806a120 2023-11-08T19:26:11.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.804+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19180c6a90 con 0x7f191806a120 2023-11-08T19:26:11.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.804+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c007290 con 0x7f191806a120 2023-11-08T19:26:11.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.804+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191815ad00 con 0x7f191806a120 2023-11-08T19:26:11.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.805+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1918074430 con 0x7f191806a120 2023-11-08T19:26:11.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.806+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f190c0073f0 con 0x7f191806a120 2023-11-08T19:26:11.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.807+0000 7f19157fa640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f18fc06ae70 0x7f18fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:11.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.807+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f190c088f50 con 0x7f191806a120 2023-11-08T19:26:11.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.807+0000 7f1917fff640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f18fc06ae70 0x7f18fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:11.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.808+0000 7f1917fff640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f18fc06ae70 0x7f18fc06d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1904006fd0 tx=0x7f1904008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:11.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:11.811+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f190c053c20 con 0x7f191806a120 2023-11-08T19:26:12.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.045+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19180c3ce0 con 0x7f191806a120 2023-11-08T19:26:12.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.046+0000 7f19157fa640 1 -- 172.21.15.119:0/346924135 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f190c017020 con 0x7f191806a120 2023-11-08T19:26:12.048 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:12.048 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:26:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:26:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f18fc06ae70 msgr2=0x7f18fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f18fc06ae70 0x7f18fc06d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1904006fd0 tx=0x7f1904008040 comp rx=0 tx=0).stop 2023-11-08T19:26:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 msgr2=0x7f191815a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191815a7c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f190c00c450 tx=0x7f190c00c910 comp rx=0 tx=0).stop 2023-11-08T19:26:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 shutdown_connections 2023-11-08T19:26:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f18fc06ae70 0x7f18fc06d330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f191815f4c0 0x7f19180c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19181447a0 0x7f19180c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 --2- 172.21.15.119:0/346924135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f191806a120 0x7f191815a7c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 >> 172.21.15.119:0/346924135 conn(0x7f19180cb610 msgr2=0x7f19180cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.049+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 shutdown_connections 2023-11-08T19:26:12.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.050+0000 7f191ed4c640 1 -- 172.21.15.119:0/346924135 wait complete. 2023-11-08T19:26:12.070 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:12.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.277+0000 7f2454eb4640 1 Processor -- start 2023-11-08T19:26:12.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.277+0000 7f2454eb4640 1 -- start start 2023-11-08T19:26:12.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f2454eb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24501447a0 0x7f245015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f2454eb4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24500cad30 0x7f245006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f2454eb4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24500c3a50 con 0x7f245006a120 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f2454eb4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2450144b80 con 0x7f24501447a0 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f2454eb4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24500c9150 con 0x7f24500cad30 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f244e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24501447a0 0x7f245015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f244e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f24501447a0 0x7f245015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44782/0 (socket says 172.21.15.119:44782) 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f244e59c640 1 -- 172.21.15.119:0/742650702 learned_addr learned my addr 172.21.15.119:0/742650702 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:12.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f244ed9d640 1 --1- 172.21.15.119:0/742650702 >> v1:172.21.15.119:6789/0 conn(0x7f245006a120 0x7f245006a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58792/0 (socket says 172.21.15.119:58792) 2023-11-08T19:26:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f244dd9b640 1 --2- 172.21.15.119:0/742650702 >> v2:172.21.15.188:3300/0 conn(0x7f24500cad30 0x7f245006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.279+0000 7f244e59c640 1 -- 172.21.15.119:0/742650702 >> v2:172.21.15.188:3300/0 conn(0x7f24500cad30 msgr2=0x7f245006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.280+0000 7f244e59c640 1 --2- 172.21.15.119:0/742650702 >> v2:172.21.15.188:3300/0 conn(0x7f24500cad30 0x7f245006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.280+0000 7f244e59c640 1 -- 172.21.15.119:0/742650702 >> v1:172.21.15.119:6789/0 conn(0x7f245006a120 legacy=0x7f245006a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.280+0000 7f244e59c640 1 -- 172.21.15.119:0/742650702 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500c9410 con 0x7f24501447a0 2023-11-08T19:26:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.280+0000 7f244d59a640 1 -- 172.21.15.119:0/742650702 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695370186 0 0) 0x7f24500c3a50 con 0x7f245006a120 2023-11-08T19:26:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.280+0000 7f244dd9b640 1 --2- 172.21.15.119:0/742650702 >> v2:172.21.15.188:3300/0 conn(0x7f24500cad30 0x7f245006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.281+0000 7f244e59c640 1 --2- 172.21.15.119:0/742650702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245015f4c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f245006df20 tx=0x7f244002b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d04de21a14e93f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.281+0000 7f244d59a640 1 -- 172.21.15.119:0/742650702 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440030880 con 0x7f24501447a0 2023-11-08T19:26:12.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.281+0000 7f244d59a640 1 -- 172.21.15.119:0/742650702 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24400309e0 con 0x7f24501447a0 2023-11-08T19:26:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.281+0000 7f244d59a640 1 -- 172.21.15.119:0/742650702 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2440030d70 con 0x7f24501447a0 2023-11-08T19:26:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.281+0000 7f2454eb4640 1 -- 172.21.15.119:0/742650702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 msgr2=0x7f245015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.281+0000 7f2454eb4640 1 --2- 172.21.15.119:0/742650702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245015f4c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f245006df20 tx=0x7f244002b9c0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.282+0000 7f2454eb4640 1 -- 172.21.15.119:0/742650702 shutdown_connections 2023-11-08T19:26:12.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.282+0000 7f2454eb4640 1 --2- 172.21.15.119:0/742650702 >> v2:172.21.15.188:3300/0 conn(0x7f24500cad30 0x7f245006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.282+0000 7f2454eb4640 1 --2- 172.21.15.119:0/742650702 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245015f4c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.282+0000 7f2454eb4640 1 -- 172.21.15.119:0/742650702 >> 172.21.15.119:0/742650702 conn(0x7f24500cb610 msgr2=0x7f24500cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.282+0000 7f2454eb4640 1 -- 172.21.15.119:0/742650702 shutdown_connections 2023-11-08T19:26:12.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.282+0000 7f2454eb4640 1 -- 172.21.15.119:0/742650702 wait complete. 2023-11-08T19:26:12.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.283+0000 7f2454eb4640 1 Processor -- start 2023-11-08T19:26:12.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.283+0000 7f2454eb4640 1 -- start start 2023-11-08T19:26:12.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.284+0000 7f2454eb4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245006a120 0x7f2450159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.284+0000 7f2454eb4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24500cad30 0x7f2450158d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.284+0000 7f2454eb4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245013d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.284+0000 7f2454eb4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245015d220 con 0x7f24501447a0 2023-11-08T19:26:12.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.284+0000 7f2454eb4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245015a370 con 0x7f24500cad30 2023-11-08T19:26:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.284+0000 7f2454eb4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2450159270 con 0x7f245006a120 2023-11-08T19:26:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245013d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245013d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44792/0 (socket says 172.21.15.119:44792) 2023-11-08T19:26:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 -- 172.21.15.119:0/3117860571 learned_addr learned my addr 172.21.15.119:0/3117860571 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 -- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245006a120 msgr2=0x7f2450159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244e59c640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245006a120 0x7f2450159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244dd9b640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24500cad30 0x7f2450158d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245006a120 0x7f2450159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 -- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24500cad30 msgr2=0x7f2450158d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24500cad30 0x7f2450158d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 -- 172.21.15.119:0/3117860571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400094d0 con 0x7f24501447a0 2023-11-08T19:26:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.285+0000 7f244ed9d640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245013d720 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2448001560 tx=0x7f244800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.286+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244800a640 con 0x7f24501447a0 2023-11-08T19:26:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.286+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244800a7a0 con 0x7f24501447a0 2023-11-08T19:26:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.286+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244800ab30 con 0x7f24501447a0 2023-11-08T19:26:12.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.286+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245015abe0 con 0x7f24501447a0 2023-11-08T19:26:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.286+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f245015cbd0 con 0x7f24501447a0 2023-11-08T19:26:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.287+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2450074430 con 0x7f24501447a0 2023-11-08T19:26:12.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.288+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f244800ac90 con 0x7f24501447a0 2023-11-08T19:26:12.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.289+0000 7f243effd640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f243006ae70 0x7f243006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.289+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f24480892b0 con 0x7f24501447a0 2023-11-08T19:26:12.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.289+0000 7f244e59c640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f243006ae70 0x7f243006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.290+0000 7f244e59c640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f243006ae70 0x7f243006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2440036000 tx=0x7f244003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.293+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2448053f60 con 0x7f24501447a0 2023-11-08T19:26:12.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.485+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f245015d360 con 0x7f24501447a0 2023-11-08T19:26:12.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.486+0000 7f243effd640 1 -- 172.21.15.119:0/3117860571 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f2448057580 con 0x7f24501447a0 2023-11-08T19:26:12.488 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:12.488 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:12.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f243006ae70 msgr2=0x7f243006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f243006ae70 0x7f243006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2440036000 tx=0x7f244003b040 comp rx=0 tx=0).stop 2023-11-08T19:26:12.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 msgr2=0x7f245013d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245013d720 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f2448001560 tx=0x7f244800cc70 comp rx=0 tx=0).stop 2023-11-08T19:26:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 shutdown_connections 2023-11-08T19:26:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f243006ae70 0x7f243006d330 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24501447a0 0x7f245013d720 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24500cad30 0x7f2450158d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 --2- 172.21.15.119:0/3117860571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f245006a120 0x7f2450159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.489+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 >> 172.21.15.119:0/3117860571 conn(0x7f24500cb610 msgr2=0x7f24500c40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.490+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 shutdown_connections 2023-11-08T19:26:12.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.490+0000 7f2454eb4640 1 -- 172.21.15.119:0/3117860571 wait complete. 2023-11-08T19:26:12.510 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:24:56.122826+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:24:57.101213+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-08T19:25:00.722259+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-08T19:26:12.511 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:12.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.718+0000 7f9eabf0f640 1 Processor -- start 2023-11-08T19:26:12.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.718+0000 7f9eabf0f640 1 -- start start 2023-11-08T19:26:12.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.720+0000 7f9eabf0f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 0x7f9ea40c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.720+0000 7f9eabf0f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea415ea60 con 0x7f9ea4144960 2023-11-08T19:26:12.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.720+0000 7f9eabf0f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea415c460 con 0x7f9ea4144580 2023-11-08T19:26:12.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.720+0000 7f9eabf0f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40c9150 con 0x7f9ea415bff0 2023-11-08T19:26:12.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.720+0000 7f9ea94aa640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9ea4144960 0x7f9ea415f1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58796/0 (socket says 172.21.15.119:58796) 2023-11-08T19:26:12.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.720+0000 7f9ea94aa640 1 -- 172.21.15.119:0/3979962426 learned_addr learned my addr 172.21.15.119:0/3979962426 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9eaa4ac640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 0x7f9ea40c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545080590 0 0) 0x7f9ea415ea60 con 0x7f9ea4144960 2023-11-08T19:26:12.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e90003520 con 0x7f9ea4144960 2023-11-08T19:26:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892314258 0 0) 0x7f9e90003520 con 0x7f9ea4144960 2023-11-08T19:26:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 msgr2=0x7f9ea40c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 0x7f9ea40c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 >> v1:172.21.15.188:6789/0 conn(0x7f9ea4144580 legacy=0x7f9ea40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea415f8e0 con 0x7f9ea4144960 2023-11-08T19:26:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f9e94001670 con 0x7f9ea4144960 2023-11-08T19:26:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 >> v1:172.21.15.119:6789/0 conn(0x7f9ea4144960 legacy=0x7f9ea415f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9e90003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e90004350 0x7f9e90004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 0x7f9e90005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40c9150 con 0x7f9e900039a0 2023-11-08T19:26:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9000ba60 con 0x7f9e90004350 2023-11-08T19:26:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.721+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9000bc20 con 0x7f9ea415bff0 2023-11-08T19:26:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.722+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9e90003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.722+0000 7f9ea9cab640 1 -- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 msgr2=0x7f9e90005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.722+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea415bff0 0x7f9e90005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.722+0000 7f9ea9cab640 1 -- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e90004350 msgr2=0x7f9e90004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:12.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.722+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9e90004350 0x7f9e90004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.722+0000 7f9ea9cab640 1 -- 172.21.15.119:0/3979962426 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90003520 con 0x7f9e900039a0 2023-11-08T19:26:12.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.723+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9e90003d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001860 tx=0x7f9e900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcbded313f4aba32 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.723+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0033630 con 0x7f9e900039a0 2023-11-08T19:26:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.723+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea00337b0 con 0x7f9e900039a0 2023-11-08T19:26:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.723+0000 7f9ea8ca9640 1 -- 172.21.15.119:0/3979962426 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0033b60 con 0x7f9e900039a0 2023-11-08T19:26:12.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eabf0f640 1 -- 172.21.15.119:0/3979962426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 msgr2=0x7f9e90003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/3979962426 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9e90003d80 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0001860 tx=0x7f9e900057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eaa4ac640 1 -- 172.21.15.119:0/3979962426 reap_dead start 2023-11-08T19:26:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eabf0f640 1 -- 172.21.15.119:0/3979962426 shutdown_connections 2023-11-08T19:26:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eabf0f640 1 -- 172.21.15.119:0/3979962426 >> 172.21.15.119:0/3979962426 conn(0x7f9ea40cb610 msgr2=0x7f9ea40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eabf0f640 1 -- 172.21.15.119:0/3979962426 shutdown_connections 2023-11-08T19:26:12.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.724+0000 7f9eabf0f640 1 -- 172.21.15.119:0/3979962426 wait complete. 2023-11-08T19:26:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.726+0000 7f9eabf0f640 1 Processor -- start 2023-11-08T19:26:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.726+0000 7f9eabf0f640 1 -- start start 2023-11-08T19:26:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9eabf0f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9ea40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9eabf0f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ea40c7cc0 0x7f9ea40aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9eabf0f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea4140ba0 0x7f9ea4141020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9eabf0f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea4141560 con 0x7f9e900039a0 2023-11-08T19:26:12.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9eabf0f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40cdcd0 con 0x7f9ea40c7cc0 2023-11-08T19:26:12.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9eabf0f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40b2270 con 0x7f9ea4140ba0 2023-11-08T19:26:12.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9ea40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9ea40cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44810/0 (socket says 172.21.15.119:44810) 2023-11-08T19:26:12.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 -- 172.21.15.119:0/1954634343 learned_addr learned my addr 172.21.15.119:0/1954634343 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:12.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 -- 172.21.15.119:0/1954634343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea4140ba0 msgr2=0x7f9ea4141020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea4140ba0 0x7f9ea4141020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 -- 172.21.15.119:0/1954634343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ea40c7cc0 msgr2=0x7f9ea40aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ea40c7cc0 0x7f9ea40aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.727+0000 7f9ea9cab640 1 -- 172.21.15.119:0/1954634343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea40cd550 con 0x7f9e900039a0 2023-11-08T19:26:12.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.728+0000 7f9ea9cab640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9ea40cc6d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0008fd0 tx=0x7f9ea0001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.728+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007050 con 0x7f9e900039a0 2023-11-08T19:26:12.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.728+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea00071b0 con 0x7f9e900039a0 2023-11-08T19:26:12.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.728+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea0007540 con 0x7f9e900039a0 2023-11-08T19:26:12.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.728+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea40cccb0 con 0x7f9e900039a0 2023-11-08T19:26:12.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.728+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea40cd270 con 0x7f9e900039a0 2023-11-08T19:26:12.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.729+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea4072f90 con 0x7f9e900039a0 2023-11-08T19:26:12.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.730+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ea00067d0 con 0x7f9e900039a0 2023-11-08T19:26:12.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.731+0000 7f9e9a7fc640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e8806aea0 0x7f9e8806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:12.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.731+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f9ea00ab620 con 0x7f9e900039a0 2023-11-08T19:26:12.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.731+0000 7f9ea94aa640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e8806aea0 0x7f9e8806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:12.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.732+0000 7f9ea94aa640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e8806aea0 0x7f9e8806d360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9ea415e240 tx=0x7f9e94001450 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:12.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.735+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ea00762f0 con 0x7f9e900039a0 2023-11-08T19:26:12.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.925+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ea415ca20 con 0x7f9e900039a0 2023-11-08T19:26:12.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.926+0000 7f9e9a7fc640 1 -- 172.21.15.119:0/1954634343 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9ea0079910 con 0x7f9e900039a0 2023-11-08T19:26:12.927 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e8806aea0 msgr2=0x7f9e8806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e8806aea0 0x7f9e8806d360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9ea415e240 tx=0x7f9e94001450 comp rx=0 tx=0).stop 2023-11-08T19:26:12.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 msgr2=0x7f9ea40cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9ea40cc6d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0008fd0 tx=0x7f9ea0001230 comp rx=0 tx=0).stop 2023-11-08T19:26:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 shutdown_connections 2023-11-08T19:26:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e8806aea0 0x7f9e8806d360 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ea4140ba0 0x7f9ea4141020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ea40c7cc0 0x7f9ea40aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 --2- 172.21.15.119:0/1954634343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9e900039a0 0x7f9ea40cc6d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.928+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 >> 172.21.15.119:0/1954634343 conn(0x7f9ea40cb610 msgr2=0x7f9ea406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.929+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 shutdown_connections 2023-11-08T19:26:12.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:12.929+0000 7f9eabf0f640 1 -- 172.21.15.119:0/1954634343 wait complete. 2023-11-08T19:26:12.948 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.149+0000 7fcc955ae640 1 Processor -- start 2023-11-08T19:26:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.149+0000 7fcc955ae640 1 -- start start 2023-11-08T19:26:13.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc955ae640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc90160530 0x7fcc900cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc955ae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc955ae640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc900d0830 con 0x7fcc90160910 2023-11-08T19:26:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc955ae640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc900b27c0 con 0x7fcc90160530 2023-11-08T19:26:13.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc955ae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900b2aa0 con 0x7fcc900ce5b0 2023-11-08T19:26:13.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc8ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc90160530 0x7fcc900cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc8ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc90160530 0x7fcc900cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44826/0 (socket says 172.21.15.119:44826) 2023-11-08T19:26:13.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.151+0000 7fcc8ffff640 1 -- 172.21.15.119:0/3572190993 learned_addr learned my addr 172.21.15.119:0/3572190993 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:13.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.152+0000 7fcc94dad640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.152+0000 7fcc8ffff640 1 -- 172.21.15.119:0/3572190993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 msgr2=0x7fcc900b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:13.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.152+0000 7fcc8ffff640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.152+0000 7fcc8ffff640 1 -- 172.21.15.119:0/3572190993 >> v1:172.21.15.188:6789/0 conn(0x7fcc90160910 legacy=0x7fcc900cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:13.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.152+0000 7fcc8ffff640 1 -- 172.21.15.119:0/3572190993 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc900d1640 con 0x7fcc90160530 2023-11-08T19:26:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.152+0000 7fcc94dad640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.153+0000 7fcc8ffff640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160530 0x7fcc900cd960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009850 tx=0x7fcc8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecbd654559aebd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc8f7fe640 1 -- 172.21.15.119:0/3572190993 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004230 con 0x7fcc90160530 2023-11-08T19:26:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc8f7fe640 1 -- 172.21.15.119:0/3572190993 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc80004390 con 0x7fcc90160530 2023-11-08T19:26:13.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc8f7fe640 1 -- 172.21.15.119:0/3572190993 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc80031830 con 0x7fcc90160530 2023-11-08T19:26:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 -- 172.21.15.119:0/3572190993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160530 msgr2=0x7fcc900cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160530 0x7fcc900cd960 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009850 tx=0x7fcc8002b8d0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 -- 172.21.15.119:0/3572190993 shutdown_connections 2023-11-08T19:26:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 --2- 172.21.15.119:0/3572190993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160530 0x7fcc900cd960 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 -- 172.21.15.119:0/3572190993 >> 172.21.15.119:0/3572190993 conn(0x7fcc900d2ec0 msgr2=0x7fcc900d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 -- 172.21.15.119:0/3572190993 shutdown_connections 2023-11-08T19:26:13.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.154+0000 7fcc955ae640 1 -- 172.21.15.119:0/3572190993 wait complete. 2023-11-08T19:26:13.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.156+0000 7fcc955ae640 1 Processor -- start 2023-11-08T19:26:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.156+0000 7fcc955ae640 1 -- start start 2023-11-08T19:26:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.157+0000 7fcc955ae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.157+0000 7fcc955ae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc90160530 0x7fcc900b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.157+0000 7fcc955ae640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 0x7fcc9006b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.157+0000 7fcc955ae640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900cced0 con 0x7fcc90160910 2023-11-08T19:26:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.157+0000 7fcc955ae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900b5ab0 con 0x7fcc90160530 2023-11-08T19:26:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.157+0000 7fcc955ae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc900ca0a0 con 0x7fcc900ce5b0 2023-11-08T19:26:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 0x7fcc9006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 0x7fcc9006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44830/0 (socket says 172.21.15.119:44830) 2023-11-08T19:26:13.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 -- 172.21.15.119:0/4024128498 learned_addr learned my addr 172.21.15.119:0/4024128498 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc87fff640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc90160530 0x7fcc900b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc8ffff640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 -- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 msgr2=0x7fcc900b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 -- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc90160530 msgr2=0x7fcc900b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc90160530 0x7fcc900b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 -- 172.21.15.119:0/4024128498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc80009460 con 0x7fcc90160910 2023-11-08T19:26:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc8ffff640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.158+0000 7fcc94dad640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 0x7fcc9006b900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800cea0 tx=0x7fcc8800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.159+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88003b90 con 0x7fcc90160910 2023-11-08T19:26:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.159+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88003d10 con 0x7fcc90160910 2023-11-08T19:26:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.159+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88004260 con 0x7fcc90160910 2023-11-08T19:26:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.159+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc900ca380 con 0x7fcc90160910 2023-11-08T19:26:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.159+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc900b5fe0 con 0x7fcc90160910 2023-11-08T19:26:13.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.160+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc900b61a0 con 0x7fcc90160910 2023-11-08T19:26:13.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.161+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc880043c0 con 0x7fcc90160910 2023-11-08T19:26:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.162+0000 7fcc8d7fa640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc7006ae70 0x7fcc7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.162+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcc8808a3c0 con 0x7fcc90160910 2023-11-08T19:26:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.162+0000 7fcc8ffff640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc7006ae70 0x7fcc7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.163+0000 7fcc8ffff640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc7006ae70 0x7fcc7006d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc80035000 tx=0x7fcc80008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.166+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc88054ff0 con 0x7fcc90160910 2023-11-08T19:26:13.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.360+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc9004ece0 con 0x7fcc90160910 2023-11-08T19:26:13.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.360+0000 7fcc8d7fa640 1 -- 172.21.15.119:0/4024128498 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc88058610 con 0x7fcc90160910 2023-11-08T19:26:13.362 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc7006ae70 msgr2=0x7fcc7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc7006ae70 0x7fcc7006d330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc80035000 tx=0x7fcc80008030 comp rx=0 tx=0).stop 2023-11-08T19:26:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 msgr2=0x7fcc9006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 0x7fcc9006b900 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcc8800cea0 tx=0x7fcc8800cf90 comp rx=0 tx=0).stop 2023-11-08T19:26:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 shutdown_connections 2023-11-08T19:26:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc7006ae70 0x7fcc7006d330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc90160910 0x7fcc9006b900 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc90160530 0x7fcc900b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 --2- 172.21.15.119:0/4024128498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc900ce5b0 0x7fcc900b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 >> 172.21.15.119:0/4024128498 conn(0x7fcc900d2ec0 msgr2=0x7fcc900b4fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 shutdown_connections 2023-11-08T19:26:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.363+0000 7fcc955ae640 1 -- 172.21.15.119:0/4024128498 wait complete. 2023-11-08T19:26:13.383 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:13.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.599+0000 7fdf85165640 1 Processor -- start 2023-11-08T19:26:13.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.599+0000 7fdf85165640 1 -- start start 2023-11-08T19:26:13.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf85165640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf80144580 0x7fdf8015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf85165640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf8015f1d0 0x7fdf800c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf85165640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8015ea60 con 0x7fdf80144960 2023-11-08T19:26:13.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf85165640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf800c9150 con 0x7fdf8015f1d0 2023-11-08T19:26:13.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf85165640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf800c9360 con 0x7fdf80144580 2023-11-08T19:26:13.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf7f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf8015f1d0 0x7fdf800c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf7f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdf8015f1d0 0x7fdf800c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44842/0 (socket says 172.21.15.119:44842) 2023-11-08T19:26:13.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf7f59e640 1 -- 172.21.15.119:0/4093971463 learned_addr learned my addr 172.21.15.119:0/4093971463 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:13.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf77fff640 1 --1- 172.21.15.119:0/4093971463 >> v1:172.21.15.119:6789/0 conn(0x7fdf80144960 0x7fdf800cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58808/0 (socket says 172.21.15.119:58808) 2023-11-08T19:26:13.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.601+0000 7fdf7ed9d640 1 --2- 172.21.15.119:0/4093971463 >> v2:172.21.15.188:3300/0 conn(0x7fdf80144580 0x7fdf8015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.602+0000 7fdf7f59e640 1 -- 172.21.15.119:0/4093971463 >> v2:172.21.15.188:3300/0 conn(0x7fdf80144580 msgr2=0x7fdf8015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:13.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.602+0000 7fdf7f59e640 1 --2- 172.21.15.119:0/4093971463 >> v2:172.21.15.188:3300/0 conn(0x7fdf80144580 0x7fdf8015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.602+0000 7fdf7f59e640 1 -- 172.21.15.119:0/4093971463 >> v1:172.21.15.119:6789/0 conn(0x7fdf80144960 legacy=0x7fdf800cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.602+0000 7fdf7f59e640 1 -- 172.21.15.119:0/4093971463 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf800c9640 con 0x7fdf8015f1d0 2023-11-08T19:26:13.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.602+0000 7fdf7e59c640 1 -- 172.21.15.119:0/4093971463 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944949555 0 0) 0x7fdf8015ea60 con 0x7fdf80144960 2023-11-08T19:26:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.603+0000 7fdf7f59e640 1 --2- 172.21.15.119:0/4093971463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf8015f1d0 0x7fdf800c3a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf800ceab0 tx=0x7fdf7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98ff1e34c6621cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.603+0000 7fdf7e59c640 1 -- 172.21.15.119:0/4093971463 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7802ca80 con 0x7fdf8015f1d0 2023-11-08T19:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.603+0000 7fdf7e59c640 1 -- 172.21.15.119:0/4093971463 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7802cbe0 con 0x7fdf8015f1d0 2023-11-08T19:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.603+0000 7fdf7e59c640 1 -- 172.21.15.119:0/4093971463 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf780343f0 con 0x7fdf8015f1d0 2023-11-08T19:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 -- 172.21.15.119:0/4093971463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf8015f1d0 msgr2=0x7fdf800c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 --2- 172.21.15.119:0/4093971463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf8015f1d0 0x7fdf800c3a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fdf800ceab0 tx=0x7fdf7802bee0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 -- 172.21.15.119:0/4093971463 shutdown_connections 2023-11-08T19:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 --2- 172.21.15.119:0/4093971463 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf8015f1d0 0x7fdf800c3a50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 --2- 172.21.15.119:0/4093971463 >> v2:172.21.15.188:3300/0 conn(0x7fdf80144580 0x7fdf8015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 -- 172.21.15.119:0/4093971463 >> 172.21.15.119:0/4093971463 conn(0x7fdf800cb610 msgr2=0x7fdf800cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 -- 172.21.15.119:0/4093971463 shutdown_connections 2023-11-08T19:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.604+0000 7fdf85165640 1 -- 172.21.15.119:0/4093971463 wait complete. 2023-11-08T19:26:13.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.606+0000 7fdf85165640 1 Processor -- start 2023-11-08T19:26:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.606+0000 7fdf85165640 1 -- start start 2023-11-08T19:26:13.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf85165640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 0x7fdf801f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf85165640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80144960 0x7fdf8013b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf85165640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf8015f1d0 0x7fdf800c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf85165640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8013cb70 con 0x7fdf80144580 2023-11-08T19:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf85165640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf801f5260 con 0x7fdf80144960 2023-11-08T19:26:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf85165640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8013bf60 con 0x7fdf8015f1d0 2023-11-08T19:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 0x7fdf801f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 0x7fdf801f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44858/0 (socket says 172.21.15.119:44858) 2023-11-08T19:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 -- 172.21.15.119:0/2866816079 learned_addr learned my addr 172.21.15.119:0/2866816079 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 -- 172.21.15.119:0/2866816079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf8015f1d0 msgr2=0x7fdf800c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf8015f1d0 0x7fdf800c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 -- 172.21.15.119:0/2866816079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80144960 msgr2=0x7fdf8013b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80144960 0x7fdf8013b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.607+0000 7fdf7ed9d640 1 -- 172.21.15.119:0/2866816079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009b30 con 0x7fdf80144580 2023-11-08T19:26:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.608+0000 7fdf7ed9d640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 0x7fdf801f4d20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d620 tx=0x7fdf7000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.608+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000b620 con 0x7fdf80144580 2023-11-08T19:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.608+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7000b780 con 0x7fdf80144580 2023-11-08T19:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.608+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8013c240 con 0x7fdf80144580 2023-11-08T19:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.608+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000bb30 con 0x7fdf80144580 2023-11-08T19:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.608+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8013c800 con 0x7fdf80144580 2023-11-08T19:26:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.609+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8013c990 con 0x7fdf80144580 2023-11-08T19:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.610+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf7000bc90 con 0x7fdf80144580 2023-11-08T19:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.612+0000 7fdf777fe640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdf6406aea0 0x7fdf6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.612+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fdf70089e10 con 0x7fdf80144580 2023-11-08T19:26:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.612+0000 7fdf77fff640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdf6406aea0 0x7fdf6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:13.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.613+0000 7fdf77fff640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdf6406aea0 0x7fdf6406d360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf68007fd0 tx=0x7fdf68007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:13.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.616+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf70054b50 con 0x7fdf80144580 2023-11-08T19:26:13.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.807+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf80074430 con 0x7fdf80144580 2023-11-08T19:26:13.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.808+0000 7fdf777fe640 1 -- 172.21.15.119:0/2866816079 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf70058170 con 0x7fdf80144580 2023-11-08T19:26:13.809 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:13.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.810+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdf6406aea0 msgr2=0x7fdf6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.810+0000 7fdf85165640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdf6406aea0 0x7fdf6406d360 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdf68007fd0 tx=0x7fdf68007920 comp rx=0 tx=0).stop 2023-11-08T19:26:13.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.810+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 msgr2=0x7fdf801f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:13.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.810+0000 7fdf85165640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 0x7fdf801f4d20 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdf7000d620 tx=0x7fdf7000dae0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 shutdown_connections 2023-11-08T19:26:13.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdf6406aea0 0x7fdf6406d360 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf8015f1d0 0x7fdf800c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf80144960 0x7fdf8013b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 --2- 172.21.15.119:0/2866816079 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf80144580 0x7fdf801f4d20 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:13.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 >> 172.21.15.119:0/2866816079 conn(0x7fdf800cb610 msgr2=0x7fdf8006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:13.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 shutdown_connections 2023-11-08T19:26:13.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:13.811+0000 7fdf85165640 1 -- 172.21.15.119:0/2866816079 wait complete. 2023-11-08T19:26:13.832 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-08T19:26:14.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.046+0000 7fe6f8839640 1 Processor -- start 2023-11-08T19:26:14.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.047+0000 7fe6f8839640 1 -- start start 2023-11-08T19:26:14.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.048+0000 7fe6f8839640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6f415d4a0 0x7fe6f40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.048+0000 7fe6f8839640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f415df50 con 0x7fe6f40cfa00 2023-11-08T19:26:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.048+0000 7fe6f8839640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f40d1e60 con 0x7fe6f4161870 2023-11-08T19:26:14.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.048+0000 7fe6f8839640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40d2830 con 0x7fe6f415d4a0 2023-11-08T19:26:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f1d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe6f40cfa00 0x7fe6f4161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58814/0 (socket says 172.21.15.119:58814) 2023-11-08T19:26:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f1d9b640 1 -- 172.21.15.119:0/4214226090 learned_addr learned my addr 172.21.15.119:0/4214226090 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:14.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f259c640 1 --2- 172.21.15.119:0/4214226090 >> v2:172.21.15.188:3300/0 conn(0x7fe6f415d4a0 0x7fe6f40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748997249 0 0) 0x7fe6f415df50 con 0x7fe6f40cfa00 2023-11-08T19:26:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6e0003520 con 0x7fe6f40cfa00 2023-11-08T19:26:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891473111 0 0) 0x7fe6e0003520 con 0x7fe6f40cfa00 2023-11-08T19:26:14.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 >> v2:172.21.15.188:3300/0 conn(0x7fe6f415d4a0 msgr2=0x7fe6f40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 --2- 172.21.15.119:0/4214226090 >> v2:172.21.15.188:3300/0 conn(0x7fe6f415d4a0 0x7fe6f40d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 >> v1:172.21.15.188:6789/0 conn(0x7fe6f4161870 legacy=0x7fe6f40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f4161d40 con 0x7fe6f40cfa00 2023-11-08T19:26:14.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.049+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe6dc001670 con 0x7fe6f40cfa00 2023-11-08T19:26:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 >> v1:172.21.15.119:6789/0 conn(0x7fe6f40cfa00 legacy=0x7fe6f4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6e0003970 0x7fe6e0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00047c0 0x7fe6e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00052a0 0x7fe6e000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40d2830 con 0x7fe6e00052a0 2023-11-08T19:26:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000c150 con 0x7fe6e00047c0 2023-11-08T19:26:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e000c310 con 0x7fe6e0003970 2023-11-08T19:26:14.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f259c640 1 --2- 172.21.15.119:0/4214226090 >> v2:172.21.15.188:3300/0 conn(0x7fe6f415d4a0 0x7fe6f40d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00052a0 0x7fe6e000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f1d9b640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00047c0 0x7fe6e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f259c640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6e0003970 0x7fe6e0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6e0003970 msgr2=0x7fe6e0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6e0003970 0x7fe6e0003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00047c0 msgr2=0x7fe6e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00047c0 0x7fe6e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/4214226090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0003520 con 0x7fe6e00052a0 2023-11-08T19:26:14.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.050+0000 7fe6f1d9b640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00047c0 0x7fe6e0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.051+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/4214226090 reap_dead start 2023-11-08T19:26:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00052a0 0x7fe6e000b980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001620 tx=0x7fe6e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc30f6c974744791 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec005270 con 0x7fe6e00052a0 2023-11-08T19:26:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec02e070 con 0x7fe6e00052a0 2023-11-08T19:26:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f159a640 1 -- 172.21.15.119:0/4214226090 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec003610 con 0x7fe6e00052a0 2023-11-08T19:26:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f8839640 1 -- 172.21.15.119:0/4214226090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00052a0 msgr2=0x7fe6e000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f8839640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00052a0 0x7fe6e000b980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001620 tx=0x7fe6e00057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f8839640 1 -- 172.21.15.119:0/4214226090 shutdown_connections 2023-11-08T19:26:14.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f8839640 1 --2- 172.21.15.119:0/4214226090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6e00052a0 0x7fe6e000b980 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f8839640 1 -- 172.21.15.119:0/4214226090 >> 172.21.15.119:0/4214226090 conn(0x7fe6f406a9f0 msgr2=0x7fe6f406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.052+0000 7fe6f8839640 1 -- 172.21.15.119:0/4214226090 shutdown_connections 2023-11-08T19:26:14.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.053+0000 7fe6f8839640 1 -- 172.21.15.119:0/4214226090 wait complete. 2023-11-08T19:26:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.054+0000 7fe6f8839640 1 Processor -- start 2023-11-08T19:26:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.054+0000 7fe6f8839640 1 -- start start 2023-11-08T19:26:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.055+0000 7fe6f8839640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00052a0 0x7fe6f4158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.055+0000 7fe6f8839640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f40cccf0 0x7fe6f41f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.055+0000 7fe6f8839640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 0x7fe6f40cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.055+0000 7fe6f8839640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40ad2a0 con 0x7fe6f40cbae0 2023-11-08T19:26:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.055+0000 7fe6f8839640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4158d40 con 0x7fe6e00052a0 2023-11-08T19:26:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.055+0000 7fe6f8839640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f41f39b0 con 0x7fe6f40cccf0 2023-11-08T19:26:14.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 0x7fe6f40cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 0x7fe6f40cbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44876/0 (socket says 172.21.15.119:44876) 2023-11-08T19:26:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/532420995 learned_addr learned my addr 172.21.15.119:0/532420995 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f259c640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00052a0 0x7fe6f4158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f1d9b640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f40cccf0 0x7fe6f41f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f40cccf0 msgr2=0x7fe6f41f3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f40cccf0 0x7fe6f41f3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00052a0 msgr2=0x7fe6f4158800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00052a0 0x7fe6f4158800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.056+0000 7fe6f2d9d640 1 -- 172.21.15.119:0/532420995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f41f3c90 con 0x7fe6f40cbae0 2023-11-08T19:26:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.057+0000 7fe6f2d9d640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 0x7fe6f40cbf60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001740 tx=0x7fe6ec000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.057+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec0035f0 con 0x7fe6f40cbae0 2023-11-08T19:26:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.057+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ec003750 con 0x7fe6f40cbae0 2023-11-08T19:26:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.057+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ec003b00 con 0x7fe6f40cbae0 2023-11-08T19:26:14.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.057+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f41f3f70 con 0x7fe6f40cbae0 2023-11-08T19:26:14.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.057+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f40cb5c0 con 0x7fe6f40cbae0 2023-11-08T19:26:14.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.058+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f40d10f0 con 0x7fe6f40cbae0 2023-11-08T19:26:14.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.059+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6ec030030 con 0x7fe6f40cbae0 2023-11-08T19:26:14.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.060+0000 7fe6eaffd640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6d406ae70 0x7fe6d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.060+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fe6ec0aa790 con 0x7fe6f40cbae0 2023-11-08T19:26:14.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.060+0000 7fe6f259c640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6d406ae70 0x7fe6d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.061+0000 7fe6f259c640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6d406ae70 0x7fe6d406d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40aac20 tx=0x7fe6e400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.064+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6ec075460 con 0x7fe6f40cbae0 2023-11-08T19:26:14.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.254+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe6f40ce930 con 0x7fe6f40cbae0 2023-11-08T19:26:14.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.255+0000 7fe6eaffd640 1 -- 172.21.15.119:0/532420995 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe6ec078a80 con 0x7fe6f40cbae0 2023-11-08T19:26:14.257 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6d406ae70 msgr2=0x7fe6d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6d406ae70 0x7fe6d406d330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe6f40aac20 tx=0x7fe6e400b040 comp rx=0 tx=0).stop 2023-11-08T19:26:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 msgr2=0x7fe6f40cbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 0x7fe6f40cbf60 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec001740 tx=0x7fe6ec000fb0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 shutdown_connections 2023-11-08T19:26:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6d406ae70 0x7fe6d406d330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6f40cbae0 0x7fe6f40cbf60 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6f40cccf0 0x7fe6f41f3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 --2- 172.21.15.119:0/532420995 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6e00052a0 0x7fe6f4158800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 >> 172.21.15.119:0/532420995 conn(0x7fe6f406a9f0 msgr2=0x7fe6f40b8000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 shutdown_connections 2023-11-08T19:26:14.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.258+0000 7fe6f8839640 1 -- 172.21.15.119:0/532420995 wait complete. 2023-11-08T19:26:14.279 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:14.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.492+0000 7f364a400640 1 Processor -- start 2023-11-08T19:26:14.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.492+0000 7f364a400640 1 -- start start 2023-11-08T19:26:14.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364a400640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3644159d90 0x7f364415a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364a400640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f364415a6b0 0x7f36440c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364a400640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36440c9360 con 0x7f36440c9c60 2023-11-08T19:26:14.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364a400640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440ca490 con 0x7f364415a6b0 2023-11-08T19:26:14.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364a400640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440ca770 con 0x7f3644159d90 2023-11-08T19:26:14.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f36437fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f364415a6b0 0x7f36440c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364899d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f36440c9c60 0x7f364406c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58824/0 (socket says 172.21.15.119:58824) 2023-11-08T19:26:14.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f364899d640 1 -- 172.21.15.119:0/2425015360 learned_addr learned my addr 172.21.15.119:0/2425015360 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:14.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.494+0000 7f3643fff640 1 --2- 172.21.15.119:0/2425015360 >> v2:172.21.15.188:3300/0 conn(0x7f3644159d90 0x7f364415a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f3642ffd640 1 -- 172.21.15.119:0/2425015360 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825646932 0 0) 0x7f36440c9360 con 0x7f36440c9c60 2023-11-08T19:26:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f3642ffd640 1 -- 172.21.15.119:0/2425015360 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c003440 con 0x7f36440c9c60 2023-11-08T19:26:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f36437fe640 1 -- 172.21.15.119:0/2425015360 >> v2:172.21.15.188:3300/0 conn(0x7f3644159d90 msgr2=0x7f364415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f36437fe640 1 --2- 172.21.15.119:0/2425015360 >> v2:172.21.15.188:3300/0 conn(0x7f3644159d90 0x7f364415a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f36437fe640 1 -- 172.21.15.119:0/2425015360 >> v1:172.21.15.119:6789/0 conn(0x7f36440c9c60 legacy=0x7f364406c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f36437fe640 1 -- 172.21.15.119:0/2425015360 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364406bbb0 con 0x7f364415a6b0 2023-11-08T19:26:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.495+0000 7f3643fff640 1 --2- 172.21.15.119:0/2425015360 >> v2:172.21.15.188:3300/0 conn(0x7f3644159d90 0x7f364415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.496+0000 7f36437fe640 1 --2- 172.21.15.119:0/2425015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364415a6b0 0x7f36440c9720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f36440cfbd0 tx=0x7f363802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70241fbee6b1a57c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.496+0000 7f3642ffd640 1 -- 172.21.15.119:0/2425015360 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638030610 con 0x7f364415a6b0 2023-11-08T19:26:14.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.496+0000 7f3642ffd640 1 -- 172.21.15.119:0/2425015360 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638030770 con 0x7f364415a6b0 2023-11-08T19:26:14.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.496+0000 7f3642ffd640 1 -- 172.21.15.119:0/2425015360 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638030b00 con 0x7f364415a6b0 2023-11-08T19:26:14.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 -- 172.21.15.119:0/2425015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364415a6b0 msgr2=0x7f36440c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 --2- 172.21.15.119:0/2425015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364415a6b0 0x7f36440c9720 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f36440cfbd0 tx=0x7f363802b960 comp rx=0 tx=0).stop 2023-11-08T19:26:14.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 -- 172.21.15.119:0/2425015360 shutdown_connections 2023-11-08T19:26:14.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 --2- 172.21.15.119:0/2425015360 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f364415a6b0 0x7f36440c9720 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 --2- 172.21.15.119:0/2425015360 >> v2:172.21.15.188:3300/0 conn(0x7f3644159d90 0x7f364415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 -- 172.21.15.119:0/2425015360 >> 172.21.15.119:0/2425015360 conn(0x7f36440cb610 msgr2=0x7f36440cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 -- 172.21.15.119:0/2425015360 shutdown_connections 2023-11-08T19:26:14.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.497+0000 7f364a400640 1 -- 172.21.15.119:0/2425015360 wait complete. 2023-11-08T19:26:14.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.499+0000 7f364a400640 1 Processor -- start 2023-11-08T19:26:14.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.499+0000 7f364a400640 1 -- start start 2023-11-08T19:26:14.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f364a400640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 0x7f364414e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f364a400640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3644159d90 0x7f36440c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f364a400640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f364415a6b0 0x7f364415ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f364a400640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441ee350 con 0x7f36440c9c60 2023-11-08T19:26:14.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f364a400640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441ee4b0 con 0x7f364415a6b0 2023-11-08T19:26:14.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f364a400640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36440b3e20 con 0x7f3644159d90 2023-11-08T19:26:14.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f3643fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 0x7f364414e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f3643fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 0x7f364414e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44900/0 (socket says 172.21.15.119:44900) 2023-11-08T19:26:14.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.500+0000 7f3643fff640 1 -- 172.21.15.119:0/330473105 learned_addr learned my addr 172.21.15.119:0/330473105 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:14.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3643fff640 1 -- 172.21.15.119:0/330473105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3644159d90 msgr2=0x7f36440c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3643fff640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3644159d90 0x7f36440c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3643fff640 1 -- 172.21.15.119:0/330473105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f364415a6b0 msgr2=0x7f364415ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3643fff640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f364415a6b0 0x7f364415ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3643fff640 1 -- 172.21.15.119:0/330473105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009400 con 0x7f36440c9c60 2023-11-08T19:26:14.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3643fff640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 0x7f364414e7d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f363400e7e0 tx=0x7f363400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634004030 con 0x7f36440c9c60 2023-11-08T19:26:14.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634004190 con 0x7f36440c9c60 2023-11-08T19:26:14.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634004520 con 0x7f36440c9c60 2023-11-08T19:26:14.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f364406fab0 con 0x7f36440c9c60 2023-11-08T19:26:14.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.501+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36440b46c0 con 0x7f36440c9c60 2023-11-08T19:26:14.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.502+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3644074430 con 0x7f36440c9c60 2023-11-08T19:26:14.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.503+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f363400c6d0 con 0x7f36440c9c60 2023-11-08T19:26:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.504+0000 7f3640ff9640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f363006aea0 0x7f363006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.504+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f3634089690 con 0x7f36440c9c60 2023-11-08T19:26:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.504+0000 7f36437fe640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f363006aea0 0x7f363006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.505+0000 7f36437fe640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f363006aea0 0x7f363006d360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3638036000 tx=0x7f3638030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.508+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3634054350 con 0x7f36440c9c60 2023-11-08T19:26:14.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.741+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f364415a210 con 0x7f36440c9c60 2023-11-08T19:26:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.742+0000 7f3640ff9640 1 -- 172.21.15.119:0/330473105 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7f3634057970 con 0x7f36440c9c60 2023-11-08T19:26:14.744 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:14.744 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:26:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:26:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.744+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f363006aea0 msgr2=0x7f363006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.744+0000 7f364a400640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f363006aea0 0x7f363006d360 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3638036000 tx=0x7f3638030000 comp rx=0 tx=0).stop 2023-11-08T19:26:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 msgr2=0x7f364414e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 0x7f364414e7d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f363400e7e0 tx=0x7f363400eca0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 shutdown_connections 2023-11-08T19:26:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f363006aea0 0x7f363006d360 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f364415a6b0 0x7f364415ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3644159d90 0x7f36440c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 --2- 172.21.15.119:0/330473105 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f36440c9c60 0x7f364414e7d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 >> 172.21.15.119:0/330473105 conn(0x7f36440cb610 msgr2=0x7f36440ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 shutdown_connections 2023-11-08T19:26:14.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.745+0000 7f364a400640 1 -- 172.21.15.119:0/330473105 wait complete. 2023-11-08T19:26:14.764 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:14.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.966+0000 7f5c6c06b640 1 Processor -- start 2023-11-08T19:26:14.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.966+0000 7f5c6c06b640 1 -- start start 2023-11-08T19:26:14.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.967+0000 7f5c6c06b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c640cfa00 0x7f5c640cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.967+0000 7f5c6c06b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c64161160 0x7f5c641615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c6c06b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6415df70 con 0x7f5c6415d6c0 2023-11-08T19:26:14.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c6c06b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c64161b30 con 0x7f5c64161160 2023-11-08T19:26:14.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c6c06b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6415daa0 con 0x7f5c640cfa00 2023-11-08T19:26:14.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c6a608640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c64161160 0x7f5c641615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c6a608640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5c64161160 0x7f5c641615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:54552/0 (socket says 172.21.15.119:54552) 2023-11-08T19:26:14.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c6a608640 1 -- 172.21.15.119:0/1862284353 learned_addr learned my addr 172.21.15.119:0/1862284353 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:14.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.968+0000 7f5c69606640 1 --2- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c640cfa00 0x7f5c640cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.969+0000 7f5c6a608640 1 -- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c640cfa00 msgr2=0x7f5c640cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:14.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.969+0000 7f5c6a608640 1 --2- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c640cfa00 0x7f5c640cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.969+0000 7f5c6a608640 1 -- 172.21.15.119:0/1862284353 >> v1:172.21.15.188:6789/0 conn(0x7f5c6415d6c0 legacy=0x7f5c640d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.969+0000 7f5c6a608640 1 -- 172.21.15.119:0/1862284353 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c640d1e60 con 0x7f5c64161160 2023-11-08T19:26:14.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.969+0000 7f5c68e05640 1 -- 172.21.15.119:0/1862284353 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3586743861 0 0) 0x7f5c6415df70 con 0x7f5c6415d6c0 2023-11-08T19:26:14.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.971+0000 7f5c6a608640 1 --2- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c64161160 0x7f5c641615f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009e50 tx=0x7f5c5c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94963f7dd5ee3861 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.971+0000 7f5c68e05640 1 -- 172.21.15.119:0/1862284353 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c02ca70 con 0x7f5c64161160 2023-11-08T19:26:14.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.971+0000 7f5c68e05640 1 -- 172.21.15.119:0/1862284353 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c02cbd0 con 0x7f5c64161160 2023-11-08T19:26:14.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.971+0000 7f5c68e05640 1 -- 172.21.15.119:0/1862284353 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0343f0 con 0x7f5c64161160 2023-11-08T19:26:14.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 -- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c64161160 msgr2=0x7f5c641615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c64161160 0x7f5c641615f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c009e50 tx=0x7f5c5c02bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 -- 172.21.15.119:0/1862284353 shutdown_connections 2023-11-08T19:26:14.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c64161160 0x7f5c641615f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/1862284353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c640cfa00 0x7f5c640cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 -- 172.21.15.119:0/1862284353 >> 172.21.15.119:0/1862284353 conn(0x7f5c6406a9f0 msgr2=0x7f5c6406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:14.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 -- 172.21.15.119:0/1862284353 shutdown_connections 2023-11-08T19:26:14.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.972+0000 7f5c6c06b640 1 -- 172.21.15.119:0/1862284353 wait complete. 2023-11-08T19:26:14.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.974+0000 7f5c6c06b640 1 Processor -- start 2023-11-08T19:26:14.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.974+0000 7f5c6c06b640 1 -- start start 2023-11-08T19:26:14.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c6c06b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 0x7f5c640cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c6c06b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6415d6c0 0x7f5c640ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c6c06b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c64161160 0x7f5c640c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c6c06b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6415add0 con 0x7f5c640cfa00 2023-11-08T19:26:14.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c6c06b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640cb790 con 0x7f5c6415d6c0 2023-11-08T19:26:14.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c6c06b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c640cab80 con 0x7f5c64161160 2023-11-08T19:26:14.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 0x7f5c640cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 0x7f5c640cc7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44906/0 (socket says 172.21.15.119:44906) 2023-11-08T19:26:14.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 -- 172.21.15.119:0/2219858281 learned_addr learned my addr 172.21.15.119:0/2219858281 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 -- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c64161160 msgr2=0x7f5c640c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c64161160 0x7f5c640c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 -- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6415d6c0 msgr2=0x7f5c640ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69606640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6415d6c0 0x7f5c640ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6415d6c0 0x7f5c640ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.975+0000 7f5c69e07640 1 -- 172.21.15.119:0/2219858281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c009b30 con 0x7f5c640cfa00 2023-11-08T19:26:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.976+0000 7f5c69e07640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 0x7f5c640cc7d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c4b0 tx=0x7f5c5400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.976+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54005480 con 0x7f5c640cfa00 2023-11-08T19:26:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.976+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c540035f0 con 0x7f5c640cfa00 2023-11-08T19:26:14.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.976+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c640cae60 con 0x7f5c640cfa00 2023-11-08T19:26:14.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.976+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c540039a0 con 0x7f5c640cfa00 2023-11-08T19:26:14.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.976+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c640cb420 con 0x7f5c640cfa00 2023-11-08T19:26:14.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.977+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c64074430 con 0x7f5c640cfa00 2023-11-08T19:26:14.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.978+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c54003b00 con 0x7f5c640cfa00 2023-11-08T19:26:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.979+0000 7f5c627fc640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c5006ae70 0x7f5c5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.980+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7f5c54088ff0 con 0x7f5c640cfa00 2023-11-08T19:26:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.980+0000 7f5c69606640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c5006ae70 0x7f5c5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.981+0000 7f5c69606640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c5006ae70 0x7f5c5006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5c641ed450 tx=0x7f5c58008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:14.984+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c54053c80 con 0x7f5c640cfa00 2023-11-08T19:26:15.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.177+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c640d10f0 con 0x7f5c640cfa00 2023-11-08T19:26:15.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.179+0000 7f5c627fc640 1 -- 172.21.15.119:0/2219858281 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7f5c54017020 con 0x7f5c640cfa00 2023-11-08T19:26:15.183 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:15.183 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:15.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c5006ae70 msgr2=0x7f5c5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c5006ae70 0x7f5c5006d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5c641ed450 tx=0x7f5c58008040 comp rx=0 tx=0).stop 2023-11-08T19:26:15.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 msgr2=0x7f5c640cc7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 0x7f5c640cc7d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400c4b0 tx=0x7f5c5400c970 comp rx=0 tx=0).stop 2023-11-08T19:26:15.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 shutdown_connections 2023-11-08T19:26:15.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c5006ae70 0x7f5c5006d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c64161160 0x7f5c640c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c6415d6c0 0x7f5c640ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 --2- 172.21.15.119:0/2219858281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c640cfa00 0x7f5c640cc7d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.182+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 >> 172.21.15.119:0/2219858281 conn(0x7f5c6406a9f0 msgr2=0x7f5c640b6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:15.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.183+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 shutdown_connections 2023-11-08T19:26:15.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.183+0000 7f5c6c06b640 1 -- 172.21.15.119:0/2219858281 wait complete. 2023-11-08T19:26:15.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:26:15.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:15.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.414+0000 7fcd41d65640 1 Processor -- start 2023-11-08T19:26:15.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.414+0000 7fcd41d65640 1 -- start start 2023-11-08T19:26:15.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd41d65640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd3c142f80 0x7fcd3c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd41d65640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcd3c15d9f0 0x7fcd3c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd41d65640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3c0c4960 con 0x7fcd3c06cb40 2023-11-08T19:26:15.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd41d65640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c143a40 con 0x7fcd3c142f80 2023-11-08T19:26:15.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd41d65640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c15df60 con 0x7fcd3c15d9f0 2023-11-08T19:26:15.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd3b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd3c142f80 0x7fcd3c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd3b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd3c142f80 0x7fcd3c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44908/0 (socket says 172.21.15.119:44908) 2023-11-08T19:26:15.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd3b7fe640 1 -- 172.21.15.119:0/1506537589 learned_addr learned my addr 172.21.15.119:0/1506537589 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:15.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.416+0000 7fcd3affd640 1 --2- 172.21.15.119:0/1506537589 >> v2:172.21.15.188:3300/0 conn(0x7fcd3c15d9f0 0x7fcd3c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.417+0000 7fcd3b7fe640 1 -- 172.21.15.119:0/1506537589 >> v2:172.21.15.188:3300/0 conn(0x7fcd3c15d9f0 msgr2=0x7fcd3c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:15.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.417+0000 7fcd3b7fe640 1 --2- 172.21.15.119:0/1506537589 >> v2:172.21.15.188:3300/0 conn(0x7fcd3c15d9f0 0x7fcd3c0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.417+0000 7fcd3b7fe640 1 -- 172.21.15.119:0/1506537589 >> v1:172.21.15.188:6789/0 conn(0x7fcd3c06cb40 legacy=0x7fcd3c0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:15.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.417+0000 7fcd3b7fe640 1 -- 172.21.15.119:0/1506537589 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c15e240 con 0x7fcd3c142f80 2023-11-08T19:26:15.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.417+0000 7fcd3affd640 1 --2- 172.21.15.119:0/1506537589 >> v2:172.21.15.188:3300/0 conn(0x7fcd3c15d9f0 0x7fcd3c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:15.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.418+0000 7fcd3b7fe640 1 --2- 172.21.15.119:0/1506537589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0cc6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c06df40 tx=0x7fcd2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3505b540bf67617a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.418+0000 7fcd3a7fc640 1 -- 172.21.15.119:0/1506537589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c031870 con 0x7fcd3c142f80 2023-11-08T19:26:15.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.418+0000 7fcd3a7fc640 1 -- 172.21.15.119:0/1506537589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c0319d0 con 0x7fcd3c142f80 2023-11-08T19:26:15.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.418+0000 7fcd3a7fc640 1 -- 172.21.15.119:0/1506537589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c031d60 con 0x7fcd3c142f80 2023-11-08T19:26:15.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 -- 172.21.15.119:0/1506537589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 msgr2=0x7fcd3c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 --2- 172.21.15.119:0/1506537589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0cc6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c06df40 tx=0x7fcd2c02b910 comp rx=0 tx=0).stop 2023-11-08T19:26:15.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 -- 172.21.15.119:0/1506537589 shutdown_connections 2023-11-08T19:26:15.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 --2- 172.21.15.119:0/1506537589 >> v2:172.21.15.188:3300/0 conn(0x7fcd3c15d9f0 0x7fcd3c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 --2- 172.21.15.119:0/1506537589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0cc6d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 -- 172.21.15.119:0/1506537589 >> 172.21.15.119:0/1506537589 conn(0x7fcd3c0cb610 msgr2=0x7fcd3c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:15.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 -- 172.21.15.119:0/1506537589 shutdown_connections 2023-11-08T19:26:15.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.419+0000 7fcd41d65640 1 -- 172.21.15.119:0/1506537589 wait complete. 2023-11-08T19:26:15.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.421+0000 7fcd41d65640 1 Processor -- start 2023-11-08T19:26:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.421+0000 7fcd41d65640 1 -- start start 2023-11-08T19:26:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.422+0000 7fcd41d65640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c06cb40 0x7fcd3c1597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.422+0000 7fcd41d65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.422+0000 7fcd41d65640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd3c15d9f0 0x7fcd3c1f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.422+0000 7fcd41d65640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c159d00 con 0x7fcd3c142f80 2023-11-08T19:26:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.422+0000 7fcd41d65640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0c71a0 con 0x7fcd3c15d9f0 2023-11-08T19:26:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.422+0000 7fcd41d65640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c15a6a0 con 0x7fcd3c06cb40 2023-11-08T19:26:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44912/0 (socket says 172.21.15.119:44912) 2023-11-08T19:26:15.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 -- 172.21.15.119:0/2563488 learned_addr learned my addr 172.21.15.119:0/2563488 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3b7fe640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c06cb40 0x7fcd3c1597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 -- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c06cb40 msgr2=0x7fcd3c1597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c06cb40 0x7fcd3c1597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 -- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd3c15d9f0 msgr2=0x7fcd3c1f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd3c15d9f0 0x7fcd3c1f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 -- 172.21.15.119:0/2563488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2c0094d0 con 0x7fcd3c142f80 2023-11-08T19:26:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.423+0000 7fcd3affd640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0afec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c7f0 tx=0x7fcd2400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.424+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd24007a30 con 0x7fcd3c142f80 2023-11-08T19:26:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.424+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd24007b90 con 0x7fcd3c142f80 2023-11-08T19:26:15.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.424+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2400ce40 con 0x7fcd3c142f80 2023-11-08T19:26:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.424+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c15a980 con 0x7fcd3c142f80 2023-11-08T19:26:15.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.424+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c15d1a0 con 0x7fcd3c142f80 2023-11-08T19:26:15.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.425+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c0c7330 con 0x7fcd3c142f80 2023-11-08T19:26:15.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.426+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd240113f0 con 0x7fcd3c142f80 2023-11-08T19:26:15.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.427+0000 7fcd33fff640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1c06ae70 0x7fcd1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.427+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fcd24089bf0 con 0x7fcd3c142f80 2023-11-08T19:26:15.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.427+0000 7fcd3b7fe640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1c06ae70 0x7fcd1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.428+0000 7fcd3b7fe640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1c06ae70 0x7fcd1c06d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c036000 tx=0x7fcd2c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.431+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd240548b0 con 0x7fcd3c142f80 2023-11-08T19:26:15.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.660+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd3c074430 con 0x7fcd3c142f80 2023-11-08T19:26:15.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.661+0000 7fcd33fff640 1 -- 172.21.15.119:0/2563488 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fcd2401f090 con 0x7fcd3c142f80 2023-11-08T19:26:15.663 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:15.663 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:26:15.664 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:26:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1c06ae70 msgr2=0x7fcd1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1c06ae70 0x7fcd1c06d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c036000 tx=0x7fcd2c03c040 comp rx=0 tx=0).stop 2023-11-08T19:26:15.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 msgr2=0x7fcd3c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0afec0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fcd2400c7f0 tx=0x7fcd2400cbf0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 shutdown_connections 2023-11-08T19:26:15.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1c06ae70 0x7fcd1c06d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd3c15d9f0 0x7fcd3c1f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3c142f80 0x7fcd3c0afec0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 --2- 172.21.15.119:0/2563488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3c06cb40 0x7fcd3c1597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 >> 172.21.15.119:0/2563488 conn(0x7fcd3c0cb610 msgr2=0x7fcd3c0cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 shutdown_connections 2023-11-08T19:26:15.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.664+0000 7fcd41d65640 1 -- 172.21.15.119:0/2563488 wait complete. 2023-11-08T19:26:15.685 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:15.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.895+0000 7fa17ac5d640 1 Processor -- start 2023-11-08T19:26:15.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.895+0000 7fa17ac5d640 1 -- start start 2023-11-08T19:26:15.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.897+0000 7fa17ac5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa17406a500 0x7fa17406a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.897+0000 7fa17ac5d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1740cf3e0 con 0x7fa17406ebd0 2023-11-08T19:26:15.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.897+0000 7fa17ac5d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1740cd8f0 con 0x7fa174161900 2023-11-08T19:26:15.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.897+0000 7fa17ac5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1740cfbd0 con 0x7fa17406a500 2023-11-08T19:26:15.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.897+0000 7fa1789f9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa17406ebd0 0x7fa17406efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58834/0 (socket says 172.21.15.119:58834) 2023-11-08T19:26:15.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.897+0000 7fa1789f9640 1 -- 172.21.15.119:0/273677353 learned_addr learned my addr 172.21.15.119:0/273677353 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:15.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1791fa640 1 --2- 172.21.15.119:0/273677353 >> v2:172.21.15.188:3300/0 conn(0x7fa17406a500 0x7fa17406a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981970505 0 0) 0x7fa1740cf3e0 con 0x7fa17406ebd0 2023-11-08T19:26:15.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa164003520 con 0x7fa17406ebd0 2023-11-08T19:26:15.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 166194292 0 0) 0x7fa164003520 con 0x7fa17406ebd0 2023-11-08T19:26:15.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 >> v2:172.21.15.188:3300/0 conn(0x7fa17406a500 msgr2=0x7fa17406a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:15.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 --2- 172.21.15.119:0/273677353 >> v2:172.21.15.188:3300/0 conn(0x7fa17406a500 0x7fa17406a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 >> v1:172.21.15.188:6789/0 conn(0x7fa174161900 legacy=0x7fa174070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1740cdbd0 con 0x7fa17406ebd0 2023-11-08T19:26:15.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fa168001670 con 0x7fa17406ebd0 2023-11-08T19:26:15.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 >> v1:172.21.15.119:6789/0 conn(0x7fa17406ebd0 legacy=0x7fa17406efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1640039c0 0x7fa164003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640047c0 0x7fa164004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640052a0 0x7fa16400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400c190 con 0x7fa1640047c0 2023-11-08T19:26:15.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400c300 con 0x7fa1640039c0 2023-11-08T19:26:15.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400c4c0 con 0x7fa1640052a0 2023-11-08T19:26:15.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa1791fa640 1 --2- 172.21.15.119:0/273677353 >> v2:172.21.15.188:3300/0 conn(0x7fa17406a500 0x7fa17406a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.898+0000 7fa173fff640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640047c0 0x7fa164004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa1789f9640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1640039c0 0x7fa164003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa173fff640 1 -- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640052a0 msgr2=0x7fa16400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa173fff640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640052a0 0x7fa16400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa173fff640 1 -- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1640039c0 msgr2=0x7fa164003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:15.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa173fff640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1640039c0 0x7fa164003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa173fff640 1 -- 172.21.15.119:0/273677353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164003520 con 0x7fa1640047c0 2023-11-08T19:26:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.899+0000 7fa1791fa640 1 -- 172.21.15.119:0/273677353 reap_dead start 2023-11-08T19:26:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.900+0000 7fa173fff640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640047c0 0x7fa164004c40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa160001750 tx=0x7fa1640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03010cce680321d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.900+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600052d0 con 0x7fa1640047c0 2023-11-08T19:26:15.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.900+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16002e050 con 0x7fa1640047c0 2023-11-08T19:26:15.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.900+0000 7fa1737fe640 1 -- 172.21.15.119:0/273677353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160003830 con 0x7fa1640047c0 2023-11-08T19:26:15.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 -- 172.21.15.119:0/273677353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640047c0 msgr2=0x7fa164004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640047c0 0x7fa164004c40 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa160001750 tx=0x7fa1640057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 -- 172.21.15.119:0/273677353 shutdown_connections 2023-11-08T19:26:15.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/273677353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1640047c0 0x7fa164004c40 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 -- 172.21.15.119:0/273677353 >> 172.21.15.119:0/273677353 conn(0x7fa1740cfd20 msgr2=0x7fa1740d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:15.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 -- 172.21.15.119:0/273677353 shutdown_connections 2023-11-08T19:26:15.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.901+0000 7fa17ac5d640 1 -- 172.21.15.119:0/273677353 wait complete. 2023-11-08T19:26:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.903+0000 7fa17ac5d640 1 Processor -- start 2023-11-08T19:26:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.903+0000 7fa17ac5d640 1 -- start start 2023-11-08T19:26:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa17ac5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640047c0 0x7fa1740b5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa17ac5d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 0x7fa1741e8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa17ac5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1741e8ed0 0x7fa1740c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa17ac5d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740c6590 con 0x7fa1740b4a30 2023-11-08T19:26:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa17ac5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740c5fd0 con 0x7fa1741e8ed0 2023-11-08T19:26:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa17ac5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1740c62b0 con 0x7fa1640047c0 2023-11-08T19:26:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa173fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 0x7fa1741e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa173fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 0x7fa1741e8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44932/0 (socket says 172.21.15.119:44932) 2023-11-08T19:26:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.904+0000 7fa173fff640 1 -- 172.21.15.119:0/3630537888 learned_addr learned my addr 172.21.15.119:0/3630537888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1789f9640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640047c0 0x7fa1740b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1791fa640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1741e8ed0 0x7fa1740c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa173fff640 1 -- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640047c0 msgr2=0x7fa1740b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa173fff640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640047c0 0x7fa1740b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa173fff640 1 -- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1741e8ed0 msgr2=0x7fa1740c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa173fff640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1741e8ed0 0x7fa1740c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa173fff640 1 -- 172.21.15.119:0/3630537888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1741e7830 con 0x7fa1740b4a30 2023-11-08T19:26:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1789f9640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640047c0 0x7fa1740b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:15.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1791fa640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1741e8ed0 0x7fa1740c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa173fff640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 0x7fa1741e8310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa160008fd0 tx=0x7fa160001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600076c0 con 0x7fa1740b4a30 2023-11-08T19:26:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160007a30 con 0x7fa1740b4a30 2023-11-08T19:26:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160006c00 con 0x7fa1740b4a30 2023-11-08T19:26:15.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.905+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1741e7b10 con 0x7fa1740b4a30 2023-11-08T19:26:15.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.906+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1741e8850 con 0x7fa1740b4a30 2023-11-08T19:26:15.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.907+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1741e7690 con 0x7fa1740b4a30 2023-11-08T19:26:15.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.908+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1600035f0 con 0x7fa1740b4a30 2023-11-08T19:26:15.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.909+0000 7fa1717fa640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa15806aea0 0x7fa15806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:15.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.909+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fa1600ab9f0 con 0x7fa1740b4a30 2023-11-08T19:26:15.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.909+0000 7fa1789f9640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa15806aea0 0x7fa15806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:15.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.910+0000 7fa1789f9640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa15806aea0 0x7fa15806d360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa17404f700 tx=0x7fa168009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:15.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:15.913+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1741e7690 con 0x7fa1740b4a30 2023-11-08T19:26:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.143+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1741e7690 con 0x7fa1740b4a30 2023-11-08T19:26:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.144+0000 7fa1717fa640 1 -- 172.21.15.119:0/3630537888 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4918 (secure 0 0 0) 0x7fa1741e7690 con 0x7fa1740b4a30 2023-11-08T19:26:16.145 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:16.146 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4239,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3272829532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3272829532},{"type":"v1","addr":"172.21.15.119:6836","nonce":3272829532}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/978965675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":978965675},{"type":"v1","addr":"172.21.15.119:6837","nonce":978965675}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/3641027527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3641027527},{"type":"v1","addr":"172.21.15.188:6834","nonce":3641027527}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:24:58.104296+0000","modified":"2023-11-08T19:25:20.218164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4254},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4254":{"gid":4254,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/2629712656","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2629712656},{"type":"v1","addr":"172.21.15.188:6835","nonce":2629712656}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T19:26:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 9 2023-11-08T19:26:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa15806aea0 msgr2=0x7fa15806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa15806aea0 0x7fa15806d360 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa17404f700 tx=0x7fa168009040 comp rx=0 tx=0).stop 2023-11-08T19:26:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 msgr2=0x7fa1741e8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 0x7fa1741e8310 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa160008fd0 tx=0x7fa160001120 comp rx=0 tx=0).stop 2023-11-08T19:26:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 shutdown_connections 2023-11-08T19:26:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa15806aea0 0x7fa15806d360 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa1741e8ed0 0x7fa1740c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1740b4a30 0x7fa1741e8310 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 --2- 172.21.15.119:0/3630537888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1640047c0 0x7fa1740b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.146+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 >> 172.21.15.119:0/3630537888 conn(0x7fa1740cfd20 msgr2=0x7fa174072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.147+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 shutdown_connections 2023-11-08T19:26:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.147+0000 7fa17ac5d640 1 -- 172.21.15.119:0/3630537888 wait complete. 2023-11-08T19:26:16.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.374+0000 7fbc658b4640 1 Processor -- start 2023-11-08T19:26:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.374+0000 7fbc658b4640 1 -- start start 2023-11-08T19:26:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc658b4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc6015d9f0 0x7fbc600cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc658b4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc6006cb40 0x7fbc6006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc658b4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6006d4d0 con 0x7fbc60142f80 2023-11-08T19:26:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc658b4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc60143a40 con 0x7fbc6006cb40 2023-11-08T19:26:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc658b4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc600ca3e0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc5f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc6006cb40 0x7fbc6006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc5f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc6006cb40 0x7fbc6006cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44948/0 (socket says 172.21.15.119:44948) 2023-11-08T19:26:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.376+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/333022127 learned_addr learned my addr 172.21.15.119:0/333022127 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.377+0000 7fbc567fc640 1 --2- 172.21.15.119:0/333022127 >> v2:172.21.15.188:3300/0 conn(0x7fbc6015d9f0 0x7fbc600cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.377+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/333022127 >> v2:172.21.15.188:3300/0 conn(0x7fbc6015d9f0 msgr2=0x7fbc600cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.377+0000 7fbc5f7fe640 1 --2- 172.21.15.119:0/333022127 >> v2:172.21.15.188:3300/0 conn(0x7fbc6015d9f0 0x7fbc600cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.377+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/333022127 >> v1:172.21.15.188:6789/0 conn(0x7fbc60142f80 legacy=0x7fbc600c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.377+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/333022127 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc600c4960 con 0x7fbc6006cb40 2023-11-08T19:26:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.378+0000 7fbc5f7fe640 1 --2- 172.21.15.119:0/333022127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6006cb40 0x7fbc6006cf90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbc6006df40 tx=0x7fbc5802e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d92ba9481fc98a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.378+0000 7fbc5e7fc640 1 -- 172.21.15.119:0/333022127 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5802ec90 con 0x7fbc6006cb40 2023-11-08T19:26:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.378+0000 7fbc5e7fc640 1 -- 172.21.15.119:0/333022127 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5802edf0 con 0x7fbc6006cb40 2023-11-08T19:26:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.378+0000 7fbc5e7fc640 1 -- 172.21.15.119:0/333022127 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58036610 con 0x7fbc6006cb40 2023-11-08T19:26:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 -- 172.21.15.119:0/333022127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6006cb40 msgr2=0x7fbc6006cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 --2- 172.21.15.119:0/333022127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6006cb40 0x7fbc6006cf90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbc6006df40 tx=0x7fbc5802e0f0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 -- 172.21.15.119:0/333022127 shutdown_connections 2023-11-08T19:26:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 --2- 172.21.15.119:0/333022127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6006cb40 0x7fbc6006cf90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 --2- 172.21.15.119:0/333022127 >> v2:172.21.15.188:3300/0 conn(0x7fbc6015d9f0 0x7fbc600cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 -- 172.21.15.119:0/333022127 >> 172.21.15.119:0/333022127 conn(0x7fbc600cb610 msgr2=0x7fbc600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 -- 172.21.15.119:0/333022127 shutdown_connections 2023-11-08T19:26:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.379+0000 7fbc658b4640 1 -- 172.21.15.119:0/333022127 wait complete. 2023-11-08T19:26:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.381+0000 7fbc658b4640 1 Processor -- start 2023-11-08T19:26:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.381+0000 7fbc658b4640 1 -- start start 2023-11-08T19:26:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc658b4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc6006cb40 0x7fbc600b3190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc658b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc60142f80 0x7fbc600b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc658b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 0x7fbc601f2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc658b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b48f0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc658b4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b3ce0 con 0x7fbc6006cb40 2023-11-08T19:26:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc658b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc600b3fc0 con 0x7fbc60142f80 2023-11-08T19:26:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc5f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 0x7fbc601f2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc5f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 0x7fbc601f2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44958/0 (socket says 172.21.15.119:44958) 2023-11-08T19:26:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/3848040880 learned_addr learned my addr 172.21.15.119:0/3848040880 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc567fc640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc60142f80 0x7fbc600b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.382+0000 7fbc5effd640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc6006cb40 0x7fbc600b3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc60142f80 msgr2=0x7fbc600b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5f7fe640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc60142f80 0x7fbc600b36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc6006cb40 msgr2=0x7fbc600b3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5f7fe640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc6006cb40 0x7fbc600b3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5f7fe640 1 -- 172.21.15.119:0/3848040880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009b30 con 0x7fbc6015d9f0 2023-11-08T19:26:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5f7fe640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 0x7fbc601f2f70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbc58038040 tx=0x7fbc58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5802ec90 con 0x7fbc6015d9f0 2023-11-08T19:26:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc58009510 con 0x7fbc6015d9f0 2023-11-08T19:26:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc600b42a0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5803f6b0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.383+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc600b1a10 con 0x7fbc6015d9f0 2023-11-08T19:26:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.385+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6015a4c0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.385+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc5803f810 con 0x7fbc6015d9f0 2023-11-08T19:26:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.386+0000 7fbc5cff9640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc4006ae70 0x7fbc4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.386+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fbc580ac290 con 0x7fbc6015d9f0 2023-11-08T19:26:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.387+0000 7fbc5effd640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc4006ae70 0x7fbc4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.387+0000 7fbc5effd640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc4006ae70 0x7fbc4006d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000a750 tx=0x7fbc50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.390+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc6015a4c0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.586+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6015a4c0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.589+0000 7fbc5cff9640 1 -- 172.21.15.119:0/3848040880 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+18781 (secure 0 0 0) 0x7fbc6015a4c0 con 0x7fbc6015d9f0 2023-11-08T19:26:16.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:16.591 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":27,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:25:40.601201+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:24:56.122826+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T19:24:57.101213+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-11-08T19:25:00.722259+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":5,"to":1}]},{"pgid":"3.c","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]},{"pgid":"4.c","mappings":[{"from":1,"to":2}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:16.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc4006ae70 msgr2=0x7fbc4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc4006ae70 0x7fbc4006d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000a750 tx=0x7fbc50008040 comp rx=0 tx=0).stop 2023-11-08T19:26:16.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 msgr2=0x7fbc601f2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 0x7fbc601f2f70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbc58038040 tx=0x7fbc58004a40 comp rx=0 tx=0).stop 2023-11-08T19:26:16.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 shutdown_connections 2023-11-08T19:26:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc4006ae70 0x7fbc4006d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc6015d9f0 0x7fbc601f2f70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc60142f80 0x7fbc600b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 --2- 172.21.15.119:0/3848040880 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc6006cb40 0x7fbc600b3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 >> 172.21.15.119:0/3848040880 conn(0x7fbc600cb610 msgr2=0x7fbc600c5460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 shutdown_connections 2023-11-08T19:26:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.591+0000 7fbc658b4640 1 -- 172.21.15.119:0/3848040880 wait complete. 2023-11-08T19:26:16.612 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.823+0000 7fd7a9753640 1 Processor -- start 2023-11-08T19:26:16.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.824+0000 7fd7a9753640 1 -- start start 2023-11-08T19:26:16.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.825+0000 7fd7a9753640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7a41447a0 0x7fd7a415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.825+0000 7fd7a9753640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 0x7fd7a406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a9753640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40c3a50 con 0x7fd7a40cb170 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a9753640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a4144b80 con 0x7fd7a41447a0 2023-11-08T19:26:16.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a9753640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c9150 con 0x7fd7a40cad30 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a37fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd7a40cb170 0x7fd7a406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58844/0 (socket says 172.21.15.119:58844) 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a37fe640 1 -- 172.21.15.119:0/3073953743 learned_addr learned my addr 172.21.15.119:0/3073953743 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/3073953743 >> v2:172.21.15.188:3300/0 conn(0x7fd7a41447a0 0x7fd7a415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd79a7fc640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 0x7fd7a406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722362064 0 0) 0x7fd7a40c3a50 con 0x7fd7a40cb170 2023-11-08T19:26:16.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.826+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd790003520 con 0x7fd7a40cb170 2023-11-08T19:26:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 713144912 0 0) 0x7fd790003520 con 0x7fd7a40cb170 2023-11-08T19:26:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 msgr2=0x7fd7a406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 0x7fd7a406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 >> v2:172.21.15.188:3300/0 conn(0x7fd7a41447a0 msgr2=0x7fd7a415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 --2- 172.21.15.119:0/3073953743 >> v2:172.21.15.188:3300/0 conn(0x7fd7a41447a0 0x7fd7a415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40c9410 con 0x7fd7a40cb170 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd79c001670 con 0x7fd7a40cb170 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 >> v1:172.21.15.119:6789/0 conn(0x7fd7a40cb170 legacy=0x7fd7a406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd790003970 0x7fd790003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7900047c0 0x7fd790004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 0x7fd790005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a4144b80 con 0x7fd790003970 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c9150 con 0x7fd7900047c0 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd79000b980 con 0x7fd7a40cad30 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.827+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd790003970 0x7fd790003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.828+0000 7fd7a2ffd640 1 -- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 msgr2=0x7fd790005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.828+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40cad30 0x7fd790005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.828+0000 7fd7a2ffd640 1 -- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7900047c0 msgr2=0x7fd790004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.828+0000 7fd79a7fc640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7900047c0 0x7fd790004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.828+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7900047c0 0x7fd790004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.828+0000 7fd7a2ffd640 1 -- 172.21.15.119:0/3073953743 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790003520 con 0x7fd790003970 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.829+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd790003970 0x7fd790003d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd794009ac0 tx=0x7fd7900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0177e134c50dbe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.829+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79402c710 con 0x7fd790003970 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.829+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79402c870 con 0x7fd790003970 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.829+0000 7fd7a27fc640 1 -- 172.21.15.119:0/3073953743 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd79402cc00 con 0x7fd790003970 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a9753640 1 -- 172.21.15.119:0/3073953743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd790003970 msgr2=0x7fd790003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a9753640 1 --2- 172.21.15.119:0/3073953743 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd790003970 0x7fd790003d90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd794009ac0 tx=0x7fd7900057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a37fe640 1 -- 172.21.15.119:0/3073953743 reap_dead start 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a9753640 1 -- 172.21.15.119:0/3073953743 shutdown_connections 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a9753640 1 -- 172.21.15.119:0/3073953743 >> 172.21.15.119:0/3073953743 conn(0x7fd7a40cb610 msgr2=0x7fd7a40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a9753640 1 -- 172.21.15.119:0/3073953743 shutdown_connections 2023-11-08T19:26:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.830+0000 7fd7a9753640 1 -- 172.21.15.119:0/3073953743 wait complete. 2023-11-08T19:26:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.831+0000 7fd7a9753640 1 Processor -- start 2023-11-08T19:26:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.832+0000 7fd7a9753640 1 -- start start 2023-11-08T19:26:16.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd7a9753640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd790003970 0x7fd7a40c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd7a9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 0x7fd7a413f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd7a9753640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40c5c60 0x7fd7a40c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd7a9753640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a413fef0 con 0x7fd7a413f530 2023-11-08T19:26:16.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd7a9753640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40a96d0 con 0x7fd790003970 2023-11-08T19:26:16.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd7a9753640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40c6a20 con 0x7fd7a40c5c60 2023-11-08T19:26:16.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 0x7fd7a413f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 0x7fd7a413f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44970/0 (socket says 172.21.15.119:44970) 2023-11-08T19:26:16.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 -- 172.21.15.119:0/124690494 learned_addr learned my addr 172.21.15.119:0/124690494 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:16.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 -- 172.21.15.119:0/124690494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40c5c60 msgr2=0x7fd7a40c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40c5c60 0x7fd7a40c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 -- 172.21.15.119:0/124690494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd790003970 msgr2=0x7fd7a40c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd790003970 0x7fd7a40c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.833+0000 7fd79a7fc640 1 -- 172.21.15.119:0/124690494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40c6d00 con 0x7fd7a413f530 2023-11-08T19:26:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.834+0000 7fd79a7fc640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 0x7fd7a413f9b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00f350 tx=0x7fd78c00f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.834+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c017600 con 0x7fd7a413f530 2023-11-08T19:26:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.834+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78c011040 con 0x7fd7a413f530 2023-11-08T19:26:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.834+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78c0179c0 con 0x7fd7a413f530 2023-11-08T19:26:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.834+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a40c6fe0 con 0x7fd7a413f530 2023-11-08T19:26:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.834+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a40c75a0 con 0x7fd7a413f530 2023-11-08T19:26:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.835+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a4074430 con 0x7fd7a413f530 2023-11-08T19:26:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.836+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd78c017b20 con 0x7fd7a413f530 2023-11-08T19:26:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.837+0000 7fd79bfff640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406ae70 0x7fd78406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.837+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6692+0+0 (secure 0 0 0) 0x7fd78c08a110 con 0x7fd7a413f530 2023-11-08T19:26:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.837+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406ae70 0x7fd78406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.838+0000 7fd7a2ffd640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406ae70 0x7fd78406d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd794036000 tx=0x7fd79402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:16.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:16.841+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd78c054e90 con 0x7fd7a413f530 2023-11-08T19:26:17.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.071+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd7a406caa0 con 0x7fd7a413f530 2023-11-08T19:26:17.073 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-11-08T19:26:17.072+0000 7f8fd7f7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:26:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.076+0000 7fd79bfff640 1 -- 172.21.15.119:0/124690494 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd78c01c090 con 0x7fd7a413f530 2023-11-08T19:26:17.078 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:26:17.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406ae70 msgr2=0x7fd78406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406ae70 0x7fd78406d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd794036000 tx=0x7fd79402e000 comp rx=0 tx=0).stop 2023-11-08T19:26:17.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 msgr2=0x7fd7a413f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 0x7fd7a413f9b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fd78c00f350 tx=0x7fd78c00f810 comp rx=0 tx=0).stop 2023-11-08T19:26:17.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 shutdown_connections 2023-11-08T19:26:17.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406ae70 0x7fd78406d330 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7a40c5c60 0x7fd7a40c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a413f530 0x7fd7a413f9b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 --2- 172.21.15.119:0/124690494 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd790003970 0x7fd7a40c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 >> 172.21.15.119:0/124690494 conn(0x7fd7a40cb610 msgr2=0x7fd7a4143550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 shutdown_connections 2023-11-08T19:26:17.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.079+0000 7fd7a9753640 1 -- 172.21.15.119:0/124690494 wait complete. 2023-11-08T19:26:17.102 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:26:17.119 INFO:tasks.ceph.mds.b.smithi188.stderr: -5493> 2023-11-08T19:25:04.147+0000 7f74f2178640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:26:17.201 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:26:17.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.316+0000 7f0957478640 1 Processor -- start 2023-11-08T19:26:17.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.316+0000 7f0957478640 1 -- start start 2023-11-08T19:26:17.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0957478640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f095015aea0 0x7f09500c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0957478640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 0x7f09500b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0957478640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095006be20 con 0x7f095006b650 2023-11-08T19:26:17.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0957478640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095006c100 con 0x7f095015aea0 2023-11-08T19:26:17.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0957478640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095006c3e0 con 0x7f09500b18c0 2023-11-08T19:26:17.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0955c75640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f095006b650 0x7f095006cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58852/0 (socket says 172.21.15.119:58852) 2023-11-08T19:26:17.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.317+0000 7f0955c75640 1 -- 172.21.15.119:0/3269666658 learned_addr learned my addr 172.21.15.119:0/3269666658 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:17.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0956476640 1 --2- 172.21.15.119:0/3269666658 >> v2:172.21.15.188:3300/0 conn(0x7f095015aea0 0x7f09500c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0956c77640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 0x7f09500b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2328324977 0 0) 0x7f095006be20 con 0x7f095006b650 2023-11-08T19:26:17.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093c003520 con 0x7f095006b650 2023-11-08T19:26:17.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2004022323 0 0) 0x7f093c003520 con 0x7f095006b650 2023-11-08T19:26:17.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 msgr2=0x7f09500b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 0x7f09500b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 >> v2:172.21.15.188:3300/0 conn(0x7f095015aea0 msgr2=0x7f09500c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 --2- 172.21.15.119:0/3269666658 >> v2:172.21.15.188:3300/0 conn(0x7f095015aea0 0x7f09500c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095006c6c0 con 0x7f095006b650 2023-11-08T19:26:17.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f0940001670 con 0x7f095006b650 2023-11-08T19:26:17.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 >> v1:172.21.15.119:6789/0 conn(0x7f095006b650 legacy=0x7f095006cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c003970 0x7f093c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 0x7f093c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f093c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095006c100 con 0x7f093c004d00 2023-11-08T19:26:17.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095006c3e0 con 0x7f093c003970 2023-11-08T19:26:17.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.318+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093c00b980 con 0x7f09500b18c0 2023-11-08T19:26:17.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956c77640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f093c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956476640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c003970 0x7f093c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956c77640 1 -- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 msgr2=0x7f093c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0955c75640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 0x7f093c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956c77640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09500b18c0 0x7f093c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956c77640 1 -- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c003970 msgr2=0x7f093c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:17.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956c77640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f093c003970 0x7f093c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.319+0000 7f0956c77640 1 -- 172.21.15.119:0/3269666658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f093c003520 con 0x7f093c004d00 2023-11-08T19:26:17.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.320+0000 7f0956c77640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f093c005180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0948009ac0 tx=0x7f093c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83798dd3b4b88c0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094802c6b0 con 0x7f093c004d00 2023-11-08T19:26:17.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094802c810 con 0x7f093c004d00 2023-11-08T19:26:17.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0955474640 1 -- 172.21.15.119:0/3269666658 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094802cba0 con 0x7f093c004d00 2023-11-08T19:26:17.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0957478640 1 -- 172.21.15.119:0/3269666658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 msgr2=0x7f093c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0957478640 1 --2- 172.21.15.119:0/3269666658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f093c005180 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f0948009ac0 tx=0x7f093c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0956c77640 1 -- 172.21.15.119:0/3269666658 reap_dead start 2023-11-08T19:26:17.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0957478640 1 -- 172.21.15.119:0/3269666658 shutdown_connections 2023-11-08T19:26:17.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.321+0000 7f0957478640 1 -- 172.21.15.119:0/3269666658 >> 172.21.15.119:0/3269666658 conn(0x7f09500b0560 msgr2=0x7f09500b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:17.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.322+0000 7f0957478640 1 -- 172.21.15.119:0/3269666658 shutdown_connections 2023-11-08T19:26:17.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.322+0000 7f0957478640 1 -- 172.21.15.119:0/3269666658 wait complete. 2023-11-08T19:26:17.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.323+0000 7f0957478640 1 Processor -- start 2023-11-08T19:26:17.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.323+0000 7f0957478640 1 -- start start 2023-11-08T19:26:17.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.324+0000 7f0957478640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f09500c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.324+0000 7f0957478640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0950157d00 0x7f09500c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.324+0000 7f0957478640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09501f0270 0x7f0950159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.324+0000 7f0957478640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c6120 con 0x7f093c004d00 2023-11-08T19:26:17.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.324+0000 7f0957478640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c6400 con 0x7f0950157d00 2023-11-08T19:26:17.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.324+0000 7f0957478640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09500c66e0 con 0x7f09501f0270 2023-11-08T19:26:17.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f09500c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f09500c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44984/0 (socket says 172.21.15.119:44984) 2023-11-08T19:26:17.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 -- 172.21.15.119:0/1167382730 learned_addr learned my addr 172.21.15.119:0/1167382730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:17.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 -- 172.21.15.119:0/1167382730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09501f0270 msgr2=0x7f0950159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09501f0270 0x7f0950159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 -- 172.21.15.119:0/1167382730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0950157d00 msgr2=0x7f09500c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:17.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0950157d00 0x7f09500c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 -- 172.21.15.119:0/1167382730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500c69c0 con 0x7f093c004d00 2023-11-08T19:26:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.325+0000 7f0956476640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f09500c5b40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f094c00f3d0 tx=0x7f094c00f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.326+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c016620 con 0x7f093c004d00 2023-11-08T19:26:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.326+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0167a0 con 0x7f093c004d00 2023-11-08T19:26:17.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.326+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c016b50 con 0x7f093c004d00 2023-11-08T19:26:17.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.326+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09500c6ca0 con 0x7f093c004d00 2023-11-08T19:26:17.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.326+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0950158180 con 0x7f093c004d00 2023-11-08T19:26:17.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.327+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095004ece0 con 0x7f093c004d00 2023-11-08T19:26:17.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.328+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f094c016cb0 con 0x7f093c004d00 2023-11-08T19:26:17.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.329+0000 7f0946ffd640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f093406ae70 0x7f093406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:17.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.329+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6778+0+0 (secure 0 0 0) 0x7f094c08aac0 con 0x7f093c004d00 2023-11-08T19:26:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.329+0000 7f0955c75640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f093406ae70 0x7f093406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.330+0000 7f0955c75640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f093406ae70 0x7f093406d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0940001200 tx=0x7f0940009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:17.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.333+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f094c055760 con 0x7f093c004d00 2023-11-08T19:26:17.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:17.561+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f09501e39b0 con 0x7f093c004d00 2023-11-08T19:26:18.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.079+0000 7f0946ffd640 1 -- 172.21.15.119:0/1167382730 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f094c058d80 con 0x7f093c004d00 2023-11-08T19:26:18.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f093406ae70 msgr2=0x7f093406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f093406ae70 0x7f093406d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0940001200 tx=0x7f0940009040 comp rx=0 tx=0).stop 2023-11-08T19:26:18.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 msgr2=0x7f09500c5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f09500c5b40 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f094c00f3d0 tx=0x7f094c00f890 comp rx=0 tx=0).stop 2023-11-08T19:26:18.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 shutdown_connections 2023-11-08T19:26:18.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f093406ae70 0x7f093406d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09501f0270 0x7f0950159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0950157d00 0x7f09500c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 --2- 172.21.15.119:0/1167382730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f093c004d00 0x7f09500c5b40 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 >> 172.21.15.119:0/1167382730 conn(0x7f09500b0560 msgr2=0x7f095006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:18.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 shutdown_connections 2023-11-08T19:26:18.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.084+0000 7f0957478640 1 -- 172.21.15.119:0/1167382730 wait complete. 2023-11-08T19:26:18.107 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:26:18.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.323+0000 7efcba5fb640 1 Processor -- start 2023-11-08T19:26:18.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.323+0000 7efcba5fb640 1 -- start start 2023-11-08T19:26:18.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.324+0000 7efcba5fb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efcb415d000 0x7efcb40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.324+0000 7efcba5fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcb406a830 0x7efcb4160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.324+0000 7efcba5fb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb406d220 con 0x7efcb40c3a50 2023-11-08T19:26:18.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.324+0000 7efcba5fb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40ccd50 con 0x7efcb406a830 2023-11-08T19:26:18.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.324+0000 7efcba5fb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40aca40 con 0x7efcb415d000 2023-11-08T19:26:18.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb8b98640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcb406a830 0x7efcb4160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcabfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efcb40c3a50 0x7efcb406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58854/0 (socket says 172.21.15.119:58854) 2023-11-08T19:26:18.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb8b98640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efcb406a830 0x7efcb4160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44988/0 (socket says 172.21.15.119:44988) 2023-11-08T19:26:18.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcabfff640 1 -- 172.21.15.119:0/2509844502 learned_addr learned my addr 172.21.15.119:0/2509844502 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:18.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb3fff640 1 --2- 172.21.15.119:0/2509844502 >> v2:172.21.15.188:3300/0 conn(0x7efcb415d000 0x7efcb40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb8b98640 1 -- 172.21.15.119:0/2509844502 >> v2:172.21.15.188:3300/0 conn(0x7efcb415d000 msgr2=0x7efcb40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:18.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb8b98640 1 --2- 172.21.15.119:0/2509844502 >> v2:172.21.15.188:3300/0 conn(0x7efcb415d000 0x7efcb40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb8b98640 1 -- 172.21.15.119:0/2509844502 >> v1:172.21.15.119:6789/0 conn(0x7efcb40c3a50 legacy=0x7efcb406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb8b98640 1 -- 172.21.15.119:0/2509844502 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40acd20 con 0x7efcb406a830 2023-11-08T19:26:18.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.325+0000 7efcb3fff640 1 --2- 172.21.15.119:0/2509844502 >> v2:172.21.15.188:3300/0 conn(0x7efcb415d000 0x7efcb40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:18.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.326+0000 7efcb8b98640 1 --2- 172.21.15.119:0/2509844502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb406a830 0x7efcb4160e90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efcb40722f0 tx=0x7efcac02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b5cc1cba002d41 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:18.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcb37fe640 1 -- 172.21.15.119:0/2509844502 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac02cbe0 con 0x7efcb406a830 2023-11-08T19:26:18.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcb37fe640 1 -- 172.21.15.119:0/2509844502 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac02cd40 con 0x7efcb406a830 2023-11-08T19:26:18.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcb37fe640 1 -- 172.21.15.119:0/2509844502 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac034660 con 0x7efcb406a830 2023-11-08T19:26:18.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcba5fb640 1 -- 172.21.15.119:0/2509844502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb406a830 msgr2=0x7efcb4160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcba5fb640 1 --2- 172.21.15.119:0/2509844502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb406a830 0x7efcb4160e90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7efcb40722f0 tx=0x7efcac02be30 comp rx=0 tx=0).stop 2023-11-08T19:26:18.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcba5fb640 1 -- 172.21.15.119:0/2509844502 shutdown_connections 2023-11-08T19:26:18.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcba5fb640 1 --2- 172.21.15.119:0/2509844502 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb406a830 0x7efcb4160e90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcba5fb640 1 --2- 172.21.15.119:0/2509844502 >> v2:172.21.15.188:3300/0 conn(0x7efcb415d000 0x7efcb40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.327+0000 7efcba5fb640 1 -- 172.21.15.119:0/2509844502 >> 172.21.15.119:0/2509844502 conn(0x7efcb40cb610 msgr2=0x7efcb40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:18.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.328+0000 7efcba5fb640 1 -- 172.21.15.119:0/2509844502 shutdown_connections 2023-11-08T19:26:18.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.328+0000 7efcba5fb640 1 -- 172.21.15.119:0/2509844502 wait complete. 2023-11-08T19:26:18.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.329+0000 7efcba5fb640 1 Processor -- start 2023-11-08T19:26:18.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.329+0000 7efcba5fb640 1 -- start start 2023-11-08T19:26:18.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.330+0000 7efcba5fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcb406a830 0x7efcb415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.330+0000 7efcba5fb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 0x7efcb414b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.330+0000 7efcba5fb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcb415d000 0x7efcb40b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.330+0000 7efcba5fb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40b0b00 con 0x7efcb40c3a50 2023-11-08T19:26:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.330+0000 7efcba5fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40af9c0 con 0x7efcb406a830 2023-11-08T19:26:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.330+0000 7efcba5fb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb40aedb0 con 0x7efcb415d000 2023-11-08T19:26:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 0x7efcb414b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 0x7efcb414b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44990/0 (socket says 172.21.15.119:44990) 2023-11-08T19:26:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 -- 172.21.15.119:0/556333317 learned_addr learned my addr 172.21.15.119:0/556333317 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:18.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcb3fff640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcb406a830 0x7efcb415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 -- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcb415d000 msgr2=0x7efcb40b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcb415d000 0x7efcb40b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 -- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcb406a830 msgr2=0x7efcb415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:18.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcb406a830 0x7efcb415fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 -- 172.21.15.119:0/556333317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac009a70 con 0x7efcb40c3a50 2023-11-08T19:26:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcabfff640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 0x7efcb414b4c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00c450 tx=0x7efc9c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c005ce0 con 0x7efcb40c3a50 2023-11-08T19:26:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c005e40 con 0x7efcb40c3a50 2023-11-08T19:26:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.331+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c0072b0 con 0x7efcb40c3a50 2023-11-08T19:26:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.332+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb40af090 con 0x7efcb40c3a50 2023-11-08T19:26:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.332+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb40af650 con 0x7efcb40c3a50 2023-11-08T19:26:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.333+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb40afb50 con 0x7efcb40c3a50 2023-11-08T19:26:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.334+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc9c007410 con 0x7efcb40c3a50 2023-11-08T19:26:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.335+0000 7efcb17fa640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efc9406aea0 0x7efc9406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.335+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6778+0+0 (secure 0 0 0) 0x7efc9c088f40 con 0x7efcb40c3a50 2023-11-08T19:26:18.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.335+0000 7efcb3fff640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efc9406aea0 0x7efc9406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:18.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.336+0000 7efcb3fff640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efc9406aea0 0x7efc9406d360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efca400b840 tx=0x7efca4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:18.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.339+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc9c053bc0 con 0x7efcb40c3a50 2023-11-08T19:26:18.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:18.528+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efcb4074430 con 0x7efcb40c3a50 2023-11-08T19:26:18.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:18.926 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:19.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.084+0000 7efcb17fa640 1 -- 172.21.15.119:0/556333317 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7efc9c017020 con 0x7efcb40c3a50 2023-11-08T19:26:19.086 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:26:19.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efc9406aea0 msgr2=0x7efc9406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efc9406aea0 0x7efc9406d360 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efca400b840 tx=0x7efca4008040 comp rx=0 tx=0).stop 2023-11-08T19:26:19.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 msgr2=0x7efcb414b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 0x7efcb414b4c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7efc9c00c450 tx=0x7efc9c00c910 comp rx=0 tx=0).stop 2023-11-08T19:26:19.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 shutdown_connections 2023-11-08T19:26:19.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efc9406aea0 0x7efc9406d360 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efcb415d000 0x7efcb40b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efcb40c3a50 0x7efcb414b4c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 --2- 172.21.15.119:0/556333317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efcb406a830 0x7efcb415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.088+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 >> 172.21.15.119:0/556333317 conn(0x7efcb40cb610 msgr2=0x7efcb415bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:19.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.089+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 shutdown_connections 2023-11-08T19:26:19.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.090+0000 7efcba5fb640 1 -- 172.21.15.119:0/556333317 wait complete. 2023-11-08T19:26:19.114 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:26:19.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.372+0000 7f609e7c0640 1 Processor -- start 2023-11-08T19:26:19.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.372+0000 7f609e7c0640 1 -- start start 2023-11-08T19:26:19.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f609e7c0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60980cba90 0x7f609815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f609e7c0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60980c43b0 con 0x7f6098144550 2023-11-08T19:26:19.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f609e7c0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f609815c340 con 0x7f609815bb10 2023-11-08T19:26:19.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f609e7c0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60980c3a80 con 0x7f60980cba90 2023-11-08T19:26:19.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f608ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60980cba90 0x7f609815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f6097fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6098144550 0x7f60980cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48112/0 (socket says 172.21.15.119:48112) 2023-11-08T19:26:19.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.374+0000 7f6097fff640 1 -- 172.21.15.119:0/2779769281 learned_addr learned my addr 172.21.15.119:0/2779769281 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:19.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.375+0000 7f608ffff640 1 -- 172.21.15.119:0/2779769281 >> v1:172.21.15.188:6789/0 conn(0x7f609815bb10 legacy=0x7f60980c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:19.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.375+0000 7f608ffff640 1 -- 172.21.15.119:0/2779769281 >> v1:172.21.15.119:6789/0 conn(0x7f6098144550 legacy=0x7f60980cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.375+0000 7f608ffff640 1 -- 172.21.15.119:0/2779769281 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980c3d60 con 0x7f60980cba90 2023-11-08T19:26:19.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.375+0000 7f60977fe640 1 -- 172.21.15.119:0/2779769281 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262233610 0 0) 0x7f60980c43b0 con 0x7f6098144550 2023-11-08T19:26:19.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.376+0000 7f608ffff640 1 --2- 172.21.15.119:0/2779769281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980cba90 0x7f609815b5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f60800097a0 tx=0x7f608002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b28c9fb793e6ce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.376+0000 7f60977fe640 1 -- 172.21.15.119:0/2779769281 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080031830 con 0x7f60980cba90 2023-11-08T19:26:19.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.376+0000 7f60977fe640 1 -- 172.21.15.119:0/2779769281 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6080031990 con 0x7f60980cba90 2023-11-08T19:26:19.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.376+0000 7f60977fe640 1 -- 172.21.15.119:0/2779769281 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080031d20 con 0x7f60980cba90 2023-11-08T19:26:19.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 -- 172.21.15.119:0/2779769281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980cba90 msgr2=0x7f609815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 --2- 172.21.15.119:0/2779769281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980cba90 0x7f609815b5d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f60800097a0 tx=0x7f608002b8d0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 -- 172.21.15.119:0/2779769281 shutdown_connections 2023-11-08T19:26:19.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 --2- 172.21.15.119:0/2779769281 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60980cba90 0x7f609815b5d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 -- 172.21.15.119:0/2779769281 >> 172.21.15.119:0/2779769281 conn(0x7f60980b0950 msgr2=0x7f60980b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:19.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 -- 172.21.15.119:0/2779769281 shutdown_connections 2023-11-08T19:26:19.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.377+0000 7f609e7c0640 1 -- 172.21.15.119:0/2779769281 wait complete. 2023-11-08T19:26:19.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.379+0000 7f609e7c0640 1 Processor -- start 2023-11-08T19:26:19.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.379+0000 7f609e7c0640 1 -- start start 2023-11-08T19:26:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f609e7c0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60980cba90 0x7f6098157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f609e7c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 0x7f60981f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f609e7c0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609815bb10 0x7f6098152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f609e7c0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f609812c990 con 0x7f6098144550 2023-11-08T19:26:19.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f609e7c0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60981f4a70 con 0x7f609815bb10 2023-11-08T19:26:19.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f609e7c0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6098157f60 con 0x7f60980cba90 2023-11-08T19:26:19.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f608ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 0x7f60981f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f608ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 0x7f60981f4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51628/0 (socket says 172.21.15.119:51628) 2023-11-08T19:26:19.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.380+0000 7f608ffff640 1 -- 172.21.15.119:0/3573904399 learned_addr learned my addr 172.21.15.119:0/3573904399 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:19.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f6097fff640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60980cba90 0x7f6098157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f609cd5d640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609815bb10 0x7f6098152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f608ffff640 1 -- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60980cba90 msgr2=0x7f6098157980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f608ffff640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60980cba90 0x7f6098157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f608ffff640 1 -- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609815bb10 msgr2=0x7f6098152a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:19.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f608ffff640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609815bb10 0x7f6098152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f608ffff640 1 -- 172.21.15.119:0/3573904399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6080009400 con 0x7f6098144550 2023-11-08T19:26:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f608ffff640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 0x7f60981f4530 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f60800097a0 tx=0x7f60800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608003aa60 con 0x7f6098144550 2023-11-08T19:26:19.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608003abc0 con 0x7f6098144550 2023-11-08T19:26:19.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60800443f0 con 0x7f6098144550 2023-11-08T19:26:19.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.381+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60980c49c0 con 0x7f6098144550 2023-11-08T19:26:19.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.382+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6098158520 con 0x7f6098144550 2023-11-08T19:26:19.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.383+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f609804ece0 con 0x7f6098144550 2023-11-08T19:26:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.384+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f608004d450 con 0x7f6098144550 2023-11-08T19:26:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.384+0000 7f60957fa640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f607806ae70 0x7f607806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:19.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.385+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6340+0+0 (secure 0 0 0) 0x7f60800abb60 con 0x7f6098144550 2023-11-08T19:26:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.385+0000 7f6097fff640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f607806ae70 0x7f607806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.385+0000 7f6097fff640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f607806ae70 0x7f607806d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6088007fd0 tx=0x7f6088009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:19.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.389+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60800768f0 con 0x7f6098144550 2023-11-08T19:26:19.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:19.578+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60980c4820 con 0x7f6098144550 2023-11-08T19:26:20.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.087+0000 7f60957fa640 1 -- 172.21.15.119:0/3573904399 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7f60980c4820 con 0x7f6098144550 2023-11-08T19:26:20.089 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-11-08T19:26:20.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f607806ae70 msgr2=0x7f607806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f607806ae70 0x7f607806d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6088007fd0 tx=0x7f6088009040 comp rx=0 tx=0).stop 2023-11-08T19:26:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 msgr2=0x7f60981f4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 0x7f60981f4530 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f60800097a0 tx=0x7f60800046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 shutdown_connections 2023-11-08T19:26:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f607806ae70 0x7f607806d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f609815bb10 0x7f6098152a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6098144550 0x7f60981f4530 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 --2- 172.21.15.119:0/3573904399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60980cba90 0x7f6098157980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.091+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 >> 172.21.15.119:0/3573904399 conn(0x7f60980b0950 msgr2=0x7f609806bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.092+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 shutdown_connections 2023-11-08T19:26:20.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.093+0000 7f609e7c0640 1 -- 172.21.15.119:0/3573904399 wait complete. 2023-11-08T19:26:20.126 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-08T19:26:20.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.315+0000 7f81cb297640 1 Processor -- start 2023-11-08T19:26:20.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.315+0000 7f81cb297640 1 -- start start 2023-11-08T19:26:20.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81cb297640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81c40cdd40 0x7f81c406ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81cb297640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c406f460 con 0x7f81c40b0320 2023-11-08T19:26:20.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81cb297640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c415d6f0 con 0x7f81c415d310 2023-11-08T19:26:20.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81cb297640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c406de80 con 0x7f81c40cdd40 2023-11-08T19:26:20.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81c8832640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81c40cdd40 0x7f81c406ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81c9834640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f81c40b0320 0x7f81c40cb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48114/0 (socket says 172.21.15.119:48114) 2023-11-08T19:26:20.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.317+0000 7f81c9834640 1 -- 172.21.15.119:0/4141932585 learned_addr learned my addr 172.21.15.119:0/4141932585 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:20.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.318+0000 7f81c8832640 1 -- 172.21.15.119:0/4141932585 >> v1:172.21.15.188:6789/0 conn(0x7f81c415d310 legacy=0x7f81c40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:20.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.318+0000 7f81c8832640 1 -- 172.21.15.119:0/4141932585 >> v1:172.21.15.119:6789/0 conn(0x7f81c40b0320 legacy=0x7f81c40cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.318+0000 7f81c8832640 1 -- 172.21.15.119:0/4141932585 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c406e160 con 0x7f81c40cdd40 2023-11-08T19:26:20.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.319+0000 7f81c8832640 1 --2- 172.21.15.119:0/4141932585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c40cdd40 0x7f81c406ef20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f81b40097a0 tx=0x7f81b402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef42422211024b62 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:20.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.319+0000 7f81bbfff640 1 -- 172.21.15.119:0/4141932585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4031830 con 0x7f81c40cdd40 2023-11-08T19:26:20.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.319+0000 7f81bbfff640 1 -- 172.21.15.119:0/4141932585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b4031990 con 0x7f81c40cdd40 2023-11-08T19:26:20.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.319+0000 7f81bbfff640 1 -- 172.21.15.119:0/4141932585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4031d20 con 0x7f81c40cdd40 2023-11-08T19:26:20.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 -- 172.21.15.119:0/4141932585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c40cdd40 msgr2=0x7f81c406ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 --2- 172.21.15.119:0/4141932585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c40cdd40 0x7f81c406ef20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f81b40097a0 tx=0x7f81b402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 -- 172.21.15.119:0/4141932585 shutdown_connections 2023-11-08T19:26:20.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 --2- 172.21.15.119:0/4141932585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c40cdd40 0x7f81c406ef20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 -- 172.21.15.119:0/4141932585 >> 172.21.15.119:0/4141932585 conn(0x7f81c40d0db0 msgr2=0x7f81c40d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:20.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 -- 172.21.15.119:0/4141932585 shutdown_connections 2023-11-08T19:26:20.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.320+0000 7f81cb297640 1 -- 172.21.15.119:0/4141932585 wait complete. 2023-11-08T19:26:20.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.321+0000 7f81cb297640 1 Processor -- start 2023-11-08T19:26:20.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.322+0000 7f81cb297640 1 -- start start 2023-11-08T19:26:20.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.322+0000 7f81cb297640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c40b0320 0x7f81c40b62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.322+0000 7f81cb297640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81c40cdd40 0x7f81c40c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81cb297640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 0x7f81c40c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81cb297640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40b4380 con 0x7f81c415d310 2023-11-08T19:26:20.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81cb297640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40b4ec0 con 0x7f81c40b0320 2023-11-08T19:26:20.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81cb297640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c40b4590 con 0x7f81c40cdd40 2023-11-08T19:26:20.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 0x7f81c40c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 0x7f81c40c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51646/0 (socket says 172.21.15.119:51646) 2023-11-08T19:26:20.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 -- 172.21.15.119:0/139483397 learned_addr learned my addr 172.21.15.119:0/139483397 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:20.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9033640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c40b0320 0x7f81c40b62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 -- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81c40cdd40 msgr2=0x7f81c40c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81c40cdd40 0x7f81c40c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 -- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c40b0320 msgr2=0x7f81c40b62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:20.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c40b0320 0x7f81c40b62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:20.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.323+0000 7f81c9834640 1 -- 172.21.15.119:0/139483397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b4009400 con 0x7f81c415d310 2023-11-08T19:26:20.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.324+0000 7f81c9834640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 0x7f81c40c6ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00cbc0 tx=0x7f81bc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:20.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.324+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00ad40 con 0x7f81c415d310 2023-11-08T19:26:20.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.324+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81bc012400 con 0x7f81c415d310 2023-11-08T19:26:20.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.324+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81bc00aea0 con 0x7f81c415d310 2023-11-08T19:26:20.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.324+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c40b4870 con 0x7f81c415d310 2023-11-08T19:26:20.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.324+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c40c75a0 con 0x7f81c415d310 2023-11-08T19:26:20.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.325+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c40c7790 con 0x7f81c415d310 2023-11-08T19:26:20.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.326+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81bc007070 con 0x7f81c415d310 2023-11-08T19:26:20.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.327+0000 7f81b9ffb640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a806aea0 0x7f81a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:20.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.327+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5781+0+0 (secure 0 0 0) 0x7f81bc0893d0 con 0x7f81c415d310 2023-11-08T19:26:20.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.327+0000 7f81c9033640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a806aea0 0x7f81a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:20.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.328+0000 7f81c9033640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a806aea0 0x7f81a806d360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f81c000b7f0 tx=0x7f81c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:20.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.332+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81bc054440 con 0x7f81c415d310 2023-11-08T19:26:20.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:20.518+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f81c406f460 con 0x7f81c415d310 2023-11-08T19:26:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.108+0000 7f81b9ffb640 1 -- 172.21.15.119:0/139483397 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f81c406f460 con 0x7f81c415d310 2023-11-08T19:26:21.110 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data_ec' removed 2023-11-08T19:26:21.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.112+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a806aea0 msgr2=0x7f81a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.112+0000 7f81cb297640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a806aea0 0x7f81a806d360 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f81c000b7f0 tx=0x7f81c0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:21.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.112+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 msgr2=0x7f81c40c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.112+0000 7f81cb297640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 0x7f81c40c6ed0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00cbc0 tx=0x7f81bc00a430 comp rx=0 tx=0).stop 2023-11-08T19:26:21.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 shutdown_connections 2023-11-08T19:26:21.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a806aea0 0x7f81a806d360 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c415d310 0x7f81c40c6ed0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81c40cdd40 0x7f81c40c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 --2- 172.21.15.119:0/139483397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c40b0320 0x7f81c40b62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 >> 172.21.15.119:0/139483397 conn(0x7f81c40d0db0 msgr2=0x7f81c40af210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 shutdown_connections 2023-11-08T19:26:21.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.113+0000 7f81cb297640 1 -- 172.21.15.119:0/139483397 wait complete. 2023-11-08T19:26:21.138 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:26:21.139 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:26:21.139 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:26:21.140 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:21.140 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:26:21.141 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:26:21.141 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:26:21.142 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:21.142 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:26:21.142 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:26:21.143 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:26:21.143 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:21.144 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:26:21.146 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:26:21.146 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:26:21.146 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:21.162 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:26:21.163 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:26:21.164 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:26:21.164 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:26:21.165 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:26:21.166 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:26:21.167 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:26:21.167 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:26:21.212 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:26:21.212 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:21.214 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:26:21.214 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:21.218 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:26:21.219 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:21.221 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:26:21.221 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:21.222 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:26:21.369 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:26:21.370 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:26:21.376 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:26:21.376 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:26:21.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.406+0000 7f26a92b0640 1 Processor -- start 2023-11-08T19:26:21.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.406+0000 7f26a92b0640 1 -- start start 2023-11-08T19:26:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.407+0000 7f26a92b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f26a4070280 0x7f26a40b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.407+0000 7f26a92b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.407+0000 7f26a92b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a40cb6e0 con 0x7f26a40c9af0 2023-11-08T19:26:21.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.407+0000 7f26a92b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a40aec60 con 0x7f26a4070280 2023-11-08T19:26:21.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.407+0000 7f26a92b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40cc840 con 0x7f26a40b2870 2023-11-08T19:26:21.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a2ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f26a40c9af0 0x7f26a40cc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48120/0 (socket says 172.21.15.119:48120) 2023-11-08T19:26:21.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a2ffd640 1 -- 172.21.15.119:0/205085927 learned_addr learned my addr 172.21.15.119:0/205085927 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a27fc640 1 --2- 172.21.15.119:0/205085927 >> v2:172.21.15.188:3300/0 conn(0x7f26a4070280 0x7f26a40b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a37fe640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a1ffb640 1 -- 172.21.15.119:0/205085927 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159076224 0 0) 0x7f26a40cb6e0 con 0x7f26a40c9af0 2023-11-08T19:26:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a1ffb640 1 -- 172.21.15.119:0/205085927 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f268c003520 con 0x7f26a40c9af0 2023-11-08T19:26:21.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a27fc640 1 -- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 msgr2=0x7f26a40c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:21.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a27fc640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a27fc640 1 -- 172.21.15.119:0/205085927 >> v1:172.21.15.119:6789/0 conn(0x7f26a40c9af0 legacy=0x7f26a40cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a27fc640 1 -- 172.21.15.119:0/205085927 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a40cad30 con 0x7f26a4070280 2023-11-08T19:26:21.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.408+0000 7f26a37fe640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.409+0000 7f26a27fc640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a40b2330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26980097a0 tx=0x7f269802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=733309bc6abe87c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a1ffb640 1 -- 172.21.15.119:0/205085927 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f269802d040 con 0x7f26a4070280 2023-11-08T19:26:21.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a1ffb640 1 -- 172.21.15.119:0/205085927 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2698031810 con 0x7f26a4070280 2023-11-08T19:26:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a1ffb640 1 -- 172.21.15.119:0/205085927 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2698031ba0 con 0x7f26a4070280 2023-11-08T19:26:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 -- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 msgr2=0x7f26a40b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a40b2330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f26980097a0 tx=0x7f269802b8b0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 -- 172.21.15.119:0/205085927 shutdown_connections 2023-11-08T19:26:21.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 --2- 172.21.15.119:0/205085927 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a40b2330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 -- 172.21.15.119:0/205085927 >> 172.21.15.119:0/205085927 conn(0x7f26a406c280 msgr2=0x7f26a406c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 -- 172.21.15.119:0/205085927 shutdown_connections 2023-11-08T19:26:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.410+0000 7f26a92b0640 1 -- 172.21.15.119:0/205085927 wait complete. 2023-11-08T19:26:21.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.412+0000 7f26a92b0640 1 Processor -- start 2023-11-08T19:26:21.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.412+0000 7f26a92b0640 1 -- start start 2023-11-08T19:26:21.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.413+0000 7f26a92b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a4155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.413+0000 7f26a92b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.413+0000 7f26a92b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a40c9af0 0x7f26a40b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.413+0000 7f26a92b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40c8d70 con 0x7f26a40c9af0 2023-11-08T19:26:21.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.413+0000 7f26a92b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40c8160 con 0x7f26a4070280 2023-11-08T19:26:21.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.413+0000 7f26a92b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a40c8440 con 0x7f26a40b2870 2023-11-08T19:26:21.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a40c9af0 0x7f26a40b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a4155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a40c9af0 0x7f26a40b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51688/0 (socket says 172.21.15.119:51688) 2023-11-08T19:26:21.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a27fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a37fe640 1 -- 172.21.15.119:0/1459564118 learned_addr learned my addr 172.21.15.119:0/1459564118 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:21.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 -- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 msgr2=0x7f26a40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 -- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a40c9af0 msgr2=0x7f26a40b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a40c9af0 0x7f26a40b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 -- 172.21.15.119:0/1459564118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f269c009a70 con 0x7f26a4070280 2023-11-08T19:26:21.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.414+0000 7f26a2ffd640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a4155700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f269400c430 tx=0x7f269400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.415+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2694005ce0 con 0x7f26a4070280 2023-11-08T19:26:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.415+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2694005e40 con 0x7f26a4070280 2023-11-08T19:26:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.415+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26940072d0 con 0x7f26a4070280 2023-11-08T19:26:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.415+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2698009400 con 0x7f26a4070280 2023-11-08T19:26:21.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.415+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a40c8a00 con 0x7f26a4070280 2023-11-08T19:26:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.416+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a4074430 con 0x7f26a4070280 2023-11-08T19:26:21.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.417+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2694007430 con 0x7f26a4070280 2023-11-08T19:26:21.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.418+0000 7f268b7fe640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f269006ae70 0x7f269006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:21.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.418+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5263+0+0 (secure 0 0 0) 0x7f2694088b70 con 0x7f26a4070280 2023-11-08T19:26:21.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.418+0000 7f26a27fc640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f269006ae70 0x7f269006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:21.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.419+0000 7f26a27fc640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f269006ae70 0x7f269006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2698036000 tx=0x7f269803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:21.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.422+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2694053e70 con 0x7f26a4070280 2023-11-08T19:26:21.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:21.623+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f26a40afaa0 con 0x7f26a4070280 2023-11-08T19:26:22.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.144+0000 7f268b7fe640 1 -- 172.21.15.119:0/1459564118 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v33) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2694017080 con 0x7f26a4070280 2023-11-08T19:26:22.146 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-11-08T19:26:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f269006ae70 msgr2=0x7f269006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f269006ae70 0x7f269006d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2698036000 tx=0x7f269803c040 comp rx=0 tx=0).stop 2023-11-08T19:26:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 msgr2=0x7f26a4155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a4155700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f269400c430 tx=0x7f269400c8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 shutdown_connections 2023-11-08T19:26:22.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f269006ae70 0x7f269006d330 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a40c9af0 0x7f26a40b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a40b2870 0x7f26a40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 --2- 172.21.15.119:0/1459564118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a4070280 0x7f26a4155700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 >> 172.21.15.119:0/1459564118 conn(0x7f26a406c280 msgr2=0x7f26a40c9ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.147+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 shutdown_connections 2023-11-08T19:26:22.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.148+0000 7f26a92b0640 1 -- 172.21.15.119:0/1459564118 wait complete. 2023-11-08T19:26:22.176 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:26:22.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.369+0000 7f2fddc00640 1 Processor -- start 2023-11-08T19:26:22.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.369+0000 7f2fddc00640 1 -- start start 2023-11-08T19:26:22.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.370+0000 7f2fddc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 0x7f2fd80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.370+0000 7f2fddc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2fd815d9f0 0x7f2fd80c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.370+0000 7f2fddc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fd80c4960 con 0x7f2fd806cb40 2023-11-08T19:26:22.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.370+0000 7f2fddc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fd8143a40 con 0x7f2fd815d9f0 2023-11-08T19:26:22.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.370+0000 7f2fddc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd815df60 con 0x7f2fd8142f80 2023-11-08T19:26:22.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.371+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 0x7f2fd80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.371+0000 7f2fd6ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2fd815d9f0 0x7f2fd80c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.371+0000 7f2fd77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 0x7f2fd80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:34516/0 (socket says 172.21.15.119:34516) 2023-11-08T19:26:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.371+0000 7f2fd77fe640 1 -- 172.21.15.119:0/2371914320 learned_addr learned my addr 172.21.15.119:0/2371914320 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.372+0000 7f2fd77fe640 1 -- 172.21.15.119:0/2371914320 >> v2:172.21.15.188:3300/0 conn(0x7f2fd815d9f0 msgr2=0x7f2fd80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.372+0000 7f2fd77fe640 1 --2- 172.21.15.119:0/2371914320 >> v2:172.21.15.188:3300/0 conn(0x7f2fd815d9f0 0x7f2fd80c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.372+0000 7f2fd77fe640 1 -- 172.21.15.119:0/2371914320 >> v1:172.21.15.188:6789/0 conn(0x7f2fd806cb40 legacy=0x7f2fd80c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.372+0000 7f2fd77fe640 1 -- 172.21.15.119:0/2371914320 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd815e240 con 0x7f2fd8142f80 2023-11-08T19:26:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.373+0000 7f2fd77fe640 1 --2- 172.21.15.119:0/2371914320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 0x7f2fd80cc6d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009e50 tx=0x7f2fc802bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4db6121ca32d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fd67fc640 1 -- 172.21.15.119:0/2371914320 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc802ca70 con 0x7f2fd8142f80 2023-11-08T19:26:22.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fd67fc640 1 -- 172.21.15.119:0/2371914320 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc802cbd0 con 0x7f2fd8142f80 2023-11-08T19:26:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fd67fc640 1 -- 172.21.15.119:0/2371914320 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc80343f0 con 0x7f2fd8142f80 2023-11-08T19:26:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fddc00640 1 -- 172.21.15.119:0/2371914320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 msgr2=0x7f2fd80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fddc00640 1 --2- 172.21.15.119:0/2371914320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 0x7f2fd80cc6d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8009e50 tx=0x7f2fc802bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fddc00640 1 -- 172.21.15.119:0/2371914320 shutdown_connections 2023-11-08T19:26:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fddc00640 1 --2- 172.21.15.119:0/2371914320 >> v2:172.21.15.188:3300/0 conn(0x7f2fd815d9f0 0x7f2fd80c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fddc00640 1 --2- 172.21.15.119:0/2371914320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd8142f80 0x7f2fd80cc6d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.374+0000 7f2fddc00640 1 -- 172.21.15.119:0/2371914320 >> 172.21.15.119:0/2371914320 conn(0x7f2fd80cb610 msgr2=0x7f2fd80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.375+0000 7f2fddc00640 1 -- 172.21.15.119:0/2371914320 shutdown_connections 2023-11-08T19:26:22.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.375+0000 7f2fddc00640 1 -- 172.21.15.119:0/2371914320 wait complete. 2023-11-08T19:26:22.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.376+0000 7f2fddc00640 1 Processor -- start 2023-11-08T19:26:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.377+0000 7f2fddc00640 1 -- start start 2023-11-08T19:26:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.377+0000 7f2fddc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd806cb40 0x7f2fd80711e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.377+0000 7f2fddc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 0x7f2fd8140760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.377+0000 7f2fddc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd815d9f0 0x7f2fd8140ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fddc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd80b0280 con 0x7f2fd8142f80 2023-11-08T19:26:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fddc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd80c6ac0 con 0x7f2fd806cb40 2023-11-08T19:26:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fddc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd80c5f40 con 0x7f2fd815d9f0 2023-11-08T19:26:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 0x7f2fd8140760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 0x7f2fd8140760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51698/0 (socket says 172.21.15.119:51698) 2023-11-08T19:26:22.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 -- 172.21.15.119:0/3636497033 learned_addr learned my addr 172.21.15.119:0/3636497033 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:22.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd77fe640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd806cb40 0x7f2fd80711e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd7fff640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd815d9f0 0x7f2fd8140ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 -- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd815d9f0 msgr2=0x7f2fd8140ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd815d9f0 0x7f2fd8140ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 -- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd806cb40 msgr2=0x7f2fd80711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd806cb40 0x7f2fd80711e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd6ffd640 1 -- 172.21.15.119:0/3636497033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc8009b30 con 0x7f2fd8142f80 2023-11-08T19:26:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.378+0000 7f2fd77fe640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd806cb40 0x7f2fd80711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:22.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.379+0000 7f2fd6ffd640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 0x7f2fd8140760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e8a0 tx=0x7f2fc000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.379+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc00094f0 con 0x7f2fd8142f80 2023-11-08T19:26:22.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.379+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc000c940 con 0x7f2fd8142f80 2023-11-08T19:26:22.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.379+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd80c6220 con 0x7f2fd8142f80 2023-11-08T19:26:22.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.379+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc000ccf0 con 0x7f2fd8142f80 2023-11-08T19:26:22.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.379+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd80c67e0 con 0x7f2fd8142f80 2023-11-08T19:26:22.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.380+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fd8074430 con 0x7f2fd8142f80 2023-11-08T19:26:22.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.381+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fc0013470 con 0x7f2fd8142f80 2023-11-08T19:26:22.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.382+0000 7f2fcffff640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fb806aea0 0x7f2fb806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.382+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2fc0089480 con 0x7f2fd8142f80 2023-11-08T19:26:22.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.382+0000 7f2fd77fe640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fb806aea0 0x7f2fb806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.383+0000 7f2fd77fe640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fb806aea0 0x7f2fb806d360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8035000 tx=0x7f2fc8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.386+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fc0054740 con 0x7f2fd8142f80 2023-11-08T19:26:22.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.644+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2fd80c4230 con 0x7f2fd8142f80 2023-11-08T19:26:22.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.645+0000 7f2fcffff640 1 -- 172.21.15.119:0/3636497033 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f2fc0057d60 con 0x7f2fd8142f80 2023-11-08T19:26:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:22.647 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:26:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-11-08T19:26:22.647 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-11-08T19:26:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg==", 2023-11-08T19:26:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.648 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:26:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:26:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.649 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-11-08T19:26:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ==", 2023-11-08T19:26:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.650 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.651 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-11-08T19:26:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw==", 2023-11-08T19:26:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.652 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:26:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:26:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.653 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ==", 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:26:22.654 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.655 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA==", 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.656 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-11-08T19:26:22.657 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A==", 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.658 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.659 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA==", 2023-11-08T19:26:22.660 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-11-08T19:26:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ==", 2023-11-08T19:26:22.662 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-11-08T19:26:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ==", 2023-11-08T19:26:22.664 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw==", 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g==", 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.669 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ==", 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:26:22.670 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:26:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.671 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-11-08T19:26:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw==", 2023-11-08T19:26:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:26:22.672 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:26:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:26:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-11-08T19:26:22.673 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ==", 2023-11-08T19:26:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:26:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:26:22.674 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:26:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-11-08T19:26:22.675 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q==", 2023-11-08T19:26:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:26:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:26:22.676 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:26:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-11-08T19:26:22.677 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ==", 2023-11-08T19:26:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:26:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:26:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:26:22.678 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:26:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-11-08T19:26:22.679 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBH4Etl5+oDFBAAMV8QbfI7L35tUuiNz9bvdw==", 2023-11-08T19:26:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:26:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:26:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:26:22.680 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-mds", 2023-11-08T19:26:22.681 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBx4EtlsuWAIxAAxuAViW82s20Z4ZFyn0+5dQ==", 2023-11-08T19:26:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-mds" 2023-11-08T19:26:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.682 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-mgr", 2023-11-08T19:26:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBx4Etlb++AIxAAnNoE5W0TSyQ8hER6RhSkFQ==", 2023-11-08T19:26:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-08T19:26:22.683 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-osd", 2023-11-08T19:26:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBx4EtlvvqAIxAAgUgmENTnHxEImjMyRZCWww==", 2023-11-08T19:26:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.684 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-osd" 2023-11-08T19:26:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.685 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rbd", 2023-11-08T19:26:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBx4EtlMwOBIxAAhcT21JV1cURXGzNADjYUXg==", 2023-11-08T19:26:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-08T19:26:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-08T19:26:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBx4Etl2wuBIxAA7pI31hNyPkrskkhkZq6Fmw==", 2023-11-08T19:26:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.687 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-08T19:26:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.bootstrap-rgw", 2023-11-08T19:26:22.688 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBx4EtlpBSBIxAA1ozAjE5FiN8SkDmamkSEYw==", 2023-11-08T19:26:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-08T19:26:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.689 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-11-08T19:26:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA==", 2023-11-08T19:26:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:26:22.690 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:26:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:26:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:22.691 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-11-08T19:26:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ==", 2023-11-08T19:26:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:26:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:26:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:26:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:26:22.692 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:22.693 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-11-08T19:26:22.693 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:26:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fb806aea0 msgr2=0x7f2fb806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fb806aea0 0x7f2fb806d360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2fc8035000 tx=0x7f2fc8030000 comp rx=0 tx=0).stop 2023-11-08T19:26:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 msgr2=0x7f2fd8140760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 0x7f2fd8140760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2fc000e8a0 tx=0x7f2fc000ed60 comp rx=0 tx=0).stop 2023-11-08T19:26:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 shutdown_connections 2023-11-08T19:26:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2fb806aea0 0x7f2fb806d360 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fd815d9f0 0x7f2fd8140ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fd8142f80 0x7f2fd8140760 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 --2- 172.21.15.119:0/3636497033 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2fd806cb40 0x7f2fd80711e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 >> 172.21.15.119:0/3636497033 conn(0x7f2fd80cb610 msgr2=0x7f2fd80b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 shutdown_connections 2023-11-08T19:26:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.648+0000 7f2fddc00640 1 -- 172.21.15.119:0/3636497033 wait complete. 2023-11-08T19:26:22.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-08T19:26:22.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.910+0000 7fe57be4f640 1 Processor -- start 2023-11-08T19:26:22.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.911+0000 7fe57be4f640 1 -- start start 2023-11-08T19:26:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.912+0000 7fe57be4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5741447a0 0x7fe57415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.912+0000 7fe57be4f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5740cb170 0x7fe57406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.912+0000 7fe57be4f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5740c3a50 con 0x7fe5740cad30 2023-11-08T19:26:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.912+0000 7fe57be4f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe574070040 con 0x7fe5740cb170 2023-11-08T19:26:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.912+0000 7fe57be4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574144b80 con 0x7fe5741447a0 2023-11-08T19:26:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5740cb170 0x7fe57406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe5740cb170 0x7fe57406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51712/0 (socket says 172.21.15.119:51712) 2023-11-08T19:26:22.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 -- 172.21.15.119:0/4048729341 learned_addr learned my addr 172.21.15.119:0/4048729341 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:22.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe579beb640 1 --2- 172.21.15.119:0/4048729341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5741447a0 0x7fe57415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 -- 172.21.15.119:0/4048729341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5741447a0 msgr2=0x7fe57415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 --2- 172.21.15.119:0/4048729341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5741447a0 0x7fe57415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 -- 172.21.15.119:0/4048729341 >> v1:172.21.15.188:6789/0 conn(0x7fe5740cad30 legacy=0x7fe57406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:22.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.913+0000 7fe57a3ec640 1 -- 172.21.15.119:0/4048729341 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740c9150 con 0x7fe5740cb170 2023-11-08T19:26:22.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.914+0000 7fe57a3ec640 1 --2- 172.21.15.119:0/4048729341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe57406a120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009e50 tx=0x7fe56c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321383d1f619475e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe578be9640 1 -- 172.21.15.119:0/4048729341 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c02ca70 con 0x7fe5740cb170 2023-11-08T19:26:22.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe578be9640 1 -- 172.21.15.119:0/4048729341 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c02cbd0 con 0x7fe5740cb170 2023-11-08T19:26:22.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe578be9640 1 -- 172.21.15.119:0/4048729341 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0343f0 con 0x7fe5740cb170 2023-11-08T19:26:22.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe57be4f640 1 -- 172.21.15.119:0/4048729341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 msgr2=0x7fe57406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe57be4f640 1 --2- 172.21.15.119:0/4048729341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe57406a120 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009e50 tx=0x7fe56c02bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe57be4f640 1 -- 172.21.15.119:0/4048729341 shutdown_connections 2023-11-08T19:26:22.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe57be4f640 1 --2- 172.21.15.119:0/4048729341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe57406a120 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe57be4f640 1 --2- 172.21.15.119:0/4048729341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5741447a0 0x7fe57415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.915+0000 7fe57be4f640 1 -- 172.21.15.119:0/4048729341 >> 172.21.15.119:0/4048729341 conn(0x7fe5740cb610 msgr2=0x7fe5740cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:22.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.916+0000 7fe57be4f640 1 -- 172.21.15.119:0/4048729341 shutdown_connections 2023-11-08T19:26:22.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.916+0000 7fe57be4f640 1 -- 172.21.15.119:0/4048729341 wait complete. 2023-11-08T19:26:22.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.917+0000 7fe57be4f640 1 Processor -- start 2023-11-08T19:26:22.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.917+0000 7fe57be4f640 1 -- start start 2023-11-08T19:26:22.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe57be4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5740cad30 0x7fe57415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe57be4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe5740c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe57be4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5741447a0 0x7fe574131350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe57be4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe57415d320 con 0x7fe5740cb170 2023-11-08T19:26:22.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe57be4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c6b20 con 0x7fe5741447a0 2023-11-08T19:26:22.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe57be4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c6d50 con 0x7fe5740cad30 2023-11-08T19:26:22.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe5793ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe5740c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.918+0000 7fe5793ea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe5740c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51718/0 (socket says 172.21.15.119:51718) 2023-11-08T19:26:22.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 -- 172.21.15.119:0/2500461814 learned_addr learned my addr 172.21.15.119:0/2500461814 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:22.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe57a3ec640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5741447a0 0x7fe574131350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 -- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5740cad30 msgr2=0x7fe57415cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe579beb640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5740cad30 0x7fe57415cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5740cad30 0x7fe57415cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 -- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5741447a0 msgr2=0x7fe574131350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:22.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5741447a0 0x7fe574131350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:22.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 -- 172.21.15.119:0/2500461814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c009b30 con 0x7fe5740cb170 2023-11-08T19:26:22.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe57a3ec640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5741447a0 0x7fe574131350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:22.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5793ea640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe5740c65e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b420 tx=0x7fe56800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.919+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568011830 con 0x7fe5740cb170 2023-11-08T19:26:22.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.920+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe568011990 con 0x7fe5740cb170 2023-11-08T19:26:22.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.920+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe568011d40 con 0x7fe5740cb170 2023-11-08T19:26:22.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.920+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5740c7030 con 0x7fe5740cb170 2023-11-08T19:26:22.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.920+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5740c75f0 con 0x7fe5740cb170 2023-11-08T19:26:22.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.921+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe574074430 con 0x7fe5740cb170 2023-11-08T19:26:22.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.922+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe56801b460 con 0x7fe5740cb170 2023-11-08T19:26:22.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.922+0000 7fe5667fc640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55806ae70 0x7fe55806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:22.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.923+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fe568088f90 con 0x7fe5740cb170 2023-11-08T19:26:22.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.923+0000 7fe579beb640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55806ae70 0x7fe55806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:22.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.923+0000 7fe579beb640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55806ae70 0x7fe55806d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe570006fd0 tx=0x7fe570008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:22.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:22.927+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5680bbca0 con 0x7fe5740cb170 2023-11-08T19:26:23.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.170+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fe57406caa0 con 0x7fe5740cb170 2023-11-08T19:26:23.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.174+0000 7fe5667fc640 1 -- 172.21.15.119:0/2500461814 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe568054250 con 0x7fe5740cb170 2023-11-08T19:26:23.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55806ae70 msgr2=0x7fe55806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55806ae70 0x7fe55806d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe570006fd0 tx=0x7fe570008040 comp rx=0 tx=0).stop 2023-11-08T19:26:23.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 msgr2=0x7fe5740c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe5740c65e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe56800b420 tx=0x7fe56800b8e0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 shutdown_connections 2023-11-08T19:26:23.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe55806ae70 0x7fe55806d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5741447a0 0x7fe574131350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5740cb170 0x7fe5740c65e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 --2- 172.21.15.119:0/2500461814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5740cad30 0x7fe57415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 >> 172.21.15.119:0/2500461814 conn(0x7fe5740cb610 msgr2=0x7fe574070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:23.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 shutdown_connections 2023-11-08T19:26:23.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.176+0000 7fe57be4f640 1 -- 172.21.15.119:0/2500461814 wait complete. 2023-11-08T19:26:23.195 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-08T19:26:23.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.402+0000 7f9785ed4640 1 Processor -- start 2023-11-08T19:26:23.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.403+0000 7f9785ed4640 1 -- start start 2023-11-08T19:26:23.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.404+0000 7f9785ed4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 0x7f978006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.404+0000 7f9785ed4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f978015f900 con 0x7f978006a120 2023-11-08T19:26:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.404+0000 7f9785ed4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97800c3a50 con 0x7f97801447a0 2023-11-08T19:26:23.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.404+0000 7f9785ed4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9780144b80 con 0x7f978015f4c0 2023-11-08T19:26:23.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f978006a120 0x7f97800c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48134/0 (socket says 172.21.15.119:48134) 2023-11-08T19:26:23.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977ffff640 1 -- 172.21.15.119:0/3529320694 learned_addr learned my addr 172.21.15.119:0/3529320694 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:23.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977effd640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 0x7f978006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927523376 0 0) 0x7f9778001490 con 0x7f978006a120 2023-11-08T19:26:23.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9768003520 con 0x7f978006a120 2023-11-08T19:26:23.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3114316374 0 0) 0x7f9768003520 con 0x7f978006a120 2023-11-08T19:26:23.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 msgr2=0x7f978006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 0x7f978006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 >> v1:172.21.15.188:6789/0 conn(0x7f97801447a0 legacy=0x7f97800cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97800c9150 con 0x7f978006a120 2023-11-08T19:26:23.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f9778001b80 con 0x7f978006a120 2023-11-08T19:26:23.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.405+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 >> v1:172.21.15.119:6789/0 conn(0x7f978006a120 legacy=0x7f97800c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977e7fc640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97680039a0 0x7f9768003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977e7fc640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 0x7f97680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977e7fc640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9768004d00 0x7f9768005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780144b80 con 0x7f9768004d00 2023-11-08T19:26:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976800ba90 con 0x7f97680039a0 2023-11-08T19:26:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976800bc50 con 0x7f978015f4c0 2023-11-08T19:26:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977ffff640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9768004d00 0x7f9768005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977effd640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 0x7f97680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977f7fe640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97680039a0 0x7f9768003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977ffff640 1 -- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 msgr2=0x7f97680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977ffff640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f978015f4c0 0x7f97680047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977ffff640 1 -- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97680039a0 msgr2=0x7f9768003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977ffff640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97680039a0 0x7f9768003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977ffff640 1 -- 172.21.15.119:0/3529320694 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768003520 con 0x7f9768004d00 2023-11-08T19:26:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.406+0000 7f977f7fe640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f97680039a0 0x7f9768003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.407+0000 7f977ffff640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9768004d00 0x7f9768005180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9778002230 tx=0x7f97680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e8a5b68a0eb3ab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97780072d0 con 0x7f9768004d00 2023-11-08T19:26:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977802f070 con 0x7f9768004d00 2023-11-08T19:26:23.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f977e7fc640 1 -- 172.21.15.119:0/3529320694 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9778007690 con 0x7f9768004d00 2023-11-08T19:26:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f9785ed4640 1 -- 172.21.15.119:0/3529320694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9768004d00 msgr2=0x7f9768005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f9785ed4640 1 --2- 172.21.15.119:0/3529320694 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9768004d00 0x7f9768005180 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9778002230 tx=0x7f97680057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f977ffff640 1 -- 172.21.15.119:0/3529320694 reap_dead start 2023-11-08T19:26:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f9785ed4640 1 -- 172.21.15.119:0/3529320694 shutdown_connections 2023-11-08T19:26:23.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f9785ed4640 1 -- 172.21.15.119:0/3529320694 >> 172.21.15.119:0/3529320694 conn(0x7f97800cb610 msgr2=0x7f97800cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f9785ed4640 1 -- 172.21.15.119:0/3529320694 shutdown_connections 2023-11-08T19:26:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.408+0000 7f9785ed4640 1 -- 172.21.15.119:0/3529320694 wait complete. 2023-11-08T19:26:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.410+0000 7f9785ed4640 1 Processor -- start 2023-11-08T19:26:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.410+0000 7f9785ed4640 1 -- start start 2023-11-08T19:26:23.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.411+0000 7f9785ed4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768004d00 0x7f978015d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.411+0000 7f9785ed4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 0x7f97800c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.411+0000 7f9785ed4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97801421f0 0x7f9780142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.411+0000 7f9785ed4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9780130450 con 0x7f97800c8410 2023-11-08T19:26:23.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.411+0000 7f9785ed4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97800c8dd0 con 0x7f9768004d00 2023-11-08T19:26:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.411+0000 7f9785ed4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9780130090 con 0x7f97801421f0 2023-11-08T19:26:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 0x7f97800c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 0x7f97800c8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51732/0 (socket says 172.21.15.119:51732) 2023-11-08T19:26:23.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 -- 172.21.15.119:0/112108118 learned_addr learned my addr 172.21.15.119:0/112108118 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977f7fe640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768004d00 0x7f978015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 -- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97801421f0 msgr2=0x7f9780142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97801421f0 0x7f9780142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 -- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768004d00 msgr2=0x7f978015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768004d00 0x7f978015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 -- 172.21.15.119:0/112108118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9780142f40 con 0x7f97800c8410 2023-11-08T19:26:23.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977f7fe640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768004d00 0x7f978015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:23.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.412+0000 7f977effd640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 0x7f97800c8890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9774010810 tx=0x7f9774010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.413+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774009610 con 0x7f97800c8410 2023-11-08T19:26:23.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.413+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977400eb60 con 0x7f97800c8410 2023-11-08T19:26:23.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.413+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977401c400 con 0x7f97800c8410 2023-11-08T19:26:23.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.413+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9780143220 con 0x7f97800c8410 2023-11-08T19:26:23.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.413+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978015dc90 con 0x7f97800c8410 2023-11-08T19:26:23.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.414+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9780130590 con 0x7f97800c8410 2023-11-08T19:26:23.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.415+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f977401c560 con 0x7f97800c8410 2023-11-08T19:26:23.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.416+0000 7f9767fff640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976c06aea0 0x7f976c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.416+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f97740898b0 con 0x7f97800c8410 2023-11-08T19:26:23.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.416+0000 7f977f7fe640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976c06aea0 0x7f976c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.417+0000 7f977f7fe640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976c06aea0 0x7f976c06d360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9770000c00 tx=0x7f9770009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.421+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9780130590 con 0x7f97800c8410 2023-11-08T19:26:23.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.684+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9780130590 con 0x7f97800c8410 2023-11-08T19:26:23.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.689+0000 7f9767fff640 1 -- 172.21.15.119:0/112108118 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9780130590 con 0x7f97800c8410 2023-11-08T19:26:23.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976c06aea0 msgr2=0x7f976c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976c06aea0 0x7f976c06d360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9770000c00 tx=0x7f9770009040 comp rx=0 tx=0).stop 2023-11-08T19:26:23.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 msgr2=0x7f97800c8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 0x7f97800c8890 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f9774010810 tx=0x7f9774010cd0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 shutdown_connections 2023-11-08T19:26:23.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976c06aea0 0x7f976c06d360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f97801421f0 0x7f9780142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f97800c8410 0x7f97800c8890 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.692+0000 7f9785ed4640 1 --2- 172.21.15.119:0/112108118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9768004d00 0x7f978015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.693+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 >> 172.21.15.119:0/112108118 conn(0x7f97800cb610 msgr2=0x7f978015e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:23.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.693+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 shutdown_connections 2023-11-08T19:26:23.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.693+0000 7f9785ed4640 1 -- 172.21.15.119:0/112108118 wait complete. 2023-11-08T19:26:23.713 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-08T19:26:23.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.927+0000 7f82d9fb6640 1 Processor -- start 2023-11-08T19:26:23.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.927+0000 7f82d9fb6640 1 -- start start 2023-11-08T19:26:23.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d9fb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 0x7f82d40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d9fb6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40c4960 con 0x7f82d415d9f0 2023-11-08T19:26:23.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d9fb6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d415df60 con 0x7f82d4142f80 2023-11-08T19:26:23.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d9fb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d4143a40 con 0x7f82d406d250 2023-11-08T19:26:23.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82caffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f82d415d9f0 0x7f82d406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48148/0 (socket says 172.21.15.119:48148) 2023-11-08T19:26:23.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82caffd640 1 -- 172.21.15.119:0/1437229148 learned_addr learned my addr 172.21.15.119:0/1437229148 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:23.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d3fff640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 0x7f82d40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015288628 0 0) 0x7f82d40c4960 con 0x7f82d415d9f0 2023-11-08T19:26:23.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.929+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003520 con 0x7f82d415d9f0 2023-11-08T19:26:23.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 702665247 0 0) 0x7f82c0003520 con 0x7f82d415d9f0 2023-11-08T19:26:23.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 msgr2=0x7f82d40cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 0x7f82d40cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 >> v1:172.21.15.188:6789/0 conn(0x7f82d4142f80 legacy=0x7f82d40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d415e240 con 0x7f82d415d9f0 2023-11-08T19:26:23.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f82bc001670 con 0x7f82d415d9f0 2023-11-08T19:26:23.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 >> v1:172.21.15.119:6789/0 conn(0x7f82d415d9f0 legacy=0x7f82d406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c00039a0 0x7f82c0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 0x7f82c0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d4143a40 con 0x7f82c00047c0 2023-11-08T19:26:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000bb10 con 0x7f82c00039a0 2023-11-08T19:26:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000bcd0 con 0x7f82d406d250 2023-11-08T19:26:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.930+0000 7f82caffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.931+0000 7f82d37fe640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c00039a0 0x7f82c0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.931+0000 7f82caffd640 1 -- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 msgr2=0x7f82c0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.931+0000 7f82caffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d406d250 0x7f82c0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.931+0000 7f82caffd640 1 -- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c00039a0 msgr2=0x7f82c0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:23.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.931+0000 7f82caffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82c00039a0 0x7f82c0003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.931+0000 7f82caffd640 1 -- 172.21.15.119:0/1437229148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0003520 con 0x7f82c00047c0 2023-11-08T19:26:23.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82caffd640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c0e0 tx=0x7f82c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f47f9119f440002 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc005ad0 con 0x7f82c00047c0 2023-11-08T19:26:23.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82bc005c30 con 0x7f82c00047c0 2023-11-08T19:26:23.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82d2ffd640 1 -- 172.21.15.119:0/1437229148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82bc02c810 con 0x7f82c00047c0 2023-11-08T19:26:23.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1437229148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 msgr2=0x7f82c0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1437229148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82c0004c40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c0e0 tx=0x7f82c00057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.932+0000 7f82d3fff640 1 -- 172.21.15.119:0/1437229148 reap_dead start 2023-11-08T19:26:23.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.933+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1437229148 shutdown_connections 2023-11-08T19:26:23.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.933+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1437229148 >> 172.21.15.119:0/1437229148 conn(0x7f82d40cb610 msgr2=0x7f82d40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:23.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.933+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1437229148 shutdown_connections 2023-11-08T19:26:23.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.933+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1437229148 wait complete. 2023-11-08T19:26:23.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.934+0000 7f82d9fb6640 1 Processor -- start 2023-11-08T19:26:23.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.934+0000 7f82d9fb6640 1 -- start start 2023-11-08T19:26:23.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.935+0000 7f82d9fb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82d41f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.935+0000 7f82d9fb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d40b0db0 0x7f82d4141f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.935+0000 7f82d9fb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82d4159930 0x7f82d41424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.935+0000 7f82d9fb6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40b1620 con 0x7f82c00047c0 2023-11-08T19:26:23.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.935+0000 7f82d9fb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d41f66a0 con 0x7f82d4159930 2023-11-08T19:26:23.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.935+0000 7f82d9fb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40b2120 con 0x7f82d40b0db0 2023-11-08T19:26:23.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82d41f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82d41f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51748/0 (socket says 172.21.15.119:51748) 2023-11-08T19:26:23.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 -- 172.21.15.119:0/1379082219 learned_addr learned my addr 172.21.15.119:0/1379082219 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:23.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 -- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d40b0db0 msgr2=0x7f82d4141f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d40b0db0 0x7f82d4141f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 -- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82d4159930 msgr2=0x7f82d41424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:23.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d3fff640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82d4159930 0x7f82d41424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82d4159930 0x7f82d41424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:23.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 -- 172.21.15.119:0/1379082219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d41f68d0 con 0x7f82c00047c0 2023-11-08T19:26:23.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.936+0000 7f82d37fe640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82d41f6160 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f82c400d900 tx=0x7f82c400ddc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.937+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4003da0 con 0x7f82c00047c0 2023-11-08T19:26:23.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.937+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c400b770 con 0x7f82c00047c0 2023-11-08T19:26:23.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.937+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400bae0 con 0x7f82c00047c0 2023-11-08T19:26:23.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.937+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d41f6bb0 con 0x7f82c00047c0 2023-11-08T19:26:23.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.937+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d415b500 con 0x7f82c00047c0 2023-11-08T19:26:23.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.938+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d415b6c0 con 0x7f82c00047c0 2023-11-08T19:26:23.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.939+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f82c400bc40 con 0x7f82c00047c0 2023-11-08T19:26:23.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.940+0000 7f82d0ff9640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f82b406ae90 0x7f82b406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:23.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.940+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f82c4089710 con 0x7f82c00047c0 2023-11-08T19:26:23.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.940+0000 7f82caffd640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f82b406ae90 0x7f82b406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:23.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.941+0000 7f82caffd640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f82b406ae90 0x7f82b406d350 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c0e0 tx=0x7f82bc007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:23.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:23.945+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82c4054930 con 0x7f82c00047c0 2023-11-08T19:26:24.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.201+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f82d41f70d0 con 0x7f82c00047c0 2023-11-08T19:26:24.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.205+0000 7f82d0ff9640 1 -- 172.21.15.119:0/1379082219 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f82c4057f50 con 0x7f82c00047c0 2023-11-08T19:26:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f82b406ae90 msgr2=0x7f82b406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f82b406ae90 0x7f82b406d350 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82bc00c0e0 tx=0x7f82bc007000 comp rx=0 tx=0).stop 2023-11-08T19:26:24.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 msgr2=0x7f82d41f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82d41f6160 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f82c400d900 tx=0x7f82c400ddc0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 shutdown_connections 2023-11-08T19:26:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f82b406ae90 0x7f82b406d350 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82d4159930 0x7f82d41424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82d40b0db0 0x7f82d4141f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 --2- 172.21.15.119:0/1379082219 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82c00047c0 0x7f82d41f6160 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 >> 172.21.15.119:0/1379082219 conn(0x7f82d40cb610 msgr2=0x7f82d406c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 shutdown_connections 2023-11-08T19:26:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.208+0000 7f82d9fb6640 1 -- 172.21.15.119:0/1379082219 wait complete. 2023-11-08T19:26:24.227 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-08T19:26:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.435+0000 7f66d2314640 1 Processor -- start 2023-11-08T19:26:24.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.436+0000 7f66d2314640 1 -- start start 2023-11-08T19:26:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.437+0000 7f66d2314640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 0x7f66cc06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.437+0000 7f66d2314640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc06be20 con 0x7f66cc15aea0 2023-11-08T19:26:24.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.437+0000 7f66d2314640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc06c100 con 0x7f66cc06d400 2023-11-08T19:26:24.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.437+0000 7f66d2314640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc06c3e0 con 0x7f66cc0c9c90 2023-11-08T19:26:24.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66d1312640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f66cc15aea0 0x7f66cc06b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48152/0 (socket says 172.21.15.119:48152) 2023-11-08T19:26:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66d1312640 1 -- 172.21.15.119:0/511326304 learned_addr learned my addr 172.21.15.119:0/511326304 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66d0b11640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 0x7f66cc06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 430923560 0 0) 0x7f66cc06be20 con 0x7f66cc15aea0 2023-11-08T19:26:24.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b8003520 con 0x7f66cc15aea0 2023-11-08T19:26:24.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1187312444 0 0) 0x7f66b8003520 con 0x7f66cc15aea0 2023-11-08T19:26:24.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 msgr2=0x7f66cc06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66c3fff640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 0x7f66cc06cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.438+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 >> v1:172.21.15.188:6789/0 conn(0x7f66cc06d400 legacy=0x7f66cc0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc06c6c0 con 0x7f66cc15aea0 2023-11-08T19:26:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f66c8001670 con 0x7f66cc15aea0 2023-11-08T19:26:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 >> v1:172.21.15.119:6789/0 conn(0x7f66cc15aea0 legacy=0x7f66cc06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 0x7f66b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8003fd0 0x7f66b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b8004d00 0x7f66b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc06c3e0 con 0x7f66b8004d00 2023-11-08T19:26:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b800bbc0 con 0x7f66b8003fd0 2023-11-08T19:26:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b800bd80 con 0x7f66cc0c9c90 2023-11-08T19:26:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66d1b13640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b8004d00 0x7f66b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66d1312640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 0x7f66b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66d0b11640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8003fd0 0x7f66b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66d1b13640 1 -- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 msgr2=0x7f66b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.439+0000 7f66d1b13640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66cc0c9c90 0x7f66b80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.440+0000 7f66d1b13640 1 -- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8003fd0 msgr2=0x7f66b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.440+0000 7f66d1b13640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8003fd0 0x7f66b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.440+0000 7f66d1b13640 1 -- 172.21.15.119:0/511326304 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b8003520 con 0x7f66b8004d00 2023-11-08T19:26:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.440+0000 7f66d0b11640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66b8003fd0 0x7f66b80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66d1b13640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b8004d00 0x7f66b8005180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f66c4001e50 tx=0x7f66b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f936038d2d72412b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4003b70 con 0x7f66b8004d00 2023-11-08T19:26:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c4003cd0 con 0x7f66b8004d00 2023-11-08T19:26:24.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66c3fff640 1 -- 172.21.15.119:0/511326304 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c4005310 con 0x7f66b8004d00 2023-11-08T19:26:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66d2314640 1 -- 172.21.15.119:0/511326304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b8004d00 msgr2=0x7f66b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66d2314640 1 --2- 172.21.15.119:0/511326304 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66b8004d00 0x7f66b8005180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f66c4001e50 tx=0x7f66b80057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66d1b13640 1 -- 172.21.15.119:0/511326304 reap_dead start 2023-11-08T19:26:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66d2314640 1 -- 172.21.15.119:0/511326304 shutdown_connections 2023-11-08T19:26:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.441+0000 7f66d2314640 1 -- 172.21.15.119:0/511326304 >> 172.21.15.119:0/511326304 conn(0x7f66cc0b0560 msgr2=0x7f66cc0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.442+0000 7f66d2314640 1 -- 172.21.15.119:0/511326304 shutdown_connections 2023-11-08T19:26:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.442+0000 7f66d2314640 1 -- 172.21.15.119:0/511326304 wait complete. 2023-11-08T19:26:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.443+0000 7f66d2314640 1 Processor -- start 2023-11-08T19:26:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.444+0000 7f66d2314640 1 -- start start 2023-11-08T19:26:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.444+0000 7f66d2314640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b8004d00 0x7f66cc0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.444+0000 7f66d2314640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 0x7f66cc157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d2314640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66cc0acfa0 0x7f66cc1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d2314640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc0cc640 con 0x7f66cc0acb40 2023-11-08T19:26:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d2314640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc0b2070 con 0x7f66cc0acfa0 2023-11-08T19:26:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d2314640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc0b2350 con 0x7f66b8004d00 2023-11-08T19:26:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 0x7f66cc157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 0x7f66cc157ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51766/0 (socket says 172.21.15.119:51766) 2023-11-08T19:26:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 -- 172.21.15.119:0/2463343228 learned_addr learned my addr 172.21.15.119:0/2463343228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 -- 172.21.15.119:0/2463343228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b8004d00 msgr2=0x7f66cc0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b8004d00 0x7f66cc0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 -- 172.21.15.119:0/2463343228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66cc0acfa0 msgr2=0x7f66cc1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66cc0acfa0 0x7f66cc1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 -- 172.21.15.119:0/2463343228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0b2630 con 0x7f66cc0acb40 2023-11-08T19:26:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.445+0000 7f66d0b11640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 0x7f66cc157ca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e7e0 tx=0x7f66bc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.446+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00c650 con 0x7f66cc0acb40 2023-11-08T19:26:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.446+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc00c7b0 con 0x7f66cc0acb40 2023-11-08T19:26:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.446+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc00cb20 con 0x7f66cc0acb40 2023-11-08T19:26:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.446+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc0b2910 con 0x7f66cc0acb40 2023-11-08T19:26:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.446+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc0b2ed0 con 0x7f66cc0acb40 2023-11-08T19:26:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.447+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66cc04ece0 con 0x7f66cc0acb40 2023-11-08T19:26:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.448+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66bc00cc80 con 0x7f66cc0acb40 2023-11-08T19:26:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.449+0000 7f66c1ffb640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66b006ae70 0x7f66b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.449+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f66bc0890f0 con 0x7f66cc0acb40 2023-11-08T19:26:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.449+0000 7f66d1312640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66b006ae70 0x7f66b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.450+0000 7f66d1312640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66b006ae70 0x7f66b006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66c8007a30 tx=0x7f66c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.453+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66bc0543b0 con 0x7f66cc0acb40 2023-11-08T19:26:24.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.712+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f66cc0ae850 con 0x7f66cc0acb40 2023-11-08T19:26:24.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.717+0000 7f66c1ffb640 1 -- 172.21.15.119:0/2463343228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f66bc0579d0 con 0x7f66cc0acb40 2023-11-08T19:26:24.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66b006ae70 msgr2=0x7f66b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66b006ae70 0x7f66b006d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f66c8007a30 tx=0x7f66c8009040 comp rx=0 tx=0).stop 2023-11-08T19:26:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 msgr2=0x7f66cc157ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 0x7f66cc157ca0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e7e0 tx=0x7f66bc00eca0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 shutdown_connections 2023-11-08T19:26:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f66b006ae70 0x7f66b006d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66cc0acfa0 0x7f66cc1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66cc0acb40 0x7f66cc157ca0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 --2- 172.21.15.119:0/2463343228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66b8004d00 0x7f66cc0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 >> 172.21.15.119:0/2463343228 conn(0x7f66cc0b0560 msgr2=0x7f66cc06fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.720+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 shutdown_connections 2023-11-08T19:26:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.721+0000 7f66d2314640 1 -- 172.21.15.119:0/2463343228 wait complete. 2023-11-08T19:26:24.739 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-08T19:26:24.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.944+0000 7f2cd975e640 1 Processor -- start 2023-11-08T19:26:24.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.944+0000 7f2cd975e640 1 -- start start 2023-11-08T19:26:24.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd975e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd415d000 0x7f2cd40cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd975e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2cd406a830 0x7f2cd4160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd975e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd406d220 con 0x7f2cd40c3a50 2023-11-08T19:26:24.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd975e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd40ccd50 con 0x7f2cd406a830 2023-11-08T19:26:24.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd975e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd40aca40 con 0x7f2cd415d000 2023-11-08T19:26:24.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd37fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2cd406a830 0x7f2cd4160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.946+0000 7f2cd2ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd415d000 0x7f2cd40cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.947+0000 7f2cd37fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2cd406a830 0x7f2cd4160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:55662/0 (socket says 172.21.15.119:55662) 2023-11-08T19:26:24.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.947+0000 7f2cd37fe640 1 -- 172.21.15.119:0/1572875050 learned_addr learned my addr 172.21.15.119:0/1572875050 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:24.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.947+0000 7f2cd37fe640 1 -- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd415d000 msgr2=0x7f2cd40cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:24.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.947+0000 7f2cd37fe640 1 --2- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd415d000 0x7f2cd40cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.947+0000 7f2cd37fe640 1 -- 172.21.15.119:0/1572875050 >> v1:172.21.15.188:6789/0 conn(0x7f2cd40c3a50 legacy=0x7f2cd406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.947+0000 7f2cd37fe640 1 -- 172.21.15.119:0/1572875050 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40acd20 con 0x7f2cd406a830 2023-11-08T19:26:24.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.949+0000 7f2cd37fe640 1 --2- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd406a830 0x7f2cd4160e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a012bd8a5c6b1946 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.949+0000 7f2cd1ffb640 1 -- 172.21.15.119:0/1572875050 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc02ca70 con 0x7f2cd406a830 2023-11-08T19:26:24.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd1ffb640 1 -- 172.21.15.119:0/1572875050 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ccc02cbd0 con 0x7f2cd406a830 2023-11-08T19:26:24.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd1ffb640 1 -- 172.21.15.119:0/1572875050 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ccc0343f0 con 0x7f2cd406a830 2023-11-08T19:26:24.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 -- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd406a830 msgr2=0x7f2cd4160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 --2- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd406a830 0x7f2cd4160e90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc009e50 tx=0x7f2ccc02bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 -- 172.21.15.119:0/1572875050 shutdown_connections 2023-11-08T19:26:24.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 --2- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd406a830 0x7f2cd4160e90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 --2- 172.21.15.119:0/1572875050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd415d000 0x7f2cd40cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 -- 172.21.15.119:0/1572875050 >> 172.21.15.119:0/1572875050 conn(0x7f2cd40cb610 msgr2=0x7f2cd40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:24.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 -- 172.21.15.119:0/1572875050 shutdown_connections 2023-11-08T19:26:24.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.950+0000 7f2cd975e640 1 -- 172.21.15.119:0/1572875050 wait complete. 2023-11-08T19:26:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.952+0000 7f2cd975e640 1 Processor -- start 2023-11-08T19:26:24.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.952+0000 7f2cd975e640 1 -- start start 2023-11-08T19:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd975e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd406a830 0x7f2cd41f22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd975e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 0x7f2cd406ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd975e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd415d000 0x7f2cd406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd975e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd4154d90 con 0x7f2cd40c3a50 2023-11-08T19:26:24.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd975e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd41f5fc0 con 0x7f2cd415d000 2023-11-08T19:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd975e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd41f6210 con 0x7f2cd406a830 2023-11-08T19:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.953+0000 7f2cd27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 0x7f2cd406ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 0x7f2cd406ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51782/0 (socket says 172.21.15.119:51782) 2023-11-08T19:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 -- 172.21.15.119:0/2993908036 learned_addr learned my addr 172.21.15.119:0/2993908036 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:24.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd2ffd640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd406a830 0x7f2cd41f22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd37fe640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd415d000 0x7f2cd406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 -- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd406a830 msgr2=0x7f2cd41f22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd406a830 0x7f2cd41f22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 -- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd415d000 msgr2=0x7f2cd406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:24.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd415d000 0x7f2cd406bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 -- 172.21.15.119:0/2993908036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009b30 con 0x7f2cd40c3a50 2023-11-08T19:26:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.954+0000 7f2cd27fc640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 0x7f2cd406ba30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.955+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc80052d0 con 0x7f2cd40c3a50 2023-11-08T19:26:24.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.955+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc8005430 con 0x7f2cd40c3a50 2023-11-08T19:26:24.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.955+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc8012870 con 0x7f2cd40c3a50 2023-11-08T19:26:24.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.955+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd41f64f0 con 0x7f2cd40c3a50 2023-11-08T19:26:24.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.955+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd41f6ab0 con 0x7f2cd40c3a50 2023-11-08T19:26:24.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.956+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd4074430 con 0x7f2cd40c3a50 2023-11-08T19:26:24.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.957+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cc80129d0 con 0x7f2cd40c3a50 2023-11-08T19:26:24.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.958+0000 7f2cbb7fe640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2cc006aea0 0x7f2cc006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:24.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.958+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f2cc8088e70 con 0x7f2cd40c3a50 2023-11-08T19:26:24.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.958+0000 7f2cd2ffd640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2cc006aea0 0x7f2cc006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:24.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.959+0000 7f2cd2ffd640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2cc006aea0 0x7f2cc006d360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40b08e0 tx=0x7f2cc4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:24.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:24.963+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2cc80541c0 con 0x7f2cd40c3a50 2023-11-08T19:26:25.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.228+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f2cd406cd50 con 0x7f2cd40c3a50 2023-11-08T19:26:25.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.232+0000 7f2cbb7fe640 1 -- 172.21.15.119:0/2993908036 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2cc80577e0 con 0x7f2cd40c3a50 2023-11-08T19:26:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.235+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2cc006aea0 msgr2=0x7f2cc006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.235+0000 7f2cd975e640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2cc006aea0 0x7f2cc006d360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40b08e0 tx=0x7f2cc4008040 comp rx=0 tx=0).stop 2023-11-08T19:26:25.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.235+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 msgr2=0x7f2cd406ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.235+0000 7f2cd975e640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 0x7f2cd406ba30 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2cc800c450 tx=0x7f2cc800c910 comp rx=0 tx=0).stop 2023-11-08T19:26:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 shutdown_connections 2023-11-08T19:26:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2cc006aea0 0x7f2cc006d360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2cd415d000 0x7f2cd406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2cd40c3a50 0x7f2cd406ba30 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 --2- 172.21.15.119:0/2993908036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2cd406a830 0x7f2cd41f22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 >> 172.21.15.119:0/2993908036 conn(0x7f2cd40cb610 msgr2=0x7f2cd406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:25.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 shutdown_connections 2023-11-08T19:26:25.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.236+0000 7f2cd975e640 1 -- 172.21.15.119:0/2993908036 wait complete. 2023-11-08T19:26:25.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-08T19:26:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.466+0000 7f453e875640 1 Processor -- start 2023-11-08T19:26:25.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.466+0000 7f453e875640 1 -- start start 2023-11-08T19:26:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.467+0000 7f453e875640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f453806fe80 0x7f453815bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453e875640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453815c160 con 0x7f45380ad710 2023-11-08T19:26:25.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453e875640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4538070310 con 0x7f453806ced0 2023-11-08T19:26:25.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453e875640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453806d2b0 con 0x7f453806fe80 2023-11-08T19:26:25.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453d072640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f453806fe80 0x7f453815bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453e074640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f45380ad710 0x7f45380adb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48156/0 (socket says 172.21.15.119:48156) 2023-11-08T19:26:25.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453e074640 1 -- 172.21.15.119:0/739281727 learned_addr learned my addr 172.21.15.119:0/739281727 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453d072640 1 -- 172.21.15.119:0/739281727 >> v1:172.21.15.188:6789/0 conn(0x7f453806ced0 legacy=0x7f45380ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453d072640 1 -- 172.21.15.119:0/739281727 >> v1:172.21.15.119:6789/0 conn(0x7f45380ad710 legacy=0x7f45380adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.468+0000 7f453d072640 1 -- 172.21.15.119:0/739281727 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45380c9820 con 0x7f453806fe80 2023-11-08T19:26:25.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.469+0000 7f453d072640 1 --2- 172.21.15.119:0/739281727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806fe80 0x7f453815bc20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45280097a0 tx=0x7f452802b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3c59e15771b26e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.470+0000 7f453c871640 1 -- 172.21.15.119:0/739281727 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452802f050 con 0x7f453806fe80 2023-11-08T19:26:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.470+0000 7f453c871640 1 -- 172.21.15.119:0/739281727 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4528031700 con 0x7f453806fe80 2023-11-08T19:26:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.470+0000 7f453c871640 1 -- 172.21.15.119:0/739281727 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528031a90 con 0x7f453806fe80 2023-11-08T19:26:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.470+0000 7f453e875640 1 -- 172.21.15.119:0/739281727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806fe80 msgr2=0x7f453815bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.470+0000 7f453e875640 1 --2- 172.21.15.119:0/739281727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806fe80 0x7f453815bc20 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f45280097a0 tx=0x7f452802b7a0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.471+0000 7f453e875640 1 -- 172.21.15.119:0/739281727 shutdown_connections 2023-11-08T19:26:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.471+0000 7f453e875640 1 --2- 172.21.15.119:0/739281727 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806fe80 0x7f453815bc20 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.471+0000 7f453e875640 1 -- 172.21.15.119:0/739281727 >> 172.21.15.119:0/739281727 conn(0x7f45380ce930 msgr2=0x7f45380ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.471+0000 7f453e875640 1 -- 172.21.15.119:0/739281727 shutdown_connections 2023-11-08T19:26:25.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.471+0000 7f453e875640 1 -- 172.21.15.119:0/739281727 wait complete. 2023-11-08T19:26:25.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.473+0000 7f453e875640 1 Processor -- start 2023-11-08T19:26:25.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.473+0000 7f453e875640 1 -- start start 2023-11-08T19:26:25.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453e875640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 0x7f4538147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453e875640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f453806fe80 0x7f45381e6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453e875640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45380ad710 0x7f45381e6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453e875640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45380ca180 con 0x7f453806ced0 2023-11-08T19:26:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453e875640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453815d040 con 0x7f45380ad710 2023-11-08T19:26:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453e875640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453815d320 con 0x7f453806fe80 2023-11-08T19:26:25.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.474+0000 7f453d873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 0x7f4538147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 0x7f4538147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51790/0 (socket says 172.21.15.119:51790) 2023-11-08T19:26:25.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 -- 172.21.15.119:0/24749649 learned_addr learned my addr 172.21.15.119:0/24749649 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:25.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453e074640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45380ad710 0x7f45381e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d072640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f453806fe80 0x7f45381e6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 -- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f453806fe80 msgr2=0x7f45381e6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f453806fe80 0x7f45381e6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 -- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45380ad710 msgr2=0x7f45381e6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45380ad710 0x7f45381e6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 -- 172.21.15.119:0/24749649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528009400 con 0x7f453806ced0 2023-11-08T19:26:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453e074640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45380ad710 0x7f45381e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:25.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.475+0000 7f453d873640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 0x7f4538147220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f453400b3f0 tx=0x7f453400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.476+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534011830 con 0x7f453806ced0 2023-11-08T19:26:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.476+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4534011990 con 0x7f453806ced0 2023-11-08T19:26:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.476+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4534011d40 con 0x7f453806ced0 2023-11-08T19:26:25.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.476+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453815d600 con 0x7f453806ced0 2023-11-08T19:26:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.476+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453815dbc0 con 0x7f453806ced0 2023-11-08T19:26:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.477+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453815ddb0 con 0x7f453806ced0 2023-11-08T19:26:25.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.478+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f453401a400 con 0x7f453806ced0 2023-11-08T19:26:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.479+0000 7f452e7fc640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f451c06aea0 0x7f451c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.479+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4534089250 con 0x7f453806ced0 2023-11-08T19:26:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.479+0000 7f453d072640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f451c06aea0 0x7f451c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.480+0000 7f453d072640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f451c06aea0 0x7f451c06d360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4528035000 tx=0x7f4528008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.485+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4534054470 con 0x7f453806ced0 2023-11-08T19:26:25.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.743+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f45380677d0 con 0x7f453806ced0 2023-11-08T19:26:25.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.746+0000 7f452e7fc640 1 -- 172.21.15.119:0/24749649 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4534057a90 con 0x7f453806ced0 2023-11-08T19:26:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f451c06aea0 msgr2=0x7f451c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f451c06aea0 0x7f451c06d360 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4528035000 tx=0x7f4528008030 comp rx=0 tx=0).stop 2023-11-08T19:26:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 msgr2=0x7f4538147220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 0x7f4538147220 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f453400b3f0 tx=0x7f453400b8b0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 shutdown_connections 2023-11-08T19:26:25.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f451c06aea0 0x7f451c06d360 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f45380ad710 0x7f45381e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f453806fe80 0x7f45381e6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 --2- 172.21.15.119:0/24749649 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f453806ced0 0x7f4538147220 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 >> 172.21.15.119:0/24749649 conn(0x7f45380ce930 msgr2=0x7f45381587c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 shutdown_connections 2023-11-08T19:26:25.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.750+0000 7f453e875640 1 -- 172.21.15.119:0/24749649 wait complete. 2023-11-08T19:26:25.770 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:25.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.979+0000 7f4a4654c640 1 Processor -- start 2023-11-08T19:26:25.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.980+0000 7f4a4654c640 1 -- start start 2023-11-08T19:26:25.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.981+0000 7f4a4654c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a401447a0 0x7f4a4015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.981+0000 7f4a4654c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4006a120 0x7f4a4006a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.981+0000 7f4a4654c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4006aa90 con 0x7f4a400cad30 2023-11-08T19:26:25.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.981+0000 7f4a4654c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a40070040 con 0x7f4a401447a0 2023-11-08T19:26:25.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.981+0000 7f4a4654c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a400c3a50 con 0x7f4a4006a120 2023-11-08T19:26:25.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.982+0000 7f4a3ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a401447a0 0x7f4a4015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.982+0000 7f4a44ae9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4006a120 0x7f4a4006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.982+0000 7f4a44ae9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4006a120 0x7f4a4006a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:34660/0 (socket says 172.21.15.119:34660) 2023-11-08T19:26:25.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.982+0000 7f4a44ae9640 1 -- 172.21.15.119:0/418509531 learned_addr learned my addr 172.21.15.119:0/418509531 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:25.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.983+0000 7f4a3ffff640 1 -- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4006a120 msgr2=0x7f4a4006a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:25.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.983+0000 7f4a3ffff640 1 --2- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4006a120 0x7f4a4006a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.983+0000 7f4a3ffff640 1 -- 172.21.15.119:0/418509531 >> v1:172.21.15.188:6789/0 conn(0x7f4a400cad30 legacy=0x7f4a4006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:25.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.983+0000 7f4a3ffff640 1 -- 172.21.15.119:0/418509531 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40144b80 con 0x7f4a401447a0 2023-11-08T19:26:25.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.984+0000 7f4a3ffff640 1 --2- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a401447a0 0x7f4a4015f4c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a300097a0 tx=0x7f4a3002b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af200a9384af49f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.985+0000 7f4a3f7fe640 1 -- 172.21.15.119:0/418509531 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3002e070 con 0x7f4a401447a0 2023-11-08T19:26:25.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.985+0000 7f4a3f7fe640 1 -- 172.21.15.119:0/418509531 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a300315a0 con 0x7f4a401447a0 2023-11-08T19:26:25.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.985+0000 7f4a3f7fe640 1 -- 172.21.15.119:0/418509531 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30031930 con 0x7f4a401447a0 2023-11-08T19:26:25.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 -- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a401447a0 msgr2=0x7f4a4015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 --2- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a401447a0 0x7f4a4015f4c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4a300097a0 tx=0x7f4a3002b840 comp rx=0 tx=0).stop 2023-11-08T19:26:25.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 -- 172.21.15.119:0/418509531 shutdown_connections 2023-11-08T19:26:25.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 --2- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4006a120 0x7f4a4006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 --2- 172.21.15.119:0/418509531 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a401447a0 0x7f4a4015f4c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 -- 172.21.15.119:0/418509531 >> 172.21.15.119:0/418509531 conn(0x7f4a400cb610 msgr2=0x7f4a400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:25.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 -- 172.21.15.119:0/418509531 shutdown_connections 2023-11-08T19:26:25.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.986+0000 7f4a4654c640 1 -- 172.21.15.119:0/418509531 wait complete. 2023-11-08T19:26:25.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.988+0000 7f4a4654c640 1 Processor -- start 2023-11-08T19:26:25.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.989+0000 7f4a4654c640 1 -- start start 2023-11-08T19:26:25.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a4654c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 0x7f4a4015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a4654c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a400cad30 0x7f4a400c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a4654c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a401447a0 0x7f4a400c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a4654c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4015ba10 con 0x7f4a4006a120 2023-11-08T19:26:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a4654c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4015bc60 con 0x7f4a400cad30 2023-11-08T19:26:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a4654c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4015bf40 con 0x7f4a401447a0 2023-11-08T19:26:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a3ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 0x7f4a4015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a3ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 0x7f4a4015a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51804/0 (socket says 172.21.15.119:51804) 2023-11-08T19:26:25.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.990+0000 7f4a3ffff640 1 -- 172.21.15.119:0/3176805455 learned_addr learned my addr 172.21.15.119:0/3176805455 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a37fff640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a400cad30 0x7f4a400c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a44ae9640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a401447a0 0x7f4a400c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3ffff640 1 -- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a401447a0 msgr2=0x7f4a400c7160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3ffff640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a401447a0 0x7f4a400c7160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3ffff640 1 -- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a400cad30 msgr2=0x7f4a400c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3ffff640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a400cad30 0x7f4a400c6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3ffff640 1 -- 172.21.15.119:0/3176805455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30009400 con 0x7f4a4006a120 2023-11-08T19:26:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3ffff640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 0x7f4a4015a390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4a30034040 tx=0x7f4a300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30032840 con 0x7f4a4006a120 2023-11-08T19:26:25.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a300329c0 con 0x7f4a4006a120 2023-11-08T19:26:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30032d70 con 0x7f4a4006a120 2023-11-08T19:26:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.991+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4015c220 con 0x7f4a4006a120 2023-11-08T19:26:25.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.992+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a40140360 con 0x7f4a4006a120 2023-11-08T19:26:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.993+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a40074430 con 0x7f4a4006a120 2023-11-08T19:26:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.993+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a3003b3f0 con 0x7f4a4006a120 2023-11-08T19:26:25.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.994+0000 7f4a3d7fa640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2006ae70 0x7f4a2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.994+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f4a300ab380 con 0x7f4a4006a120 2023-11-08T19:26:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.995+0000 7f4a37fff640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2006ae70 0x7f4a2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.995+0000 7f4a37fff640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2006ae70 0x7f4a2006d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:25.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:25.998+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a30076630 con 0x7f4a4006a120 2023-11-08T19:26:26.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.175+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a40143b70 con 0x7f4a4006a120 2023-11-08T19:26:26.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.177+0000 7f4a3d7fa640 1 -- 172.21.15.119:0/3176805455 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13062 (secure 0 0 0) 0x7f4a30079c50 con 0x7f4a4006a120 2023-11-08T19:26:26.178 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:26.179 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:22.150203+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:26.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2006ae70 msgr2=0x7f4a2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2006ae70 0x7f4a2006d330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4a28006fd0 tx=0x7f4a28008040 comp rx=0 tx=0).stop 2023-11-08T19:26:26.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 msgr2=0x7f4a4015a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 0x7f4a4015a390 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f4a30034040 tx=0x7f4a300046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 shutdown_connections 2023-11-08T19:26:26.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2006ae70 0x7f4a2006d330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a401447a0 0x7f4a400c7160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a400cad30 0x7f4a400c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 --2- 172.21.15.119:0/3176805455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4006a120 0x7f4a4015a390 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 >> 172.21.15.119:0/3176805455 conn(0x7f4a400cb610 msgr2=0x7f4a400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 shutdown_connections 2023-11-08T19:26:26.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.179+0000 7f4a4654c640 1 -- 172.21.15.119:0/3176805455 wait complete. 2023-11-08T19:26:26.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:26.200 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:26.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.415+0000 7f09f754c640 1 Processor -- start 2023-11-08T19:26:26.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.415+0000 7f09f754c640 1 -- start start 2023-11-08T19:26:26.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f754c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09f0143600 0x7f09f00c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f754c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f09f015d9f0 0x7f09f015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f754c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f015e440 con 0x7f09f006cb40 2023-11-08T19:26:26.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f754c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f00c4f50 con 0x7f09f0143600 2023-11-08T19:26:26.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f754c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f00c4960 con 0x7f09f015d9f0 2023-11-08T19:26:26.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09f0143600 0x7f09f00c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f09f0143600 0x7f09f00c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51820/0 (socket says 172.21.15.119:51820) 2023-11-08T19:26:26.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 -- 172.21.15.119:0/712894466 learned_addr learned my addr 172.21.15.119:0/712894466 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:26.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f4ae7640 1 --2- 172.21.15.119:0/712894466 >> v2:172.21.15.188:3300/0 conn(0x7f09f015d9f0 0x7f09f015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 -- 172.21.15.119:0/712894466 >> v2:172.21.15.188:3300/0 conn(0x7f09f015d9f0 msgr2=0x7f09f015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:26.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 --2- 172.21.15.119:0/712894466 >> v2:172.21.15.188:3300/0 conn(0x7f09f015d9f0 0x7f09f015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 -- 172.21.15.119:0/712894466 >> v1:172.21.15.188:6789/0 conn(0x7f09f006cb40 legacy=0x7f09f006cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:26.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.417+0000 7f09f52e8640 1 -- 172.21.15.119:0/712894466 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f015e720 con 0x7f09f0143600 2023-11-08T19:26:26.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.419+0000 7f09f52e8640 1 --2- 172.21.15.119:0/712894466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f0143600 0x7f09f00c5370 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09f00c91e0 tx=0x7f09ec02e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49c4ad02f7a1ed1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.419+0000 7f09e7fff640 1 -- 172.21.15.119:0/712894466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec02ee10 con 0x7f09f0143600 2023-11-08T19:26:26.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.419+0000 7f09e7fff640 1 -- 172.21.15.119:0/712894466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec036440 con 0x7f09f0143600 2023-11-08T19:26:26.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.419+0000 7f09e7fff640 1 -- 172.21.15.119:0/712894466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0367d0 con 0x7f09f0143600 2023-11-08T19:26:26.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 -- 172.21.15.119:0/712894466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f0143600 msgr2=0x7f09f00c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 --2- 172.21.15.119:0/712894466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f0143600 0x7f09f00c5370 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f09f00c91e0 tx=0x7f09ec02e060 comp rx=0 tx=0).stop 2023-11-08T19:26:26.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 -- 172.21.15.119:0/712894466 shutdown_connections 2023-11-08T19:26:26.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 --2- 172.21.15.119:0/712894466 >> v2:172.21.15.188:3300/0 conn(0x7f09f015d9f0 0x7f09f015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 --2- 172.21.15.119:0/712894466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f0143600 0x7f09f00c5370 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 -- 172.21.15.119:0/712894466 >> 172.21.15.119:0/712894466 conn(0x7f09f00cb610 msgr2=0x7f09f00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 -- 172.21.15.119:0/712894466 shutdown_connections 2023-11-08T19:26:26.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.420+0000 7f09f754c640 1 -- 172.21.15.119:0/712894466 wait complete. 2023-11-08T19:26:26.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.422+0000 7f09f754c640 1 Processor -- start 2023-11-08T19:26:26.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.422+0000 7f09f754c640 1 -- start start 2023-11-08T19:26:26.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.423+0000 7f09f754c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f006cb40 0x7f09f015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.423+0000 7f09f754c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f0143600 0x7f09f00b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f754c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 0x7f09f00b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f754c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f015d1e0 con 0x7f09f015d9f0 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f754c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f00b3790 con 0x7f09f006cb40 2023-11-08T19:26:26.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f754c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f01f36a0 con 0x7f09f0143600 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f5ae9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 0x7f09f00b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f5ae9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 0x7f09f00b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51834/0 (socket says 172.21.15.119:51834) 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f5ae9640 1 -- 172.21.15.119:0/1873761653 learned_addr learned my addr 172.21.15.119:0/1873761653 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f4ae7640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f0143600 0x7f09f00b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f52e8640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f006cb40 0x7f09f015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.424+0000 7f09f5ae9640 1 -- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f0143600 msgr2=0x7f09f00b1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f5ae9640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f0143600 0x7f09f00b1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f5ae9640 1 -- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f006cb40 msgr2=0x7f09f015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f5ae9640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f006cb40 0x7f09f015cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f5ae9640 1 -- 172.21.15.119:0/1873761653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec00b770 con 0x7f09f015d9f0 2023-11-08T19:26:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f5ae9640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 0x7f09f00b2520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09e800c780 tx=0x7f09e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8003d80 con 0x7f09f015d9f0 2023-11-08T19:26:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e800ce30 con 0x7f09f015d9f0 2023-11-08T19:26:26.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e8005290 con 0x7f09f015d9f0 2023-11-08T19:26:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f00b2b80 con 0x7f09f015d9f0 2023-11-08T19:26:26.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.425+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f00b3140 con 0x7f09f015d9f0 2023-11-08T19:26:26.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.427+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f00b3900 con 0x7f09f015d9f0 2023-11-08T19:26:26.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.428+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09e80053f0 con 0x7f09f015d9f0 2023-11-08T19:26:26.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.428+0000 7f09e5ffb640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f09d406aea0 0x7f09d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.428+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f09e8088630 con 0x7f09f015d9f0 2023-11-08T19:26:26.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.429+0000 7f09f52e8640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f09d406aea0 0x7f09d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.429+0000 7f09f52e8640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f09d406aea0 0x7f09d406d360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09ec038000 tx=0x7f09ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.432+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09e8053980 con 0x7f09f015d9f0 2023-11-08T19:26:26.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.621+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09f0074430 con 0x7f09f015d9f0 2023-11-08T19:26:26.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.622+0000 7f09e5ffb640 1 -- 172.21.15.119:0/1873761653 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f09e8056fa0 con 0x7f09f015d9f0 2023-11-08T19:26:26.623 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:26.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f09d406aea0 msgr2=0x7f09d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f09d406aea0 0x7f09d406d360 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f09ec038000 tx=0x7f09ec008040 comp rx=0 tx=0).stop 2023-11-08T19:26:26.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 msgr2=0x7f09f00b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 0x7f09f00b2520 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f09e800c780 tx=0x7f09e800cc40 comp rx=0 tx=0).stop 2023-11-08T19:26:26.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 shutdown_connections 2023-11-08T19:26:26.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f09d406aea0 0x7f09d406d360 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09f015d9f0 0x7f09f00b2520 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09f0143600 0x7f09f00b1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 --2- 172.21.15.119:0/1873761653 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f09f006cb40 0x7f09f015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 >> 172.21.15.119:0/1873761653 conn(0x7f09f00cb610 msgr2=0x7f09f00cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.624+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 shutdown_connections 2023-11-08T19:26:26.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.625+0000 7f09f754c640 1 -- 172.21.15.119:0/1873761653 wait complete. 2023-11-08T19:26:26.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:26:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.843+0000 7f79db59e640 1 Processor -- start 2023-11-08T19:26:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.844+0000 7f79db59e640 1 -- start start 2023-11-08T19:26:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.845+0000 7f79db59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.845+0000 7f79db59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b18c0 0x7f79dc0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.845+0000 7f79db59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79dc06be20 con 0x7f79dc06b650 2023-11-08T19:26:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.845+0000 7f79db59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79dc06c100 con 0x7f79dc0b18c0 2023-11-08T19:26:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.845+0000 7f79db59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc06c3e0 con 0x7f79dc15aea0 2023-11-08T19:26:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.845+0000 7f79dad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b18c0 0x7f79dc0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79dad9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f79dc0b18c0 0x7f79dc0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51846/0 (socket says 172.21.15.119:51846) 2023-11-08T19:26:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79dad9d640 1 -- 172.21.15.119:0/2293366887 learned_addr learned my addr 172.21.15.119:0/2293366887 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79da59c640 1 --2- 172.21.15.119:0/2293366887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79dad9d640 1 -- 172.21.15.119:0/2293366887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 msgr2=0x7f79dc0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79dad9d640 1 --2- 172.21.15.119:0/2293366887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79dad9d640 1 -- 172.21.15.119:0/2293366887 >> v1:172.21.15.188:6789/0 conn(0x7f79dc06b650 legacy=0x7f79dc06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.846+0000 7f79dad9d640 1 -- 172.21.15.119:0/2293366887 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79dc06c6c0 con 0x7f79dc0b18c0 2023-11-08T19:26:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.847+0000 7f79dad9d640 1 --2- 172.21.15.119:0/2293366887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0b1d50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009e50 tx=0x7f79d402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b327702af7cf3263 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.848+0000 7f79d959a640 1 -- 172.21.15.119:0/2293366887 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d402caf0 con 0x7f79dc0b18c0 2023-11-08T19:26:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.848+0000 7f79d959a640 1 -- 172.21.15.119:0/2293366887 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d402cc50 con 0x7f79dc0b18c0 2023-11-08T19:26:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.848+0000 7f79d959a640 1 -- 172.21.15.119:0/2293366887 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d40343f0 con 0x7f79dc0b18c0 2023-11-08T19:26:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.848+0000 7f79db59e640 1 -- 172.21.15.119:0/2293366887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 msgr2=0x7f79dc0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.848+0000 7f79db59e640 1 --2- 172.21.15.119:0/2293366887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0b1d50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009e50 tx=0x7f79d402bf50 comp rx=0 tx=0).stop 2023-11-08T19:26:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.849+0000 7f79db59e640 1 -- 172.21.15.119:0/2293366887 shutdown_connections 2023-11-08T19:26:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.849+0000 7f79db59e640 1 --2- 172.21.15.119:0/2293366887 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0b1d50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.849+0000 7f79db59e640 1 --2- 172.21.15.119:0/2293366887 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.849+0000 7f79db59e640 1 -- 172.21.15.119:0/2293366887 >> 172.21.15.119:0/2293366887 conn(0x7f79dc0b0560 msgr2=0x7f79dc0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.849+0000 7f79db59e640 1 -- 172.21.15.119:0/2293366887 shutdown_connections 2023-11-08T19:26:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.849+0000 7f79db59e640 1 -- 172.21.15.119:0/2293366887 wait complete. 2023-11-08T19:26:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.851+0000 7f79db59e640 1 Processor -- start 2023-11-08T19:26:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.851+0000 7f79db59e640 1 -- start start 2023-11-08T19:26:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79db59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79dc06b650 0x7f79dc15b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79db59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79db59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc1f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79db59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0cae10 con 0x7f79dc0b18c0 2023-11-08T19:26:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79db59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0c9ff0 con 0x7f79dc06b650 2023-11-08T19:26:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79db59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79dc0ca290 con 0x7f79dc15aea0 2023-11-08T19:26:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51852/0 (socket says 172.21.15.119:51852) 2023-11-08T19:26:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 -- 172.21.15.119:0/64780567 learned_addr learned my addr 172.21.15.119:0/64780567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 -- 172.21.15.119:0/64780567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 msgr2=0x7f79dc1f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc1f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 -- 172.21.15.119:0/64780567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79dc06b650 msgr2=0x7f79dc15b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79dc06b650 0x7f79dc15b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.852+0000 7f79d9d9b640 1 -- 172.21.15.119:0/64780567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4009b30 con 0x7f79dc0b18c0 2023-11-08T19:26:26.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.853+0000 7f79d9d9b640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0c9ab0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c400 tx=0x7f79d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.853+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d00052d0 con 0x7f79dc0b18c0 2023-11-08T19:26:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.853+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d0005430 con 0x7f79dc0b18c0 2023-11-08T19:26:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.853+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc0ca570 con 0x7f79dc0b18c0 2023-11-08T19:26:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.853+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0003830 con 0x7f79dc0b18c0 2023-11-08T19:26:26.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.853+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79dc0cab30 con 0x7f79dc0b18c0 2023-11-08T19:26:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.854+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79dc0c5bd0 con 0x7f79dc0b18c0 2023-11-08T19:26:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.855+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79d0003990 con 0x7f79dc0b18c0 2023-11-08T19:26:26.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.856+0000 7f79caffd640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79bc06ae70 0x7f79bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.856+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7f79d0088990 con 0x7f79dc0b18c0 2023-11-08T19:26:26.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.856+0000 7f79da59c640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79bc06ae70 0x7f79bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.857+0000 7f79da59c640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79bc06ae70 0x7f79bc06d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0098f0 tx=0x7f79cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:26.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:26.860+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79dc0c5bd0 con 0x7f79dc0b18c0 2023-11-08T19:26:27.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.051+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f79dc0c5bd0 con 0x7f79dc0b18c0 2023-11-08T19:26:27.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.052+0000 7f79caffd640 1 -- 172.21.15.119:0/64780567 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7f79dc0c5bd0 con 0x7f79dc0b18c0 2023-11-08T19:26:27.054 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:27.055 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-11-08T19:26:27.055 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:26:27.055 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-11-08T19:26:27.055 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-11-08T19:26:27.056 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:26:27.056 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-11-08T19:26:27.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79bc06ae70 msgr2=0x7f79bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79bc06ae70 0x7f79bc06d330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0098f0 tx=0x7f79cc008040 comp rx=0 tx=0).stop 2023-11-08T19:26:27.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 msgr2=0x7f79dc0c9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0c9ab0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c400 tx=0x7f79d000c8c0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 shutdown_connections 2023-11-08T19:26:27.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f79bc06ae70 0x7f79bc06d330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f79dc15aea0 0x7f79dc1f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f79dc0b18c0 0x7f79dc0c9ab0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 --2- 172.21.15.119:0/64780567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f79dc06b650 0x7f79dc15b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 >> 172.21.15.119:0/64780567 conn(0x7f79dc0b0560 msgr2=0x7f79dc071120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:27.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 shutdown_connections 2023-11-08T19:26:27.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.055+0000 7f79db59e640 1 -- 172.21.15.119:0/64780567 wait complete. 2023-11-08T19:26:27.074 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:26:27.075 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:26:27.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.279+0000 7fbd8c5d0640 1 Processor -- start 2023-11-08T19:26:27.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.279+0000 7fbd8c5d0640 1 -- start start 2023-11-08T19:26:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.280+0000 7fbd8c5d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 0x7fbd8406a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8c5d0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd840cc6d0 0x7fbd840ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8c5d0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd84162ae0 con 0x7fbd8406ba30 2023-11-08T19:26:27.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8c5d0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd84165de0 con 0x7fbd840cc6d0 2023-11-08T19:26:27.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8c5d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cd050 con 0x7fbd84144690 2023-11-08T19:26:27.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8ab6d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbd8406ba30 0x7fbd8406bec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48166/0 (socket says 172.21.15.119:48166) 2023-11-08T19:26:27.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/3322394485 learned_addr learned my addr 172.21.15.119:0/3322394485 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:27.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709941745 0 0) 0x7fbd84162ae0 con 0x7fbd8406ba30 2023-11-08T19:26:27.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8a36c640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 0x7fbd8406a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd70003520 con 0x7fbd8406ba30 2023-11-08T19:26:27.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.281+0000 7fbd89b6b640 1 --2- 172.21.15.119:0/3322394485 >> v2:172.21.15.188:3300/0 conn(0x7fbd840cc6d0 0x7fbd840ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 797598075 0 0) 0x7fbd70003520 con 0x7fbd8406ba30 2023-11-08T19:26:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 msgr2=0x7fbd8406a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 0x7fbd8406a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 >> v2:172.21.15.188:3300/0 conn(0x7fbd840cc6d0 msgr2=0x7fbd840ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 --2- 172.21.15.119:0/3322394485 >> v2:172.21.15.188:3300/0 conn(0x7fbd840cc6d0 0x7fbd840ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd84165160 con 0x7fbd8406ba30 2023-11-08T19:26:27.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fbd7c0019f0 con 0x7fbd8406ba30 2023-11-08T19:26:27.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 >> v1:172.21.15.119:6789/0 conn(0x7fbd8406ba30 legacy=0x7fbd8406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 0x7fbd70003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70003fa0 0x7fbd700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd70004d00 0x7fbd70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd84165de0 con 0x7fbd70004d00 2023-11-08T19:26:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840cd050 con 0x7fbd70003fa0 2023-11-08T19:26:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7000ba80 con 0x7fbd84144690 2023-11-08T19:26:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.282+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd70004d00 0x7fbd70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd8a36c640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 0x7fbd70003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd89b6b640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70003fa0 0x7fbd700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 msgr2=0x7fbd70003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd84144690 0x7fbd70003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70003fa0 msgr2=0x7fbd700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:27.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70003fa0 0x7fbd700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/3322394485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd70003520 con 0x7fbd70004d00 2023-11-08T19:26:27.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.283+0000 7fbd89b6b640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70003fa0 0x7fbd700047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:27.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd70004d00 0x7fbd70005180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00a860 tx=0x7fbd700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c709e12f97fb2c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:27.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c034630 con 0x7fbd70004d00 2023-11-08T19:26:27.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c02f070 con 0x7fbd70004d00 2023-11-08T19:26:27.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8936a640 1 -- 172.21.15.119:0/3322394485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c034a30 con 0x7fbd70004d00 2023-11-08T19:26:27.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/3322394485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd70004d00 msgr2=0x7fbd70005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/3322394485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd70004d00 0x7fbd70005180 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c00a860 tx=0x7fbd700057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.284+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/3322394485 reap_dead start 2023-11-08T19:26:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.285+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/3322394485 shutdown_connections 2023-11-08T19:26:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.285+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/3322394485 >> 172.21.15.119:0/3322394485 conn(0x7fbd840cb610 msgr2=0x7fbd840cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.285+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/3322394485 shutdown_connections 2023-11-08T19:26:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.285+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/3322394485 wait complete. 2023-11-08T19:26:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.286+0000 7fbd8c5d0640 1 Processor -- start 2023-11-08T19:26:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.286+0000 7fbd8c5d0640 1 -- start start 2023-11-08T19:26:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.287+0000 7fbd8c5d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70004d00 0x7fbd84143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.287+0000 7fbd8c5d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd840b1870 0x7fbd84146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.287+0000 7fbd8c5d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 0x7fbd841466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.287+0000 7fbd8c5d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b2210 con 0x7fbd840b1cf0 2023-11-08T19:26:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.287+0000 7fbd8c5d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b2a20 con 0x7fbd70004d00 2023-11-08T19:26:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.287+0000 7fbd8c5d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd840b4cb0 con 0x7fbd840b1870 2023-11-08T19:26:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 0x7fbd841466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 0x7fbd841466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51866/0 (socket says 172.21.15.119:51866) 2023-11-08T19:26:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/446121117 learned_addr learned my addr 172.21.15.119:0/446121117 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd840b1870 msgr2=0x7fbd84146180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd840b1870 0x7fbd84146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8a36c640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70004d00 0x7fbd84143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70004d00 msgr2=0x7fbd84143e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70004d00 0x7fbd84143e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 -- 172.21.15.119:0/446121117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840b2460 con 0x7fbd840b1cf0 2023-11-08T19:26:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.288+0000 7fbd8ab6d640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 0x7fbd841466c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02cba0 tx=0x7fbd7c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.289+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c001fd0 con 0x7fbd840b1cf0 2023-11-08T19:26:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.289+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd7c034430 con 0x7fbd840b1cf0 2023-11-08T19:26:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.289+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd7c0347a0 con 0x7fbd840b1cf0 2023-11-08T19:26:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.289+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd840b2740 con 0x7fbd840b1cf0 2023-11-08T19:26:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.289+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd840c7060 con 0x7fbd840b1cf0 2023-11-08T19:26:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.290+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd840b4e20 con 0x7fbd840b1cf0 2023-11-08T19:26:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.291+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd7c031030 con 0x7fbd840b1cf0 2023-11-08T19:26:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.291+0000 7fbd7affd640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd6806aea0 0x7fbd6806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.292+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5177+0+0 (secure 0 0 0) 0x7fbd7c0aa820 con 0x7fbd840b1cf0 2023-11-08T19:26:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.292+0000 7fbd8a36c640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd6806aea0 0x7fbd6806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.292+0000 7fbd8a36c640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd6806aea0 0x7fbd6806d360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbd80006fd0 tx=0x7fbd80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:27.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.296+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd7c075af0 con 0x7fbd840b1cf0 2023-11-08T19:26:27.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:27.487+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbd84071d20 con 0x7fbd840b1cf0 2023-11-08T19:26:28.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.235+0000 7fbd7affd640 1 -- 172.21.15.119:0/446121117 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbd84071d20 con 0x7fbd840b1cf0 2023-11-08T19:26:28.237 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-11-08T19:26:28.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd6806aea0 msgr2=0x7fbd6806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd6806aea0 0x7fbd6806d360 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbd80006fd0 tx=0x7fbd80008040 comp rx=0 tx=0).stop 2023-11-08T19:26:28.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 msgr2=0x7fbd841466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 0x7fbd841466c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbd7c02cba0 tx=0x7fbd7c001d80 comp rx=0 tx=0).stop 2023-11-08T19:26:28.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 shutdown_connections 2023-11-08T19:26:28.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd6806aea0 0x7fbd6806d360 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd840b1cf0 0x7fbd841466c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd840b1870 0x7fbd84146180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 --2- 172.21.15.119:0/446121117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd70004d00 0x7fbd84143e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.239+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 >> 172.21.15.119:0/446121117 conn(0x7fbd840cb610 msgr2=0x7fbd840c5160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:28.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.240+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 shutdown_connections 2023-11-08T19:26:28.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.240+0000 7fbd8c5d0640 1 -- 172.21.15.119:0/446121117 wait complete. 2023-11-08T19:26:28.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:26:28.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.472+0000 7fcdc484f640 1 Processor -- start 2023-11-08T19:26:28.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.472+0000 7fcdc484f640 1 -- start start 2023-11-08T19:26:28.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.473+0000 7fcdc484f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cd020 0x7fcdc00ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.473+0000 7fcdc484f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcdc00cb890 0x7fcdc00ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdc484f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdc00c9400 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdc484f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdc00cd8c0 con 0x7fcdc00cb890 2023-11-08T19:26:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdc484f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00ad810 con 0x7fcdc00cd020 2023-11-08T19:26:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdbed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcdc00cb890 0x7fcdc00ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdbe59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcdc00cb4b0 0x7fcdc006f970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48168/0 (socket says 172.21.15.119:48168) 2023-11-08T19:26:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdbe59c640 1 -- 172.21.15.119:0/3059864907 learned_addr learned my addr 172.21.15.119:0/3059864907 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:28.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdb7fff640 1 --2- 172.21.15.119:0/3059864907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cd020 0x7fcdc00ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdbdd9b640 1 -- 172.21.15.119:0/3059864907 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597313508 0 0) 0x7fcdc00c9400 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.474+0000 7fcdbdd9b640 1 -- 172.21.15.119:0/3059864907 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdac003520 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.475+0000 7fcdbed9d640 1 -- 172.21.15.119:0/3059864907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cd020 msgr2=0x7fcdc00ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.475+0000 7fcdbed9d640 1 --2- 172.21.15.119:0/3059864907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cd020 0x7fcdc00ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.475+0000 7fcdbed9d640 1 -- 172.21.15.119:0/3059864907 >> v1:172.21.15.119:6789/0 conn(0x7fcdc00cb4b0 legacy=0x7fcdc006f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.475+0000 7fcdbed9d640 1 -- 172.21.15.119:0/3059864907 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdc00cd5e0 con 0x7fcdc00cb890 2023-11-08T19:26:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.476+0000 7fcdbed9d640 1 --2- 172.21.15.119:0/3059864907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb890 0x7fcdc00ca160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009d90 tx=0x7fcdb802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d921c1148992bca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.476+0000 7fcdbdd9b640 1 -- 172.21.15.119:0/3059864907 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb802cbc0 con 0x7fcdc00cb890 2023-11-08T19:26:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.476+0000 7fcdbdd9b640 1 -- 172.21.15.119:0/3059864907 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb802cd20 con 0x7fcdc00cb890 2023-11-08T19:26:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.476+0000 7fcdbdd9b640 1 -- 172.21.15.119:0/3059864907 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb8034640 con 0x7fcdc00cb890 2023-11-08T19:26:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.476+0000 7fcdc484f640 1 -- 172.21.15.119:0/3059864907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb890 msgr2=0x7fcdc00ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.476+0000 7fcdc484f640 1 --2- 172.21.15.119:0/3059864907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb890 0x7fcdc00ca160 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8009d90 tx=0x7fcdb802be10 comp rx=0 tx=0).stop 2023-11-08T19:26:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.477+0000 7fcdc484f640 1 -- 172.21.15.119:0/3059864907 shutdown_connections 2023-11-08T19:26:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.477+0000 7fcdc484f640 1 --2- 172.21.15.119:0/3059864907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb890 0x7fcdc00ca160 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.477+0000 7fcdc484f640 1 --2- 172.21.15.119:0/3059864907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cd020 0x7fcdc00ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.477+0000 7fcdc484f640 1 -- 172.21.15.119:0/3059864907 >> 172.21.15.119:0/3059864907 conn(0x7fcdc00b0950 msgr2=0x7fcdc00b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.477+0000 7fcdc484f640 1 -- 172.21.15.119:0/3059864907 shutdown_connections 2023-11-08T19:26:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.477+0000 7fcdc484f640 1 -- 172.21.15.119:0/3059864907 wait complete. 2023-11-08T19:26:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.479+0000 7fcdc484f640 1 Processor -- start 2023-11-08T19:26:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.479+0000 7fcdc484f640 1 -- start start 2023-11-08T19:26:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdc484f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 0x7fcdc00c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdc484f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cb890 0x7fcdc00c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdc484f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00cd020 0x7fcdc00c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdc484f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc00c89d0 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdc484f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc015b520 con 0x7fcdc00cd020 2023-11-08T19:26:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdc484f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc015b800 con 0x7fcdc00cb890 2023-11-08T19:26:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdbe59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 0x7fcdc00c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdbe59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 0x7fcdc00c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51888/0 (socket says 172.21.15.119:51888) 2023-11-08T19:26:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.480+0000 7fcdbe59c640 1 -- 172.21.15.119:0/1273308660 learned_addr learned my addr 172.21.15.119:0/1273308660 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdbe59c640 1 -- 172.21.15.119:0/1273308660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cb890 msgr2=0x7fcdc00c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdbe59c640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cb890 0x7fcdc00c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdbe59c640 1 -- 172.21.15.119:0/1273308660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00cd020 msgr2=0x7fcdc00c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdbe59c640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00cd020 0x7fcdc00c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdbe59c640 1 -- 172.21.15.119:0/1273308660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8009a70 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdbe59c640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 0x7fcdc00c64e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c390 tx=0x7fcdb000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0005ce0 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdb0005e40 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb00072f0 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc015bae0 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.481+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc015c0a0 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.482+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc004ece0 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.484+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcdb0007450 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.485+0000 7fcdb6ffd640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcda006ae70 0x7fcda006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.485+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5547+0+0 (secure 0 0 0) 0x7fcdb0089060 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.485+0000 7fcdb7fff640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcda006ae70 0x7fcda006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.486+0000 7fcdb7fff640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcda006ae70 0x7fcda006d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c5ef0 tx=0x7fcda8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.489+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcdb0054250 con 0x7fcdc00cb4b0 2023-11-08T19:26:28.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:28.683+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fcdc01462f0 con 0x7fcdc00cb4b0 2023-11-08T19:26:29.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.250+0000 7fcdb6ffd640 1 -- 172.21.15.119:0/1273308660 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7fcdc01462f0 con 0x7fcdc00cb4b0 2023-11-08T19:26:29.252 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-11-08T19:26:29.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcda006ae70 msgr2=0x7fcda006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcda006ae70 0x7fcda006d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c5ef0 tx=0x7fcda8008040 comp rx=0 tx=0).stop 2023-11-08T19:26:29.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 msgr2=0x7fcdc00c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 0x7fcdc00c64e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fcdb000c390 tx=0x7fcdb000c850 comp rx=0 tx=0).stop 2023-11-08T19:26:29.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 shutdown_connections 2023-11-08T19:26:29.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcda006ae70 0x7fcda006d330 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcdc00c5ef0 tx=0x7fcda8008040 comp rx=0 tx=0).stop 2023-11-08T19:26:29.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcdc00cd020 0x7fcdc00c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcdc00cb890 0x7fcdc00c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 --2- 172.21.15.119:0/1273308660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcdc00cb4b0 0x7fcdc00c64e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 >> 172.21.15.119:0/1273308660 conn(0x7fcdc00b0950 msgr2=0x7fcdc00c39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.254+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 shutdown_connections 2023-11-08T19:26:29.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.255+0000 7fcdc484f640 1 -- 172.21.15.119:0/1273308660 wait complete. 2023-11-08T19:26:29.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:26:29.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.477+0000 7f65d4947640 1 Processor -- start 2023-11-08T19:26:29.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.478+0000 7f65d4947640 1 -- start start 2023-11-08T19:26:29.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65d4947640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f65d0160530 0x7f65d00cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65d4947640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65d4947640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65d00d3ac0 0x7f65d00ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65d4947640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00ce920 con 0x7f65d00d3ac0 2023-11-08T19:26:29.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65d4947640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65d00b27c0 con 0x7f65d0160530 2023-11-08T19:26:29.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65d4947640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00b2aa0 con 0x7f65d0160910 2023-11-08T19:26:29.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65cffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65d00d3ac0 0x7f65d00ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65cffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f65d00d3ac0 0x7f65d00ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47570/0 (socket says 172.21.15.119:47570) 2023-11-08T19:26:29.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.479+0000 7f65cffff640 1 -- 172.21.15.119:0/2378534756 learned_addr learned my addr 172.21.15.119:0/2378534756 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:29.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.480+0000 7f65cffff640 1 -- 172.21.15.119:0/2378534756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 msgr2=0x7f65d00cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:29.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.480+0000 7f65cffff640 1 --2- 172.21.15.119:0/2378534756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.480+0000 7f65cffff640 1 -- 172.21.15.119:0/2378534756 >> v2:172.21.15.188:3300/0 conn(0x7f65d0160530 msgr2=0x7f65d00cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:29.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.480+0000 7f65cffff640 1 --2- 172.21.15.119:0/2378534756 >> v2:172.21.15.188:3300/0 conn(0x7f65d0160530 0x7f65d00cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.480+0000 7f65cffff640 1 -- 172.21.15.119:0/2378534756 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d00d1640 con 0x7f65d00d3ac0 2023-11-08T19:26:29.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.481+0000 7f65cffff640 1 --2- 172.21.15.119:0/2378534756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d00d3ac0 0x7f65d00ce3e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f65d006a7d0 tx=0x7f65c802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4f016cda938f0d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.481+0000 7f65ceffd640 1 -- 172.21.15.119:0/2378534756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c802ca70 con 0x7f65d00d3ac0 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65ceffd640 1 -- 172.21.15.119:0/2378534756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c802cbd0 con 0x7f65d00d3ac0 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65ceffd640 1 -- 172.21.15.119:0/2378534756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80343f0 con 0x7f65d00d3ac0 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 -- 172.21.15.119:0/2378534756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d00d3ac0 msgr2=0x7f65d00ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 --2- 172.21.15.119:0/2378534756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d00d3ac0 0x7f65d00ce3e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f65d006a7d0 tx=0x7f65c802bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 -- 172.21.15.119:0/2378534756 shutdown_connections 2023-11-08T19:26:29.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 --2- 172.21.15.119:0/2378534756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d00d3ac0 0x7f65d00ce3e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 --2- 172.21.15.119:0/2378534756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 --2- 172.21.15.119:0/2378534756 >> v2:172.21.15.188:3300/0 conn(0x7f65d0160530 0x7f65d00cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 -- 172.21.15.119:0/2378534756 >> 172.21.15.119:0/2378534756 conn(0x7f65d00d2ec0 msgr2=0x7f65d00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:29.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 -- 172.21.15.119:0/2378534756 shutdown_connections 2023-11-08T19:26:29.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.482+0000 7f65d4947640 1 -- 172.21.15.119:0/2378534756 wait complete. 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.484+0000 7f65d4947640 1 Processor -- start 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.484+0000 7f65d4947640 1 -- start start 2023-11-08T19:26:29.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.485+0000 7f65d4947640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65d00d3ac0 0x7f65d015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.485+0000 7f65d4947640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 0x7f65d01533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.485+0000 7f65d4947640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.485+0000 7f65d4947640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00c6890 con 0x7f65d0160530 2023-11-08T19:26:29.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.485+0000 7f65d4947640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00cacc0 con 0x7f65d00d3ac0 2023-11-08T19:26:29.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.485+0000 7f65d4947640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00cafa0 con 0x7f65d0160910 2023-11-08T19:26:29.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 0x7f65d01533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 0x7f65d01533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47576/0 (socket says 172.21.15.119:47576) 2023-11-08T19:26:29.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 -- 172.21.15.119:0/173925201 learned_addr learned my addr 172.21.15.119:0/173925201 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:29.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65cffff640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65cf7fe640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65d00d3ac0 0x7f65d015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 -- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 msgr2=0x7f65d00b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 -- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65d00d3ac0 msgr2=0x7f65d015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:29.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65d00d3ac0 0x7f65d015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:29.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 -- 172.21.15.119:0/173925201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c8009b30 con 0x7f65d0160530 2023-11-08T19:26:29.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65cf7fe640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65d00d3ac0 0x7f65d015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:29.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.486+0000 7f65c7fff640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 0x7f65d01533d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f65b800b410 tx=0x7f65b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.487+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8011830 con 0x7f65d0160530 2023-11-08T19:26:29.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.487+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65b80119b0 con 0x7f65d0160530 2023-11-08T19:26:29.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.487+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00cb280 con 0x7f65d0160530 2023-11-08T19:26:29.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.487+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65b8011d60 con 0x7f65d0160530 2023-11-08T19:26:29.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.487+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d00cb840 con 0x7f65d0160530 2023-11-08T19:26:29.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.488+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d00cba30 con 0x7f65d0160530 2023-11-08T19:26:29.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.489+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65b801a400 con 0x7f65d0160530 2023-11-08T19:26:29.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.490+0000 7f65ccff9640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f65b006aea0 0x7f65b006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:29.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.490+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5929+0+0 (secure 0 0 0) 0x7f65b8089830 con 0x7f65d0160530 2023-11-08T19:26:29.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.490+0000 7f65cf7fe640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f65b006aea0 0x7f65b006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:29.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.491+0000 7f65cf7fe640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f65b006aea0 0x7f65b006d360 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f65c000a8f0 tx=0x7f65c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:29.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.495+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65b80547f0 con 0x7f65d0160530 2023-11-08T19:26:29.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:29.720+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f65d004ece0 con 0x7f65d0160530 2023-11-08T19:26:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.622+0000 7f65ccff9640 1 -- 172.21.15.119:0/173925201 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f65b8057e10 con 0x7f65d0160530 2023-11-08T19:26:30.624 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 5 and data pool 6 2023-11-08T19:26:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.625+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f65b006aea0 msgr2=0x7f65b006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.625+0000 7f65d4947640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f65b006aea0 0x7f65b006d360 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f65c000a8f0 tx=0x7f65c0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.625+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 msgr2=0x7f65d01533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.625+0000 7f65d4947640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 0x7f65d01533d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f65b800b410 tx=0x7f65b800b8d0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 shutdown_connections 2023-11-08T19:26:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f65b006aea0 0x7f65b006d360 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65d0160910 0x7f65d00b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65d0160530 0x7f65d01533d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 --2- 172.21.15.119:0/173925201 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65d00d3ac0 0x7f65d015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 >> 172.21.15.119:0/173925201 conn(0x7f65d00d2ec0 msgr2=0x7f65d006c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 shutdown_connections 2023-11-08T19:26:30.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.626+0000 7f65d4947640 1 -- 172.21.15.119:0/173925201 wait complete. 2023-11-08T19:26:30.653 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:30.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.858+0000 7f1a23c34640 1 Processor -- start 2023-11-08T19:26:30.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.858+0000 7f1a23c34640 1 -- start start 2023-11-08T19:26:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.859+0000 7f1a23c34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a1c142f80 0x7f1a1c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.859+0000 7f1a23c34640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1c0c4960 con 0x7f1a1c15d9f0 2023-11-08T19:26:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.859+0000 7f1a23c34640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1c15df60 con 0x7f1a1c06cb40 2023-11-08T19:26:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.859+0000 7f1a23c34640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a1c143a40 con 0x7f1a1c142f80 2023-11-08T19:26:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a219d0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a1c142f80 0x7f1a1c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a211cf640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1a1c15d9f0 0x7f1a1c0c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41694/0 (socket says 172.21.15.119:41694) 2023-11-08T19:26:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a211cf640 1 -- 172.21.15.119:0/3885305988 learned_addr learned my addr 172.21.15.119:0/3885305988 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:30.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a209ce640 1 -- 172.21.15.119:0/3885305988 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507133096 0 0) 0x7f1a1c0c4960 con 0x7f1a1c15d9f0 2023-11-08T19:26:30.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a209ce640 1 -- 172.21.15.119:0/3885305988 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a04005730 con 0x7f1a1c15d9f0 2023-11-08T19:26:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a219d0640 1 -- 172.21.15.119:0/3885305988 >> v1:172.21.15.188:6789/0 conn(0x7f1a1c06cb40 legacy=0x7f1a1c0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a219d0640 1 -- 172.21.15.119:0/3885305988 >> v1:172.21.15.119:6789/0 conn(0x7f1a1c15d9f0 legacy=0x7f1a1c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.860+0000 7f1a219d0640 1 -- 172.21.15.119:0/3885305988 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1c15e240 con 0x7f1a1c142f80 2023-11-08T19:26:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.862+0000 7f1a219d0640 1 --2- 172.21.15.119:0/3885305988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c142f80 0x7f1a1c0cc6d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009d90 tx=0x7f1a0c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ed9af556fbdba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.862+0000 7f1a209ce640 1 -- 172.21.15.119:0/3885305988 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c02cd00 con 0x7f1a1c142f80 2023-11-08T19:26:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.862+0000 7f1a209ce640 1 -- 172.21.15.119:0/3885305988 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c02ce60 con 0x7f1a1c142f80 2023-11-08T19:26:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.862+0000 7f1a209ce640 1 -- 172.21.15.119:0/3885305988 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c034660 con 0x7f1a1c142f80 2023-11-08T19:26:30.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 -- 172.21.15.119:0/3885305988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c142f80 msgr2=0x7f1a1c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 --2- 172.21.15.119:0/3885305988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c142f80 0x7f1a1c0cc6d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009d90 tx=0x7f1a0c02bf50 comp rx=0 tx=0).stop 2023-11-08T19:26:30.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 -- 172.21.15.119:0/3885305988 shutdown_connections 2023-11-08T19:26:30.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 --2- 172.21.15.119:0/3885305988 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c142f80 0x7f1a1c0cc6d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 -- 172.21.15.119:0/3885305988 >> 172.21.15.119:0/3885305988 conn(0x7f1a1c0cb610 msgr2=0x7f1a1c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:30.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 -- 172.21.15.119:0/3885305988 shutdown_connections 2023-11-08T19:26:30.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.863+0000 7f1a23c34640 1 -- 172.21.15.119:0/3885305988 wait complete. 2023-11-08T19:26:30.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.864+0000 7f1a23c34640 1 Processor -- start 2023-11-08T19:26:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 -- start start 2023-11-08T19:26:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 0x7f1a1c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a1c142f80 0x7f1a1c1420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c15d9f0 0x7f1a1c142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0c6d00 con 0x7f1a1c06cb40 2023-11-08T19:26:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c1ef730 con 0x7f1a1c15d9f0 2023-11-08T19:26:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.865+0000 7f1a23c34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1c0c63d0 con 0x7f1a1c142f80 2023-11-08T19:26:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 0x7f1a1c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 0x7f1a1c1596a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47600/0 (socket says 172.21.15.119:47600) 2023-11-08T19:26:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 -- 172.21.15.119:0/1716584069 learned_addr learned my addr 172.21.15.119:0/1716584069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 -- 172.21.15.119:0/1716584069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a1c142f80 msgr2=0x7f1a1c1420c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a1c142f80 0x7f1a1c1420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 -- 172.21.15.119:0/1716584069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c15d9f0 msgr2=0x7f1a1c142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c15d9f0 0x7f1a1c142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 -- 172.21.15.119:0/1716584069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c009a70 con 0x7f1a1c06cb40 2023-11-08T19:26:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.866+0000 7f1a219d0640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 0x7f1a1c1596a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009d90 tx=0x7f1a0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.867+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c03d600 con 0x7f1a1c06cb40 2023-11-08T19:26:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.867+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c03d7a0 con 0x7f1a1c06cb40 2023-11-08T19:26:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.867+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c03db50 con 0x7f1a1c06cb40 2023-11-08T19:26:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.867+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1c0c66b0 con 0x7f1a1c06cb40 2023-11-08T19:26:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.867+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1c159be0 con 0x7f1a1c06cb40 2023-11-08T19:26:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.868+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a1c074430 con 0x7f1a1c06cb40 2023-11-08T19:26:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.869+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a0c03b040 con 0x7f1a1c06cb40 2023-11-08T19:26:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.870+0000 7f1a1a7fc640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a0806ae70 0x7f1a0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.870+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f1a0c0ab970 con 0x7f1a1c06cb40 2023-11-08T19:26:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.870+0000 7f1a211cf640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a0806ae70 0x7f1a0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.871+0000 7f1a211cf640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a0806ae70 0x7f1a0806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b0700 tx=0x7f1a10007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:30.874+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a0c0769f0 con 0x7f1a1c06cb40 2023-11-08T19:26:31.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.068+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a1c0c4230 con 0x7f1a1c06cb40 2023-11-08T19:26:31.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.070+0000 7f1a1a7fc640 1 -- 172.21.15.119:0/1716584069 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16787 (secure 0 0 0) 0x7f1a0c039070 con 0x7f1a1c06cb40 2023-11-08T19:26:31.072 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:31.073 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":38,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:29.722344+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:31.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a0806ae70 msgr2=0x7f1a0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a0806ae70 0x7f1a0806d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c0b0700 tx=0x7f1a10007930 comp rx=0 tx=0).stop 2023-11-08T19:26:31.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 msgr2=0x7f1a1c1596a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 0x7f1a1c1596a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009d90 tx=0x7f1a0c004a40 comp rx=0 tx=0).stop 2023-11-08T19:26:31.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 shutdown_connections 2023-11-08T19:26:31.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a0806ae70 0x7f1a0806d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1c15d9f0 0x7f1a1c142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a1c142f80 0x7f1a1c1420c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 --2- 172.21.15.119:0/1716584069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a1c06cb40 0x7f1a1c1596a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.073+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 >> 172.21.15.119:0/1716584069 conn(0x7f1a1c0cb610 msgr2=0x7f1a1c13fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.074+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 shutdown_connections 2023-11-08T19:26:31.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.074+0000 7f1a23c34640 1 -- 172.21.15.119:0/1716584069 wait complete. 2023-11-08T19:26:31.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.294+0000 7f295e00e640 1 Processor -- start 2023-11-08T19:26:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.294+0000 7f295e00e640 1 -- start start 2023-11-08T19:26:31.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.296+0000 7f295e00e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f295806a120 0x7f295806a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.296+0000 7f295e00e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295806aa70 con 0x7f29581447a0 2023-11-08T19:26:31.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.296+0000 7f295e00e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29580c3a50 con 0x7f295815f4c0 2023-11-08T19:26:31.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.296+0000 7f295e00e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295815f930 con 0x7f295806a120 2023-11-08T19:26:31.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.296+0000 7f29577fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f29581447a0 0x7f29580cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41698/0 (socket says 172.21.15.119:41698) 2023-11-08T19:26:31.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.296+0000 7f29577fe640 1 -- 172.21.15.119:0/3011620712 learned_addr learned my addr 172.21.15.119:0/3011620712 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:31.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2957fff640 1 --2- 172.21.15.119:0/3011620712 >> v2:172.21.15.188:3300/0 conn(0x7f295806a120 0x7f295806a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3876456600 0 0) 0x7f295806aa70 con 0x7f29581447a0 2023-11-08T19:26:31.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2944003520 con 0x7f29581447a0 2023-11-08T19:26:31.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 64364770 0 0) 0x7f2944003520 con 0x7f29581447a0 2023-11-08T19:26:31.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 >> v2:172.21.15.188:3300/0 conn(0x7f295806a120 msgr2=0x7f295806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 --2- 172.21.15.119:0/3011620712 >> v2:172.21.15.188:3300/0 conn(0x7f295806a120 0x7f295806a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 >> v1:172.21.15.188:6789/0 conn(0x7f295815f4c0 legacy=0x7f295806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2958144b80 con 0x7f29581447a0 2023-11-08T19:26:31.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2948001690 con 0x7f29581447a0 2023-11-08T19:26:31.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.297+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 >> v1:172.21.15.119:6789/0 conn(0x7f29581447a0 legacy=0x7f29580cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2956ffd640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29440039c0 0x7f2944003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2956ffd640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29440047c0 0x7f2944004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2956ffd640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2944005180 0x7f294400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295815f930 con 0x7f2944005180 2023-11-08T19:26:31.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294400c0b0 con 0x7f29440039c0 2023-11-08T19:26:31.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294400c270 con 0x7f29440047c0 2023-11-08T19:26:31.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2957fff640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2944005180 0x7f294400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2957fff640 1 -- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29440047c0 msgr2=0x7f2944004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2957fff640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29440047c0 0x7f2944004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2957fff640 1 -- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29440039c0 msgr2=0x7f2944003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f29577fe640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29440039c0 0x7f2944003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2957fff640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29440039c0 0x7f2944003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.298+0000 7f2957fff640 1 -- 172.21.15.119:0/3011620712 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2944003520 con 0x7f2944005180 2023-11-08T19:26:31.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.299+0000 7f2957fff640 1 -- 172.21.15.119:0/3011620712 reap_dead start 2023-11-08T19:26:31.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f2957fff640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2944005180 0x7f294400b980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2950009fc0 tx=0x7f29440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ce7bcbd70c46b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295002cc60 con 0x7f2944005180 2023-11-08T19:26:31.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f295000f040 con 0x7f2944005180 2023-11-08T19:26:31.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f2956ffd640 1 -- 172.21.15.119:0/3011620712 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950003990 con 0x7f2944005180 2023-11-08T19:26:31.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f295e00e640 1 -- 172.21.15.119:0/3011620712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2944005180 msgr2=0x7f294400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f295e00e640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2944005180 0x7f294400b980 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2950009fc0 tx=0x7f29440057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f295e00e640 1 -- 172.21.15.119:0/3011620712 shutdown_connections 2023-11-08T19:26:31.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f295e00e640 1 --2- 172.21.15.119:0/3011620712 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2944005180 0x7f294400b980 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.300+0000 7f295e00e640 1 -- 172.21.15.119:0/3011620712 >> 172.21.15.119:0/3011620712 conn(0x7f29580cb610 msgr2=0x7f29580cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.301+0000 7f295e00e640 1 -- 172.21.15.119:0/3011620712 shutdown_connections 2023-11-08T19:26:31.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.301+0000 7f295e00e640 1 -- 172.21.15.119:0/3011620712 wait complete. 2023-11-08T19:26:31.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.303+0000 7f295e00e640 1 Processor -- start 2023-11-08T19:26:31.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.303+0000 7f295e00e640 1 -- start start 2023-11-08T19:26:31.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f295e00e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2944005180 0x7f295815d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f295e00e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 0x7f295815cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f295e00e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29581f4270 0x7f29581f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f295e00e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29581f3e50 con 0x7f29580b3940 2023-11-08T19:26:31.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f295e00e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29581333c0 con 0x7f29581f4270 2023-11-08T19:26:31.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f295e00e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29581f3b40 con 0x7f2944005180 2023-11-08T19:26:31.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f294ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 0x7f295815cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f294ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 0x7f295815cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47626/0 (socket says 172.21.15.119:47626) 2023-11-08T19:26:31.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.304+0000 7f294ffff640 1 -- 172.21.15.119:0/4050934645 learned_addr learned my addr 172.21.15.119:0/4050934645 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:31.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f29577fe640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2944005180 0x7f295815d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f2957fff640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29581f4270 0x7f29581f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f294ffff640 1 -- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2944005180 msgr2=0x7f295815d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f294ffff640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2944005180 0x7f295815d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f294ffff640 1 -- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29581f4270 msgr2=0x7f29581f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f294ffff640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29581f4270 0x7f29581f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f294ffff640 1 -- 172.21.15.119:0/4050934645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29580b3dc0 con 0x7f29580b3940 2023-11-08T19:26:31.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f294ffff640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 0x7f295815cd80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f294000c400 tx=0x7f294000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29400052d0 con 0x7f29580b3940 2023-11-08T19:26:31.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2940005470 con 0x7f29580b3940 2023-11-08T19:26:31.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.305+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2940003830 con 0x7f29580b3940 2023-11-08T19:26:31.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.306+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29580b4020 con 0x7f29580b3940 2023-11-08T19:26:31.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.306+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29580b45e0 con 0x7f29580b3940 2023-11-08T19:26:31.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.307+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29581f40c0 con 0x7f29580b3940 2023-11-08T19:26:31.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.308+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2940003990 con 0x7f29580b3940 2023-11-08T19:26:31.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.309+0000 7f2954ff9640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f293806ae70 0x7f293806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.309+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2940088880 con 0x7f29580b3940 2023-11-08T19:26:31.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.309+0000 7f29577fe640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f293806ae70 0x7f293806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.310+0000 7f29577fe640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f293806ae70 0x7f293806d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2958071ea0 tx=0x7f2948009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.313+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2940057320 con 0x7f29580b3940 2023-11-08T19:26:31.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.510+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2958074430 con 0x7f29580b3940 2023-11-08T19:26:31.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.515+0000 7f2954ff9640 1 -- 172.21.15.119:0/4050934645 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f2940017090 con 0x7f29580b3940 2023-11-08T19:26:31.516 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:31.517 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:31.255061+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:31.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.517+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f293806ae70 msgr2=0x7f293806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.517+0000 7f295e00e640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f293806ae70 0x7f293806d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2958071ea0 tx=0x7f2948009040 comp rx=0 tx=0).stop 2023-11-08T19:26:31.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 msgr2=0x7f295815cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 0x7f295815cd80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f294000c400 tx=0x7f294000c8c0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 shutdown_connections 2023-11-08T19:26:31.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f293806ae70 0x7f293806d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f29581f4270 0x7f29581f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f29580b3940 0x7f295815cd80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 --2- 172.21.15.119:0/4050934645 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2944005180 0x7f295815d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 >> 172.21.15.119:0/4050934645 conn(0x7f29580cb610 msgr2=0x7f29580c52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 shutdown_connections 2023-11-08T19:26:31.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.518+0000 7f295e00e640 1 -- 172.21.15.119:0/4050934645 wait complete. 2023-11-08T19:26:31.538 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:26:31.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.747+0000 7f5d52939640 1 Processor -- start 2023-11-08T19:26:31.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.747+0000 7f5d52939640 1 -- start start 2023-11-08T19:26:31.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d52939640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d4c144580 0x7f5d4c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d52939640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d4c144960 0x7f5d4c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d52939640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d52939640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c0c9150 con 0x7f5d4c144960 2023-11-08T19:26:31.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d52939640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c15ea60 con 0x7f5d4c144580 2023-11-08T19:26:31.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d52939640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0cb270 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d4b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d4c144960 0x7f5d4c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d4b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d4c144960 0x7f5d4c0cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47638/0 (socket says 172.21.15.119:47638) 2023-11-08T19:26:31.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d4b7fe640 1 -- 172.21.15.119:0/195763623 learned_addr learned my addr 172.21.15.119:0/195763623 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:31.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d4bfff640 1 --2- 172.21.15.119:0/195763623 >> v2:172.21.15.188:3300/0 conn(0x7f5d4c144580 0x7f5d4c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d50ed6640 1 --2- 172.21.15.119:0/195763623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.749+0000 7f5d4b7fe640 1 -- 172.21.15.119:0/195763623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15f1d0 msgr2=0x7f5d4c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.750+0000 7f5d4b7fe640 1 --2- 172.21.15.119:0/195763623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.750+0000 7f5d4b7fe640 1 -- 172.21.15.119:0/195763623 >> v2:172.21.15.188:3300/0 conn(0x7f5d4c144580 msgr2=0x7f5d4c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:31.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.750+0000 7f5d4b7fe640 1 --2- 172.21.15.119:0/195763623 >> v2:172.21.15.188:3300/0 conn(0x7f5d4c144580 0x7f5d4c15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.750+0000 7f5d4b7fe640 1 -- 172.21.15.119:0/195763623 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c0c9360 con 0x7f5d4c144960 2023-11-08T19:26:31.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.750+0000 7f5d4bfff640 1 --2- 172.21.15.119:0/195763623 >> v2:172.21.15.188:3300/0 conn(0x7f5d4c144580 0x7f5d4c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:31.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.751+0000 7f5d4b7fe640 1 --2- 172.21.15.119:0/195763623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c144960 0x7f5d4c0cad30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ceab0 tx=0x7f5d4002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713f47e41e16f55 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.751+0000 7f5d4affd640 1 -- 172.21.15.119:0/195763623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400317e0 con 0x7f5d4c144960 2023-11-08T19:26:31.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.751+0000 7f5d4affd640 1 -- 172.21.15.119:0/195763623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40031940 con 0x7f5d4c144960 2023-11-08T19:26:31.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.751+0000 7f5d4affd640 1 -- 172.21.15.119:0/195763623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40031cd0 con 0x7f5d4c144960 2023-11-08T19:26:31.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 -- 172.21.15.119:0/195763623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c144960 msgr2=0x7f5d4c0cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 --2- 172.21.15.119:0/195763623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c144960 0x7f5d4c0cad30 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0ceab0 tx=0x7f5d4002b880 comp rx=0 tx=0).stop 2023-11-08T19:26:31.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 -- 172.21.15.119:0/195763623 shutdown_connections 2023-11-08T19:26:31.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 --2- 172.21.15.119:0/195763623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 --2- 172.21.15.119:0/195763623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c144960 0x7f5d4c0cad30 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 --2- 172.21.15.119:0/195763623 >> v2:172.21.15.188:3300/0 conn(0x7f5d4c144580 0x7f5d4c15bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 -- 172.21.15.119:0/195763623 >> 172.21.15.119:0/195763623 conn(0x7f5d4c0cb610 msgr2=0x7f5d4c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:31.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 -- 172.21.15.119:0/195763623 shutdown_connections 2023-11-08T19:26:31.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.752+0000 7f5d52939640 1 -- 172.21.15.119:0/195763623 wait complete. 2023-11-08T19:26:31.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.754+0000 7f5d52939640 1 Processor -- start 2023-11-08T19:26:31.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.754+0000 7f5d52939640 1 -- start start 2023-11-08T19:26:31.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d52939640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c144580 0x7f5d4c15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d52939640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c144960 0x7f5d4c13dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d52939640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d52939640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0b4bd0 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d52939640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c15acc0 con 0x7f5d4c144580 2023-11-08T19:26:31.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d52939640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c13e120 con 0x7f5d4c144960 2023-11-08T19:26:31.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d50ed6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d50ed6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c7a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47654/0 (socket says 172.21.15.119:47654) 2023-11-08T19:26:31.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.755+0000 7f5d50ed6640 1 -- 172.21.15.119:0/3978229579 learned_addr learned my addr 172.21.15.119:0/3978229579 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:31.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d4bfff640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c144580 0x7f5d4c15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d4b7fe640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c144960 0x7f5d4c13dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d50ed6640 1 -- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c144960 msgr2=0x7f5d4c13dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d50ed6640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c144960 0x7f5d4c13dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d50ed6640 1 -- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c144580 msgr2=0x7f5d4c15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:31.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d50ed6640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c144580 0x7f5d4c15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:31.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d50ed6640 1 -- 172.21.15.119:0/3978229579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400094d0 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d4bfff640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c144580 0x7f5d4c15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:31.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d50ed6640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c7a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cc80 tx=0x7f5d4400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4400ce30 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44003990 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44003d20 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c13cb40 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.756+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c13d100 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.758+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c074430 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.758+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d44003e80 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.759+0000 7f5d48ff9640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:31.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.759+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f5d44089b40 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.760+0000 7f5d4bfff640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:31.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.760+0000 7f5d4bfff640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0c7950 tx=0x7f5d3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:31.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.764+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d44054a10 con 0x7f5d4c15f1d0 2023-11-08T19:26:31.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:31.992+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5d4c06cb10 con 0x7f5d4c15f1d0 2023-11-08T19:26:32.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.643+0000 7f5d48ff9640 1 -- 172.21.15.119:0/3978229579 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5d4401d070 con 0x7f5d4c15f1d0 2023-11-08T19:26:32.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 msgr2=0x7f5d3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0c7950 tx=0x7f5d3c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:32.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 msgr2=0x7f5d4c0c7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c7a00 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400cc80 tx=0x7f5d4400a5c0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 shutdown_connections 2023-11-08T19:26:32.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15f1d0 0x7f5d4c0c7a00 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c144960 0x7f5d4c13dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.646+0000 7f5d52939640 1 --2- 172.21.15.119:0/3978229579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c144580 0x7f5d4c15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.647+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 >> 172.21.15.119:0/3978229579 conn(0x7f5d4c0cb610 msgr2=0x7f5d4c0cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.647+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 shutdown_connections 2023-11-08T19:26:32.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.647+0000 7f5d52939640 1 -- 172.21.15.119:0/3978229579 wait complete. 2023-11-08T19:26:32.669 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:26:32.670 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:26:32.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.883+0000 7f29327d0640 1 Processor -- start 2023-11-08T19:26:32.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.883+0000 7f29327d0640 1 -- start start 2023-11-08T19:26:32.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.885+0000 7f29327d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 0x7f292c0cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.885+0000 7f29327d0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292c15df50 con 0x7f292c0cfa00 2023-11-08T19:26:32.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.885+0000 7f29327d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292c0d1e60 con 0x7f292c15d4a0 2023-11-08T19:26:32.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.885+0000 7f29327d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292c0d03b0 con 0x7f292c161870 2023-11-08T19:26:32.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.885+0000 7f2923fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f292c0cfa00 0x7f292c161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41710/0 (socket says 172.21.15.119:41710) 2023-11-08T19:26:32.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.885+0000 7f2923fff640 1 -- 172.21.15.119:0/3595919370 learned_addr learned my addr 172.21.15.119:0/3595919370 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:32.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f2930d6d640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 0x7f292c0cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252977460 0 0) 0x7f292c15df50 con 0x7f292c0cfa00 2023-11-08T19:26:32.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2918003520 con 0x7f292c0cfa00 2023-11-08T19:26:32.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3902913772 0 0) 0x7f2918003520 con 0x7f292c0cfa00 2023-11-08T19:26:32.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 msgr2=0x7f292c0cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:32.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 0x7f292c0cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 >> v1:172.21.15.188:6789/0 conn(0x7f292c15d4a0 legacy=0x7f292c0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c161d40 con 0x7f292c0cfa00 2023-11-08T19:26:32.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2914001670 con 0x7f292c0cfa00 2023-11-08T19:26:32.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.886+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 >> v1:172.21.15.119:6789/0 conn(0x7f292c0cfa00 legacy=0x7f292c161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292b7fe640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 0x7f29180039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292b7fe640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2918003f70 0x7f29180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292b7fe640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918004d00 0x7f2918005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c0d03b0 con 0x7f2918003f70 2023-11-08T19:26:32.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291800bb70 con 0x7f2918004d00 2023-11-08T19:26:32.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291800bd30 con 0x7f292c161870 2023-11-08T19:26:32.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2923fff640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2918003f70 0x7f29180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f292bfff640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 0x7f29180039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2930d6d640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918004d00 0x7f2918005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2923fff640 1 -- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 msgr2=0x7f29180039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:32.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2923fff640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f292c161870 0x7f29180039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2923fff640 1 -- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918004d00 msgr2=0x7f2918005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:32.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2923fff640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2918004d00 0x7f2918005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.887+0000 7f2923fff640 1 -- 172.21.15.119:0/3595919370 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2918003520 con 0x7f2918003f70 2023-11-08T19:26:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.888+0000 7f2923fff640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2918003f70 0x7f29180047c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f291400c0e0 tx=0x7f29180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bc285a5b0b593a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914005ad0 con 0x7f2918003f70 2023-11-08T19:26:32.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2914005c30 con 0x7f2918003f70 2023-11-08T19:26:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f292b7fe640 1 -- 172.21.15.119:0/3595919370 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291402ca50 con 0x7f2918003f70 2023-11-08T19:26:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f29327d0640 1 -- 172.21.15.119:0/3595919370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2918003f70 msgr2=0x7f29180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f29327d0640 1 --2- 172.21.15.119:0/3595919370 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2918003f70 0x7f29180047c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f291400c0e0 tx=0x7f29180057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f2930d6d640 1 -- 172.21.15.119:0/3595919370 reap_dead start 2023-11-08T19:26:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f29327d0640 1 -- 172.21.15.119:0/3595919370 shutdown_connections 2023-11-08T19:26:32.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f29327d0640 1 -- 172.21.15.119:0/3595919370 >> 172.21.15.119:0/3595919370 conn(0x7f292c06a9f0 msgr2=0x7f292c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:32.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.889+0000 7f29327d0640 1 -- 172.21.15.119:0/3595919370 shutdown_connections 2023-11-08T19:26:32.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.890+0000 7f29327d0640 1 -- 172.21.15.119:0/3595919370 wait complete. 2023-11-08T19:26:32.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.891+0000 7f29327d0640 1 Processor -- start 2023-11-08T19:26:32.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.891+0000 7f29327d0640 1 -- start start 2023-11-08T19:26:32.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.892+0000 7f29327d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2918003f70 0x7f292c157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.892+0000 7f29327d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 0x7f292c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.892+0000 7f29327d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f292c0aaba0 0x7f292c0ab020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.892+0000 7f29327d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c0ab560 con 0x7f292c1f3250 2023-11-08T19:26:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.892+0000 7f29327d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c0ca0c0 con 0x7f292c0aaba0 2023-11-08T19:26:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.892+0000 7f29327d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292c1f3d10 con 0x7f2918003f70 2023-11-08T19:26:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 0x7f292c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 0x7f292c1f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47674/0 (socket says 172.21.15.119:47674) 2023-11-08T19:26:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 -- 172.21.15.119:0/2386645300 learned_addr learned my addr 172.21.15.119:0/2386645300 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:32.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f292bfff640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2918003f70 0x7f292c157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 -- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2918003f70 msgr2=0x7f292c157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2918003f70 0x7f292c157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 -- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f292c0aaba0 msgr2=0x7f292c0ab020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f292c0aaba0 0x7f292c0ab020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:32.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 -- 172.21.15.119:0/2386645300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c1f3ff0 con 0x7f292c1f3250 2023-11-08T19:26:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.893+0000 7f2923fff640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 0x7f292c1f36d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f291400c0e0 tx=0x7f2914001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.894+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2914039070 con 0x7f292c1f3250 2023-11-08T19:26:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.894+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291402cc20 con 0x7f292c1f3250 2023-11-08T19:26:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.894+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291403c040 con 0x7f292c1f3250 2023-11-08T19:26:32.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.894+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292c0ca310 con 0x7f292c1f3250 2023-11-08T19:26:32.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.894+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292c0ccac0 con 0x7f292c1f3250 2023-11-08T19:26:32.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.895+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292c0d10f0 con 0x7f292c1f3250 2023-11-08T19:26:32.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.896+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2914032070 con 0x7f292c1f3250 2023-11-08T19:26:32.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.897+0000 7f29297fa640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f290c06aea0 0x7f290c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.897+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f29140aa920 con 0x7f292c1f3250 2023-11-08T19:26:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.897+0000 7f292bfff640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f290c06aea0 0x7f290c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.898+0000 7f292bfff640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f290c06aea0 0x7f290c06d360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f291c007fd0 tx=0x7f291c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:32.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:32.901+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2914075880 con 0x7f292c1f3250 2023-11-08T19:26:33.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.129+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f292c0ce960 con 0x7f292c1f3250 2023-11-08T19:26:33.143 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:26:33.142+0000 7f60437d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:26:33.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.652+0000 7f29297fa640 1 -- 172.21.15.119:0/2386645300 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2914078ea0 con 0x7f292c1f3250 2023-11-08T19:26:33.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f290c06aea0 msgr2=0x7f290c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:33.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f290c06aea0 0x7f290c06d360 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f291c007fd0 tx=0x7f291c009040 comp rx=0 tx=0).stop 2023-11-08T19:26:33.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 msgr2=0x7f292c1f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:33.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 0x7f292c1f36d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f291400c0e0 tx=0x7f2914001930 comp rx=0 tx=0).stop 2023-11-08T19:26:33.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 shutdown_connections 2023-11-08T19:26:33.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f290c06aea0 0x7f290c06d360 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f292c0aaba0 0x7f292c0ab020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f292c1f3250 0x7f292c1f36d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 --2- 172.21.15.119:0/2386645300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2918003f70 0x7f292c157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.655+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 >> 172.21.15.119:0/2386645300 conn(0x7f292c06a9f0 msgr2=0x7f292c15ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:33.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.656+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 shutdown_connections 2023-11-08T19:26:33.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.656+0000 7f29327d0640 1 -- 172.21.15.119:0/2386645300 wait complete. 2023-11-08T19:26:33.679 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:26:33.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:33.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.885+0000 7f252c84b640 1 Processor -- start 2023-11-08T19:26:33.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.885+0000 7f252c84b640 1 -- start start 2023-11-08T19:26:33.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252c84b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2528142f80 0x7f25280cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252c84b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f252815d9f0 0x7f25280c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252c84b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252806cb40 0x7f252806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252c84b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2528143a40 con 0x7f2528142f80 2023-11-08T19:26:33.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252c84b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252806d510 con 0x7f252815d9f0 2023-11-08T19:26:33.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252c84b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25280c4960 con 0x7f252806cb40 2023-11-08T19:26:33.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2528142f80 0x7f25280cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2528142f80 0x7f25280cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47684/0 (socket says 172.21.15.119:47684) 2023-11-08T19:26:33.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.887+0000 7f252659c640 1 -- 172.21.15.119:0/3426950485 learned_addr learned my addr 172.21.15.119:0/3426950485 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:33.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f2525d9b640 1 --2- 172.21.15.119:0/3426950485 >> v2:172.21.15.188:3300/0 conn(0x7f252815d9f0 0x7f25280c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f2526d9d640 1 --2- 172.21.15.119:0/3426950485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252806cb40 0x7f252806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f252659c640 1 -- 172.21.15.119:0/3426950485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252806cb40 msgr2=0x7f252806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:33.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f252659c640 1 --2- 172.21.15.119:0/3426950485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252806cb40 0x7f252806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f252659c640 1 -- 172.21.15.119:0/3426950485 >> v2:172.21.15.188:3300/0 conn(0x7f252815d9f0 msgr2=0x7f25280c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f252659c640 1 --2- 172.21.15.119:0/3426950485 >> v2:172.21.15.188:3300/0 conn(0x7f252815d9f0 0x7f25280c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.888+0000 7f252659c640 1 -- 172.21.15.119:0/3426950485 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252815df60 con 0x7f2528142f80 2023-11-08T19:26:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.889+0000 7f252659c640 1 --2- 172.21.15.119:0/3426950485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2528142f80 0x7f25280cc6d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f252806df40 tx=0x7f251802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d390b76bb5b4b2b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:33.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.890+0000 7f252559a640 1 -- 172.21.15.119:0/3426950485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518031870 con 0x7f2528142f80 2023-11-08T19:26:33.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.890+0000 7f252559a640 1 -- 172.21.15.119:0/3426950485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180319d0 con 0x7f2528142f80 2023-11-08T19:26:33.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.890+0000 7f252559a640 1 -- 172.21.15.119:0/3426950485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518031d60 con 0x7f2528142f80 2023-11-08T19:26:33.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.890+0000 7f252c84b640 1 -- 172.21.15.119:0/3426950485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2528142f80 msgr2=0x7f25280cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:33.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.890+0000 7f252c84b640 1 --2- 172.21.15.119:0/3426950485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2528142f80 0x7f25280cc6d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f252806df40 tx=0x7f251802b910 comp rx=0 tx=0).stop 2023-11-08T19:26:33.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 -- 172.21.15.119:0/3426950485 shutdown_connections 2023-11-08T19:26:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 --2- 172.21.15.119:0/3426950485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252806cb40 0x7f252806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 --2- 172.21.15.119:0/3426950485 >> v2:172.21.15.188:3300/0 conn(0x7f252815d9f0 0x7f25280c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 --2- 172.21.15.119:0/3426950485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2528142f80 0x7f25280cc6d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 -- 172.21.15.119:0/3426950485 >> 172.21.15.119:0/3426950485 conn(0x7f25280cb610 msgr2=0x7f25280cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 -- 172.21.15.119:0/3426950485 shutdown_connections 2023-11-08T19:26:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.891+0000 7f252c84b640 1 -- 172.21.15.119:0/3426950485 wait complete. 2023-11-08T19:26:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.892+0000 7f252c84b640 1 Processor -- start 2023-11-08T19:26:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 -- start start 2023-11-08T19:26:33.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 0x7f25281591b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2528142f80 0x7f2528158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252815d9f0 0x7f252813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2528159a30 con 0x7f252806cb40 2023-11-08T19:26:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25280c7290 con 0x7f2528142f80 2023-11-08T19:26:33.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.893+0000 7f252c84b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25280c7570 con 0x7f252815d9f0 2023-11-08T19:26:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 0x7f25281591b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 0x7f25281591b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47686/0 (socket says 172.21.15.119:47686) 2023-11-08T19:26:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 -- 172.21.15.119:0/66079257 learned_addr learned my addr 172.21.15.119:0/66079257 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f2525d9b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2528142f80 0x7f2528158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 -- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252815d9f0 msgr2=0x7f252813ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f2526d9d640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252815d9f0 0x7f252813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252815d9f0 0x7f252813ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 -- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2528142f80 msgr2=0x7f2528158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2528142f80 0x7f2528158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:33.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 -- 172.21.15.119:0/66079257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180094d0 con 0x7f252806cb40 2023-11-08T19:26:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f2525d9b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2528142f80 0x7f2528158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.894+0000 7f252659c640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 0x7f25281591b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f251802b950 tx=0x7f25180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.895+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251803ad30 con 0x7f252806cb40 2023-11-08T19:26:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.895+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25180443f0 con 0x7f252806cb40 2023-11-08T19:26:33.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.895+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25280c7850 con 0x7f252806cb40 2023-11-08T19:26:33.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.895+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25180447a0 con 0x7f252806cb40 2023-11-08T19:26:33.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.895+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25280c7e10 con 0x7f252806cb40 2023-11-08T19:26:33.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.896+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25280c8000 con 0x7f252806cb40 2023-11-08T19:26:33.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.897+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f251804d450 con 0x7f252806cb40 2023-11-08T19:26:33.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.898+0000 7f2516ffd640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f250806ae70 0x7f250806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:33.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.898+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f251802d040 con 0x7f252806cb40 2023-11-08T19:26:33.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.898+0000 7f2525d9b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f250806ae70 0x7f250806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:33.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.899+0000 7f2525d9b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f250806ae70 0x7f250806d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25281f1fa0 tx=0x7f251c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:33.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:33.903+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2518076cd0 con 0x7f252806cb40 2023-11-08T19:26:34.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.128+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2528074430 con 0x7f252806cb40 2023-11-08T19:26:34.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.129+0000 7f2516ffd640 1 -- 172.21.15.119:0/66079257 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f2518035070 con 0x7f252806cb40 2023-11-08T19:26:34.131 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:34.132 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:29.722778+0000","modified":"2023-11-08T19:26:33.132121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/4172930163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4172930163},{"type":"v1","addr":"172.21.15.188:6834","nonce":4172930163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:34.132 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-11-08T19:26:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f250806ae70 msgr2=0x7f250806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f250806ae70 0x7f250806d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f25281f1fa0 tx=0x7f251c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 msgr2=0x7f25281591b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 0x7f25281591b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f251802b950 tx=0x7f25180046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 shutdown_connections 2023-11-08T19:26:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f250806ae70 0x7f250806d330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252815d9f0 0x7f252813ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2528142f80 0x7f2528158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 --2- 172.21.15.119:0/66079257 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252806cb40 0x7f25281591b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.131+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 >> 172.21.15.119:0/66079257 conn(0x7f25280cb610 msgr2=0x7f25280cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.132+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 shutdown_connections 2023-11-08T19:26:34.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.132+0000 7f252c84b640 1 -- 172.21.15.119:0/66079257 wait complete. 2023-11-08T19:26:34.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:34.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.362+0000 7f11eaca2640 1 Processor -- start 2023-11-08T19:26:34.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.362+0000 7f11eaca2640 1 -- start start 2023-11-08T19:26:34.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.364+0000 7f11eaca2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11e41447a0 0x7f11e415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.364+0000 7f11eaca2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e40c3a50 con 0x7f11e406a120 2023-11-08T19:26:34.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.364+0000 7f11eaca2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e4144b80 con 0x7f11e40cad30 2023-11-08T19:26:34.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.364+0000 7f11eaca2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e4070040 con 0x7f11e41447a0 2023-11-08T19:26:34.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.364+0000 7f11e923f640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f11e406a120 0x7f11e406a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41716/0 (socket says 172.21.15.119:41716) 2023-11-08T19:26:34.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.364+0000 7f11e923f640 1 -- 172.21.15.119:0/784230181 learned_addr learned my addr 172.21.15.119:0/784230181 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:34.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e8a3e640 1 --2- 172.21.15.119:0/784230181 >> v2:172.21.15.188:3300/0 conn(0x7f11e41447a0 0x7f11e415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439856761 0 0) 0x7f11e40c3a50 con 0x7f11e406a120 2023-11-08T19:26:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d4003520 con 0x7f11e406a120 2023-11-08T19:26:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939671837 0 0) 0x7f11d4003520 con 0x7f11e406a120 2023-11-08T19:26:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 >> v2:172.21.15.188:3300/0 conn(0x7f11e41447a0 msgr2=0x7f11e415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 --2- 172.21.15.119:0/784230181 >> v2:172.21.15.188:3300/0 conn(0x7f11e41447a0 0x7f11e415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 >> v1:172.21.15.188:6789/0 conn(0x7f11e40cad30 legacy=0x7f11e406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40c9150 con 0x7f11e406a120 2023-11-08T19:26:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f11dc001a10 con 0x7f11e406a120 2023-11-08T19:26:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 >> v1:172.21.15.119:6789/0 conn(0x7f11e406a120 legacy=0x7f11e406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d40039c0 0x7f11d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40047c0 0x7f11d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11d40051d0 0x7f11d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e4070040 con 0x7f11d40051d0 2023-11-08T19:26:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d400c120 con 0x7f11d40047c0 2023-11-08T19:26:34.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.365+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d400c2e0 con 0x7f11d40039c0 2023-11-08T19:26:34.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e923f640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11d40051d0 0x7f11d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e3fff640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40047c0 0x7f11d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e8a3e640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d40039c0 0x7f11d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e923f640 1 -- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d40039c0 msgr2=0x7f11d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e923f640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11d40039c0 0x7f11d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e923f640 1 -- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40047c0 msgr2=0x7f11d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e923f640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40047c0 0x7f11d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.366+0000 7f11e923f640 1 -- 172.21.15.119:0/784230181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d4003520 con 0x7f11d40051d0 2023-11-08T19:26:34.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.367+0000 7f11e923f640 1 -- 172.21.15.119:0/784230181 reap_dead start 2023-11-08T19:26:34.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.367+0000 7f11e923f640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11d40051d0 0x7f11d400b980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00a7e0 tx=0x7f11d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9eff6d46aafd906 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.367+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc034630 con 0x7f11d40051d0 2023-11-08T19:26:34.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.367+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc02f070 con 0x7f11d40051d0 2023-11-08T19:26:34.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.367+0000 7f11e37fe640 1 -- 172.21.15.119:0/784230181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc0349f0 con 0x7f11d40051d0 2023-11-08T19:26:34.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 -- 172.21.15.119:0/784230181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11d40051d0 msgr2=0x7f11d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11d40051d0 0x7f11d400b980 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f11dc00a7e0 tx=0x7f11d40057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 -- 172.21.15.119:0/784230181 shutdown_connections 2023-11-08T19:26:34.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 --2- 172.21.15.119:0/784230181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11d40051d0 0x7f11d400b980 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 -- 172.21.15.119:0/784230181 >> 172.21.15.119:0/784230181 conn(0x7f11e40cb610 msgr2=0x7f11e40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 -- 172.21.15.119:0/784230181 shutdown_connections 2023-11-08T19:26:34.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.368+0000 7f11eaca2640 1 -- 172.21.15.119:0/784230181 wait complete. 2023-11-08T19:26:34.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.370+0000 7f11eaca2640 1 Processor -- start 2023-11-08T19:26:34.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.370+0000 7f11eaca2640 1 -- start start 2023-11-08T19:26:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11eaca2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40051d0 0x7f11e415a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11eaca2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11e40c4110 0x7f11e40c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11eaca2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 0x7f11e40c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11eaca2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c4ad0 con 0x7f11e40c5e00 2023-11-08T19:26:34.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11eaca2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c6b30 con 0x7f11d40051d0 2023-11-08T19:26:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11eaca2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c6e10 con 0x7f11e40c4110 2023-11-08T19:26:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.371+0000 7f11e923f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 0x7f11e40c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 0x7f11e40c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47702/0 (socket says 172.21.15.119:47702) 2023-11-08T19:26:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 -- 172.21.15.119:0/3793970187 learned_addr learned my addr 172.21.15.119:0/3793970187 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:34.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e3fff640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11e40c4110 0x7f11e40c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e8a3e640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40051d0 0x7f11e415a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 -- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11e40c4110 msgr2=0x7f11e40c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11e40c4110 0x7f11e40c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 -- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40051d0 msgr2=0x7f11e415a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40051d0 0x7f11e415a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 -- 172.21.15.119:0/3793970187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40c70f0 con 0x7f11e40c5e00 2023-11-08T19:26:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.372+0000 7f11e923f640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 0x7f11e40c6280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000f80 tx=0x7f11dc001cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.373+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc034840 con 0x7f11e40c5e00 2023-11-08T19:26:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.373+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc0349a0 con 0x7f11e40c5e00 2023-11-08T19:26:34.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.373+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc034d50 con 0x7f11e40c5e00 2023-11-08T19:26:34.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.373+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e40c73d0 con 0x7f11e40c5e00 2023-11-08T19:26:34.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.373+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e40c7bd0 con 0x7f11e40c5e00 2023-11-08T19:26:34.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.374+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e40c78f0 con 0x7f11e40c5e00 2023-11-08T19:26:34.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.375+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11dc031030 con 0x7f11e40c5e00 2023-11-08T19:26:34.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.376+0000 7f11e17fa640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11c806ae70 0x7f11c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.376+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f11dc0ab4a0 con 0x7f11e40c5e00 2023-11-08T19:26:34.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.376+0000 7f11e8a3e640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11c806ae70 0x7f11c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.377+0000 7f11e8a3e640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11c806ae70 0x7f11c806d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11d8006fd0 tx=0x7f11d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.380+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11e40c78f0 con 0x7f11e40c5e00 2023-11-08T19:26:34.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.572+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11e40c78f0 con 0x7f11e40c5e00 2023-11-08T19:26:34.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.574+0000 7f11e17fa640 1 -- 172.21.15.119:0/3793970187 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16774 (secure 0 0 0) 0x7f11dc045020 con 0x7f11e40c5e00 2023-11-08T19:26:34.576 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:34.577 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":39,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:31.255061+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:34.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.576+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11c806ae70 msgr2=0x7f11c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11c806ae70 0x7f11c806d330 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11d8006fd0 tx=0x7f11d8008040 comp rx=0 tx=0).stop 2023-11-08T19:26:34.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 msgr2=0x7f11e40c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 0x7f11e40c6280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f11dc000f80 tx=0x7f11dc001cd0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 shutdown_connections 2023-11-08T19:26:34.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11c806ae70 0x7f11c806d330 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11e40c5e00 0x7f11e40c6280 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11e40c4110 0x7f11e40c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 --2- 172.21.15.119:0/3793970187 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11d40051d0 0x7f11e415a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 >> 172.21.15.119:0/3793970187 conn(0x7f11e40cb610 msgr2=0x7f11e40715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 shutdown_connections 2023-11-08T19:26:34.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.577+0000 7f11eaca2640 1 -- 172.21.15.119:0/3793970187 wait complete. 2023-11-08T19:26:34.599 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:26:34.599 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.800+0000 7f914fa1a640 1 Processor -- start 2023-11-08T19:26:34.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.800+0000 7f914fa1a640 1 -- start start 2023-11-08T19:26:34.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.802+0000 7f914fa1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f91480cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.802+0000 7f914fa1a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f914815d9f0 0x7f91480c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.802+0000 7f914fa1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f914806cb40 0x7f914806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.802+0000 7f914fa1a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9148143a40 con 0x7f914815d9f0 2023-11-08T19:26:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.802+0000 7f914fa1a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914806d510 con 0x7f914806cb40 2023-11-08T19:26:34.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.802+0000 7f914fa1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91480c4960 con 0x7f9148142f80 2023-11-08T19:26:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f914815d9f0 0x7f91480c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914d7b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f91480cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914dfb7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f914806cb40 0x7f914806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f914815d9f0 0x7f91480c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47718/0 (socket says 172.21.15.119:47718) 2023-11-08T19:26:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 -- 172.21.15.119:0/2604764222 learned_addr learned my addr 172.21.15.119:0/2604764222 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 -- 172.21.15.119:0/2604764222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 msgr2=0x7f91480cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 --2- 172.21.15.119:0/2604764222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f91480cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 -- 172.21.15.119:0/2604764222 >> v2:172.21.15.188:3300/0 conn(0x7f914806cb40 msgr2=0x7f914806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:34.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 --2- 172.21.15.119:0/2604764222 >> v2:172.21.15.188:3300/0 conn(0x7f914806cb40 0x7f914806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.803+0000 7f914cfb5640 1 -- 172.21.15.119:0/2604764222 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914815df60 con 0x7f914815d9f0 2023-11-08T19:26:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.804+0000 7f914cfb5640 1 --2- 172.21.15.119:0/2604764222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91480c9cd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f91480cb550 tx=0x7f913802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb3b55017bd1670 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.805+0000 7f913ffff640 1 -- 172.21.15.119:0/2604764222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91380317d0 con 0x7f914815d9f0 2023-11-08T19:26:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.805+0000 7f913ffff640 1 -- 172.21.15.119:0/2604764222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9138031930 con 0x7f914815d9f0 2023-11-08T19:26:34.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.805+0000 7f913ffff640 1 -- 172.21.15.119:0/2604764222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138031cc0 con 0x7f914815d9f0 2023-11-08T19:26:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.805+0000 7f914fa1a640 1 -- 172.21.15.119:0/2604764222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 msgr2=0x7f91480c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.805+0000 7f914fa1a640 1 --2- 172.21.15.119:0/2604764222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91480c9cd0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f91480cb550 tx=0x7f913802b870 comp rx=0 tx=0).stop 2023-11-08T19:26:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 -- 172.21.15.119:0/2604764222 shutdown_connections 2023-11-08T19:26:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 --2- 172.21.15.119:0/2604764222 >> v2:172.21.15.188:3300/0 conn(0x7f914806cb40 0x7f914806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 --2- 172.21.15.119:0/2604764222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91480c9cd0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 --2- 172.21.15.119:0/2604764222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f91480cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 -- 172.21.15.119:0/2604764222 >> 172.21.15.119:0/2604764222 conn(0x7f91480cb610 msgr2=0x7f91480cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 -- 172.21.15.119:0/2604764222 shutdown_connections 2023-11-08T19:26:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.806+0000 7f914fa1a640 1 -- 172.21.15.119:0/2604764222 wait complete. 2023-11-08T19:26:34.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.808+0000 7f914fa1a640 1 Processor -- start 2023-11-08T19:26:34.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.808+0000 7f914fa1a640 1 -- start start 2023-11-08T19:26:34.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914fa1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914806cb40 0x7f914812bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914fa1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f914812f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914fa1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91481f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914fa1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91481304b0 con 0x7f914815d9f0 2023-11-08T19:26:34.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914fa1a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480c8f30 con 0x7f914806cb40 2023-11-08T19:26:34.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914fa1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91481f2c50 con 0x7f9148142f80 2023-11-08T19:26:34.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91481f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91481f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47722/0 (socket says 172.21.15.119:47722) 2023-11-08T19:26:34.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 -- 172.21.15.119:0/1396619939 learned_addr learned my addr 172.21.15.119:0/1396619939 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:34.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914cfb5640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f914812f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914d7b6640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914806cb40 0x7f914812bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 -- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 msgr2=0x7f914812f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f914812f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 -- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914806cb40 msgr2=0x7f914812bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:34.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914806cb40 0x7f914812bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:34.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.810+0000 7f914dfb7640 1 -- 172.21.15.119:0/1396619939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91380094d0 con 0x7f914815d9f0 2023-11-08T19:26:34.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.811+0000 7f914dfb7640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91481f2710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f914000e7f0 tx=0x7f914000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.811+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9140009490 con 0x7f914815d9f0 2023-11-08T19:26:34.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.811+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914000c730 con 0x7f914815d9f0 2023-11-08T19:26:34.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.811+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914000cac0 con 0x7f914815d9f0 2023-11-08T19:26:34.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.811+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f914812f830 con 0x7f914815d9f0 2023-11-08T19:26:34.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.811+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f914812fdf0 con 0x7f914815d9f0 2023-11-08T19:26:34.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.812+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91481305f0 con 0x7f914815d9f0 2023-11-08T19:26:34.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.813+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f914000cc20 con 0x7f914815d9f0 2023-11-08T19:26:34.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.815+0000 7f913dffb640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f912c06aea0 0x7f912c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:34.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.815+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f91400897e0 con 0x7f914815d9f0 2023-11-08T19:26:34.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.815+0000 7f914d7b6640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f912c06aea0 0x7f912c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:34.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.816+0000 7f914d7b6640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f912c06aea0 0x7f912c06d360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9144007380 tx=0x7f914400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:34.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:34.819+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f91481305f0 con 0x7f914815d9f0 2023-11-08T19:26:35.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.009+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f914815e520 con 0x7f912c06aea0 2023-11-08T19:26:35.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.035+0000 7f913dffb640 1 -- 172.21.15.119:0/1396619939 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286750 (secure 0 0 0) 0x7f91440143d0 con 0x7f912c06aea0 2023-11-08T19:26:35.041 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:35.043 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:26:35.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f912c06aea0 msgr2=0x7f912c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f912c06aea0 0x7f912c06d360 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9144007380 tx=0x7f914400b040 comp rx=0 tx=0).stop 2023-11-08T19:26:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 msgr2=0x7f91481f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91481f2710 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f914000e7f0 tx=0x7f914000ecb0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 shutdown_connections 2023-11-08T19:26:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f912c06aea0 0x7f912c06d360 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f914815d9f0 0x7f91481f2710 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9148142f80 0x7f914812f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 --2- 172.21.15.119:0/1396619939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f914806cb40 0x7f914812bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.042+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 >> 172.21.15.119:0/1396619939 conn(0x7f91480cb610 msgr2=0x7f91480c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.043+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 shutdown_connections 2023-11-08T19:26:35.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.043+0000 7f914fa1a640 1 -- 172.21.15.119:0/1396619939 wait complete. 2023-11-08T19:26:35.064 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-11-08T19:26:34.692556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220856,"kb_used_data":4664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753864,"statfs":{"total":773094113280,"available":772867956736,"internally_reserved":0,"allocated":4775936,"data_stored":2709486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266157+0000","last_change":"2023-11-08T19:26:30.292305+0000","last_active":"2023-11-08T19:26:31.266157+0000","last_peered":"2023-11-08T19:26:31.266157+0000","last_clean":"2023-11-08T19:26:31.266157+0000","last_became_active":"2023-11-08T19:26:30.292030+0000","last_became_peered":"2023-11-08T19:26:30.292030+0000","last_unstale":"2023-11-08T19:26:31.266157+0000","last_undegraded":"2023-11-08T19:26:31.266157+0000","last_fullsized":"2023-11-08T19:26:31.266157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:13.373857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265463+0000","last_change":"2023-11-08T19:26:30.296762+0000","last_active":"2023-11-08T19:26:31.265463+0000","last_peered":"2023-11-08T19:26:31.265463+0000","last_clean":"2023-11-08T19:26:31.265463+0000","last_became_active":"2023-11-08T19:26:30.296491+0000","last_became_peered":"2023-11-08T19:26:30.296491+0000","last_unstale":"2023-11-08T19:26:31.265463+0000","last_undegraded":"2023-11-08T19:26:31.265463+0000","last_fullsized":"2023-11-08T19:26:31.265463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:59.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395518+0000","last_change":"2023-11-08T19:26:30.292119+0000","last_active":"2023-11-08T19:26:31.395518+0000","last_peered":"2023-11-08T19:26:31.395518+0000","last_clean":"2023-11-08T19:26:31.395518+0000","last_became_active":"2023-11-08T19:26:30.291809+0000","last_became_peered":"2023-11-08T19:26:30.291809+0000","last_unstale":"2023-11-08T19:26:31.395518+0000","last_undegraded":"2023-11-08T19:26:31.395518+0000","last_fullsized":"2023-11-08T19:26:31.395518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:52.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399333+0000","last_change":"2023-11-08T19:26:30.291436+0000","last_active":"2023-11-08T19:26:31.399333+0000","last_peered":"2023-11-08T19:26:31.399333+0000","last_clean":"2023-11-08T19:26:31.399333+0000","last_became_active":"2023-11-08T19:26:30.289196+0000","last_became_peered":"2023-11-08T19:26:30.289196+0000","last_unstale":"2023-11-08T19:26:31.399333+0000","last_undegraded":"2023-11-08T19:26:31.399333+0000","last_fullsized":"2023-11-08T19:26:31.399333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:25:46.534304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399048+0000","last_change":"2023-11-08T19:26:30.291337+0000","last_active":"2023-11-08T19:26:31.399048+0000","last_peered":"2023-11-08T19:26:31.399048+0000","last_clean":"2023-11-08T19:26:31.399048+0000","last_became_active":"2023-11-08T19:26:30.291053+0000","last_became_peered":"2023-11-08T19:26:30.291053+0000","last_unstale":"2023-11-08T19:26:31.399048+0000","last_undegraded":"2023-11-08T19:26:31.399048+0000","last_fullsized":"2023-11-08T19:26:31.399048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:03.031932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396341+0000","last_change":"2023-11-08T19:26:30.293253+0000","last_active":"2023-11-08T19:26:31.396341+0000","last_peered":"2023-11-08T19:26:31.396341+0000","last_clean":"2023-11-08T19:26:31.396341+0000","last_became_active":"2023-11-08T19:26:30.292988+0000","last_became_peered":"2023-11-08T19:26:30.292988+0000","last_unstale":"2023-11-08T19:26:31.396341+0000","last_undegraded":"2023-11-08T19:26:31.396341+0000","last_fullsized":"2023-11-08T19:26:31.396341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:09.389740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268478+0000","last_change":"2023-11-08T19:26:30.300921+0000","last_active":"2023-11-08T19:26:31.268478+0000","last_peered":"2023-11-08T19:26:31.268478+0000","last_clean":"2023-11-08T19:26:31.268478+0000","last_became_active":"2023-11-08T19:26:30.300622+0000","last_became_peered":"2023-11-08T19:26:30.300622+0000","last_unstale":"2023-11-08T19:26:31.268478+0000","last_undegraded":"2023-11-08T19:26:31.268478+0000","last_fullsized":"2023-11-08T19:26:31.268478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:02.915393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265503+0000","last_change":"2023-11-08T19:26:30.291795+0000","last_active":"2023-11-08T19:26:31.265503+0000","last_peered":"2023-11-08T19:26:31.265503+0000","last_clean":"2023-11-08T19:26:31.265503+0000","last_became_active":"2023-11-08T19:26:30.290648+0000","last_became_peered":"2023-11-08T19:26:30.290648+0000","last_unstale":"2023-11-08T19:26:31.265503+0000","last_undegraded":"2023-11-08T19:26:31.265503+0000","last_fullsized":"2023-11-08T19:26:31.265503+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:07.011653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395933+0000","last_change":"2023-11-08T19:26:30.293533+0000","last_active":"2023-11-08T19:26:31.395933+0000","last_peered":"2023-11-08T19:26:31.395933+0000","last_clean":"2023-11-08T19:26:31.395933+0000","last_became_active":"2023-11-08T19:26:30.292642+0000","last_became_peered":"2023-11-08T19:26:30.292642+0000","last_unstale":"2023-11-08T19:26:31.395933+0000","last_undegraded":"2023-11-08T19:26:31.395933+0000","last_fullsized":"2023-11-08T19:26:31.395933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269803+0000","last_change":"2023-11-08T19:26:30.293410+0000","last_active":"2023-11-08T19:26:31.269803+0000","last_peered":"2023-11-08T19:26:31.269803+0000","last_clean":"2023-11-08T19:26:31.269803+0000","last_became_active":"2023-11-08T19:26:30.293155+0000","last_became_peered":"2023-11-08T19:26:30.293155+0000","last_unstale":"2023-11-08T19:26:31.269803+0000","last_undegraded":"2023-11-08T19:26:31.269803+0000","last_fullsized":"2023-11-08T19:26:31.269803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:07.011653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269472+0000","last_change":"2023-11-08T19:26:30.290241+0000","last_active":"2023-11-08T19:26:31.269472+0000","last_peered":"2023-11-08T19:26:31.269472+0000","last_clean":"2023-11-08T19:26:31.269472+0000","last_became_active":"2023-11-08T19:26:30.289967+0000","last_became_peered":"2023-11-08T19:26:30.289967+0000","last_unstale":"2023-11-08T19:26:31.269472+0000","last_undegraded":"2023-11-08T19:26:31.269472+0000","last_fullsized":"2023-11-08T19:26:31.269472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:59.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395309+0000","last_change":"2023-11-08T19:26:30.291603+0000","last_active":"2023-11-08T19:26:31.395309+0000","last_peered":"2023-11-08T19:26:31.395309+0000","last_clean":"2023-11-08T19:26:31.395309+0000","last_became_active":"2023-11-08T19:26:30.291260+0000","last_became_peered":"2023-11-08T19:26:30.291260+0000","last_unstale":"2023-11-08T19:26:31.395309+0000","last_undegraded":"2023-11-08T19:26:31.395309+0000","last_fullsized":"2023-11-08T19:26:31.395309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:47.968547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268428+0000","last_change":"2023-11-08T19:26:30.298138+0000","last_active":"2023-11-08T19:26:31.268428+0000","last_peered":"2023-11-08T19:26:31.268428+0000","last_clean":"2023-11-08T19:26:31.268428+0000","last_became_active":"2023-11-08T19:26:30.297875+0000","last_became_peered":"2023-11-08T19:26:30.297875+0000","last_unstale":"2023-11-08T19:26:31.268428+0000","last_undegraded":"2023-11-08T19:26:31.268428+0000","last_fullsized":"2023-11-08T19:26:31.268428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:04.627278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396793+0000","last_change":"2023-11-08T19:26:30.296178+0000","last_active":"2023-11-08T19:26:31.396793+0000","last_peered":"2023-11-08T19:26:31.396793+0000","last_clean":"2023-11-08T19:26:31.396793+0000","last_became_active":"2023-11-08T19:26:30.293002+0000","last_became_peered":"2023-11-08T19:26:30.293002+0000","last_unstale":"2023-11-08T19:26:31.396793+0000","last_undegraded":"2023-11-08T19:26:31.396793+0000","last_fullsized":"2023-11-08T19:26:31.396793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268805+0000","last_change":"2023-11-08T19:26:30.297919+0000","last_active":"2023-11-08T19:26:31.268805+0000","last_peered":"2023-11-08T19:26:31.268805+0000","last_clean":"2023-11-08T19:26:31.268805+0000","last_became_active":"2023-11-08T19:26:30.297574+0000","last_became_peered":"2023-11-08T19:26:30.297574+0000","last_unstale":"2023-11-08T19:26:31.268805+0000","last_undegraded":"2023-11-08T19:26:31.268805+0000","last_fullsized":"2023-11-08T19:26:31.268805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:13:21.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396225+0000","last_change":"2023-11-08T19:26:30.290191+0000","last_active":"2023-11-08T19:26:31.396225+0000","last_peered":"2023-11-08T19:26:31.396225+0000","last_clean":"2023-11-08T19:26:31.396225+0000","last_became_active":"2023-11-08T19:26:30.289922+0000","last_became_peered":"2023-11-08T19:26:30.289922+0000","last_unstale":"2023-11-08T19:26:31.396225+0000","last_undegraded":"2023-11-08T19:26:31.396225+0000","last_fullsized":"2023-11-08T19:26:31.396225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.513281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398270+0000","last_change":"2023-11-08T19:26:30.303342+0000","last_active":"2023-11-08T19:26:31.398270+0000","last_peered":"2023-11-08T19:26:31.398270+0000","last_clean":"2023-11-08T19:26:31.398270+0000","last_became_active":"2023-11-08T19:26:30.302853+0000","last_became_peered":"2023-11-08T19:26:30.302853+0000","last_unstale":"2023-11-08T19:26:31.398270+0000","last_undegraded":"2023-11-08T19:26:31.398270+0000","last_fullsized":"2023-11-08T19:26:31.398270+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:13:21.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269638+0000","last_change":"2023-11-08T19:26:30.286808+0000","last_active":"2023-11-08T19:26:31.269638+0000","last_peered":"2023-11-08T19:26:31.269638+0000","last_clean":"2023-11-08T19:26:31.269638+0000","last_became_active":"2023-11-08T19:26:30.285864+0000","last_became_peered":"2023-11-08T19:26:30.285864+0000","last_unstale":"2023-11-08T19:26:31.269638+0000","last_undegraded":"2023-11-08T19:26:31.269638+0000","last_fullsized":"2023-11-08T19:26:31.269638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:54.424411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399066+0000","last_change":"2023-11-08T19:26:30.298280+0000","last_active":"2023-11-08T19:26:31.399066+0000","last_peered":"2023-11-08T19:26:31.399066+0000","last_clean":"2023-11-08T19:26:31.399066+0000","last_became_active":"2023-11-08T19:26:30.297903+0000","last_became_peered":"2023-11-08T19:26:30.297903+0000","last_unstale":"2023-11-08T19:26:31.399066+0000","last_undegraded":"2023-11-08T19:26:31.399066+0000","last_fullsized":"2023-11-08T19:26:31.399066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:53.174998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.267912+0000","last_change":"2023-11-08T19:26:30.289026+0000","last_active":"2023-11-08T19:26:31.267912+0000","last_peered":"2023-11-08T19:26:31.267912+0000","last_clean":"2023-11-08T19:26:31.267912+0000","last_became_active":"2023-11-08T19:26:30.287461+0000","last_became_peered":"2023-11-08T19:26:30.287461+0000","last_unstale":"2023-11-08T19:26:31.267912+0000","last_undegraded":"2023-11-08T19:26:31.267912+0000","last_fullsized":"2023-11-08T19:26:31.267912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:11:32.070628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268778+0000","last_change":"2023-11-08T19:26:30.301598+0000","last_active":"2023-11-08T19:26:31.268778+0000","last_peered":"2023-11-08T19:26:31.268778+0000","last_clean":"2023-11-08T19:26:31.268778+0000","last_became_active":"2023-11-08T19:26:30.301322+0000","last_became_peered":"2023-11-08T19:26:30.301322+0000","last_unstale":"2023-11-08T19:26:31.268778+0000","last_undegraded":"2023-11-08T19:26:31.268778+0000","last_fullsized":"2023-11-08T19:26:31.268778+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:58.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397422+0000","last_change":"2023-11-08T19:26:30.296888+0000","last_active":"2023-11-08T19:26:31.397422+0000","last_peered":"2023-11-08T19:26:31.397422+0000","last_clean":"2023-11-08T19:26:31.397422+0000","last_became_active":"2023-11-08T19:26:30.296541+0000","last_became_peered":"2023-11-08T19:26:30.296541+0000","last_unstale":"2023-11-08T19:26:31.397422+0000","last_undegraded":"2023-11-08T19:26:31.397422+0000","last_fullsized":"2023-11-08T19:26:31.397422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:29.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397319+0000","last_change":"2023-11-08T19:26:30.297332+0000","last_active":"2023-11-08T19:26:31.397319+0000","last_peered":"2023-11-08T19:26:31.397319+0000","last_clean":"2023-11-08T19:26:31.397319+0000","last_became_active":"2023-11-08T19:26:30.296705+0000","last_became_peered":"2023-11-08T19:26:30.296705+0000","last_unstale":"2023-11-08T19:26:31.397319+0000","last_undegraded":"2023-11-08T19:26:31.397319+0000","last_fullsized":"2023-11-08T19:26:31.397319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:34.753858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398254+0000","last_change":"2023-11-08T19:26:30.283303+0000","last_active":"2023-11-08T19:26:31.398254+0000","last_peered":"2023-11-08T19:26:31.398254+0000","last_clean":"2023-11-08T19:26:31.398254+0000","last_became_active":"2023-11-08T19:26:30.283026+0000","last_became_peered":"2023-11-08T19:26:30.283026+0000","last_unstale":"2023-11-08T19:26:31.398254+0000","last_undegraded":"2023-11-08T19:26:31.398254+0000","last_fullsized":"2023-11-08T19:26:31.398254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:27:22.361334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270220+0000","last_change":"2023-11-08T19:26:30.282351+0000","last_active":"2023-11-08T19:26:31.270220+0000","last_peered":"2023-11-08T19:26:31.270220+0000","last_clean":"2023-11-08T19:26:31.270220+0000","last_became_active":"2023-11-08T19:26:30.282016+0000","last_became_peered":"2023-11-08T19:26:30.282016+0000","last_unstale":"2023-11-08T19:26:31.270220+0000","last_undegraded":"2023-11-08T19:26:31.270220+0000","last_fullsized":"2023-11-08T19:26:31.270220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:57.123261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397886+0000","last_change":"2023-11-08T19:26:30.302590+0000","last_active":"2023-11-08T19:26:31.397886+0000","last_peered":"2023-11-08T19:26:31.397886+0000","last_clean":"2023-11-08T19:26:31.397886+0000","last_became_active":"2023-11-08T19:26:30.302368+0000","last_became_peered":"2023-11-08T19:26:30.302368+0000","last_unstale":"2023-11-08T19:26:31.397886+0000","last_undegraded":"2023-11-08T19:26:31.397886+0000","last_fullsized":"2023-11-08T19:26:31.397886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269026+0000","last_change":"2023-11-08T19:26:30.283391+0000","last_active":"2023-11-08T19:26:31.269026+0000","last_peered":"2023-11-08T19:26:31.269026+0000","last_clean":"2023-11-08T19:26:31.269026+0000","last_became_active":"2023-11-08T19:26:30.283079+0000","last_became_peered":"2023-11-08T19:26:30.283079+0000","last_unstale":"2023-11-08T19:26:31.269026+0000","last_undegraded":"2023-11-08T19:26:31.269026+0000","last_fullsized":"2023-11-08T19:26:31.269026+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:35:37.209163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269086+0000","last_change":"2023-11-08T19:26:30.293540+0000","last_active":"2023-11-08T19:26:31.269086+0000","last_peered":"2023-11-08T19:26:31.269086+0000","last_clean":"2023-11-08T19:26:31.269086+0000","last_became_active":"2023-11-08T19:26:30.292296+0000","last_became_peered":"2023-11-08T19:26:30.292296+0000","last_unstale":"2023-11-08T19:26:31.269086+0000","last_undegraded":"2023-11-08T19:26:31.269086+0000","last_fullsized":"2023-11-08T19:26:31.269086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:05.882531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268053+0000","last_change":"2023-11-08T19:26:30.295364+0000","last_active":"2023-11-08T19:26:31.268053+0000","last_peered":"2023-11-08T19:26:31.268053+0000","last_clean":"2023-11-08T19:26:31.268053+0000","last_became_active":"2023-11-08T19:26:30.295042+0000","last_became_peered":"2023-11-08T19:26:30.295042+0000","last_unstale":"2023-11-08T19:26:31.268053+0000","last_undegraded":"2023-11-08T19:26:31.268053+0000","last_fullsized":"2023-11-08T19:26:31.268053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:41.723320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.267788+0000","last_change":"2023-11-08T19:26:30.291570+0000","last_active":"2023-11-08T19:26:31.267788+0000","last_peered":"2023-11-08T19:26:31.267788+0000","last_clean":"2023-11-08T19:26:31.267788+0000","last_became_active":"2023-11-08T19:26:30.290459+0000","last_became_peered":"2023-11-08T19:26:30.290459+0000","last_unstale":"2023-11-08T19:26:31.267788+0000","last_undegraded":"2023-11-08T19:26:31.267788+0000","last_fullsized":"2023-11-08T19:26:31.267788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:33.467785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397691+0000","last_change":"2023-11-08T19:26:30.297583+0000","last_active":"2023-11-08T19:26:31.397691+0000","last_peered":"2023-11-08T19:26:31.397691+0000","last_clean":"2023-11-08T19:26:31.397691+0000","last_became_active":"2023-11-08T19:26:30.297075+0000","last_became_peered":"2023-11-08T19:26:30.297075+0000","last_unstale":"2023-11-08T19:26:31.397691+0000","last_undegraded":"2023-11-08T19:26:31.397691+0000","last_fullsized":"2023-11-08T19:26:31.397691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:02.915393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396597+0000","last_change":"2023-11-08T19:26:30.299427+0000","last_active":"2023-11-08T19:26:31.396597+0000","last_peered":"2023-11-08T19:26:31.396597+0000","last_clean":"2023-11-08T19:26:31.396597+0000","last_became_active":"2023-11-08T19:26:30.299151+0000","last_became_peered":"2023-11-08T19:26:30.299151+0000","last_unstale":"2023-11-08T19:26:31.396597+0000","last_undegraded":"2023-11-08T19:26:31.396597+0000","last_fullsized":"2023-11-08T19:26:31.396597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:47.968547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397986+0000","last_change":"2023-11-08T19:26:30.294571+0000","last_active":"2023-11-08T19:26:31.397986+0000","last_peered":"2023-11-08T19:26:31.397986+0000","last_clean":"2023-11-08T19:26:31.397986+0000","last_became_active":"2023-11-08T19:26:30.294206+0000","last_became_peered":"2023-11-08T19:26:30.294206+0000","last_unstale":"2023-11-08T19:26:31.397986+0000","last_undegraded":"2023-11-08T19:26:31.397986+0000","last_fullsized":"2023-11-08T19:26:31.397986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:58.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270455+0000","last_change":"2023-11-08T19:26:29.291331+0000","last_active":"2023-11-08T19:26:31.270455+0000","last_peered":"2023-11-08T19:26:31.270455+0000","last_clean":"2023-11-08T19:26:31.270455+0000","last_became_active":"2023-11-08T19:26:29.291064+0000","last_became_peered":"2023-11-08T19:26:29.291064+0000","last_unstale":"2023-11-08T19:26:31.270455+0000","last_undegraded":"2023-11-08T19:26:31.270455+0000","last_fullsized":"2023-11-08T19:26:31.270455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:14.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397676+0000","last_change":"2023-11-08T19:26:30.303309+0000","last_active":"2023-11-08T19:26:31.397676+0000","last_peered":"2023-11-08T19:26:31.397676+0000","last_clean":"2023-11-08T19:26:31.397676+0000","last_became_active":"2023-11-08T19:26:30.302985+0000","last_became_peered":"2023-11-08T19:26:30.302985+0000","last_unstale":"2023-11-08T19:26:31.397676+0000","last_undegraded":"2023-11-08T19:26:31.397676+0000","last_fullsized":"2023-11-08T19:26:31.397676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396809+0000","last_change":"2023-11-08T19:26:29.284298+0000","last_active":"2023-11-08T19:26:31.396809+0000","last_peered":"2023-11-08T19:26:31.396809+0000","last_clean":"2023-11-08T19:26:31.396809+0000","last_became_active":"2023-11-08T19:26:29.283401+0000","last_became_peered":"2023-11-08T19:26:29.283401+0000","last_unstale":"2023-11-08T19:26:31.396809+0000","last_undegraded":"2023-11-08T19:26:31.396809+0000","last_fullsized":"2023-11-08T19:26:31.396809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:47:19.011806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398666+0000","last_change":"2023-11-08T19:26:30.289864+0000","last_active":"2023-11-08T19:26:31.398666+0000","last_peered":"2023-11-08T19:26:31.398666+0000","last_clean":"2023-11-08T19:26:31.398666+0000","last_became_active":"2023-11-08T19:26:30.288162+0000","last_became_peered":"2023-11-08T19:26:30.288162+0000","last_unstale":"2023-11-08T19:26:31.398666+0000","last_undegraded":"2023-11-08T19:26:31.398666+0000","last_fullsized":"2023-11-08T19:26:31.398666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:02.915393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397938+0000","last_change":"2023-11-08T19:26:29.290793+0000","last_active":"2023-11-08T19:26:31.397938+0000","last_peered":"2023-11-08T19:26:31.397938+0000","last_clean":"2023-11-08T19:26:31.397938+0000","last_became_active":"2023-11-08T19:26:29.290546+0000","last_became_peered":"2023-11-08T19:26:29.290546+0000","last_unstale":"2023-11-08T19:26:31.397938+0000","last_undegraded":"2023-11-08T19:26:31.397938+0000","last_fullsized":"2023-11-08T19:26:31.397938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:37.487567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265934+0000","last_change":"2023-11-08T19:26:30.296181+0000","last_active":"2023-11-08T19:26:31.265934+0000","last_peered":"2023-11-08T19:26:31.265934+0000","last_clean":"2023-11-08T19:26:31.265934+0000","last_became_active":"2023-11-08T19:26:30.295855+0000","last_became_peered":"2023-11-08T19:26:30.295855+0000","last_unstale":"2023-11-08T19:26:31.265934+0000","last_undegraded":"2023-11-08T19:26:31.265934+0000","last_fullsized":"2023-11-08T19:26:31.265934+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:46.858474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397550+0000","last_change":"2023-11-08T19:26:29.288519+0000","last_active":"2023-11-08T19:26:31.397550+0000","last_peered":"2023-11-08T19:26:31.397550+0000","last_clean":"2023-11-08T19:26:31.397550+0000","last_became_active":"2023-11-08T19:26:29.288184+0000","last_became_peered":"2023-11-08T19:26:29.288184+0000","last_unstale":"2023-11-08T19:26:31.397550+0000","last_undegraded":"2023-11-08T19:26:31.397550+0000","last_fullsized":"2023-11-08T19:26:31.397550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:56.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395253+0000","last_change":"2023-11-08T19:26:30.291538+0000","last_active":"2023-11-08T19:26:31.395253+0000","last_peered":"2023-11-08T19:26:31.395253+0000","last_clean":"2023-11-08T19:26:31.395253+0000","last_became_active":"2023-11-08T19:26:30.291229+0000","last_became_peered":"2023-11-08T19:26:30.291229+0000","last_unstale":"2023-11-08T19:26:31.395253+0000","last_undegraded":"2023-11-08T19:26:31.395253+0000","last_fullsized":"2023-11-08T19:26:31.395253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:52.601811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269023+0000","last_change":"2023-11-08T19:26:29.285627+0000","last_active":"2023-11-08T19:26:31.269023+0000","last_peered":"2023-11-08T19:26:31.269023+0000","last_clean":"2023-11-08T19:26:31.269023+0000","last_became_active":"2023-11-08T19:26:29.284572+0000","last_became_peered":"2023-11-08T19:26:29.284572+0000","last_unstale":"2023-11-08T19:26:31.269023+0000","last_undegraded":"2023-11-08T19:26:31.269023+0000","last_fullsized":"2023-11-08T19:26:31.269023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:51.079022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269123+0000","last_change":"2023-11-08T19:26:30.294358+0000","last_active":"2023-11-08T19:26:31.269123+0000","last_peered":"2023-11-08T19:26:31.269123+0000","last_clean":"2023-11-08T19:26:31.269123+0000","last_became_active":"2023-11-08T19:26:30.294086+0000","last_became_peered":"2023-11-08T19:26:30.294086+0000","last_unstale":"2023-11-08T19:26:31.269123+0000","last_undegraded":"2023-11-08T19:26:31.269123+0000","last_fullsized":"2023-11-08T19:26:31.269123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:09.911521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398512+0000","last_change":"2023-11-08T19:26:29.295383+0000","last_active":"2023-11-08T19:26:31.398512+0000","last_peered":"2023-11-08T19:26:31.398512+0000","last_clean":"2023-11-08T19:26:31.398512+0000","last_became_active":"2023-11-08T19:26:29.295105+0000","last_became_peered":"2023-11-08T19:26:29.295105+0000","last_unstale":"2023-11-08T19:26:31.398512+0000","last_undegraded":"2023-11-08T19:26:31.398512+0000","last_fullsized":"2023-11-08T19:26:31.398512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:08.068311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397476+0000","last_change":"2023-11-08T19:26:30.297007+0000","last_active":"2023-11-08T19:26:31.397476+0000","last_peered":"2023-11-08T19:26:31.397476+0000","last_clean":"2023-11-08T19:26:31.397476+0000","last_became_active":"2023-11-08T19:26:30.296566+0000","last_became_peered":"2023-11-08T19:26:30.296566+0000","last_unstale":"2023-11-08T19:26:31.397476+0000","last_undegraded":"2023-11-08T19:26:31.397476+0000","last_fullsized":"2023-11-08T19:26:31.397476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:34.753858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397324+0000","last_change":"2023-11-08T19:26:29.279306+0000","last_active":"2023-11-08T19:26:31.397324+0000","last_peered":"2023-11-08T19:26:31.397324+0000","last_clean":"2023-11-08T19:26:31.397324+0000","last_became_active":"2023-11-08T19:26:29.279041+0000","last_became_peered":"2023-11-08T19:26:29.279041+0000","last_unstale":"2023-11-08T19:26:31.397324+0000","last_undegraded":"2023-11-08T19:26:31.397324+0000","last_fullsized":"2023-11-08T19:26:31.397324+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:53.409533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265990+0000","last_change":"2023-11-08T19:26:30.290971+0000","last_active":"2023-11-08T19:26:31.265990+0000","last_peered":"2023-11-08T19:26:31.265990+0000","last_clean":"2023-11-08T19:26:31.265990+0000","last_became_active":"2023-11-08T19:26:30.289214+0000","last_became_peered":"2023-11-08T19:26:30.289214+0000","last_unstale":"2023-11-08T19:26:31.265990+0000","last_undegraded":"2023-11-08T19:26:31.265990+0000","last_fullsized":"2023-11-08T19:26:31.265990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:10.539596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396121+0000","last_change":"2023-11-08T19:26:29.292941+0000","last_active":"2023-11-08T19:26:31.396121+0000","last_peered":"2023-11-08T19:26:31.396121+0000","last_clean":"2023-11-08T19:26:31.396121+0000","last_became_active":"2023-11-08T19:26:29.289972+0000","last_became_peered":"2023-11-08T19:26:29.289972+0000","last_unstale":"2023-11-08T19:26:31.396121+0000","last_undegraded":"2023-11-08T19:26:31.396121+0000","last_fullsized":"2023-11-08T19:26:31.396121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:53.409533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399416+0000","last_change":"2023-11-08T19:26:30.290971+0000","last_active":"2023-11-08T19:26:31.399416+0000","last_peered":"2023-11-08T19:26:31.399416+0000","last_clean":"2023-11-08T19:26:31.399416+0000","last_became_active":"2023-11-08T19:26:30.290500+0000","last_became_peered":"2023-11-08T19:26:30.290500+0000","last_unstale":"2023-11-08T19:26:31.399416+0000","last_undegraded":"2023-11-08T19:26:31.399416+0000","last_fullsized":"2023-11-08T19:26:31.399416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.112156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266255+0000","last_change":"2023-11-08T19:26:29.280352+0000","last_active":"2023-11-08T19:26:31.266255+0000","last_peered":"2023-11-08T19:26:31.266255+0000","last_clean":"2023-11-08T19:26:31.266255+0000","last_became_active":"2023-11-08T19:26:29.280082+0000","last_became_peered":"2023-11-08T19:26:29.280082+0000","last_unstale":"2023-11-08T19:26:31.266255+0000","last_undegraded":"2023-11-08T19:26:31.266255+0000","last_fullsized":"2023-11-08T19:26:31.266255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:10.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397226+0000","last_change":"2023-11-08T19:26:30.287963+0000","last_active":"2023-11-08T19:26:31.397226+0000","last_peered":"2023-11-08T19:26:31.397226+0000","last_clean":"2023-11-08T19:26:31.397226+0000","last_became_active":"2023-11-08T19:26:30.287540+0000","last_became_peered":"2023-11-08T19:26:30.287540+0000","last_unstale":"2023-11-08T19:26:31.397226+0000","last_undegraded":"2023-11-08T19:26:31.397226+0000","last_fullsized":"2023-11-08T19:26:31.397226+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:29.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270028+0000","last_change":"2023-11-08T19:26:29.275731+0000","last_active":"2023-11-08T19:26:31.270028+0000","last_peered":"2023-11-08T19:26:31.270028+0000","last_clean":"2023-11-08T19:26:31.270028+0000","last_became_active":"2023-11-08T19:26:29.275364+0000","last_became_peered":"2023-11-08T19:26:29.275364+0000","last_unstale":"2023-11-08T19:26:31.270028+0000","last_undegraded":"2023-11-08T19:26:31.270028+0000","last_fullsized":"2023-11-08T19:26:31.270028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:10.487434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396962+0000","last_change":"2023-11-08T19:26:30.288811+0000","last_active":"2023-11-08T19:26:31.396962+0000","last_peered":"2023-11-08T19:26:31.396962+0000","last_clean":"2023-11-08T19:26:31.396962+0000","last_became_active":"2023-11-08T19:26:30.288245+0000","last_became_peered":"2023-11-08T19:26:30.288245+0000","last_unstale":"2023-11-08T19:26:31.396962+0000","last_undegraded":"2023-11-08T19:26:31.396962+0000","last_fullsized":"2023-11-08T19:26:31.396962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:29.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270281+0000","last_change":"2023-11-08T19:26:29.286720+0000","last_active":"2023-11-08T19:26:31.270281+0000","last_peered":"2023-11-08T19:26:31.270281+0000","last_clean":"2023-11-08T19:26:31.270281+0000","last_became_active":"2023-11-08T19:26:29.286426+0000","last_became_peered":"2023-11-08T19:26:29.286426+0000","last_unstale":"2023-11-08T19:26:31.270281+0000","last_undegraded":"2023-11-08T19:26:31.270281+0000","last_fullsized":"2023-11-08T19:26:31.270281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:45.570382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398697+0000","last_change":"2023-11-08T19:26:30.300346+0000","last_active":"2023-11-08T19:26:31.398697+0000","last_peered":"2023-11-08T19:26:31.398697+0000","last_clean":"2023-11-08T19:26:31.398697+0000","last_became_active":"2023-11-08T19:26:30.299506+0000","last_became_peered":"2023-11-08T19:26:30.299506+0000","last_unstale":"2023-11-08T19:26:31.398697+0000","last_undegraded":"2023-11-08T19:26:31.398697+0000","last_fullsized":"2023-11-08T19:26:31.398697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:58.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266334+0000","last_change":"2023-11-08T19:26:29.283737+0000","last_active":"2023-11-08T19:26:31.266334+0000","last_peered":"2023-11-08T19:26:31.266334+0000","last_clean":"2023-11-08T19:26:31.266334+0000","last_became_active":"2023-11-08T19:26:29.283432+0000","last_became_peered":"2023-11-08T19:26:29.283432+0000","last_unstale":"2023-11-08T19:26:31.266334+0000","last_undegraded":"2023-11-08T19:26:31.266334+0000","last_fullsized":"2023-11-08T19:26:31.266334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:04.867654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395262+0000","last_change":"2023-11-08T19:26:30.292146+0000","last_active":"2023-11-08T19:26:31.395262+0000","last_peered":"2023-11-08T19:26:31.395262+0000","last_clean":"2023-11-08T19:26:31.395262+0000","last_became_active":"2023-11-08T19:26:30.291846+0000","last_became_peered":"2023-11-08T19:26:30.291846+0000","last_unstale":"2023-11-08T19:26:31.395262+0000","last_undegraded":"2023-11-08T19:26:31.395262+0000","last_fullsized":"2023-11-08T19:26:31.395262+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:09.389740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266392+0000","last_change":"2023-11-08T19:26:29.280380+0000","last_active":"2023-11-08T19:26:31.266392+0000","last_peered":"2023-11-08T19:26:31.266392+0000","last_clean":"2023-11-08T19:26:31.266392+0000","last_became_active":"2023-11-08T19:26:29.279951+0000","last_became_peered":"2023-11-08T19:26:29.279951+0000","last_unstale":"2023-11-08T19:26:31.266392+0000","last_undegraded":"2023-11-08T19:26:31.266392+0000","last_fullsized":"2023-11-08T19:26:31.266392+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:08.896644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265546+0000","last_change":"2023-11-08T19:26:30.293261+0000","last_active":"2023-11-08T19:26:31.265546+0000","last_peered":"2023-11-08T19:26:31.265546+0000","last_clean":"2023-11-08T19:26:31.265546+0000","last_became_active":"2023-11-08T19:26:30.291279+0000","last_became_peered":"2023-11-08T19:26:30.291279+0000","last_unstale":"2023-11-08T19:26:31.265546+0000","last_undegraded":"2023-11-08T19:26:31.265546+0000","last_fullsized":"2023-11-08T19:26:31.265546+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:53.027281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396429+0000","last_change":"2023-11-08T19:26:29.286345+0000","last_active":"2023-11-08T19:26:31.396429+0000","last_peered":"2023-11-08T19:26:31.396429+0000","last_clean":"2023-11-08T19:26:31.396429+0000","last_became_active":"2023-11-08T19:26:29.286094+0000","last_became_peered":"2023-11-08T19:26:29.286094+0000","last_unstale":"2023-11-08T19:26:31.396429+0000","last_undegraded":"2023-11-08T19:26:31.396429+0000","last_fullsized":"2023-11-08T19:26:31.396429+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:47:19.011806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396516+0000","last_change":"2023-11-08T19:26:30.294135+0000","last_active":"2023-11-08T19:26:31.396516+0000","last_peered":"2023-11-08T19:26:31.396516+0000","last_clean":"2023-11-08T19:26:31.396516+0000","last_became_active":"2023-11-08T19:26:30.293861+0000","last_became_peered":"2023-11-08T19:26:30.293861+0000","last_unstale":"2023-11-08T19:26:31.396516+0000","last_undegraded":"2023-11-08T19:26:31.396516+0000","last_fullsized":"2023-11-08T19:26:31.396516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:52.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.267125+0000","last_change":"2023-11-08T19:26:29.293542+0000","last_active":"2023-11-08T19:26:31.267125+0000","last_peered":"2023-11-08T19:26:31.267125+0000","last_clean":"2023-11-08T19:26:31.267125+0000","last_became_active":"2023-11-08T19:26:29.293196+0000","last_became_peered":"2023-11-08T19:26:29.293196+0000","last_unstale":"2023-11-08T19:26:31.267125+0000","last_undegraded":"2023-11-08T19:26:31.267125+0000","last_fullsized":"2023-11-08T19:26:31.267125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:11.337250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266157+0000","last_change":"2023-11-08T19:26:30.292582+0000","last_active":"2023-11-08T19:26:31.266157+0000","last_peered":"2023-11-08T19:26:31.266157+0000","last_clean":"2023-11-08T19:26:31.266157+0000","last_became_active":"2023-11-08T19:26:30.291190+0000","last_became_peered":"2023-11-08T19:26:30.291190+0000","last_unstale":"2023-11-08T19:26:31.266157+0000","last_undegraded":"2023-11-08T19:26:31.266157+0000","last_fullsized":"2023-11-08T19:26:31.266157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:51.089873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396356+0000","last_change":"2023-11-08T19:26:29.295530+0000","last_active":"2023-11-08T19:26:31.396356+0000","last_peered":"2023-11-08T19:26:31.396356+0000","last_clean":"2023-11-08T19:26:31.396356+0000","last_became_active":"2023-11-08T19:26:29.295031+0000","last_became_peered":"2023-11-08T19:26:29.295031+0000","last_unstale":"2023-11-08T19:26:31.396356+0000","last_undegraded":"2023-11-08T19:26:31.396356+0000","last_fullsized":"2023-11-08T19:26:31.396356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:02:45.733805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266091+0000","last_change":"2023-11-08T19:26:30.297083+0000","last_active":"2023-11-08T19:26:31.266091+0000","last_peered":"2023-11-08T19:26:31.266091+0000","last_clean":"2023-11-08T19:26:31.266091+0000","last_became_active":"2023-11-08T19:26:30.294852+0000","last_became_peered":"2023-11-08T19:26:30.294852+0000","last_unstale":"2023-11-08T19:26:31.266091+0000","last_undegraded":"2023-11-08T19:26:31.266091+0000","last_fullsized":"2023-11-08T19:26:31.266091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:35.950771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269478+0000","last_change":"2023-11-08T19:26:29.291169+0000","last_active":"2023-11-08T19:26:31.269478+0000","last_peered":"2023-11-08T19:26:31.269478+0000","last_clean":"2023-11-08T19:26:31.269478+0000","last_became_active":"2023-11-08T19:26:29.290831+0000","last_became_peered":"2023-11-08T19:26:29.290831+0000","last_unstale":"2023-11-08T19:26:31.269478+0000","last_undegraded":"2023-11-08T19:26:31.269478+0000","last_fullsized":"2023-11-08T19:26:31.269478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:28.104951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270278+0000","last_change":"2023-11-08T19:26:30.301149+0000","last_active":"2023-11-08T19:26:31.270278+0000","last_peered":"2023-11-08T19:26:31.270278+0000","last_clean":"2023-11-08T19:26:31.270278+0000","last_became_active":"2023-11-08T19:26:30.300881+0000","last_became_peered":"2023-11-08T19:26:30.300881+0000","last_unstale":"2023-11-08T19:26:31.270278+0000","last_undegraded":"2023-11-08T19:26:31.270278+0000","last_fullsized":"2023-11-08T19:26:31.270278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:49.873565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398786+0000","last_change":"2023-11-08T19:26:29.287399+0000","last_active":"2023-11-08T19:26:31.398786+0000","last_peered":"2023-11-08T19:26:31.398786+0000","last_clean":"2023-11-08T19:26:31.398786+0000","last_became_active":"2023-11-08T19:26:29.287124+0000","last_became_peered":"2023-11-08T19:26:29.287124+0000","last_unstale":"2023-11-08T19:26:31.398786+0000","last_undegraded":"2023-11-08T19:26:31.398786+0000","last_fullsized":"2023-11-08T19:26:31.398786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:08.374863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398422+0000","last_change":"2023-11-08T19:26:30.294708+0000","last_active":"2023-11-08T19:26:31.398422+0000","last_peered":"2023-11-08T19:26:31.398422+0000","last_clean":"2023-11-08T19:26:31.398422+0000","last_became_active":"2023-11-08T19:26:30.294350+0000","last_became_peered":"2023-11-08T19:26:30.294350+0000","last_unstale":"2023-11-08T19:26:31.398422+0000","last_undegraded":"2023-11-08T19:26:31.398422+0000","last_fullsized":"2023-11-08T19:26:31.398422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:04.627278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399236+0000","last_change":"2023-11-08T19:26:29.280768+0000","last_active":"2023-11-08T19:26:31.399236+0000","last_peered":"2023-11-08T19:26:31.399236+0000","last_clean":"2023-11-08T19:26:31.399236+0000","last_became_active":"2023-11-08T19:26:29.280498+0000","last_became_peered":"2023-11-08T19:26:29.280498+0000","last_unstale":"2023-11-08T19:26:31.399236+0000","last_undegraded":"2023-11-08T19:26:31.399236+0000","last_fullsized":"2023-11-08T19:26:31.399236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:24:37.687613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399634+0000","last_change":"2023-11-08T19:26:30.298373+0000","last_active":"2023-11-08T19:26:31.399634+0000","last_peered":"2023-11-08T19:26:31.399634+0000","last_clean":"2023-11-08T19:26:31.399634+0000","last_became_active":"2023-11-08T19:26:30.296777+0000","last_became_peered":"2023-11-08T19:26:30.296777+0000","last_unstale":"2023-11-08T19:26:31.399634+0000","last_undegraded":"2023-11-08T19:26:31.399634+0000","last_fullsized":"2023-11-08T19:26:31.399634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.426409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396709+0000","last_change":"2023-11-08T19:26:29.291077+0000","last_active":"2023-11-08T19:26:31.396709+0000","last_peered":"2023-11-08T19:26:31.396709+0000","last_clean":"2023-11-08T19:26:31.396709+0000","last_became_active":"2023-11-08T19:26:29.290498+0000","last_became_peered":"2023-11-08T19:26:29.290498+0000","last_unstale":"2023-11-08T19:26:31.396709+0000","last_undegraded":"2023-11-08T19:26:31.396709+0000","last_fullsized":"2023-11-08T19:26:31.396709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:56.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396629+0000","last_change":"2023-11-08T19:26:30.298812+0000","last_active":"2023-11-08T19:26:31.396629+0000","last_peered":"2023-11-08T19:26:31.396629+0000","last_clean":"2023-11-08T19:26:31.396629+0000","last_became_active":"2023-11-08T19:26:30.298552+0000","last_became_peered":"2023-11-08T19:26:30.298552+0000","last_unstale":"2023-11-08T19:26:31.396629+0000","last_undegraded":"2023-11-08T19:26:31.396629+0000","last_fullsized":"2023-11-08T19:26:31.396629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398957+0000","last_change":"2023-11-08T19:26:29.279568+0000","last_active":"2023-11-08T19:26:31.398957+0000","last_peered":"2023-11-08T19:26:31.398957+0000","last_clean":"2023-11-08T19:26:31.398957+0000","last_became_active":"2023-11-08T19:26:29.279251+0000","last_became_peered":"2023-11-08T19:26:29.279251+0000","last_unstale":"2023-11-08T19:26:31.398957+0000","last_undegraded":"2023-11-08T19:26:31.398957+0000","last_fullsized":"2023-11-08T19:26:31.398957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:38:11.038990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395384+0000","last_change":"2023-11-08T19:26:30.287043+0000","last_active":"2023-11-08T19:26:31.395384+0000","last_peered":"2023-11-08T19:26:31.395384+0000","last_clean":"2023-11-08T19:26:31.395384+0000","last_became_active":"2023-11-08T19:26:30.286688+0000","last_became_peered":"2023-11-08T19:26:30.286688+0000","last_unstale":"2023-11-08T19:26:31.395384+0000","last_undegraded":"2023-11-08T19:26:31.395384+0000","last_fullsized":"2023-11-08T19:26:31.395384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.513281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398690+0000","last_change":"2023-11-08T19:26:29.289719+0000","last_active":"2023-11-08T19:26:31.398690+0000","last_peered":"2023-11-08T19:26:31.398690+0000","last_clean":"2023-11-08T19:26:31.398690+0000","last_became_active":"2023-11-08T19:26:29.289421+0000","last_became_peered":"2023-11-08T19:26:29.289421+0000","last_unstale":"2023-11-08T19:26:31.398690+0000","last_undegraded":"2023-11-08T19:26:31.398690+0000","last_fullsized":"2023-11-08T19:26:31.398690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:41:12.299524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269924+0000","last_change":"2023-11-08T19:26:30.289369+0000","last_active":"2023-11-08T19:26:31.269924+0000","last_peered":"2023-11-08T19:26:31.269924+0000","last_clean":"2023-11-08T19:26:31.269924+0000","last_became_active":"2023-11-08T19:26:30.289068+0000","last_became_peered":"2023-11-08T19:26:30.289068+0000","last_unstale":"2023-11-08T19:26:31.269924+0000","last_undegraded":"2023-11-08T19:26:31.269924+0000","last_fullsized":"2023-11-08T19:26:31.269924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:53.027281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397732+0000","last_change":"2023-11-08T19:26:29.289157+0000","last_active":"2023-11-08T19:26:31.397732+0000","last_peered":"2023-11-08T19:26:31.397732+0000","last_clean":"2023-11-08T19:26:31.397732+0000","last_became_active":"2023-11-08T19:26:29.288842+0000","last_became_peered":"2023-11-08T19:26:29.288842+0000","last_unstale":"2023-11-08T19:26:31.397732+0000","last_undegraded":"2023-11-08T19:26:31.397732+0000","last_fullsized":"2023-11-08T19:26:31.397732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:51.586934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266326+0000","last_change":"2023-11-08T19:26:30.290576+0000","last_active":"2023-11-08T19:26:31.266326+0000","last_peered":"2023-11-08T19:26:31.266326+0000","last_clean":"2023-11-08T19:26:31.266326+0000","last_became_active":"2023-11-08T19:26:30.288089+0000","last_became_peered":"2023-11-08T19:26:30.288089+0000","last_unstale":"2023-11-08T19:26:31.266326+0000","last_undegraded":"2023-11-08T19:26:31.266326+0000","last_fullsized":"2023-11-08T19:26:31.266326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:48.437724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398390+0000","last_change":"2023-11-08T19:26:29.274366+0000","last_active":"2023-11-08T19:26:31.398390+0000","last_peered":"2023-11-08T19:26:31.398390+0000","last_clean":"2023-11-08T19:26:31.398390+0000","last_became_active":"2023-11-08T19:26:29.274047+0000","last_became_peered":"2023-11-08T19:26:29.274047+0000","last_unstale":"2023-11-08T19:26:31.398390+0000","last_undegraded":"2023-11-08T19:26:31.398390+0000","last_fullsized":"2023-11-08T19:26:31.398390+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:37.210520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396805+0000","last_change":"2023-11-08T19:26:30.296734+0000","last_active":"2023-11-08T19:26:31.396805+0000","last_peered":"2023-11-08T19:26:31.396805+0000","last_clean":"2023-11-08T19:26:31.396805+0000","last_became_active":"2023-11-08T19:26:30.295198+0000","last_became_peered":"2023-11-08T19:26:30.295198+0000","last_unstale":"2023-11-08T19:26:31.396805+0000","last_undegraded":"2023-11-08T19:26:31.396805+0000","last_fullsized":"2023-11-08T19:26:31.396805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.513281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269963+0000","last_change":"2023-11-08T19:26:29.288157+0000","last_active":"2023-11-08T19:26:31.269963+0000","last_peered":"2023-11-08T19:26:31.269963+0000","last_clean":"2023-11-08T19:26:31.269963+0000","last_became_active":"2023-11-08T19:26:29.286582+0000","last_became_peered":"2023-11-08T19:26:29.286582+0000","last_unstale":"2023-11-08T19:26:31.269963+0000","last_undegraded":"2023-11-08T19:26:31.269963+0000","last_fullsized":"2023-11-08T19:26:31.269963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:33.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397573+0000","last_change":"2023-11-08T19:26:30.287582+0000","last_active":"2023-11-08T19:26:31.397573+0000","last_peered":"2023-11-08T19:26:31.397573+0000","last_clean":"2023-11-08T19:26:31.397573+0000","last_became_active":"2023-11-08T19:26:30.287248+0000","last_became_peered":"2023-11-08T19:26:30.287248+0000","last_unstale":"2023-11-08T19:26:31.397573+0000","last_undegraded":"2023-11-08T19:26:31.397573+0000","last_fullsized":"2023-11-08T19:26:31.397573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:04.627278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399483+0000","last_change":"2023-11-08T19:26:29.282704+0000","last_active":"2023-11-08T19:26:31.399483+0000","last_peered":"2023-11-08T19:26:31.399483+0000","last_clean":"2023-11-08T19:26:31.399483+0000","last_became_active":"2023-11-08T19:26:29.282409+0000","last_became_peered":"2023-11-08T19:26:29.282409+0000","last_unstale":"2023-11-08T19:26:31.399483+0000","last_undegraded":"2023-11-08T19:26:31.399483+0000","last_fullsized":"2023-11-08T19:26:31.399483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:05.247438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265604+0000","last_change":"2023-11-08T19:26:30.288274+0000","last_active":"2023-11-08T19:26:31.265604+0000","last_peered":"2023-11-08T19:26:31.265604+0000","last_clean":"2023-11-08T19:26:31.265604+0000","last_became_active":"2023-11-08T19:26:30.287595+0000","last_became_peered":"2023-11-08T19:26:30.287595+0000","last_unstale":"2023-11-08T19:26:31.265604+0000","last_undegraded":"2023-11-08T19:26:31.265604+0000","last_fullsized":"2023-11-08T19:26:31.265604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:49.873565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396045+0000","last_change":"2023-11-08T19:26:29.275592+0000","last_active":"2023-11-08T19:26:31.396045+0000","last_peered":"2023-11-08T19:26:31.396045+0000","last_clean":"2023-11-08T19:26:31.396045+0000","last_became_active":"2023-11-08T19:26:29.275331+0000","last_became_peered":"2023-11-08T19:26:29.275331+0000","last_unstale":"2023-11-08T19:26:31.396045+0000","last_undegraded":"2023-11-08T19:26:31.396045+0000","last_fullsized":"2023-11-08T19:26:31.396045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:56.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396464+0000","last_change":"2023-11-08T19:26:30.281266+0000","last_active":"2023-11-08T19:26:31.396464+0000","last_peered":"2023-11-08T19:26:31.396464+0000","last_clean":"2023-11-08T19:26:31.396464+0000","last_became_active":"2023-11-08T19:26:30.280526+0000","last_became_peered":"2023-11-08T19:26:30.280526+0000","last_unstale":"2023-11-08T19:26:31.396464+0000","last_undegraded":"2023-11-08T19:26:31.396464+0000","last_fullsized":"2023-11-08T19:26:31.396464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395772+0000","last_change":"2023-11-08T19:26:29.277771+0000","last_active":"2023-11-08T19:26:31.395772+0000","last_peered":"2023-11-08T19:26:31.395772+0000","last_clean":"2023-11-08T19:26:31.395772+0000","last_became_active":"2023-11-08T19:26:29.277549+0000","last_became_peered":"2023-11-08T19:26:29.277549+0000","last_unstale":"2023-11-08T19:26:31.395772+0000","last_undegraded":"2023-11-08T19:26:31.395772+0000","last_fullsized":"2023-11-08T19:26:31.395772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:37.487567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397347+0000","last_change":"2023-11-08T19:26:30.293828+0000","last_active":"2023-11-08T19:26:31.397347+0000","last_peered":"2023-11-08T19:26:31.397347+0000","last_clean":"2023-11-08T19:26:31.397347+0000","last_became_active":"2023-11-08T19:26:30.293566+0000","last_became_peered":"2023-11-08T19:26:30.293566+0000","last_unstale":"2023-11-08T19:26:31.397347+0000","last_undegraded":"2023-11-08T19:26:31.397347+0000","last_fullsized":"2023-11-08T19:26:31.397347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:52.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268030+0000","last_change":"2023-11-08T19:26:29.276400+0000","last_active":"2023-11-08T19:26:31.268030+0000","last_peered":"2023-11-08T19:26:31.268030+0000","last_clean":"2023-11-08T19:26:31.268030+0000","last_became_active":"2023-11-08T19:26:29.275593+0000","last_became_peered":"2023-11-08T19:26:29.275593+0000","last_unstale":"2023-11-08T19:26:31.268030+0000","last_undegraded":"2023-11-08T19:26:31.268030+0000","last_fullsized":"2023-11-08T19:26:31.268030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:18.797947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397149+0000","last_change":"2023-11-08T19:26:30.296837+0000","last_active":"2023-11-08T19:26:31.397149+0000","last_peered":"2023-11-08T19:26:31.397149+0000","last_clean":"2023-11-08T19:26:31.397149+0000","last_became_active":"2023-11-08T19:26:30.295170+0000","last_became_peered":"2023-11-08T19:26:30.295170+0000","last_unstale":"2023-11-08T19:26:31.397149+0000","last_undegraded":"2023-11-08T19:26:31.397149+0000","last_fullsized":"2023-11-08T19:26:31.397149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396358+0000","last_change":"2023-11-08T19:26:29.288055+0000","last_active":"2023-11-08T19:26:31.396358+0000","last_peered":"2023-11-08T19:26:31.396358+0000","last_clean":"2023-11-08T19:26:31.396358+0000","last_became_active":"2023-11-08T19:26:29.287741+0000","last_became_peered":"2023-11-08T19:26:29.287741+0000","last_unstale":"2023-11-08T19:26:31.396358+0000","last_undegraded":"2023-11-08T19:26:31.396358+0000","last_fullsized":"2023-11-08T19:26:31.396358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:11:31.055751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397163+0000","last_change":"2023-11-08T19:26:30.297062+0000","last_active":"2023-11-08T19:26:31.397163+0000","last_peered":"2023-11-08T19:26:31.397163+0000","last_clean":"2023-11-08T19:26:31.397163+0000","last_became_active":"2023-11-08T19:26:30.296426+0000","last_became_peered":"2023-11-08T19:26:30.296426+0000","last_unstale":"2023-11-08T19:26:31.397163+0000","last_undegraded":"2023-11-08T19:26:31.397163+0000","last_fullsized":"2023-11-08T19:26:31.397163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265791+0000","last_change":"2023-11-08T19:26:29.282479+0000","last_active":"2023-11-08T19:26:31.265791+0000","last_peered":"2023-11-08T19:26:31.265791+0000","last_clean":"2023-11-08T19:26:31.265791+0000","last_became_active":"2023-11-08T19:26:29.280695+0000","last_became_peered":"2023-11-08T19:26:29.280695+0000","last_unstale":"2023-11-08T19:26:31.265791+0000","last_undegraded":"2023-11-08T19:26:31.265791+0000","last_fullsized":"2023-11-08T19:26:31.265791+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:03.612401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269004+0000","last_change":"2023-11-08T19:26:30.295473+0000","last_active":"2023-11-08T19:26:31.269004+0000","last_peered":"2023-11-08T19:26:31.269004+0000","last_clean":"2023-11-08T19:26:31.269004+0000","last_became_active":"2023-11-08T19:26:30.294289+0000","last_became_peered":"2023-11-08T19:26:30.294289+0000","last_unstale":"2023-11-08T19:26:31.269004+0000","last_undegraded":"2023-11-08T19:26:31.269004+0000","last_fullsized":"2023-11-08T19:26:31.269004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:11.791162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397971+0000","last_change":"2023-11-08T19:26:29.290756+0000","last_active":"2023-11-08T19:26:31.397971+0000","last_peered":"2023-11-08T19:26:31.397971+0000","last_clean":"2023-11-08T19:26:31.397971+0000","last_became_active":"2023-11-08T19:26:29.290392+0000","last_became_peered":"2023-11-08T19:26:29.290392+0000","last_unstale":"2023-11-08T19:26:31.397971+0000","last_undegraded":"2023-11-08T19:26:31.397971+0000","last_fullsized":"2023-11-08T19:26:31.397971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:40.708443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-08T19:26:30.271560+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:26:30.271560+0000","last_peered":"2023-11-08T19:26:30.271560+0000","last_clean":"2023-11-08T19:26:30.271560+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:26:30.271560+0000","last_undegraded":"2023-11-08T19:26:30.271560+0000","last_fullsized":"2023-11-08T19:26:30.271560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.4589999999999996,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":1.1679999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.96699999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":4.6980000000000004,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.3260000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.204},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.6150000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.2949999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.399},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3700000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.415},{"interface":"front","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.123}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":196293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.1559999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.747},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":4.6399999999999997,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.93300000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.609,"5min":4.609,"15min":4.609},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.4339999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.383},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.0860000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.47}]},{"osd":5,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.23},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.732}]},{"osd":7,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.526},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.337}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":189203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.339}]},{"osd":1,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":7.6779999999999999,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.2350000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":7.6479999999999997,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.0129999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4650000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.573,"5min":1.573,"15min":1.573},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.90200000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.546,"15min":1.546},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2849999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"min":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.488},{"interface":"front","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.526}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":189695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.62},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.9940000000000002,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.8319999999999999,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":4.1710000000000003,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.077}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":4.335,"5min":4.335,"15min":4.335},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.645,"5min":1.645,"15min":1.645},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":3.9239999999999999,"5min":3.9239999999999999,"15min":3.9239999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.645,"15min":1.645},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":1.262,"5min":1.262,"15min":1.262},"max":{"1min":4.4950000000000001,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.1830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.3999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":4.5750000000000002,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":1.1299999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.778}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":4.2770000000000001,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.861},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.54}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":2.149},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.2380000000000004,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":2.375}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":4.2220000000000004,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":2.4860000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":2.528},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.736}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":188142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.436}]},{"osd":1,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.75600000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.032}]},{"osd":5,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.90800000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.95499999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.2370000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":189181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.246},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.0449999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.004},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.1799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":778474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.2889999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1180000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.95999999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.198},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.397}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.45},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.2190000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.5710000000000002,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.92900000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.256},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.306}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:35.072 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:26:35.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.321+0000 7f0f85621640 1 Processor -- start 2023-11-08T19:26:35.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.322+0000 7f0f85621640 1 -- start start 2023-11-08T19:26:35.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.323+0000 7f0f85621640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f8015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.323+0000 7f0f85621640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0f800cb170 0x7f0f8006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.323+0000 7f0f85621640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f800c3a50 con 0x7f0f800cad30 2023-11-08T19:26:35.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.323+0000 7f0f85621640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f80070040 con 0x7f0f800cb170 2023-11-08T19:26:35.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.323+0000 7f0f85621640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80144b80 con 0x7f0f801447a0 2023-11-08T19:26:35.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.324+0000 7f0f7effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f8015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.324+0000 7f0f7f7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0f800cb170 0x7f0f8006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.324+0000 7f0f7effd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f8015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:35962/0 (socket says 172.21.15.119:35962) 2023-11-08T19:26:35.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.324+0000 7f0f7effd640 1 -- 172.21.15.119:0/295178234 learned_addr learned my addr 172.21.15.119:0/295178234 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:35.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.325+0000 7f0f7effd640 1 -- 172.21.15.119:0/295178234 >> v2:172.21.15.188:3300/0 conn(0x7f0f800cb170 msgr2=0x7f0f8006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:35.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.325+0000 7f0f7effd640 1 --2- 172.21.15.119:0/295178234 >> v2:172.21.15.188:3300/0 conn(0x7f0f800cb170 0x7f0f8006a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.325+0000 7f0f7effd640 1 -- 172.21.15.119:0/295178234 >> v1:172.21.15.188:6789/0 conn(0x7f0f800cad30 legacy=0x7f0f8006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.325+0000 7f0f7effd640 1 -- 172.21.15.119:0/295178234 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800c9150 con 0x7f0f801447a0 2023-11-08T19:26:35.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.326+0000 7f0f7effd640 1 --2- 172.21.15.119:0/295178234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f8015f4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009870 tx=0x7f0f7002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35bb2b724f56e103 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.327+0000 7f0f7dffb640 1 -- 172.21.15.119:0/295178234 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70030840 con 0x7f0f801447a0 2023-11-08T19:26:35.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.327+0000 7f0f7dffb640 1 -- 172.21.15.119:0/295178234 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f700309a0 con 0x7f0f801447a0 2023-11-08T19:26:35.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.327+0000 7f0f7dffb640 1 -- 172.21.15.119:0/295178234 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f70030d30 con 0x7f0f801447a0 2023-11-08T19:26:35.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 -- 172.21.15.119:0/295178234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 msgr2=0x7f0f8015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 --2- 172.21.15.119:0/295178234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f8015f4c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009870 tx=0x7f0f7002b980 comp rx=0 tx=0).stop 2023-11-08T19:26:35.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 -- 172.21.15.119:0/295178234 shutdown_connections 2023-11-08T19:26:35.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 --2- 172.21.15.119:0/295178234 >> v2:172.21.15.188:3300/0 conn(0x7f0f800cb170 0x7f0f8006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 --2- 172.21.15.119:0/295178234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f8015f4c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 -- 172.21.15.119:0/295178234 >> 172.21.15.119:0/295178234 conn(0x7f0f800cb610 msgr2=0x7f0f800cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 -- 172.21.15.119:0/295178234 shutdown_connections 2023-11-08T19:26:35.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.328+0000 7f0f85621640 1 -- 172.21.15.119:0/295178234 wait complete. 2023-11-08T19:26:35.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.330+0000 7f0f85621640 1 Processor -- start 2023-11-08T19:26:35.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.330+0000 7f0f85621640 1 -- start start 2023-11-08T19:26:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.331+0000 7f0f85621640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800cad30 0x7f0f8013f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.331+0000 7f0f85621640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 0x7f0f8013ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f85621640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f801f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f85621640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801f7120 con 0x7f0f800cb170 2023-11-08T19:26:35.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f85621640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80141840 con 0x7f0f800cad30 2023-11-08T19:26:35.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f85621640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f800b4300 con 0x7f0f801447a0 2023-11-08T19:26:35.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 0x7f0f8013ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 0x7f0f8013ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47734/0 (socket says 172.21.15.119:47734) 2023-11-08T19:26:35.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 -- 172.21.15.119:0/2207734311 learned_addr learned my addr 172.21.15.119:0/2207734311 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:35.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7effd640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800cad30 0x7f0f8013f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7f7fe640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f801f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 -- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 msgr2=0x7f0f801f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f801f6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 -- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800cad30 msgr2=0x7f0f8013f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800cad30 0x7f0f8013f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7e7fc640 1 -- 172.21.15.119:0/2207734311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f700094d0 con 0x7f0f800cb170 2023-11-08T19:26:35.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.332+0000 7f0f7effd640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800cad30 0x7f0f8013f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:35.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.333+0000 7f0f7e7fc640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 0x7f0f8013ed20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c400 tx=0x7f0f6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.333+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680052d0 con 0x7f0f800cb170 2023-11-08T19:26:35.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.333+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68005430 con 0x7f0f800cb170 2023-11-08T19:26:35.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.333+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f800b45e0 con 0x7f0f800cb170 2023-11-08T19:26:35.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.333+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68003830 con 0x7f0f800cb170 2023-11-08T19:26:35.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.333+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f800b48c0 con 0x7f0f800cb170 2023-11-08T19:26:35.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.334+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f801416a0 con 0x7f0f800cb170 2023-11-08T19:26:35.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.335+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f68003990 con 0x7f0f800cb170 2023-11-08T19:26:35.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.336+0000 7f0f777fe640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f6006ae70 0x7f0f6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.336+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f0f68088ce0 con 0x7f0f800cb170 2023-11-08T19:26:35.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.336+0000 7f0f7effd640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f6006ae70 0x7f0f6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.337+0000 7f0f7effd640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f6006ae70 0x7f0f6006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f70036000 tx=0x7f0f7003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.340+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f801416a0 con 0x7f0f800cb170 2023-11-08T19:26:35.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.529+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f800c96f0 con 0x7f0f6006ae70 2023-11-08T19:26:35.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.540+0000 7f0f777fe640 1 -- 172.21.15.119:0/2207734311 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286750 (secure 0 0 0) 0x7f0f70030c10 con 0x7f0f6006ae70 2023-11-08T19:26:35.547 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:35.549 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:26:35.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f6006ae70 msgr2=0x7f0f6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f6006ae70 0x7f0f6006d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f70036000 tx=0x7f0f7003b040 comp rx=0 tx=0).stop 2023-11-08T19:26:35.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 msgr2=0x7f0f8013ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 0x7f0f8013ed20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0f6800c400 tx=0x7f0f6800c8c0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 shutdown_connections 2023-11-08T19:26:35.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f6006ae70 0x7f0f6006d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f801447a0 0x7f0f801f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f800cb170 0x7f0f8013ed20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 --2- 172.21.15.119:0/2207734311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f800cad30 0x7f0f8013f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 >> 172.21.15.119:0/2207734311 conn(0x7f0f800cb610 msgr2=0x7f0f800c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 shutdown_connections 2023-11-08T19:26:35.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.546+0000 7f0f85621640 1 -- 172.21.15.119:0/2207734311 wait complete. 2023-11-08T19:26:35.569 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-11-08T19:26:34.692556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":158,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220856,"kb_used_data":4664,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753864,"statfs":{"total":773094113280,"available":772867956736,"internally_reserved":0,"allocated":4775936,"data_stored":2709486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001923"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266157+0000","last_change":"2023-11-08T19:26:30.292305+0000","last_active":"2023-11-08T19:26:31.266157+0000","last_peered":"2023-11-08T19:26:31.266157+0000","last_clean":"2023-11-08T19:26:31.266157+0000","last_became_active":"2023-11-08T19:26:30.292030+0000","last_became_peered":"2023-11-08T19:26:30.292030+0000","last_unstale":"2023-11-08T19:26:31.266157+0000","last_undegraded":"2023-11-08T19:26:31.266157+0000","last_fullsized":"2023-11-08T19:26:31.266157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:13.373857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265463+0000","last_change":"2023-11-08T19:26:30.296762+0000","last_active":"2023-11-08T19:26:31.265463+0000","last_peered":"2023-11-08T19:26:31.265463+0000","last_clean":"2023-11-08T19:26:31.265463+0000","last_became_active":"2023-11-08T19:26:30.296491+0000","last_became_peered":"2023-11-08T19:26:30.296491+0000","last_unstale":"2023-11-08T19:26:31.265463+0000","last_undegraded":"2023-11-08T19:26:31.265463+0000","last_fullsized":"2023-11-08T19:26:31.265463+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:59.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395518+0000","last_change":"2023-11-08T19:26:30.292119+0000","last_active":"2023-11-08T19:26:31.395518+0000","last_peered":"2023-11-08T19:26:31.395518+0000","last_clean":"2023-11-08T19:26:31.395518+0000","last_became_active":"2023-11-08T19:26:30.291809+0000","last_became_peered":"2023-11-08T19:26:30.291809+0000","last_unstale":"2023-11-08T19:26:31.395518+0000","last_undegraded":"2023-11-08T19:26:31.395518+0000","last_fullsized":"2023-11-08T19:26:31.395518+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:52.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399333+0000","last_change":"2023-11-08T19:26:30.291436+0000","last_active":"2023-11-08T19:26:31.399333+0000","last_peered":"2023-11-08T19:26:31.399333+0000","last_clean":"2023-11-08T19:26:31.399333+0000","last_became_active":"2023-11-08T19:26:30.289196+0000","last_became_peered":"2023-11-08T19:26:30.289196+0000","last_unstale":"2023-11-08T19:26:31.399333+0000","last_undegraded":"2023-11-08T19:26:31.399333+0000","last_fullsized":"2023-11-08T19:26:31.399333+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:25:46.534304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399048+0000","last_change":"2023-11-08T19:26:30.291337+0000","last_active":"2023-11-08T19:26:31.399048+0000","last_peered":"2023-11-08T19:26:31.399048+0000","last_clean":"2023-11-08T19:26:31.399048+0000","last_became_active":"2023-11-08T19:26:30.291053+0000","last_became_peered":"2023-11-08T19:26:30.291053+0000","last_unstale":"2023-11-08T19:26:31.399048+0000","last_undegraded":"2023-11-08T19:26:31.399048+0000","last_fullsized":"2023-11-08T19:26:31.399048+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:03.031932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396341+0000","last_change":"2023-11-08T19:26:30.293253+0000","last_active":"2023-11-08T19:26:31.396341+0000","last_peered":"2023-11-08T19:26:31.396341+0000","last_clean":"2023-11-08T19:26:31.396341+0000","last_became_active":"2023-11-08T19:26:30.292988+0000","last_became_peered":"2023-11-08T19:26:30.292988+0000","last_unstale":"2023-11-08T19:26:31.396341+0000","last_undegraded":"2023-11-08T19:26:31.396341+0000","last_fullsized":"2023-11-08T19:26:31.396341+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:09.389740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268478+0000","last_change":"2023-11-08T19:26:30.300921+0000","last_active":"2023-11-08T19:26:31.268478+0000","last_peered":"2023-11-08T19:26:31.268478+0000","last_clean":"2023-11-08T19:26:31.268478+0000","last_became_active":"2023-11-08T19:26:30.300622+0000","last_became_peered":"2023-11-08T19:26:30.300622+0000","last_unstale":"2023-11-08T19:26:31.268478+0000","last_undegraded":"2023-11-08T19:26:31.268478+0000","last_fullsized":"2023-11-08T19:26:31.268478+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:02.915393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265503+0000","last_change":"2023-11-08T19:26:30.291795+0000","last_active":"2023-11-08T19:26:31.265503+0000","last_peered":"2023-11-08T19:26:31.265503+0000","last_clean":"2023-11-08T19:26:31.265503+0000","last_became_active":"2023-11-08T19:26:30.290648+0000","last_became_peered":"2023-11-08T19:26:30.290648+0000","last_unstale":"2023-11-08T19:26:31.265503+0000","last_undegraded":"2023-11-08T19:26:31.265503+0000","last_fullsized":"2023-11-08T19:26:31.265503+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:07.011653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395933+0000","last_change":"2023-11-08T19:26:30.293533+0000","last_active":"2023-11-08T19:26:31.395933+0000","last_peered":"2023-11-08T19:26:31.395933+0000","last_clean":"2023-11-08T19:26:31.395933+0000","last_became_active":"2023-11-08T19:26:30.292642+0000","last_became_peered":"2023-11-08T19:26:30.292642+0000","last_unstale":"2023-11-08T19:26:31.395933+0000","last_undegraded":"2023-11-08T19:26:31.395933+0000","last_fullsized":"2023-11-08T19:26:31.395933+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269803+0000","last_change":"2023-11-08T19:26:30.293410+0000","last_active":"2023-11-08T19:26:31.269803+0000","last_peered":"2023-11-08T19:26:31.269803+0000","last_clean":"2023-11-08T19:26:31.269803+0000","last_became_active":"2023-11-08T19:26:30.293155+0000","last_became_peered":"2023-11-08T19:26:30.293155+0000","last_unstale":"2023-11-08T19:26:31.269803+0000","last_undegraded":"2023-11-08T19:26:31.269803+0000","last_fullsized":"2023-11-08T19:26:31.269803+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:07.011653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269472+0000","last_change":"2023-11-08T19:26:30.290241+0000","last_active":"2023-11-08T19:26:31.269472+0000","last_peered":"2023-11-08T19:26:31.269472+0000","last_clean":"2023-11-08T19:26:31.269472+0000","last_became_active":"2023-11-08T19:26:30.289967+0000","last_became_peered":"2023-11-08T19:26:30.289967+0000","last_unstale":"2023-11-08T19:26:31.269472+0000","last_undegraded":"2023-11-08T19:26:31.269472+0000","last_fullsized":"2023-11-08T19:26:31.269472+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:59.297108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395309+0000","last_change":"2023-11-08T19:26:30.291603+0000","last_active":"2023-11-08T19:26:31.395309+0000","last_peered":"2023-11-08T19:26:31.395309+0000","last_clean":"2023-11-08T19:26:31.395309+0000","last_became_active":"2023-11-08T19:26:30.291260+0000","last_became_peered":"2023-11-08T19:26:30.291260+0000","last_unstale":"2023-11-08T19:26:31.395309+0000","last_undegraded":"2023-11-08T19:26:31.395309+0000","last_fullsized":"2023-11-08T19:26:31.395309+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:47.968547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268428+0000","last_change":"2023-11-08T19:26:30.298138+0000","last_active":"2023-11-08T19:26:31.268428+0000","last_peered":"2023-11-08T19:26:31.268428+0000","last_clean":"2023-11-08T19:26:31.268428+0000","last_became_active":"2023-11-08T19:26:30.297875+0000","last_became_peered":"2023-11-08T19:26:30.297875+0000","last_unstale":"2023-11-08T19:26:31.268428+0000","last_undegraded":"2023-11-08T19:26:31.268428+0000","last_fullsized":"2023-11-08T19:26:31.268428+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:04.627278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396793+0000","last_change":"2023-11-08T19:26:30.296178+0000","last_active":"2023-11-08T19:26:31.396793+0000","last_peered":"2023-11-08T19:26:31.396793+0000","last_clean":"2023-11-08T19:26:31.396793+0000","last_became_active":"2023-11-08T19:26:30.293002+0000","last_became_peered":"2023-11-08T19:26:30.293002+0000","last_unstale":"2023-11-08T19:26:31.396793+0000","last_undegraded":"2023-11-08T19:26:31.396793+0000","last_fullsized":"2023-11-08T19:26:31.396793+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268805+0000","last_change":"2023-11-08T19:26:30.297919+0000","last_active":"2023-11-08T19:26:31.268805+0000","last_peered":"2023-11-08T19:26:31.268805+0000","last_clean":"2023-11-08T19:26:31.268805+0000","last_became_active":"2023-11-08T19:26:30.297574+0000","last_became_peered":"2023-11-08T19:26:30.297574+0000","last_unstale":"2023-11-08T19:26:31.268805+0000","last_undegraded":"2023-11-08T19:26:31.268805+0000","last_fullsized":"2023-11-08T19:26:31.268805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:13:21.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396225+0000","last_change":"2023-11-08T19:26:30.290191+0000","last_active":"2023-11-08T19:26:31.396225+0000","last_peered":"2023-11-08T19:26:31.396225+0000","last_clean":"2023-11-08T19:26:31.396225+0000","last_became_active":"2023-11-08T19:26:30.289922+0000","last_became_peered":"2023-11-08T19:26:30.289922+0000","last_unstale":"2023-11-08T19:26:31.396225+0000","last_undegraded":"2023-11-08T19:26:31.396225+0000","last_fullsized":"2023-11-08T19:26:31.396225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.513281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398270+0000","last_change":"2023-11-08T19:26:30.303342+0000","last_active":"2023-11-08T19:26:31.398270+0000","last_peered":"2023-11-08T19:26:31.398270+0000","last_clean":"2023-11-08T19:26:31.398270+0000","last_became_active":"2023-11-08T19:26:30.302853+0000","last_became_peered":"2023-11-08T19:26:30.302853+0000","last_unstale":"2023-11-08T19:26:31.398270+0000","last_undegraded":"2023-11-08T19:26:31.398270+0000","last_fullsized":"2023-11-08T19:26:31.398270+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:13:21.925214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269638+0000","last_change":"2023-11-08T19:26:30.286808+0000","last_active":"2023-11-08T19:26:31.269638+0000","last_peered":"2023-11-08T19:26:31.269638+0000","last_clean":"2023-11-08T19:26:31.269638+0000","last_became_active":"2023-11-08T19:26:30.285864+0000","last_became_peered":"2023-11-08T19:26:30.285864+0000","last_unstale":"2023-11-08T19:26:31.269638+0000","last_undegraded":"2023-11-08T19:26:31.269638+0000","last_fullsized":"2023-11-08T19:26:31.269638+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:54.424411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399066+0000","last_change":"2023-11-08T19:26:30.298280+0000","last_active":"2023-11-08T19:26:31.399066+0000","last_peered":"2023-11-08T19:26:31.399066+0000","last_clean":"2023-11-08T19:26:31.399066+0000","last_became_active":"2023-11-08T19:26:30.297903+0000","last_became_peered":"2023-11-08T19:26:30.297903+0000","last_unstale":"2023-11-08T19:26:31.399066+0000","last_undegraded":"2023-11-08T19:26:31.399066+0000","last_fullsized":"2023-11-08T19:26:31.399066+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:53.174998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.267912+0000","last_change":"2023-11-08T19:26:30.289026+0000","last_active":"2023-11-08T19:26:31.267912+0000","last_peered":"2023-11-08T19:26:31.267912+0000","last_clean":"2023-11-08T19:26:31.267912+0000","last_became_active":"2023-11-08T19:26:30.287461+0000","last_became_peered":"2023-11-08T19:26:30.287461+0000","last_unstale":"2023-11-08T19:26:31.267912+0000","last_undegraded":"2023-11-08T19:26:31.267912+0000","last_fullsized":"2023-11-08T19:26:31.267912+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:11:32.070628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268778+0000","last_change":"2023-11-08T19:26:30.301598+0000","last_active":"2023-11-08T19:26:31.268778+0000","last_peered":"2023-11-08T19:26:31.268778+0000","last_clean":"2023-11-08T19:26:31.268778+0000","last_became_active":"2023-11-08T19:26:30.301322+0000","last_became_peered":"2023-11-08T19:26:30.301322+0000","last_unstale":"2023-11-08T19:26:31.268778+0000","last_undegraded":"2023-11-08T19:26:31.268778+0000","last_fullsized":"2023-11-08T19:26:31.268778+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:58.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397422+0000","last_change":"2023-11-08T19:26:30.296888+0000","last_active":"2023-11-08T19:26:31.397422+0000","last_peered":"2023-11-08T19:26:31.397422+0000","last_clean":"2023-11-08T19:26:31.397422+0000","last_became_active":"2023-11-08T19:26:30.296541+0000","last_became_peered":"2023-11-08T19:26:30.296541+0000","last_unstale":"2023-11-08T19:26:31.397422+0000","last_undegraded":"2023-11-08T19:26:31.397422+0000","last_fullsized":"2023-11-08T19:26:31.397422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:29.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397319+0000","last_change":"2023-11-08T19:26:30.297332+0000","last_active":"2023-11-08T19:26:31.397319+0000","last_peered":"2023-11-08T19:26:31.397319+0000","last_clean":"2023-11-08T19:26:31.397319+0000","last_became_active":"2023-11-08T19:26:30.296705+0000","last_became_peered":"2023-11-08T19:26:30.296705+0000","last_unstale":"2023-11-08T19:26:31.397319+0000","last_undegraded":"2023-11-08T19:26:31.397319+0000","last_fullsized":"2023-11-08T19:26:31.397319+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:34.753858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398254+0000","last_change":"2023-11-08T19:26:30.283303+0000","last_active":"2023-11-08T19:26:31.398254+0000","last_peered":"2023-11-08T19:26:31.398254+0000","last_clean":"2023-11-08T19:26:31.398254+0000","last_became_active":"2023-11-08T19:26:30.283026+0000","last_became_peered":"2023-11-08T19:26:30.283026+0000","last_unstale":"2023-11-08T19:26:31.398254+0000","last_undegraded":"2023-11-08T19:26:31.398254+0000","last_fullsized":"2023-11-08T19:26:31.398254+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:27:22.361334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270220+0000","last_change":"2023-11-08T19:26:30.282351+0000","last_active":"2023-11-08T19:26:31.270220+0000","last_peered":"2023-11-08T19:26:31.270220+0000","last_clean":"2023-11-08T19:26:31.270220+0000","last_became_active":"2023-11-08T19:26:30.282016+0000","last_became_peered":"2023-11-08T19:26:30.282016+0000","last_unstale":"2023-11-08T19:26:31.270220+0000","last_undegraded":"2023-11-08T19:26:31.270220+0000","last_fullsized":"2023-11-08T19:26:31.270220+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:57.123261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397886+0000","last_change":"2023-11-08T19:26:30.302590+0000","last_active":"2023-11-08T19:26:31.397886+0000","last_peered":"2023-11-08T19:26:31.397886+0000","last_clean":"2023-11-08T19:26:31.397886+0000","last_became_active":"2023-11-08T19:26:30.302368+0000","last_became_peered":"2023-11-08T19:26:30.302368+0000","last_unstale":"2023-11-08T19:26:31.397886+0000","last_undegraded":"2023-11-08T19:26:31.397886+0000","last_fullsized":"2023-11-08T19:26:31.397886+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269026+0000","last_change":"2023-11-08T19:26:30.283391+0000","last_active":"2023-11-08T19:26:31.269026+0000","last_peered":"2023-11-08T19:26:31.269026+0000","last_clean":"2023-11-08T19:26:31.269026+0000","last_became_active":"2023-11-08T19:26:30.283079+0000","last_became_peered":"2023-11-08T19:26:30.283079+0000","last_unstale":"2023-11-08T19:26:31.269026+0000","last_undegraded":"2023-11-08T19:26:31.269026+0000","last_fullsized":"2023-11-08T19:26:31.269026+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:35:37.209163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269086+0000","last_change":"2023-11-08T19:26:30.293540+0000","last_active":"2023-11-08T19:26:31.269086+0000","last_peered":"2023-11-08T19:26:31.269086+0000","last_clean":"2023-11-08T19:26:31.269086+0000","last_became_active":"2023-11-08T19:26:30.292296+0000","last_became_peered":"2023-11-08T19:26:30.292296+0000","last_unstale":"2023-11-08T19:26:31.269086+0000","last_undegraded":"2023-11-08T19:26:31.269086+0000","last_fullsized":"2023-11-08T19:26:31.269086+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:05.882531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268053+0000","last_change":"2023-11-08T19:26:30.295364+0000","last_active":"2023-11-08T19:26:31.268053+0000","last_peered":"2023-11-08T19:26:31.268053+0000","last_clean":"2023-11-08T19:26:31.268053+0000","last_became_active":"2023-11-08T19:26:30.295042+0000","last_became_peered":"2023-11-08T19:26:30.295042+0000","last_unstale":"2023-11-08T19:26:31.268053+0000","last_undegraded":"2023-11-08T19:26:31.268053+0000","last_fullsized":"2023-11-08T19:26:31.268053+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:41.723320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.267788+0000","last_change":"2023-11-08T19:26:30.291570+0000","last_active":"2023-11-08T19:26:31.267788+0000","last_peered":"2023-11-08T19:26:31.267788+0000","last_clean":"2023-11-08T19:26:31.267788+0000","last_became_active":"2023-11-08T19:26:30.290459+0000","last_became_peered":"2023-11-08T19:26:30.290459+0000","last_unstale":"2023-11-08T19:26:31.267788+0000","last_undegraded":"2023-11-08T19:26:31.267788+0000","last_fullsized":"2023-11-08T19:26:31.267788+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:33.467785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397691+0000","last_change":"2023-11-08T19:26:30.297583+0000","last_active":"2023-11-08T19:26:31.397691+0000","last_peered":"2023-11-08T19:26:31.397691+0000","last_clean":"2023-11-08T19:26:31.397691+0000","last_became_active":"2023-11-08T19:26:30.297075+0000","last_became_peered":"2023-11-08T19:26:30.297075+0000","last_unstale":"2023-11-08T19:26:31.397691+0000","last_undegraded":"2023-11-08T19:26:31.397691+0000","last_fullsized":"2023-11-08T19:26:31.397691+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:02.915393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396597+0000","last_change":"2023-11-08T19:26:30.299427+0000","last_active":"2023-11-08T19:26:31.396597+0000","last_peered":"2023-11-08T19:26:31.396597+0000","last_clean":"2023-11-08T19:26:31.396597+0000","last_became_active":"2023-11-08T19:26:30.299151+0000","last_became_peered":"2023-11-08T19:26:30.299151+0000","last_unstale":"2023-11-08T19:26:31.396597+0000","last_undegraded":"2023-11-08T19:26:31.396597+0000","last_fullsized":"2023-11-08T19:26:31.396597+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:47.968547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397986+0000","last_change":"2023-11-08T19:26:30.294571+0000","last_active":"2023-11-08T19:26:31.397986+0000","last_peered":"2023-11-08T19:26:31.397986+0000","last_clean":"2023-11-08T19:26:31.397986+0000","last_became_active":"2023-11-08T19:26:30.294206+0000","last_became_peered":"2023-11-08T19:26:30.294206+0000","last_unstale":"2023-11-08T19:26:31.397986+0000","last_undegraded":"2023-11-08T19:26:31.397986+0000","last_fullsized":"2023-11-08T19:26:31.397986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:58.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270455+0000","last_change":"2023-11-08T19:26:29.291331+0000","last_active":"2023-11-08T19:26:31.270455+0000","last_peered":"2023-11-08T19:26:31.270455+0000","last_clean":"2023-11-08T19:26:31.270455+0000","last_became_active":"2023-11-08T19:26:29.291064+0000","last_became_peered":"2023-11-08T19:26:29.291064+0000","last_unstale":"2023-11-08T19:26:31.270455+0000","last_undegraded":"2023-11-08T19:26:31.270455+0000","last_fullsized":"2023-11-08T19:26:31.270455+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:14.298716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397676+0000","last_change":"2023-11-08T19:26:30.303309+0000","last_active":"2023-11-08T19:26:31.397676+0000","last_peered":"2023-11-08T19:26:31.397676+0000","last_clean":"2023-11-08T19:26:31.397676+0000","last_became_active":"2023-11-08T19:26:30.302985+0000","last_became_peered":"2023-11-08T19:26:30.302985+0000","last_unstale":"2023-11-08T19:26:31.397676+0000","last_undegraded":"2023-11-08T19:26:31.397676+0000","last_fullsized":"2023-11-08T19:26:31.397676+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396809+0000","last_change":"2023-11-08T19:26:29.284298+0000","last_active":"2023-11-08T19:26:31.396809+0000","last_peered":"2023-11-08T19:26:31.396809+0000","last_clean":"2023-11-08T19:26:31.396809+0000","last_became_active":"2023-11-08T19:26:29.283401+0000","last_became_peered":"2023-11-08T19:26:29.283401+0000","last_unstale":"2023-11-08T19:26:31.396809+0000","last_undegraded":"2023-11-08T19:26:31.396809+0000","last_fullsized":"2023-11-08T19:26:31.396809+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:47:19.011806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398666+0000","last_change":"2023-11-08T19:26:30.289864+0000","last_active":"2023-11-08T19:26:31.398666+0000","last_peered":"2023-11-08T19:26:31.398666+0000","last_clean":"2023-11-08T19:26:31.398666+0000","last_became_active":"2023-11-08T19:26:30.288162+0000","last_became_peered":"2023-11-08T19:26:30.288162+0000","last_unstale":"2023-11-08T19:26:31.398666+0000","last_undegraded":"2023-11-08T19:26:31.398666+0000","last_fullsized":"2023-11-08T19:26:31.398666+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:02.915393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397938+0000","last_change":"2023-11-08T19:26:29.290793+0000","last_active":"2023-11-08T19:26:31.397938+0000","last_peered":"2023-11-08T19:26:31.397938+0000","last_clean":"2023-11-08T19:26:31.397938+0000","last_became_active":"2023-11-08T19:26:29.290546+0000","last_became_peered":"2023-11-08T19:26:29.290546+0000","last_unstale":"2023-11-08T19:26:31.397938+0000","last_undegraded":"2023-11-08T19:26:31.397938+0000","last_fullsized":"2023-11-08T19:26:31.397938+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:37.487567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265934+0000","last_change":"2023-11-08T19:26:30.296181+0000","last_active":"2023-11-08T19:26:31.265934+0000","last_peered":"2023-11-08T19:26:31.265934+0000","last_clean":"2023-11-08T19:26:31.265934+0000","last_became_active":"2023-11-08T19:26:30.295855+0000","last_became_peered":"2023-11-08T19:26:30.295855+0000","last_unstale":"2023-11-08T19:26:31.265934+0000","last_undegraded":"2023-11-08T19:26:31.265934+0000","last_fullsized":"2023-11-08T19:26:31.265934+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:46.858474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397550+0000","last_change":"2023-11-08T19:26:29.288519+0000","last_active":"2023-11-08T19:26:31.397550+0000","last_peered":"2023-11-08T19:26:31.397550+0000","last_clean":"2023-11-08T19:26:31.397550+0000","last_became_active":"2023-11-08T19:26:29.288184+0000","last_became_peered":"2023-11-08T19:26:29.288184+0000","last_unstale":"2023-11-08T19:26:31.397550+0000","last_undegraded":"2023-11-08T19:26:31.397550+0000","last_fullsized":"2023-11-08T19:26:31.397550+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:56.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395253+0000","last_change":"2023-11-08T19:26:30.291538+0000","last_active":"2023-11-08T19:26:31.395253+0000","last_peered":"2023-11-08T19:26:31.395253+0000","last_clean":"2023-11-08T19:26:31.395253+0000","last_became_active":"2023-11-08T19:26:30.291229+0000","last_became_peered":"2023-11-08T19:26:30.291229+0000","last_unstale":"2023-11-08T19:26:31.395253+0000","last_undegraded":"2023-11-08T19:26:31.395253+0000","last_fullsized":"2023-11-08T19:26:31.395253+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:52.601811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269023+0000","last_change":"2023-11-08T19:26:29.285627+0000","last_active":"2023-11-08T19:26:31.269023+0000","last_peered":"2023-11-08T19:26:31.269023+0000","last_clean":"2023-11-08T19:26:31.269023+0000","last_became_active":"2023-11-08T19:26:29.284572+0000","last_became_peered":"2023-11-08T19:26:29.284572+0000","last_unstale":"2023-11-08T19:26:31.269023+0000","last_undegraded":"2023-11-08T19:26:31.269023+0000","last_fullsized":"2023-11-08T19:26:31.269023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:51.079022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269123+0000","last_change":"2023-11-08T19:26:30.294358+0000","last_active":"2023-11-08T19:26:31.269123+0000","last_peered":"2023-11-08T19:26:31.269123+0000","last_clean":"2023-11-08T19:26:31.269123+0000","last_became_active":"2023-11-08T19:26:30.294086+0000","last_became_peered":"2023-11-08T19:26:30.294086+0000","last_unstale":"2023-11-08T19:26:31.269123+0000","last_undegraded":"2023-11-08T19:26:31.269123+0000","last_fullsized":"2023-11-08T19:26:31.269123+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:09.911521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398512+0000","last_change":"2023-11-08T19:26:29.295383+0000","last_active":"2023-11-08T19:26:31.398512+0000","last_peered":"2023-11-08T19:26:31.398512+0000","last_clean":"2023-11-08T19:26:31.398512+0000","last_became_active":"2023-11-08T19:26:29.295105+0000","last_became_peered":"2023-11-08T19:26:29.295105+0000","last_unstale":"2023-11-08T19:26:31.398512+0000","last_undegraded":"2023-11-08T19:26:31.398512+0000","last_fullsized":"2023-11-08T19:26:31.398512+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:20:08.068311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397476+0000","last_change":"2023-11-08T19:26:30.297007+0000","last_active":"2023-11-08T19:26:31.397476+0000","last_peered":"2023-11-08T19:26:31.397476+0000","last_clean":"2023-11-08T19:26:31.397476+0000","last_became_active":"2023-11-08T19:26:30.296566+0000","last_became_peered":"2023-11-08T19:26:30.296566+0000","last_unstale":"2023-11-08T19:26:31.397476+0000","last_undegraded":"2023-11-08T19:26:31.397476+0000","last_fullsized":"2023-11-08T19:26:31.397476+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:34.753858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397324+0000","last_change":"2023-11-08T19:26:29.279306+0000","last_active":"2023-11-08T19:26:31.397324+0000","last_peered":"2023-11-08T19:26:31.397324+0000","last_clean":"2023-11-08T19:26:31.397324+0000","last_became_active":"2023-11-08T19:26:29.279041+0000","last_became_peered":"2023-11-08T19:26:29.279041+0000","last_unstale":"2023-11-08T19:26:31.397324+0000","last_undegraded":"2023-11-08T19:26:31.397324+0000","last_fullsized":"2023-11-08T19:26:31.397324+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:53.409533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265990+0000","last_change":"2023-11-08T19:26:30.290971+0000","last_active":"2023-11-08T19:26:31.265990+0000","last_peered":"2023-11-08T19:26:31.265990+0000","last_clean":"2023-11-08T19:26:31.265990+0000","last_became_active":"2023-11-08T19:26:30.289214+0000","last_became_peered":"2023-11-08T19:26:30.289214+0000","last_unstale":"2023-11-08T19:26:31.265990+0000","last_undegraded":"2023-11-08T19:26:31.265990+0000","last_fullsized":"2023-11-08T19:26:31.265990+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:10.539596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396121+0000","last_change":"2023-11-08T19:26:29.292941+0000","last_active":"2023-11-08T19:26:31.396121+0000","last_peered":"2023-11-08T19:26:31.396121+0000","last_clean":"2023-11-08T19:26:31.396121+0000","last_became_active":"2023-11-08T19:26:29.289972+0000","last_became_peered":"2023-11-08T19:26:29.289972+0000","last_unstale":"2023-11-08T19:26:31.396121+0000","last_undegraded":"2023-11-08T19:26:31.396121+0000","last_fullsized":"2023-11-08T19:26:31.396121+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:53.409533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399416+0000","last_change":"2023-11-08T19:26:30.290971+0000","last_active":"2023-11-08T19:26:31.399416+0000","last_peered":"2023-11-08T19:26:31.399416+0000","last_clean":"2023-11-08T19:26:31.399416+0000","last_became_active":"2023-11-08T19:26:30.290500+0000","last_became_peered":"2023-11-08T19:26:30.290500+0000","last_unstale":"2023-11-08T19:26:31.399416+0000","last_undegraded":"2023-11-08T19:26:31.399416+0000","last_fullsized":"2023-11-08T19:26:31.399416+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:23:38.112156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266255+0000","last_change":"2023-11-08T19:26:29.280352+0000","last_active":"2023-11-08T19:26:31.266255+0000","last_peered":"2023-11-08T19:26:31.266255+0000","last_clean":"2023-11-08T19:26:31.266255+0000","last_became_active":"2023-11-08T19:26:29.280082+0000","last_became_peered":"2023-11-08T19:26:29.280082+0000","last_unstale":"2023-11-08T19:26:31.266255+0000","last_undegraded":"2023-11-08T19:26:31.266255+0000","last_fullsized":"2023-11-08T19:26:31.266255+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:10.776285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397226+0000","last_change":"2023-11-08T19:26:30.287963+0000","last_active":"2023-11-08T19:26:31.397226+0000","last_peered":"2023-11-08T19:26:31.397226+0000","last_clean":"2023-11-08T19:26:31.397226+0000","last_became_active":"2023-11-08T19:26:30.287540+0000","last_became_peered":"2023-11-08T19:26:30.287540+0000","last_unstale":"2023-11-08T19:26:31.397226+0000","last_undegraded":"2023-11-08T19:26:31.397226+0000","last_fullsized":"2023-11-08T19:26:31.397226+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:29.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270028+0000","last_change":"2023-11-08T19:26:29.275731+0000","last_active":"2023-11-08T19:26:31.270028+0000","last_peered":"2023-11-08T19:26:31.270028+0000","last_clean":"2023-11-08T19:26:31.270028+0000","last_became_active":"2023-11-08T19:26:29.275364+0000","last_became_peered":"2023-11-08T19:26:29.275364+0000","last_unstale":"2023-11-08T19:26:31.270028+0000","last_undegraded":"2023-11-08T19:26:31.270028+0000","last_fullsized":"2023-11-08T19:26:31.270028+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:10.487434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396962+0000","last_change":"2023-11-08T19:26:30.288811+0000","last_active":"2023-11-08T19:26:31.396962+0000","last_peered":"2023-11-08T19:26:31.396962+0000","last_clean":"2023-11-08T19:26:31.396962+0000","last_became_active":"2023-11-08T19:26:30.288245+0000","last_became_peered":"2023-11-08T19:26:30.288245+0000","last_unstale":"2023-11-08T19:26:31.396962+0000","last_undegraded":"2023-11-08T19:26:31.396962+0000","last_fullsized":"2023-11-08T19:26:31.396962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:29.119828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270281+0000","last_change":"2023-11-08T19:26:29.286720+0000","last_active":"2023-11-08T19:26:31.270281+0000","last_peered":"2023-11-08T19:26:31.270281+0000","last_clean":"2023-11-08T19:26:31.270281+0000","last_became_active":"2023-11-08T19:26:29.286426+0000","last_became_peered":"2023-11-08T19:26:29.286426+0000","last_unstale":"2023-11-08T19:26:31.270281+0000","last_undegraded":"2023-11-08T19:26:31.270281+0000","last_fullsized":"2023-11-08T19:26:31.270281+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:45.570382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398697+0000","last_change":"2023-11-08T19:26:30.300346+0000","last_active":"2023-11-08T19:26:31.398697+0000","last_peered":"2023-11-08T19:26:31.398697+0000","last_clean":"2023-11-08T19:26:31.398697+0000","last_became_active":"2023-11-08T19:26:30.299506+0000","last_became_peered":"2023-11-08T19:26:30.299506+0000","last_unstale":"2023-11-08T19:26:31.398697+0000","last_undegraded":"2023-11-08T19:26:31.398697+0000","last_fullsized":"2023-11-08T19:26:31.398697+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:58.692489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266334+0000","last_change":"2023-11-08T19:26:29.283737+0000","last_active":"2023-11-08T19:26:31.266334+0000","last_peered":"2023-11-08T19:26:31.266334+0000","last_clean":"2023-11-08T19:26:31.266334+0000","last_became_active":"2023-11-08T19:26:29.283432+0000","last_became_peered":"2023-11-08T19:26:29.283432+0000","last_unstale":"2023-11-08T19:26:31.266334+0000","last_undegraded":"2023-11-08T19:26:31.266334+0000","last_fullsized":"2023-11-08T19:26:31.266334+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:04.867654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395262+0000","last_change":"2023-11-08T19:26:30.292146+0000","last_active":"2023-11-08T19:26:31.395262+0000","last_peered":"2023-11-08T19:26:31.395262+0000","last_clean":"2023-11-08T19:26:31.395262+0000","last_became_active":"2023-11-08T19:26:30.291846+0000","last_became_peered":"2023-11-08T19:26:30.291846+0000","last_unstale":"2023-11-08T19:26:31.395262+0000","last_undegraded":"2023-11-08T19:26:31.395262+0000","last_fullsized":"2023-11-08T19:26:31.395262+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:09.389740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266392+0000","last_change":"2023-11-08T19:26:29.280380+0000","last_active":"2023-11-08T19:26:31.266392+0000","last_peered":"2023-11-08T19:26:31.266392+0000","last_clean":"2023-11-08T19:26:31.266392+0000","last_became_active":"2023-11-08T19:26:29.279951+0000","last_became_peered":"2023-11-08T19:26:29.279951+0000","last_unstale":"2023-11-08T19:26:31.266392+0000","last_undegraded":"2023-11-08T19:26:31.266392+0000","last_fullsized":"2023-11-08T19:26:31.266392+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:08.896644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265546+0000","last_change":"2023-11-08T19:26:30.293261+0000","last_active":"2023-11-08T19:26:31.265546+0000","last_peered":"2023-11-08T19:26:31.265546+0000","last_clean":"2023-11-08T19:26:31.265546+0000","last_became_active":"2023-11-08T19:26:30.291279+0000","last_became_peered":"2023-11-08T19:26:30.291279+0000","last_unstale":"2023-11-08T19:26:31.265546+0000","last_undegraded":"2023-11-08T19:26:31.265546+0000","last_fullsized":"2023-11-08T19:26:31.265546+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:53.027281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396429+0000","last_change":"2023-11-08T19:26:29.286345+0000","last_active":"2023-11-08T19:26:31.396429+0000","last_peered":"2023-11-08T19:26:31.396429+0000","last_clean":"2023-11-08T19:26:31.396429+0000","last_became_active":"2023-11-08T19:26:29.286094+0000","last_became_peered":"2023-11-08T19:26:29.286094+0000","last_unstale":"2023-11-08T19:26:31.396429+0000","last_undegraded":"2023-11-08T19:26:31.396429+0000","last_fullsized":"2023-11-08T19:26:31.396429+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:47:19.011806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396516+0000","last_change":"2023-11-08T19:26:30.294135+0000","last_active":"2023-11-08T19:26:31.396516+0000","last_peered":"2023-11-08T19:26:31.396516+0000","last_clean":"2023-11-08T19:26:31.396516+0000","last_became_active":"2023-11-08T19:26:30.293861+0000","last_became_peered":"2023-11-08T19:26:30.293861+0000","last_unstale":"2023-11-08T19:26:31.396516+0000","last_undegraded":"2023-11-08T19:26:31.396516+0000","last_fullsized":"2023-11-08T19:26:31.396516+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:52.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.267125+0000","last_change":"2023-11-08T19:26:29.293542+0000","last_active":"2023-11-08T19:26:31.267125+0000","last_peered":"2023-11-08T19:26:31.267125+0000","last_clean":"2023-11-08T19:26:31.267125+0000","last_became_active":"2023-11-08T19:26:29.293196+0000","last_became_peered":"2023-11-08T19:26:29.293196+0000","last_unstale":"2023-11-08T19:26:31.267125+0000","last_undegraded":"2023-11-08T19:26:31.267125+0000","last_fullsized":"2023-11-08T19:26:31.267125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:11.337250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266157+0000","last_change":"2023-11-08T19:26:30.292582+0000","last_active":"2023-11-08T19:26:31.266157+0000","last_peered":"2023-11-08T19:26:31.266157+0000","last_clean":"2023-11-08T19:26:31.266157+0000","last_became_active":"2023-11-08T19:26:30.291190+0000","last_became_peered":"2023-11-08T19:26:30.291190+0000","last_unstale":"2023-11-08T19:26:31.266157+0000","last_undegraded":"2023-11-08T19:26:31.266157+0000","last_fullsized":"2023-11-08T19:26:31.266157+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:51.089873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396356+0000","last_change":"2023-11-08T19:26:29.295530+0000","last_active":"2023-11-08T19:26:31.396356+0000","last_peered":"2023-11-08T19:26:31.396356+0000","last_clean":"2023-11-08T19:26:31.396356+0000","last_became_active":"2023-11-08T19:26:29.295031+0000","last_became_peered":"2023-11-08T19:26:29.295031+0000","last_unstale":"2023-11-08T19:26:31.396356+0000","last_undegraded":"2023-11-08T19:26:31.396356+0000","last_fullsized":"2023-11-08T19:26:31.396356+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:02:45.733805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266091+0000","last_change":"2023-11-08T19:26:30.297083+0000","last_active":"2023-11-08T19:26:31.266091+0000","last_peered":"2023-11-08T19:26:31.266091+0000","last_clean":"2023-11-08T19:26:31.266091+0000","last_became_active":"2023-11-08T19:26:30.294852+0000","last_became_peered":"2023-11-08T19:26:30.294852+0000","last_unstale":"2023-11-08T19:26:31.266091+0000","last_undegraded":"2023-11-08T19:26:31.266091+0000","last_fullsized":"2023-11-08T19:26:31.266091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:35.950771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269478+0000","last_change":"2023-11-08T19:26:29.291169+0000","last_active":"2023-11-08T19:26:31.269478+0000","last_peered":"2023-11-08T19:26:31.269478+0000","last_clean":"2023-11-08T19:26:31.269478+0000","last_became_active":"2023-11-08T19:26:29.290831+0000","last_became_peered":"2023-11-08T19:26:29.290831+0000","last_unstale":"2023-11-08T19:26:31.269478+0000","last_undegraded":"2023-11-08T19:26:31.269478+0000","last_fullsized":"2023-11-08T19:26:31.269478+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:28.104951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.270278+0000","last_change":"2023-11-08T19:26:30.301149+0000","last_active":"2023-11-08T19:26:31.270278+0000","last_peered":"2023-11-08T19:26:31.270278+0000","last_clean":"2023-11-08T19:26:31.270278+0000","last_became_active":"2023-11-08T19:26:30.300881+0000","last_became_peered":"2023-11-08T19:26:30.300881+0000","last_unstale":"2023-11-08T19:26:31.270278+0000","last_undegraded":"2023-11-08T19:26:31.270278+0000","last_fullsized":"2023-11-08T19:26:31.270278+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:49.873565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398786+0000","last_change":"2023-11-08T19:26:29.287399+0000","last_active":"2023-11-08T19:26:31.398786+0000","last_peered":"2023-11-08T19:26:31.398786+0000","last_clean":"2023-11-08T19:26:31.398786+0000","last_became_active":"2023-11-08T19:26:29.287124+0000","last_became_peered":"2023-11-08T19:26:29.287124+0000","last_unstale":"2023-11-08T19:26:31.398786+0000","last_undegraded":"2023-11-08T19:26:31.398786+0000","last_fullsized":"2023-11-08T19:26:31.398786+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:08.374863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398422+0000","last_change":"2023-11-08T19:26:30.294708+0000","last_active":"2023-11-08T19:26:31.398422+0000","last_peered":"2023-11-08T19:26:31.398422+0000","last_clean":"2023-11-08T19:26:31.398422+0000","last_became_active":"2023-11-08T19:26:30.294350+0000","last_became_peered":"2023-11-08T19:26:30.294350+0000","last_unstale":"2023-11-08T19:26:31.398422+0000","last_undegraded":"2023-11-08T19:26:31.398422+0000","last_fullsized":"2023-11-08T19:26:31.398422+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:04.627278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":13,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399236+0000","last_change":"2023-11-08T19:26:29.280768+0000","last_active":"2023-11-08T19:26:31.399236+0000","last_peered":"2023-11-08T19:26:31.399236+0000","last_clean":"2023-11-08T19:26:31.399236+0000","last_became_active":"2023-11-08T19:26:29.280498+0000","last_became_peered":"2023-11-08T19:26:29.280498+0000","last_unstale":"2023-11-08T19:26:31.399236+0000","last_undegraded":"2023-11-08T19:26:31.399236+0000","last_fullsized":"2023-11-08T19:26:31.399236+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:24:37.687613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399634+0000","last_change":"2023-11-08T19:26:30.298373+0000","last_active":"2023-11-08T19:26:31.399634+0000","last_peered":"2023-11-08T19:26:31.399634+0000","last_clean":"2023-11-08T19:26:31.399634+0000","last_became_active":"2023-11-08T19:26:30.296777+0000","last_became_peered":"2023-11-08T19:26:30.296777+0000","last_unstale":"2023-11-08T19:26:31.399634+0000","last_undegraded":"2023-11-08T19:26:31.399634+0000","last_fullsized":"2023-11-08T19:26:31.399634+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:36.426409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396709+0000","last_change":"2023-11-08T19:26:29.291077+0000","last_active":"2023-11-08T19:26:31.396709+0000","last_peered":"2023-11-08T19:26:31.396709+0000","last_clean":"2023-11-08T19:26:31.396709+0000","last_became_active":"2023-11-08T19:26:29.290498+0000","last_became_peered":"2023-11-08T19:26:29.290498+0000","last_unstale":"2023-11-08T19:26:31.396709+0000","last_undegraded":"2023-11-08T19:26:31.396709+0000","last_fullsized":"2023-11-08T19:26:31.396709+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:56.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396629+0000","last_change":"2023-11-08T19:26:30.298812+0000","last_active":"2023-11-08T19:26:31.396629+0000","last_peered":"2023-11-08T19:26:31.396629+0000","last_clean":"2023-11-08T19:26:31.396629+0000","last_became_active":"2023-11-08T19:26:30.298552+0000","last_became_peered":"2023-11-08T19:26:30.298552+0000","last_unstale":"2023-11-08T19:26:31.396629+0000","last_undegraded":"2023-11-08T19:26:31.396629+0000","last_fullsized":"2023-11-08T19:26:31.396629+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:46.585259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398957+0000","last_change":"2023-11-08T19:26:29.279568+0000","last_active":"2023-11-08T19:26:31.398957+0000","last_peered":"2023-11-08T19:26:31.398957+0000","last_clean":"2023-11-08T19:26:31.398957+0000","last_became_active":"2023-11-08T19:26:29.279251+0000","last_became_peered":"2023-11-08T19:26:29.279251+0000","last_unstale":"2023-11-08T19:26:31.398957+0000","last_undegraded":"2023-11-08T19:26:31.398957+0000","last_fullsized":"2023-11-08T19:26:31.398957+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:38:11.038990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395384+0000","last_change":"2023-11-08T19:26:30.287043+0000","last_active":"2023-11-08T19:26:31.395384+0000","last_peered":"2023-11-08T19:26:31.395384+0000","last_clean":"2023-11-08T19:26:31.395384+0000","last_became_active":"2023-11-08T19:26:30.286688+0000","last_became_peered":"2023-11-08T19:26:30.286688+0000","last_unstale":"2023-11-08T19:26:31.395384+0000","last_undegraded":"2023-11-08T19:26:31.395384+0000","last_fullsized":"2023-11-08T19:26:31.395384+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.513281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398690+0000","last_change":"2023-11-08T19:26:29.289719+0000","last_active":"2023-11-08T19:26:31.398690+0000","last_peered":"2023-11-08T19:26:31.398690+0000","last_clean":"2023-11-08T19:26:31.398690+0000","last_became_active":"2023-11-08T19:26:29.289421+0000","last_became_peered":"2023-11-08T19:26:29.289421+0000","last_unstale":"2023-11-08T19:26:31.398690+0000","last_undegraded":"2023-11-08T19:26:31.398690+0000","last_fullsized":"2023-11-08T19:26:31.398690+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:41:12.299524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269924+0000","last_change":"2023-11-08T19:26:30.289369+0000","last_active":"2023-11-08T19:26:31.269924+0000","last_peered":"2023-11-08T19:26:31.269924+0000","last_clean":"2023-11-08T19:26:31.269924+0000","last_became_active":"2023-11-08T19:26:30.289068+0000","last_became_peered":"2023-11-08T19:26:30.289068+0000","last_unstale":"2023-11-08T19:26:31.269924+0000","last_undegraded":"2023-11-08T19:26:31.269924+0000","last_fullsized":"2023-11-08T19:26:31.269924+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:53.027281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397732+0000","last_change":"2023-11-08T19:26:29.289157+0000","last_active":"2023-11-08T19:26:31.397732+0000","last_peered":"2023-11-08T19:26:31.397732+0000","last_clean":"2023-11-08T19:26:31.397732+0000","last_became_active":"2023-11-08T19:26:29.288842+0000","last_became_peered":"2023-11-08T19:26:29.288842+0000","last_unstale":"2023-11-08T19:26:31.397732+0000","last_undegraded":"2023-11-08T19:26:31.397732+0000","last_fullsized":"2023-11-08T19:26:31.397732+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:51.586934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.266326+0000","last_change":"2023-11-08T19:26:30.290576+0000","last_active":"2023-11-08T19:26:31.266326+0000","last_peered":"2023-11-08T19:26:31.266326+0000","last_clean":"2023-11-08T19:26:31.266326+0000","last_became_active":"2023-11-08T19:26:30.288089+0000","last_became_peered":"2023-11-08T19:26:30.288089+0000","last_unstale":"2023-11-08T19:26:31.266326+0000","last_undegraded":"2023-11-08T19:26:31.266326+0000","last_fullsized":"2023-11-08T19:26:31.266326+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:48.437724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.398390+0000","last_change":"2023-11-08T19:26:29.274366+0000","last_active":"2023-11-08T19:26:31.398390+0000","last_peered":"2023-11-08T19:26:31.398390+0000","last_clean":"2023-11-08T19:26:31.398390+0000","last_became_active":"2023-11-08T19:26:29.274047+0000","last_became_peered":"2023-11-08T19:26:29.274047+0000","last_unstale":"2023-11-08T19:26:31.398390+0000","last_undegraded":"2023-11-08T19:26:31.398390+0000","last_fullsized":"2023-11-08T19:26:31.398390+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:00:37.210520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396805+0000","last_change":"2023-11-08T19:26:30.296734+0000","last_active":"2023-11-08T19:26:31.396805+0000","last_peered":"2023-11-08T19:26:31.396805+0000","last_clean":"2023-11-08T19:26:31.396805+0000","last_became_active":"2023-11-08T19:26:30.295198+0000","last_became_peered":"2023-11-08T19:26:30.295198+0000","last_unstale":"2023-11-08T19:26:31.396805+0000","last_undegraded":"2023-11-08T19:26:31.396805+0000","last_fullsized":"2023-11-08T19:26:31.396805+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.513281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269963+0000","last_change":"2023-11-08T19:26:29.288157+0000","last_active":"2023-11-08T19:26:31.269963+0000","last_peered":"2023-11-08T19:26:31.269963+0000","last_clean":"2023-11-08T19:26:31.269963+0000","last_became_active":"2023-11-08T19:26:29.286582+0000","last_became_peered":"2023-11-08T19:26:29.286582+0000","last_unstale":"2023-11-08T19:26:31.269963+0000","last_undegraded":"2023-11-08T19:26:31.269963+0000","last_fullsized":"2023-11-08T19:26:31.269963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:33.738981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397573+0000","last_change":"2023-11-08T19:26:30.287582+0000","last_active":"2023-11-08T19:26:31.397573+0000","last_peered":"2023-11-08T19:26:31.397573+0000","last_clean":"2023-11-08T19:26:31.397573+0000","last_became_active":"2023-11-08T19:26:30.287248+0000","last_became_peered":"2023-11-08T19:26:30.287248+0000","last_unstale":"2023-11-08T19:26:31.397573+0000","last_undegraded":"2023-11-08T19:26:31.397573+0000","last_fullsized":"2023-11-08T19:26:31.397573+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:04.627278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.399483+0000","last_change":"2023-11-08T19:26:29.282704+0000","last_active":"2023-11-08T19:26:31.399483+0000","last_peered":"2023-11-08T19:26:31.399483+0000","last_clean":"2023-11-08T19:26:31.399483+0000","last_became_active":"2023-11-08T19:26:29.282409+0000","last_became_peered":"2023-11-08T19:26:29.282409+0000","last_unstale":"2023-11-08T19:26:31.399483+0000","last_undegraded":"2023-11-08T19:26:31.399483+0000","last_fullsized":"2023-11-08T19:26:31.399483+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:05.247438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265604+0000","last_change":"2023-11-08T19:26:30.288274+0000","last_active":"2023-11-08T19:26:31.265604+0000","last_peered":"2023-11-08T19:26:31.265604+0000","last_clean":"2023-11-08T19:26:31.265604+0000","last_became_active":"2023-11-08T19:26:30.287595+0000","last_became_peered":"2023-11-08T19:26:30.287595+0000","last_unstale":"2023-11-08T19:26:31.265604+0000","last_undegraded":"2023-11-08T19:26:31.265604+0000","last_fullsized":"2023-11-08T19:26:31.265604+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:49.873565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396045+0000","last_change":"2023-11-08T19:26:29.275592+0000","last_active":"2023-11-08T19:26:31.396045+0000","last_peered":"2023-11-08T19:26:31.396045+0000","last_clean":"2023-11-08T19:26:31.396045+0000","last_became_active":"2023-11-08T19:26:29.275331+0000","last_became_peered":"2023-11-08T19:26:29.275331+0000","last_unstale":"2023-11-08T19:26:31.396045+0000","last_undegraded":"2023-11-08T19:26:31.396045+0000","last_fullsized":"2023-11-08T19:26:31.396045+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:23:56.108384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396464+0000","last_change":"2023-11-08T19:26:30.281266+0000","last_active":"2023-11-08T19:26:31.396464+0000","last_peered":"2023-11-08T19:26:31.396464+0000","last_clean":"2023-11-08T19:26:31.396464+0000","last_became_active":"2023-11-08T19:26:30.280526+0000","last_became_peered":"2023-11-08T19:26:30.280526+0000","last_unstale":"2023-11-08T19:26:31.396464+0000","last_undegraded":"2023-11-08T19:26:31.396464+0000","last_fullsized":"2023-11-08T19:26:31.396464+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.395772+0000","last_change":"2023-11-08T19:26:29.277771+0000","last_active":"2023-11-08T19:26:31.395772+0000","last_peered":"2023-11-08T19:26:31.395772+0000","last_clean":"2023-11-08T19:26:31.395772+0000","last_became_active":"2023-11-08T19:26:29.277549+0000","last_became_peered":"2023-11-08T19:26:29.277549+0000","last_unstale":"2023-11-08T19:26:31.395772+0000","last_undegraded":"2023-11-08T19:26:31.395772+0000","last_fullsized":"2023-11-08T19:26:31.395772+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:37.487567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397347+0000","last_change":"2023-11-08T19:26:30.293828+0000","last_active":"2023-11-08T19:26:31.397347+0000","last_peered":"2023-11-08T19:26:31.397347+0000","last_clean":"2023-11-08T19:26:31.397347+0000","last_became_active":"2023-11-08T19:26:30.293566+0000","last_became_peered":"2023-11-08T19:26:30.293566+0000","last_unstale":"2023-11-08T19:26:31.397347+0000","last_undegraded":"2023-11-08T19:26:31.397347+0000","last_fullsized":"2023-11-08T19:26:31.397347+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:16:52.093899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.268030+0000","last_change":"2023-11-08T19:26:29.276400+0000","last_active":"2023-11-08T19:26:31.268030+0000","last_peered":"2023-11-08T19:26:31.268030+0000","last_clean":"2023-11-08T19:26:31.268030+0000","last_became_active":"2023-11-08T19:26:29.275593+0000","last_became_peered":"2023-11-08T19:26:29.275593+0000","last_unstale":"2023-11-08T19:26:31.268030+0000","last_undegraded":"2023-11-08T19:26:31.268030+0000","last_fullsized":"2023-11-08T19:26:31.268030+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:48:18.797947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397149+0000","last_change":"2023-11-08T19:26:30.296837+0000","last_active":"2023-11-08T19:26:31.397149+0000","last_peered":"2023-11-08T19:26:31.397149+0000","last_clean":"2023-11-08T19:26:31.397149+0000","last_became_active":"2023-11-08T19:26:30.295170+0000","last_became_peered":"2023-11-08T19:26:30.295170+0000","last_unstale":"2023-11-08T19:26:31.397149+0000","last_undegraded":"2023-11-08T19:26:31.397149+0000","last_fullsized":"2023-11-08T19:26:31.397149+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.396358+0000","last_change":"2023-11-08T19:26:29.288055+0000","last_active":"2023-11-08T19:26:31.396358+0000","last_peered":"2023-11-08T19:26:31.396358+0000","last_clean":"2023-11-08T19:26:31.396358+0000","last_became_active":"2023-11-08T19:26:29.287741+0000","last_became_peered":"2023-11-08T19:26:29.287741+0000","last_unstale":"2023-11-08T19:26:31.396358+0000","last_undegraded":"2023-11-08T19:26:31.396358+0000","last_fullsized":"2023-11-08T19:26:31.396358+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:11:31.055751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397163+0000","last_change":"2023-11-08T19:26:30.297062+0000","last_active":"2023-11-08T19:26:31.397163+0000","last_peered":"2023-11-08T19:26:31.397163+0000","last_clean":"2023-11-08T19:26:31.397163+0000","last_became_active":"2023-11-08T19:26:30.296426+0000","last_became_peered":"2023-11-08T19:26:30.296426+0000","last_unstale":"2023-11-08T19:26:31.397163+0000","last_undegraded":"2023-11-08T19:26:31.397163+0000","last_fullsized":"2023-11-08T19:26:31.397163+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:11.502311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.265791+0000","last_change":"2023-11-08T19:26:29.282479+0000","last_active":"2023-11-08T19:26:31.265791+0000","last_peered":"2023-11-08T19:26:31.265791+0000","last_clean":"2023-11-08T19:26:31.265791+0000","last_became_active":"2023-11-08T19:26:29.280695+0000","last_became_peered":"2023-11-08T19:26:29.280695+0000","last_unstale":"2023-11-08T19:26:31.265791+0000","last_undegraded":"2023-11-08T19:26:31.265791+0000","last_fullsized":"2023-11-08T19:26:31.265791+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:03.612401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.269004+0000","last_change":"2023-11-08T19:26:30.295473+0000","last_active":"2023-11-08T19:26:31.269004+0000","last_peered":"2023-11-08T19:26:31.269004+0000","last_clean":"2023-11-08T19:26:31.269004+0000","last_became_active":"2023-11-08T19:26:30.294289+0000","last_became_peered":"2023-11-08T19:26:30.294289+0000","last_unstale":"2023-11-08T19:26:31.269004+0000","last_undegraded":"2023-11-08T19:26:31.269004+0000","last_fullsized":"2023-11-08T19:26:31.269004+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:29.236367+0000","last_clean_scrub_stamp":"2023-11-08T19:26:29.236367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:11.791162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-08T19:26:31.397971+0000","last_change":"2023-11-08T19:26:29.290756+0000","last_active":"2023-11-08T19:26:31.397971+0000","last_peered":"2023-11-08T19:26:31.397971+0000","last_clean":"2023-11-08T19:26:31.397971+0000","last_became_active":"2023-11-08T19:26:29.290392+0000","last_became_peered":"2023-11-08T19:26:29.290392+0000","last_unstale":"2023-11-08T19:26:31.397971+0000","last_undegraded":"2023-11-08T19:26:31.397971+0000","last_fullsized":"2023-11-08T19:26:31.397971+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:28.221490+0000","last_clean_scrub_stamp":"2023-11-08T19:26:28.221490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:40.708443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":181,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-11-08T19:26:30.271560+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:26:30.271560+0000","last_peered":"2023-11-08T19:26:30.271560+0000","last_clean":"2023-11-08T19:26:30.271560+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:26:30.271560+0000","last_undegraded":"2023-11-08T19:26:30.271560+0000","last_fullsized":"2023-11-08T19:26:30.271560+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.113,"15min":1.113},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.7509999999999999,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":4.4589999999999996,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":1.1679999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":4.5330000000000004,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.96699999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":1.5429999999999999},"min":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":4.6980000000000004,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.3260000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"max":{"1min":2.2240000000000002,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.204},{"interface":"front","average":{"1min":1.6020000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"min":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":4.6150000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.2949999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.532,"5min":1.532,"15min":1.532},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":2.2869999999999999,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.82799999999999996}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4750000000000001,"5min":1.4750000000000001,"15min":1.4750000000000001},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.399},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":2.1680000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3700000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"min":{"1min":1.145,"5min":1.145,"15min":1.145},"max":{"1min":2.427,"5min":2.427,"15min":2.427},"last":1.415},{"interface":"front","average":{"1min":1.663,"5min":1.663,"15min":1.663},"min":{"1min":1.1890000000000001,"5min":1.1890000000000001,"15min":1.1890000000000001},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":1.123}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":454656,"data_stored":196293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.1559999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.747},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":4.6399999999999997,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.93300000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":4.609,"5min":4.609,"15min":4.609},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.861,"5min":1.861,"15min":1.861},"last":1.4339999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":4.7190000000000003,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.383},{"interface":"front","average":{"1min":1.181,"5min":1.181,"15min":1.181},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.0860000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.496,"5min":1.496,"15min":1.496},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.47}]},{"osd":5,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.9079999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.23},{"interface":"front","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.732}]},{"osd":7,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.526},{"interface":"front","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.0640000000000001,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.337}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":189203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.76,"5min":1.76,"15min":1.76},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.236,"5min":1.236,"15min":1.236},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.339}]},{"osd":1,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":7.6779999999999999,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.2350000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.167,"15min":1.167},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.0640000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":7.6479999999999997,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.0129999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.2120000000000002,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.4650000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.347,"5min":2.347,"15min":2.347},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.573,"5min":1.573,"15min":1.573},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.90200000000000002}]},{"osd":6,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.546,"15min":1.546},"min":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"max":{"1min":2.8650000000000002,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"max":{"1min":2.6190000000000002,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.2849999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:25:51 2023","interfaces":[{"interface":"back","average":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"min":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"max":{"1min":3.2730000000000001,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.488},{"interface":"front","average":{"1min":1.595,"5min":1.595,"15min":1.595},"min":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":2.4670000000000001,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.526}]}]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":189695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.02,"5min":2.02,"15min":2.02},"last":0.62},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"max":{"1min":2.5819999999999999,"5min":2.5819999999999999,"15min":2.5819999999999999},"last":0.84799999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":0.91600000000000004},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.9940000000000002,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":4.0410000000000004,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":3.2069999999999999,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3,"15min":1.3},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.8319999999999999,"5min":3.8319999999999999,"15min":3.8319999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":1.401,"5min":1.401,"15min":1.401},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.5710000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.0189999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.55,"15min":1.55},"min":{"1min":0.872,"5min":0.872,"15min":0.872},"max":{"1min":4.2350000000000003,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"max":{"1min":4.1710000000000003,"5min":4.1710000000000003,"15min":4.1710000000000003},"last":1.077}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":4.335,"5min":4.335,"15min":4.335},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.645,"5min":1.645,"15min":1.645},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":3.9239999999999999,"5min":3.9239999999999999,"15min":3.9239999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.645,"5min":1.645,"15min":1.645},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.756,"5min":1.756,"15min":1.756},"min":{"1min":1.262,"5min":1.262,"15min":1.262},"max":{"1min":4.4950000000000001,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.1830000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27484,"kb_used_data":456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344356,"statfs":{"total":96636764160,"available":96608620544,"internally_reserved":0,"allocated":466944,"data_stored":197357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":3.972,"5min":3.972,"15min":3.972},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":3.1520000000000001,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.3999999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":1.6839999999999999},{"interface":"front","average":{"1min":1.137,"5min":1.137,"15min":1.137},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":4.5750000000000002,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":1.1299999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.353,"15min":1.353},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":3.9079999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":2.73,"5min":2.73,"15min":2.73},"last":1.778}]},{"osd":4,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.486,"15min":1.486},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":4.2770000000000001,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.861},{"interface":"front","average":{"1min":1.482,"5min":1.482,"15min":1.482},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":3.052,"5min":3.052,"15min":3.052},"last":1.54}]},{"osd":5,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.53},"min":{"1min":0.873,"5min":0.873,"15min":0.873},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":2.149},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":4.2380000000000004,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":2.375}]},{"osd":6,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":4.2220000000000004,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.8220000000000001},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":3.1230000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":2.4860000000000002}]},{"osd":7,"last update":"Wed Nov 8 19:25:46 2023","interfaces":[{"interface":"back","average":{"1min":1.758,"5min":1.758,"15min":1.758},"min":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"max":{"1min":4.407,"5min":4.407,"15min":4.407},"last":2.528},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":4.3849999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.736}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":438272,"data_stored":188142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.88,"15min":0.88},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.4890000000000001,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.436}]},{"osd":1,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.68200000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.53,"5min":1.53,"15min":1.53},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.75600000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":3.2959999999999998,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.399,"5min":1.399,"15min":1.399},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.278,"5min":2.278,"15min":2.278},"last":1.032}]},{"osd":5,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.458,"15min":1.458},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":2.2000000000000002,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.536,"5min":2.536,"15min":2.536},"last":0.90800000000000003}]},{"osd":6,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.411,"15min":1.411},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.0910000000000002,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.95499999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:25:48 2023","interfaces":[{"interface":"back","average":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.6549999999999998,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.3439999999999999,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.2370000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574873,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":442368,"data_stored":189181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.755,"5min":0.755,"15min":0.755},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.65700000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.2709999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.71099999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.516,"5min":1.516,"15min":1.516},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.5369999999999999,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":0.81999999999999995}]},{"osd":4,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.246},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.0449999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.4569999999999999,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":2.4089999999999998,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.0229999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.6019999999999999,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.393,"5min":1.393,"15min":1.393},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.2190000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.0409999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.004},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.1799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":1032192,"data_stored":778474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":1.2889999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.997,"5min":0.997,"15min":0.997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.1180000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":0.95999999999999996}]},{"osd":4,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.9239999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.198},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":2.4039999999999999,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.397}]},{"osd":5,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":2.4369999999999998,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.45},{"interface":"front","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":2.4239999999999999,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.2190000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.5710000000000002,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.431,"15min":1.431},"min":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"max":{"1min":1.9890000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":0.92900000000000005}]},{"osd":7,"last update":"Wed Nov 8 19:25:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.256},{"interface":"front","average":{"1min":1.514,"5min":1.514,"15min":1.514},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":1.306}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:26:35.579 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:26:35.580 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:26:35.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.835+0000 7f2a6541f640 1 Processor -- start 2023-11-08T19:26:35.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.835+0000 7f2a6541f640 1 -- start start 2023-11-08T19:26:35.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.836+0000 7f2a6541f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a60144580 0x7f2a6015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.836+0000 7f2a6541f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a6015f1d0 0x7f2a600c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.836+0000 7f2a6541f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6015ea60 con 0x7f2a60144960 2023-11-08T19:26:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.836+0000 7f2a6541f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600c9150 con 0x7f2a60144580 2023-11-08T19:26:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.836+0000 7f2a6541f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a600c9360 con 0x7f2a6015f1d0 2023-11-08T19:26:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5e7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2a60144960 0x7f2a600cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41728/0 (socket says 172.21.15.119:41728) 2023-11-08T19:26:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5e7fc640 1 -- 172.21.15.119:0/1240782934 learned_addr learned my addr 172.21.15.119:0/1240782934 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:35.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5effd640 1 --2- 172.21.15.119:0/1240782934 >> v2:172.21.15.119:3300/0 conn(0x7f2a60144580 0x7f2a6015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5dffb640 1 -- 172.21.15.119:0/1240782934 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344821794 0 0) 0x7f2a6015ea60 con 0x7f2a60144960 2023-11-08T19:26:35.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5f7fe640 1 --2- 172.21.15.119:0/1240782934 >> v2:172.21.15.188:3300/0 conn(0x7f2a6015f1d0 0x7f2a600c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5dffb640 1 -- 172.21.15.119:0/1240782934 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a48003520 con 0x7f2a60144960 2023-11-08T19:26:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5effd640 1 -- 172.21.15.119:0/1240782934 >> v2:172.21.15.188:3300/0 conn(0x7f2a6015f1d0 msgr2=0x7f2a600c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5effd640 1 --2- 172.21.15.119:0/1240782934 >> v2:172.21.15.188:3300/0 conn(0x7f2a6015f1d0 0x7f2a600c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5effd640 1 -- 172.21.15.119:0/1240782934 >> v1:172.21.15.119:6789/0 conn(0x7f2a60144960 legacy=0x7f2a600cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5effd640 1 -- 172.21.15.119:0/1240782934 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a600c9640 con 0x7f2a60144580 2023-11-08T19:26:35.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.837+0000 7f2a5f7fe640 1 --2- 172.21.15.119:0/1240782934 >> v2:172.21.15.188:3300/0 conn(0x7f2a6015f1d0 0x7f2a600c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:35.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.838+0000 7f2a5effd640 1 --2- 172.21.15.119:0/1240782934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a6015bff0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2a6006d420 tx=0x7f2a5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed401b3a62b718f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a5dffb640 1 -- 172.21.15.119:0/1240782934 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5002e070 con 0x7f2a60144580 2023-11-08T19:26:35.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a5dffb640 1 -- 172.21.15.119:0/1240782934 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a500315a0 con 0x7f2a60144580 2023-11-08T19:26:35.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a5dffb640 1 -- 172.21.15.119:0/1240782934 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50031930 con 0x7f2a60144580 2023-11-08T19:26:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a6541f640 1 -- 172.21.15.119:0/1240782934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 msgr2=0x7f2a6015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1240782934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a6015bff0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2a6006d420 tx=0x7f2a5002b840 comp rx=0 tx=0).stop 2023-11-08T19:26:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a6541f640 1 -- 172.21.15.119:0/1240782934 shutdown_connections 2023-11-08T19:26:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1240782934 >> v2:172.21.15.188:3300/0 conn(0x7f2a6015f1d0 0x7f2a600c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1240782934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a6015bff0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.839+0000 7f2a6541f640 1 -- 172.21.15.119:0/1240782934 >> 172.21.15.119:0/1240782934 conn(0x7f2a600cb610 msgr2=0x7f2a600cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:35.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.840+0000 7f2a6541f640 1 -- 172.21.15.119:0/1240782934 shutdown_connections 2023-11-08T19:26:35.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.840+0000 7f2a6541f640 1 -- 172.21.15.119:0/1240782934 wait complete. 2023-11-08T19:26:35.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.841+0000 7f2a6541f640 1 Processor -- start 2023-11-08T19:26:35.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.841+0000 7f2a6541f640 1 -- start start 2023-11-08T19:26:35.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a6541f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a601f0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a6541f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a60144960 0x7f2a600c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a6541f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6015f1d0 0x7f2a6015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a6541f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c6e70 con 0x7f2a60144580 2023-11-08T19:26:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a6541f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c6260 con 0x7f2a6015f1d0 2023-11-08T19:26:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a6541f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a600c6540 con 0x7f2a60144960 2023-11-08T19:26:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.842+0000 7f2a5effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a601f0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a601f0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47758/0 (socket says 172.21.15.119:47758) 2023-11-08T19:26:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 -- 172.21.15.119:0/1710040069 learned_addr learned my addr 172.21.15.119:0/1710040069 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 -- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a60144960 msgr2=0x7f2a600c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a60144960 0x7f2a600c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 -- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6015f1d0 msgr2=0x7f2a6015a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5f7fe640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6015f1d0 0x7f2a6015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6015f1d0 0x7f2a6015a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 -- 172.21.15.119:0/1710040069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a50009400 con 0x7f2a60144580 2023-11-08T19:26:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a5effd640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a601f0850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009890 tx=0x7f2a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50032840 con 0x7f2a60144580 2023-11-08T19:26:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.843+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a50031f00 con 0x7f2a60144580 2023-11-08T19:26:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.844+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a50032be0 con 0x7f2a60144580 2023-11-08T19:26:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.844+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a600c6820 con 0x7f2a60144580 2023-11-08T19:26:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.844+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a600b4b40 con 0x7f2a60144580 2023-11-08T19:26:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.845+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a60074430 con 0x7f2a60144580 2023-11-08T19:26:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.846+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a50032d40 con 0x7f2a60144580 2023-11-08T19:26:35.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.847+0000 7f2a477fe640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a4c06aea0 0x7f2a4c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:35.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.847+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2a500ab600 con 0x7f2a60144580 2023-11-08T19:26:35.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.847+0000 7f2a5e7fc640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a4c06aea0 0x7f2a4c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:35.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.848+0000 7f2a5e7fc640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a4c06aea0 0x7f2a4c06d360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a6013f180 tx=0x7f2a54007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:35.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:35.851+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a50076600 con 0x7f2a60144580 2023-11-08T19:26:36.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.089+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f2a6015c9f0 con 0x7f2a60144580 2023-11-08T19:26:36.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.093+0000 7f2a477fe640 1 -- 172.21.15.119:0/1710040069 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2a50079c20 con 0x7f2a60144580 2023-11-08T19:26:36.095 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-11-08T19:26:36.096 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:26:36.096 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2023-11-08T19:26:36.096 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2023-11-08T19:26:36.096 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:26:36.097 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-11-08T19:26:36.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a4c06aea0 msgr2=0x7f2a4c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a4c06aea0 0x7f2a4c06d360 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a6013f180 tx=0x7f2a54007930 comp rx=0 tx=0).stop 2023-11-08T19:26:36.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 msgr2=0x7f2a601f0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a601f0850 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2a50009890 tx=0x7f2a500046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 shutdown_connections 2023-11-08T19:26:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a4c06aea0 0x7f2a4c06d360 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a6015f1d0 0x7f2a6015a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a60144960 0x7f2a600c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 --2- 172.21.15.119:0/1710040069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a60144580 0x7f2a601f0850 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 >> 172.21.15.119:0/1710040069 conn(0x7f2a600cb610 msgr2=0x7f2a6006a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 shutdown_connections 2023-11-08T19:26:36.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.096+0000 7f2a6541f640 1 -- 172.21.15.119:0/1710040069 wait complete. 2023-11-08T19:26:36.116 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:36.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.318+0000 7fc33df27640 1 Processor -- start 2023-11-08T19:26:36.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.318+0000 7fc33df27640 1 -- start start 2023-11-08T19:26:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.319+0000 7fc33df27640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc3380cac30 0x7fc33815f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.319+0000 7fc33df27640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3380adf20 0x7fc33815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.319+0000 7fc33df27640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33815f8c0 con 0x7fc3380ad9d0 2023-11-08T19:26:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.319+0000 7fc33df27640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc338071c70 con 0x7fc3380cac30 2023-11-08T19:26:36.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.319+0000 7fc33df27640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380ca2e0 con 0x7fc3380adf20 2023-11-08T19:26:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33cf25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc3380cac30 0x7fc33815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33d726640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc3380ad9d0 0x7fc3380c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41744/0 (socket says 172.21.15.119:41744) 2023-11-08T19:26:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33d726640 1 -- 172.21.15.119:0/296562882 learned_addr learned my addr 172.21.15.119:0/296562882 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc337fff640 1 --2- 172.21.15.119:0/296562882 >> v2:172.21.15.188:3300/0 conn(0x7fc3380adf20 0x7fc33815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc3377fe640 1 -- 172.21.15.119:0/296562882 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2212398191 0 0) 0x7fc33815f8c0 con 0x7fc3380ad9d0 2023-11-08T19:26:36.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33cf25640 1 -- 172.21.15.119:0/296562882 >> v2:172.21.15.188:3300/0 conn(0x7fc3380adf20 msgr2=0x7fc33815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:36.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33cf25640 1 --2- 172.21.15.119:0/296562882 >> v2:172.21.15.188:3300/0 conn(0x7fc3380adf20 0x7fc33815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33cf25640 1 -- 172.21.15.119:0/296562882 >> v1:172.21.15.119:6789/0 conn(0x7fc3380ad9d0 legacy=0x7fc3380c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc33cf25640 1 -- 172.21.15.119:0/296562882 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380ca5c0 con 0x7fc3380cac30 2023-11-08T19:26:36.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.320+0000 7fc337fff640 1 --2- 172.21.15.119:0/296562882 >> v2:172.21.15.188:3300/0 conn(0x7fc3380adf20 0x7fc33815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:36.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.322+0000 7fc33cf25640 1 --2- 172.21.15.119:0/296562882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380cac30 0x7fc33815f380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc3380ce940 tx=0x7fc32c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=177de75c37cb19f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.322+0000 7fc3377fe640 1 -- 172.21.15.119:0/296562882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c031830 con 0x7fc3380cac30 2023-11-08T19:26:36.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.322+0000 7fc3377fe640 1 -- 172.21.15.119:0/296562882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c031990 con 0x7fc3380cac30 2023-11-08T19:26:36.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.322+0000 7fc3377fe640 1 -- 172.21.15.119:0/296562882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c031d20 con 0x7fc3380cac30 2023-11-08T19:26:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.322+0000 7fc33df27640 1 -- 172.21.15.119:0/296562882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380cac30 msgr2=0x7fc33815f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.322+0000 7fc33df27640 1 --2- 172.21.15.119:0/296562882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380cac30 0x7fc33815f380 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc3380ce940 tx=0x7fc32c02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.323+0000 7fc33df27640 1 -- 172.21.15.119:0/296562882 shutdown_connections 2023-11-08T19:26:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.323+0000 7fc33df27640 1 --2- 172.21.15.119:0/296562882 >> v2:172.21.15.188:3300/0 conn(0x7fc3380adf20 0x7fc33815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.323+0000 7fc33df27640 1 --2- 172.21.15.119:0/296562882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380cac30 0x7fc33815f380 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.323+0000 7fc33df27640 1 -- 172.21.15.119:0/296562882 >> 172.21.15.119:0/296562882 conn(0x7fc3380b0560 msgr2=0x7fc3380b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.323+0000 7fc33df27640 1 -- 172.21.15.119:0/296562882 shutdown_connections 2023-11-08T19:26:36.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.323+0000 7fc33df27640 1 -- 172.21.15.119:0/296562882 wait complete. 2023-11-08T19:26:36.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.324+0000 7fc33df27640 1 Processor -- start 2023-11-08T19:26:36.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.324+0000 7fc33df27640 1 -- start start 2023-11-08T19:26:36.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33df27640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 0x7fc338155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33df27640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3380adf20 0x7fc3380af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33df27640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3380cac30 0x7fc3381efb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33df27640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381535b0 con 0x7fc3380ad9d0 2023-11-08T19:26:36.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33df27640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338157c70 con 0x7fc3380adf20 2023-11-08T19:26:36.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33df27640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc338156630 con 0x7fc3380cac30 2023-11-08T19:26:36.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33cf25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 0x7fc338155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.325+0000 7fc33cf25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 0x7fc338155260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47772/0 (socket says 172.21.15.119:47772) 2023-11-08T19:26:36.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 -- 172.21.15.119:0/1011605870 learned_addr learned my addr 172.21.15.119:0/1011605870 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:36.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc337fff640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3380adf20 0x7fc3380af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 -- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3380cac30 msgr2=0x7fc3381efb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33d726640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3380cac30 0x7fc3381efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3380cac30 0x7fc3381efb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 -- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3380adf20 msgr2=0x7fc3380af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3380adf20 0x7fc3380af7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 -- 172.21.15.119:0/1011605870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c009400 con 0x7fc3380ad9d0 2023-11-08T19:26:36.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc337fff640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3380adf20 0x7fc3380af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.326+0000 7fc33cf25640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 0x7fc338155260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc32c000c00 tx=0x7fc32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.327+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c004030 con 0x7fc3380ad9d0 2023-11-08T19:26:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.327+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c004190 con 0x7fc3380ad9d0 2023-11-08T19:26:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.327+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c03b860 con 0x7fc3380ad9d0 2023-11-08T19:26:36.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.327+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc338155ab0 con 0x7fc3380ad9d0 2023-11-08T19:26:36.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.327+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc338156070 con 0x7fc3380ad9d0 2023-11-08T19:26:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.328+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33804ece0 con 0x7fc3380ad9d0 2023-11-08T19:26:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.329+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc32c043070 con 0x7fc3380ad9d0 2023-11-08T19:26:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.330+0000 7fc3357fa640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc32406ae70 0x7fc32406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:36.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.330+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fc32c0ab500 con 0x7fc3380ad9d0 2023-11-08T19:26:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.330+0000 7fc337fff640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc32406ae70 0x7fc32406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.331+0000 7fc337fff640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc32406ae70 0x7fc32406d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc328006fd0 tx=0x7fc328008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:36.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.334+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc32c076590 con 0x7fc3380ad9d0 2023-11-08T19:26:36.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.559+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3380b1110 con 0x7fc3380ad9d0 2023-11-08T19:26:36.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.561+0000 7fc3357fa640 1 -- 172.21.15.119:0/1011605870 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fc32c035090 con 0x7fc3380ad9d0 2023-11-08T19:26:36.563 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:36.563 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:29.722778+0000","modified":"2023-11-08T19:26:33.132121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/4172930163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4172930163},{"type":"v1","addr":"172.21.15.188:6834","nonce":4172930163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:36.564 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-11-08T19:26:36.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc32406ae70 msgr2=0x7fc32406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc32406ae70 0x7fc32406d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fc328006fd0 tx=0x7fc328008040 comp rx=0 tx=0).stop 2023-11-08T19:26:36.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 msgr2=0x7fc338155260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:36.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 0x7fc338155260 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc32c000c00 tx=0x7fc32c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 shutdown_connections 2023-11-08T19:26:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc32406ae70 0x7fc32406d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3380cac30 0x7fc3381efb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3380adf20 0x7fc3380af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 --2- 172.21.15.119:0/1011605870 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3380ad9d0 0x7fc338155260 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:36.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 >> 172.21.15.119:0/1011605870 conn(0x7fc3380b0560 msgr2=0x7fc3380b2680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:36.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 shutdown_connections 2023-11-08T19:26:36.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:36.563+0000 7fc33df27640 1 -- 172.21.15.119:0/1011605870 wait complete. 2023-11-08T19:26:36.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:26:29.722778+0000', 'modified': '2023-11-08T19:26:33.132121+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5310}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5310': {'gid': 5310, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.188:6834/4172930163', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4172930163}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4172930163}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:26:36.986 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:26:36.987 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:26:37.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.211+0000 7f2b334a3640 1 Processor -- start 2023-11-08T19:26:37.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.211+0000 7f2b334a3640 1 -- start start 2023-11-08T19:26:37.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.212+0000 7f2b334a3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b2c0d1c50 0x7f2b2c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.212+0000 7f2b334a3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0b2fb0 con 0x7f2b2c0d0d40 2023-11-08T19:26:37.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.212+0000 7f2b334a3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c0ceeb0 con 0x7f2b2c15d2e0 2023-11-08T19:26:37.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.212+0000 7f2b334a3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0cebd0 con 0x7f2b2c0d1c50 2023-11-08T19:26:37.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.212+0000 7f2b32ca2640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2b2c0d0d40 0x7f2b2c0b3980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41748/0 (socket says 172.21.15.119:41748) 2023-11-08T19:26:37.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.212+0000 7f2b32ca2640 1 -- 172.21.15.119:0/2378213231 learned_addr learned my addr 172.21.15.119:0/2378213231 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:37.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2378213231 >> v2:172.21.15.188:3300/0 conn(0x7f2b2c0d1c50 0x7f2b2c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243406912 0 0) 0x7f2b2c0b2fb0 con 0x7f2b2c0d0d40 2023-11-08T19:26:37.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b18003520 con 0x7f2b2c0d0d40 2023-11-08T19:26:37.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3286430369 0 0) 0x7f2b18003520 con 0x7f2b2c0d0d40 2023-11-08T19:26:37.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 >> v2:172.21.15.188:3300/0 conn(0x7f2b2c0d1c50 msgr2=0x7f2b2c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 --2- 172.21.15.119:0/2378213231 >> v2:172.21.15.188:3300/0 conn(0x7f2b2c0d1c50 0x7f2b2c0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 >> v1:172.21.15.188:6789/0 conn(0x7f2b2c15d2e0 legacy=0x7f2b2c0d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0b2960 con 0x7f2b2c0d0d40 2023-11-08T19:26:37.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2b240019f0 con 0x7f2b2c0d0d40 2023-11-08T19:26:37.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 >> v1:172.21.15.119:6789/0 conn(0x7f2b2c0d0d40 legacy=0x7f2b2c0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b180039c0 0x7f2b18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b180047c0 0x7f2b18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.213+0000 7f2b3149f640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b18005180 0x7f2b1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0cebd0 con 0x7f2b18005180 2023-11-08T19:26:37.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1800c060 con 0x7f2b180039c0 2023-11-08T19:26:37.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1800c220 con 0x7f2b180047c0 2023-11-08T19:26:37.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b18005180 0x7f2b1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b324a1640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b180039c0 0x7f2b18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b180047c0 0x7f2b18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b32ca2640 1 -- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b180047c0 msgr2=0x7f2b18004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b180047c0 0x7f2b18004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b32ca2640 1 -- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b180039c0 msgr2=0x7f2b18003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b180039c0 0x7f2b18003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b32ca2640 1 -- 172.21.15.119:0/2378213231 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18003520 con 0x7f2b18005180 2023-11-08T19:26:37.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.214+0000 7f2b324a1640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b180039c0 0x7f2b18003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:37.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.215+0000 7f2b32ca2640 1 -- 172.21.15.119:0/2378213231 reap_dead start 2023-11-08T19:26:37.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.215+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b18005180 0x7f2b1800b980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400a760 tx=0x7f2b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540a55c294e0ae86 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24034630 con 0x7f2b18005180 2023-11-08T19:26:37.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2402f070 con 0x7f2b18005180 2023-11-08T19:26:37.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b3149f640 1 -- 172.21.15.119:0/2378213231 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24034a10 con 0x7f2b18005180 2023-11-08T19:26:37.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 -- 172.21.15.119:0/2378213231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b18005180 msgr2=0x7f2b1800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b18005180 0x7f2b1800b980 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400a760 tx=0x7f2b180057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 -- 172.21.15.119:0/2378213231 shutdown_connections 2023-11-08T19:26:37.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2378213231 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b18005180 0x7f2b1800b980 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 -- 172.21.15.119:0/2378213231 >> 172.21.15.119:0/2378213231 conn(0x7f2b2c0d2f60 msgr2=0x7f2b2c0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 -- 172.21.15.119:0/2378213231 shutdown_connections 2023-11-08T19:26:37.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.216+0000 7f2b334a3640 1 -- 172.21.15.119:0/2378213231 wait complete. 2023-11-08T19:26:37.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.218+0000 7f2b334a3640 1 Processor -- start 2023-11-08T19:26:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.218+0000 7f2b334a3640 1 -- start start 2023-11-08T19:26:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.219+0000 7f2b334a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b18005180 0x7f2b2c159f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.219+0000 7f2b334a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 0x7f2b2c06b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.219+0000 7f2b334a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2c06be20 0x7f2b2c0c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.219+0000 7f2b334a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c1f6cd0 con 0x7f2b2c0c9190 2023-11-08T19:26:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.219+0000 7f2b334a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c06c2a0 con 0x7f2b2c06be20 2023-11-08T19:26:37.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.219+0000 7f2b334a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c1f5e00 con 0x7f2b18005180 2023-11-08T19:26:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 0x7f2b2c06b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 0x7f2b2c06b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47798/0 (socket says 172.21.15.119:47798) 2023-11-08T19:26:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2843625169 learned_addr learned my addr 172.21.15.119:0/2843625169 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2c06be20 0x7f2b2c0c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b324a1640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b18005180 0x7f2b2c159f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b18005180 msgr2=0x7f2b2c159f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b18005180 0x7f2b2c159f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2c06be20 msgr2=0x7f2b2c0c6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2c06be20 0x7f2b2c0c6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2843625169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c1f6050 con 0x7f2b2c0c9190 2023-11-08T19:26:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2c06be20 0x7f2b2c0c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 0x7f2b2c06b8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c010860 tx=0x7f2b1c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.220+0000 7f2b22ffd640 1 -- 172.21.15.119:0/2843625169 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c009610 con 0x7f2b2c0c9190 2023-11-08T19:26:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.221+0000 7f2b22ffd640 1 -- 172.21.15.119:0/2843625169 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00eb60 con 0x7f2b2c0c9190 2023-11-08T19:26:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.221+0000 7f2b22ffd640 1 -- 172.21.15.119:0/2843625169 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c01c400 con 0x7f2b2c0c9190 2023-11-08T19:26:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.221+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c1f6330 con 0x7f2b2c0c9190 2023-11-08T19:26:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.221+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c1f68f0 con 0x7f2b2c0c9190 2023-11-08T19:26:37.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.223+0000 7f2b22ffd640 1 -- 172.21.15.119:0/2843625169 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b1c01c560 con 0x7f2b2c0c9190 2023-11-08T19:26:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.224+0000 7f2b22ffd640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b1006aea0 0x7f2b1006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.224+0000 7f2b22ffd640 1 -- 172.21.15.119:0/2843625169 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b1c089c80 con 0x7f2b2c0c9190 2023-11-08T19:26:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.225+0000 7f2b324a1640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b1006aea0 0x7f2b1006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.225+0000 7f2b37371740 1 Processor -- start 2023-11-08T19:26:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.225+0000 7f2b324a1640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b1006aea0 0x7f2b1006d360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2b280015b0 tx=0x7f2b28009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.225+0000 7f2b37371740 1 -- start start 2023-11-08T19:26:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.226+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12080 0x562e329e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.226+0000 7f2b37371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.226+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e329e54c0 0x562e32a15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.226+0000 7f2b37371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e329577e0 con 0x562e32a12460 2023-11-08T19:26:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.226+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e32954ed0 con 0x562e32a12080 2023-11-08T19:26:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.226+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e329507c0 con 0x562e329e54c0 2023-11-08T19:26:37.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a14e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47806/0 (socket says 172.21.15.119:47806) 2023-11-08T19:26:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2111667202 learned_addr learned my addr 172.21.15.119:0/2111667202 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b324a1640 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12080 0x562e329e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e329e54c0 0x562e32a15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e329e54c0 msgr2=0x562e32a15350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e329e54c0 0x562e32a15350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12080 msgr2=0x562e329e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12080 0x562e329e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 -- 172.21.15.119:0/2111667202 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e32923b10 con 0x562e32a12460 2023-11-08T19:26:37.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e329e54c0 0x562e32a15350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:37.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a14e10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c008010 tx=0x7f2b1c0583e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b751cb15daeacb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.227+0000 7f2b0ffff640 1 -- 172.21.15.119:0/2111667202 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c010070 con 0x562e32a12460 2023-11-08T19:26:37.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b0ffff640 1 -- 172.21.15.119:0/2111667202 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c070170 con 0x562e32a12460 2023-11-08T19:26:37.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b0ffff640 1 -- 172.21.15.119:0/2111667202 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c057e40 con 0x562e32a12460 2023-11-08T19:26:37.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 -- 172.21.15.119:0/2111667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 msgr2=0x562e32a14e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a14e10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c008010 tx=0x7f2b1c0583e0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 -- 172.21.15.119:0/2111667202 shutdown_connections 2023-11-08T19:26:37.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e329e54c0 0x562e32a15350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a14e10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 --2- 172.21.15.119:0/2111667202 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12080 0x562e329e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 -- 172.21.15.119:0/2111667202 >> 172.21.15.119:0/2111667202 conn(0x562e329e7290 msgr2=0x562e32a2fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 -- 172.21.15.119:0/2111667202 shutdown_connections 2023-11-08T19:26:37.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.228+0000 7f2b37371740 1 -- 172.21.15.119:0/2111667202 wait complete. 2023-11-08T19:26:37.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.229+0000 7f2b37371740 1 -- start start 2023-11-08T19:26:37.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.229+0000 7f2b37371740 1 Processor -- start 2023-11-08T19:26:37.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.230+0000 7f2b37371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a3fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.230+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12460 0x562e32a443e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.230+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a18310 0x562e32a03fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.230+0000 7f2b37371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e328edc40 con 0x562e32a12080 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.230+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e328ed8e0 con 0x562e32a12460 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.230+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e326b3f20 con 0x562e32a18310 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b37371740 10 client.0 _ll_register_callbacks cb 0x562e3290fcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a3fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a3fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47808/0 (socket says 172.21.15.119:47808) 2023-11-08T19:26:37.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 -- 172.21.15.119:0/529691999 learned_addr learned my addr 172.21.15.119:0/529691999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 -- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a18310 msgr2=0x562e32a03fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a18310 0x562e32a03fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 -- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12460 msgr2=0x562e32a443e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12460 0x562e32a443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 -- 172.21.15.119:0/529691999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c1f68f0 con 0x562e32a12080 2023-11-08T19:26:37.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b324a1640 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a3fef0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2b280136f0 tx=0x7f2b28001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206aca3d845c92d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.231+0000 7f2b0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:26:37.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/529691999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800c050 con 0x562e32a12080 2023-11-08T19:26:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/529691999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b28005310 con 0x562e32a12080 2023-11-08T19:26:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e32924c80 con 0x562e32a12080 2023-11-08T19:26:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/529691999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b280035f0 con 0x562e32a12080 2023-11-08T19:26:37.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 10 client.5472 fetch_fsmap learned FSMap version 18 2023-11-08T19:26:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x562e3294ee40 con 0x562e32a12080 2023-11-08T19:26:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/529691999 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e32924c80 con 0x562e32a12080 2023-11-08T19:26:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/529691999 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b28003750 con 0x562e32a12080 2023-11-08T19:26:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 10 client.5472 fetch_fsmap finished waiting for FSMap version 18 2023-11-08T19:26:37.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 10 client.5472 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:26:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 20 client.5472 populate_metadata read hostname 'smithi119' 2023-11-08T19:26:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a04500 0x562e32a30340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 4 client.5472 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:26:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.232+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 --> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e329497a0 con 0x562e32a04500 2023-11-08T19:26:37.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.233+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a04500 0x562e32a30340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.234+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a04500 0x562e32a30340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f18e9a5dcc53630 server_cookie=eaac9eb82b21cc0a in_seq=0 out_seq=0 2023-11-08T19:26:37.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.234+0000 7f2b0d7fa640 10 client.5472 ms_handle_connect on v2:172.21.15.188:6832/4172930163 2023-11-08T19:26:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.235+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/529691999 <== mds.0 v2:172.21.15.188:6832/4172930163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e329497a0 con 0x562e32a04500 2023-11-08T19:26:37.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.235+0000 7f2b0d7fa640 10 client.5472 handle_command_reply: tid=0 2023-11-08T19:26:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 client.5472 shutdown 2023-11-08T19:26:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 msgr2=0x562e32a3fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a3fef0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f2b280136f0 tx=0x7f2b28001190 comp rx=0 tx=0).stop 2023-11-08T19:26:37.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 shutdown_connections 2023-11-08T19:26:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a04500 0x562e32a30340 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a18310 0x562e32a03fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a12460 0x562e32a443e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b0d7fa640 0 client.5472 ms_handle_reset on v2:172.21.15.188:6832/4172930163 2023-11-08T19:26:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 --2- 172.21.15.119:0/529691999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a3fef0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 >> 172.21.15.119:0/529691999 conn(0x562e329e54c0 msgr2=0x562e32a29c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 shutdown_connections 2023-11-08T19:26:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 1 -- 172.21.15.119:0/529691999 wait complete. 2023-11-08T19:26:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.236+0000 7f2b37371740 20 client.5472 trim_cache size 0 max 16384 2023-11-08T19:26:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.253+0000 7f2b37371740 1 Processor -- start 2023-11-08T19:26:37.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.253+0000 7f2b37371740 1 -- start start 2023-11-08T19:26:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b37371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a30530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12460 0x562e32a14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a2cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b37371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e329e9b50 con 0x562e32a12080 2023-11-08T19:26:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e329e5ad0 con 0x562e32a18310 2023-11-08T19:26:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e3291c3b0 con 0x562e32a12460 2023-11-08T19:26:37.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b324a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a30530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b324a1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a30530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47810/0 (socket says 172.21.15.119:47810) 2023-11-08T19:26:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.254+0000 7f2b324a1640 1 -- 172.21.15.119:0/3893217989 learned_addr learned my addr 172.21.15.119:0/3893217989 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a2cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12460 0x562e32a14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b324a1640 1 -- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12460 msgr2=0x562e32a14890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b324a1640 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12460 0x562e32a14890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b324a1640 1 -- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 msgr2=0x562e32a2cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b324a1640 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a2cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b324a1640 1 -- 172.21.15.119:0/3893217989 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e328ecb90 con 0x562e32a12080 2023-11-08T19:26:37.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b32ca2640 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a2cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:37.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b324a1640 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a30530 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2b280014c0 tx=0x7f2b2801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de3752748b332940 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b20ff9640 1 -- 172.21.15.119:0/3893217989 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2800c070 con 0x562e32a12080 2023-11-08T19:26:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b20ff9640 1 -- 172.21.15.119:0/3893217989 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2800d040 con 0x562e32a12080 2023-11-08T19:26:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.255+0000 7f2b20ff9640 1 -- 172.21.15.119:0/3893217989 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28003c60 con 0x562e32a12080 2023-11-08T19:26:37.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 -- 172.21.15.119:0/3893217989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 msgr2=0x562e32a30530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a30530 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2b280014c0 tx=0x7f2b2801f000 comp rx=0 tx=0).stop 2023-11-08T19:26:37.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 -- 172.21.15.119:0/3893217989 shutdown_connections 2023-11-08T19:26:37.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a2cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12460 0x562e32a14890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 --2- 172.21.15.119:0/3893217989 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12080 0x562e32a30530 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 -- 172.21.15.119:0/3893217989 >> 172.21.15.119:0/3893217989 conn(0x562e329e54c0 msgr2=0x562e32a110b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 -- 172.21.15.119:0/3893217989 shutdown_connections 2023-11-08T19:26:37.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.256+0000 7f2b37371740 1 -- 172.21.15.119:0/3893217989 wait complete. 2023-11-08T19:26:37.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.257+0000 7f2b37371740 1 -- start start 2023-11-08T19:26:37.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.257+0000 7f2b37371740 1 Processor -- start 2023-11-08T19:26:37.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12080 0x562e32a9a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a439a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a14890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e32a33320 con 0x562e32a12460 2023-11-08T19:26:37.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e32a53360 con 0x562e32a18310 2023-11-08T19:26:37.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e32aa4160 con 0x562e32a12080 2023-11-08T19:26:37.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b37371740 10 client.0 _ll_register_callbacks cb 0x7f2b2800e510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:26:37.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b31ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b31ca0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47820/0 (socket says 172.21.15.119:47820) 2023-11-08T19:26:37.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b31ca0640 1 -- 172.21.15.119:0/536192673 learned_addr learned my addr 172.21.15.119:0/536192673 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.258+0000 7f2b31ca0640 1 -- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12080 msgr2=0x562e32a9a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12080 0x562e32a9a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b31ca0640 1 -- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 msgr2=0x562e32a14890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a14890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b31ca0640 1 -- 172.21.15.119:0/536192673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e3294ee40 con 0x562e32a12460 2023-11-08T19:26:37.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b31ca0640 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a439a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c056ca0 tx=0x7f2b1c058ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b66ec9a3707af6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:26:37.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/536192673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c057e40 con 0x562e32a12460 2023-11-08T19:26:37.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/536192673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7f2b1c057fa0 con 0x562e32a12460 2023-11-08T19:26:37.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b1c070170 con 0x562e32a12460 2023-11-08T19:26:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/536192673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b1c0771f0 con 0x562e32a12460 2023-11-08T19:26:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b37371740 10 client.5478 fetch_fsmap learned FSMap version 18 2023-11-08T19:26:37.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.259+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x562e32912180 con 0x562e32a12460 2023-11-08T19:26:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/536192673 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c070170 con 0x562e32a12460 2023-11-08T19:26:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/536192673 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b1c024070 con 0x562e32a12460 2023-11-08T19:26:37.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b37371740 10 client.5478 fetch_fsmap finished waiting for FSMap version 18 2023-11-08T19:26:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b37371740 10 client.5478 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:26:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b37371740 20 client.5478 populate_metadata read hostname 'smithi119' 2023-11-08T19:26:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b37371740 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a14fe0 0x562e32a31440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b37371740 4 client.5478 mds_command: new command op to 5310 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:26:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 --> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e329494c0 con 0x562e32a14fe0 2023-11-08T19:26:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.260+0000 7f2b324a1640 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a14fe0 0x562e32a31440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.261+0000 7f2b324a1640 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a14fe0 0x562e32a31440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e56d3e6ab88847c1 server_cookie=e1c2975ad5b6c95e in_seq=0 out_seq=0 2023-11-08T19:26:37.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.261+0000 7f2b0d7fa640 10 client.5478 ms_handle_connect on v2:172.21.15.188:6832/4172930163 2023-11-08T19:26:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b0d7fa640 1 -- 172.21.15.119:0/536192673 <== mds.0 v2:172.21.15.188:6832/4172930163 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x562e329494c0 con 0x562e32a14fe0 2023-11-08T19:26:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b0d7fa640 10 client.5478 handle_command_reply: tid=0 2023-11-08T19:26:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 client.5478 shutdown 2023-11-08T19:26:37.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 msgr2=0x562e32a439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a439a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c056ca0 tx=0x7f2b1c058ad0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 shutdown_connections 2023-11-08T19:26:37.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:6832/4172930163,v1:172.21.15.188:6834/4172930163] conn(0x562e32a14fe0 0x562e32a31440 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562e32a18310 0x562e32a14890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562e32a12460 0x562e32a439a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b0d7fa640 0 client.5478 ms_handle_reset on v2:172.21.15.188:6832/4172930163 2023-11-08T19:26:37.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 --2- 172.21.15.119:0/536192673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562e32a12080 0x562e32a9a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 >> 172.21.15.119:0/536192673 conn(0x562e329e54c0 msgr2=0x562e329e4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 shutdown_connections 2023-11-08T19:26:37.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.262+0000 7f2b37371740 1 -- 172.21.15.119:0/536192673 wait complete. 2023-11-08T19:26:37.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b37371740 20 client.5478 trim_cache size 0 max 16384 2023-11-08T19:26:37.292 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:26:37.292 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:26:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:26:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5310, 2023-11-08T19:26:37.293 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:26:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:26:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:26:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.6387951230000004, 2023-11-08T19:26:37.294 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 18, 2023-11-08T19:26:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 39, 2023-11-08T19:26:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 39, 2023-11-08T19:26:37.295 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 15.88431087 2023-11-08T19:26:37.296 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:26:37.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b1006aea0 msgr2=0x7f2b1006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b1006aea0 0x7f2b1006d360 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2b280015b0 tx=0x7f2b28009040 comp rx=0 tx=0).stop 2023-11-08T19:26:37.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 msgr2=0x7f2b2c06b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 0x7f2b2c06b8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c010860 tx=0x7f2b1c010d20 comp rx=0 tx=0).stop 2023-11-08T19:26:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 shutdown_connections 2023-11-08T19:26:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b1006aea0 0x7f2b1006d360 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b2c06be20 0x7f2b2c0c6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b2c0c9190 0x7f2b2c06b8e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 --2- 172.21.15.119:0/2843625169 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b18005180 0x7f2b2c159f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 >> 172.21.15.119:0/2843625169 conn(0x7f2b2c0d2f60 msgr2=0x7f2b2c0b4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 shutdown_connections 2023-11-08T19:26:37.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.263+0000 7f2b334a3640 1 -- 172.21.15.119:0/2843625169 wait complete. 2023-11-08T19:26:37.302 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:26:37.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.517+0000 7fdc55f54640 1 Processor -- start 2023-11-08T19:26:37.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.519+0000 7fdc55f54640 1 -- start start 2023-11-08T19:26:37.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55f54640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc500ac8d0 0x7fdc500accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55f54640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc5006bc10 0x7fdc5006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55f54640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5006b780 con 0x7fdc500adee0 2023-11-08T19:26:37.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55f54640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5015db20 con 0x7fdc500ac8d0 2023-11-08T19:26:37.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55f54640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc500ad1f0 con 0x7fdc5006bc10 2023-11-08T19:26:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc500ac8d0 0x7fdc500accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc500ac8d0 0x7fdc500accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47824/0 (socket says 172.21.15.119:47824) 2023-11-08T19:26:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc4ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc500adee0 0x7fdc5015d410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41760/0 (socket says 172.21.15.119:41760) 2023-11-08T19:26:37.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 -- 172.21.15.119:0/1917063064 learned_addr learned my addr 172.21.15.119:0/1917063064 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55753640 1 --2- 172.21.15.119:0/1917063064 >> v2:172.21.15.188:3300/0 conn(0x7fdc5006bc10 0x7fdc5006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 -- 172.21.15.119:0/1917063064 >> v2:172.21.15.188:3300/0 conn(0x7fdc5006bc10 msgr2=0x7fdc5006c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 --2- 172.21.15.119:0/1917063064 >> v2:172.21.15.188:3300/0 conn(0x7fdc5006bc10 0x7fdc5006c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 -- 172.21.15.119:0/1917063064 >> v1:172.21.15.119:6789/0 conn(0x7fdc500adee0 legacy=0x7fdc5015d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc54f52640 1 -- 172.21.15.119:0/1917063064 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5015dd30 con 0x7fdc500ac8d0 2023-11-08T19:26:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.520+0000 7fdc55753640 1 --2- 172.21.15.119:0/1917063064 >> v2:172.21.15.188:3300/0 conn(0x7fdc5006bc10 0x7fdc5006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:37.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.521+0000 7fdc54f52640 1 --2- 172.21.15.119:0/1917063064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500ac8d0 0x7fdc500accb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009870 tx=0x7fdc4402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10249d1266f6310f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.521+0000 7fdc4f7fe640 1 -- 172.21.15.119:0/1917063064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44030850 con 0x7fdc500ac8d0 2023-11-08T19:26:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.521+0000 7fdc4f7fe640 1 -- 172.21.15.119:0/1917063064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc440309b0 con 0x7fdc500ac8d0 2023-11-08T19:26:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.521+0000 7fdc4f7fe640 1 -- 172.21.15.119:0/1917063064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44030d40 con 0x7fdc500ac8d0 2023-11-08T19:26:37.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 -- 172.21.15.119:0/1917063064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500ac8d0 msgr2=0x7fdc500accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 --2- 172.21.15.119:0/1917063064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500ac8d0 0x7fdc500accb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009870 tx=0x7fdc4402b990 comp rx=0 tx=0).stop 2023-11-08T19:26:37.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 -- 172.21.15.119:0/1917063064 shutdown_connections 2023-11-08T19:26:37.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 --2- 172.21.15.119:0/1917063064 >> v2:172.21.15.188:3300/0 conn(0x7fdc5006bc10 0x7fdc5006c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 --2- 172.21.15.119:0/1917063064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500ac8d0 0x7fdc500accb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 -- 172.21.15.119:0/1917063064 >> 172.21.15.119:0/1917063064 conn(0x7fdc500b0560 msgr2=0x7fdc500b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 -- 172.21.15.119:0/1917063064 shutdown_connections 2023-11-08T19:26:37.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.522+0000 7fdc55f54640 1 -- 172.21.15.119:0/1917063064 wait complete. 2023-11-08T19:26:37.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.524+0000 7fdc55f54640 1 Processor -- start 2023-11-08T19:26:37.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.524+0000 7fdc55f54640 1 -- start start 2023-11-08T19:26:37.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55f54640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc5006bc10 0x7fdc5014e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55f54640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc500ac8d0 0x7fdc50160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55f54640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 0x7fdc50160960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55f54640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5014e5d0 con 0x7fdc500adee0 2023-11-08T19:26:37.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55f54640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc500b4060 con 0x7fdc500ac8d0 2023-11-08T19:26:37.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55f54640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50152780 con 0x7fdc5006bc10 2023-11-08T19:26:37.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.525+0000 7fdc55753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 0x7fdc50160960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 0x7fdc50160960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:47828/0 (socket says 172.21.15.119:47828) 2023-11-08T19:26:37.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 -- 172.21.15.119:0/3357468486 learned_addr learned my addr 172.21.15.119:0/3357468486 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:37.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc54f52640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc5006bc10 0x7fdc5014e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc4ffff640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc500ac8d0 0x7fdc50160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 -- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc5006bc10 msgr2=0x7fdc5014e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc5006bc10 0x7fdc5014e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 -- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc500ac8d0 msgr2=0x7fdc50160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc500ac8d0 0x7fdc50160420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 -- 172.21.15.119:0/3357468486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440094d0 con 0x7fdc500adee0 2023-11-08T19:26:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc55753640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 0x7fdc50160960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800cef0 tx=0x7fdc4800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800d840 con 0x7fdc500adee0 2023-11-08T19:26:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.526+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4800d9c0 con 0x7fdc500adee0 2023-11-08T19:26:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.527+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4800dd70 con 0x7fdc500adee0 2023-11-08T19:26:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.527+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc50152a60 con 0x7fdc500adee0 2023-11-08T19:26:37.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.527+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc50153020 con 0x7fdc500adee0 2023-11-08T19:26:37.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.528+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc50066920 con 0x7fdc500adee0 2023-11-08T19:26:37.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.529+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc4801e3b0 con 0x7fdc500adee0 2023-11-08T19:26:37.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.529+0000 7fdc4d7fa640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3c06ae70 0x7fdc3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:37.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.530+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fdc4808a240 con 0x7fdc500adee0 2023-11-08T19:26:37.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.530+0000 7fdc54f52640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3c06ae70 0x7fdc3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:37.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.530+0000 7fdc54f52640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3c06ae70 0x7fdc3c06d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc44036000 tx=0x7fdc4403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:37.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.534+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc48055260 con 0x7fdc500adee0 2023-11-08T19:26:37.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.763+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc50153240 con 0x7fdc500adee0 2023-11-08T19:26:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.763+0000 7fdc4d7fa640 1 -- 172.21.15.119:0/3357468486 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc48058880 con 0x7fdc500adee0 2023-11-08T19:26:37.766 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:26:37.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3c06ae70 msgr2=0x7fdc3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3c06ae70 0x7fdc3c06d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdc44036000 tx=0x7fdc4403b040 comp rx=0 tx=0).stop 2023-11-08T19:26:37.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 msgr2=0x7fdc50160960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:37.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 0x7fdc50160960 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800cef0 tx=0x7fdc4800a710 comp rx=0 tx=0).stop 2023-11-08T19:26:37.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 shutdown_connections 2023-11-08T19:26:37.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3c06ae70 0x7fdc3c06d330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc500adee0 0x7fdc50160960 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc500ac8d0 0x7fdc50160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 --2- 172.21.15.119:0/3357468486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc5006bc10 0x7fdc5014e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:37.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 >> 172.21.15.119:0/3357468486 conn(0x7fdc500b0560 msgr2=0x7fdc50070960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:37.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 shutdown_connections 2023-11-08T19:26:37.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:37.766+0000 7fdc55f54640 1 -- 172.21.15.119:0/3357468486 wait complete. 2023-11-08T19:26:37.788 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:26:37.788 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:26:37.789 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:26:37.789 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:26:37.790 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:37.790 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:26:37.790 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:26:37.791 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:26:37.791 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:37.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:37.792 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:26:37.845 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:26:37.845 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:26:37.845 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:26:37.846 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.846 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:26:37.846 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.846 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:37.846 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:37.847 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:26:37.847 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:37.847 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:37.847 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:26:37.847 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:26:37.848 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:37.848 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:26:37.848 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:37.848 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:26:37.848 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42273sec preferred_lft 42273sec 2023-11-08T19:26:37.848 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:26:37.849 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.849 INFO:teuthology.orchestra.run.smithi119.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.850 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::e06c:64ff:fe55:3f8b/64 scope link 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi119.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:26:37.851 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c086:39ff:fe1e:9142/64 scope link 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi119.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:26:37.852 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::494:cbff:fed6:d560/64 scope link 2023-11-08T19:26:37.853 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:26:37.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:37.855 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:26:37.949 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:26:37.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:37.954 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:26:38.008 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:26:38.009 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:26:38.009 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:26:38.009 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:26:38.009 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:26:38.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:38.010 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:26:38.011 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:26:38.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:26:38.011 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:26:38.012 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:26:38.012 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:26:38.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:38.195 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:26:38.248 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:26:38.249 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:26:38.250 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:38.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:38.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:26:38.341 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:26:38.341 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:38.390 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:26:38.391 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:26:38.391 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:38.441 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T19:26:38.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:38.442 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:26:38.579 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:26:38.579 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:26:38.613 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:26:38.613 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:26:38.613 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:26:38.614 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:26:38.614 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:26:38.615 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-08T19:26:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:26:38.615 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:26:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:26:38.616 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:26:38.619 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:26:38.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:38.620 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:38.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:38.706 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:26:38.778 INFO:teuthology.orchestra.run.smithi119.stdout:client5490 2023-11-08T19:26:38.779 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:38.779 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:38.834 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create snap-test-pool 2023-11-08T19:26:39.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.051+0000 7fe47abd7640 1 Processor -- start 2023-11-08T19:26:39.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.052+0000 7fe47abd7640 1 -- start start 2023-11-08T19:26:39.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.053+0000 7fe47abd7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe4740c3a50 0x7fe4740cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.053+0000 7fe47abd7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47406d220 con 0x7fe47415d000 2023-11-08T19:26:39.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.053+0000 7fe47abd7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4740aca40 con 0x7fe47406a830 2023-11-08T19:26:39.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.053+0000 7fe47abd7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4740ccd50 con 0x7fe4740c3a50 2023-11-08T19:26:39.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe478973640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe47415d000 0x7fe47406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58288/0 (socket says 172.21.15.119:58288) 2023-11-08T19:26:39.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe478973640 1 -- 172.21.15.119:0/1350249114 learned_addr learned my addr 172.21.15.119:0/1350249114 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:39.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe479174640 1 --2- 172.21.15.119:0/1350249114 >> v2:172.21.15.119:3300/0 conn(0x7fe4740c3a50 0x7fe4740cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe4737fe640 1 -- 172.21.15.119:0/1350249114 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510667892 0 0) 0x7fe47406d220 con 0x7fe47415d000 2023-11-08T19:26:39.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe4737fe640 1 -- 172.21.15.119:0/1350249114 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe464003520 con 0x7fe47415d000 2023-11-08T19:26:39.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe479174640 1 -- 172.21.15.119:0/1350249114 >> v1:172.21.15.188:6789/0 conn(0x7fe47406a830 legacy=0x7fe474160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe479174640 1 -- 172.21.15.119:0/1350249114 >> v1:172.21.15.119:6789/0 conn(0x7fe47415d000 legacy=0x7fe47406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.054+0000 7fe479174640 1 -- 172.21.15.119:0/1350249114 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4740acd20 con 0x7fe4740c3a50 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.055+0000 7fe479174640 1 --2- 172.21.15.119:0/1350249114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4740c3a50 0x7fe4740cc810 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009d90 tx=0x7fe46c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e52f350967ee44d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe4737fe640 1 -- 172.21.15.119:0/1350249114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c02cbd0 con 0x7fe4740c3a50 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe4737fe640 1 -- 172.21.15.119:0/1350249114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c02cd30 con 0x7fe4740c3a50 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe4737fe640 1 -- 172.21.15.119:0/1350249114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c02cea0 con 0x7fe4740c3a50 2023-11-08T19:26:39.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe47abd7640 1 -- 172.21.15.119:0/1350249114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4740c3a50 msgr2=0x7fe4740cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe47abd7640 1 --2- 172.21.15.119:0/1350249114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4740c3a50 0x7fe4740cc810 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009d90 tx=0x7fe46c02be20 comp rx=0 tx=0).stop 2023-11-08T19:26:39.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe47abd7640 1 -- 172.21.15.119:0/1350249114 shutdown_connections 2023-11-08T19:26:39.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe47abd7640 1 --2- 172.21.15.119:0/1350249114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4740c3a50 0x7fe4740cc810 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe47abd7640 1 -- 172.21.15.119:0/1350249114 >> 172.21.15.119:0/1350249114 conn(0x7fe4740cb610 msgr2=0x7fe4740cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.056+0000 7fe47abd7640 1 -- 172.21.15.119:0/1350249114 shutdown_connections 2023-11-08T19:26:39.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.057+0000 7fe47abd7640 1 -- 172.21.15.119:0/1350249114 wait complete. 2023-11-08T19:26:39.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.058+0000 7fe47abd7640 1 Processor -- start 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.058+0000 7fe47abd7640 1 -- start start 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.059+0000 7fe47abd7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47406a830 0x7fe474154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.059+0000 7fe47abd7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4740c3a50 0x7fe4740b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.059+0000 7fe47abd7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 0x7fe4741f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.059+0000 7fe47abd7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b3440 con 0x7fe47415d000 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.059+0000 7fe47abd7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b27c0 con 0x7fe4740c3a50 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.059+0000 7fe47abd7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4740b2aa0 con 0x7fe47406a830 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 0x7fe4741f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 0x7fe4741f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55310/0 (socket says 172.21.15.119:55310) 2023-11-08T19:26:39.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 -- 172.21.15.119:0/10005045 learned_addr learned my addr 172.21.15.119:0/10005045 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe478973640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47406a830 0x7fe474154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe473fff640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4740c3a50 0x7fe4740b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 -- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47406a830 msgr2=0x7fe474154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47406a830 0x7fe474154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 -- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4740c3a50 msgr2=0x7fe4740b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4740c3a50 0x7fe4740b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 -- 172.21.15.119:0/10005045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c009a70 con 0x7fe47415d000 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.060+0000 7fe479174640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 0x7fe4741f60b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe46c02be60 tx=0x7fe46c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.061+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c004250 con 0x7fe47415d000 2023-11-08T19:26:39.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.061+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe46c0043d0 con 0x7fe47415d000 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.061+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c004800 con 0x7fe47415d000 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.061+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4740b2d80 con 0x7fe47415d000 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.061+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe474154e70 con 0x7fe47415d000 2023-11-08T19:26:39.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.062+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4740b3580 con 0x7fe47415d000 2023-11-08T19:26:39.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.063+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe46c03b070 con 0x7fe47415d000 2023-11-08T19:26:39.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.064+0000 7fe4717fa640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe45806aea0 0x7fe45806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.064+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6029+0+0 (secure 0 0 0) 0x7fe46c0ab8c0 con 0x7fe47415d000 2023-11-08T19:26:39.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.065+0000 7fe478973640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe45806aea0 0x7fe45806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.065+0000 7fe478973640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe45806aea0 0x7fe45806d360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe468007fd0 tx=0x7fe468009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.069+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4740b3580 con 0x7fe47415d000 2023-11-08T19:26:39.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.260+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "snap-test-pool"} v 0) v1 -- 0x7fe4740b3580 con 0x7fe47415d000 2023-11-08T19:26:39.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.692+0000 7fe4717fa640 1 -- 172.21.15.119:0/10005045 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "snap-test-pool"}]=0 pool 'snap-test-pool' created v40) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe4740b3580 con 0x7fe47415d000 2023-11-08T19:26:39.694 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'snap-test-pool' created 2023-11-08T19:26:39.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.696+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe45806aea0 msgr2=0x7fe45806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.696+0000 7fe47abd7640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe45806aea0 0x7fe45806d360 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe468007fd0 tx=0x7fe468009040 comp rx=0 tx=0).stop 2023-11-08T19:26:39.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.696+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 msgr2=0x7fe4741f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.696+0000 7fe47abd7640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 0x7fe4741f60b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe46c02be60 tx=0x7fe46c004a40 comp rx=0 tx=0).stop 2023-11-08T19:26:39.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 shutdown_connections 2023-11-08T19:26:39.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe45806aea0 0x7fe45806d360 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe47415d000 0x7fe4741f60b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4740c3a50 0x7fe4740b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 --2- 172.21.15.119:0/10005045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe47406a830 0x7fe474154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 >> 172.21.15.119:0/10005045 conn(0x7fe4740cb610 msgr2=0x7fe4740cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.697+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 shutdown_connections 2023-11-08T19:26:39.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.698+0000 7fe47abd7640 1 -- 172.21.15.119:0/10005045 wait complete. 2023-11-08T19:26:39.724 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool mksnap snap3 2023-11-08T19:26:39.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.807+0000 7f1044370f40 1 Processor -- start 2023-11-08T19:26:39.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.807+0000 7f1044370f40 1 -- start start 2023-11-08T19:26:39.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.808+0000 7f1044370f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8a9bdd20 0x561b8a9bb580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.808+0000 7f1044370f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 0x561b8aa700a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.808+0000 7f1044370f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 0x561b8a9d5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.808+0000 7f1044370f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a93d340 con 0x561b8aa659a0 2023-11-08T19:26:39.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.808+0000 7f1044370f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a8d7a70 con 0x561b8a9db280 2023-11-08T19:26:39.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.808+0000 7f1044370f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b8a8d7710 con 0x561b8a9bdd20 2023-11-08T19:26:39.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 0x561b8a9d5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 0x561b8a9d5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55338/0 (socket says 172.21.15.119:55338) 2023-11-08T19:26:39.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 -- 172.21.15.119:0/1313523163 learned_addr learned my addr 172.21.15.119:0/1313523163 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:39.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 -- 172.21.15.119:0/1313523163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8a9bdd20 msgr2=0x561b8a9bb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8a9bdd20 0x561b8a9bb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 -- 172.21.15.119:0/1313523163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 msgr2=0x561b8aa700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:39.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 0x561b8aa700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 -- 172.21.15.119:0/1313523163 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102c005de0 con 0x561b8aa659a0 2023-11-08T19:26:39.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f1042673640 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 0x561b8a9d5580 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f103c00a490 tx=0x7f103c00a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2267f380a3a7cb3c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.809+0000 7f103bfff640 1 -- 172.21.15.119:0/1313523163 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c00b980 con 0x561b8aa659a0 2023-11-08T19:26:39.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f103bfff640 1 -- 172.21.15.119:0/1313523163 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c00bb00 con 0x561b8aa659a0 2023-11-08T19:26:39.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f103bfff640 1 -- 172.21.15.119:0/1313523163 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c008460 con 0x561b8aa659a0 2023-11-08T19:26:39.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 -- 172.21.15.119:0/1313523163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 msgr2=0x561b8a9d5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 0x561b8a9d5580 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f103c00a490 tx=0x7f103c00a950 comp rx=0 tx=0).stop 2023-11-08T19:26:39.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 -- 172.21.15.119:0/1313523163 shutdown_connections 2023-11-08T19:26:39.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8aa659a0 0x561b8a9d5580 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 0x561b8aa700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 --2- 172.21.15.119:0/1313523163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8a9bdd20 0x561b8a9bb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 -- 172.21.15.119:0/1313523163 >> 172.21.15.119:0/1313523163 conn(0x561b8a98c570 msgr2=0x561b8a9d9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:39.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 -- 172.21.15.119:0/1313523163 shutdown_connections 2023-11-08T19:26:39.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.810+0000 7f1044370f40 1 -- 172.21.15.119:0/1313523163 wait complete. 2023-11-08T19:26:39.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.811+0000 7f1044370f40 1 Processor -- start 2023-11-08T19:26:39.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.811+0000 7f1044370f40 1 -- start start 2023-11-08T19:26:39.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1044370f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 0x561b8aa574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1044370f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 0x561b8aa5f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1044370f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8aa659a0 0x561b8aa5f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1044370f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8a8cb520 con 0x561b8a9bdd20 2023-11-08T19:26:39.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1044370f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8aa5d2e0 con 0x561b8a9db280 2023-11-08T19:26:39.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1044370f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561b8aa5c9f0 con 0x561b8aa659a0 2023-11-08T19:26:39.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1041e72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 0x561b8aa574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1041e72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 0x561b8aa574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55348/0 (socket says 172.21.15.119:55348) 2023-11-08T19:26:39.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.812+0000 7f1041e72640 1 -- 172.21.15.119:0/2648278824 learned_addr learned my addr 172.21.15.119:0/2648278824 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:39.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1041e72640 1 -- 172.21.15.119:0/2648278824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8aa659a0 msgr2=0x561b8aa5f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1041e72640 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8aa659a0 0x561b8aa5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1041e72640 1 -- 172.21.15.119:0/2648278824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 msgr2=0x561b8aa5f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:39.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1041e72640 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 0x561b8aa5f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:39.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1041e72640 1 -- 172.21.15.119:0/2648278824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c005cc0 con 0x561b8a9bdd20 2023-11-08T19:26:39.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1041e72640 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 0x561b8aa574c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f103400a3e0 tx=0x7f103400a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:39.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340116b0 con 0x561b8a9bdd20 2023-11-08T19:26:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034011830 con 0x561b8a9bdd20 2023-11-08T19:26:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034011be0 con 0x561b8a9bdd20 2023-11-08T19:26:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561b8a8e3680 con 0x561b8a9bdd20 2023-11-08T19:26:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.813+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561b8a8d6830 con 0x561b8a9bdd20 2023-11-08T19:26:39.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.816+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1034011d40 con 0x561b8a9bdd20 2023-11-08T19:26:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.817+0000 7f103affd640 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1024069850 0x7f102406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.817+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6411+0+0 (secure 0 0 0) 0x7f1034088770 con 0x561b8a9bdd20 2023-11-08T19:26:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.817+0000 7f1041671640 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1024069850 0x7f102406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.817+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- pool_op(create snap pool 7 tid 1 name snap3 v0) v4 -- 0x561b8a9472c0 con 0x561b8a9bdd20 2023-11-08T19:26:39.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:39.818+0000 7f1041671640 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1024069850 0x7f102406bd10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f102c054000 tx=0x7f102c04c070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.704+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v41) v1 ==== 43+0+0 (secure 0 0 0) 0x7f10340535f0 con 0x561b8a9bdd20 2023-11-08T19:26:40.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.704+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=41}) v3 -- 0x7f10240725a0 con 0x561b8a9bdd20 2023-11-08T19:26:40.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.708+0000 7f103affd640 1 -- 172.21.15.119:0/2648278824 <== mon.0 v2:172.21.15.119:3300/0 7 ==== osd_map(41..41 src has 1..41) v4 ==== 718+0+0 (secure 0 0 0) 0x7f10340579b0 con 0x561b8a9bdd20 2023-11-08T19:26:40.711 INFO:teuthology.orchestra.run.smithi119.stdout:created pool snap-test-pool snap snap3 2023-11-08T19:26:40.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.710+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1024069850 msgr2=0x7f102406bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.710+0000 7f1044370f40 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1024069850 0x7f102406bd10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f102c054000 tx=0x7f102c04c070 comp rx=0 tx=0).stop 2023-11-08T19:26:40.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.710+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 msgr2=0x561b8aa574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.710+0000 7f1044370f40 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 0x561b8aa574c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f103400a3e0 tx=0x7f103400a8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 shutdown_connections 2023-11-08T19:26:40.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1024069850 0x7f102406bd10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561b8aa659a0 0x561b8aa5f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561b8a9db280 0x561b8aa5f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 --2- 172.21.15.119:0/2648278824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561b8a9bdd20 0x561b8aa574c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 >> 172.21.15.119:0/2648278824 conn(0x561b8a98c570 msgr2=0x561b8a9bb320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 shutdown_connections 2023-11-08T19:26:40.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.711+0000 7f1044370f40 1 -- 172.21.15.119:0/2648278824 wait complete. 2023-11-08T19:26:40.721 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs snap-test-pool 2023-11-08T19:26:40.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.912+0000 7f3c25fa9640 1 Processor -- start 2023-11-08T19:26:40.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.913+0000 7f3c25fa9640 1 -- start start 2023-11-08T19:26:40.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.914+0000 7f3c25fa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c2015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.914+0000 7f3c25fa9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c20144960 0x7f3c200cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.914+0000 7f3c25fa9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2015ea60 con 0x7f3c2015f1d0 2023-11-08T19:26:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.914+0000 7f3c25fa9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c200c9150 con 0x7f3c20144960 2023-11-08T19:26:40.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.914+0000 7f3c25fa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200cb270 con 0x7f3c20144580 2023-11-08T19:26:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1effd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c20144960 0x7f3c200cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c2015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1f7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c2015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:53776/0 (socket says 172.21.15.119:53776) 2023-11-08T19:26:40.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1effd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3c20144960 0x7f3c200cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56424/0 (socket says 172.21.15.119:56424) 2023-11-08T19:26:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1f7fe640 1 -- 172.21.15.119:0/3866537739 learned_addr learned my addr 172.21.15.119:0/3866537739 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1ffff640 1 --1- 172.21.15.119:0/3866537739 >> v1:172.21.15.188:6789/0 conn(0x7f3c2015f1d0 0x7f3c200c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:48418/0 (socket says 172.21.15.119:48418) 2023-11-08T19:26:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1effd640 1 -- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 msgr2=0x7f3c2015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.915+0000 7f3c1effd640 1 --2- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c2015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.916+0000 7f3c1effd640 1 -- 172.21.15.119:0/3866537739 >> v1:172.21.15.188:6789/0 conn(0x7f3c2015f1d0 legacy=0x7f3c200c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.916+0000 7f3c1effd640 1 -- 172.21.15.119:0/3866537739 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200c9360 con 0x7f3c20144960 2023-11-08T19:26:40.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.916+0000 7f3c1f7fe640 1 --2- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c2015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:40.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.917+0000 7f3c1effd640 1 --2- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c20144960 0x7f3c200cad30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b73d06b9b217e8ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c1e7fc640 1 -- 172.21.15.119:0/3866537739 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14031780 con 0x7f3c20144960 2023-11-08T19:26:40.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c1e7fc640 1 -- 172.21.15.119:0/3866537739 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c140318e0 con 0x7f3c20144960 2023-11-08T19:26:40.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c1e7fc640 1 -- 172.21.15.119:0/3866537739 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14031c70 con 0x7f3c20144960 2023-11-08T19:26:40.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 -- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c20144960 msgr2=0x7f3c200cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 --2- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c20144960 0x7f3c200cad30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3c140097a0 tx=0x7f3c1402b820 comp rx=0 tx=0).stop 2023-11-08T19:26:40.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 -- 172.21.15.119:0/3866537739 shutdown_connections 2023-11-08T19:26:40.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 --2- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c20144960 0x7f3c200cad30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 --2- 172.21.15.119:0/3866537739 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c2015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 -- 172.21.15.119:0/3866537739 >> 172.21.15.119:0/3866537739 conn(0x7f3c200cb610 msgr2=0x7f3c200cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 -- 172.21.15.119:0/3866537739 shutdown_connections 2023-11-08T19:26:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.918+0000 7f3c25fa9640 1 -- 172.21.15.119:0/3866537739 wait complete. 2023-11-08T19:26:40.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.920+0000 7f3c25fa9640 1 Processor -- start 2023-11-08T19:26:40.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.920+0000 7f3c25fa9640 1 -- start start 2023-11-08T19:26:40.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.921+0000 7f3c25fa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c20152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.921+0000 7f3c25fa9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c20144960 0x7f3c200cdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.921+0000 7f3c25fa9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c2015f1d0 0x7f3c200c6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.921+0000 7f3c25fa9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200cd7e0 con 0x7f3c20144960 2023-11-08T19:26:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.921+0000 7f3c25fa9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20152ba0 con 0x7f3c2015f1d0 2023-11-08T19:26:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.921+0000 7f3c25fa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c200ccbd0 con 0x7f3c20144580 2023-11-08T19:26:40.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c20144960 0x7f3c200cdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c20144960 0x7f3c200cdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55362/0 (socket says 172.21.15.119:55362) 2023-11-08T19:26:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 -- 172.21.15.119:0/4223381222 learned_addr learned my addr 172.21.15.119:0/4223381222 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1f7fe640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c20152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1ffff640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c2015f1d0 0x7f3c200c6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 -- 172.21.15.119:0/4223381222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 msgr2=0x7f3c20152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c20144580 0x7f3c20152660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 -- 172.21.15.119:0/4223381222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c2015f1d0 msgr2=0x7f3c200c6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:40.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c2015f1d0 0x7f3c200c6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:40.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 -- 172.21.15.119:0/4223381222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c100094d0 con 0x7f3c20144960 2023-11-08T19:26:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.922+0000 7f3c1effd640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3c20144960 0x7f3c200cdc70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3c14034040 tx=0x7f3c140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.923+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14032820 con 0x7f3c20144960 2023-11-08T19:26:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.923+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c14032980 con 0x7f3c20144960 2023-11-08T19:26:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.923+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c14032d30 con 0x7f3c20144960 2023-11-08T19:26:40.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.923+0000 7f3c25fa9640 1 -- 172.21.15.119:0/4223381222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c14009400 con 0x7f3c20144960 2023-11-08T19:26:40.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.923+0000 7f3c25fa9640 1 -- 172.21.15.119:0/4223381222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c200cd190 con 0x7f3c20144960 2023-11-08T19:26:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.924+0000 7f3c25fa9640 1 -- 172.21.15.119:0/4223381222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c200cd990 con 0x7f3c20144960 2023-11-08T19:26:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.925+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c1404e440 con 0x7f3c20144960 2023-11-08T19:26:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.926+0000 7f3c07fff640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c0c06ae70 0x7f3c0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.926+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6450+0+0 (secure 0 0 0) 0x7f3c140aba20 con 0x7f3c20144960 2023-11-08T19:26:40.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.926+0000 7f3c1f7fe640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c0c06ae70 0x7f3c0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.927+0000 7f3c1f7fe640 1 --2- 172.21.15.119:0/4223381222 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c0c06ae70 0x7f3c0c06d330 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c10009cf0 tx=0x7f3c10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:40.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:40.931+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c14076890 con 0x7f3c20144960 2023-11-08T19:26:41.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.157+0000 7f3c25fa9640 1 -- 172.21.15.119:0/4223381222 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"} v 0) v1 -- 0x7f3c200c4130 con 0x7f3c20144960 2023-11-08T19:26:41.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.157+0000 7f3c07fff640 1 -- 172.21.15.119:0/4223381222 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "snap-test-pool"}]=-95 pool(7) already has mon-managed snaps; can't attach pool to fs v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f3c14079eb0 con 0x7f3c20144960 2023-11-08T19:26:41.159 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOTSUP: pool(7) already has mon-managed snaps; can't attach pool to fs 2023-11-08T19:26:41.174 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:26:41.175 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool snap-test-pool rmsnap snap3 2023-11-08T19:26:41.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.278+0000 7fdc90245f40 1 Processor -- start 2023-11-08T19:26:41.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.278+0000 7fdc90245f40 1 -- start start 2023-11-08T19:26:41.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc90245f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de25840f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc90245f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de24f0cc0 0x560de2583010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc90245f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de2584da0 0x560de257a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc90245f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de245c340 con 0x560de24f0830 2023-11-08T19:26:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc90245f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de23f6a70 con 0x560de24f0cc0 2023-11-08T19:26:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc90245f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560de23f6710 con 0x560de2584da0 2023-11-08T19:26:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc8d5a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de25840f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc8d5a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de25840f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55386/0 (socket says 172.21.15.119:55386) 2023-11-08T19:26:41.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.279+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/3970345274 learned_addr learned my addr 172.21.15.119:0/3970345274 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8ddaa640 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de2584da0 0x560de257a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de2584da0 msgr2=0x560de257a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8d5a9640 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de2584da0 0x560de257a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8cda8640 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de24f0cc0 0x560de2583010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de24f0cc0 msgr2=0x560de2583010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8d5a9640 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de24f0cc0 0x560de2583010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/3970345274 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc88005de0 con 0x560de24f0830 2023-11-08T19:26:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8cda8640 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de24f0cc0 0x560de2583010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc8d5a9640 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de25840f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000a500 tx=0x7fdc8000a9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edaa34eccc50323 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc877fe640 1 -- 172.21.15.119:0/3970345274 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc800115f0 con 0x560de24f0830 2023-11-08T19:26:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc877fe640 1 -- 172.21.15.119:0/3970345274 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc80011750 con 0x560de24f0830 2023-11-08T19:26:41.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.280+0000 7fdc877fe640 1 -- 172.21.15.119:0/3970345274 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80011ae0 con 0x560de24f0830 2023-11-08T19:26:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 -- 172.21.15.119:0/3970345274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 msgr2=0x560de25840f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de25840f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdc8000a500 tx=0x7fdc8000a9c0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 -- 172.21.15.119:0/3970345274 shutdown_connections 2023-11-08T19:26:41.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de2584da0 0x560de257a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de24f0cc0 0x560de2583010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 --2- 172.21.15.119:0/3970345274 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de25840f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 -- 172.21.15.119:0/3970345274 >> 172.21.15.119:0/3970345274 conn(0x560de24aca30 msgr2=0x560de2582380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 -- 172.21.15.119:0/3970345274 shutdown_connections 2023-11-08T19:26:41.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.281+0000 7fdc90245f40 1 -- 172.21.15.119:0/3970345274 wait complete. 2023-11-08T19:26:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.282+0000 7fdc90245f40 1 Processor -- start 2023-11-08T19:26:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.282+0000 7fdc90245f40 1 -- start start 2023-11-08T19:26:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc90245f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de24f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc90245f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de24f0cc0 0x560de2575dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc90245f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de2584da0 0x560de257b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc90245f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de23ea520 con 0x560de24f0830 2023-11-08T19:26:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc90245f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560de2576310 con 0x560de2584da0 2023-11-08T19:26:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc90245f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560de249b0d0 con 0x560de24f0cc0 2023-11-08T19:26:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc8d5a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de24f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc8d5a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de24f2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55388/0 (socket says 172.21.15.119:55388) 2023-11-08T19:26:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.283+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/4121093311 learned_addr learned my addr 172.21.15.119:0/4121093311 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8ddaa640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de2584da0 0x560de257b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8cda8640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de24f0cc0 0x560de2575dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de24f0cc0 msgr2=0x560de2575dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8d5a9640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de24f0cc0 0x560de2575dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de2584da0 msgr2=0x560de257b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8d5a9640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de2584da0 0x560de257b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8d5a9640 1 -- 172.21.15.119:0/4121093311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8000d040 con 0x560de24f0830 2023-11-08T19:26:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc8d5a9640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de24f2330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x560de24ad210 tx=0x7fdc800033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc80009050 con 0x560de24f0830 2023-11-08T19:26:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc80011e30 con 0x560de24f0830 2023-11-08T19:26:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc800115f0 con 0x560de24f0830 2023-11-08T19:26:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.284+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560de2402680 con 0x560de24f0830 2023-11-08T19:26:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.285+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560de23f5830 con 0x560de24f0830 2023-11-08T19:26:41.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.287+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc8001c3f0 con 0x560de24f0830 2023-11-08T19:26:41.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.287+0000 7fdc867fc640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc70069820 0x7fdc7006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.288+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6450+0+0 (secure 0 0 0) 0x7fdc8008a290 con 0x560de24f0830 2023-11-08T19:26:41.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.288+0000 7fdc8cda8640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc70069820 0x7fdc7006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.288+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- pool_op(delete snap pool 7 tid 1 name snap3 v0) v4 -- 0x560de24662c0 con 0x560de24f0830 2023-11-08T19:26:41.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.288+0000 7fdc8cda8640 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc70069820 0x7fdc7006bce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x560de24dcc70 tx=0x7fdc78005790 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.725+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v42) v1 ==== 43+0+0 (secure 0 0 0) 0x7fdc800550e0 con 0x560de24f0830 2023-11-08T19:26:41.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.725+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=42}) v3 -- 0x7fdc700725d0 con 0x560de24f0830 2023-11-08T19:26:41.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.728+0000 7fdc867fc640 1 -- 172.21.15.119:0/4121093311 <== mon.0 v2:172.21.15.119:3300/0 7 ==== osd_map(42..42 src has 1..42) v4 ==== 643+0+0 (secure 0 0 0) 0x7fdc8008a0d0 con 0x560de24f0830 2023-11-08T19:26:41.730 INFO:teuthology.orchestra.run.smithi119.stdout:removed pool snap-test-pool snap snap3 2023-11-08T19:26:41.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc70069820 msgr2=0x7fdc7006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc70069820 0x7fdc7006bce0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x560de24dcc70 tx=0x7fdc78005790 comp rx=0 tx=0).stop 2023-11-08T19:26:41.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 msgr2=0x560de24f2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de24f2330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x560de24ad210 tx=0x7fdc800033a0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 shutdown_connections 2023-11-08T19:26:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc70069820 0x7fdc7006bce0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560de2584da0 0x560de257b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560de24f0cc0 0x560de2575dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 --2- 172.21.15.119:0/4121093311 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560de24f0830 0x560de24f2330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.729+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 >> 172.21.15.119:0/4121093311 conn(0x560de24aca30 msgr2=0x560de24b2410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.730+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 shutdown_connections 2023-11-08T19:26:41.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.730+0000 7fdc90245f40 1 -- 172.21.15.119:0/4121093311 wait complete. 2023-11-08T19:26:41.737 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete snap-test-pool 2023-11-08T19:26:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.937+0000 7fb47d3c9640 1 Processor -- start 2023-11-08T19:26:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.938+0000 7fb47d3c9640 1 -- start start 2023-11-08T19:26:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.939+0000 7fb47d3c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4781596b0 0x7fb47806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb47d3c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb47806ba30 0x7fb47806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb47d3c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4780ca370 con 0x7fb47806fe70 2023-11-08T19:26:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb47d3c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47806cbb0 con 0x7fb47806ba30 2023-11-08T19:26:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb47d3c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb478159a90 con 0x7fb4781596b0 2023-11-08T19:26:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb46e7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb47806ba30 0x7fb47806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb46e7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb47806ba30 0x7fb47806f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55400/0 (socket says 172.21.15.119:55400) 2023-11-08T19:26:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb4777fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb47806fe70 0x7fb47815d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58300/0 (socket says 172.21.15.119:58300) 2023-11-08T19:26:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb46e7fc640 1 -- 172.21.15.119:0/215629414 learned_addr learned my addr 172.21.15.119:0/215629414 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb476ffd640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4781596b0 0x7fb47806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb46e7fc640 1 -- 172.21.15.119:0/215629414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4781596b0 msgr2=0x7fb47806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb46e7fc640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4781596b0 0x7fb47806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb46e7fc640 1 -- 172.21.15.119:0/215629414 >> v1:172.21.15.119:6789/0 conn(0x7fb47806fe70 legacy=0x7fb47815d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.941+0000 7fb46e7fc640 1 -- 172.21.15.119:0/215629414 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47806bf30 con 0x7fb47806ba30 2023-11-08T19:26:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.940+0000 7fb4767fc640 1 -- 172.21.15.119:0/215629414 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2733926268 0 0) 0x7fb4780ca370 con 0x7fb47806fe70 2023-11-08T19:26:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.941+0000 7fb476ffd640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4781596b0 0x7fb47806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.942+0000 7fb46e7fc640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806ba30 0x7fb47806f930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb460009870 tx=0x7fb46002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20edce8b356f7a41 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.942+0000 7fb4767fc640 1 -- 172.21.15.119:0/215629414 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030840 con 0x7fb47806ba30 2023-11-08T19:26:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.942+0000 7fb4767fc640 1 -- 172.21.15.119:0/215629414 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4600309a0 con 0x7fb47806ba30 2023-11-08T19:26:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.942+0000 7fb4767fc640 1 -- 172.21.15.119:0/215629414 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030d30 con 0x7fb47806ba30 2023-11-08T19:26:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.942+0000 7fb47d3c9640 1 -- 172.21.15.119:0/215629414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806ba30 msgr2=0x7fb47806f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.942+0000 7fb47d3c9640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806ba30 0x7fb47806f930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb460009870 tx=0x7fb46002b980 comp rx=0 tx=0).stop 2023-11-08T19:26:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.943+0000 7fb47d3c9640 1 -- 172.21.15.119:0/215629414 shutdown_connections 2023-11-08T19:26:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.943+0000 7fb47d3c9640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806ba30 0x7fb47806f930 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.943+0000 7fb47d3c9640 1 --2- 172.21.15.119:0/215629414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4781596b0 0x7fb47806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.943+0000 7fb47d3c9640 1 -- 172.21.15.119:0/215629414 >> 172.21.15.119:0/215629414 conn(0x7fb4780cb610 msgr2=0x7fb4780cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.943+0000 7fb47d3c9640 1 -- 172.21.15.119:0/215629414 shutdown_connections 2023-11-08T19:26:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.943+0000 7fb47d3c9640 1 -- 172.21.15.119:0/215629414 wait complete. 2023-11-08T19:26:41.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.944+0000 7fb47d3c9640 1 Processor -- start 2023-11-08T19:26:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.944+0000 7fb47d3c9640 1 -- start start 2023-11-08T19:26:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.945+0000 7fb47d3c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb47806ba30 0x7fb4781f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.945+0000 7fb47d3c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806fe70 0x7fb478146ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.945+0000 7fb47d3c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4781596b0 0x7fb478147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.945+0000 7fb47d3c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781f33c0 con 0x7fb47806fe70 2023-11-08T19:26:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.945+0000 7fb47d3c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478158ce0 con 0x7fb4781596b0 2023-11-08T19:26:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.945+0000 7fb47d3c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4781f35f0 con 0x7fb47806ba30 2023-11-08T19:26:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806fe70 0x7fb478146ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806fe70 0x7fb478146ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55414/0 (socket says 172.21.15.119:55414) 2023-11-08T19:26:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 -- 172.21.15.119:0/2709526728 learned_addr learned my addr 172.21.15.119:0/2709526728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb4777fe640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4781596b0 0x7fb478147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 -- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb47806ba30 msgr2=0x7fb4781f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb476ffd640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb47806ba30 0x7fb4781f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb47806ba30 0x7fb4781f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 -- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4781596b0 msgr2=0x7fb478147720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4781596b0 0x7fb478147720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 -- 172.21.15.119:0/2709526728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600094d0 con 0x7fb47806fe70 2023-11-08T19:26:41.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb4777fe640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4781596b0 0x7fb478147720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.946+0000 7fb46e7fc640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb47806fe70 0x7fb478146ca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb460000c00 tx=0x7fb460004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.947+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460030810 con 0x7fb47806fe70 2023-11-08T19:26:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.947+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460043400 con 0x7fb47806fe70 2023-11-08T19:26:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.947+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4600437d0 con 0x7fb47806fe70 2023-11-08T19:26:41.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.947+0000 7fb47d3c9640 1 -- 172.21.15.119:0/2709526728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4781f38d0 con 0x7fb47806fe70 2023-11-08T19:26:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.947+0000 7fb47d3c9640 1 -- 172.21.15.119:0/2709526728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4780aca40 con 0x7fb47806fe70 2023-11-08T19:26:41.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.948+0000 7fb47d3c9640 1 -- 172.21.15.119:0/2709526728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4781f3df0 con 0x7fb47806fe70 2023-11-08T19:26:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.949+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb460043930 con 0x7fb47806fe70 2023-11-08T19:26:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.950+0000 7fb46ffff640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb45806ae70 0x7fb45806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.950+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fb46003e070 con 0x7fb47806fe70 2023-11-08T19:26:41.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.950+0000 7fb476ffd640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb45806ae70 0x7fb45806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:41.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.950+0000 7fb476ffd640 1 --2- 172.21.15.119:0/2709526728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb45806ae70 0x7fb45806d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb46800b840 tx=0x7fb468008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:41.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:41.954+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb460077940 con 0x7fb47806fe70 2023-11-08T19:26:42.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.148+0000 7fb47d3c9640 1 -- 172.21.15.119:0/2709526728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "snap-test-pool"} v 0) v1 -- 0x7fb478074430 con 0x7fb47806fe70 2023-11-08T19:26:42.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.149+0000 7fb46ffff640 1 -- 172.21.15.119:0/2709526728 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "snap-test-pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v42) v1 ==== 296+0+0 (secure 0 0 0) 0x7fb4600340a0 con 0x7fb47806fe70 2023-11-08T19:26:42.151 INFO:teuthology.orchestra.run.smithi119.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool snap-test-pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-08T19:26:42.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:26:42.162 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:26:42.162 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:42.271 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:42.401 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:26:42.402 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:42.402 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:42.461 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:42.462 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:42.516 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.731+0000 7ff24764f640 1 Processor -- start 2023-11-08T19:26:42.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.731+0000 7ff24764f640 1 -- start start 2023-11-08T19:26:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff24764f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24015d410 0x7ff2400ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff24764f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff24015e5f0 0x7ff24015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff24764f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24006b570 con 0x7ff24015dd30 2023-11-08T19:26:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff24764f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24006c1f0 con 0x7ff24015d410 2023-11-08T19:26:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff24764f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff24006b850 con 0x7ff24015e5f0 2023-11-08T19:26:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff2453eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24015d410 0x7ff2400ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff2453eb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24015d410 0x7ff2400ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55416/0 (socket says 172.21.15.119:55416) 2023-11-08T19:26:42.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.733+0000 7ff2453eb640 1 -- 172.21.15.119:0/1463085990 learned_addr learned my addr 172.21.15.119:0/1463085990 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:42.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.734+0000 7ff244bea640 1 --2- 172.21.15.119:0/1463085990 >> v2:172.21.15.188:3300/0 conn(0x7ff24015e5f0 0x7ff24015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.734+0000 7ff2453eb640 1 -- 172.21.15.119:0/1463085990 >> v2:172.21.15.188:3300/0 conn(0x7ff24015e5f0 msgr2=0x7ff24015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:42.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.734+0000 7ff2453eb640 1 --2- 172.21.15.119:0/1463085990 >> v2:172.21.15.188:3300/0 conn(0x7ff24015e5f0 0x7ff24015d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.734+0000 7ff2453eb640 1 -- 172.21.15.119:0/1463085990 >> v1:172.21.15.119:6789/0 conn(0x7ff24015dd30 legacy=0x7ff240160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.734+0000 7ff2453eb640 1 -- 172.21.15.119:0/1463085990 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24006bb30 con 0x7ff24015d410 2023-11-08T19:26:42.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.735+0000 7ff2453eb640 1 --2- 172.21.15.119:0/1463085990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015d410 0x7ff2400ce040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff240070060 tx=0x7ff23c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697ecab3606226d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.738+0000 7ff237fff640 1 -- 172.21.15.119:0/1463085990 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c004230 con 0x7ff24015d410 2023-11-08T19:26:42.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.738+0000 7ff237fff640 1 -- 172.21.15.119:0/1463085990 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c004390 con 0x7ff24015d410 2023-11-08T19:26:42.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.738+0000 7ff237fff640 1 -- 172.21.15.119:0/1463085990 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c003810 con 0x7ff24015d410 2023-11-08T19:26:42.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 -- 172.21.15.119:0/1463085990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015d410 msgr2=0x7ff2400ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 --2- 172.21.15.119:0/1463085990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015d410 0x7ff2400ce040 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff240070060 tx=0x7ff23c02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 -- 172.21.15.119:0/1463085990 shutdown_connections 2023-11-08T19:26:42.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 --2- 172.21.15.119:0/1463085990 >> v2:172.21.15.188:3300/0 conn(0x7ff24015e5f0 0x7ff24015d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 --2- 172.21.15.119:0/1463085990 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015d410 0x7ff2400ce040 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 -- 172.21.15.119:0/1463085990 >> 172.21.15.119:0/1463085990 conn(0x7ff2400d0db0 msgr2=0x7ff2400d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 -- 172.21.15.119:0/1463085990 shutdown_connections 2023-11-08T19:26:42.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.739+0000 7ff24764f640 1 -- 172.21.15.119:0/1463085990 wait complete. 2023-11-08T19:26:42.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.741+0000 7ff24764f640 1 Processor -- start 2023-11-08T19:26:42.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.741+0000 7ff24764f640 1 -- start start 2023-11-08T19:26:42.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff24764f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d410 0x7ff240148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff24764f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 0x7ff240159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff24764f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff24015e5f0 0x7ff2400ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff24764f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240155860 con 0x7ff24015dd30 2023-11-08T19:26:42.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff24764f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400c9450 con 0x7ff24015e5f0 2023-11-08T19:26:42.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff24764f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400c9730 con 0x7ff24015d410 2023-11-08T19:26:42.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.742+0000 7ff244bea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 0x7ff240159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 0x7ff240159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55428/0 (socket says 172.21.15.119:55428) 2023-11-08T19:26:42.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 -- 172.21.15.119:0/2924397112 learned_addr learned my addr 172.21.15.119:0/2924397112 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:42.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff2453eb640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d410 0x7ff240148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff245bec640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff24015e5f0 0x7ff2400ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 -- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d410 msgr2=0x7ff240148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d410 0x7ff240148150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 -- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff24015e5f0 msgr2=0x7ff2400ca4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff24015e5f0 0x7ff2400ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 -- 172.21.15.119:0/2924397112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c009460 con 0x7ff24015dd30 2023-11-08T19:26:42.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.743+0000 7ff244bea640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 0x7ff240159cf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff23000d800 tx=0x7ff23000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.744+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000b6c0 con 0x7ff24015dd30 2023-11-08T19:26:42.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.744+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23000b820 con 0x7ff24015dd30 2023-11-08T19:26:42.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.744+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23000bbb0 con 0x7ff24015dd30 2023-11-08T19:26:42.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.744+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2400c9a10 con 0x7ff24015dd30 2023-11-08T19:26:42.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.744+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2400c9fd0 con 0x7ff24015dd30 2023-11-08T19:26:42.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.745+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2400ca160 con 0x7ff24015dd30 2023-11-08T19:26:42.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.746+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff23000bd10 con 0x7ff24015dd30 2023-11-08T19:26:42.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.747+0000 7ff235ffb640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:42.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.747+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7ff230089f50 con 0x7ff24015dd30 2023-11-08T19:26:42.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.747+0000 7ff2453eb640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:42.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.748+0000 7ff2453eb640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff23c035000 tx=0x7ff23c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:42.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.751+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff230054c60 con 0x7ff24015dd30 2023-11-08T19:26:42.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.945+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff240074430 con 0x7ff24015dd30 2023-11-08T19:26:42.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.947+0000 7ff235ffb640 1 -- 172.21.15.119:0/2924397112 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18606 (secure 0 0 0) 0x7ff230058280 con 0x7ff24015dd30 2023-11-08T19:26:42.949 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:42.949 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":42,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:41.714462+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:42.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 msgr2=0x7ff22406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff23c035000 tx=0x7ff23c008030 comp rx=0 tx=0).stop 2023-11-08T19:26:42.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 msgr2=0x7ff240159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:42.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 0x7ff240159cf0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff23000d800 tx=0x7ff23000dcc0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 shutdown_connections 2023-11-08T19:26:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff24015e5f0 0x7ff2400ca4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24015dd30 0x7ff240159cf0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 --2- 172.21.15.119:0/2924397112 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d410 0x7ff240148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:42.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 >> 172.21.15.119:0/2924397112 conn(0x7ff2400d0db0 msgr2=0x7ff2400ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:42.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 shutdown_connections 2023-11-08T19:26:42.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:42.950+0000 7ff24764f640 1 -- 172.21.15.119:0/2924397112 wait complete. 2023-11-08T19:26:42.969 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:26:27.536718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:26:28.733331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:42.970 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:43.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.175+0000 7f74f330c640 1 Processor -- start 2023-11-08T19:26:43.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.176+0000 7f74f330c640 1 -- start start 2023-11-08T19:26:43.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.177+0000 7f74f330c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec0cfa00 0x7f74ec0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.177+0000 7f74f330c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74ec161160 0x7f74ec161590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.177+0000 7f74f330c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec15df50 con 0x7f74ec15d4a0 2023-11-08T19:26:43.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.177+0000 7f74f330c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec0d1e60 con 0x7f74ec161160 2023-11-08T19:26:43.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.177+0000 7f74f330c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0d0380 con 0x7f74ec0cfa00 2023-11-08T19:26:43.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.178+0000 7f74f18a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74ec161160 0x7f74ec161590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.178+0000 7f74f08a7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec0cfa00 0x7f74ec0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.178+0000 7f74f18a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f74ec161160 0x7f74ec161590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56494/0 (socket says 172.21.15.119:56494) 2023-11-08T19:26:43.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.178+0000 7f74f18a9640 1 -- 172.21.15.119:0/39441024 learned_addr learned my addr 172.21.15.119:0/39441024 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:43.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.179+0000 7f74f18a9640 1 -- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec0cfa00 msgr2=0x7f74ec0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.179+0000 7f74f18a9640 1 --2- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec0cfa00 0x7f74ec0cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.179+0000 7f74f18a9640 1 -- 172.21.15.119:0/39441024 >> v1:172.21.15.188:6789/0 conn(0x7f74ec15d4a0 legacy=0x7f74ec0d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.179+0000 7f74f18a9640 1 -- 172.21.15.119:0/39441024 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec161ad0 con 0x7f74ec161160 2023-11-08T19:26:43.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.179+0000 7f74f08a7640 1 --2- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec0cfa00 0x7f74ec0cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:26:43.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.180+0000 7f74f18a9640 1 --2- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec161160 0x7f74ec161590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009870 tx=0x7f74e402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7ee469a7904319b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.180+0000 7f74e3fff640 1 -- 172.21.15.119:0/39441024 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4031640 con 0x7f74ec161160 2023-11-08T19:26:43.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.180+0000 7f74e3fff640 1 -- 172.21.15.119:0/39441024 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e40317a0 con 0x7f74ec161160 2023-11-08T19:26:43.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.180+0000 7f74e3fff640 1 -- 172.21.15.119:0/39441024 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e4031b30 con 0x7f74ec161160 2023-11-08T19:26:43.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 -- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec161160 msgr2=0x7f74ec161590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 --2- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec161160 0x7f74ec161590 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f74e4009870 tx=0x7f74e402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 -- 172.21.15.119:0/39441024 shutdown_connections 2023-11-08T19:26:43.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 --2- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec161160 0x7f74ec161590 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 --2- 172.21.15.119:0/39441024 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec0cfa00 0x7f74ec0cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 -- 172.21.15.119:0/39441024 >> 172.21.15.119:0/39441024 conn(0x7f74ec06a9f0 msgr2=0x7f74ec06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 -- 172.21.15.119:0/39441024 shutdown_connections 2023-11-08T19:26:43.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.181+0000 7f74f330c640 1 -- 172.21.15.119:0/39441024 wait complete. 2023-11-08T19:26:43.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.183+0000 7f74f330c640 1 Processor -- start 2023-11-08T19:26:43.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.184+0000 7f74f330c640 1 -- start start 2023-11-08T19:26:43.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f330c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 0x7f74ec0c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f330c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec15d4a0 0x7f74ec0cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f330c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec161160 0x7f74ec156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f330c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0c7400 con 0x7f74ec0cfa00 2023-11-08T19:26:43.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f330c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0c5b20 con 0x7f74ec15d4a0 2023-11-08T19:26:43.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f330c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec0cc060 con 0x7f74ec161160 2023-11-08T19:26:43.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f10a8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 0x7f74ec0c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f10a8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 0x7f74ec0c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55436/0 (socket says 172.21.15.119:55436) 2023-11-08T19:26:43.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.185+0000 7f74f10a8640 1 -- 172.21.15.119:0/1770284279 learned_addr learned my addr 172.21.15.119:0/1770284279 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:43.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f08a7640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec15d4a0 0x7f74ec0cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f18a9640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec161160 0x7f74ec156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f10a8640 1 -- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec161160 msgr2=0x7f74ec156600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f10a8640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec161160 0x7f74ec156600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f10a8640 1 -- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec15d4a0 msgr2=0x7f74ec0cbb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f10a8640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec15d4a0 0x7f74ec0cbb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f10a8640 1 -- 172.21.15.119:0/1770284279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e8009a70 con 0x7f74ec0cfa00 2023-11-08T19:26:43.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f18a9640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec161160 0x7f74ec156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:43.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f10a8640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 0x7f74ec0c55e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00c430 tx=0x7f74dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0052d0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc005450 con 0x7f74ec0cfa00 2023-11-08T19:26:43.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc007260 con 0x7f74ec0cfa00 2023-11-08T19:26:43.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.186+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e40094d0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.187+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec0cc5c0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.188+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ec074430 con 0x7f74ec0cfa00 2023-11-08T19:26:43.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.189+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74dc0073c0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.190+0000 7f74e1ffb640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f74d806ae70 0x7f74d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.190+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f74dc088dd0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.190+0000 7f74f08a7640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f74d806ae70 0x7f74d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.191+0000 7f74f08a7640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f74d806ae70 0x7f74d806d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0b7080 tx=0x7f74e800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.194+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74dc053af0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.384+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74ec0d10f0 con 0x7f74ec0cfa00 2023-11-08T19:26:43.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.384+0000 7f74e1ffb640 1 -- 172.21.15.119:0/1770284279 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f74dc08a020 con 0x7f74ec0cfa00 2023-11-08T19:26:43.386 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f74d806ae70 msgr2=0x7f74d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f74d806ae70 0x7f74d806d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f74ec0b7080 tx=0x7f74e800b040 comp rx=0 tx=0).stop 2023-11-08T19:26:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 msgr2=0x7f74ec0c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 0x7f74ec0c55e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00c430 tx=0x7f74dc00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 shutdown_connections 2023-11-08T19:26:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f74d806ae70 0x7f74d806d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f74ec161160 0x7f74ec156600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74ec15d4a0 0x7f74ec0cbb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 --2- 172.21.15.119:0/1770284279 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f74ec0cfa00 0x7f74ec0c55e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 >> 172.21.15.119:0/1770284279 conn(0x7f74ec06a9f0 msgr2=0x7f74ec06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 shutdown_connections 2023-11-08T19:26:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.387+0000 7f74f330c640 1 -- 172.21.15.119:0/1770284279 wait complete. 2023-11-08T19:26:43.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.624+0000 7fc70b261640 1 Processor -- start 2023-11-08T19:26:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.625+0000 7fc70b261640 1 -- start start 2023-11-08T19:26:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.626+0000 7fc70b261640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc704144580 0x7fc70415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.626+0000 7fc70b261640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 0x7fc7040c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.626+0000 7fc70b261640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc70415ea60 con 0x7fc704144960 2023-11-08T19:26:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.626+0000 7fc70b261640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7040c9150 con 0x7fc704144580 2023-11-08T19:26:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.626+0000 7fc70b261640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c9360 con 0x7fc70415f1d0 2023-11-08T19:26:43.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.627+0000 7fc7097fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 0x7fc7040c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.627+0000 7fc7097fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 0x7fc7040c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:53880/0 (socket says 172.21.15.119:53880) 2023-11-08T19:26:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.627+0000 7fc7097fe640 1 -- 172.21.15.119:0/3763624232 learned_addr learned my addr 172.21.15.119:0/3763624232 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.627+0000 7fc708ffd640 1 --2- 172.21.15.119:0/3763624232 >> v2:172.21.15.188:3300/0 conn(0x7fc704144580 0x7fc70415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.628+0000 7fc7097fe640 1 -- 172.21.15.119:0/3763624232 >> v2:172.21.15.188:3300/0 conn(0x7fc704144580 msgr2=0x7fc70415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.628+0000 7fc7097fe640 1 --2- 172.21.15.119:0/3763624232 >> v2:172.21.15.188:3300/0 conn(0x7fc704144580 0x7fc70415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.628+0000 7fc7097fe640 1 -- 172.21.15.119:0/3763624232 >> v1:172.21.15.188:6789/0 conn(0x7fc704144960 legacy=0x7fc7040cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.628+0000 7fc7097fe640 1 -- 172.21.15.119:0/3763624232 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7040c9640 con 0x7fc70415f1d0 2023-11-08T19:26:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.629+0000 7fc7097fe640 1 --2- 172.21.15.119:0/3763624232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 0x7fc7040c3a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7040c9290 tx=0x7fc6fc02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a17be64d173a98 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.630+0000 7fc7037fe640 1 -- 172.21.15.119:0/3763624232 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc031640 con 0x7fc70415f1d0 2023-11-08T19:26:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.630+0000 7fc7037fe640 1 -- 172.21.15.119:0/3763624232 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6fc0317a0 con 0x7fc70415f1d0 2023-11-08T19:26:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.630+0000 7fc7037fe640 1 -- 172.21.15.119:0/3763624232 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc031b30 con 0x7fc70415f1d0 2023-11-08T19:26:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 -- 172.21.15.119:0/3763624232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 msgr2=0x7fc7040c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 --2- 172.21.15.119:0/3763624232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 0x7fc7040c3a50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc7040c9290 tx=0x7fc6fc02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 -- 172.21.15.119:0/3763624232 shutdown_connections 2023-11-08T19:26:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 --2- 172.21.15.119:0/3763624232 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc70415f1d0 0x7fc7040c3a50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 --2- 172.21.15.119:0/3763624232 >> v2:172.21.15.188:3300/0 conn(0x7fc704144580 0x7fc70415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 -- 172.21.15.119:0/3763624232 >> 172.21.15.119:0/3763624232 conn(0x7fc7040cb610 msgr2=0x7fc7040cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 -- 172.21.15.119:0/3763624232 shutdown_connections 2023-11-08T19:26:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.631+0000 7fc70b261640 1 -- 172.21.15.119:0/3763624232 wait complete. 2023-11-08T19:26:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.633+0000 7fc70b261640 1 Processor -- start 2023-11-08T19:26:43.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.634+0000 7fc70b261640 1 -- start start 2023-11-08T19:26:43.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc70b261640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 0x7fc70415a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc70b261640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc704144960 0x7fc7040c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc70b261640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc70415f1d0 0x7fc704140d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc70b261640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c7220 con 0x7fc704144580 2023-11-08T19:26:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc70b261640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70415aa80 con 0x7fc70415f1d0 2023-11-08T19:26:43.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc70b261640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7040c74c0 con 0x7fc704144960 2023-11-08T19:26:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc708ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 0x7fc70415a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc708ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 0x7fc70415a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55444/0 (socket says 172.21.15.119:55444) 2023-11-08T19:26:43.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.635+0000 7fc708ffd640 1 -- 172.21.15.119:0/3455447444 learned_addr learned my addr 172.21.15.119:0/3455447444 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc703fff640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc704144960 0x7fc7040c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc7097fe640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc70415f1d0 0x7fc704140d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc708ffd640 1 -- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc704144960 msgr2=0x7fc7040c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc708ffd640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc704144960 0x7fc7040c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc708ffd640 1 -- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc70415f1d0 msgr2=0x7fc704140d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc708ffd640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc70415f1d0 0x7fc704140d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc708ffd640 1 -- 172.21.15.119:0/3455447444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0094d0 con 0x7fc704144580 2023-11-08T19:26:43.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc708ffd640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 0x7fc70415a540 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009db0 tx=0x7fc6f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4014480 con 0x7fc704144580 2023-11-08T19:26:43.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f400f040 con 0x7fc704144580 2023-11-08T19:26:43.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.636+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7040c6420 con 0x7fc704144580 2023-11-08T19:26:43.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.637+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4014800 con 0x7fc704144580 2023-11-08T19:26:43.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.637+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7040c69e0 con 0x7fc704144580 2023-11-08T19:26:43.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.638+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc704074430 con 0x7fc704144580 2023-11-08T19:26:43.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.639+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6f4014960 con 0x7fc704144580 2023-11-08T19:26:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.640+0000 7fc7017fa640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc6f006ae70 0x7fc6f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:43.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.640+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fc6f408a7c0 con 0x7fc704144580 2023-11-08T19:26:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.640+0000 7fc703fff640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc6f006ae70 0x7fc6f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:43.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.641+0000 7fc703fff640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc6f006ae70 0x7fc6f006d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7040c48d0 tx=0x7fc6f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:43.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.644+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6f4055570 con 0x7fc704144580 2023-11-08T19:26:43.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.832+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc70406cb10 con 0x7fc704144580 2023-11-08T19:26:43.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.833+0000 7fc7017fa640 1 -- 172.21.15.119:0/3455447444 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6f4058b90 con 0x7fc704144580 2023-11-08T19:26:43.835 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc6f006ae70 msgr2=0x7fc6f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc6f006ae70 0x7fc6f006d330 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc7040c48d0 tx=0x7fc6f8009040 comp rx=0 tx=0).stop 2023-11-08T19:26:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 msgr2=0x7fc70415a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 0x7fc70415a540 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009db0 tx=0x7fc6f400c490 comp rx=0 tx=0).stop 2023-11-08T19:26:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 shutdown_connections 2023-11-08T19:26:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc6f006ae70 0x7fc6f006d330 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc70415f1d0 0x7fc704140d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc704144960 0x7fc7040c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 --2- 172.21.15.119:0/3455447444 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc704144580 0x7fc70415a540 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:43.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.835+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 >> 172.21.15.119:0/3455447444 conn(0x7fc7040cb610 msgr2=0x7fc70415dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:43.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.836+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 shutdown_connections 2023-11-08T19:26:43.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:43.836+0000 7fc70b261640 1 -- 172.21.15.119:0/3455447444 wait complete. 2023-11-08T19:26:43.856 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.052+0000 7f3cba8bb640 1 Processor -- start 2023-11-08T19:26:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.052+0000 7f3cba8bb640 1 -- start start 2023-11-08T19:26:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.053+0000 7f3cba8bb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3cb415e9a0 0x7f3cb40cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.053+0000 7f3cba8bb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb4071720 con 0x7f3cb406b5e0 2023-11-08T19:26:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.053+0000 7f3cba8bb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb40cf5e0 con 0x7f3cb415deb0 2023-11-08T19:26:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.053+0000 7f3cba8bb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb40cf8c0 con 0x7f3cb415e9a0 2023-11-08T19:26:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cba0ba640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3cb406b5e0 0x7f3cb406ba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58316/0 (socket says 172.21.15.119:58316) 2023-11-08T19:26:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cba0ba640 1 -- 172.21.15.119:0/1078083620 learned_addr learned my addr 172.21.15.119:0/1078083620 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:44.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb90b8640 1 --2- 172.21.15.119:0/1078083620 >> v2:172.21.15.188:3300/0 conn(0x7f3cb415e9a0 0x7f3cb40cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588509902 0 0) 0x7f3cb4071720 con 0x7f3cb406b5e0 2023-11-08T19:26:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ca4005730 con 0x7f3cb406b5e0 2023-11-08T19:26:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2943506143 0 0) 0x7f3ca4005730 con 0x7f3cb406b5e0 2023-11-08T19:26:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 >> v2:172.21.15.188:3300/0 conn(0x7f3cb415e9a0 msgr2=0x7f3cb40cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 --2- 172.21.15.119:0/1078083620 >> v2:172.21.15.188:3300/0 conn(0x7f3cb415e9a0 0x7f3cb40cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 >> v1:172.21.15.188:6789/0 conn(0x7f3cb415deb0 legacy=0x7f3cb415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40710d0 con 0x7f3cb406b5e0 2023-11-08T19:26:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f3cac001670 con 0x7f3cb406b5e0 2023-11-08T19:26:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.054+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 >> v1:172.21.15.119:6789/0 conn(0x7f3cb406b5e0 legacy=0x7f3cb406ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cb88b7640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca4005b80 0x7f3ca4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cb88b7640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca40069d0 0x7f3ca4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cb88b7640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3ca400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40cf8c0 con 0x7f3ca4007390 2023-11-08T19:26:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca400e330 con 0x7f3ca40069d0 2023-11-08T19:26:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca400e4f0 con 0x7f3ca4005b80 2023-11-08T19:26:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cba0ba640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3ca400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cba0ba640 1 -- 172.21.15.119:0/1078083620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca4005b80 msgr2=0x7f3ca4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cba0ba640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ca4005b80 0x7f3ca4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cba0ba640 1 -- 172.21.15.119:0/1078083620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca40069d0 msgr2=0x7f3ca4006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cba0ba640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ca40069d0 0x7f3ca4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.055+0000 7f3cba0ba640 1 -- 172.21.15.119:0/1078083620 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca4005730 con 0x7f3ca4007390 2023-11-08T19:26:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.056+0000 7f3cba0ba640 1 -- 172.21.15.119:0/1078083620 reap_dead start 2023-11-08T19:26:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.056+0000 7f3cba0ba640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3ca400db90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c240 tx=0x7f3ca4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60658084ce1c9eec server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac005ad0 con 0x7f3ca4007390 2023-11-08T19:26:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac005c70 con 0x7f3ca4007390 2023-11-08T19:26:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cb88b7640 1 -- 172.21.15.119:0/1078083620 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac02c8b0 con 0x7f3ca4007390 2023-11-08T19:26:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 -- 172.21.15.119:0/1078083620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 msgr2=0x7f3ca400db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3ca400db90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3cac00c240 tx=0x7f3ca4007a00 comp rx=0 tx=0).stop 2023-11-08T19:26:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 -- 172.21.15.119:0/1078083620 shutdown_connections 2023-11-08T19:26:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/1078083620 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3ca400db90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 -- 172.21.15.119:0/1078083620 >> 172.21.15.119:0/1078083620 conn(0x7f3cb40ceac0 msgr2=0x7f3cb40ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 -- 172.21.15.119:0/1078083620 shutdown_connections 2023-11-08T19:26:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.057+0000 7f3cba8bb640 1 -- 172.21.15.119:0/1078083620 wait complete. 2023-11-08T19:26:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.059+0000 7f3cba8bb640 1 Processor -- start 2023-11-08T19:26:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.059+0000 7f3cba8bb640 1 -- start start 2023-11-08T19:26:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cba8bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3cb40c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cba8bb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb4157110 0x7f3cb414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cba8bb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cb40c2f00 0x7f3cb40c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cba8bb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c75e0 con 0x7f3ca4007390 2023-11-08T19:26:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cba8bb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c38c0 con 0x7f3cb4157110 2023-11-08T19:26:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cba8bb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c5450 con 0x7f3cb40c2f00 2023-11-08T19:26:44.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cb98b9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3cb40c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cb98b9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3cb40c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55458/0 (socket says 172.21.15.119:55458) 2023-11-08T19:26:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.060+0000 7f3cb98b9640 1 -- 172.21.15.119:0/3145662518 learned_addr learned my addr 172.21.15.119:0/3145662518 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cba0ba640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cb40c2f00 0x7f3cb40c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb90b8640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb4157110 0x7f3cb414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb98b9640 1 -- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cb40c2f00 msgr2=0x7f3cb40c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb98b9640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cb40c2f00 0x7f3cb40c3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb98b9640 1 -- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb4157110 msgr2=0x7f3cb414bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb98b9640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb4157110 0x7f3cb414bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb98b9640 1 -- 172.21.15.119:0/3145662518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40c6c40 con 0x7f3ca4007390 2023-11-08T19:26:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb98b9640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3cb40c4e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800c840 tx=0x7f3ca800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca8003b70 con 0x7f3ca4007390 2023-11-08T19:26:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca8003cf0 con 0x7f3ca4007390 2023-11-08T19:26:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800aa90 con 0x7f3ca4007390 2023-11-08T19:26:44.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.061+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb40c6f20 con 0x7f3ca4007390 2023-11-08T19:26:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.062+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb40b1510 con 0x7f3ca4007390 2023-11-08T19:26:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.062+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb404ece0 con 0x7f3ca4007390 2023-11-08T19:26:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.063+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ca800abf0 con 0x7f3ca4007390 2023-11-08T19:26:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.064+0000 7f3cb27fc640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c9806ae70 0x7f3c9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.064+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f3ca8089830 con 0x7f3ca4007390 2023-11-08T19:26:44.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.065+0000 7f3cb90b8640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c9806ae70 0x7f3c9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.065+0000 7f3cb90b8640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c9806ae70 0x7f3c9806d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40c56a0 tx=0x7f3ca0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.069+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ca80545e0 con 0x7f3ca4007390 2023-11-08T19:26:44.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.256+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3cb40cb1c0 con 0x7f3ca4007390 2023-11-08T19:26:44.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.257+0000 7f3cb27fc640 1 -- 172.21.15.119:0/3145662518 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ca8003e60 con 0x7f3ca4007390 2023-11-08T19:26:44.259 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:44.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c9806ae70 msgr2=0x7f3c9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c9806ae70 0x7f3c9806d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3cb40c56a0 tx=0x7f3ca0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:44.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 msgr2=0x7f3cb40c4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3cb40c4e00 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3ca800c840 tx=0x7f3ca800cd00 comp rx=0 tx=0).stop 2023-11-08T19:26:44.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 shutdown_connections 2023-11-08T19:26:44.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3c9806ae70 0x7f3c9806d330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3cb40c2f00 0x7f3cb40c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3cb4157110 0x7f3cb414bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 --2- 172.21.15.119:0/3145662518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3ca4007390 0x7f3cb40c4e00 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 >> 172.21.15.119:0/3145662518 conn(0x7f3cb40ceac0 msgr2=0x7f3cb406bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 shutdown_connections 2023-11-08T19:26:44.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.260+0000 7f3cba8bb640 1 -- 172.21.15.119:0/3145662518 wait complete. 2023-11-08T19:26:44.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:44.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.493+0000 7f7435e44640 1 Processor -- start 2023-11-08T19:26:44.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.493+0000 7f7435e44640 1 -- start start 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f7435e44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7430144960 0x7f743015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f7435e44640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f743015f1d0 0x7f74300c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f7435e44640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743015ea60 con 0x7f7430144580 2023-11-08T19:26:44.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f7435e44640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74300c9150 con 0x7f743015f1d0 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f7435e44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74300c9360 con 0x7f7430144960 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f742ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f743015f1d0 0x7f74300c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f742ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f743015f1d0 0x7f74300c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55462/0 (socket says 172.21.15.119:55462) 2023-11-08T19:26:44.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f742ffff640 1 -- 172.21.15.119:0/1057493032 learned_addr learned my addr 172.21.15.119:0/1057493032 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.495+0000 7f742effd640 1 --2- 172.21.15.119:0/1057493032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7430144960 0x7f743015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.496+0000 7f742ffff640 1 -- 172.21.15.119:0/1057493032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7430144960 msgr2=0x7f743015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.496+0000 7f742ffff640 1 --2- 172.21.15.119:0/1057493032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7430144960 0x7f743015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.496+0000 7f742ffff640 1 -- 172.21.15.119:0/1057493032 >> v1:172.21.15.188:6789/0 conn(0x7f7430144580 legacy=0x7f74300cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.496+0000 7f742ffff640 1 -- 172.21.15.119:0/1057493032 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74300c9640 con 0x7f743015f1d0 2023-11-08T19:26:44.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.497+0000 7f742ffff640 1 --2- 172.21.15.119:0/1057493032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f743015f1d0 0x7f74300c3a50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f74300c9290 tx=0x7f742802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6465e9932c3b3a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.497+0000 7f742e7fc640 1 -- 172.21.15.119:0/1057493032 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742802ca50 con 0x7f743015f1d0 2023-11-08T19:26:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.497+0000 7f742e7fc640 1 -- 172.21.15.119:0/1057493032 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742802cbb0 con 0x7f743015f1d0 2023-11-08T19:26:44.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.497+0000 7f742e7fc640 1 -- 172.21.15.119:0/1057493032 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74280343f0 con 0x7f743015f1d0 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 -- 172.21.15.119:0/1057493032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f743015f1d0 msgr2=0x7f74300c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 --2- 172.21.15.119:0/1057493032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f743015f1d0 0x7f74300c3a50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f74300c9290 tx=0x7f742802beb0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 -- 172.21.15.119:0/1057493032 shutdown_connections 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 --2- 172.21.15.119:0/1057493032 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f743015f1d0 0x7f74300c3a50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 --2- 172.21.15.119:0/1057493032 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7430144960 0x7f743015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 -- 172.21.15.119:0/1057493032 >> 172.21.15.119:0/1057493032 conn(0x7f74300cb610 msgr2=0x7f74300cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 -- 172.21.15.119:0/1057493032 shutdown_connections 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.498+0000 7f7435e44640 1 -- 172.21.15.119:0/1057493032 wait complete. 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.499+0000 7f7435e44640 1 Processor -- start 2023-11-08T19:26:44.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.500+0000 7f7435e44640 1 -- start start 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f7435e44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 0x7f74301f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f7435e44640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7430144960 0x7f74300c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f7435e44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f743015f1d0 0x7f74300c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f7435e44640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743013b170 con 0x7f7430144580 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f7435e44640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743013b3e0 con 0x7f7430144960 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f7435e44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743013b6c0 con 0x7f743015f1d0 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 0x7f74301f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 0x7f74301f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55468/0 (socket says 172.21.15.119:55468) 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 -- 172.21.15.119:0/3499819935 learned_addr learned my addr 172.21.15.119:0/3499819935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:44.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 -- 172.21.15.119:0/3499819935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f743015f1d0 msgr2=0x7f74300c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f743015f1d0 0x7f74300c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 -- 172.21.15.119:0/3499819935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7430144960 msgr2=0x7f74300c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7430144960 0x7f74300c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.501+0000 7f742f7fe640 1 -- 172.21.15.119:0/3499819935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7428009b30 con 0x7f7430144580 2023-11-08T19:26:44.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.502+0000 7f742f7fe640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 0x7f74301f02f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f742000c450 tx=0x7f742000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.502+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74200052d0 con 0x7f7430144580 2023-11-08T19:26:44.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.502+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7420005430 con 0x7f7430144580 2023-11-08T19:26:44.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.502+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7420012870 con 0x7f7430144580 2023-11-08T19:26:44.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.502+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743013b9a0 con 0x7f7430144580 2023-11-08T19:26:44.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.502+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743013cec0 con 0x7f7430144580 2023-11-08T19:26:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.503+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743013bec0 con 0x7f7430144580 2023-11-08T19:26:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.504+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74200129d0 con 0x7f7430144580 2023-11-08T19:26:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.505+0000 7f7417fff640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f741c06ae70 0x7f741c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.505+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f74200894f0 con 0x7f7430144580 2023-11-08T19:26:44.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.505+0000 7f742effd640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f741c06ae70 0x7f741c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.505+0000 7f742effd640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f741c06ae70 0x7f741c06d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f74300c4630 tx=0x7f7424008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.509+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f743013bec0 con 0x7f7430144580 2023-11-08T19:26:44.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.688+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f743013bec0 con 0x7f7430144580 2023-11-08T19:26:44.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.689+0000 7f7417fff640 1 -- 172.21.15.119:0/3499819935 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v42) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7420054200 con 0x7f7430144580 2023-11-08T19:26:44.690 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f741c06ae70 msgr2=0x7f741c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f741c06ae70 0x7f741c06d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f74300c4630 tx=0x7f7424008040 comp rx=0 tx=0).stop 2023-11-08T19:26:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 msgr2=0x7f74301f02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 0x7f74301f02f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f742000c450 tx=0x7f742000c910 comp rx=0 tx=0).stop 2023-11-08T19:26:44.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 shutdown_connections 2023-11-08T19:26:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f741c06ae70 0x7f741c06d330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f743015f1d0 0x7f74300c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7430144960 0x7f74300c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 --2- 172.21.15.119:0/3499819935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7430144580 0x7f74301f02f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 >> 172.21.15.119:0/3499819935 conn(0x7f74300cb610 msgr2=0x7f74300cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 shutdown_connections 2023-11-08T19:26:44.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.691+0000 7f7435e44640 1 -- 172.21.15.119:0/3499819935 wait complete. 2023-11-08T19:26:44.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:26:44.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.912+0000 7ffab5f41640 1 Processor -- start 2023-11-08T19:26:44.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.912+0000 7ffab5f41640 1 -- start start 2023-11-08T19:26:44.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.913+0000 7ffab5f41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 0x7ffab00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.913+0000 7ffab5f41640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffab006cb40 0x7ffab006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.913+0000 7ffab5f41640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab00c4960 con 0x7ffab0142f80 2023-11-08T19:26:44.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.913+0000 7ffab5f41640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffab0143a40 con 0x7ffab006cb40 2023-11-08T19:26:44.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.913+0000 7ffab5f41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00ca3e0 con 0x7ffab015d9f0 2023-11-08T19:26:44.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.914+0000 7ffaaf7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffab0142f80 0x7ffab00c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58330/0 (socket says 172.21.15.119:58330) 2023-11-08T19:26:44.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.914+0000 7ffaaf7fe640 1 -- 172.21.15.119:0/3815356783 learned_addr learned my addr 172.21.15.119:0/3815356783 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.914+0000 7ffaaeffd640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 0x7ffab00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.914+0000 7ffaaffff640 1 --2- 172.21.15.119:0/3815356783 >> v2:172.21.15.188:3300/0 conn(0x7ffab006cb40 0x7ffab006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.914+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2759049529 0 0) 0x7ffab00c4960 con 0x7ffab0142f80 2023-11-08T19:26:44.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.914+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa9c005730 con 0x7ffab0142f80 2023-11-08T19:26:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2541313497 0 0) 0x7ffa9c005730 con 0x7ffab0142f80 2023-11-08T19:26:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 msgr2=0x7ffab00cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 0x7ffab00cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 >> v2:172.21.15.188:3300/0 conn(0x7ffab006cb40 msgr2=0x7ffab006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 --2- 172.21.15.119:0/3815356783 >> v2:172.21.15.188:3300/0 conn(0x7ffab006cb40 0x7ffab006cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab015df60 con 0x7ffab0142f80 2023-11-08T19:26:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7ffaa0001670 con 0x7ffab0142f80 2023-11-08T19:26:44.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaaffff640 1 --2- 172.21.15.119:0/3815356783 >> v2:172.21.15.188:3300/0 conn(0x7ffab006cb40 0x7ffab006cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:44.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 >> v1:172.21.15.119:6789/0 conn(0x7ffab0142f80 legacy=0x7ffab00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa9c005b80 0x7ffa9c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 0x7ffa9c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa9c006f10 0x7ffa9c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0143a40 con 0x7ffa9c006f10 2023-11-08T19:26:44.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00ca3e0 con 0x7ffa9c005b80 2023-11-08T19:26:44.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c00dc40 con 0x7ffab015d9f0 2023-11-08T19:26:44.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaaffff640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa9c006f10 0x7ffa9c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaaffff640 1 -- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 msgr2=0x7ffa9c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaaeffd640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 0x7ffa9c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.915+0000 7ffaaf7fe640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa9c005b80 0x7ffa9c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.916+0000 7ffaaffff640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffab015d9f0 0x7ffa9c0069d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.916+0000 7ffaaffff640 1 -- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa9c005b80 msgr2=0x7ffa9c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:44.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.916+0000 7ffaaffff640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa9c005b80 0x7ffa9c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.916+0000 7ffaaffff640 1 -- 172.21.15.119:0/3815356783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c005730 con 0x7ffa9c006f10 2023-11-08T19:26:44.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.916+0000 7ffaaf7fe640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa9c005b80 0x7ffa9c005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:44.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffaaffff640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa9c006f10 0x7ffa9c007390 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80073e0 tx=0x7ffa9c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d94dd017a206966 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa802cdc0 con 0x7ffa9c006f10 2023-11-08T19:26:44.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa8034440 con 0x7ffa9c006f10 2023-11-08T19:26:44.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffaae7fc640 1 -- 172.21.15.119:0/3815356783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa80347b0 con 0x7ffa9c006f10 2023-11-08T19:26:44.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffab5f41640 1 -- 172.21.15.119:0/3815356783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa9c006f10 msgr2=0x7ffa9c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffab5f41640 1 --2- 172.21.15.119:0/3815356783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa9c006f10 0x7ffa9c007390 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80073e0 tx=0x7ffa9c007a00 comp rx=0 tx=0).stop 2023-11-08T19:26:44.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.917+0000 7ffaaffff640 1 -- 172.21.15.119:0/3815356783 reap_dead start 2023-11-08T19:26:44.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.918+0000 7ffab5f41640 1 -- 172.21.15.119:0/3815356783 shutdown_connections 2023-11-08T19:26:44.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.918+0000 7ffab5f41640 1 -- 172.21.15.119:0/3815356783 >> 172.21.15.119:0/3815356783 conn(0x7ffab00cb610 msgr2=0x7ffab00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:44.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.918+0000 7ffab5f41640 1 -- 172.21.15.119:0/3815356783 shutdown_connections 2023-11-08T19:26:44.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.918+0000 7ffab5f41640 1 -- 172.21.15.119:0/3815356783 wait complete. 2023-11-08T19:26:44.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.919+0000 7ffab5f41640 1 Processor -- start 2023-11-08T19:26:44.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.920+0000 7ffab5f41640 1 -- start start 2023-11-08T19:26:44.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.920+0000 7ffab5f41640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa9c006f10 0x7ffab00b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffab5f41640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 0x7ffab00b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffab5f41640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab01f5da0 0x7ffab0140a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffab5f41640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b0580 con 0x7ffab013d620 2023-11-08T19:26:44.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffab5f41640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b3570 con 0x7ffab01f5da0 2023-11-08T19:26:44.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffab5f41640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab00b3a60 con 0x7ffa9c006f10 2023-11-08T19:26:44.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 0x7ffab00b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 0x7ffab00b3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55484/0 (socket says 172.21.15.119:55484) 2023-11-08T19:26:44.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 -- 172.21.15.119:0/2615710935 learned_addr learned my addr 172.21.15.119:0/2615710935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:44.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaf7fe640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa9c006f10 0x7ffab00b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 -- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa9c006f10 msgr2=0x7ffab00b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa9c006f10 0x7ffab00b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaffff640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab01f5da0 0x7ffab0140a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 -- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab01f5da0 msgr2=0x7ffab0140a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:44.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab01f5da0 0x7ffab0140a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:44.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.921+0000 7ffaaeffd640 1 -- 172.21.15.119:0/2615710935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00b3d40 con 0x7ffab013d620 2023-11-08T19:26:44.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.922+0000 7ffaaeffd640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 0x7ffab00b3030 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffa98010860 tx=0x7ffa98010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.922+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98009610 con 0x7ffab013d620 2023-11-08T19:26:44.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.922+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9800eb60 con 0x7ffab013d620 2023-11-08T19:26:44.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.922+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9801c400 con 0x7ffab013d620 2023-11-08T19:26:44.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.922+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab00b4020 con 0x7ffab013d620 2023-11-08T19:26:44.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.922+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab00b45e0 con 0x7ffab013d620 2023-11-08T19:26:44.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.923+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab0074430 con 0x7ffab013d620 2023-11-08T19:26:44.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.924+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa9801c560 con 0x7ffab013d620 2023-11-08T19:26:44.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.925+0000 7ffaa7fff640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa9006ae70 0x7ffa9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:44.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.926+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7ffa98089f50 con 0x7ffab013d620 2023-11-08T19:26:44.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.926+0000 7ffaaf7fe640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa9006ae70 0x7ffa9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:44.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.926+0000 7ffaaf7fe640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa9006ae70 0x7ffa9006d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00017a0 tx=0x7ffaa0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:44.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:44.930+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa98054d10 con 0x7ffab013d620 2023-11-08T19:26:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.126+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffab00c4230 con 0x7ffab013d620 2023-11-08T19:26:45.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.127+0000 7ffaa7fff640 1 -- 172.21.15.119:0/2615710935 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa98058330 con 0x7ffab013d620 2023-11-08T19:26:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:26:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa9006ae70 msgr2=0x7ffa9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa9006ae70 0x7ffa9006d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ffaa00017a0 tx=0x7ffaa0009040 comp rx=0 tx=0).stop 2023-11-08T19:26:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 msgr2=0x7ffab00b3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 0x7ffab00b3030 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ffa98010860 tx=0x7ffa98010d20 comp rx=0 tx=0).stop 2023-11-08T19:26:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 shutdown_connections 2023-11-08T19:26:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa9006ae70 0x7ffa9006d330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffab01f5da0 0x7ffab0140a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffab013d620 0x7ffab00b3030 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 --2- 172.21.15.119:0/2615710935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa9c006f10 0x7ffab00b0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 >> 172.21.15.119:0/2615710935 conn(0x7ffab00cb610 msgr2=0x7ffab00b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.129+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 shutdown_connections 2023-11-08T19:26:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.130+0000 7ffab5f41640 1 -- 172.21.15.119:0/2615710935 wait complete. 2023-11-08T19:26:45.149 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:26:45.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:45.150 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:26:45.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:26:45.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:26:45.262 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:26:45.316 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:45.317 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:45.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:26:45.379 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:45.380 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:45.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:26:45.439 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:45.439 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:45.499 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:26:45.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:45.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.713+0000 7fec38c85640 1 Processor -- start 2023-11-08T19:26:45.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.713+0000 7fec38c85640 1 -- start start 2023-11-08T19:26:45.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.714+0000 7fec38c85640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 0x7fec340accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.714+0000 7fec38c85640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec340adee0 0x7fec3415d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.714+0000 7fec38c85640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3406b780 con 0x7fec34160c50 2023-11-08T19:26:45.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.714+0000 7fec38c85640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec340ad1f0 con 0x7fec340adee0 2023-11-08T19:26:45.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.715+0000 7fec38c85640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3415dd30 con 0x7fec340ac8d0 2023-11-08T19:26:45.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.715+0000 7fec337fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 0x7fec340accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.715+0000 7fec2bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fec340adee0 0x7fec3415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.715+0000 7fec337fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 0x7fec340accb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:53958/0 (socket says 172.21.15.119:53958) 2023-11-08T19:26:45.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.715+0000 7fec337fe640 1 -- 172.21.15.119:0/3397367812 learned_addr learned my addr 172.21.15.119:0/3397367812 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:45.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.716+0000 7fec337fe640 1 -- 172.21.15.119:0/3397367812 >> v2:172.21.15.188:3300/0 conn(0x7fec340adee0 msgr2=0x7fec3415d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:45.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.716+0000 7fec337fe640 1 --2- 172.21.15.119:0/3397367812 >> v2:172.21.15.188:3300/0 conn(0x7fec340adee0 0x7fec3415d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.716+0000 7fec337fe640 1 -- 172.21.15.119:0/3397367812 >> v1:172.21.15.188:6789/0 conn(0x7fec34160c50 legacy=0x7fec341610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.716+0000 7fec337fe640 1 -- 172.21.15.119:0/3397367812 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3415e010 con 0x7fec340ac8d0 2023-11-08T19:26:45.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.717+0000 7fec337fe640 1 --2- 172.21.15.119:0/3397367812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 0x7fec340accb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec24009870 tx=0x7fec2402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aaf3887c098980 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:45.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.718+0000 7fec32ffd640 1 -- 172.21.15.119:0/3397367812 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24030840 con 0x7fec340ac8d0 2023-11-08T19:26:45.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.718+0000 7fec32ffd640 1 -- 172.21.15.119:0/3397367812 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec240309a0 con 0x7fec340ac8d0 2023-11-08T19:26:45.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec32ffd640 1 -- 172.21.15.119:0/3397367812 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec24030d30 con 0x7fec340ac8d0 2023-11-08T19:26:45.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 -- 172.21.15.119:0/3397367812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 msgr2=0x7fec340accb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 --2- 172.21.15.119:0/3397367812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 0x7fec340accb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fec24009870 tx=0x7fec2402b980 comp rx=0 tx=0).stop 2023-11-08T19:26:45.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 -- 172.21.15.119:0/3397367812 shutdown_connections 2023-11-08T19:26:45.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 --2- 172.21.15.119:0/3397367812 >> v2:172.21.15.188:3300/0 conn(0x7fec340adee0 0x7fec3415d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 --2- 172.21.15.119:0/3397367812 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec340ac8d0 0x7fec340accb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 -- 172.21.15.119:0/3397367812 >> 172.21.15.119:0/3397367812 conn(0x7fec340b0560 msgr2=0x7fec340b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:45.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 -- 172.21.15.119:0/3397367812 shutdown_connections 2023-11-08T19:26:45.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.719+0000 7fec38c85640 1 -- 172.21.15.119:0/3397367812 wait complete. 2023-11-08T19:26:45.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.722+0000 7fec38c85640 1 Processor -- start 2023-11-08T19:26:45.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.722+0000 7fec38c85640 1 -- start start 2023-11-08T19:26:45.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec38c85640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec340ac8d0 0x7fec34159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec38c85640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 0x7fec340b1540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec38c85640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec34160c50 0x7fec341f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec38c85640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34159760 con 0x7fec340adee0 2023-11-08T19:26:45.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec38c85640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec34159560 con 0x7fec340ac8d0 2023-11-08T19:26:45.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec38c85640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec340b1eb0 con 0x7fec34160c50 2023-11-08T19:26:45.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 0x7fec340b1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec2bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 0x7fec340b1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55500/0 (socket says 172.21.15.119:55500) 2023-11-08T19:26:45.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.723+0000 7fec2bfff640 1 -- 172.21.15.119:0/3629428687 learned_addr learned my addr 172.21.15.119:0/3629428687 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:45.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec337fe640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec340ac8d0 0x7fec34159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec33fff640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec34160c50 0x7fec341f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec2bfff640 1 -- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec34160c50 msgr2=0x7fec341f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec2bfff640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec34160c50 0x7fec341f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec2bfff640 1 -- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec340ac8d0 msgr2=0x7fec34159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec2bfff640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec340ac8d0 0x7fec34159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec2bfff640 1 -- 172.21.15.119:0/3629428687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec240094d0 con 0x7fec340adee0 2023-11-08T19:26:45.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec2bfff640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 0x7fec340b1540 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00e9b0 tx=0x7fec1c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:45.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.724+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c009670 con 0x7fec340adee0 2023-11-08T19:26:45.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.725+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec1c00c9a0 con 0x7fec340adee0 2023-11-08T19:26:45.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.725+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec340b2190 con 0x7fec340adee0 2023-11-08T19:26:45.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.725+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1c00cd50 con 0x7fec340adee0 2023-11-08T19:26:45.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.725+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec34159970 con 0x7fec340adee0 2023-11-08T19:26:45.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.726+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3404ece0 con 0x7fec340adee0 2023-11-08T19:26:45.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.727+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec1c013470 con 0x7fec340adee0 2023-11-08T19:26:45.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.728+0000 7fec30ff9640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1406ae70 0x7fec1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:45.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.728+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fec1c089b70 con 0x7fec340adee0 2023-11-08T19:26:45.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.728+0000 7fec337fe640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1406ae70 0x7fec1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:45.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.729+0000 7fec337fe640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1406ae70 0x7fec1406d330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fec24036000 tx=0x7fec2403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:45.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.732+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec1c054910 con 0x7fec340adee0 2023-11-08T19:26:45.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.958+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec340ad6b0 con 0x7fec340adee0 2023-11-08T19:26:45.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.960+0000 7fec30ff9640 1 -- 172.21.15.119:0/3629428687 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fec1c017090 con 0x7fec340adee0 2023-11-08T19:26:45.962 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:45.962 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:29.722778+0000","modified":"2023-11-08T19:26:33.132121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/4172930163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4172930163},{"type":"v1","addr":"172.21.15.188:6834","nonce":4172930163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:45.962 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-11-08T19:26:45.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.962+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1406ae70 msgr2=0x7fec1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.962+0000 7fec38c85640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1406ae70 0x7fec1406d330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fec24036000 tx=0x7fec2403b040 comp rx=0 tx=0).stop 2023-11-08T19:26:45.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.962+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 msgr2=0x7fec340b1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:45.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.962+0000 7fec38c85640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 0x7fec340b1540 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00e9b0 tx=0x7fec1c00ee70 comp rx=0 tx=0).stop 2023-11-08T19:26:45.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 shutdown_connections 2023-11-08T19:26:45.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fec1406ae70 0x7fec1406d330 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fec34160c50 0x7fec341f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fec340adee0 0x7fec340b1540 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 --2- 172.21.15.119:0/3629428687 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fec340ac8d0 0x7fec34159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:45.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 >> 172.21.15.119:0/3629428687 conn(0x7fec340b0560 msgr2=0x7fec340c45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:45.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 shutdown_connections 2023-11-08T19:26:45.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:45.963+0000 7fec38c85640 1 -- 172.21.15.119:0/3629428687 wait complete. 2023-11-08T19:26:45.984 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:46.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.192+0000 7f1cd1785640 1 Processor -- start 2023-11-08T19:26:46.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.193+0000 7f1cd1785640 1 -- start start 2023-11-08T19:26:46.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.194+0000 7f1cd1785640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ccc15d0e0 0x7f1ccc0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.194+0000 7f1cd1785640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ccc15d4c0 0x7f1ccc0cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.194+0000 7f1cd1785640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ccc0c7100 con 0x7f1ccc0cffa0 2023-11-08T19:26:46.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.194+0000 7f1cd1785640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc0d04f0 con 0x7f1ccc15d4c0 2023-11-08T19:26:46.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.194+0000 7f1cd1785640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ccc0d07d0 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.194+0000 7f1cd0f84640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1ccc0cffa0 0x7f1ccc0d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58336/0 (socket says 172.21.15.119:58336) 2023-11-08T19:26:46.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1cd0f84640 1 -- 172.21.15.119:0/2343513728 learned_addr learned my addr 172.21.15.119:0/2343513728 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:46.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccb7fe640 1 --2- 172.21.15.119:0/2343513728 >> v2:172.21.15.119:3300/0 conn(0x7f1ccc15d4c0 0x7f1ccc0cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccbfff640 1 --2- 172.21.15.119:0/2343513728 >> v2:172.21.15.188:3300/0 conn(0x7f1ccc15d0e0 0x7f1ccc0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccaffd640 1 -- 172.21.15.119:0/2343513728 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961103112 0 0) 0x7f1ccc0c7100 con 0x7f1ccc0cffa0 2023-11-08T19:26:46.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccaffd640 1 -- 172.21.15.119:0/2343513728 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cb8003520 con 0x7f1ccc0cffa0 2023-11-08T19:26:46.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2343513728 >> v2:172.21.15.188:3300/0 conn(0x7f1ccc15d0e0 msgr2=0x7f1ccc0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:46.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccb7fe640 1 --2- 172.21.15.119:0/2343513728 >> v2:172.21.15.188:3300/0 conn(0x7f1ccc15d0e0 0x7f1ccc0cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2343513728 >> v1:172.21.15.119:6789/0 conn(0x7f1ccc0cffa0 legacy=0x7f1ccc0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.195+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2343513728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc0c6ab0 con 0x7f1ccc15d4c0 2023-11-08T19:26:46.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.197+0000 7f1ccb7fe640 1 --2- 172.21.15.119:0/2343513728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d4c0 0x7f1ccc0cdfb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc066df0 tx=0x7f1cb402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee9d591f7e3d057 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.197+0000 7f1ccaffd640 1 -- 172.21.15.119:0/2343513728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4030610 con 0x7f1ccc15d4c0 2023-11-08T19:26:46.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.197+0000 7f1ccaffd640 1 -- 172.21.15.119:0/2343513728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb4030770 con 0x7f1ccc15d4c0 2023-11-08T19:26:46.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.197+0000 7f1ccaffd640 1 -- 172.21.15.119:0/2343513728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4030b00 con 0x7f1ccc15d4c0 2023-11-08T19:26:46.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.197+0000 7f1cd1785640 1 -- 172.21.15.119:0/2343513728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d4c0 msgr2=0x7f1ccc0cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2343513728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d4c0 0x7f1ccc0cdfb0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc066df0 tx=0x7f1cb402b960 comp rx=0 tx=0).stop 2023-11-08T19:26:46.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 -- 172.21.15.119:0/2343513728 shutdown_connections 2023-11-08T19:26:46.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2343513728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d4c0 0x7f1ccc0cdfb0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2343513728 >> v2:172.21.15.188:3300/0 conn(0x7f1ccc15d0e0 0x7f1ccc0cda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 -- 172.21.15.119:0/2343513728 >> 172.21.15.119:0/2343513728 conn(0x7f1ccc0d2f60 msgr2=0x7f1ccc0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:46.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 -- 172.21.15.119:0/2343513728 shutdown_connections 2023-11-08T19:26:46.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.198+0000 7f1cd1785640 1 -- 172.21.15.119:0/2343513728 wait complete. 2023-11-08T19:26:46.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.199+0000 7f1cd1785640 1 Processor -- start 2023-11-08T19:26:46.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.200+0000 7f1cd1785640 1 -- start start 2023-11-08T19:26:46.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1cd1785640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc0cffa0 0x7f1ccc154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1cd1785640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 0x7f1ccc0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1cd1785640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ccc15d4c0 0x7f1ccc1e6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1cd1785640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc15fa30 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1cd1785640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc15ee20 con 0x7f1ccc0cffa0 2023-11-08T19:26:46.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1cd1785640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ccc15f100 con 0x7f1ccc15d4c0 2023-11-08T19:26:46.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 0x7f1ccc0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 0x7f1ccc0b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55512/0 (socket says 172.21.15.119:55512) 2023-11-08T19:26:46.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2581844255 learned_addr learned my addr 172.21.15.119:0/2581844255 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:46.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2581844255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ccc15d4c0 msgr2=0x7f1ccc1e6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ccc15d4c0 0x7f1ccc1e6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2581844255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc0cffa0 msgr2=0x7f1ccc154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc0cffa0 0x7f1ccc154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.201+0000 7f1ccb7fe640 1 -- 172.21.15.119:0/2581844255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009400 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.202+0000 7f1ccb7fe640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 0x7f1ccc0b6e80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009890 tx=0x7f1cb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.202+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb403f070 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.202+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb4030e10 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.202+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ccc15f3e0 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.202+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb403ac40 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.202+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ccc0b73c0 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.203+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ccc04ece0 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.204+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1cb40433f0 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.205+0000 7f1cc8ff9640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1cac06ae70 0x7f1cac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.205+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f1cb40ab760 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.205+0000 7f1ccbfff640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1cac06ae70 0x7f1cac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.206+0000 7f1ccbfff640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1cac06ae70 0x7f1cac06d330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0070b0 tx=0x7f1cbc007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.209+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1cb4076470 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.405+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ccc15fb70 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.407+0000 7f1cc8ff9640 1 -- 172.21.15.119:0/2581844255 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18606 (secure 0 0 0) 0x7f1cb4079a90 con 0x7f1ccc15d0e0 2023-11-08T19:26:46.410 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:46.411 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":42,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:41.714462+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.410+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1cac06ae70 msgr2=0x7f1cac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.410+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1cac06ae70 0x7f1cac06d330 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0070b0 tx=0x7f1cbc007040 comp rx=0 tx=0).stop 2023-11-08T19:26:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.410+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 msgr2=0x7f1ccc0b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.410+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 0x7f1ccc0b6e80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009890 tx=0x7f1cb40046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 shutdown_connections 2023-11-08T19:26:46.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1cac06ae70 0x7f1cac06d330 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ccc15d4c0 0x7f1ccc1e6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ccc15d0e0 0x7f1ccc0b6e80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 --2- 172.21.15.119:0/2581844255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ccc0cffa0 0x7f1ccc154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 >> 172.21.15.119:0/2581844255 conn(0x7f1ccc0d2f60 msgr2=0x7f1ccc15bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 shutdown_connections 2023-11-08T19:26:46.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.411+0000 7f1cd1785640 1 -- 172.21.15.119:0/2581844255 wait complete. 2023-11-08T19:26:46.432 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:26:27.536718+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:26:28.733331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:46.433 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:46.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.652+0000 7f7b65a26640 1 Processor -- start 2023-11-08T19:26:46.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.652+0000 7f7b65a26640 1 -- start start 2023-11-08T19:26:46.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b65a26640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7b6006d250 0x7f7b6015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b65a26640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6015e440 con 0x7f7b6015d9f0 2023-11-08T19:26:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b65a26640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6015e720 con 0x7f7b60143600 2023-11-08T19:26:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b65a26640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b600c4f50 con 0x7f7b6006d250 2023-11-08T19:26:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b5e7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7b6015d9f0 0x7f7b6006cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58338/0 (socket says 172.21.15.119:58338) 2023-11-08T19:26:46.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b5e7fc640 1 -- 172.21.15.119:0/1310931175 learned_addr learned my addr 172.21.15.119:0/1310931175 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b5f7fe640 1 --2- 172.21.15.119:0/1310931175 >> v2:172.21.15.188:3300/0 conn(0x7f7b6006d250 0x7f7b6015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299381900 0 0) 0x7f7b6015e440 con 0x7f7b6015d9f0 2023-11-08T19:26:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.654+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b48003520 con 0x7f7b6015d9f0 2023-11-08T19:26:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1309601708 0 0) 0x7f7b48003520 con 0x7f7b6015d9f0 2023-11-08T19:26:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 >> v2:172.21.15.188:3300/0 conn(0x7f7b6006d250 msgr2=0x7f7b6015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:46.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 --2- 172.21.15.119:0/1310931175 >> v2:172.21.15.188:3300/0 conn(0x7f7b6006d250 0x7f7b6015de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 >> v1:172.21.15.188:6789/0 conn(0x7f7b60143600 legacy=0x7f7b600c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b600c4b70 con 0x7f7b6015d9f0 2023-11-08T19:26:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f7b54001670 con 0x7f7b6015d9f0 2023-11-08T19:26:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 >> v1:172.21.15.119:6789/0 conn(0x7f7b6015d9f0 legacy=0x7f7b6006cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480052a0 0x7f7b4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b600c4f50 con 0x7f7b480039c0 2023-11-08T19:26:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4800c1c0 con 0x7f7b480047c0 2023-11-08T19:26:46.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4800e590 con 0x7f7b480052a0 2023-11-08T19:26:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.655+0000 7f7b5effd640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5e7fc640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5effd640 1 -- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480052a0 msgr2=0x7f7b4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5effd640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480052a0 0x7f7b4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5effd640 1 -- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b480047c0 msgr2=0x7f7b48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5effd640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b480047c0 0x7f7b48004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5effd640 1 -- 172.21.15.119:0/1310931175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b48003520 con 0x7f7b480039c0 2023-11-08T19:26:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.656+0000 7f7b5f7fe640 1 -- 172.21.15.119:0/1310931175 reap_dead start 2023-11-08T19:26:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.657+0000 7f7b5effd640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b50001e50 tx=0x7f7b480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc190c29307a9539 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.657+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50003b70 con 0x7f7b480039c0 2023-11-08T19:26:46.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.657+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b50003cd0 con 0x7f7b480039c0 2023-11-08T19:26:46.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.657+0000 7f7b5dffb640 1 -- 172.21.15.119:0/1310931175 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b50005310 con 0x7f7b480039c0 2023-11-08T19:26:46.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 -- 172.21.15.119:0/1310931175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b480039c0 msgr2=0x7f7b48003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7b50001e50 tx=0x7f7b480057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 -- 172.21.15.119:0/1310931175 shutdown_connections 2023-11-08T19:26:46.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1310931175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b480039c0 0x7f7b48003de0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 -- 172.21.15.119:0/1310931175 >> 172.21.15.119:0/1310931175 conn(0x7f7b600cb610 msgr2=0x7f7b600cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:46.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 -- 172.21.15.119:0/1310931175 shutdown_connections 2023-11-08T19:26:46.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.658+0000 7f7b65a26640 1 -- 172.21.15.119:0/1310931175 wait complete. 2023-11-08T19:26:46.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.660+0000 7f7b65a26640 1 Processor -- start 2023-11-08T19:26:46.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.660+0000 7f7b65a26640 1 -- start start 2023-11-08T19:26:46.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b65a26640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480039c0 0x7f7b6015b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b65a26640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b600c7630 0x7f7b600713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b65a26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 0x7f7b600718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b65a26640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015cc50 con 0x7f7b600c7ab0 2023-11-08T19:26:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b65a26640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015bbb0 con 0x7f7b600c7630 2023-11-08T19:26:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b65a26640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6015be90 con 0x7f7b480039c0 2023-11-08T19:26:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 0x7f7b600718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 0x7f7b600718e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55538/0 (socket says 172.21.15.119:55538) 2023-11-08T19:26:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 -- 172.21.15.119:0/1180536077 learned_addr learned my addr 172.21.15.119:0/1180536077 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5e7fc640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b600c7630 0x7f7b600713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 -- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480039c0 msgr2=0x7f7b6015b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480039c0 0x7f7b6015b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 -- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b600c7630 msgr2=0x7f7b600713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b600c7630 0x7f7b600713a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.661+0000 7f7b5f7fe640 1 -- 172.21.15.119:0/1180536077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6015c170 con 0x7f7b600c7ab0 2023-11-08T19:26:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.662+0000 7f7b5f7fe640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 0x7f7b600718e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7b6006dc30 tx=0x7f7b5800d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.662+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58017070 con 0x7f7b600c7ab0 2023-11-08T19:26:46.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.662+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b58004020 con 0x7f7b600c7ab0 2023-11-08T19:26:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.662+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58004210 con 0x7f7b600c7ab0 2023-11-08T19:26:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.662+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6015c450 con 0x7f7b600c7ab0 2023-11-08T19:26:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.662+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b600cd770 con 0x7f7b600c7ab0 2023-11-08T19:26:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.663+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b6015c970 con 0x7f7b600c7ab0 2023-11-08T19:26:46.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.664+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b58004370 con 0x7f7b600c7ab0 2023-11-08T19:26:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.665+0000 7f7b47fff640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7b4c06aea0 0x7f7b4c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.665+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f7b580890d0 con 0x7f7b600c7ab0 2023-11-08T19:26:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.665+0000 7f7b5effd640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7b4c06aea0 0x7f7b4c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:46.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.666+0000 7f7b5effd640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7b4c06aea0 0x7f7b4c06d360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b50034000 tx=0x7f7b50030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:46.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.669+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b6015c970 con 0x7f7b600c7ab0 2023-11-08T19:26:46.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.863+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b6015c970 con 0x7f7b600c7ab0 2023-11-08T19:26:46.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.864+0000 7f7b47fff640 1 -- 172.21.15.119:0/1180536077 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b6015c970 con 0x7f7b600c7ab0 2023-11-08T19:26:46.866 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:46.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7b4c06aea0 msgr2=0x7f7b4c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7b4c06aea0 0x7f7b4c06d360 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b50034000 tx=0x7f7b50030000 comp rx=0 tx=0).stop 2023-11-08T19:26:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 msgr2=0x7f7b600718e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 0x7f7b600718e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7b6006dc30 tx=0x7f7b5800d8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 shutdown_connections 2023-11-08T19:26:46.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7b4c06aea0 0x7f7b4c06d360 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7b600c7ab0 0x7f7b600718e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b600c7630 0x7f7b600713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 --2- 172.21.15.119:0/1180536077 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b480039c0 0x7f7b6015b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:46.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 >> 172.21.15.119:0/1180536077 conn(0x7f7b600cb610 msgr2=0x7f7b6013fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:46.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 shutdown_connections 2023-11-08T19:26:46.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:46.867+0000 7f7b65a26640 1 -- 172.21.15.119:0/1180536077 wait complete. 2023-11-08T19:26:46.887 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:26:47.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.101+0000 7fc18024b640 1 Processor -- start 2023-11-08T19:26:47.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.101+0000 7fc18024b640 1 -- start start 2023-11-08T19:26:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.102+0000 7fc18024b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc17815d9f0 0x7fc17815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.102+0000 7fc18024b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc17806cb40 0x7fc17806cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.102+0000 7fc18024b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc17806d4a0 con 0x7fc178143600 2023-11-08T19:26:47.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.102+0000 7fc18024b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780c4f50 con 0x7fc17815d9f0 2023-11-08T19:26:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.102+0000 7fc18024b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1780c4960 con 0x7fc17806cb40 2023-11-08T19:26:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17d7e6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc17815d9f0 0x7fc17815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17dfe7640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc178143600 0x7fc1780c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58352/0 (socket says 172.21.15.119:58352) 2023-11-08T19:26:47.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17dfe7640 1 -- 172.21.15.119:0/1033658918 learned_addr learned my addr 172.21.15.119:0/1033658918 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:47.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17e7e8640 1 --2- 172.21.15.119:0/1033658918 >> v2:172.21.15.188:3300/0 conn(0x7fc17806cb40 0x7fc17806cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17d7e6640 1 -- 172.21.15.119:0/1033658918 >> v2:172.21.15.188:3300/0 conn(0x7fc17806cb40 msgr2=0x7fc17806cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:47.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17d7e6640 1 --2- 172.21.15.119:0/1033658918 >> v2:172.21.15.188:3300/0 conn(0x7fc17806cb40 0x7fc17806cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17d7e6640 1 -- 172.21.15.119:0/1033658918 >> v1:172.21.15.119:6789/0 conn(0x7fc178143600 legacy=0x7fc1780c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17cfe5640 1 -- 172.21.15.119:0/1033658918 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814347042 0 0) 0x7fc17806d4a0 con 0x7fc178143600 2023-11-08T19:26:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17d7e6640 1 -- 172.21.15.119:0/1033658918 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17815e440 con 0x7fc17815d9f0 2023-11-08T19:26:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.103+0000 7fc17e7e8640 1 --2- 172.21.15.119:0/1033658918 >> v2:172.21.15.188:3300/0 conn(0x7fc17806cb40 0x7fc17806cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:47.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.105+0000 7fc17d7e6640 1 --2- 172.21.15.119:0/1033658918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17815de60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc1780cc650 tx=0x7fc16c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5630ab4fd1bcd422 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.105+0000 7fc17cfe5640 1 -- 172.21.15.119:0/1033658918 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c02e070 con 0x7fc17815d9f0 2023-11-08T19:26:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.105+0000 7fc17cfe5640 1 -- 172.21.15.119:0/1033658918 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16c0315a0 con 0x7fc17815d9f0 2023-11-08T19:26:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.105+0000 7fc17cfe5640 1 -- 172.21.15.119:0/1033658918 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16c031930 con 0x7fc17815d9f0 2023-11-08T19:26:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.105+0000 7fc18024b640 1 -- 172.21.15.119:0/1033658918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 msgr2=0x7fc17815de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.105+0000 7fc18024b640 1 --2- 172.21.15.119:0/1033658918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17815de60 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc1780cc650 tx=0x7fc16c02b840 comp rx=0 tx=0).stop 2023-11-08T19:26:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.106+0000 7fc18024b640 1 -- 172.21.15.119:0/1033658918 shutdown_connections 2023-11-08T19:26:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.106+0000 7fc18024b640 1 --2- 172.21.15.119:0/1033658918 >> v2:172.21.15.188:3300/0 conn(0x7fc17806cb40 0x7fc17806cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.106+0000 7fc18024b640 1 --2- 172.21.15.119:0/1033658918 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17815de60 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.106+0000 7fc18024b640 1 -- 172.21.15.119:0/1033658918 >> 172.21.15.119:0/1033658918 conn(0x7fc1780cb610 msgr2=0x7fc1780cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.106+0000 7fc18024b640 1 -- 172.21.15.119:0/1033658918 shutdown_connections 2023-11-08T19:26:47.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.106+0000 7fc18024b640 1 -- 172.21.15.119:0/1033658918 wait complete. 2023-11-08T19:26:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.107+0000 7fc18024b640 1 Processor -- start 2023-11-08T19:26:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.108+0000 7fc18024b640 1 -- start start 2023-11-08T19:26:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc18024b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc17806cb40 0x7fc1780af710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc18024b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc178143600 0x7fc17806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc18024b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc18024b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780c83c0 con 0x7fc17815d9f0 2023-11-08T19:26:47.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc18024b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780b48a0 con 0x7fc17806cb40 2023-11-08T19:26:47.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc18024b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1780af280 con 0x7fc178143600 2023-11-08T19:26:47.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17806bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55550/0 (socket says 172.21.15.119:55550) 2023-11-08T19:26:47.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 -- 172.21.15.119:0/2141326439 learned_addr learned my addr 172.21.15.119:0/2141326439 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:47.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17dfe7640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc17806cb40 0x7fc1780af710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 -- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc178143600 msgr2=0x7fc17806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17d7e6640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc178143600 0x7fc17806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc178143600 0x7fc17806ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 -- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc17806cb40 msgr2=0x7fc1780af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.109+0000 7fc17e7e8640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc17806cb40 0x7fc1780af710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc17e7e8640 1 -- 172.21.15.119:0/2141326439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c009400 con 0x7fc17815d9f0 2023-11-08T19:26:47.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc17dfe7640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc17806cb40 0x7fc1780af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:47.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc17e7e8640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17806bf70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ce40 tx=0x7fc17000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000d840 con 0x7fc17815d9f0 2023-11-08T19:26:47.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17000d9a0 con 0x7fc17815d9f0 2023-11-08T19:26:47.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1780c80e0 con 0x7fc17815d9f0 2023-11-08T19:26:47.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17000dd50 con 0x7fc17815d9f0 2023-11-08T19:26:47.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.110+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1780aef10 con 0x7fc17815d9f0 2023-11-08T19:26:47.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.111+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1780aea90 con 0x7fc17815d9f0 2023-11-08T19:26:47.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.112+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc17001e3b0 con 0x7fc17815d9f0 2023-11-08T19:26:47.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.114+0000 7fc1767fc640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc16406aea0 0x7fc16406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.114+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fc17008a200 con 0x7fc17815d9f0 2023-11-08T19:26:47.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.114+0000 7fc17dfe7640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc16406aea0 0x7fc16406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.115+0000 7fc17dfe7640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc16406aea0 0x7fc16406d360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc168000f60 tx=0x7fc168009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.118+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1780aea90 con 0x7fc17815d9f0 2023-11-08T19:26:47.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.309+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1780aea90 con 0x7fc17815d9f0 2023-11-08T19:26:47.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.309+0000 7fc1767fc640 1 -- 172.21.15.119:0/2141326439 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1780aea90 con 0x7fc17815d9f0 2023-11-08T19:26:47.311 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:47.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc16406aea0 msgr2=0x7fc16406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc16406aea0 0x7fc16406d360 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc168000f60 tx=0x7fc168009040 comp rx=0 tx=0).stop 2023-11-08T19:26:47.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 msgr2=0x7fc17806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17806bf70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc17000ce40 tx=0x7fc17000a710 comp rx=0 tx=0).stop 2023-11-08T19:26:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 shutdown_connections 2023-11-08T19:26:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc16406aea0 0x7fc16406d360 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc17815d9f0 0x7fc17806bf70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc178143600 0x7fc17806ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 --2- 172.21.15.119:0/2141326439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc17806cb40 0x7fc1780af710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 >> 172.21.15.119:0/2141326439 conn(0x7fc1780cb610 msgr2=0x7fc178072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 shutdown_connections 2023-11-08T19:26:47.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.312+0000 7fc18024b640 1 -- 172.21.15.119:0/2141326439 wait complete. 2023-11-08T19:26:47.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:26:47.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.541+0000 7fb8396bb640 1 Processor -- start 2023-11-08T19:26:47.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.541+0000 7fb8396bb640 1 -- start start 2023-11-08T19:26:47.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.543+0000 7fb8396bb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb834144580 0x7fb83415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.543+0000 7fb8396bb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb83415f1d0 0x7fb8340c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.543+0000 7fb8396bb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83415ea60 con 0x7fb834144960 2023-11-08T19:26:47.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.543+0000 7fb8396bb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340c9150 con 0x7fb83415f1d0 2023-11-08T19:26:47.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.543+0000 7fb8396bb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340c9360 con 0x7fb834144580 2023-11-08T19:26:47.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.543+0000 7fb8337fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb83415f1d0 0x7fb8340c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb8337fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb83415f1d0 0x7fb8340c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55562/0 (socket says 172.21.15.119:55562) 2023-11-08T19:26:47.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb8337fe640 1 -- 172.21.15.119:0/2461131941 learned_addr learned my addr 172.21.15.119:0/2461131941 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:47.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb832ffd640 1 --2- 172.21.15.119:0/2461131941 >> v2:172.21.15.188:3300/0 conn(0x7fb834144580 0x7fb83415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb8337fe640 1 -- 172.21.15.119:0/2461131941 >> v2:172.21.15.188:3300/0 conn(0x7fb834144580 msgr2=0x7fb83415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:47.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb8337fe640 1 --2- 172.21.15.119:0/2461131941 >> v2:172.21.15.188:3300/0 conn(0x7fb834144580 0x7fb83415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb8337fe640 1 -- 172.21.15.119:0/2461131941 >> v1:172.21.15.119:6789/0 conn(0x7fb834144960 legacy=0x7fb8340cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb8337fe640 1 -- 172.21.15.119:0/2461131941 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340c9640 con 0x7fb83415f1d0 2023-11-08T19:26:47.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.544+0000 7fb832ffd640 1 --2- 172.21.15.119:0/2461131941 >> v2:172.21.15.188:3300/0 conn(0x7fb834144580 0x7fb83415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:47.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.545+0000 7fb8337fe640 1 --2- 172.21.15.119:0/2461131941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340c3a50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb8340ceab0 tx=0x7fb82c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6151a42fe89f704c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.546+0000 7fb8327fc640 1 -- 172.21.15.119:0/2461131941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c02ca80 con 0x7fb83415f1d0 2023-11-08T19:26:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.546+0000 7fb8327fc640 1 -- 172.21.15.119:0/2461131941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c02cbe0 con 0x7fb83415f1d0 2023-11-08T19:26:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.546+0000 7fb8327fc640 1 -- 172.21.15.119:0/2461131941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0343f0 con 0x7fb83415f1d0 2023-11-08T19:26:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.546+0000 7fb8396bb640 1 -- 172.21.15.119:0/2461131941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 msgr2=0x7fb8340c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.546+0000 7fb8396bb640 1 --2- 172.21.15.119:0/2461131941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340c3a50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fb8340ceab0 tx=0x7fb82c02bee0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.547+0000 7fb8396bb640 1 -- 172.21.15.119:0/2461131941 shutdown_connections 2023-11-08T19:26:47.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.547+0000 7fb8396bb640 1 --2- 172.21.15.119:0/2461131941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340c3a50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.547+0000 7fb8396bb640 1 --2- 172.21.15.119:0/2461131941 >> v2:172.21.15.188:3300/0 conn(0x7fb834144580 0x7fb83415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.547+0000 7fb8396bb640 1 -- 172.21.15.119:0/2461131941 >> 172.21.15.119:0/2461131941 conn(0x7fb8340cb610 msgr2=0x7fb8340cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.547+0000 7fb8396bb640 1 -- 172.21.15.119:0/2461131941 shutdown_connections 2023-11-08T19:26:47.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.547+0000 7fb8396bb640 1 -- 172.21.15.119:0/2461131941 wait complete. 2023-11-08T19:26:47.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.549+0000 7fb8396bb640 1 Processor -- start 2023-11-08T19:26:47.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.549+0000 7fb8396bb640 1 -- start start 2023-11-08T19:26:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8396bb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb834144580 0x7fb8341f6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8396bb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb834144960 0x7fb8340ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8396bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8396bb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340cc800 con 0x7fb83415f1d0 2023-11-08T19:26:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8396bb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8341f68f0 con 0x7fb834144960 2023-11-08T19:26:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8396bb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8341f6bd0 con 0x7fb834144580 2023-11-08T19:26:47.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55566/0 (socket says 172.21.15.119:55566) 2023-11-08T19:26:47.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 -- 172.21.15.119:0/3720691228 learned_addr learned my addr 172.21.15.119:0/3720691228 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:47.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb82bfff640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb834144960 0x7fb8340ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 -- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb834144580 msgr2=0x7fb8341f6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb834144580 0x7fb8341f6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 -- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb834144960 msgr2=0x7fb8340ce110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb834144960 0x7fb8340ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.551+0000 7fb8337fe640 1 -- 172.21.15.119:0/3720691228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c009b30 con 0x7fb83415f1d0 2023-11-08T19:26:47.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.552+0000 7fb8337fe640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340712e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb82c036040 tx=0x7fb82c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.552+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c009510 con 0x7fb83415f1d0 2023-11-08T19:26:47.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.552+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c03e450 con 0x7fb83415f1d0 2023-11-08T19:26:47.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.552+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c03e800 con 0x7fb83415f1d0 2023-11-08T19:26:47.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.552+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8341f6eb0 con 0x7fb83415f1d0 2023-11-08T19:26:47.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.552+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8340ce650 con 0x7fb83415f1d0 2023-11-08T19:26:47.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.554+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb834074430 con 0x7fb83415f1d0 2023-11-08T19:26:47.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.555+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb82c02cd50 con 0x7fb83415f1d0 2023-11-08T19:26:47.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.556+0000 7fb82b7fe640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb81406ae70 0x7fb81406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.557+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fb82c0ac470 con 0x7fb83415f1d0 2023-11-08T19:26:47.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.557+0000 7fb832ffd640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb81406ae70 0x7fb81406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.557+0000 7fb832ffd640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb81406ae70 0x7fb81406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb824006fd0 tx=0x7fb824008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.561+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb82c077220 con 0x7fb83415f1d0 2023-11-08T19:26:47.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.748+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb83406cb10 con 0x7fb83415f1d0 2023-11-08T19:26:47.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.749+0000 7fb82b7fe640 1 -- 172.21.15.119:0/3720691228 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb82c07a840 con 0x7fb83415f1d0 2023-11-08T19:26:47.751 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:26:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.751+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb81406ae70 msgr2=0x7fb81406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.751+0000 7fb8396bb640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb81406ae70 0x7fb81406d330 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb824006fd0 tx=0x7fb824008040 comp rx=0 tx=0).stop 2023-11-08T19:26:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.751+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 msgr2=0x7fb8340712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.751+0000 7fb8396bb640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340712e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fb82c036040 tx=0x7fb82c004a40 comp rx=0 tx=0).stop 2023-11-08T19:26:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 shutdown_connections 2023-11-08T19:26:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb81406ae70 0x7fb81406d330 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb83415f1d0 0x7fb8340712e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb834144960 0x7fb8340ce110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 --2- 172.21.15.119:0/3720691228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb834144580 0x7fb8341f6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 >> 172.21.15.119:0/3720691228 conn(0x7fb8340cb610 msgr2=0x7fb83406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 shutdown_connections 2023-11-08T19:26:47.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.752+0000 7fb8396bb640 1 -- 172.21.15.119:0/3720691228 wait complete. 2023-11-08T19:26:47.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:47.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.979+0000 7fe3c547c640 1 Processor -- start 2023-11-08T19:26:47.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.979+0000 7fe3c547c640 1 -- start start 2023-11-08T19:26:47.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.980+0000 7fe3c547c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe3c015bff0 0x7fe3c00c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.980+0000 7fe3c547c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c015ea60 con 0x7fe3c0144580 2023-11-08T19:26:47.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.980+0000 7fe3c547c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3c015c460 con 0x7fe3c0144960 2023-11-08T19:26:47.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.980+0000 7fe3c547c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3c00c9150 con 0x7fe3c015bff0 2023-11-08T19:26:47.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.981+0000 7fe3beffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe3c0144580 0x7fe3c00cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58360/0 (socket says 172.21.15.119:58360) 2023-11-08T19:26:47.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.981+0000 7fe3beffd640 1 -- 172.21.15.119:0/1468548526 learned_addr learned my addr 172.21.15.119:0/1468548526 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:47.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.981+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1468548526 >> v2:172.21.15.188:3300/0 conn(0x7fe3c015bff0 0x7fe3c00c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.981+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3889091007 0 0) 0x7fe3c015ea60 con 0x7fe3c0144580 2023-11-08T19:26:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.981+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3a8003520 con 0x7fe3c0144580 2023-11-08T19:26:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 892094305 0 0) 0x7fe3a8003520 con 0x7fe3c0144580 2023-11-08T19:26:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 >> v2:172.21.15.188:3300/0 conn(0x7fe3c015bff0 msgr2=0x7fe3c00c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 --2- 172.21.15.119:0/1468548526 >> v2:172.21.15.188:3300/0 conn(0x7fe3c015bff0 0x7fe3c00c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 >> v1:172.21.15.188:6789/0 conn(0x7fe3c0144960 legacy=0x7fe3c015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c015f8e0 con 0x7fe3c0144580 2023-11-08T19:26:47.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe3b0001670 con 0x7fe3c0144580 2023-11-08T19:26:47.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 >> v1:172.21.15.119:6789/0 conn(0x7fe3c0144580 legacy=0x7fe3c00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a80039c0 0x7fe3a8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80047c0 0x7fe3a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a80051d0 0x7fe3a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00c9150 con 0x7fe3a80051d0 2023-11-08T19:26:47.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800c120 con 0x7fe3a80047c0 2023-11-08T19:26:47.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a800c2e0 con 0x7fe3a80039c0 2023-11-08T19:26:47.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a80051d0 0x7fe3a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a80039c0 msgr2=0x7fe3a8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:47.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3a80039c0 0x7fe3a8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80047c0 msgr2=0x7fe3a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:47.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3be7fc640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80047c0 0x7fe3a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80047c0 0x7fe3a8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.982+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1468548526 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8003520 con 0x7fe3a80051d0 2023-11-08T19:26:47.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.983+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1468548526 reap_dead start 2023-11-08T19:26:47.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a80051d0 0x7fe3a800b980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009f90 tx=0x7fe3a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad746b5eb6d0393 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b802cc20 con 0x7fe3a80051d0 2023-11-08T19:26:47.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b800f040 con 0x7fe3a80051d0 2023-11-08T19:26:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3bdffb640 1 -- 172.21.15.119:0/1468548526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8003990 con 0x7fe3a80051d0 2023-11-08T19:26:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3c547c640 1 -- 172.21.15.119:0/1468548526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a80051d0 msgr2=0x7fe3a800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a80051d0 0x7fe3a800b980 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8009f90 tx=0x7fe3a80057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3c547c640 1 -- 172.21.15.119:0/1468548526 shutdown_connections 2023-11-08T19:26:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.984+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1468548526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3a80051d0 0x7fe3a800b980 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.985+0000 7fe3c547c640 1 -- 172.21.15.119:0/1468548526 >> 172.21.15.119:0/1468548526 conn(0x7fe3c00cb610 msgr2=0x7fe3c00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.985+0000 7fe3c547c640 1 -- 172.21.15.119:0/1468548526 shutdown_connections 2023-11-08T19:26:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.985+0000 7fe3c547c640 1 -- 172.21.15.119:0/1468548526 wait complete. 2023-11-08T19:26:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.986+0000 7fe3c547c640 1 Processor -- start 2023-11-08T19:26:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.986+0000 7fe3c547c640 1 -- start start 2023-11-08T19:26:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3c547c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80051d0 0x7fe3c00b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3c547c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c00c5f20 0x7fe3c015b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3c547c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 0x7fe3c00c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3c547c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c015b8e0 con 0x7fe3c0071670 2023-11-08T19:26:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3c547c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00c6490 con 0x7fe3a80051d0 2023-11-08T19:26:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3c547c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c00c6770 con 0x7fe3c00c5f20 2023-11-08T19:26:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 0x7fe3c00c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.987+0000 7fe3bf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 0x7fe3c00c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55570/0 (socket says 172.21.15.119:55570) 2023-11-08T19:26:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1438740881 learned_addr learned my addr 172.21.15.119:0/1438740881 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3beffd640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80051d0 0x7fe3c00b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3be7fc640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c00c5f20 0x7fe3c015b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c00c5f20 msgr2=0x7fe3c015b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c00c5f20 0x7fe3c015b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80051d0 msgr2=0x7fe3c00b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80051d0 0x7fe3c00b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 -- 172.21.15.119:0/1438740881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c006fa40 con 0x7fe3c0071670 2023-11-08T19:26:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3bf7fe640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 0x7fe3c00c7f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe3b802c690 tx=0x7fe3b8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8003da0 con 0x7fe3c0071670 2023-11-08T19:26:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b80358d0 con 0x7fe3c0071670 2023-11-08T19:26:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b8035c60 con 0x7fe3c0071670 2023-11-08T19:26:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.988+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c006fd20 con 0x7fe3c0071670 2023-11-08T19:26:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.989+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c0071af0 con 0x7fe3c0071670 2023-11-08T19:26:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.990+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c015dd70 con 0x7fe3c0071670 2023-11-08T19:26:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.990+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3b8035dc0 con 0x7fe3c0071670 2023-11-08T19:26:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.991+0000 7fe3a77fe640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe3ac06aea0 0x7fe3ac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.991+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fe3b80ac590 con 0x7fe3c0071670 2023-11-08T19:26:48.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.992+0000 7fe3beffd640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe3ac06aea0 0x7fe3ac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.992+0000 7fe3beffd640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe3ac06aea0 0x7fe3ac06d360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0001860 tx=0x7fe3b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:47.995+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3b80773c0 con 0x7fe3c0071670 2023-11-08T19:26:48.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.182+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fe3c006cb10 con 0x7fe3c0071670 2023-11-08T19:26:48.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.183+0000 7fe3a77fe640 1 -- 172.21.15.119:0/1438740881 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v42) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe3b807a9e0 con 0x7fe3c0071670 2023-11-08T19:26:48.185 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:48.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe3ac06aea0 msgr2=0x7fe3ac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe3ac06aea0 0x7fe3ac06d360 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fe3b0001860 tx=0x7fe3b0009040 comp rx=0 tx=0).stop 2023-11-08T19:26:48.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 msgr2=0x7fe3c00c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 0x7fe3c00c7f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fe3b802c690 tx=0x7fe3b8009750 comp rx=0 tx=0).stop 2023-11-08T19:26:48.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 shutdown_connections 2023-11-08T19:26:48.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe3ac06aea0 0x7fe3ac06d360 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3c0071670 0x7fe3c00c7f40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3c00c5f20 0x7fe3c015b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 --2- 172.21.15.119:0/1438740881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe3a80051d0 0x7fe3c00b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 >> 172.21.15.119:0/1438740881 conn(0x7fe3c00cb610 msgr2=0x7fe3c0143620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.186+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 shutdown_connections 2023-11-08T19:26:48.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.187+0000 7fe3c547c640 1 -- 172.21.15.119:0/1438740881 wait complete. 2023-11-08T19:26:48.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:48.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.415+0000 7fbdc6cd4640 1 Processor -- start 2023-11-08T19:26:48.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.415+0000 7fbdc6cd4640 1 -- start start 2023-11-08T19:26:48.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.416+0000 7fbdc6cd4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdc015d9f0 0x7fbdc00cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.416+0000 7fbdc6cd4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbdc006cb40 0x7fbdc006cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.416+0000 7fbdc6cd4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc006d4d0 con 0x7fbdc0142f80 2023-11-08T19:26:48.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.416+0000 7fbdc6cd4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc0143a40 con 0x7fbdc015d9f0 2023-11-08T19:26:48.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.416+0000 7fbdc6cd4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc00ca3e0 con 0x7fbdc006cb40 2023-11-08T19:26:48.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdc015d9f0 0x7fbdc00cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbdc015d9f0 0x7fbdc00cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55580/0 (socket says 172.21.15.119:55580) 2023-11-08T19:26:48.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 -- 172.21.15.119:0/66529856 learned_addr learned my addr 172.21.15.119:0/66529856 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:48.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdc5271640 1 --2- 172.21.15.119:0/66529856 >> v2:172.21.15.188:3300/0 conn(0x7fbdc006cb40 0x7fbdc006cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 -- 172.21.15.119:0/66529856 >> v2:172.21.15.188:3300/0 conn(0x7fbdc006cb40 msgr2=0x7fbdc006cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 --2- 172.21.15.119:0/66529856 >> v2:172.21.15.188:3300/0 conn(0x7fbdc006cb40 0x7fbdc006cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 -- 172.21.15.119:0/66529856 >> v1:172.21.15.188:6789/0 conn(0x7fbdc0142f80 legacy=0x7fbdc00c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.417+0000 7fbdbffff640 1 -- 172.21.15.119:0/66529856 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc00c4960 con 0x7fbdc015d9f0 2023-11-08T19:26:48.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.419+0000 7fbdbffff640 1 --2- 172.21.15.119:0/66529856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc00cc6d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00c41e0 tx=0x7fbdac02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d87abe2b62cea99e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.419+0000 7fbdbf7fe640 1 -- 172.21.15.119:0/66529856 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac030860 con 0x7fbdc015d9f0 2023-11-08T19:26:48.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.419+0000 7fbdbf7fe640 1 -- 172.21.15.119:0/66529856 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdac0309c0 con 0x7fbdc015d9f0 2023-11-08T19:26:48.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.419+0000 7fbdbf7fe640 1 -- 172.21.15.119:0/66529856 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac030d50 con 0x7fbdc015d9f0 2023-11-08T19:26:48.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.419+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/66529856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 msgr2=0x7fbdc00cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.419+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/66529856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc00cc6d0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00c41e0 tx=0x7fbdac02b9a0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.420+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/66529856 shutdown_connections 2023-11-08T19:26:48.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.420+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/66529856 >> v2:172.21.15.188:3300/0 conn(0x7fbdc006cb40 0x7fbdc006cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.420+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/66529856 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc00cc6d0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.420+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/66529856 >> 172.21.15.119:0/66529856 conn(0x7fbdc00cb610 msgr2=0x7fbdc00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.420+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/66529856 shutdown_connections 2023-11-08T19:26:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.420+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/66529856 wait complete. 2023-11-08T19:26:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.421+0000 7fbdc6cd4640 1 Processor -- start 2023-11-08T19:26:48.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 -- start start 2023-11-08T19:26:48.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc006cb40 0x7fbdc0141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc0142f80 0x7fbdc00af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00c8190 con 0x7fbdc015d9f0 2023-11-08T19:26:48.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00c70f0 con 0x7fbdc006cb40 2023-11-08T19:26:48.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.422+0000 7fbdc6cd4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc00c73d0 con 0x7fbdc0142f80 2023-11-08T19:26:48.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc015c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:55582/0 (socket says 172.21.15.119:55582) 2023-11-08T19:26:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 -- 172.21.15.119:0/3222175861 learned_addr learned my addr 172.21.15.119:0/3222175861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 -- 172.21.15.119:0/3222175861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc0142f80 msgr2=0x7fbdc00af160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc0142f80 0x7fbdc00af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 -- 172.21.15.119:0/3222175861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc006cb40 msgr2=0x7fbdc0141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc006cb40 0x7fbdc0141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.423+0000 7fbdc5271640 1 -- 172.21.15.119:0/3222175861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdac0094d0 con 0x7fbdc015d9f0 2023-11-08T19:26:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.424+0000 7fbdc5271640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc015c4d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8007e60 tx=0x7fbdb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.424+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8014420 con 0x7fbdc015d9f0 2023-11-08T19:26:48.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.424+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80145a0 con 0x7fbdc015d9f0 2023-11-08T19:26:48.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.424+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8014950 con 0x7fbdc015d9f0 2023-11-08T19:26:48.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.424+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc00c76b0 con 0x7fbdc015d9f0 2023-11-08T19:26:48.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.424+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc00c7c70 con 0x7fbdc015d9f0 2023-11-08T19:26:48.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.425+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc0074430 con 0x7fbdc015d9f0 2023-11-08T19:26:48.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.426+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbdb8014ab0 con 0x7fbdc015d9f0 2023-11-08T19:26:48.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.427+0000 7fbdbd7fa640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbda406ae70 0x7fbda406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.427+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7fbdb808a260 con 0x7fbdc015d9f0 2023-11-08T19:26:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.427+0000 7fbdc4a70640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbda406ae70 0x7fbda406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.428+0000 7fbdc4a70640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbda406ae70 0x7fbda406d330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000f60 tx=0x7fbdb40079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.431+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbdb8055010 con 0x7fbdc015d9f0 2023-11-08T19:26:48.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.660+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdc00b4d90 con 0x7fbdc015d9f0 2023-11-08T19:26:48.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.662+0000 7fbdbd7fa640 1 -- 172.21.15.119:0/3222175861 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fbdb8058630 con 0x7fbdc015d9f0 2023-11-08T19:26:48.663 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:48.664 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:29.722778+0000","modified":"2023-11-08T19:26:33.132121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/4172930163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4172930163},{"type":"v1","addr":"172.21.15.188:6834","nonce":4172930163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:48.664 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-11-08T19:26:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.664+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbda406ae70 msgr2=0x7fbda406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.664+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbda406ae70 0x7fbda406d330 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4000f60 tx=0x7fbdb40079b0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.664+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 msgr2=0x7fbdc015c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.664+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc015c4d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8007e60 tx=0x7fbdb800e4b0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 shutdown_connections 2023-11-08T19:26:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbda406ae70 0x7fbda406d330 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbdc015d9f0 0x7fbdc015c4d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdc0142f80 0x7fbdc00af160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 --2- 172.21.15.119:0/3222175861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdc006cb40 0x7fbdc0141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 >> 172.21.15.119:0/3222175861 conn(0x7fbdc00cb610 msgr2=0x7fbdc00b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 shutdown_connections 2023-11-08T19:26:48.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.665+0000 7fbdc6cd4640 1 -- 172.21.15.119:0/3222175861 wait complete. 2023-11-08T19:26:48.686 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:48.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.890+0000 7ff6536a2640 1 Processor -- start 2023-11-08T19:26:48.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.890+0000 7ff6536a2640 1 -- start start 2023-11-08T19:26:48.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff6536a2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff64c0c9720 0x7ff64c0c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff6536a2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c0c9360 con 0x7ff64c159d90 2023-11-08T19:26:48.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff6536a2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c06d3c0 con 0x7ff64c1610c0 2023-11-08T19:26:48.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff6536a2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c0ca490 con 0x7ff64c0c9720 2023-11-08T19:26:48.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff65143e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff64c159d90 0x7ff64c15a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48078/0 (socket says 172.21.15.119:48078) 2023-11-08T19:26:48.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff65143e640 1 -- 172.21.15.119:0/2536879002 learned_addr learned my addr 172.21.15.119:0/2536879002 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:48.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477914860 0 0) 0x7ff64c0c9360 con 0x7ff64c159d90 2023-11-08T19:26:48.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff638003520 con 0x7ff64c159d90 2023-11-08T19:26:48.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.892+0000 7ff650c3d640 1 --2- 172.21.15.119:0/2536879002 >> v2:172.21.15.188:3300/0 conn(0x7ff64c0c9720 0x7ff64c0c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 916400177 0 0) 0x7ff638003520 con 0x7ff64c159d90 2023-11-08T19:26:48.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 >> v2:172.21.15.188:3300/0 conn(0x7ff64c0c9720 msgr2=0x7ff64c0c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 --2- 172.21.15.119:0/2536879002 >> v2:172.21.15.188:3300/0 conn(0x7ff64c0c9720 0x7ff64c0c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 >> v1:172.21.15.188:6789/0 conn(0x7ff64c1610c0 legacy=0x7ff64c1634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0ca770 con 0x7ff64c159d90 2023-11-08T19:26:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7ff648001670 con 0x7ff64c159d90 2023-11-08T19:26:48.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 >> v1:172.21.15.119:6789/0 conn(0x7ff64c159d90 legacy=0x7ff64c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff638003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6380047c0 0x7ff638004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff638005180 0x7ff63800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0ca490 con 0x7ff638003970 2023-11-08T19:26:48.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800c0d0 con 0x7ff638005180 2023-11-08T19:26:48.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800e4a0 con 0x7ff6380047c0 2023-11-08T19:26:48.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.893+0000 7ff65143e640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff638003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff651c3f640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff638005180 0x7ff63800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff65143e640 1 -- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6380047c0 msgr2=0x7ff638004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff65143e640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6380047c0 0x7ff638004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff65143e640 1 -- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff638005180 msgr2=0x7ff63800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:48.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff65143e640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff638005180 0x7ff63800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff65143e640 1 -- 172.21.15.119:0/2536879002 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff638003520 con 0x7ff638003970 2023-11-08T19:26:48.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.894+0000 7ff651c3f640 1 -- 172.21.15.119:0/2536879002 reap_dead start 2023-11-08T19:26:48.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.895+0000 7ff65143e640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff638003d90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff64800c240 tx=0x7ff6380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d7f6ea61cac5ff server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.895+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648005ad0 con 0x7ff638003970 2023-11-08T19:26:48.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.895+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff648005c50 con 0x7ff638003970 2023-11-08T19:26:48.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.895+0000 7ff643fff640 1 -- 172.21.15.119:0/2536879002 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64802c8b0 con 0x7ff638003970 2023-11-08T19:26:48.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 -- 172.21.15.119:0/2536879002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 msgr2=0x7ff638003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff638003d90 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff64800c240 tx=0x7ff6380057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 -- 172.21.15.119:0/2536879002 shutdown_connections 2023-11-08T19:26:48.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 --2- 172.21.15.119:0/2536879002 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff638003d90 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 -- 172.21.15.119:0/2536879002 >> 172.21.15.119:0/2536879002 conn(0x7ff64c0cb610 msgr2=0x7ff64c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:48.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 -- 172.21.15.119:0/2536879002 shutdown_connections 2023-11-08T19:26:48.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.896+0000 7ff6536a2640 1 -- 172.21.15.119:0/2536879002 wait complete. 2023-11-08T19:26:48.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.897+0000 7ff6536a2640 1 Processor -- start 2023-11-08T19:26:48.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.898+0000 7ff6536a2640 1 -- start start 2023-11-08T19:26:48.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff6536a2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff64c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff6536a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64c14ffb0 0x7ff64c151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff6536a2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff64c150430 0x7ff64c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff6536a2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c151580 con 0x7ff638003970 2023-11-08T19:26:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff6536a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0b4460 con 0x7ff64c14ffb0 2023-11-08T19:26:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff6536a2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c150a00 con 0x7ff64c150430 2023-11-08T19:26:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff64c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff64c0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33460/0 (socket says 172.21.15.119:33460) 2023-11-08T19:26:48.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 -- 172.21.15.119:0/3502936181 learned_addr learned my addr 172.21.15.119:0/3502936181 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:48.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff650c3d640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64c14ffb0 0x7ff64c151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff651c3f640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff64c150430 0x7ff64c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 -- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff64c150430 msgr2=0x7ff64c152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff64c150430 0x7ff64c152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 -- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64c14ffb0 msgr2=0x7ff64c151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64c14ffb0 0x7ff64c151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.899+0000 7ff65143e640 1 -- 172.21.15.119:0/3502936181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c150ce0 con 0x7ff638003970 2023-11-08T19:26:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.900+0000 7ff65143e640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff64c0b3f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff648001930 tx=0x7ff64802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.900+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64803c040 con 0x7ff638003970 2023-11-08T19:26:48.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.900+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff64802ce00 con 0x7ff638003970 2023-11-08T19:26:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.900+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff648007290 con 0x7ff638003970 2023-11-08T19:26:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.900+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c150fc0 con 0x7ff638003970 2023-11-08T19:26:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.900+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c13cbf0 con 0x7ff638003970 2023-11-08T19:26:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.901+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c13cd80 con 0x7ff638003970 2023-11-08T19:26:48.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.902+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff648032070 con 0x7ff638003970 2023-11-08T19:26:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.903+0000 7ff641ffb640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff63006aea0 0x7ff63006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.903+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7ff6480aaf70 con 0x7ff638003970 2023-11-08T19:26:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.903+0000 7ff650c3d640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff63006aea0 0x7ff63006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:48.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.904+0000 7ff650c3d640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff63006aea0 0x7ff63006d360 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006a90 tx=0x7ff63c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:48.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:48.908+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff648075c80 con 0x7ff638003970 2023-11-08T19:26:49.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:49.080 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:26:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.100+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff64c074430 con 0x7ff638003970 2023-11-08T19:26:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.102+0000 7ff641ffb640 1 -- 172.21.15.119:0/3502936181 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18606 (secure 0 0 0) 0x7ff6480792a0 con 0x7ff638003970 2023-11-08T19:26:49.104 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:49.104 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":42,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:41.714462+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:49.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff63006aea0 msgr2=0x7ff63006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff63006aea0 0x7ff63006d360 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff63c006a90 tx=0x7ff63c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 msgr2=0x7ff64c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff64c0b3f20 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff648001930 tx=0x7ff64802e070 comp rx=0 tx=0).stop 2023-11-08T19:26:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 shutdown_connections 2023-11-08T19:26:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff63006aea0 0x7ff63006d360 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff64c150430 0x7ff64c152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff64c14ffb0 0x7ff64c151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 --2- 172.21.15.119:0/3502936181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff638003970 0x7ff64c0b3f20 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 >> 172.21.15.119:0/3502936181 conn(0x7ff64c0cb610 msgr2=0x7ff64c06fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 shutdown_connections 2023-11-08T19:26:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.105+0000 7ff6536a2640 1 -- 172.21.15.119:0/3502936181 wait complete. 2023-11-08T19:26:49.124 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:26:49.125 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:49.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.380+0000 7ff6e8bf5640 1 Processor -- start 2023-11-08T19:26:49.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.380+0000 7ff6e8bf5640 1 -- start start 2023-11-08T19:26:49.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e8bf5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff6e415b080 0x7ff6e40ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e8bf5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6e4161250 0x7ff6e41616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e8bf5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40cda70 con 0x7ff6e40cf8c0 2023-11-08T19:26:49.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e8bf5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e4161fc0 con 0x7ff6e415b080 2023-11-08T19:26:49.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e8bf5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40cdcc0 con 0x7ff6e4161250 2023-11-08T19:26:49.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff6e415b080 0x7ff6e40ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6dbfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff6e40cf8c0 0x7ff6e40cfd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48086/0 (socket says 172.21.15.119:48086) 2023-11-08T19:26:49.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6dbfff640 1 -- 172.21.15.119:0/3090474293 learned_addr learned my addr 172.21.15.119:0/3090474293 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:49.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.382+0000 7ff6e259c640 1 --2- 172.21.15.119:0/3090474293 >> v2:172.21.15.119:3300/0 conn(0x7ff6e415b080 0x7ff6e40ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33462/0 (socket says 172.21.15.119:33462) 2023-11-08T19:26:49.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.383+0000 7ff6e259c640 1 -- 172.21.15.119:0/3090474293 >> v2:172.21.15.188:3300/0 conn(0x7ff6e4161250 msgr2=0x7ff6e41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:49.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.383+0000 7ff6e259c640 1 --2- 172.21.15.119:0/3090474293 >> v2:172.21.15.188:3300/0 conn(0x7ff6e4161250 0x7ff6e41616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.383+0000 7ff6e259c640 1 -- 172.21.15.119:0/3090474293 >> v1:172.21.15.119:6789/0 conn(0x7ff6e40cf8c0 legacy=0x7ff6e40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.383+0000 7ff6e259c640 1 -- 172.21.15.119:0/3090474293 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40cdfa0 con 0x7ff6e415b080 2023-11-08T19:26:49.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.383+0000 7ff6e1d9b640 1 -- 172.21.15.119:0/3090474293 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379080883 0 0) 0x7ff6e40cda70 con 0x7ff6e40cf8c0 2023-11-08T19:26:49.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.384+0000 7ff6e259c640 1 --2- 172.21.15.119:0/3090474293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e415b080 0x7ff6e40ce7a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4072810 tx=0x7ff6d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac10f56a6aa6ed8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.384+0000 7ff6e1d9b640 1 -- 172.21.15.119:0/3090474293 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d402e070 con 0x7ff6e415b080 2023-11-08T19:26:49.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.384+0000 7ff6e1d9b640 1 -- 172.21.15.119:0/3090474293 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d40315a0 con 0x7ff6e415b080 2023-11-08T19:26:49.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.384+0000 7ff6e1d9b640 1 -- 172.21.15.119:0/3090474293 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4031930 con 0x7ff6e415b080 2023-11-08T19:26:49.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/3090474293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e415b080 msgr2=0x7ff6e40ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/3090474293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e415b080 0x7ff6e40ce7a0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4072810 tx=0x7ff6d402b840 comp rx=0 tx=0).stop 2023-11-08T19:26:49.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/3090474293 shutdown_connections 2023-11-08T19:26:49.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/3090474293 >> v2:172.21.15.188:3300/0 conn(0x7ff6e4161250 0x7ff6e41616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/3090474293 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e415b080 0x7ff6e40ce7a0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/3090474293 >> 172.21.15.119:0/3090474293 conn(0x7ff6e406a9f0 msgr2=0x7ff6e406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/3090474293 shutdown_connections 2023-11-08T19:26:49.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.385+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/3090474293 wait complete. 2023-11-08T19:26:49.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.387+0000 7ff6e8bf5640 1 Processor -- start 2023-11-08T19:26:49.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.387+0000 7ff6e8bf5640 1 -- start start 2023-11-08T19:26:49.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e8bf5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6e40cf8c0 0x7ff6e4151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e8bf5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6e415b080 0x7ff6e415e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e8bf5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 0x7ff6e4156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e8bf5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e4151bc0 con 0x7ff6e4161250 2023-11-08T19:26:49.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e8bf5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40c7b70 con 0x7ff6e415b080 2023-11-08T19:26:49.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e8bf5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e415eaf0 con 0x7ff6e40cf8c0 2023-11-08T19:26:49.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.388+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 0x7ff6e4156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 0x7ff6e4156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33464/0 (socket says 172.21.15.119:33464) 2023-11-08T19:26:49.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 -- 172.21.15.119:0/733750709 learned_addr learned my addr 172.21.15.119:0/733750709 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:49.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e259c640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6e40cf8c0 0x7ff6e4151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6dbfff640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6e415b080 0x7ff6e415e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 -- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6e40cf8c0 msgr2=0x7ff6e4151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6e40cf8c0 0x7ff6e4151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 -- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6e415b080 msgr2=0x7ff6e415e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6e415b080 0x7ff6e415e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 -- 172.21.15.119:0/733750709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d4009400 con 0x7ff6e4161250 2023-11-08T19:26:49.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6dbfff640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6e415b080 0x7ff6e415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:49.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6e2d9d640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 0x7ff6e4156c10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b9a0 tx=0x7ff6dc00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.389+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc00cba0 con 0x7ff6e4161250 2023-11-08T19:26:49.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.390+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc00cd20 con 0x7ff6e4161250 2023-11-08T19:26:49.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.390+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc014680 con 0x7ff6e4161250 2023-11-08T19:26:49.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.390+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e4159af0 con 0x7ff6e4161250 2023-11-08T19:26:49.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.390+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e40c80a0 con 0x7ff6e4161250 2023-11-08T19:26:49.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.391+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e4074430 con 0x7ff6e4161250 2023-11-08T19:26:49.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.392+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6dc01d3b0 con 0x7ff6e4161250 2023-11-08T19:26:49.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.393+0000 7ff6daffd640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff6c406ae70 0x7ff6c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.393+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7ff6dc089880 con 0x7ff6e4161250 2023-11-08T19:26:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.393+0000 7ff6e259c640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff6c406ae70 0x7ff6c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.394+0000 7ff6e259c640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff6c406ae70 0x7ff6c406d330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4036000 tx=0x7ff6d4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.397+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6dc054630 con 0x7ff6e4161250 2023-11-08T19:26:49.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.623+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6e40cf6a0 con 0x7ff6e4161250 2023-11-08T19:26:49.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.624+0000 7ff6daffd640 1 -- 172.21.15.119:0/733750709 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7ff6dc019090 con 0x7ff6e4161250 2023-11-08T19:26:49.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:49.626 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:29.722778+0000","modified":"2023-11-08T19:26:33.132121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/4172930163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4172930163},{"type":"v1","addr":"172.21.15.188:6834","nonce":4172930163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:49.626 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-11-08T19:26:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff6c406ae70 msgr2=0x7ff6c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff6c406ae70 0x7ff6c406d330 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff6d4036000 tx=0x7ff6d4008030 comp rx=0 tx=0).stop 2023-11-08T19:26:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 msgr2=0x7ff6e4156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 0x7ff6e4156c10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc00b9a0 tx=0x7ff6dc00be60 comp rx=0 tx=0).stop 2023-11-08T19:26:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 shutdown_connections 2023-11-08T19:26:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff6c406ae70 0x7ff6c406d330 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6e4161250 0x7ff6e4156c10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6e415b080 0x7ff6e415e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 --2- 172.21.15.119:0/733750709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6e40cf8c0 0x7ff6e4151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.626+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 >> 172.21.15.119:0/733750709 conn(0x7ff6e406a9f0 msgr2=0x7ff6e40d22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.627+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 shutdown_connections 2023-11-08T19:26:49.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.627+0000 7ff6e8bf5640 1 -- 172.21.15.119:0/733750709 wait complete. 2023-11-08T19:26:49.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:49.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.857+0000 7f335300e640 1 Processor -- start 2023-11-08T19:26:49.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.857+0000 7f335300e640 1 -- start start 2023-11-08T19:26:49.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.858+0000 7f335300e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f334c159d90 0x7f334c15a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.858+0000 7f335300e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f334c15a6b0 0x7f334c0c9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.858+0000 7f335300e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f334c0c9360 con 0x7f334c0c9c60 2023-11-08T19:26:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.858+0000 7f335300e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0ca490 con 0x7f334c15a6b0 2023-11-08T19:26:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.858+0000 7f335300e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c0ca770 con 0x7f334c159d90 2023-11-08T19:26:49.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f334c15a6b0 0x7f334c0c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f334c15a6b0 0x7f334c0c9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33480/0 (socket says 172.21.15.119:33480) 2023-11-08T19:26:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 -- 172.21.15.119:0/2888222608 learned_addr learned my addr 172.21.15.119:0/2888222608 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f33515ab640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f334c0c9c60 0x7f334c06c3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48096/0 (socket says 172.21.15.119:48096) 2023-11-08T19:26:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f3350daa640 1 --2- 172.21.15.119:0/2888222608 >> v2:172.21.15.188:3300/0 conn(0x7f334c159d90 0x7f334c15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 -- 172.21.15.119:0/2888222608 >> v2:172.21.15.188:3300/0 conn(0x7f334c159d90 msgr2=0x7f334c15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 --2- 172.21.15.119:0/2888222608 >> v2:172.21.15.188:3300/0 conn(0x7f334c159d90 0x7f334c15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 -- 172.21.15.119:0/2888222608 >> v1:172.21.15.119:6789/0 conn(0x7f334c0c9c60 legacy=0x7f334c06c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f334bfff640 1 -- 172.21.15.119:0/2888222608 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c06bbb0 con 0x7f334c15a6b0 2023-11-08T19:26:49.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.859+0000 7f3350daa640 1 --2- 172.21.15.119:0/2888222608 >> v2:172.21.15.188:3300/0 conn(0x7f334c159d90 0x7f334c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:49.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.860+0000 7f334bfff640 1 --2- 172.21.15.119:0/2888222608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c15a6b0 0x7f334c0c9720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f334c0cfbd0 tx=0x7f333c02e220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6144f05a64eeffa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f334b7fe640 1 -- 172.21.15.119:0/2888222608 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c02edc0 con 0x7f334c15a6b0 2023-11-08T19:26:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f334b7fe640 1 -- 172.21.15.119:0/2888222608 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f333c0363f0 con 0x7f334c15a6b0 2023-11-08T19:26:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f334b7fe640 1 -- 172.21.15.119:0/2888222608 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c036780 con 0x7f334c15a6b0 2023-11-08T19:26:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 -- 172.21.15.119:0/2888222608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c15a6b0 msgr2=0x7f334c0c9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 --2- 172.21.15.119:0/2888222608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c15a6b0 0x7f334c0c9720 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f334c0cfbd0 tx=0x7f333c02e220 comp rx=0 tx=0).stop 2023-11-08T19:26:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 -- 172.21.15.119:0/2888222608 shutdown_connections 2023-11-08T19:26:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 --2- 172.21.15.119:0/2888222608 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c15a6b0 0x7f334c0c9720 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 --2- 172.21.15.119:0/2888222608 >> v2:172.21.15.188:3300/0 conn(0x7f334c159d90 0x7f334c15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 -- 172.21.15.119:0/2888222608 >> 172.21.15.119:0/2888222608 conn(0x7f334c0cb610 msgr2=0x7f334c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:49.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.861+0000 7f335300e640 1 -- 172.21.15.119:0/2888222608 shutdown_connections 2023-11-08T19:26:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.862+0000 7f335300e640 1 -- 172.21.15.119:0/2888222608 wait complete. 2023-11-08T19:26:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.863+0000 7f335300e640 1 Processor -- start 2023-11-08T19:26:49.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.863+0000 7f335300e640 1 -- start start 2023-11-08T19:26:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.864+0000 7f335300e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 0x7f334c0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.864+0000 7f335300e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f334c159d90 0x7f334c0cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.864+0000 7f335300e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f334c15a6b0 0x7f334c14a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.864+0000 7f335300e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0c5bd0 con 0x7f334c0c9c60 2023-11-08T19:26:49.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.864+0000 7f335300e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0afa30 con 0x7f334c15a6b0 2023-11-08T19:26:49.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.864+0000 7f335300e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c0c8270 con 0x7f334c159d90 2023-11-08T19:26:49.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 0x7f334c0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 0x7f334c0ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33490/0 (socket says 172.21.15.119:33490) 2023-11-08T19:26:49.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 -- 172.21.15.119:0/2222840889 learned_addr learned my addr 172.21.15.119:0/2222840889 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:49.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 -- 172.21.15.119:0/2222840889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f334c159d90 msgr2=0x7f334c0cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f334c159d90 0x7f334c0cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 -- 172.21.15.119:0/2222840889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f334c15a6b0 msgr2=0x7f334c14a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f334c15a6b0 0x7f334c14a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 -- 172.21.15.119:0/2222840889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009b30 con 0x7f334c0c9c60 2023-11-08T19:26:49.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f3350daa640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 0x7f334c0ac8c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f333400c7f0 tx=0x7f333400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.865+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3334007a30 con 0x7f334c0c9c60 2023-11-08T19:26:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.866+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3334007b90 con 0x7f334c0c9c60 2023-11-08T19:26:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.866+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333400ce40 con 0x7f334c0c9c60 2023-11-08T19:26:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.866+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c06fab0 con 0x7f334c0c9c60 2023-11-08T19:26:49.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.866+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c0c8830 con 0x7f334c0c9c60 2023-11-08T19:26:49.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.867+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0c8a20 con 0x7f334c0c9c60 2023-11-08T19:26:49.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.868+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33340113f0 con 0x7f334c0c9c60 2023-11-08T19:26:49.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.869+0000 7f33497fa640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f332c06aea0 0x7f332c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:49.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.869+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f3334089a60 con 0x7f334c0c9c60 2023-11-08T19:26:49.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.869+0000 7f334bfff640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f332c06aea0 0x7f332c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:49.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.870+0000 7f334bfff640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f332c06aea0 0x7f332c06d360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f333c037000 tx=0x7f333c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:49.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:49.873+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3334054810 con 0x7f334c0c9c60 2023-11-08T19:26:50.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.099+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f334c074430 con 0x7f334c0c9c60 2023-11-08T19:26:50.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.100+0000 7f33497fa640 1 -- 172.21.15.119:0/2222840889 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f333401f090 con 0x7f334c0c9c60 2023-11-08T19:26:50.102 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:50.102 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:26:29.722778+0000","modified":"2023-11-08T19:26:33.132121+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5310},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5310":{"gid":5310,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.188:6834/4172930163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4172930163},{"type":"v1","addr":"172.21.15.188:6834","nonce":4172930163}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T19:26:50.103 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 18 2023-11-08T19:26:50.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f332c06aea0 msgr2=0x7f332c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f332c06aea0 0x7f332c06d360 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f333c037000 tx=0x7f333c030000 comp rx=0 tx=0).stop 2023-11-08T19:26:50.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 msgr2=0x7f334c0ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 0x7f334c0ac8c0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f333400c7f0 tx=0x7f333400cbf0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 shutdown_connections 2023-11-08T19:26:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f332c06aea0 0x7f332c06d360 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f334c15a6b0 0x7f334c14a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f334c159d90 0x7f334c0cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 --2- 172.21.15.119:0/2222840889 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f334c0c9c60 0x7f334c0ac8c0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 >> 172.21.15.119:0/2222840889 conn(0x7f334c0cb610 msgr2=0x7f334c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.103+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 shutdown_connections 2023-11-08T19:26:50.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.104+0000 7f335300e640 1 -- 172.21.15.119:0/2222840889 wait complete. 2023-11-08T19:26:50.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:50.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.330+0000 7f47e86b4640 1 Processor -- start 2023-11-08T19:26:50.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.330+0000 7f47e86b4640 1 -- start start 2023-11-08T19:26:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.332+0000 7f47e86b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 0x7f47e006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.332+0000 7f47e86b4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e015f900 con 0x7f47e01447a0 2023-11-08T19:26:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.332+0000 7f47e86b4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47e00c3a50 con 0x7f47e006a120 2023-11-08T19:26:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.332+0000 7f47e86b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0144b80 con 0x7f47e015f4c0 2023-11-08T19:26:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.332+0000 7f47e6450640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f47e01447a0 0x7f47e00cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48106/0 (socket says 172.21.15.119:48106) 2023-11-08T19:26:50.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.332+0000 7f47e6450640 1 -- 172.21.15.119:0/2849014145 learned_addr learned my addr 172.21.15.119:0/2849014145 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:50.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e5c4f640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 0x7f47e006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327521389 0 0) 0x7f47dc001110 con 0x7f47e01447a0 2023-11-08T19:26:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47cc003520 con 0x7f47e01447a0 2023-11-08T19:26:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2668674902 0 0) 0x7f47cc003520 con 0x7f47e01447a0 2023-11-08T19:26:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 msgr2=0x7f47e006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:50.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 0x7f47e006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 >> v1:172.21.15.188:6789/0 conn(0x7f47e006a120 legacy=0x7f47e00c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e00c9150 con 0x7f47e01447a0 2023-11-08T19:26:50.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f47dc001800 con 0x7f47e01447a0 2023-11-08T19:26:50.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 >> v1:172.21.15.119:6789/0 conn(0x7f47e01447a0 legacy=0x7f47e00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47cc0039a0 0x7f47cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 0x7f47cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47cc004d00 0x7f47cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e0144b80 con 0x7f47cc004d00 2023-11-08T19:26:50.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc00ba60 con 0x7f47cc0039a0 2023-11-08T19:26:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc00bc20 con 0x7f47e015f4c0 2023-11-08T19:26:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.333+0000 7f47e6c51640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47cc004d00 0x7f47cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e5c4f640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 0x7f47cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6c51640 1 -- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 msgr2=0x7f47cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:50.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6450640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47cc0039a0 0x7f47cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6c51640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e015f4c0 0x7f47cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6c51640 1 -- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47cc0039a0 msgr2=0x7f47cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6c51640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47cc0039a0 0x7f47cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6c51640 1 -- 172.21.15.119:0/2849014145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc003520 con 0x7f47cc004d00 2023-11-08T19:26:50.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.334+0000 7f47e6450640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47cc0039a0 0x7f47cc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:50.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.335+0000 7f47e6c51640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47cc004d00 0x7f47cc005180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f47d8001e20 tx=0x7f47cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab23874b2683396a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.335+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8003b70 con 0x7f47cc004d00 2023-11-08T19:26:50.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.335+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d8003cd0 con 0x7f47cc004d00 2023-11-08T19:26:50.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.335+0000 7f47e544e640 1 -- 172.21.15.119:0/2849014145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d8005310 con 0x7f47cc004d00 2023-11-08T19:26:50.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e86b4640 1 -- 172.21.15.119:0/2849014145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47cc004d00 msgr2=0x7f47cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e86b4640 1 --2- 172.21.15.119:0/2849014145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47cc004d00 0x7f47cc005180 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f47d8001e20 tx=0x7f47cc0057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e6c51640 1 -- 172.21.15.119:0/2849014145 reap_dead start 2023-11-08T19:26:50.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e86b4640 1 -- 172.21.15.119:0/2849014145 shutdown_connections 2023-11-08T19:26:50.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e86b4640 1 -- 172.21.15.119:0/2849014145 >> 172.21.15.119:0/2849014145 conn(0x7f47e00cb610 msgr2=0x7f47e00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:50.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e86b4640 1 -- 172.21.15.119:0/2849014145 shutdown_connections 2023-11-08T19:26:50.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.336+0000 7f47e86b4640 1 -- 172.21.15.119:0/2849014145 wait complete. 2023-11-08T19:26:50.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.338+0000 7f47e86b4640 1 Processor -- start 2023-11-08T19:26:50.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.338+0000 7f47e86b4640 1 -- start start 2023-11-08T19:26:50.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e86b4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 0x7f47e00c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e86b4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e00b40c0 0x7f47e01f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e86b4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e00c6320 0x7f47e00c6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e86b4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e00c5bd0 con 0x7f47e00b3ce0 2023-11-08T19:26:50.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e86b4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f3280 con 0x7f47e00b40c0 2023-11-08T19:26:50.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e86b4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e01f3560 con 0x7f47e00c6320 2023-11-08T19:26:50.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e6450640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 0x7f47e00c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e6450640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 0x7f47e00c7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33504/0 (socket says 172.21.15.119:33504) 2023-11-08T19:26:50.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.339+0000 7f47e6450640 1 -- 172.21.15.119:0/252770443 learned_addr learned my addr 172.21.15.119:0/252770443 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:50.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6c51640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e00c6320 0x7f47e00c6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e5c4f640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e00b40c0 0x7f47e01f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6450640 1 -- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e00c6320 msgr2=0x7f47e00c6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6450640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e00c6320 0x7f47e00c6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6450640 1 -- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e00b40c0 msgr2=0x7f47e01f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6450640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e00b40c0 0x7f47e01f2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6450640 1 -- 172.21.15.119:0/252770443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e01f3840 con 0x7f47e00b3ce0 2023-11-08T19:26:50.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e6450640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 0x7f47e00c7ae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47dc001270 tx=0x7f47dc00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc004cb0 con 0x7f47e00b3ce0 2023-11-08T19:26:50.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47dc004e10 con 0x7f47e00b3ce0 2023-11-08T19:26:50.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0072d0 con 0x7f47e00b3ce0 2023-11-08T19:26:50.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.340+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47e00c5e20 con 0x7f47e00b3ce0 2023-11-08T19:26:50.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.341+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47e015b4d0 con 0x7f47e00b3ce0 2023-11-08T19:26:50.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.342+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47e0074430 con 0x7f47e00b3ce0 2023-11-08T19:26:50.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.343+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47dc007430 con 0x7f47e00b3ce0 2023-11-08T19:26:50.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.344+0000 7f47d6ffd640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47c406ae70 0x7f47c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.344+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f47dc089380 con 0x7f47e00b3ce0 2023-11-08T19:26:50.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.344+0000 7f47e5c4f640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47c406ae70 0x7f47c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.345+0000 7f47e5c4f640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47c406ae70 0x7f47c406d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009950 tx=0x7f47d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.348+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47dc054000 con 0x7f47e00b3ce0 2023-11-08T19:26:50.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.542+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f47e015f900 con 0x7f47e00b3ce0 2023-11-08T19:26:50.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.545+0000 7f47d6ffd640 1 -- 172.21.15.119:0/252770443 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18606 (secure 0 0 0) 0x7f47e015f900 con 0x7f47e00b3ce0 2023-11-08T19:26:50.547 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:50.547 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":42,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:41.714462+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:26:27.536718+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-08T19:26:28.733331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:50.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47c406ae70 msgr2=0x7f47c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47c406ae70 0x7f47c406d330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f47d0009950 tx=0x7f47d0008040 comp rx=0 tx=0).stop 2023-11-08T19:26:50.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 msgr2=0x7f47e00c7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 0x7f47e00c7ae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47dc001270 tx=0x7f47dc00df70 comp rx=0 tx=0).stop 2023-11-08T19:26:50.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 shutdown_connections 2023-11-08T19:26:50.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f47c406ae70 0x7f47c406d330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f47e00c6320 0x7f47e00c6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f47e00b40c0 0x7f47e01f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 --2- 172.21.15.119:0/252770443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f47e00b3ce0 0x7f47e00c7ae0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 >> 172.21.15.119:0/252770443 conn(0x7f47e00cb610 msgr2=0x7f47e00cd1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 shutdown_connections 2023-11-08T19:26:50.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.548+0000 7f47e86b4640 1 -- 172.21.15.119:0/252770443 wait complete. 2023-11-08T19:26:50.569 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:26:50.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.783+0000 7f89dc421640 1 Processor -- start 2023-11-08T19:26:50.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.784+0000 7f89dc421640 1 -- start start 2023-11-08T19:26:50.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89dc421640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89d415d410 0x7f89d40ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89dc421640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d406b570 con 0x7f89d415e5f0 2023-11-08T19:26:50.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89dc421640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89d406b850 con 0x7f89d415df00 2023-11-08T19:26:50.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89dc421640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89d406c1f0 con 0x7f89d415d410 2023-11-08T19:26:50.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89da1bd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89d415d410 0x7f89d40ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89da1bd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f89d415d410 0x7f89d40ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33514/0 (socket says 172.21.15.119:33514) 2023-11-08T19:26:50.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89da1bd640 1 -- 172.21.15.119:0/2789617585 learned_addr learned my addr 172.21.15.119:0/2789617585 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:50.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.785+0000 7f89d99bc640 1 --1- 172.21.15.119:0/2789617585 >> v1:172.21.15.119:6789/0 conn(0x7f89d415e5f0 0x7f89d415d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48110/0 (socket says 172.21.15.119:48110) 2023-11-08T19:26:50.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.786+0000 7f89da1bd640 1 -- 172.21.15.119:0/2789617585 >> v1:172.21.15.188:6789/0 conn(0x7f89d415df00 legacy=0x7f89d406dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:50.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.786+0000 7f89da1bd640 1 -- 172.21.15.119:0/2789617585 >> v1:172.21.15.119:6789/0 conn(0x7f89d415e5f0 legacy=0x7f89d415d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.786+0000 7f89da1bd640 1 -- 172.21.15.119:0/2789617585 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d406bb30 con 0x7f89d415d410 2023-11-08T19:26:50.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.787+0000 7f89da1bd640 1 --2- 172.21.15.119:0/2789617585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415d410 0x7f89d40ce040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009860 tx=0x7f89d002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=656d5e4648dcc284 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.788+0000 7f89d91bb640 1 -- 172.21.15.119:0/2789617585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0031640 con 0x7f89d415d410 2023-11-08T19:26:50.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.788+0000 7f89d91bb640 1 -- 172.21.15.119:0/2789617585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d00317a0 con 0x7f89d415d410 2023-11-08T19:26:50.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.788+0000 7f89d91bb640 1 -- 172.21.15.119:0/2789617585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d0031b30 con 0x7f89d415d410 2023-11-08T19:26:50.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 -- 172.21.15.119:0/2789617585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415d410 msgr2=0x7f89d40ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 --2- 172.21.15.119:0/2789617585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415d410 0x7f89d40ce040 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f89d0009860 tx=0x7f89d002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 -- 172.21.15.119:0/2789617585 shutdown_connections 2023-11-08T19:26:50.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 --2- 172.21.15.119:0/2789617585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415d410 0x7f89d40ce040 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 -- 172.21.15.119:0/2789617585 >> 172.21.15.119:0/2789617585 conn(0x7f89d40d0db0 msgr2=0x7f89d40d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:50.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 -- 172.21.15.119:0/2789617585 shutdown_connections 2023-11-08T19:26:50.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.789+0000 7f89dc421640 1 -- 172.21.15.119:0/2789617585 wait complete. 2023-11-08T19:26:50.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.791+0000 7f89dc421640 1 Processor -- start 2023-11-08T19:26:50.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.791+0000 7f89dc421640 1 -- start start 2023-11-08T19:26:50.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89dc421640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89d415d410 0x7f89d40cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89dc421640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 0x7f89d40c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89dc421640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89d415e5f0 0x7f89d414f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89dc421640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d415eb60 con 0x7f89d415df00 2023-11-08T19:26:50.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89dc421640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40c7a00 con 0x7f89d415e5f0 2023-11-08T19:26:50.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89dc421640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89d40cfca0 con 0x7f89d415d410 2023-11-08T19:26:50.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89d99bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 0x7f89d40c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89d99bc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 0x7f89d40c74c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33528/0 (socket says 172.21.15.119:33528) 2023-11-08T19:26:50.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.792+0000 7f89d99bc640 1 -- 172.21.15.119:0/2249685896 learned_addr learned my addr 172.21.15.119:0/2249685896 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:50.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89da1bd640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89d415d410 0x7f89d40cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89da9be640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89d415e5f0 0x7f89d414f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89d99bc640 1 -- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89d415d410 msgr2=0x7f89d40cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89d99bc640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89d415d410 0x7f89d40cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89d99bc640 1 -- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89d415e5f0 msgr2=0x7f89d414f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:50.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89d99bc640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89d415e5f0 0x7f89d414f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:50.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89d99bc640 1 -- 172.21.15.119:0/2249685896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d0009470 con 0x7f89d415df00 2023-11-08T19:26:50.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89d99bc640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 0x7f89d40c74c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c3e0 tx=0x7f89c400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40074e0 con 0x7f89d415df00 2023-11-08T19:26:50.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c4007640 con 0x7f89d415df00 2023-11-08T19:26:50.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c40049e0 con 0x7f89d415df00 2023-11-08T19:26:50.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.793+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d40cff80 con 0x7f89d415df00 2023-11-08T19:26:50.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.794+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d414f700 con 0x7f89d415df00 2023-11-08T19:26:50.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.795+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d415ed30 con 0x7f89d415df00 2023-11-08T19:26:50.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.796+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89c4004b40 con 0x7f89d415df00 2023-11-08T19:26:50.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.796+0000 7f89caffd640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f89b806ae70 0x7f89b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:50.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.797+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6467+0+0 (secure 0 0 0) 0x7f89c4089300 con 0x7f89d415df00 2023-11-08T19:26:50.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.797+0000 7f89da1bd640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f89b806ae70 0x7f89b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:50.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.798+0000 7f89da1bd640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f89b806ae70 0x7f89b806d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89d0036000 tx=0x7f89d003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:50.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:50.801+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89d415ed30 con 0x7f89d415df00 2023-11-08T19:26:51.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.027+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f89d415ed30 con 0x7f89d415df00 2023-11-08T19:26:51.029 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-11-08T19:26:51.028+0000 7f8fd7f7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:26:51.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.031+0000 7f89caffd640 1 -- 172.21.15.119:0/2249685896 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) v1 ==== 161+0+0 (secure 0 0 0) 0x7f89d415ed30 con 0x7f89d415df00 2023-11-08T19:26:51.032 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:26:51.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.034+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f89b806ae70 msgr2=0x7f89b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.034+0000 7f89dc421640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f89b806ae70 0x7f89b806d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f89d0036000 tx=0x7f89d003c040 comp rx=0 tx=0).stop 2023-11-08T19:26:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.034+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 msgr2=0x7f89d40c74c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.034+0000 7f89dc421640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 0x7f89d40c74c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f89c400c3e0 tx=0x7f89c400c8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 shutdown_connections 2023-11-08T19:26:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f89b806ae70 0x7f89b806d330 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89d415e5f0 0x7f89d414f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89d415df00 0x7f89d40c74c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 --2- 172.21.15.119:0/2249685896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89d415d410 0x7f89d40cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 >> 172.21.15.119:0/2249685896 conn(0x7f89d40d0db0 msgr2=0x7f89d415c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:51.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 shutdown_connections 2023-11-08T19:26:51.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.035+0000 7f89dc421640 1 -- 172.21.15.119:0/2249685896 wait complete. 2023-11-08T19:26:51.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:26:51.056 INFO:tasks.ceph.mds.b.smithi188.stderr: -1869> 2023-11-08T19:26:33.142+0000 7f60437d6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:26:51.120 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:26:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.260+0000 7f0030da9640 1 Processor -- start 2023-11-08T19:26:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.260+0000 7f0030da9640 1 -- start start 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.261+0000 7f0030da9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c150350 0x7f002c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f0030da9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f002c0ca040 0x7f002c0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f0030da9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c15a990 con 0x7f002c0716a0 2023-11-08T19:26:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f0030da9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c0711a0 con 0x7f002c0ca040 2023-11-08T19:26:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f0030da9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c06fe30 con 0x7f002c150350 2023-11-08T19:26:51.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f002a59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c150350 0x7f002c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f0029d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f002c0716a0 0x7f002c0c9930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:37780/0 (socket says 172.21.15.119:37780) 2023-11-08T19:26:51.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f0029d9b640 1 -- 172.21.15.119:0/2856784145 learned_addr learned my addr 172.21.15.119:0/2856784145 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.262+0000 7f002ad9d640 1 --2- 172.21.15.119:0/2856784145 >> v2:172.21.15.188:3300/0 conn(0x7f002c0ca040 0x7f002c0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.263+0000 7f002959a640 1 -- 172.21.15.119:0/2856784145 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435889075 0 0) 0x7f002c15a990 con 0x7f002c0716a0 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.263+0000 7f002959a640 1 -- 172.21.15.119:0/2856784145 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0018003520 con 0x7f002c0716a0 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.263+0000 7f002a59c640 1 -- 172.21.15.119:0/2856784145 >> v2:172.21.15.188:3300/0 conn(0x7f002c0ca040 msgr2=0x7f002c0cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:51.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.263+0000 7f002a59c640 1 --2- 172.21.15.119:0/2856784145 >> v2:172.21.15.188:3300/0 conn(0x7f002c0ca040 0x7f002c0cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.263+0000 7f002a59c640 1 -- 172.21.15.119:0/2856784145 >> v1:172.21.15.188:6789/0 conn(0x7f002c0716a0 legacy=0x7f002c0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.263+0000 7f002a59c640 1 -- 172.21.15.119:0/2856784145 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c070100 con 0x7f002c150350 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.265+0000 7f002a59c640 1 --2- 172.21.15.119:0/2856784145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c150350 0x7f002c15a450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f001c00bfa0 tx=0x7f001c02e0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eb50124ffe8a4d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.265+0000 7f002959a640 1 -- 172.21.15.119:0/2856784145 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c02ea40 con 0x7f002c150350 2023-11-08T19:26:51.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.265+0000 7f002959a640 1 -- 172.21.15.119:0/2856784145 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001c02eba0 con 0x7f002c150350 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.265+0000 7f002959a640 1 -- 172.21.15.119:0/2856784145 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001c009540 con 0x7f002c150350 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 -- 172.21.15.119:0/2856784145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c150350 msgr2=0x7f002c15a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 --2- 172.21.15.119:0/2856784145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c150350 0x7f002c15a450 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f001c00bfa0 tx=0x7f001c02e0a0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 -- 172.21.15.119:0/2856784145 shutdown_connections 2023-11-08T19:26:51.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 --2- 172.21.15.119:0/2856784145 >> v2:172.21.15.188:3300/0 conn(0x7f002c0ca040 0x7f002c0cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 --2- 172.21.15.119:0/2856784145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c150350 0x7f002c15a450 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 -- 172.21.15.119:0/2856784145 >> 172.21.15.119:0/2856784145 conn(0x7f002c0b0950 msgr2=0x7f002c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 -- 172.21.15.119:0/2856784145 shutdown_connections 2023-11-08T19:26:51.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.266+0000 7f0030da9640 1 -- 172.21.15.119:0/2856784145 wait complete. 2023-11-08T19:26:51.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.268+0000 7f0030da9640 1 Processor -- start 2023-11-08T19:26:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.268+0000 7f0030da9640 1 -- start start 2023-11-08T19:26:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0030da9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0716a0 0x7f002c14ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0030da9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 0x7f002c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0030da9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c150350 0x7f002c152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0030da9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c15aed0 con 0x7f002c0ca040 2023-11-08T19:26:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0030da9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0c6b20 con 0x7f002c150350 2023-11-08T19:26:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0030da9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0c5fa0 con 0x7f002c0716a0 2023-11-08T19:26:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0029d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 0x7f002c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0029d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 0x7f002c0c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33538/0 (socket says 172.21.15.119:33538) 2023-11-08T19:26:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0029d9b640 1 -- 172.21.15.119:0/2174818919 learned_addr learned my addr 172.21.15.119:0/2174818919 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f002a59c640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0716a0 0x7f002c14ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0029d9b640 1 -- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0716a0 msgr2=0x7f002c14ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f002ad9d640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c150350 0x7f002c152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.269+0000 7f0029d9b640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0716a0 0x7f002c14ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0029d9b640 1 -- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c150350 msgr2=0x7f002c152970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0029d9b640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c150350 0x7f002c152970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0029d9b640 1 -- 172.21.15.119:0/2174818919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c00bc80 con 0x7f002c0ca040 2023-11-08T19:26:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f002ad9d640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c150350 0x7f002c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0029d9b640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 0x7f002c0c8210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f001400c4d0 tx=0x7f001400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014005ce0 con 0x7f002c0ca040 2023-11-08T19:26:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0014005e40 con 0x7f002c0ca040 2023-11-08T19:26:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014007270 con 0x7f002c0ca040 2023-11-08T19:26:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c0c6280 con 0x7f002c0ca040 2023-11-08T19:26:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.270+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c0c6840 con 0x7f002c0ca040 2023-11-08T19:26:51.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.271+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002c15b040 con 0x7f002c0ca040 2023-11-08T19:26:51.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.272+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0014007490 con 0x7f002c0ca040 2023-11-08T19:26:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.273+0000 7f0022ffd640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f000c06ae70 0x7f000c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.273+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6525+0+0 (secure 0 0 0) 0x7f0014088d80 con 0x7f002c0ca040 2023-11-08T19:26:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.273+0000 7f002a59c640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f000c06ae70 0x7f000c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:51.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.274+0000 7f002a59c640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f000c06ae70 0x7f000c06d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f001c037000 tx=0x7f001c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:51.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.278+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f002c15b040 con 0x7f002c0ca040 2023-11-08T19:26:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:51.496+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f002c15b010 con 0x7f002c0ca040 2023-11-08T19:26:52.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.034+0000 7f0022ffd640 1 -- 172.21.15.119:0/2174818919 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7f00140170a0 con 0x7f002c0ca040 2023-11-08T19:26:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f000c06ae70 msgr2=0x7f000c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f000c06ae70 0x7f000c06d330 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f001c037000 tx=0x7f001c032000 comp rx=0 tx=0).stop 2023-11-08T19:26:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 msgr2=0x7f002c0c8210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 0x7f002c0c8210 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f001400c4d0 tx=0x7f001400c990 comp rx=0 tx=0).stop 2023-11-08T19:26:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 shutdown_connections 2023-11-08T19:26:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f000c06ae70 0x7f000c06d330 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f002c150350 0x7f002c152970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f002c0ca040 0x7f002c0c8210 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 --2- 172.21.15.119:0/2174818919 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f002c0716a0 0x7f002c14ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.037+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 >> 172.21.15.119:0/2174818919 conn(0x7f002c0b0950 msgr2=0x7f002c14f610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:52.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.038+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 shutdown_connections 2023-11-08T19:26:52.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.038+0000 7f0030da9640 1 -- 172.21.15.119:0/2174818919 wait complete. 2023-11-08T19:26:52.059 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:26:52.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.279+0000 7efd5700d640 1 Processor -- start 2023-11-08T19:26:52.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.280+0000 7efd5700d640 1 -- start start 2023-11-08T19:26:52.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.281+0000 7efd5700d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd500cfb80 0x7efd500cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.281+0000 7efd5700d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd50160a00 0x7efd500b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.281+0000 7efd5700d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd500d16e0 con 0x7efd500d04a0 2023-11-08T19:26:52.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.281+0000 7efd5700d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd500d0990 con 0x7efd500cfb80 2023-11-08T19:26:52.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.281+0000 7efd5700d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500d1980 con 0x7efd50160a00 2023-11-08T19:26:52.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.282+0000 7efd5680c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd50160a00 0x7efd500b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.282+0000 7efd5680c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd50160a00 0x7efd500b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:44854/0 (socket says 172.21.15.119:44854) 2023-11-08T19:26:52.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.282+0000 7efd5680c640 1 -- 172.21.15.119:0/2960340369 learned_addr learned my addr 172.21.15.119:0/2960340369 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:52.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.282+0000 7efd5600b640 1 --2- 172.21.15.119:0/2960340369 >> v2:172.21.15.188:3300/0 conn(0x7efd500cfb80 0x7efd500cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.283+0000 7efd5600b640 1 -- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd50160a00 msgr2=0x7efd500b6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:52.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.283+0000 7efd5600b640 1 --2- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd50160a00 0x7efd500b6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.283+0000 7efd5600b640 1 -- 172.21.15.119:0/2960340369 >> v1:172.21.15.188:6789/0 conn(0x7efd500d04a0 legacy=0x7efd501602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.283+0000 7efd5600b640 1 -- 172.21.15.119:0/2960340369 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500d1c60 con 0x7efd500cfb80 2023-11-08T19:26:52.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.284+0000 7efd5600b640 1 --2- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd500cfb80 0x7efd500cff60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0097a0 tx=0x7efd4c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce7ac9d904cbc803 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:52.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.284+0000 7efd55009640 1 -- 172.21.15.119:0/2960340369 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c02d040 con 0x7efd500cfb80 2023-11-08T19:26:52.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.284+0000 7efd55009640 1 -- 172.21.15.119:0/2960340369 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4c031810 con 0x7efd500cfb80 2023-11-08T19:26:52.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.284+0000 7efd55009640 1 -- 172.21.15.119:0/2960340369 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4c031ba0 con 0x7efd500cfb80 2023-11-08T19:26:52.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 -- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd500cfb80 msgr2=0x7efd500cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 --2- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd500cfb80 0x7efd500cff60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efd4c0097a0 tx=0x7efd4c02b8b0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 -- 172.21.15.119:0/2960340369 shutdown_connections 2023-11-08T19:26:52.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 --2- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd50160a00 0x7efd500b6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 --2- 172.21.15.119:0/2960340369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd500cfb80 0x7efd500cff60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 -- 172.21.15.119:0/2960340369 >> 172.21.15.119:0/2960340369 conn(0x7efd500d2f60 msgr2=0x7efd500d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:52.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 -- 172.21.15.119:0/2960340369 shutdown_connections 2023-11-08T19:26:52.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.285+0000 7efd5700d640 1 -- 172.21.15.119:0/2960340369 wait complete. 2023-11-08T19:26:52.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.287+0000 7efd5700d640 1 Processor -- start 2023-11-08T19:26:52.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.287+0000 7efd5700d640 1 -- start start 2023-11-08T19:26:52.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5700d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd500cfb80 0x7efd50155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5700d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 0x7efd5015f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5700d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd50160a00 0x7efd500b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5700d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd50155a60 con 0x7efd500d04a0 2023-11-08T19:26:52.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5700d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500cca90 con 0x7efd50160a00 2023-11-08T19:26:52.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5700d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500cbe80 con 0x7efd500cfb80 2023-11-08T19:26:52.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 0x7efd5015f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 0x7efd5015f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33542/0 (socket says 172.21.15.119:33542) 2023-11-08T19:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 -- 172.21.15.119:0/2797714358 learned_addr learned my addr 172.21.15.119:0/2797714358 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5680c640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd50160a00 0x7efd500b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5600b640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd500cfb80 0x7efd50155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 -- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd500cfb80 msgr2=0x7efd50155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd500cfb80 0x7efd50155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 -- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd50160a00 msgr2=0x7efd500b7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:52.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd50160a00 0x7efd500b7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:52.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.289+0000 7efd5580a640 1 -- 172.21.15.119:0/2797714358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009400 con 0x7efd500d04a0 2023-11-08T19:26:52.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.290+0000 7efd5580a640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 0x7efd5015f5e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efd4400c400 tx=0x7efd4400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:52.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.290+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd440052d0 con 0x7efd500d04a0 2023-11-08T19:26:52.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.290+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd44005450 con 0x7efd500d04a0 2023-11-08T19:26:52.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.290+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd500cc160 con 0x7efd500d04a0 2023-11-08T19:26:52.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.290+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd44007260 con 0x7efd500d04a0 2023-11-08T19:26:52.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.290+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd500cc720 con 0x7efd500d04a0 2023-11-08T19:26:52.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.291+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5004ece0 con 0x7efd500d04a0 2023-11-08T19:26:52.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.292+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd440073c0 con 0x7efd500d04a0 2023-11-08T19:26:52.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.293+0000 7efd42ffd640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efd3406ae70 0x7efd3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:52.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.294+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6525+0+0 (secure 0 0 0) 0x7efd44088e10 con 0x7efd500d04a0 2023-11-08T19:26:52.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.294+0000 7efd5600b640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efd3406ae70 0x7efd3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:52.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.294+0000 7efd5600b640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efd3406ae70 0x7efd3406d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efd4c036000 tx=0x7efd4c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:52.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.297+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd4408c030 con 0x7efd500d04a0 2023-11-08T19:26:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:52.487+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efd5006baa0 con 0x7efd500d04a0 2023-11-08T19:26:53.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.044+0000 7efd42ffd640 1 -- 172.21.15.119:0/2797714358 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7efd44053af0 con 0x7efd500d04a0 2023-11-08T19:26:53.046 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:26:53.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.048+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efd3406ae70 msgr2=0x7efd3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.048+0000 7efd5700d640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efd3406ae70 0x7efd3406d330 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efd4c036000 tx=0x7efd4c03c040 comp rx=0 tx=0).stop 2023-11-08T19:26:53.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 msgr2=0x7efd5015f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 0x7efd5015f5e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efd4400c400 tx=0x7efd4400c8c0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 shutdown_connections 2023-11-08T19:26:53.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efd3406ae70 0x7efd3406d330 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7efd4c036000 tx=0x7efd4c03c040 comp rx=0 tx=0).stop 2023-11-08T19:26:53.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd50160a00 0x7efd500b7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efd500d04a0 0x7efd5015f5e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 --2- 172.21.15.119:0/2797714358 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd500cfb80 0x7efd50155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.049+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 >> 172.21.15.119:0/2797714358 conn(0x7efd500d2f60 msgr2=0x7efd5006bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:53.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.050+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 shutdown_connections 2023-11-08T19:26:53.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.050+0000 7efd5700d640 1 -- 172.21.15.119:0/2797714358 wait complete. 2023-11-08T19:26:53.071 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:26:53.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.282+0000 7fdc8359e640 1 Processor -- start 2023-11-08T19:26:53.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.282+0000 7fdc8359e640 1 -- start start 2023-11-08T19:26:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.283+0000 7fdc8359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8415d2e0 0x7fdc840d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.283+0000 7fdc8359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc840d0d40 0x7fdc840d1a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.283+0000 7fdc8359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc840ce610 0x7fdc840cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.283+0000 7fdc8359e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc840b2fb0 con 0x7fdc840ce610 2023-11-08T19:26:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.283+0000 7fdc8359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc840ceeb0 con 0x7fdc840d0d40 2023-11-08T19:26:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.283+0000 7fdc8359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840cebd0 con 0x7fdc8415d2e0 2023-11-08T19:26:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc840ce610 0x7fdc840cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc840ce610 0x7fdc840cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33544/0 (socket says 172.21.15.119:33544) 2023-11-08T19:26:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 -- 172.21.15.119:0/4036950189 learned_addr learned my addr 172.21.15.119:0/4036950189 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:53.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc8259c640 1 --2- 172.21.15.119:0/4036950189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8415d2e0 0x7fdc840d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc81d9b640 1 --2- 172.21.15.119:0/4036950189 >> v2:172.21.15.188:3300/0 conn(0x7fdc840d0d40 0x7fdc840d1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 -- 172.21.15.119:0/4036950189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8415d2e0 msgr2=0x7fdc840d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 --2- 172.21.15.119:0/4036950189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8415d2e0 0x7fdc840d1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 -- 172.21.15.119:0/4036950189 >> v2:172.21.15.188:3300/0 conn(0x7fdc840d0d40 msgr2=0x7fdc840d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 --2- 172.21.15.119:0/4036950189 >> v2:172.21.15.188:3300/0 conn(0x7fdc840d0d40 0x7fdc840d1a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.284+0000 7fdc82d9d640 1 -- 172.21.15.119:0/4036950189 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc840b2960 con 0x7fdc840ce610 2023-11-08T19:26:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.285+0000 7fdc82d9d640 1 --2- 172.21.15.119:0/4036950189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc840ce610 0x7fdc840cf620 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdc84066df0 tx=0x7fdc7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a1e65d03afdf67 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8159a640 1 -- 172.21.15.119:0/4036950189 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c030860 con 0x7fdc840ce610 2023-11-08T19:26:53.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8159a640 1 -- 172.21.15.119:0/4036950189 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c0309c0 con 0x7fdc840ce610 2023-11-08T19:26:53.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8159a640 1 -- 172.21.15.119:0/4036950189 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c030d50 con 0x7fdc840ce610 2023-11-08T19:26:53.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 -- 172.21.15.119:0/4036950189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc840ce610 msgr2=0x7fdc840cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 --2- 172.21.15.119:0/4036950189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc840ce610 0x7fdc840cf620 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdc84066df0 tx=0x7fdc7c02b9a0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 -- 172.21.15.119:0/4036950189 shutdown_connections 2023-11-08T19:26:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 --2- 172.21.15.119:0/4036950189 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc840ce610 0x7fdc840cf620 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 --2- 172.21.15.119:0/4036950189 >> v2:172.21.15.188:3300/0 conn(0x7fdc840d0d40 0x7fdc840d1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 --2- 172.21.15.119:0/4036950189 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8415d2e0 0x7fdc840d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 -- 172.21.15.119:0/4036950189 >> 172.21.15.119:0/4036950189 conn(0x7fdc840d2f60 msgr2=0x7fdc840d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:53.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 -- 172.21.15.119:0/4036950189 shutdown_connections 2023-11-08T19:26:53.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.286+0000 7fdc8359e640 1 -- 172.21.15.119:0/4036950189 wait complete. 2023-11-08T19:26:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.288+0000 7fdc8359e640 1 Processor -- start 2023-11-08T19:26:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.288+0000 7fdc8359e640 1 -- start start 2023-11-08T19:26:53.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc8359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840ce610 0x7fdc84150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc8359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840d0d40 0x7fdc8415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc8359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 0x7fdc841e6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc8359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840c8670 con 0x7fdc8415d2e0 2023-11-08T19:26:53.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc8359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840ca1f0 con 0x7fdc840d0d40 2023-11-08T19:26:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc8359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc840c9120 con 0x7fdc840ce610 2023-11-08T19:26:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc82d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 0x7fdc841e6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc82d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 0x7fdc841e6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33560/0 (socket says 172.21.15.119:33560) 2023-11-08T19:26:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.289+0000 7fdc82d9d640 1 -- 172.21.15.119:0/2624706047 learned_addr learned my addr 172.21.15.119:0/2624706047 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc8259c640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840ce610 0x7fdc84150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc81d9b640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840d0d40 0x7fdc8415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc82d9d640 1 -- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840ce610 msgr2=0x7fdc84150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc82d9d640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840ce610 0x7fdc84150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc82d9d640 1 -- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840d0d40 msgr2=0x7fdc8415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc82d9d640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840d0d40 0x7fdc8415a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc82d9d640 1 -- 172.21.15.119:0/2624706047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c0094d0 con 0x7fdc8415d2e0 2023-11-08T19:26:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc82d9d640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 0x7fdc841e6b10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c02b9e0 tx=0x7fdc7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c0307f0 con 0x7fdc8415d2e0 2023-11-08T19:26:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc7c039d90 con 0x7fdc8415d2e0 2023-11-08T19:26:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7c043650 con 0x7fdc8415d2e0 2023-11-08T19:26:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.290+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc84151850 con 0x7fdc8415d2e0 2023-11-08T19:26:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.291+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc840c8b60 con 0x7fdc8415d2e0 2023-11-08T19:26:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.292+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc840c68c0 con 0x7fdc8415d2e0 2023-11-08T19:26:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.293+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc7c04c3b0 con 0x7fdc8415d2e0 2023-11-08T19:26:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.293+0000 7fdc7affd640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6406ae70 0x7fdc6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.294+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6087+0+0 (secure 0 0 0) 0x7fdc7c0abf10 con 0x7fdc8415d2e0 2023-11-08T19:26:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.294+0000 7fdc8259c640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6406ae70 0x7fdc6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.294+0000 7fdc8259c640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6406ae70 0x7fdc6406d330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00a970 tx=0x7fdc6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.298+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc7c076e30 con 0x7fdc8415d2e0 2023-11-08T19:26:53.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.489+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc8404ece0 con 0x7fdc8415d2e0 2023-11-08T19:26:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.624+0000 7fdc7affd640 1 -- 172.21.15.119:0/2624706047 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdc7c07a450 con 0x7fdc8415d2e0 2023-11-08T19:26:53.626 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-11-08T19:26:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.628+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6406ae70 msgr2=0x7fdc6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.628+0000 7fdc8359e640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6406ae70 0x7fdc6406d330 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00a970 tx=0x7fdc6c008040 comp rx=0 tx=0).stop 2023-11-08T19:26:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.628+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 msgr2=0x7fdc841e6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.628+0000 7fdc8359e640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 0x7fdc841e6b10 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdc7c02b9e0 tx=0x7fdc7c004730 comp rx=0 tx=0).stop 2023-11-08T19:26:53.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.629+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 shutdown_connections 2023-11-08T19:26:53.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.629+0000 7fdc8359e640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6406ae70 0x7fdc6406d330 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.629+0000 7fdc8359e640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8415d2e0 0x7fdc841e6b10 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.629+0000 7fdc8359e640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc840d0d40 0x7fdc8415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.629+0000 7fdc8359e640 1 --2- 172.21.15.119:0/2624706047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc840ce610 0x7fdc84150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.629+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 >> 172.21.15.119:0/2624706047 conn(0x7fdc840d2f60 msgr2=0x7fdc8415c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:53.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.630+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 shutdown_connections 2023-11-08T19:26:53.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.630+0000 7fdc8359e640 1 -- 172.21.15.119:0/2624706047 wait complete. 2023-11-08T19:26:53.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:26:53.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.863+0000 7feaa4b00640 1 Processor -- start 2023-11-08T19:26:53.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.863+0000 7feaa4b00640 1 -- start start 2023-11-08T19:26:53.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.864+0000 7feaa4b00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa00d1b90 0x7feaa00ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.864+0000 7feaa4b00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feaa00ce990 0x7feaa00b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.864+0000 7feaa4b00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa00b6470 con 0x7feaa015d000 2023-11-08T19:26:53.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.864+0000 7feaa4b00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa00cf180 con 0x7feaa00ce990 2023-11-08T19:26:53.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.864+0000 7feaa4b00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00cf460 con 0x7feaa00d1b90 2023-11-08T19:26:53.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feaa00ce990 0x7feaa00b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feaa00ce990 0x7feaa00b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33576/0 (socket says 172.21.15.119:33576) 2023-11-08T19:26:53.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 -- 172.21.15.119:0/3264390291 learned_addr learned my addr 172.21.15.119:0/3264390291 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 -- 172.21.15.119:0/3264390291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa00d1b90 msgr2=0x7feaa00ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 --2- 172.21.15.119:0/3264390291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa00d1b90 0x7feaa00ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 -- 172.21.15.119:0/3264390291 >> v1:172.21.15.188:6789/0 conn(0x7feaa015d000 legacy=0x7feaa00d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:53.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.865+0000 7fea9ed9d640 1 -- 172.21.15.119:0/3264390291 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa00b5e20 con 0x7feaa00ce990 2023-11-08T19:26:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.866+0000 7fea9ed9d640 1 --2- 172.21.15.119:0/3264390291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00ce990 0x7feaa00b6900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea98009870 tx=0x7fea9802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d3bcfe92ec8ec9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7fea9dd9b640 1 -- 172.21.15.119:0/3264390291 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98031640 con 0x7feaa00ce990 2023-11-08T19:26:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7fea9dd9b640 1 -- 172.21.15.119:0/3264390291 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea980317a0 con 0x7feaa00ce990 2023-11-08T19:26:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7fea9dd9b640 1 -- 172.21.15.119:0/3264390291 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea98031b30 con 0x7feaa00ce990 2023-11-08T19:26:53.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7feaa4b00640 1 -- 172.21.15.119:0/3264390291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00ce990 msgr2=0x7feaa00b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7feaa4b00640 1 --2- 172.21.15.119:0/3264390291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00ce990 0x7feaa00b6900 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea98009870 tx=0x7fea9802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7feaa4b00640 1 -- 172.21.15.119:0/3264390291 shutdown_connections 2023-11-08T19:26:53.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7feaa4b00640 1 --2- 172.21.15.119:0/3264390291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00ce990 0x7feaa00b6900 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7feaa4b00640 1 --2- 172.21.15.119:0/3264390291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa00d1b90 0x7feaa00ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.867+0000 7feaa4b00640 1 -- 172.21.15.119:0/3264390291 >> 172.21.15.119:0/3264390291 conn(0x7feaa009c260 msgr2=0x7feaa00d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:53.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.868+0000 7feaa4b00640 1 -- 172.21.15.119:0/3264390291 shutdown_connections 2023-11-08T19:26:53.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.868+0000 7feaa4b00640 1 -- 172.21.15.119:0/3264390291 wait complete. 2023-11-08T19:26:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.869+0000 7feaa4b00640 1 Processor -- start 2023-11-08T19:26:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.869+0000 7feaa4b00640 1 -- start start 2023-11-08T19:26:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.870+0000 7feaa4b00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaa00ce990 0x7feaa0156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.870+0000 7feaa4b00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 0x7feaa00cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.870+0000 7feaa4b00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa015d000 0x7feaa00cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.870+0000 7feaa4b00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa00ca320 con 0x7feaa00d1b90 2023-11-08T19:26:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.870+0000 7feaa4b00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa01f3970 con 0x7feaa00ce990 2023-11-08T19:26:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.870+0000 7feaa4b00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa01f3c50 con 0x7feaa015d000 2023-11-08T19:26:53.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 0x7feaa00cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 0x7feaa00cc140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33584/0 (socket says 172.21.15.119:33584) 2023-11-08T19:26:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 -- 172.21.15.119:0/948772251 learned_addr learned my addr 172.21.15.119:0/948772251 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea9e59c640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaa00ce990 0x7feaa0156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea9ed9d640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa015d000 0x7feaa00cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 -- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa015d000 msgr2=0x7feaa00cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa015d000 0x7feaa00cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 -- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaa00ce990 msgr2=0x7feaa0156a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaa00ce990 0x7feaa0156a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 -- 172.21.15.119:0/948772251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea980094d0 con 0x7feaa00d1b90 2023-11-08T19:26:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea9ed9d640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa015d000 0x7feaa00cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:53.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.871+0000 7fea95d9b640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 0x7feaa00cc140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea8800ba50 tx=0x7fea8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.872+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800cbe0 con 0x7feaa00d1b90 2023-11-08T19:26:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.872+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800cd60 con 0x7feaa00d1b90 2023-11-08T19:26:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.872+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88014680 con 0x7feaa00d1b90 2023-11-08T19:26:53.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.872+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa01f3f30 con 0x7feaa00d1b90 2023-11-08T19:26:53.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.872+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa0156f90 con 0x7feaa00d1b90 2023-11-08T19:26:53.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.873+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa004ece0 con 0x7feaa00d1b90 2023-11-08T19:26:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.874+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea8801d3b0 con 0x7feaa00d1b90 2023-11-08T19:26:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.875+0000 7fea977fe640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.875+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fea88089620 con 0x7feaa00d1b90 2023-11-08T19:26:53.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.875+0000 7fea9e59c640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:53.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.876+0000 7fea9e59c640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea8006ae70 0x7fea8006d330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fea90004e50 tx=0x7fea90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:53.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:53.879+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea88054670 con 0x7feaa00d1b90 2023-11-08T19:26:54.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.072+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7feaa00d3640 con 0x7feaa00d1b90 2023-11-08T19:26:54.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.078+0000 7fea977fe640 1 -- 172.21.15.119:0/948772251 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v12) v1 ==== 104+0+0 (secure 0 0 0) 0x7fea88057c90 con 0x7feaa00d1b90 2023-11-08T19:26:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.080+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea8006ae70 msgr2=0x7fea8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.080+0000 7feaa4b00640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea8006ae70 0x7fea8006d330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fea90004e50 tx=0x7fea90009040 comp rx=0 tx=0).stop 2023-11-08T19:26:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.080+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 msgr2=0x7feaa00cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.080+0000 7feaa4b00640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 0x7feaa00cc140 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea8800ba50 tx=0x7fea8800bf10 comp rx=0 tx=0).stop 2023-11-08T19:26:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.080+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 shutdown_connections 2023-11-08T19:26:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fea8006ae70 0x7fea8006d330 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feaa015d000 0x7feaa00cc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feaa00d1b90 0x7feaa00cc140 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 --2- 172.21.15.119:0/948772251 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feaa00ce990 0x7feaa0156a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 >> 172.21.15.119:0/948772251 conn(0x7feaa009c260 msgr2=0x7feaa00b72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 shutdown_connections 2023-11-08T19:26:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.081+0000 7feaa4b00640 1 -- 172.21.15.119:0/948772251 wait complete. 2023-11-08T19:26:54.099 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails' 2023-11-08T19:26:54.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.307+0000 7f01bab97640 1 Processor -- start 2023-11-08T19:26:54.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.307+0000 7f01bab97640 1 -- start start 2023-11-08T19:26:54.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.308+0000 7f01bab97640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01b41447a0 0x7f01b415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01bab97640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01b406a120 0x7f01b406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01bab97640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b406aa70 con 0x7f01b40cad30 2023-11-08T19:26:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01bab97640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b4070040 con 0x7f01b406a120 2023-11-08T19:26:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01bab97640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40c3a50 con 0x7f01b41447a0 2023-11-08T19:26:54.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01b406a120 0x7f01b406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01b406a120 0x7f01b406a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33586/0 (socket says 172.21.15.119:33586) 2023-11-08T19:26:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 -- 172.21.15.119:0/3635468585 learned_addr learned my addr 172.21.15.119:0/3635468585 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 -- 172.21.15.119:0/3635468585 >> v2:172.21.15.188:3300/0 conn(0x7f01b41447a0 msgr2=0x7f01b415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:54.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 --2- 172.21.15.119:0/3635468585 >> v2:172.21.15.188:3300/0 conn(0x7f01b41447a0 0x7f01b415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 -- 172.21.15.119:0/3635468585 >> v1:172.21.15.188:6789/0 conn(0x7f01b40cad30 legacy=0x7f01b406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:54.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.309+0000 7f01b9134640 1 -- 172.21.15.119:0/3635468585 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4144b80 con 0x7f01b406a120 2023-11-08T19:26:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.310+0000 7f01b9134640 1 --2- 172.21.15.119:0/3635468585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b406a120 0x7f01b406a530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b1850 tx=0x7f01ac02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7261c82b99b3dca5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01b37fe640 1 -- 172.21.15.119:0/3635468585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac02ca70 con 0x7f01b406a120 2023-11-08T19:26:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01b37fe640 1 -- 172.21.15.119:0/3635468585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac02cbd0 con 0x7f01b406a120 2023-11-08T19:26:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01b37fe640 1 -- 172.21.15.119:0/3635468585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac0343f0 con 0x7f01b406a120 2023-11-08T19:26:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 -- 172.21.15.119:0/3635468585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b406a120 msgr2=0x7f01b406a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 --2- 172.21.15.119:0/3635468585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b406a120 0x7f01b406a530 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f01b40b1850 tx=0x7f01ac02bed0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 -- 172.21.15.119:0/3635468585 shutdown_connections 2023-11-08T19:26:54.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 --2- 172.21.15.119:0/3635468585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b406a120 0x7f01b406a530 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 --2- 172.21.15.119:0/3635468585 >> v2:172.21.15.188:3300/0 conn(0x7f01b41447a0 0x7f01b415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 -- 172.21.15.119:0/3635468585 >> 172.21.15.119:0/3635468585 conn(0x7f01b40cb610 msgr2=0x7f01b40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 -- 172.21.15.119:0/3635468585 shutdown_connections 2023-11-08T19:26:54.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.311+0000 7f01bab97640 1 -- 172.21.15.119:0/3635468585 wait complete. 2023-11-08T19:26:54.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.313+0000 7f01bab97640 1 Processor -- start 2023-11-08T19:26:54.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.313+0000 7f01bab97640 1 -- start start 2023-11-08T19:26:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01bab97640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b406a120 0x7f01b41f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01bab97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 0x7f01b40b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01bab97640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b41447a0 0x7f01b415c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01bab97640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c6d10 con 0x7f01b40cad30 2023-11-08T19:26:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01bab97640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40b1c10 con 0x7f01b406a120 2023-11-08T19:26:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01bab97640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40b1ef0 con 0x7f01b41447a0 2023-11-08T19:26:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 0x7f01b40b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 0x7f01b40b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33594/0 (socket says 172.21.15.119:33594) 2023-11-08T19:26:54.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.314+0000 7f01b3fff640 1 -- 172.21.15.119:0/498281865 learned_addr learned my addr 172.21.15.119:0/498281865 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b8933640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b406a120 0x7f01b41f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b3fff640 1 -- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b41447a0 msgr2=0x7f01b415c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b3fff640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b41447a0 0x7f01b415c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b3fff640 1 -- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b406a120 msgr2=0x7f01b41f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b3fff640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b406a120 0x7f01b41f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b3fff640 1 -- 172.21.15.119:0/498281865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac009b30 con 0x7f01b40cad30 2023-11-08T19:26:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b8933640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b406a120 0x7f01b41f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b3fff640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 0x7f01b40b4580 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b3e0 tx=0x7f01a000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011830 con 0x7f01b40cad30 2023-11-08T19:26:54.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a00119b0 con 0x7f01b40cad30 2023-11-08T19:26:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40b21d0 con 0x7f01b40cad30 2023-11-08T19:26:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011d60 con 0x7f01b40cad30 2023-11-08T19:26:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.315+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40b1830 con 0x7f01b40cad30 2023-11-08T19:26:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.317+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b40c6e80 con 0x7f01b40cad30 2023-11-08T19:26:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.317+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01a001a400 con 0x7f01b40cad30 2023-11-08T19:26:54.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.318+0000 7f01b17fa640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f019806ae70 0x7f019806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.318+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f01a00897a0 con 0x7f01b40cad30 2023-11-08T19:26:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.318+0000 7f01b8933640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f019806ae70 0x7f019806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.319+0000 7f01b8933640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f019806ae70 0x7f019806d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f01a8006fd0 tx=0x7f01a8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:54.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.322+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01b40c6e80 con 0x7f01b40cad30 2023-11-08T19:26:54.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.586+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]} v 0) v1 -- 0x7f01b40c6e50 con 0x7f01b40cad30 2023-11-08T19:26:54.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.799+0000 7f01b17fa640 1 -- 172.21.15.119:0/498281865 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_attaching_pools_with_snaps_to_fs_fails"]}]=0 v135) v1 ==== 173+0+0 (secure 0 0 0) 0x7f01a00150f0 con 0x7f01b40cad30 2023-11-08T19:26:54.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f019806ae70 msgr2=0x7f019806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f019806ae70 0x7f019806d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f01a8006fd0 tx=0x7f01a8008040 comp rx=0 tx=0).stop 2023-11-08T19:26:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 msgr2=0x7f01b40b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 0x7f01b40b4580 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f01a000b3e0 tx=0x7f01a000b8a0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 shutdown_connections 2023-11-08T19:26:54.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f019806ae70 0x7f019806d330 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01b41447a0 0x7f01b415c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01b40cad30 0x7f01b40b4580 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 --2- 172.21.15.119:0/498281865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01b406a120 0x7f01b41f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:54.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 >> 172.21.15.119:0/498281865 conn(0x7f01b40cb610 msgr2=0x7f01b40c4210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:54.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 shutdown_connections 2023-11-08T19:26:54.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:54.802+0000 7f01bab97640 1 -- 172.21.15.119:0/498281865 wait complete. 2023-11-08T19:26:54.821 INFO:tasks.cephfs_test_runner:Test that attempt to attach pool with snapshots to an fs fails ... ok 2023-11-08T19:26:54.822 INFO:tasks.cephfs_test_runner:Starting test: test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that creation of monitor managed snaps fails for pools attached 2023-11-08T19:26:54.822 INFO:tasks.cephfs_test_runner:test_disallow_monitor_managed_snaps_for_fs_pools (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-08T19:26:54.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:26:55.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.018+0000 7f19307a9640 1 Processor -- start 2023-11-08T19:26:55.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.018+0000 7f19307a9640 1 -- start start 2023-11-08T19:26:55.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.019+0000 7f19307a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f192806a120 0x7f192806a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.019+0000 7f19307a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192806aa70 con 0x7f19281447a0 2023-11-08T19:26:55.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f19307a9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19280c3a50 con 0x7f192815f4c0 2023-11-08T19:26:55.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f19307a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192815f930 con 0x7f192806a120 2023-11-08T19:26:55.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f192e545640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f19281447a0 0x7f19280cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48124/0 (socket says 172.21.15.119:48124) 2023-11-08T19:26:55.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f192e545640 1 -- 172.21.15.119:0/779489948 learned_addr learned my addr 172.21.15.119:0/779489948 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:55.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f192ed46640 1 --2- 172.21.15.119:0/779489948 >> v2:172.21.15.119:3300/0 conn(0x7f192806a120 0x7f192806a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f192ed46640 1 -- 172.21.15.119:0/779489948 >> v1:172.21.15.188:6789/0 conn(0x7f192815f4c0 legacy=0x7f192806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f192ed46640 1 -- 172.21.15.119:0/779489948 >> v1:172.21.15.119:6789/0 conn(0x7f19281447a0 legacy=0x7f19280cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.020+0000 7f192ed46640 1 -- 172.21.15.119:0/779489948 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1928144b80 con 0x7f192806a120 2023-11-08T19:26:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.021+0000 7f192ed46640 1 --2- 172.21.15.119:0/779489948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192806a120 0x7f192806a530 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f192815fa70 tx=0x7f192002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3c8385d6cf081b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f192d543640 1 -- 172.21.15.119:0/779489948 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192002cc60 con 0x7f192806a120 2023-11-08T19:26:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f192d543640 1 -- 172.21.15.119:0/779489948 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192002cdc0 con 0x7f192806a120 2023-11-08T19:26:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f192d543640 1 -- 172.21.15.119:0/779489948 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920034660 con 0x7f192806a120 2023-11-08T19:26:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 -- 172.21.15.119:0/779489948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192806a120 msgr2=0x7f192806a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 --2- 172.21.15.119:0/779489948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192806a120 0x7f192806a530 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f192815fa70 tx=0x7f192002beb0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 -- 172.21.15.119:0/779489948 shutdown_connections 2023-11-08T19:26:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 --2- 172.21.15.119:0/779489948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192806a120 0x7f192806a530 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 -- 172.21.15.119:0/779489948 >> 172.21.15.119:0/779489948 conn(0x7f19280cb610 msgr2=0x7f19280cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 -- 172.21.15.119:0/779489948 shutdown_connections 2023-11-08T19:26:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.022+0000 7f19307a9640 1 -- 172.21.15.119:0/779489948 wait complete. 2023-11-08T19:26:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.024+0000 7f19307a9640 1 Processor -- start 2023-11-08T19:26:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.024+0000 7f19307a9640 1 -- start start 2023-11-08T19:26:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f19307a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192806a120 0x7f19281f30c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f19307a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19281447a0 0x7f192813f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f19307a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 0x7f19280c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f19307a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19280cc6d0 con 0x7f192815f4c0 2023-11-08T19:26:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f19307a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192813f6b0 con 0x7f19281447a0 2023-11-08T19:26:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f19307a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19280c8710 con 0x7f192806a120 2023-11-08T19:26:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f192ed46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 0x7f19280c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f192ed46640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 0x7f19280c7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33606/0 (socket says 172.21.15.119:33606) 2023-11-08T19:26:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.025+0000 7f192ed46640 1 -- 172.21.15.119:0/2087890696 learned_addr learned my addr 172.21.15.119:0/2087890696 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192ed46640 1 -- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192806a120 msgr2=0x7f19281f30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192dd44640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19281447a0 0x7f192813f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192e545640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192806a120 0x7f19281f30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192ed46640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192806a120 0x7f19281f30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192ed46640 1 -- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19281447a0 msgr2=0x7f192813f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192ed46640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19281447a0 0x7f192813f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192ed46640 1 -- 172.21.15.119:0/2087890696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1920009a70 con 0x7f192815f4c0 2023-11-08T19:26:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f192ed46640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 0x7f19280c7b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1920037040 tx=0x7f1920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920040650 con 0x7f192815f4c0 2023-11-08T19:26:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.026+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200407b0 con 0x7f192815f4c0 2023-11-08T19:26:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.027+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920040b20 con 0x7f192815f4c0 2023-11-08T19:26:55.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.027+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19280c89f0 con 0x7f192815f4c0 2023-11-08T19:26:55.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.027+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19280cc920 con 0x7f192815f4c0 2023-11-08T19:26:55.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.028+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19280ccb10 con 0x7f192815f4c0 2023-11-08T19:26:55.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.029+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1920040c80 con 0x7f192815f4c0 2023-11-08T19:26:55.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.030+0000 7f191effd640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f190c06ae70 0x7f190c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.030+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f19200ac520 con 0x7f192815f4c0 2023-11-08T19:26:55.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.030+0000 7f192e545640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f190c06ae70 0x7f190c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.030+0000 7f192e545640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f190c06ae70 0x7f190c06d330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1924000f60 tx=0x7f1924009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.034+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1920077610 con 0x7f192815f4c0 2023-11-08T19:26:55.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.225+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1928074430 con 0x7f192815f4c0 2023-11-08T19:26:55.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.226+0000 7f191effd640 1 -- 172.21.15.119:0/2087890696 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14989 (secure 0 0 0) 0x7f192007ac30 con 0x7f192815f4c0 2023-11-08T19:26:55.228 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:55.228 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":45,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:53.621726+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6834/4172930163":"2023-11-09T19:26:51.028910+0000","172.21.15.188:6832/4172930163":"2023-11-09T19:26:51.028910+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":7,"snaps":[{"begin":1,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:26:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.228+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f190c06ae70 msgr2=0x7f190c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.228+0000 7f19307a9640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f190c06ae70 0x7f190c06d330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1924000f60 tx=0x7f1924009040 comp rx=0 tx=0).stop 2023-11-08T19:26:55.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.228+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 msgr2=0x7f19280c7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.228+0000 7f19307a9640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 0x7f19280c7b50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1920037040 tx=0x7f1920004a40 comp rx=0 tx=0).stop 2023-11-08T19:26:55.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 shutdown_connections 2023-11-08T19:26:55.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f190c06ae70 0x7f190c06d330 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f192815f4c0 0x7f19280c7b50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19281447a0 0x7f192813f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 --2- 172.21.15.119:0/2087890696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f192806a120 0x7f19281f30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 >> 172.21.15.119:0/2087890696 conn(0x7f19280cb610 msgr2=0x7f1928143a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 shutdown_connections 2023-11-08T19:26:55.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.229+0000 7f19307a9640 1 -- 172.21.15.119:0/2087890696 wait complete. 2023-11-08T19:26:55.250 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:26:55.251 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:26:55.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.469+0000 7fed9e687640 1 Processor -- start 2023-11-08T19:26:55.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.469+0000 7fed9e687640 1 -- start start 2023-11-08T19:26:55.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed9e687640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed98144580 0x7fed9815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed9e687640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed9e687640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9815ea60 con 0x7fed98144960 2023-11-08T19:26:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed9e687640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980c9150 con 0x7fed98144580 2023-11-08T19:26:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed9e687640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c9360 con 0x7fed9815f1d0 2023-11-08T19:26:55.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed97fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed98144580 0x7fed9815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed97fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fed98144580 0x7fed9815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33614/0 (socket says 172.21.15.119:33614) 2023-11-08T19:26:55.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed97fff640 1 -- 172.21.15.119:0/1523183908 learned_addr learned my addr 172.21.15.119:0/1523183908 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:55.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.471+0000 7fed9cc24640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.472+0000 7fed97fff640 1 -- 172.21.15.119:0/1523183908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 msgr2=0x7fed980c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.472+0000 7fed97fff640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.472+0000 7fed97fff640 1 -- 172.21.15.119:0/1523183908 >> v1:172.21.15.188:6789/0 conn(0x7fed98144960 legacy=0x7fed980cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.472+0000 7fed97fff640 1 -- 172.21.15.119:0/1523183908 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980c9640 con 0x7fed98144580 2023-11-08T19:26:55.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.472+0000 7fed9cc24640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:55.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.474+0000 7fed97fff640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed9815bff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fed980c9290 tx=0x7fed8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b241a07d1c1fbce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.474+0000 7fed977fe640 1 -- 172.21.15.119:0/1523183908 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88031640 con 0x7fed98144580 2023-11-08T19:26:55.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.474+0000 7fed977fe640 1 -- 172.21.15.119:0/1523183908 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed880317a0 con 0x7fed98144580 2023-11-08T19:26:55.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.474+0000 7fed977fe640 1 -- 172.21.15.119:0/1523183908 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88031b30 con 0x7fed98144580 2023-11-08T19:26:55.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 -- 172.21.15.119:0/1523183908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 msgr2=0x7fed9815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed9815bff0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fed980c9290 tx=0x7fed8802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 -- 172.21.15.119:0/1523183908 shutdown_connections 2023-11-08T19:26:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 --2- 172.21.15.119:0/1523183908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed9815bff0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 -- 172.21.15.119:0/1523183908 >> 172.21.15.119:0/1523183908 conn(0x7fed980cb610 msgr2=0x7fed980cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 -- 172.21.15.119:0/1523183908 shutdown_connections 2023-11-08T19:26:55.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.475+0000 7fed9e687640 1 -- 172.21.15.119:0/1523183908 wait complete. 2023-11-08T19:26:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.477+0000 7fed9e687640 1 Processor -- start 2023-11-08T19:26:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.477+0000 7fed9e687640 1 -- start start 2023-11-08T19:26:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.478+0000 7fed9e687640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed980c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.478+0000 7fed9e687640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed98144960 0x7fed98140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.478+0000 7fed9e687640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.478+0000 7fed9e687640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed98140ea0 con 0x7fed98144580 2023-11-08T19:26:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.478+0000 7fed9e687640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c71e0 con 0x7fed98144960 2023-11-08T19:26:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.478+0000 7fed9e687640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c74c0 con 0x7fed9815f1d0 2023-11-08T19:26:55.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed980c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed980c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33624/0 (socket says 172.21.15.119:33624) 2023-11-08T19:26:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 -- 172.21.15.119:0/70862499 learned_addr learned my addr 172.21.15.119:0/70862499 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 -- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 msgr2=0x7fed980ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed9cc24640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 -- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed98144960 msgr2=0x7fed98140960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed8ffff640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed98144960 0x7fed98140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed98144960 0x7fed98140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.479+0000 7fed97fff640 1 -- 172.21.15.119:0/70862499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed880094d0 con 0x7fed98144580 2023-11-08T19:26:55.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.480+0000 7fed97fff640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed980c65e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fed8802b930 tx=0x7fed880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.480+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8803ac60 con 0x7fed98144580 2023-11-08T19:26:55.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.480+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8803ade0 con 0x7fed98144580 2023-11-08T19:26:55.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.480+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed88044630 con 0x7fed98144580 2023-11-08T19:26:55.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.480+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed980c77a0 con 0x7fed98144580 2023-11-08T19:26:55.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.480+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed980c6c70 con 0x7fed98144580 2023-11-08T19:26:55.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.481+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed980c7cc0 con 0x7fed98144580 2023-11-08T19:26:55.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.482+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fed8802f050 con 0x7fed98144580 2023-11-08T19:26:55.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.483+0000 7fed957fa640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed7806ae70 0x7fed7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.483+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fed880ab4e0 con 0x7fed98144580 2023-11-08T19:26:55.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.483+0000 7fed8ffff640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed7806ae70 0x7fed7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.484+0000 7fed8ffff640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed7806ae70 0x7fed7806d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed980c4270 tx=0x7fed80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.488+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fed980c7cc0 con 0x7fed98144580 2023-11-08T19:26:55.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.676+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed980c7cc0 con 0x7fed98144580 2023-11-08T19:26:55.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.677+0000 7fed957fa640 1 -- 172.21.15.119:0/70862499 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed88035020 con 0x7fed98144580 2023-11-08T19:26:55.679 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:26:55.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.679+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed7806ae70 msgr2=0x7fed7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed7806ae70 0x7fed7806d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fed980c4270 tx=0x7fed80009040 comp rx=0 tx=0).stop 2023-11-08T19:26:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 msgr2=0x7fed980c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed980c65e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fed8802b930 tx=0x7fed880046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 shutdown_connections 2023-11-08T19:26:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fed7806ae70 0x7fed7806d330 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fed9815f1d0 0x7fed980ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fed98144960 0x7fed98140960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 --2- 172.21.15.119:0/70862499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fed98144580 0x7fed980c65e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 >> 172.21.15.119:0/70862499 conn(0x7fed980cb610 msgr2=0x7fed9815c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 shutdown_connections 2023-11-08T19:26:55.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.680+0000 7fed9e687640 1 -- 172.21.15.119:0/70862499 wait complete. 2023-11-08T19:26:55.700 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:26:55.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.911+0000 7f8e408ce640 1 Processor -- start 2023-11-08T19:26:55.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.912+0000 7f8e408ce640 1 -- start start 2023-11-08T19:26:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.913+0000 7f8e408ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8e3c160530 0x7f8e3c0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.913+0000 7f8e408ce640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0d0830 con 0x7f8e3c0ce5b0 2023-11-08T19:26:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.913+0000 7f8e408ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3c0b27c0 con 0x7f8e3c160910 2023-11-08T19:26:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.913+0000 7f8e408ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3c0b2aa0 con 0x7f8e3c160530 2023-11-08T19:26:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.913+0000 7f8e3b59e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8e3c0ce5b0 0x7f8e3c0b2f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48140/0 (socket says 172.21.15.119:48140) 2023-11-08T19:26:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.913+0000 7f8e3b59e640 1 -- 172.21.15.119:0/2143021167 learned_addr learned my addr 172.21.15.119:0/2143021167 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:55.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/2143021167 >> v2:172.21.15.188:3300/0 conn(0x7f8e3c160530 0x7f8e3c0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2342519816 0 0) 0x7f8e3c0d0830 con 0x7f8e3c0ce5b0 2023-11-08T19:26:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e28003520 con 0x7f8e3c0ce5b0 2023-11-08T19:26:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1208293178 0 0) 0x7f8e28003520 con 0x7f8e3c0ce5b0 2023-11-08T19:26:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 >> v2:172.21.15.188:3300/0 conn(0x7f8e3c160530 msgr2=0x7f8e3c0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 --2- 172.21.15.119:0/2143021167 >> v2:172.21.15.188:3300/0 conn(0x7f8e3c160530 0x7f8e3c0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 >> v1:172.21.15.188:6789/0 conn(0x7f8e3c160910 legacy=0x7f8e3c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c0d1640 con 0x7f8e3c0ce5b0 2023-11-08T19:26:55.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f8e34001670 con 0x7f8e3c0ce5b0 2023-11-08T19:26:55.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 >> v1:172.21.15.119:6789/0 conn(0x7f8e3c0ce5b0 legacy=0x7f8e3c0b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e280039c0 0x7f8e28003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280047c0 0x7f8e28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e28005180 0x7f8e2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0b2aa0 con 0x7f8e280039c0 2023-11-08T19:26:55.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2800c080 con 0x7f8e280047c0 2023-11-08T19:26:55.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2800c240 con 0x7f8e28005180 2023-11-08T19:26:55.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.914+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e280039c0 0x7f8e28003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3a59c640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280047c0 0x7f8e28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3ad9d640 1 -- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e28005180 msgr2=0x7f8e2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e28005180 0x7f8e2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3ad9d640 1 -- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280047c0 msgr2=0x7f8e28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:55.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280047c0 0x7f8e28004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3ad9d640 1 -- 172.21.15.119:0/2143021167 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28003520 con 0x7f8e280039c0 2023-11-08T19:26:55.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3a59c640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280047c0 0x7f8e28004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:55.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.915+0000 7f8e3b59e640 1 -- 172.21.15.119:0/2143021167 reap_dead start 2023-11-08T19:26:55.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.916+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e280039c0 0x7f8e28003de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00e170 tx=0x7f8e280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ddfa521d426e498 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.916+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c02cc50 con 0x7f8e280039c0 2023-11-08T19:26:55.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.916+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c00f040 con 0x7f8e280039c0 2023-11-08T19:26:55.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.916+0000 7f8e39d9b640 1 -- 172.21.15.119:0/2143021167 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c034440 con 0x7f8e280039c0 2023-11-08T19:26:55.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 -- 172.21.15.119:0/2143021167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e280039c0 msgr2=0x7f8e28003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e280039c0 0x7f8e28003de0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00e170 tx=0x7f8e280057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 -- 172.21.15.119:0/2143021167 shutdown_connections 2023-11-08T19:26:55.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 --2- 172.21.15.119:0/2143021167 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e280039c0 0x7f8e28003de0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 -- 172.21.15.119:0/2143021167 >> 172.21.15.119:0/2143021167 conn(0x7f8e3c0d2ec0 msgr2=0x7f8e3c0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:55.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 -- 172.21.15.119:0/2143021167 shutdown_connections 2023-11-08T19:26:55.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.917+0000 7f8e408ce640 1 -- 172.21.15.119:0/2143021167 wait complete. 2023-11-08T19:26:55.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.919+0000 7f8e408ce640 1 Processor -- start 2023-11-08T19:26:55.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.919+0000 7f8e408ce640 1 -- start start 2023-11-08T19:26:55.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e408ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280039c0 0x7f8e3c0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e408ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 0x7f8e3c1f60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e408ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3c14fef0 0x7f8e3c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e408ce640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0b72e0 con 0x7f8e3c150440 2023-11-08T19:26:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e408ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0afb50 con 0x7f8e280039c0 2023-11-08T19:26:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e408ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e3c0b6fb0 con 0x7f8e3c14fef0 2023-11-08T19:26:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e3a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 0x7f8e3c1f60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e3a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 0x7f8e3c1f60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33634/0 (socket says 172.21.15.119:33634) 2023-11-08T19:26:55.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.920+0000 7f8e3a59c640 1 -- 172.21.15.119:0/558541668 learned_addr learned my addr 172.21.15.119:0/558541668 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:55.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3b59e640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3c14fef0 0x7f8e3c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280039c0 0x7f8e3c0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3a59c640 1 -- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3c14fef0 msgr2=0x7f8e3c1f6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3a59c640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3c14fef0 0x7f8e3c1f6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3a59c640 1 -- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280039c0 msgr2=0x7f8e3c0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:55.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3a59c640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280039c0 0x7f8e3c0b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:55.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3a59c640 1 -- 172.21.15.119:0/558541668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3c1f6b70 con 0x7f8e3c150440 2023-11-08T19:26:55.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3b59e640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3c14fef0 0x7f8e3c1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:55.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e3a59c640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 0x7f8e3c1f60f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400d7d0 tx=0x7f8e2400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24017070 con 0x7f8e3c150440 2023-11-08T19:26:55.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e24005070 con 0x7f8e3c150440 2023-11-08T19:26:55.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c1f7060 con 0x7f8e3c150440 2023-11-08T19:26:55.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24005400 con 0x7f8e3c150440 2023-11-08T19:26:55.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.921+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3c0c81c0 con 0x7f8e3c150440 2023-11-08T19:26:55.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.922+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3c04ece0 con 0x7f8e3c150440 2023-11-08T19:26:55.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.923+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e24007030 con 0x7f8e3c150440 2023-11-08T19:26:55.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.924+0000 7f8e337fe640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8e1c06ae70 0x7f8e1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.924+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f8e24088e90 con 0x7f8e3c150440 2023-11-08T19:26:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.925+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8e1c06ae70 0x7f8e1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:55.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.925+0000 7f8e3ad9d640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8e1c06ae70 0x7f8e1c06d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c036000 tx=0x7f8e2c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:55.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:55.929+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e24053f70 con 0x7f8e3c150440 2023-11-08T19:26:56.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.121+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f8e3c0c6560 con 0x7f8e3c150440 2023-11-08T19:26:56.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.122+0000 7f8e337fe640 1 -- 172.21.15.119:0/558541668 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v45) v1 ==== 103+0+11 (secure 0 0 0) 0x7f8e2408a020 con 0x7f8e3c150440 2023-11-08T19:26:56.124 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8e1c06ae70 msgr2=0x7f8e1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8e1c06ae70 0x7f8e1c06d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c036000 tx=0x7f8e2c02e000 comp rx=0 tx=0).stop 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 msgr2=0x7f8e3c1f60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 0x7f8e3c1f60f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400d7d0 tx=0x7f8e2400dc90 comp rx=0 tx=0).stop 2023-11-08T19:26:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 shutdown_connections 2023-11-08T19:26:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8e1c06ae70 0x7f8e1c06d330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e3c14fef0 0x7f8e3c1f6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8e3c150440 0x7f8e3c1f60f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 --2- 172.21.15.119:0/558541668 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e280039c0 0x7f8e3c0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 >> 172.21.15.119:0/558541668 conn(0x7f8e3c0d2ec0 msgr2=0x7f8e3c06ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:56.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.125+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 shutdown_connections 2023-11-08T19:26:56.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.126+0000 7f8e408ce640 1 -- 172.21.15.119:0/558541668 wait complete. 2023-11-08T19:26:56.145 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2023-11-08T19:26:56.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.356+0000 7f4b6106f640 1 Processor -- start 2023-11-08T19:26:56.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.356+0000 7f4b6106f640 1 -- start start 2023-11-08T19:26:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b6106f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4b5c0cc810 0x7f4b5c0c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b6106f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b5c15d180 0x7f4b5c06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b6106f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b5c0ae410 con 0x7f4b5c06a390 2023-11-08T19:26:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b6106f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c06ba30 con 0x7f4b5c15d180 2023-11-08T19:26:56.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b6106f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b5c06bcf0 con 0x7f4b5c0cc810 2023-11-08T19:26:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b5b59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b5c15d180 0x7f4b5c06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b5b59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4b5c15d180 0x7f4b5c06cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33642/0 (socket says 172.21.15.119:33642) 2023-11-08T19:26:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b5b59e640 1 -- 172.21.15.119:0/3271274412 learned_addr learned my addr 172.21.15.119:0/3271274412 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b5ad9d640 1 --1- 172.21.15.119:0/3271274412 >> v1:172.21.15.119:6789/0 conn(0x7f4b5c06a390 0x7f4b5c15ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48144/0 (socket says 172.21.15.119:48144) 2023-11-08T19:26:56.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.358+0000 7f4b5a59c640 1 --2- 172.21.15.119:0/3271274412 >> v2:172.21.15.188:3300/0 conn(0x7f4b5c0cc810 0x7f4b5c0c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.359+0000 7f4b5b59e640 1 -- 172.21.15.119:0/3271274412 >> v2:172.21.15.188:3300/0 conn(0x7f4b5c0cc810 msgr2=0x7f4b5c0c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.359+0000 7f4b5b59e640 1 --2- 172.21.15.119:0/3271274412 >> v2:172.21.15.188:3300/0 conn(0x7f4b5c0cc810 0x7f4b5c0c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.359+0000 7f4b5b59e640 1 -- 172.21.15.119:0/3271274412 >> v1:172.21.15.119:6789/0 conn(0x7f4b5c06a390 legacy=0x7f4b5c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.359+0000 7f4b5b59e640 1 -- 172.21.15.119:0/3271274412 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b5c06bfd0 con 0x7f4b5c15d180 2023-11-08T19:26:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.360+0000 7f4b5b59e640 1 --2- 172.21.15.119:0/3271274412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c15d180 0x7f4b5c06cf70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b1850 tx=0x7f4b5402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc0a47d23df9a2d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.360+0000 7f4b59d9b640 1 -- 172.21.15.119:0/3271274412 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5402cb10 con 0x7f4b5c15d180 2023-11-08T19:26:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.360+0000 7f4b59d9b640 1 -- 172.21.15.119:0/3271274412 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5402cc70 con 0x7f4b5c15d180 2023-11-08T19:26:56.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.360+0000 7f4b59d9b640 1 -- 172.21.15.119:0/3271274412 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b540343f0 con 0x7f4b5c15d180 2023-11-08T19:26:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 -- 172.21.15.119:0/3271274412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c15d180 msgr2=0x7f4b5c06cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 --2- 172.21.15.119:0/3271274412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c15d180 0x7f4b5c06cf70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0b1850 tx=0x7f4b5402bf70 comp rx=0 tx=0).stop 2023-11-08T19:26:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 -- 172.21.15.119:0/3271274412 shutdown_connections 2023-11-08T19:26:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 --2- 172.21.15.119:0/3271274412 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c15d180 0x7f4b5c06cf70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 --2- 172.21.15.119:0/3271274412 >> v2:172.21.15.188:3300/0 conn(0x7f4b5c0cc810 0x7f4b5c0c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 -- 172.21.15.119:0/3271274412 >> 172.21.15.119:0/3271274412 conn(0x7f4b5c0cb610 msgr2=0x7f4b5c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 -- 172.21.15.119:0/3271274412 shutdown_connections 2023-11-08T19:26:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.361+0000 7f4b6106f640 1 -- 172.21.15.119:0/3271274412 wait complete. 2023-11-08T19:26:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.363+0000 7f4b6106f640 1 Processor -- start 2023-11-08T19:26:56.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.363+0000 7f4b6106f640 1 -- start start 2023-11-08T19:26:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b6106f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 0x7f4b5c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b6106f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b5c0cc810 0x7f4b5c070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b6106f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b5c15d180 0x7f4b5c070ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b6106f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c0b4420 con 0x7f4b5c06a390 2023-11-08T19:26:56.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b6106f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c15d650 con 0x7f4b5c15d180 2023-11-08T19:26:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b6106f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5c071400 con 0x7f4b5c0cc810 2023-11-08T19:26:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b5ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 0x7f4b5c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b5ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 0x7f4b5c0b26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33658/0 (socket says 172.21.15.119:33658) 2023-11-08T19:26:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.364+0000 7f4b5ad9d640 1 -- 172.21.15.119:0/1194896898 learned_addr learned my addr 172.21.15.119:0/1194896898 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:56.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b5ad9d640 1 -- 172.21.15.119:0/1194896898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b5c0cc810 msgr2=0x7f4b5c070980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b5ad9d640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b5c0cc810 0x7f4b5c070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b5ad9d640 1 -- 172.21.15.119:0/1194896898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b5c15d180 msgr2=0x7f4b5c070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b5ad9d640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b5c15d180 0x7f4b5c070ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b5ad9d640 1 -- 172.21.15.119:0/1194896898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b54009b30 con 0x7f4b5c06a390 2023-11-08T19:26:56.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b5ad9d640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 0x7f4b5c0b26a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c500 tx=0x7f4b4c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c007500 con 0x7f4b5c06a390 2023-11-08T19:26:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c007660 con 0x7f4b5c06a390 2023-11-08T19:26:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0049e0 con 0x7f4b5c06a390 2023-11-08T19:26:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.365+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b5c15d8a0 con 0x7f4b5c06a390 2023-11-08T19:26:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.366+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b5c15de60 con 0x7f4b5c06a390 2023-11-08T19:26:56.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.367+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b5c074430 con 0x7f4b5c06a390 2023-11-08T19:26:56.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.368+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b4c004b40 con 0x7f4b5c06a390 2023-11-08T19:26:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.369+0000 7f4b437fe640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4b4806aea0 0x7f4b4806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.369+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f4b4c088700 con 0x7f4b5c06a390 2023-11-08T19:26:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.369+0000 7f4b5a59c640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4b4806aea0 0x7f4b4806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.369+0000 7f4b5a59c640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4b4806aea0 0x7f4b4806d360 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c15ae70 tx=0x7f4b50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:56.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.373+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4b4c053890 con 0x7f4b5c06a390 2023-11-08T19:26:56.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.635+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) v1 -- 0x7f4b5c06cd50 con 0x7f4b5c06a390 2023-11-08T19:26:56.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.811+0000 7f4b437fe640 1 -- 172.21.15.119:0/1194896898 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v137) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4b4c0180f0 con 0x7f4b5c06a390 2023-11-08T19:26:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4b4806aea0 msgr2=0x7f4b4806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4b4806aea0 0x7f4b4806d360 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c15ae70 tx=0x7f4b50008040 comp rx=0 tx=0).stop 2023-11-08T19:26:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 msgr2=0x7f4b5c0b26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 0x7f4b5c0b26a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00c500 tx=0x7f4b4c00c9c0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 shutdown_connections 2023-11-08T19:26:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4b4806aea0 0x7f4b4806d360 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4b5c15d180 0x7f4b5c070ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4b5c0cc810 0x7f4b5c070980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.815+0000 7f4b6106f640 1 --2- 172.21.15.119:0/1194896898 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4b5c06a390 0x7f4b5c0b26a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.816+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 >> 172.21.15.119:0/1194896898 conn(0x7f4b5c0cb610 msgr2=0x7f4b5c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.816+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 shutdown_connections 2023-11-08T19:26:56.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:56.816+0000 7f4b6106f640 1 -- 172.21.15.119:0/1194896898 wait complete. 2023-11-08T19:26:56.837 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:26:57.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.047+0000 7fb6dcba5640 1 Processor -- start 2023-11-08T19:26:57.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.047+0000 7fb6dcba5640 1 -- start start 2023-11-08T19:26:57.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.048+0000 7fb6dcba5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d815b2b0 0x7fb6d8071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.048+0000 7fb6dcba5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb6d815b690 0x7fb6d80ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.048+0000 7fb6dcba5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6d80c9440 con 0x7fb6d80cf5e0 2023-11-08T19:26:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.048+0000 7fb6dcba5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d80c9690 con 0x7fb6d815b690 2023-11-08T19:26:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.048+0000 7fb6dcba5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c9970 con 0x7fb6d815b2b0 2023-11-08T19:26:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb6d815b690 0x7fb6d80ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb6d815b690 0x7fb6d80ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33660/0 (socket says 172.21.15.119:33660) 2023-11-08T19:26:57.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/2496557064 learned_addr learned my addr 172.21.15.119:0/2496557064 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d77fe640 1 --2- 172.21.15.119:0/2496557064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d815b2b0 0x7fb6d8071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/2496557064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d815b2b0 msgr2=0x7fb6d8071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 --2- 172.21.15.119:0/2496557064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d815b2b0 0x7fb6d8071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/2496557064 >> v1:172.21.15.188:6789/0 conn(0x7fb6d80cf5e0 legacy=0x7fb6d80ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.049+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/2496557064 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d80c9c50 con 0x7fb6d815b690 2023-11-08T19:26:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.050+0000 7fb6d6ffd640 1 --2- 172.21.15.119:0/2496557064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b690 0x7fb6d80ca450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009870 tx=0x7fb6cc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1486d5b6ee8840d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6d67fc640 1 -- 172.21.15.119:0/2496557064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc031640 con 0x7fb6d815b690 2023-11-08T19:26:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6d67fc640 1 -- 172.21.15.119:0/2496557064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc0317a0 con 0x7fb6d815b690 2023-11-08T19:26:57.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6d67fc640 1 -- 172.21.15.119:0/2496557064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc031b30 con 0x7fb6d815b690 2023-11-08T19:26:57.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 -- 172.21.15.119:0/2496557064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b690 msgr2=0x7fb6d80ca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/2496557064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b690 0x7fb6d80ca450 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc009870 tx=0x7fb6cc02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 -- 172.21.15.119:0/2496557064 shutdown_connections 2023-11-08T19:26:57.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/2496557064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b690 0x7fb6d80ca450 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/2496557064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d815b2b0 0x7fb6d8071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 -- 172.21.15.119:0/2496557064 >> 172.21.15.119:0/2496557064 conn(0x7fb6d80ceac0 msgr2=0x7fb6d80ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.051+0000 7fb6dcba5640 1 -- 172.21.15.119:0/2496557064 shutdown_connections 2023-11-08T19:26:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.052+0000 7fb6dcba5640 1 -- 172.21.15.119:0/2496557064 wait complete. 2023-11-08T19:26:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.053+0000 7fb6dcba5640 1 Processor -- start 2023-11-08T19:26:57.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.053+0000 7fb6dcba5640 1 -- start start 2023-11-08T19:26:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.054+0000 7fb6dcba5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cf5e0 0x7fb6d81500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.054+0000 7fb6dcba5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 0x7fb6d815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.054+0000 7fb6dcba5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815b690 0x7fb6d815e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.054+0000 7fb6dcba5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80aeae0 con 0x7fb6d815b2b0 2023-11-08T19:26:57.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.054+0000 7fb6dcba5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d806b680 con 0x7fb6d815b690 2023-11-08T19:26:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.054+0000 7fb6dcba5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d80c5100 con 0x7fb6d80cf5e0 2023-11-08T19:26:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 0x7fb6d815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 0x7fb6d815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33666/0 (socket says 172.21.15.119:33666) 2023-11-08T19:26:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/308231941 learned_addr learned my addr 172.21.15.119:0/308231941 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:57.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d77fe640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cf5e0 0x7fb6d81500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d7fff640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815b690 0x7fb6d815e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cf5e0 msgr2=0x7fb6d81500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cf5e0 0x7fb6d81500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815b690 msgr2=0x7fb6d815e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815b690 0x7fb6d815e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 -- 172.21.15.119:0/308231941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc0094d0 con 0x7fb6d815b2b0 2023-11-08T19:26:57.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d77fe640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cf5e0 0x7fb6d81500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:57.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d7fff640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815b690 0x7fb6d815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:57.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.055+0000 7fb6d6ffd640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 0x7fb6d815dc40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc000c00 tx=0x7fb6cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.056+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc03ac60 con 0x7fb6d815b2b0 2023-11-08T19:26:57.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.056+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6cc03adc0 con 0x7fb6d815b2b0 2023-11-08T19:26:57.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.056+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6cc044630 con 0x7fb6d815b2b0 2023-11-08T19:26:57.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.056+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d806b920 con 0x7fb6d815b2b0 2023-11-08T19:26:57.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.056+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d80c53e0 con 0x7fb6d815b2b0 2023-11-08T19:26:57.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.057+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d806be40 con 0x7fb6d815b2b0 2023-11-08T19:26:57.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.058+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb6cc02f050 con 0x7fb6d815b2b0 2023-11-08T19:26:57.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.059+0000 7fb6bffff640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb6c406ae70 0x7fb6c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.059+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fb6cc0ab4d0 con 0x7fb6d815b2b0 2023-11-08T19:26:57.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.059+0000 7fb6d77fe640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb6c406ae70 0x7fb6c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.060+0000 7fb6d77fe640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb6c406ae70 0x7fb6c406d330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8006fd0 tx=0x7fb6c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.064+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb6d806be40 con 0x7fb6d815b2b0 2023-11-08T19:26:57.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.239+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb6d806be40 con 0x7fb6d815b2b0 2023-11-08T19:26:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.245+0000 7fb6bffff640 1 -- 172.21.15.119:0/308231941 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v13)=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb6d806be40 con 0x7fb6d815b2b0 2023-11-08T19:26:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.247+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb6c406ae70 msgr2=0x7fb6c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.247+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb6c406ae70 0x7fb6c406d330 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8006fd0 tx=0x7fb6c8008040 comp rx=0 tx=0).stop 2023-11-08T19:26:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 msgr2=0x7fb6d815dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 0x7fb6d815dc40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc000c00 tx=0x7fb6cc0046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 shutdown_connections 2023-11-08T19:26:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb6c406ae70 0x7fb6c406d330 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb6d815b690 0x7fb6d815e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb6d815b2b0 0x7fb6d815dc40 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 --2- 172.21.15.119:0/308231941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb6d80cf5e0 0x7fb6d81500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 >> 172.21.15.119:0/308231941 conn(0x7fb6d80ceac0 msgr2=0x7fb6d80b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:57.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 shutdown_connections 2023-11-08T19:26:57.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.248+0000 7fb6dcba5640 1 -- 172.21.15.119:0/308231941 wait complete. 2023-11-08T19:26:57.267 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:26:57.268 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:57.382 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:26:57.506 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:26:57.506 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:57.507 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:57.569 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:26:57.569 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:57.570 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:57.631 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:26:57.631 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:26:57.632 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:57.693 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:26:57.694 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:26:57.695 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:26:57.756 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:26:57.757 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:26:57.757 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:26:57.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.978+0000 7f27d104b640 1 Processor -- start 2023-11-08T19:26:57.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.978+0000 7f27d104b640 1 -- start start 2023-11-08T19:26:57.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27d104b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 0x7f27cc15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27d104b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f27cc0cad30 0x7f27cc06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27d104b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc0c3a50 con 0x7f27cc0cb170 2023-11-08T19:26:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27d104b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc144b80 con 0x7f27cc0cad30 2023-11-08T19:26:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27d104b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0c9150 con 0x7f27cc1447a0 2023-11-08T19:26:57.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27cb59e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f27cc0cb170 0x7f27cc06a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48160/0 (socket says 172.21.15.119:48160) 2023-11-08T19:26:57.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.980+0000 7f27cb59e640 1 -- 172.21.15.119:0/2713106269 learned_addr learned my addr 172.21.15.119:0/2713106269 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27cad9d640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 0x7f27cc15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27ca59c640 1 --2- 172.21.15.119:0/2713106269 >> v2:172.21.15.188:3300/0 conn(0x7f27cc0cad30 0x7f27cc06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110244163 0 0) 0x7f27cc0c3a50 con 0x7f27cc0cb170 2023-11-08T19:26:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b8003520 con 0x7f27cc0cb170 2023-11-08T19:26:57.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435605234 0 0) 0x7f27b8003520 con 0x7f27cc0cb170 2023-11-08T19:26:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 msgr2=0x7f27cc15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 0x7f27cc15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 >> v2:172.21.15.188:3300/0 conn(0x7f27cc0cad30 msgr2=0x7f27cc06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 --2- 172.21.15.119:0/2713106269 >> v2:172.21.15.188:3300/0 conn(0x7f27cc0cad30 0x7f27cc06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0c9410 con 0x7f27cc0cb170 2023-11-08T19:26:57.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f27c4001670 con 0x7f27cc0cb170 2023-11-08T19:26:57.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 >> v1:172.21.15.119:6789/0 conn(0x7f27cc0cb170 legacy=0x7f27cc06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 0x7f27b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b8003ef0 0x7f27b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27b8004d00 0x7f27b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc144b80 con 0x7f27b8003ef0 2023-11-08T19:26:57.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0c9150 con 0x7f27b8004d00 2023-11-08T19:26:57.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.981+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b800b980 con 0x7f27cc1447a0 2023-11-08T19:26:57.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27ca59c640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b8003ef0 0x7f27b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27ca59c640 1 -- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 msgr2=0x7f27b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27cad9d640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 0x7f27b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27ca59c640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27cc1447a0 0x7f27b8003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27ca59c640 1 -- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27b8004d00 msgr2=0x7f27b8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27ca59c640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27b8004d00 0x7f27b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.982+0000 7f27ca59c640 1 -- 172.21.15.119:0/2713106269 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b8003520 con 0x7f27b8003ef0 2023-11-08T19:26:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.983+0000 7f27ca59c640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b8003ef0 0x7f27b80047c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009ac0 tx=0x7f27b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=387366c25ef1f8ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.983+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b402c6d0 con 0x7f27b8003ef0 2023-11-08T19:26:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.983+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b402c830 con 0x7f27b8003ef0 2023-11-08T19:26:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.983+0000 7f27c9d9b640 1 -- 172.21.15.119:0/2713106269 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b402cbc0 con 0x7f27b8003ef0 2023-11-08T19:26:57.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27d104b640 1 -- 172.21.15.119:0/2713106269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b8003ef0 msgr2=0x7f27b80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27d104b640 1 --2- 172.21.15.119:0/2713106269 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27b8003ef0 0x7f27b80047c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009ac0 tx=0x7f27b80057f0 comp rx=0 tx=0).stop 2023-11-08T19:26:57.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27cb59e640 1 -- 172.21.15.119:0/2713106269 reap_dead start 2023-11-08T19:26:57.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27d104b640 1 -- 172.21.15.119:0/2713106269 shutdown_connections 2023-11-08T19:26:57.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27d104b640 1 -- 172.21.15.119:0/2713106269 >> 172.21.15.119:0/2713106269 conn(0x7f27cc0cb610 msgr2=0x7f27cc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:57.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27d104b640 1 -- 172.21.15.119:0/2713106269 shutdown_connections 2023-11-08T19:26:57.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.984+0000 7f27d104b640 1 -- 172.21.15.119:0/2713106269 wait complete. 2023-11-08T19:26:57.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.986+0000 7f27d104b640 1 Processor -- start 2023-11-08T19:26:57.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.986+0000 7f27d104b640 1 -- start start 2023-11-08T19:26:57.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27d104b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27b8003ef0 0x7f27cc15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27d104b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27cc0cc6d0 0x7f27cc0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27d104b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 0x7f27cc0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:57.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27d104b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0c7200 con 0x7f27cc0cd090 2023-11-08T19:26:57.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27d104b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0c8e90 con 0x7f27cc0cc6d0 2023-11-08T19:26:57.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27d104b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc15aa10 con 0x7f27b8003ef0 2023-11-08T19:26:57.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27cb59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 0x7f27cc0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27cb59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 0x7f27cc0b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33690/0 (socket says 172.21.15.119:33690) 2023-11-08T19:26:57.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.987+0000 7f27cb59e640 1 -- 172.21.15.119:0/4206136594 learned_addr learned my addr 172.21.15.119:0/4206136594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:57.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cad9d640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27b8003ef0 0x7f27cc15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:57.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cb59e640 1 -- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27b8003ef0 msgr2=0x7f27cc15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:57.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cb59e640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27b8003ef0 0x7f27cc15a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cb59e640 1 -- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27cc0cc6d0 msgr2=0x7f27cc0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cb59e640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27cc0cc6d0 0x7f27cc0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cb59e640 1 -- 172.21.15.119:0/4206136594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0c8bb0 con 0x7f27cc0cd090 2023-11-08T19:26:58.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27cb59e640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 0x7f27cc0b35b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0c9290 tx=0x7f27c400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4007230 con 0x7f27cc0cd090 2023-11-08T19:26:58.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c400ced0 con 0x7f27cc0cd090 2023-11-08T19:26:58.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40075d0 con 0x7f27cc0cd090 2023-11-08T19:26:58.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc0b2570 con 0x7f27cc0cd090 2023-11-08T19:26:58.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.988+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc0b2b30 con 0x7f27cc0cd090 2023-11-08T19:26:58.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.989+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc0b23d0 con 0x7f27cc0cd090 2023-11-08T19:26:58.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.990+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27c40047e0 con 0x7f27cc0cd090 2023-11-08T19:26:58.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.992+0000 7f27c37fe640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27ac06aea0 0x7f27ac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.992+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7f27c4088ac0 con 0x7f27cc0cd090 2023-11-08T19:26:58.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.992+0000 7f27cad9d640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27ac06aea0 0x7f27ac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.993+0000 7f27cad9d640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27ac06aea0 0x7f27ac06d360 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0b17c0 tx=0x7f27bc00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:57.996+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27cc0b23d0 con 0x7f27cc0cd090 2023-11-08T19:26:58.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.222+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27cc0b23d0 con 0x7f27cc0cd090 2023-11-08T19:26:58.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.224+0000 7f27c37fe640 1 -- 172.21.15.119:0/4206136594 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f27cc0b23d0 con 0x7f27cc0cd090 2023-11-08T19:26:58.225 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:26:58.226 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":20,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/127250916","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":127250916},{"type":"v1","addr":"172.21.15.188:6835","nonce":127250916}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":4918,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/1091810753","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":1091810753},{"type":"v1","addr":"172.21.15.188:6834","nonce":1091810753}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":20},{"gid":5304,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3938413520","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3938413520},{"type":"v1","addr":"172.21.15.119:6836","nonce":3938413520}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5307,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3008284252","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3008284252},{"type":"v1","addr":"172.21.15.119:6837","nonce":3008284252}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[]} 2023-11-08T19:26:58.226 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 20 2023-11-08T19:26:58.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27ac06aea0 msgr2=0x7f27ac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27ac06aea0 0x7f27ac06d360 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0b17c0 tx=0x7f27bc00b040 comp rx=0 tx=0).stop 2023-11-08T19:26:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 msgr2=0x7f27cc0b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 0x7f27cc0b35b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0c9290 tx=0x7f27c400cb90 comp rx=0 tx=0).stop 2023-11-08T19:26:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 shutdown_connections 2023-11-08T19:26:58.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27ac06aea0 0x7f27ac06d360 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27cc0cd090 0x7f27cc0b35b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27cc0cc6d0 0x7f27cc0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 --2- 172.21.15.119:0/4206136594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27b8003ef0 0x7f27cc15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 >> 172.21.15.119:0/4206136594 conn(0x7f27cc0cb610 msgr2=0x7f27cc070d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 shutdown_connections 2023-11-08T19:26:58.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.226+0000 7f27d104b640 1 -- 172.21.15.119:0/4206136594 wait complete. 2023-11-08T19:26:58.247 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:26:58.247 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:26:58.248 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:26:58.248 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:58.248 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:26:58.249 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:26:58.249 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:26:58.249 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:58.250 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:26:58.250 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:26:58.250 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:26:58.250 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:58.251 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:26:58.251 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:26:58.251 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:26:58.252 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:26:58.268 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:26:58.268 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:26:58.269 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:26:58.270 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:26:58.273 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:26:58.273 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:26:58.275 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:26:58.276 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:26:58.317 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:26:58.317 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:58.323 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:26:58.324 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:58.325 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:26:58.325 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:58.332 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:26:58.333 DEBUG:teuthology.parallel:result is None 2023-11-08T19:26:58.333 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:26:58.477 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:26:58.478 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:26:58.487 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:26:58.490 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:26:58.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.520+0000 7fa9a5eda640 1 Processor -- start 2023-11-08T19:26:58.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.520+0000 7fa9a5eda640 1 -- start start 2023-11-08T19:26:58.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.521+0000 7fa9a5eda640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9a015d410 0x7fa9a00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.521+0000 7fa9a5eda640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a006b570 con 0x7fa9a015df00 2023-11-08T19:26:58.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.521+0000 7fa9a5eda640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a006b850 con 0x7fa9a015e5f0 2023-11-08T19:26:58.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.521+0000 7fa9a5eda640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a006c1f0 con 0x7fa9a015d410 2023-11-08T19:26:58.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.521+0000 7fa99ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa9a015df00 0x7fa9a006dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48190/0 (socket says 172.21.15.119:48190) 2023-11-08T19:26:58.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.521+0000 7fa99ffff640 1 -- 172.21.15.119:0/4037592035 learned_addr learned my addr 172.21.15.119:0/4037592035 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:58.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99f7fe640 1 --2- 172.21.15.119:0/4037592035 >> v2:172.21.15.188:3300/0 conn(0x7fa9a015d410 0x7fa9a00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227302827 0 0) 0x7fa9a006b570 con 0x7fa9a015df00 2023-11-08T19:26:58.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa988003520 con 0x7fa9a015df00 2023-11-08T19:26:58.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 52461795 0 0) 0x7fa9a006b850 con 0x7fa9a015e5f0 2023-11-08T19:26:58.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a006b570 con 0x7fa9a015e5f0 2023-11-08T19:26:58.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99f7fe640 1 -- 172.21.15.119:0/4037592035 >> v1:172.21.15.188:6789/0 conn(0x7fa9a015e5f0 legacy=0x7fa9a015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99f7fe640 1 -- 172.21.15.119:0/4037592035 >> v1:172.21.15.119:6789/0 conn(0x7fa9a015df00 legacy=0x7fa9a006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.522+0000 7fa99f7fe640 1 -- 172.21.15.119:0/4037592035 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a006bb30 con 0x7fa9a015d410 2023-11-08T19:26:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.523+0000 7fa99f7fe640 1 --2- 172.21.15.119:0/4037592035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015d410 0x7fa9a00ce040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa990009790 tx=0x7fa99002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=881f5201e00711c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99002d040 con 0x7fa9a015d410 2023-11-08T19:26:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990031810 con 0x7fa9a015d410 2023-11-08T19:26:58.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa99e7fc640 1 -- 172.21.15.119:0/4037592035 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990031ba0 con 0x7fa9a015d410 2023-11-08T19:26:58.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 -- 172.21.15.119:0/4037592035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015d410 msgr2=0x7fa9a00ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/4037592035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015d410 0x7fa9a00ce040 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa990009790 tx=0x7fa99002b8b0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 -- 172.21.15.119:0/4037592035 shutdown_connections 2023-11-08T19:26:58.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/4037592035 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015d410 0x7fa9a00ce040 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 -- 172.21.15.119:0/4037592035 >> 172.21.15.119:0/4037592035 conn(0x7fa9a00d0db0 msgr2=0x7fa9a00d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:58.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 -- 172.21.15.119:0/4037592035 shutdown_connections 2023-11-08T19:26:58.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.524+0000 7fa9a5eda640 1 -- 172.21.15.119:0/4037592035 wait complete. 2023-11-08T19:26:58.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.526+0000 7fa9a5eda640 1 Processor -- start 2023-11-08T19:26:58.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.526+0000 7fa9a5eda640 1 -- start start 2023-11-08T19:26:58.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa9a5eda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 0x7fa9a014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa9a5eda640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015df00 0x7fa9a01601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa9a5eda640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015e5f0 0x7fa9a015c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa9a5eda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a015f9e0 con 0x7fa9a015d410 2023-11-08T19:26:58.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa9a5eda640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a015ee60 con 0x7fa9a015df00 2023-11-08T19:26:58.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa9a5eda640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a015f140 con 0x7fa9a015e5f0 2023-11-08T19:26:58.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 0x7fa9a014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa99f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 0x7fa9a014b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33728/0 (socket says 172.21.15.119:33728) 2023-11-08T19:26:58.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.527+0000 7fa99f7fe640 1 -- 172.21.15.119:0/628825940 learned_addr learned my addr 172.21.15.119:0/628825940 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:58.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99effd640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015df00 0x7fa9a01601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99ffff640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015e5f0 0x7fa9a015c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99f7fe640 1 -- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015e5f0 msgr2=0x7fa9a015c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99f7fe640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015e5f0 0x7fa9a015c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99f7fe640 1 -- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015df00 msgr2=0x7fa9a01601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:58.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99f7fe640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015df00 0x7fa9a01601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:58.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99f7fe640 1 -- 172.21.15.119:0/628825940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900093a0 con 0x7fa9a015d410 2023-11-08T19:26:58.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99ffff640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015e5f0 0x7fa9a015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:26:58.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99effd640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015df00 0x7fa9a01601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:26:58.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.528+0000 7fa99f7fe640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 0x7fa9a014b1f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa9900098b0 tx=0x7fa9900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.529+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa99003ad60 con 0x7fa9a015d410 2023-11-08T19:26:58.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.529+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa990005990 con 0x7fa9a015d410 2023-11-08T19:26:58.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.529+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa990005b40 con 0x7fa9a015d410 2023-11-08T19:26:58.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.529+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a015f420 con 0x7fa9a015d410 2023-11-08T19:26:58.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.529+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a014b730 con 0x7fa9a015d410 2023-11-08T19:26:58.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.530+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a0074430 con 0x7fa9a015d410 2023-11-08T19:26:58.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.531+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa99004d440 con 0x7fa9a015d410 2023-11-08T19:26:58.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.532+0000 7fa987fff640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa98c06ae70 0x7fa98c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:58.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.532+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5644+0+0 (secure 0 0 0) 0x7fa9900ab890 con 0x7fa9a015d410 2023-11-08T19:26:58.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.532+0000 7fa99effd640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa98c06ae70 0x7fa98c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:58.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.533+0000 7fa99effd640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa98c06ae70 0x7fa98c06d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015a2f0 tx=0x7fa994009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:58.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.536+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa990076970 con 0x7fa9a015d410 2023-11-08T19:26:58.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:58.745+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa9a015d920 con 0x7fa9a015d410 2023-11-08T19:26:59.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.667+0000 7fa987fff640 1 -- 172.21.15.119:0/628825940 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa990035080 con 0x7fa9a015d410 2023-11-08T19:26:59.669 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-11-08T19:26:59.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa98c06ae70 msgr2=0x7fa98c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:59.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa98c06ae70 0x7fa98c06d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa9a015a2f0 tx=0x7fa994009040 comp rx=0 tx=0).stop 2023-11-08T19:26:59.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 msgr2=0x7fa9a014b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:59.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 0x7fa9a014b1f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa9900098b0 tx=0x7fa9900046c0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 shutdown_connections 2023-11-08T19:26:59.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa98c06ae70 0x7fa98c06d330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9a015e5f0 0x7fa9a015c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9a015df00 0x7fa9a01601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 --2- 172.21.15.119:0/628825940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa9a015d410 0x7fa9a014b1f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 >> 172.21.15.119:0/628825940 conn(0x7fa9a00d0db0 msgr2=0x7fa9a0160dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 shutdown_connections 2023-11-08T19:26:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.671+0000 7fa9a5eda640 1 -- 172.21.15.119:0/628825940 wait complete. 2023-11-08T19:26:59.691 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:26:59.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.894+0000 7efeaa500640 1 Processor -- start 2023-11-08T19:26:59.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.894+0000 7efeaa500640 1 -- start start 2023-11-08T19:26:59.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efeaa500640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efea406a390 0x7efea40c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:59.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efeaa500640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efea415d180 0x7efea406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:59.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efeaa500640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea40ae410 con 0x7efea40cc810 2023-11-08T19:26:59.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efeaa500640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea406ba30 con 0x7efea406a390 2023-11-08T19:26:59.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efeaa500640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea406bcf0 con 0x7efea415d180 2023-11-08T19:26:59.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efea3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efea406a390 0x7efea40c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:59.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efe9bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efea40cc810 0x7efea415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37546/0 (socket says 172.21.15.119:37546) 2023-11-08T19:26:59.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.896+0000 7efe9bfff640 1 -- 172.21.15.119:0/3491754472 learned_addr learned my addr 172.21.15.119:0/3491754472 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:26:59.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.897+0000 7efea8a9d640 1 --2- 172.21.15.119:0/3491754472 >> v2:172.21.15.188:3300/0 conn(0x7efea415d180 0x7efea406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:59.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.897+0000 7efea3fff640 1 -- 172.21.15.119:0/3491754472 >> v2:172.21.15.188:3300/0 conn(0x7efea415d180 msgr2=0x7efea406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:26:59.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.897+0000 7efea3fff640 1 --2- 172.21.15.119:0/3491754472 >> v2:172.21.15.188:3300/0 conn(0x7efea415d180 0x7efea406cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.897+0000 7efea3fff640 1 -- 172.21.15.119:0/3491754472 >> v1:172.21.15.119:6789/0 conn(0x7efea40cc810 legacy=0x7efea415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:59.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.897+0000 7efea3fff640 1 -- 172.21.15.119:0/3491754472 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea406bfd0 con 0x7efea406a390 2023-11-08T19:26:59.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.898+0000 7efea3fff640 1 --2- 172.21.15.119:0/3491754472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea406a390 0x7efea40c4460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efea40c49a0 tx=0x7efe9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244a42557d6995bd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:59.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.898+0000 7efea37fe640 1 -- 172.21.15.119:0/3491754472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94031830 con 0x7efea406a390 2023-11-08T19:26:59.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.898+0000 7efea37fe640 1 -- 172.21.15.119:0/3491754472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe94031990 con 0x7efea406a390 2023-11-08T19:26:59.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.898+0000 7efea37fe640 1 -- 172.21.15.119:0/3491754472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94031d20 con 0x7efea406a390 2023-11-08T19:26:59.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 -- 172.21.15.119:0/3491754472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea406a390 msgr2=0x7efea40c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:59.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 --2- 172.21.15.119:0/3491754472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea406a390 0x7efea40c4460 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efea40c49a0 tx=0x7efe9402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 -- 172.21.15.119:0/3491754472 shutdown_connections 2023-11-08T19:26:59.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 --2- 172.21.15.119:0/3491754472 >> v2:172.21.15.188:3300/0 conn(0x7efea415d180 0x7efea406cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 --2- 172.21.15.119:0/3491754472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea406a390 0x7efea40c4460 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 -- 172.21.15.119:0/3491754472 >> 172.21.15.119:0/3491754472 conn(0x7efea40cb610 msgr2=0x7efea40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:26:59.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 -- 172.21.15.119:0/3491754472 shutdown_connections 2023-11-08T19:26:59.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.899+0000 7efeaa500640 1 -- 172.21.15.119:0/3491754472 wait complete. 2023-11-08T19:26:59.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.901+0000 7efeaa500640 1 Processor -- start 2023-11-08T19:26:59.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.901+0000 7efeaa500640 1 -- start start 2023-11-08T19:26:59.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efeaa500640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efea406a390 0x7efea4154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:59.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efeaa500640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efea40cc810 0x7efea40b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:59.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efeaa500640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 0x7efea40ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:59.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efeaa500640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea4155be0 con 0x7efea415d180 2023-11-08T19:26:59.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efeaa500640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea4155130 con 0x7efea406a390 2023-11-08T19:26:59.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efeaa500640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea40b14b0 con 0x7efea40cc810 2023-11-08T19:26:59.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efea8a9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 0x7efea40ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:59.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efea8a9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 0x7efea40ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51798/0 (socket says 172.21.15.119:51798) 2023-11-08T19:26:59.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efea8a9d640 1 -- 172.21.15.119:0/2808642749 learned_addr learned my addr 172.21.15.119:0/2808642749 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:26:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.902+0000 7efea3fff640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efea406a390 0x7efea4154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efe9bfff640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efea40cc810 0x7efea40b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea8a9d640 1 -- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efea40cc810 msgr2=0x7efea40b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea8a9d640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efea40cc810 0x7efea40b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea8a9d640 1 -- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efea406a390 msgr2=0x7efea4154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:26:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea8a9d640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efea406a390 0x7efea4154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:26:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea8a9d640 1 -- 172.21.15.119:0/2808642749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe94009400 con 0x7efea415d180 2023-11-08T19:26:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea8a9d640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 0x7efea40ae870 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007e60 tx=0x7efe9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c014420 con 0x7efea415d180 2023-11-08T19:26:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9c0145a0 con 0x7efea415d180 2023-11-08T19:26:59.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9c014950 con 0x7efea415d180 2023-11-08T19:26:59.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea415f0b0 con 0x7efea415d180 2023-11-08T19:26:59.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.903+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea415f670 con 0x7efea415d180 2023-11-08T19:26:59.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.904+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea406cd50 con 0x7efea415d180 2023-11-08T19:26:59.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.905+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe9c014ab0 con 0x7efea415d180 2023-11-08T19:26:59.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.907+0000 7efea17fa640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe8406ae70 0x7efe8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:26:59.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.907+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7efe9c089f50 con 0x7efea415d180 2023-11-08T19:26:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.907+0000 7efea3fff640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe8406ae70 0x7efe8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:26:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.908+0000 7efea3fff640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe8406ae70 0x7efe8406d330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe94036000 tx=0x7efe9403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:26:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:26:59.911+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe9c055010 con 0x7efea415d180 2023-11-08T19:27:00.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.166+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efea40cafc0 con 0x7efea415d180 2023-11-08T19:27:00.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.167+0000 7efea17fa640 1 -- 172.21.15.119:0/2808642749 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efe9c058630 con 0x7efea415d180 2023-11-08T19:27:00.169 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:00.169 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:27:00.169 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-11-08T19:27:00.169 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.169 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-11-08T19:27:00.170 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg==", 2023-11-08T19:27:00.170 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.170 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.170 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:00.171 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:00.171 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.171 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.172 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-11-08T19:27:00.172 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ==", 2023-11-08T19:27:00.172 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.172 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.172 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:00.173 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:00.173 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.173 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.174 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-11-08T19:27:00.174 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw==", 2023-11-08T19:27:00.174 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.174 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.174 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:00.175 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:00.175 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.175 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.175 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.175 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.176 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-11-08T19:27:00.176 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ==", 2023-11-08T19:27:00.176 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.176 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.177 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:00.177 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:00.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.177 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.177 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.178 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-11-08T19:27:00.178 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA==", 2023-11-08T19:27:00.178 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.178 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.179 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-11-08T19:27:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A==", 2023-11-08T19:27:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.180 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.181 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-11-08T19:27:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA==", 2023-11-08T19:27:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.182 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-11-08T19:27:00.183 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ==", 2023-11-08T19:27:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-11-08T19:27:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ==", 2023-11-08T19:27:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.185 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.186 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-11-08T19:27:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw==", 2023-11-08T19:27:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.187 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.188 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-11-08T19:27:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g==", 2023-11-08T19:27:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.189 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.190 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-11-08T19:27:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ==", 2023-11-08T19:27:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:00.191 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-11-08T19:27:00.192 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw==", 2023-11-08T19:27:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-11-08T19:27:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:00.193 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-11-08T19:27:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ==", 2023-11-08T19:27:00.194 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:27:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:27:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:27:00.195 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-11-08T19:27:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q==", 2023-11-08T19:27:00.196 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:27:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:27:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:27:00.197 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-11-08T19:27:00.198 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ==", 2023-11-08T19:27:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:27:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:27:00.199 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:27:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-11-08T19:27:00.200 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBH4Etl5+oDFBAAMV8QbfI7L35tUuiNz9bvdw==", 2023-11-08T19:27:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:27:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-11-08T19:27:00.202 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA==", 2023-11-08T19:27:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:27:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-11-08T19:27:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ==", 2023-11-08T19:27:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:00.204 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:27:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:00.205 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-11-08T19:27:00.206 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:27:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe8406ae70 msgr2=0x7efe8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe8406ae70 0x7efe8406d330 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efe94036000 tx=0x7efe9403c040 comp rx=0 tx=0).stop 2023-11-08T19:27:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 msgr2=0x7efea40ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 0x7efea40ae870 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efe9c007e60 tx=0x7efe9c00e4b0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 shutdown_connections 2023-11-08T19:27:00.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe8406ae70 0x7efe8406d330 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efea415d180 0x7efea40ae870 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efea40cc810 0x7efea40b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 --2- 172.21.15.119:0/2808642749 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efea406a390 0x7efea4154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.169+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 >> 172.21.15.119:0/2808642749 conn(0x7efea40cb610 msgr2=0x7efea40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.170+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 shutdown_connections 2023-11-08T19:27:00.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.170+0000 7efeaa500640 1 -- 172.21.15.119:0/2808642749 wait complete. 2023-11-08T19:27:00.210 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:00.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.424+0000 7f344afc7640 1 Processor -- start 2023-11-08T19:27:00.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.424+0000 7f344afc7640 1 -- start start 2023-11-08T19:27:00.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f344afc7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 0x7f344415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f344afc7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34440c3a50 con 0x7f34440cb170 2023-11-08T19:27:00.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f344afc7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3444144b80 con 0x7f34440cad30 2023-11-08T19:27:00.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f344afc7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3444070040 con 0x7f34441447a0 2023-11-08T19:27:00.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f3449564640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f34440cb170 0x7f344406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37548/0 (socket says 172.21.15.119:37548) 2023-11-08T19:27:00.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f3449564640 1 -- 172.21.15.119:0/3834726853 learned_addr learned my addr 172.21.15.119:0/3834726853 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:00.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f3448d63640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 0x7f344415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.426+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738873588 0 0) 0x7f34440c3a50 con 0x7f34440cb170 2023-11-08T19:27:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3434003520 con 0x7f34440cb170 2023-11-08T19:27:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3053738735 0 0) 0x7f3434003520 con 0x7f34440cb170 2023-11-08T19:27:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 msgr2=0x7f344415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 0x7f344415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 >> v1:172.21.15.188:6789/0 conn(0x7f34440cad30 legacy=0x7f344406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34440c9150 con 0x7f34440cb170 2023-11-08T19:27:00.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f343c0019f0 con 0x7f34440cb170 2023-11-08T19:27:00.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 >> v1:172.21.15.119:6789/0 conn(0x7f34440cb170 legacy=0x7f344406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34340039a0 0x7f3434003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 0x7f3434005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3444070040 con 0x7f34340039a0 2023-11-08T19:27:00.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343400ba30 con 0x7f34340047c0 2023-11-08T19:27:00.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343400bbf0 con 0x7f34441447a0 2023-11-08T19:27:00.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.427+0000 7f3448d63640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34340039a0 0x7f3434003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3449564640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 0x7f3434005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3443fff640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3448d63640 1 -- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 msgr2=0x7f3434005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3448d63640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34441447a0 0x7f3434005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3448d63640 1 -- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 msgr2=0x7f3434004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3448d63640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3448d63640 1 -- 172.21.15.119:0/3834726853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3434003520 con 0x7f34340039a0 2023-11-08T19:27:00.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.428+0000 7f3443fff640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34340047c0 0x7f3434004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:00.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.429+0000 7f3448d63640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34340039a0 0x7f3434003dc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3438009a90 tx=0x7f34340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dac37b2d451db5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.429+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343802c660 con 0x7f34340039a0 2023-11-08T19:27:00.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.429+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343802c7c0 con 0x7f34340039a0 2023-11-08T19:27:00.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.429+0000 7f34437fe640 1 -- 172.21.15.119:0/3834726853 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343802cb50 con 0x7f34340039a0 2023-11-08T19:27:00.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f344afc7640 1 -- 172.21.15.119:0/3834726853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34340039a0 msgr2=0x7f3434003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f344afc7640 1 --2- 172.21.15.119:0/3834726853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34340039a0 0x7f3434003dc0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3438009a90 tx=0x7f34340057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f3449564640 1 -- 172.21.15.119:0/3834726853 reap_dead start 2023-11-08T19:27:00.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f344afc7640 1 -- 172.21.15.119:0/3834726853 shutdown_connections 2023-11-08T19:27:00.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f344afc7640 1 -- 172.21.15.119:0/3834726853 >> 172.21.15.119:0/3834726853 conn(0x7f34440cb610 msgr2=0x7f34440cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:00.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f344afc7640 1 -- 172.21.15.119:0/3834726853 shutdown_connections 2023-11-08T19:27:00.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.430+0000 7f344afc7640 1 -- 172.21.15.119:0/3834726853 wait complete. 2023-11-08T19:27:00.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.432+0000 7f344afc7640 1 Processor -- start 2023-11-08T19:27:00.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.432+0000 7f344afc7640 1 -- start start 2023-11-08T19:27:00.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f344afc7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34340039a0 0x7f344415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f344afc7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f344415a420 0x7f3444142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f344afc7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 0x7f34441401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f344afc7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344415c650 con 0x7f34441431c0 2023-11-08T19:27:00.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f344afc7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344415c930 con 0x7f344415a420 2023-11-08T19:27:00.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f344afc7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f344415cc10 con 0x7f34340039a0 2023-11-08T19:27:00.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f3449564640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 0x7f34441401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f3449564640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 0x7f34441401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51828/0 (socket says 172.21.15.119:51828) 2023-11-08T19:27:00.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.433+0000 7f3449564640 1 -- 172.21.15.119:0/4202477296 learned_addr learned my addr 172.21.15.119:0/4202477296 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:00.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3448d63640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34340039a0 0x7f344415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3443fff640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f344415a420 0x7f3444142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3449564640 1 -- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34340039a0 msgr2=0x7f344415d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3449564640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34340039a0 0x7f344415d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3449564640 1 -- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f344415a420 msgr2=0x7f3444142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3449564640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f344415a420 0x7f3444142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3449564640 1 -- 172.21.15.119:0/4202477296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344415cef0 con 0x7f34441431c0 2023-11-08T19:27:00.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f3449564640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 0x7f34441401a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f343c0012e0 tx=0x7f343c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c00bb80 con 0x7f34441431c0 2023-11-08T19:27:00.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c00bd00 con 0x7f34441431c0 2023-11-08T19:27:00.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c0072b0 con 0x7f34441431c0 2023-11-08T19:27:00.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344415d1d0 con 0x7f34441431c0 2023-11-08T19:27:00.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.434+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344415df10 con 0x7f34441431c0 2023-11-08T19:27:00.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.435+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344415d6f0 con 0x7f34441431c0 2023-11-08T19:27:00.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.437+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f343c007410 con 0x7f34441431c0 2023-11-08T19:27:00.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.438+0000 7f34417fa640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f342806aea0 0x7f342806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.438+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f343c089850 con 0x7f34441431c0 2023-11-08T19:27:00.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.438+0000 7f3448d63640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f342806aea0 0x7f342806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.438+0000 7f3448d63640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f342806aea0 0x7f342806d360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3438036000 tx=0x7f3438030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.442+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f344415d6f0 con 0x7f34441431c0 2023-11-08T19:27:00.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.632+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f344415d6f0 con 0x7f34441431c0 2023-11-08T19:27:00.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.634+0000 7f34417fa640 1 -- 172.21.15.119:0/4202477296 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14815 (secure 0 0 0) 0x7f344415d6f0 con 0x7f34441431c0 2023-11-08T19:27:00.636 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:00.637 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":50,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:26:59.656678+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:00.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f342806aea0 msgr2=0x7f342806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f342806aea0 0x7f342806d360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3438036000 tx=0x7f3438030000 comp rx=0 tx=0).stop 2023-11-08T19:27:00.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 msgr2=0x7f34441401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 0x7f34441401a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f343c0012e0 tx=0x7f343c00b4a0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 shutdown_connections 2023-11-08T19:27:00.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f342806aea0 0x7f342806d360 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34441431c0 0x7f34441401a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f344415a420 0x7f3444142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 --2- 172.21.15.119:0/4202477296 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34340039a0 0x7f344415d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.637+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 >> 172.21.15.119:0/4202477296 conn(0x7f34440cb610 msgr2=0x7f34440ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:00.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.638+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 shutdown_connections 2023-11-08T19:27:00.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.638+0000 7f344afc7640 1 -- 172.21.15.119:0/4202477296 wait complete. 2023-11-08T19:27:00.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:00.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:00.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.877+0000 7fdf19c39640 1 Processor -- start 2023-11-08T19:27:00.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.877+0000 7fdf19c39640 1 -- start start 2023-11-08T19:27:00.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf19c39640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 0x7fdf1415bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf19c39640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdf140cd000 0x7fdf140b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf19c39640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf140ce380 con 0x7fdf14144750 2023-11-08T19:27:00.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf19c39640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf140cd4b0 con 0x7fdf140cd000 2023-11-08T19:27:00.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf19c39640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140cd770 con 0x7fdf1406a830 2023-11-08T19:27:00.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf137fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdf14144750 0x7fdf1406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37550/0 (socket says 172.21.15.119:37550) 2023-11-08T19:27:00.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.879+0000 7fdf137fe640 1 -- 172.21.15.119:0/3588630114 learned_addr learned my addr 172.21.15.119:0/3588630114 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:00.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525544630 0 0) 0x7fdf140ce380 con 0x7fdf14144750 2023-11-08T19:27:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf13fff640 1 --2- 172.21.15.119:0/3588630114 >> v2:172.21.15.188:3300/0 conn(0x7fdf140cd000 0x7fdf140b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf0bfff640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 0x7fdf1415bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf00003520 con 0x7fdf14144750 2023-11-08T19:27:00.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1801510528 0 0) 0x7fdf00003520 con 0x7fdf14144750 2023-11-08T19:27:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 msgr2=0x7fdf1415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 0x7fdf1415bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 >> v2:172.21.15.188:3300/0 conn(0x7fdf140cd000 msgr2=0x7fdf140b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 --2- 172.21.15.119:0/3588630114 >> v2:172.21.15.188:3300/0 conn(0x7fdf140cd000 0x7fdf140b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140cda50 con 0x7fdf14144750 2023-11-08T19:27:00.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fdf04001670 con 0x7fdf14144750 2023-11-08T19:27:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 >> v1:172.21.15.119:6789/0 conn(0x7fdf14144750 legacy=0x7fdf1406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf000047c0 0x7fdf00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 0x7fdf00005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140cd4b0 con 0x7fdf00003970 2023-11-08T19:27:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140cd770 con 0x7fdf000047c0 2023-11-08T19:27:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0000b980 con 0x7fdf1406a830 2023-11-08T19:27:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.880+0000 7fdf137fe640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf0bfff640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf000047c0 0x7fdf00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf13fff640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 0x7fdf00005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf137fe640 1 -- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 msgr2=0x7fdf00005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf137fe640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf1406a830 0x7fdf00005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf137fe640 1 -- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf000047c0 msgr2=0x7fdf00004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:00.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf137fe640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf000047c0 0x7fdf00004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf137fe640 1 -- 172.21.15.119:0/3588630114 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf00003520 con 0x7fdf00003970 2023-11-08T19:27:00.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.881+0000 7fdf0bfff640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf000047c0 0x7fdf00004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:00.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.882+0000 7fdf137fe640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf00003d90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c290 tx=0x7fdf000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7934b0697f82bf91 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.882+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04005ad0 con 0x7fdf00003970 2023-11-08T19:27:00.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.882+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf04005c30 con 0x7fdf00003970 2023-11-08T19:27:00.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.882+0000 7fdf12ffd640 1 -- 172.21.15.119:0/3588630114 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0402c8d0 con 0x7fdf00003970 2023-11-08T19:27:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf19c39640 1 -- 172.21.15.119:0/3588630114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 msgr2=0x7fdf00003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf19c39640 1 --2- 172.21.15.119:0/3588630114 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf00003d90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400c290 tx=0x7fdf000057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf13fff640 1 -- 172.21.15.119:0/3588630114 reap_dead start 2023-11-08T19:27:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf19c39640 1 -- 172.21.15.119:0/3588630114 shutdown_connections 2023-11-08T19:27:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf19c39640 1 -- 172.21.15.119:0/3588630114 >> 172.21.15.119:0/3588630114 conn(0x7fdf140b08e0 msgr2=0x7fdf140b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf19c39640 1 -- 172.21.15.119:0/3588630114 shutdown_connections 2023-11-08T19:27:00.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.883+0000 7fdf19c39640 1 -- 172.21.15.119:0/3588630114 wait complete. 2023-11-08T19:27:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.885+0000 7fdf19c39640 1 Processor -- start 2023-11-08T19:27:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.885+0000 7fdf19c39640 1 -- start start 2023-11-08T19:27:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf19c39640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf14158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf19c39640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf14138da0 0x7fdf140adb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf19c39640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf140ae040 0x7fdf141f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf19c39640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140ae4c0 con 0x7fdf00003970 2023-11-08T19:27:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf19c39640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14158a70 con 0x7fdf140ae040 2023-11-08T19:27:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf19c39640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf140ac8c0 con 0x7fdf14138da0 2023-11-08T19:27:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf14158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf137fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf14158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51848/0 (socket says 172.21.15.119:51848) 2023-11-08T19:27:00.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf137fe640 1 -- 172.21.15.119:0/1085225567 learned_addr learned my addr 172.21.15.119:0/1085225567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:00.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.886+0000 7fdf13fff640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf140ae040 0x7fdf141f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf0bfff640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf14138da0 0x7fdf140adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf137fe640 1 -- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf14138da0 msgr2=0x7fdf140adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf137fe640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf14138da0 0x7fdf140adb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf137fe640 1 -- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf140ae040 msgr2=0x7fdf141f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf137fe640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf140ae040 0x7fdf141f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf137fe640 1 -- 172.21.15.119:0/1085225567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf140acba0 con 0x7fdf00003970 2023-11-08T19:27:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf13fff640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf140ae040 0x7fdf141f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf137fe640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf14158530 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402c540 tx=0x7fdf04000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0402cce0 con 0x7fdf00003970 2023-11-08T19:27:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0402ce60 con 0x7fdf00003970 2023-11-08T19:27:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf140ace80 con 0x7fdf00003970 2023-11-08T19:27:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.887+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf040072b0 con 0x7fdf00003970 2023-11-08T19:27:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.888+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf140ad440 con 0x7fdf00003970 2023-11-08T19:27:00.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.889+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1404ece0 con 0x7fdf00003970 2023-11-08T19:27:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.890+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdf04032070 con 0x7fdf00003970 2023-11-08T19:27:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.891+0000 7fdf10ff9640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdef406ae70 0x7fdef406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.891+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdf0403c040 con 0x7fdf00003970 2023-11-08T19:27:00.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.891+0000 7fdf0bfff640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdef406ae70 0x7fdef406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:00.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.891+0000 7fdf0bfff640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdef406ae70 0x7fdef406d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf140ae7e0 tx=0x7fdefc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:00.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:00.895+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdf04075b10 con 0x7fdf00003970 2023-11-08T19:27:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.086+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdf1415c1d0 con 0x7fdf00003970 2023-11-08T19:27:01.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.087+0000 7fdf10ff9640 1 -- 172.21.15.119:0/1085225567 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdf04001c80 con 0x7fdf00003970 2023-11-08T19:27:01.089 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:01.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdef406ae70 msgr2=0x7fdef406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdef406ae70 0x7fdef406d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdf140ae7e0 tx=0x7fdefc008040 comp rx=0 tx=0).stop 2023-11-08T19:27:01.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 msgr2=0x7fdf14158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf14158530 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402c540 tx=0x7fdf04000fe0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 shutdown_connections 2023-11-08T19:27:01.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdef406ae70 0x7fdef406d330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdf140ae040 0x7fdf141f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdf14138da0 0x7fdf140adb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 --2- 172.21.15.119:0/1085225567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdf00003970 0x7fdf14158530 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 >> 172.21.15.119:0/1085225567 conn(0x7fdf140b08e0 msgr2=0x7fdf1406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 shutdown_connections 2023-11-08T19:27:01.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.090+0000 7fdf19c39640 1 -- 172.21.15.119:0/1085225567 wait complete. 2023-11-08T19:27:01.109 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:01.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.319+0000 7fd14ff78640 1 Processor -- start 2023-11-08T19:27:01.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.319+0000 7fd14ff78640 1 -- start start 2023-11-08T19:27:01.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.321+0000 7fd14ff78640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 0x7fd1480cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.321+0000 7fd14ff78640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1480ced30 con 0x7fd1480cf670 2023-11-08T19:27:01.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.321+0000 7fd14ff78640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1480ce120 con 0x7fd14806a830 2023-11-08T19:27:01.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.321+0000 7fd14ff78640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480ce400 con 0x7fd1480cade0 2023-11-08T19:27:01.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.321+0000 7fd14d513640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd1480cf670 0x7fd14806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37554/0 (socket says 172.21.15.119:37554) 2023-11-08T19:27:01.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.321+0000 7fd14d513640 1 -- 172.21.15.119:0/151141939 learned_addr learned my addr 172.21.15.119:0/151141939 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:01.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14dd14640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 0x7fd1480cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634477324 0 0) 0x7fd1480ced30 con 0x7fd1480cf670 2023-11-08T19:27:01.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd134003520 con 0x7fd1480cf670 2023-11-08T19:27:01.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3817843406 0 0) 0x7fd134003520 con 0x7fd1480cf670 2023-11-08T19:27:01.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 msgr2=0x7fd1480cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 0x7fd1480cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 >> v1:172.21.15.188:6789/0 conn(0x7fd14806a830 legacy=0x7fd14815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480ce6e0 con 0x7fd1480cf670 2023-11-08T19:27:01.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd138001670 con 0x7fd1480cf670 2023-11-08T19:27:01.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 >> v1:172.21.15.119:6789/0 conn(0x7fd1480cf670 legacy=0x7fd14806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd134003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1340047c0 0x7fd134004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 0x7fd134005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480ce400 con 0x7fd1340039a0 2023-11-08T19:27:01.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13400bac0 con 0x7fd1340047c0 2023-11-08T19:27:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.322+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13400bc80 con 0x7fd1480cade0 2023-11-08T19:27:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14dd14640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd134003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14d513640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1340047c0 0x7fd134004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14dd14640 1 -- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 msgr2=0x7fd134005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14e515640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 0x7fd134005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14dd14640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1480cade0 0x7fd134005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14dd14640 1 -- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1340047c0 msgr2=0x7fd134004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14dd14640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1340047c0 0x7fd134004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14dd14640 1 -- 172.21.15.119:0/151141939 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134003520 con 0x7fd1340039a0 2023-11-08T19:27:01.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.323+0000 7fd14d513640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1340047c0 0x7fd134004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:01.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.324+0000 7fd14dd14640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd134003dc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd144009a90 tx=0x7fd1340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5317204d5a385abb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.324+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14402c680 con 0x7fd1340039a0 2023-11-08T19:27:01.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14402c7e0 con 0x7fd1340039a0 2023-11-08T19:27:01.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14cd12640 1 -- 172.21.15.119:0/151141939 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14402cb70 con 0x7fd1340039a0 2023-11-08T19:27:01.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14ff78640 1 -- 172.21.15.119:0/151141939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 msgr2=0x7fd134003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14ff78640 1 --2- 172.21.15.119:0/151141939 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd134003dc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd144009a90 tx=0x7fd1340057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14e515640 1 -- 172.21.15.119:0/151141939 reap_dead start 2023-11-08T19:27:01.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14ff78640 1 -- 172.21.15.119:0/151141939 shutdown_connections 2023-11-08T19:27:01.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14ff78640 1 -- 172.21.15.119:0/151141939 >> 172.21.15.119:0/151141939 conn(0x7fd1480cfd00 msgr2=0x7fd1480d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14ff78640 1 -- 172.21.15.119:0/151141939 shutdown_connections 2023-11-08T19:27:01.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.325+0000 7fd14ff78640 1 -- 172.21.15.119:0/151141939 wait complete. 2023-11-08T19:27:01.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.327+0000 7fd14ff78640 1 Processor -- start 2023-11-08T19:27:01.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.327+0000 7fd14ff78640 1 -- start start 2023-11-08T19:27:01.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.328+0000 7fd14ff78640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd1480c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.328+0000 7fd14ff78640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1480c9550 0x7fd1481f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.328+0000 7fd14ff78640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14806d1b0 0x7fd1480c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.328+0000 7fd14ff78640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1481def70 con 0x7fd1340039a0 2023-11-08T19:27:01.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.328+0000 7fd14ff78640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480c3a30 con 0x7fd1480c9550 2023-11-08T19:27:01.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.328+0000 7fd14ff78640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1481f6750 con 0x7fd14806d1b0 2023-11-08T19:27:01.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd1480c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd1480c34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51870/0 (socket says 172.21.15.119:51870) 2023-11-08T19:27:01.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 -- 172.21.15.119:0/2946722333 learned_addr learned my addr 172.21.15.119:0/2946722333 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:01.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14d513640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1480c9550 0x7fd1481f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14e515640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14806d1b0 0x7fd1480c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 -- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14806d1b0 msgr2=0x7fd1480c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14806d1b0 0x7fd1480c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 -- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1480c9550 msgr2=0x7fd1481f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1480c9550 0x7fd1481f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 -- 172.21.15.119:0/2946722333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1481f69d0 con 0x7fd1340039a0 2023-11-08T19:27:01.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14e515640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14806d1b0 0x7fd1480c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:01.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14d513640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1480c9550 0x7fd1481f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:01.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.329+0000 7fd14dd14640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd1480c34f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd144009a90 tx=0x7fd144004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.330+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14402dea0 con 0x7fd1340039a0 2023-11-08T19:27:01.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.330+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144003810 con 0x7fd1340039a0 2023-11-08T19:27:01.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.330+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144003ba0 con 0x7fd1340039a0 2023-11-08T19:27:01.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.330+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1481f6cb0 con 0x7fd1340039a0 2023-11-08T19:27:01.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.330+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1481df180 con 0x7fd1340039a0 2023-11-08T19:27:01.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.331+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1481df310 con 0x7fd1340039a0 2023-11-08T19:27:01.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.332+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd144003d00 con 0x7fd1340039a0 2023-11-08T19:27:01.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.333+0000 7fd13e7fc640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.333+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd1440ab460 con 0x7fd1340039a0 2023-11-08T19:27:01.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.333+0000 7fd14d513640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.334+0000 7fd14d513640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd138001200 tx=0x7fd138009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.337+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1440765c0 con 0x7fd1340039a0 2023-11-08T19:27:01.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.527+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fd14806a610 con 0x7fd1340039a0 2023-11-08T19:27:01.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.528+0000 7fd13e7fc640 1 -- 172.21.15.119:0/2946722333 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v50) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd1440a9e60 con 0x7fd1340039a0 2023-11-08T19:27:01.529 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 msgr2=0x7fd12c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd138001200 tx=0x7fd138009040 comp rx=0 tx=0).stop 2023-11-08T19:27:01.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 msgr2=0x7fd1480c34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd1480c34f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fd144009a90 tx=0x7fd144004730 comp rx=0 tx=0).stop 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 shutdown_connections 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14806d1b0 0x7fd1480c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1480c9550 0x7fd1481f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 --2- 172.21.15.119:0/2946722333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1340039a0 0x7fd1480c34f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 >> 172.21.15.119:0/2946722333 conn(0x7fd1480cfd00 msgr2=0x7fd1481e4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 shutdown_connections 2023-11-08T19:27:01.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.530+0000 7fd14ff78640 1 -- 172.21.15.119:0/2946722333 wait complete. 2023-11-08T19:27:01.551 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:27:01.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.767+0000 7fe52efc0640 1 Processor -- start 2023-11-08T19:27:01.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.768+0000 7fe52efc0640 1 -- start start 2023-11-08T19:27:01.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.769+0000 7fe52efc0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5280d1260 0x7fe528160580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.769+0000 7fe52efc0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe528160ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.769+0000 7fe52efc0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280d21e0 con 0x7fe5280d1640 2023-11-08T19:27:01.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.769+0000 7fe52efc0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe528161000 con 0x7fe5280d1260 2023-11-08T19:27:01.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.769+0000 7fe52efc0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280ce760 con 0x7fe5280b2010 2023-11-08T19:27:01.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe528160ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe528160ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:36278/0 (socket says 172.21.15.119:36278) 2023-11-08T19:27:01.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 -- 172.21.15.119:0/667807861 learned_addr learned my addr 172.21.15.119:0/667807861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe52cd5c640 1 --2- 172.21.15.119:0/667807861 >> v2:172.21.15.188:3300/0 conn(0x7fe5280d1260 0x7fe528160580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 -- 172.21.15.119:0/667807861 >> v2:172.21.15.188:3300/0 conn(0x7fe5280d1260 msgr2=0x7fe528160580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 --2- 172.21.15.119:0/667807861 >> v2:172.21.15.188:3300/0 conn(0x7fe5280d1260 0x7fe528160580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 -- 172.21.15.119:0/667807861 >> v1:172.21.15.188:6789/0 conn(0x7fe5280d1640 legacy=0x7fe5280d1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.770+0000 7fe527fff640 1 -- 172.21.15.119:0/667807861 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5280b2580 con 0x7fe5280b2010 2023-11-08T19:27:01.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.772+0000 7fe527fff640 1 --2- 172.21.15.119:0/667807861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe528160ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009870 tx=0x7fe51c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33400e0d17b56ea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.772+0000 7fe5277fe640 1 -- 172.21.15.119:0/667807861 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c0317c0 con 0x7fe5280b2010 2023-11-08T19:27:01.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.772+0000 7fe5277fe640 1 -- 172.21.15.119:0/667807861 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c031920 con 0x7fe5280b2010 2023-11-08T19:27:01.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.772+0000 7fe5277fe640 1 -- 172.21.15.119:0/667807861 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c031cb0 con 0x7fe5280b2010 2023-11-08T19:27:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 -- 172.21.15.119:0/667807861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 msgr2=0x7fe528160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 --2- 172.21.15.119:0/667807861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe528160ac0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009870 tx=0x7fe51c02b860 comp rx=0 tx=0).stop 2023-11-08T19:27:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 -- 172.21.15.119:0/667807861 shutdown_connections 2023-11-08T19:27:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 --2- 172.21.15.119:0/667807861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe528160ac0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 --2- 172.21.15.119:0/667807861 >> v2:172.21.15.188:3300/0 conn(0x7fe5280d1260 0x7fe528160580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 -- 172.21.15.119:0/667807861 >> 172.21.15.119:0/667807861 conn(0x7fe5280d30f0 msgr2=0x7fe5280d34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 -- 172.21.15.119:0/667807861 shutdown_connections 2023-11-08T19:27:01.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.773+0000 7fe52efc0640 1 -- 172.21.15.119:0/667807861 wait complete. 2023-11-08T19:27:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.775+0000 7fe52efc0640 1 Processor -- start 2023-11-08T19:27:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.775+0000 7fe52efc0640 1 -- start start 2023-11-08T19:27:01.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52efc0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe5280affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52efc0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5280d1260 0x7fe52815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52efc0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 0x7fe52815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52efc0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52815c7b0 con 0x7fe5280d1640 2023-11-08T19:27:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52efc0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280cb1d0 con 0x7fe5280d1260 2023-11-08T19:27:01.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52efc0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5280caef0 con 0x7fe5280b2010 2023-11-08T19:27:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 0x7fe52815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 0x7fe52815c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51874/0 (socket says 172.21.15.119:51874) 2023-11-08T19:27:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 -- 172.21.15.119:0/423600507 learned_addr learned my addr 172.21.15.119:0/423600507 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52cd5c640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe5280affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 -- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 msgr2=0x7fe5280affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe5280affd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 -- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5280d1260 msgr2=0x7fe52815bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5280d1260 0x7fe52815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.776+0000 7fe52d55d640 1 -- 172.21.15.119:0/423600507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe51c0094d0 con 0x7fe5280d1640 2023-11-08T19:27:01.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.777+0000 7fe52d55d640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 0x7fe52815c270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe520001eb0 tx=0x7fe52000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.777+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5200052d0 con 0x7fe5280d1640 2023-11-08T19:27:01.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.777+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe520005430 con 0x7fe5280d1640 2023-11-08T19:27:01.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.777+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe520003810 con 0x7fe5280d1640 2023-11-08T19:27:01.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.777+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5280c6cc0 con 0x7fe5280d1640 2023-11-08T19:27:01.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.777+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5280c7280 con 0x7fe5280d1640 2023-11-08T19:27:01.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.778+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528074430 con 0x7fe5280d1640 2023-11-08T19:27:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.779+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe520003970 con 0x7fe5280d1640 2023-11-08T19:27:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.781+0000 7fe5257fa640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe51406ae70 0x7fe51406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.781+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe520088a20 con 0x7fe5280d1640 2023-11-08T19:27:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.781+0000 7fe52cd5c640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe51406ae70 0x7fe51406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:01.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.782+0000 7fe52cd5c640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe51406ae70 0x7fe51406d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5180073f0 tx=0x7fe51800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:01.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.784+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe520053b70 con 0x7fe5280d1640 2023-11-08T19:27:01.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.980+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe5280d21e0 con 0x7fe5280d1640 2023-11-08T19:27:01.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.981+0000 7fe5257fa640 1 -- 172.21.15.119:0/423600507 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+137 (secure 0 0 0) 0x7fe5280d21e0 con 0x7fe5280d1640 2023-11-08T19:27:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:01.983 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-11-08T19:27:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-11-08T19:27:01.983 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-11-08T19:27:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 7, 2023-11-08T19:27:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "snap-test-pool" 2023-11-08T19:27:01.984 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:01.985 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-11-08T19:27:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.983+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe51406ae70 msgr2=0x7fe51406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.983+0000 7fe52efc0640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe51406ae70 0x7fe51406d330 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe5180073f0 tx=0x7fe51800b040 comp rx=0 tx=0).stop 2023-11-08T19:27:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.983+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 msgr2=0x7fe52815c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.983+0000 7fe52efc0640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 0x7fe52815c270 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe520001eb0 tx=0x7fe52000c890 comp rx=0 tx=0).stop 2023-11-08T19:27:01.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 shutdown_connections 2023-11-08T19:27:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe51406ae70 0x7fe51406d330 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5280d1640 0x7fe52815c270 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5280d1260 0x7fe52815bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 --2- 172.21.15.119:0/423600507 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5280b2010 0x7fe5280affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:01.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 >> 172.21.15.119:0/423600507 conn(0x7fe5280d30f0 msgr2=0x7fe52815e660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 shutdown_connections 2023-11-08T19:27:01.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:01.984+0000 7fe52efc0640 1 -- 172.21.15.119:0/423600507 wait complete. 2023-11-08T19:27:02.004 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:27:02.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:27:02.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.218+0000 7f8db9275640 1 Processor -- start 2023-11-08T19:27:02.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.219+0000 7f8db9275640 1 -- start start 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db9275640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8db4144580 0x7f8db415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db9275640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8db415f1d0 0x7f8db40c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db9275640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db415ea60 con 0x7f8db4144960 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db9275640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40c9150 con 0x7f8db4144580 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db9275640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db40c9360 con 0x7f8db415f1d0 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db2ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8db4144580 0x7f8db415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8dabfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8db4144960 0x7f8db40cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37556/0 (socket says 172.21.15.119:37556) 2023-11-08T19:27:02.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8dabfff640 1 -- 172.21.15.119:0/4246106493 learned_addr learned my addr 172.21.15.119:0/4246106493 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:02.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db2ffd640 1 -- 172.21.15.119:0/4246106493 >> v2:172.21.15.188:3300/0 conn(0x7f8db415f1d0 msgr2=0x7f8db40c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:02.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db2ffd640 1 --2- 172.21.15.119:0/4246106493 >> v2:172.21.15.188:3300/0 conn(0x7f8db415f1d0 0x7f8db40c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db2ffd640 1 -- 172.21.15.119:0/4246106493 >> v1:172.21.15.119:6789/0 conn(0x7f8db4144960 legacy=0x7f8db40cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.221+0000 7f8db2ffd640 1 -- 172.21.15.119:0/4246106493 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db40c9640 con 0x7f8db4144580 2023-11-08T19:27:02.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.222+0000 7f8db2ffd640 1 --2- 172.21.15.119:0/4246106493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db415bff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8db406d420 tx=0x7f8da402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37bd510dfd7a99 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db27fc640 1 -- 172.21.15.119:0/4246106493 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da402e070 con 0x7f8db4144580 2023-11-08T19:27:02.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db27fc640 1 -- 172.21.15.119:0/4246106493 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da40315a0 con 0x7f8db4144580 2023-11-08T19:27:02.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db27fc640 1 -- 172.21.15.119:0/4246106493 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4031930 con 0x7f8db4144580 2023-11-08T19:27:02.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db9275640 1 -- 172.21.15.119:0/4246106493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 msgr2=0x7f8db415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db9275640 1 --2- 172.21.15.119:0/4246106493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db415bff0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f8db406d420 tx=0x7f8da402b840 comp rx=0 tx=0).stop 2023-11-08T19:27:02.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db9275640 1 -- 172.21.15.119:0/4246106493 shutdown_connections 2023-11-08T19:27:02.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db9275640 1 --2- 172.21.15.119:0/4246106493 >> v2:172.21.15.188:3300/0 conn(0x7f8db415f1d0 0x7f8db40c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db9275640 1 --2- 172.21.15.119:0/4246106493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db415bff0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.223+0000 7f8db9275640 1 -- 172.21.15.119:0/4246106493 >> 172.21.15.119:0/4246106493 conn(0x7f8db40cb610 msgr2=0x7f8db40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.224+0000 7f8db9275640 1 -- 172.21.15.119:0/4246106493 shutdown_connections 2023-11-08T19:27:02.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.224+0000 7f8db9275640 1 -- 172.21.15.119:0/4246106493 wait complete. 2023-11-08T19:27:02.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.225+0000 7f8db9275640 1 Processor -- start 2023-11-08T19:27:02.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.225+0000 7f8db9275640 1 -- start start 2023-11-08T19:27:02.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.226+0000 7f8db9275640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db4132280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.226+0000 7f8db9275640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db4144960 0x7f8db40b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.226+0000 7f8db9275640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db415f1d0 0x7f8db40c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.226+0000 7f8db9275640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c6530 con 0x7f8db4144580 2023-11-08T19:27:02.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.226+0000 7f8db9275640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db40c5c70 con 0x7f8db4144960 2023-11-08T19:27:02.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.226+0000 7f8db9275640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db41327c0 con 0x7f8db415f1d0 2023-11-08T19:27:02.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db4132280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db4132280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51886/0 (socket says 172.21.15.119:51886) 2023-11-08T19:27:02.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 -- 172.21.15.119:0/2971275065 learned_addr learned my addr 172.21.15.119:0/2971275065 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:02.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db37fe640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db415f1d0 0x7f8db40c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8dabfff640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db4144960 0x7f8db40b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 -- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db415f1d0 msgr2=0x7f8db40c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db415f1d0 0x7f8db40c7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 -- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db4144960 msgr2=0x7f8db40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db4144960 0x7f8db40b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.227+0000 7f8db2ffd640 1 -- 172.21.15.119:0/2971275065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4009400 con 0x7f8db4144580 2023-11-08T19:27:02.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.228+0000 7f8db2ffd640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db4132280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009860 tx=0x7f8da40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.228+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4032900 con 0x7f8db4144580 2023-11-08T19:27:02.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.228+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da4032a60 con 0x7f8db4144580 2023-11-08T19:27:02.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.228+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db40c5ee0 con 0x7f8db4144580 2023-11-08T19:27:02.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.228+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da4032e10 con 0x7f8db4144580 2023-11-08T19:27:02.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.228+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db413f090 con 0x7f8db4144580 2023-11-08T19:27:02.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.229+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db406cb10 con 0x7f8db4144580 2023-11-08T19:27:02.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.230+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8da40453f0 con 0x7f8db4144580 2023-11-08T19:27:02.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.231+0000 7f8dab7fe640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8d9406ae70 0x7f8d9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:02.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.231+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8da40ab7e0 con 0x7f8db4144580 2023-11-08T19:27:02.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.231+0000 7f8dabfff640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8d9406ae70 0x7f8d9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:02.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.232+0000 7f8dabfff640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8d9406ae70 0x7f8d9406d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8db40c7dc0 tx=0x7f8d9c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:02.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.235+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8da40769d0 con 0x7f8db4144580 2023-11-08T19:27:02.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.411+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8db40ca910 con 0x7f8db4144580 2023-11-08T19:27:02.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.858+0000 7f8dab7fe640 1 -- 172.21.15.119:0/2971275065 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8da4079ff0 con 0x7f8db4144580 2023-11-08T19:27:02.861 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-11-08T19:27:02.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8d9406ae70 msgr2=0x7f8d9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8d9406ae70 0x7f8d9406d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f8db40c7dc0 tx=0x7f8d9c009040 comp rx=0 tx=0).stop 2023-11-08T19:27:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 msgr2=0x7f8db4132280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db4132280 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8da4009860 tx=0x7f8da40046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 shutdown_connections 2023-11-08T19:27:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8d9406ae70 0x7f8d9406d330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8db415f1d0 0x7f8db40c7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8db4144960 0x7f8db40b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 --2- 172.21.15.119:0/2971275065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8db4144580 0x7f8db4132280 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 >> 172.21.15.119:0/2971275065 conn(0x7f8db40cb610 msgr2=0x7f8db406ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 shutdown_connections 2023-11-08T19:27:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:02.862+0000 7f8db9275640 1 -- 172.21.15.119:0/2971275065 wait complete. 2023-11-08T19:27:02.889 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:27:03.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.085+0000 7fa896c07640 1 Processor -- start 2023-11-08T19:27:03.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.085+0000 7fa896c07640 1 -- start start 2023-11-08T19:27:03.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa896c07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900719c0 0x7fa8900ced70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa896c07640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa890071da0 0x7fa8900cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa896c07640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8900cdcc0 0x7fa8900cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa896c07640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89006a530 con 0x7fa890071da0 2023-11-08T19:27:03.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa896c07640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8900cf2b0 con 0x7fa8900cdcc0 2023-11-08T19:27:03.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa896c07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900c9cd0 con 0x7fa8900719c0 2023-11-08T19:27:03.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa88ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa890071da0 0x7fa8900cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa88ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa890071da0 0x7fa8900cd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51896/0 (socket says 172.21.15.119:51896) 2023-11-08T19:27:03.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa88ffff640 1 -- 172.21.15.119:0/1096705256 learned_addr learned my addr 172.21.15.119:0/1096705256 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:03.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.087+0000 7fa8951a4640 1 --2- 172.21.15.119:0/1096705256 >> v2:172.21.15.188:3300/0 conn(0x7fa8900cdcc0 0x7fa8900cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.088+0000 7fa8949a3640 1 --2- 172.21.15.119:0/1096705256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900719c0 0x7fa8900ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.088+0000 7fa88ffff640 1 -- 172.21.15.119:0/1096705256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900719c0 msgr2=0x7fa8900ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:03.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.088+0000 7fa88ffff640 1 --2- 172.21.15.119:0/1096705256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900719c0 0x7fa8900ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.088+0000 7fa88ffff640 1 -- 172.21.15.119:0/1096705256 >> v2:172.21.15.188:3300/0 conn(0x7fa8900cdcc0 msgr2=0x7fa8900cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:03.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.088+0000 7fa88ffff640 1 --2- 172.21.15.119:0/1096705256 >> v2:172.21.15.188:3300/0 conn(0x7fa8900cdcc0 0x7fa8900cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.088+0000 7fa88ffff640 1 -- 172.21.15.119:0/1096705256 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8900c9fb0 con 0x7fa890071da0 2023-11-08T19:27:03.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.089+0000 7fa88ffff640 1 --2- 172.21.15.119:0/1096705256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa890071da0 0x7fa8900cd0b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8900d0380 tx=0x7fa87c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9586822b0d800659 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.089+0000 7fa88f7fe640 1 -- 172.21.15.119:0/1096705256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c031860 con 0x7fa890071da0 2023-11-08T19:27:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.089+0000 7fa88f7fe640 1 -- 172.21.15.119:0/1096705256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87c0319c0 con 0x7fa890071da0 2023-11-08T19:27:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.089+0000 7fa88f7fe640 1 -- 172.21.15.119:0/1096705256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87c031d50 con 0x7fa890071da0 2023-11-08T19:27:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 -- 172.21.15.119:0/1096705256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa890071da0 msgr2=0x7fa8900cd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 --2- 172.21.15.119:0/1096705256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa890071da0 0x7fa8900cd0b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa8900d0380 tx=0x7fa87c02b900 comp rx=0 tx=0).stop 2023-11-08T19:27:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 -- 172.21.15.119:0/1096705256 shutdown_connections 2023-11-08T19:27:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 --2- 172.21.15.119:0/1096705256 >> v2:172.21.15.188:3300/0 conn(0x7fa8900cdcc0 0x7fa8900cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 --2- 172.21.15.119:0/1096705256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa890071da0 0x7fa8900cd0b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 --2- 172.21.15.119:0/1096705256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900719c0 0x7fa8900ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 -- 172.21.15.119:0/1096705256 >> 172.21.15.119:0/1096705256 conn(0x7fa8900cf4e0 msgr2=0x7fa8900cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 -- 172.21.15.119:0/1096705256 shutdown_connections 2023-11-08T19:27:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.090+0000 7fa896c07640 1 -- 172.21.15.119:0/1096705256 wait complete. 2023-11-08T19:27:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.092+0000 7fa896c07640 1 Processor -- start 2023-11-08T19:27:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.092+0000 7fa896c07640 1 -- start start 2023-11-08T19:27:03.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa896c07640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 0x7fa8900c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa896c07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa890071da0 0x7fa8900c4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa896c07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900cdcc0 0x7fa8900c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa896c07640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900c8d30 con 0x7fa8900719c0 2023-11-08T19:27:03.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa896c07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa890152e90 con 0x7fa890071da0 2023-11-08T19:27:03.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa896c07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900c80b0 con 0x7fa8900cdcc0 2023-11-08T19:27:03.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa8949a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 0x7fa8900c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa8949a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 0x7fa8900c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51898/0 (socket says 172.21.15.119:51898) 2023-11-08T19:27:03.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.093+0000 7fa8949a3640 1 -- 172.21.15.119:0/3296043357 learned_addr learned my addr 172.21.15.119:0/3296043357 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:03.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa88ffff640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa890071da0 0x7fa8900c4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa8949a3640 1 -- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900cdcc0 msgr2=0x7fa8900c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa8949a3640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900cdcc0 0x7fa8900c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa8949a3640 1 -- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa890071da0 msgr2=0x7fa8900c4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa8949a3640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa890071da0 0x7fa8900c4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa8949a3640 1 -- 172.21.15.119:0/3296043357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0094d0 con 0x7fa8900719c0 2023-11-08T19:27:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa8949a3640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 0x7fa8900c3010 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa88400d800 tx=0x7fa88400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8840096a0 con 0x7fa8900719c0 2023-11-08T19:27:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa884009800 con 0x7fa8900719c0 2023-11-08T19:27:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa884009b90 con 0x7fa8900719c0 2023-11-08T19:27:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8900c8390 con 0x7fa8900719c0 2023-11-08T19:27:03.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.094+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8900c8950 con 0x7fa8900719c0 2023-11-08T19:27:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.095+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89004ece0 con 0x7fa8900719c0 2023-11-08T19:27:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.097+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa884009cf0 con 0x7fa8900719c0 2023-11-08T19:27:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.098+0000 7fa88d7fa640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa87406ae70 0x7fa87406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.098+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5900+0+0 (secure 0 0 0) 0x7fa884089ac0 con 0x7fa8900719c0 2023-11-08T19:27:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.098+0000 7fa88ffff640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa87406ae70 0x7fa87406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.099+0000 7fa88ffff640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa87406ae70 0x7fa87406d330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa87c036000 tx=0x7fa87c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.102+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa884054b40 con 0x7fa8900719c0 2023-11-08T19:27:03.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.291+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa890065df0 con 0x7fa8900719c0 2023-11-08T19:27:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.872+0000 7fa88d7fa640 1 -- 172.21.15.119:0/3296043357 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa884089e60 con 0x7fa8900719c0 2023-11-08T19:27:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-11-08T19:27:03.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.877+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa87406ae70 msgr2=0x7fa87406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.877+0000 7fa896c07640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa87406ae70 0x7fa87406d330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa87c036000 tx=0x7fa87c03c040 comp rx=0 tx=0).stop 2023-11-08T19:27:03.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.877+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 msgr2=0x7fa8900c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:03.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.877+0000 7fa896c07640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 0x7fa8900c3010 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fa88400d800 tx=0x7fa88400dcc0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.878+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 shutdown_connections 2023-11-08T19:27:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.878+0000 7fa896c07640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa87406ae70 0x7fa87406d330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.878+0000 7fa896c07640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8900cdcc0 0x7fa8900c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.878+0000 7fa896c07640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa890071da0 0x7fa8900c4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.878+0000 7fa896c07640 1 --2- 172.21.15.119:0/3296043357 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8900719c0 0x7fa8900c3010 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.878+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 >> 172.21.15.119:0/3296043357 conn(0x7fa8900cf4e0 msgr2=0x7fa8900ce130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.879+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 shutdown_connections 2023-11-08T19:27:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:03.879+0000 7fa896c07640 1 -- 172.21.15.119:0/3296043357 wait complete. 2023-11-08T19:27:03.914 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:27:04.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.097+0000 7f4c3a97c640 1 Processor -- start 2023-11-08T19:27:04.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.097+0000 7f4c3a97c640 1 -- start start 2023-11-08T19:27:04.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.098+0000 7f4c3a97c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4c3415d000 0x7f4c340cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.098+0000 7f4c3a97c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 0x7f4c34160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.098+0000 7f4c3a97c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c3406d220 con 0x7f4c340c3a50 2023-11-08T19:27:04.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.098+0000 7f4c3a97c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340ccd50 con 0x7f4c3415d000 2023-11-08T19:27:04.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.098+0000 7f4c3a97c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340aca40 con 0x7f4c3406a830 2023-11-08T19:27:04.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.099+0000 7f4c38f19640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 0x7f4c34160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.099+0000 7f4c38f19640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 0x7f4c34160e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:36300/0 (socket says 172.21.15.119:36300) 2023-11-08T19:27:04.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.099+0000 7f4c38f19640 1 -- 172.21.15.119:0/2224770785 learned_addr learned my addr 172.21.15.119:0/2224770785 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:04.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.099+0000 7f4c2b7fe640 1 --1- 172.21.15.119:0/2224770785 >> v1:172.21.15.188:6789/0 conn(0x7f4c340c3a50 0x7f4c3406a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:46496/0 (socket says 172.21.15.119:46496) 2023-11-08T19:27:04.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.099+0000 7f4c33fff640 1 --2- 172.21.15.119:0/2224770785 >> v2:172.21.15.188:3300/0 conn(0x7f4c3415d000 0x7f4c340cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.100+0000 7f4c38f19640 1 -- 172.21.15.119:0/2224770785 >> v2:172.21.15.188:3300/0 conn(0x7f4c3415d000 msgr2=0x7f4c340cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:04.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.100+0000 7f4c38f19640 1 --2- 172.21.15.119:0/2224770785 >> v2:172.21.15.188:3300/0 conn(0x7f4c3415d000 0x7f4c340cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.100+0000 7f4c38f19640 1 -- 172.21.15.119:0/2224770785 >> v1:172.21.15.188:6789/0 conn(0x7f4c340c3a50 legacy=0x7f4c3406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.100+0000 7f4c38f19640 1 -- 172.21.15.119:0/2224770785 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340acd20 con 0x7f4c3406a830 2023-11-08T19:27:04.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.101+0000 7f4c38f19640 1 --2- 172.21.15.119:0/2224770785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 0x7f4c34160e90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009e50 tx=0x7f4c2c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a91bb6fc10ce878 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.101+0000 7f4c337fe640 1 -- 172.21.15.119:0/2224770785 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c02ca70 con 0x7f4c3406a830 2023-11-08T19:27:04.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c337fe640 1 -- 172.21.15.119:0/2224770785 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c02cbd0 con 0x7f4c3406a830 2023-11-08T19:27:04.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c337fe640 1 -- 172.21.15.119:0/2224770785 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c0343f0 con 0x7f4c3406a830 2023-11-08T19:27:04.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 -- 172.21.15.119:0/2224770785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 msgr2=0x7f4c34160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/2224770785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 0x7f4c34160e90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4c2c009e50 tx=0x7f4c2c02bed0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 -- 172.21.15.119:0/2224770785 shutdown_connections 2023-11-08T19:27:04.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/2224770785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c3406a830 0x7f4c34160e90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/2224770785 >> v2:172.21.15.188:3300/0 conn(0x7f4c3415d000 0x7f4c340cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 -- 172.21.15.119:0/2224770785 >> 172.21.15.119:0/2224770785 conn(0x7f4c340cb610 msgr2=0x7f4c340cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 -- 172.21.15.119:0/2224770785 shutdown_connections 2023-11-08T19:27:04.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.102+0000 7f4c3a97c640 1 -- 172.21.15.119:0/2224770785 wait complete. 2023-11-08T19:27:04.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.104+0000 7f4c3a97c640 1 Processor -- start 2023-11-08T19:27:04.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.104+0000 7f4c3a97c640 1 -- start start 2023-11-08T19:27:04.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c3a97c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 0x7f4c340af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c3a97c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c340c3a50 0x7f4c341f4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c3a97c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c3415d000 0x7f4c3414b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c3a97c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341f5100 con 0x7f4c3406a830 2023-11-08T19:27:04.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c3a97c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341f53e0 con 0x7f4c3415d000 2023-11-08T19:27:04.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c3a97c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341f56c0 con 0x7f4c340c3a50 2023-11-08T19:27:04.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.105+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 0x7f4c340af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 0x7f4c340af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51908/0 (socket says 172.21.15.119:51908) 2023-11-08T19:27:04.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 -- 172.21.15.119:0/1022078018 learned_addr learned my addr 172.21.15.119:0/1022078018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:04.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c2b7fe640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c340c3a50 0x7f4c341f4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c38f19640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c3415d000 0x7f4c3414b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 -- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c340c3a50 msgr2=0x7f4c341f4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c340c3a50 0x7f4c341f4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 -- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c3415d000 msgr2=0x7f4c3414b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c3415d000 0x7f4c3414b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 -- 172.21.15.119:0/1022078018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009b30 con 0x7f4c3406a830 2023-11-08T19:27:04.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c33fff640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 0x7f4c340af160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9b0 tx=0x7f4c2400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.106+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c24009670 con 0x7f4c3406a830 2023-11-08T19:27:04.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.107+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2400c9a0 con 0x7f4c3406a830 2023-11-08T19:27:04.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.107+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2400cd50 con 0x7f4c3406a830 2023-11-08T19:27:04.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.107+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c341f59a0 con 0x7f4c3406a830 2023-11-08T19:27:04.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.107+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c340b0a50 con 0x7f4c3406a830 2023-11-08T19:27:04.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.108+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c34074430 con 0x7f4c3406a830 2023-11-08T19:27:04.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.109+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c24013470 con 0x7f4c3406a830 2023-11-08T19:27:04.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.110+0000 7f4c317fa640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4c1406ae70 0x7f4c1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:04.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.110+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f4c24089aa0 con 0x7f4c3406a830 2023-11-08T19:27:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.110+0000 7f4c2b7fe640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4c1406ae70 0x7f4c1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:04.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.111+0000 7f4c2b7fe640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4c1406ae70 0x7f4c1406d330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c340cb050 tx=0x7f4c1c007c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:04.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.115+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c24054910 con 0x7f4c3406a830 2023-11-08T19:27:04.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.326+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4c3406cd50 con 0x7f4c3406a830 2023-11-08T19:27:04.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.898+0000 7f4c317fa640 1 -- 172.21.15.119:0/1022078018 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v22) v1 ==== 172+0+0 (secure 0 0 0) 0x7f4c240170c0 con 0x7f4c3406a830 2023-11-08T19:27:04.906 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 8 and data pool 9 2023-11-08T19:27:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.912+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4c1406ae70 msgr2=0x7f4c1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.912+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4c1406ae70 0x7f4c1406d330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4c340cb050 tx=0x7f4c1c007c20 comp rx=0 tx=0).stop 2023-11-08T19:27:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.912+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 msgr2=0x7f4c340af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:04.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.912+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 0x7f4c340af160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4c2400e9b0 tx=0x7f4c2400ee70 comp rx=0 tx=0).stop 2023-11-08T19:27:04.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.915+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 shutdown_connections 2023-11-08T19:27:04.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.915+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4c1406ae70 0x7f4c1406d330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.915+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c3415d000 0x7f4c3414b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.915+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c340c3a50 0x7f4c341f4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.915+0000 7f4c3a97c640 1 --2- 172.21.15.119:0/1022078018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4c3406a830 0x7f4c340af160 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:04.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.915+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 >> 172.21.15.119:0/1022078018 conn(0x7f4c340cb610 msgr2=0x7f4c3406ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:04.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.916+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 shutdown_connections 2023-11-08T19:27:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:04.916+0000 7f4c3a97c640 1 -- 172.21.15.119:0/1022078018 wait complete. 2023-11-08T19:27:04.945 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:05.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.115+0000 7fb8fac15640 1 Processor -- start 2023-11-08T19:27:05.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.115+0000 7fb8fac15640 1 -- start start 2023-11-08T19:27:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8fac15640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8f40cc810 0x7fb8f40c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8fac15640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8f415d180 0x7fb8f406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8fac15640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f40ae410 con 0x7fb8f406a390 2023-11-08T19:27:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8fac15640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f406ba30 con 0x7fb8f40cc810 2023-11-08T19:27:05.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8fac15640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f406bcf0 con 0x7fb8f415d180 2023-11-08T19:27:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8f40cc810 0x7fb8f40c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8f40cc810 0x7fb8f40c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51924/0 (socket says 172.21.15.119:51924) 2023-11-08T19:27:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 -- 172.21.15.119:0/1864449101 learned_addr learned my addr 172.21.15.119:0/1864449101 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8f91b2640 1 --2- 172.21.15.119:0/1864449101 >> v2:172.21.15.188:3300/0 conn(0x7fb8f415d180 0x7fb8f406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 -- 172.21.15.119:0/1864449101 >> v2:172.21.15.188:3300/0 conn(0x7fb8f415d180 msgr2=0x7fb8f406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 --2- 172.21.15.119:0/1864449101 >> v2:172.21.15.188:3300/0 conn(0x7fb8f415d180 0x7fb8f406cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 -- 172.21.15.119:0/1864449101 >> v1:172.21.15.188:6789/0 conn(0x7fb8f406a390 legacy=0x7fb8f415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8ebfff640 1 -- 172.21.15.119:0/1864449101 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f406bfd0 con 0x7fb8f40cc810 2023-11-08T19:27:05.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.117+0000 7fb8f91b2640 1 --2- 172.21.15.119:0/1864449101 >> v2:172.21.15.188:3300/0 conn(0x7fb8f415d180 0x7fb8f406cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:05.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.119+0000 7fb8ebfff640 1 --2- 172.21.15.119:0/1864449101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f40cc810 0x7fb8f40c4460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40c49a0 tx=0x7fb8e402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c977c80287fc775e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.119+0000 7fb8eb7fe640 1 -- 172.21.15.119:0/1864449101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4031870 con 0x7fb8f40cc810 2023-11-08T19:27:05.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.119+0000 7fb8eb7fe640 1 -- 172.21.15.119:0/1864449101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e40319d0 con 0x7fb8f40cc810 2023-11-08T19:27:05.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.119+0000 7fb8eb7fe640 1 -- 172.21.15.119:0/1864449101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e4031d60 con 0x7fb8f40cc810 2023-11-08T19:27:05.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 -- 172.21.15.119:0/1864449101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f40cc810 msgr2=0x7fb8f40c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 --2- 172.21.15.119:0/1864449101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f40cc810 0x7fb8f40c4460 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40c49a0 tx=0x7fb8e402b910 comp rx=0 tx=0).stop 2023-11-08T19:27:05.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 -- 172.21.15.119:0/1864449101 shutdown_connections 2023-11-08T19:27:05.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 --2- 172.21.15.119:0/1864449101 >> v2:172.21.15.188:3300/0 conn(0x7fb8f415d180 0x7fb8f406cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 --2- 172.21.15.119:0/1864449101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f40cc810 0x7fb8f40c4460 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 -- 172.21.15.119:0/1864449101 >> 172.21.15.119:0/1864449101 conn(0x7fb8f40cb610 msgr2=0x7fb8f40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:05.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 -- 172.21.15.119:0/1864449101 shutdown_connections 2023-11-08T19:27:05.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.120+0000 7fb8fac15640 1 -- 172.21.15.119:0/1864449101 wait complete. 2023-11-08T19:27:05.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.121+0000 7fb8fac15640 1 Processor -- start 2023-11-08T19:27:05.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.122+0000 7fb8fac15640 1 -- start start 2023-11-08T19:27:05.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.122+0000 7fb8fac15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f406a390 0x7fb8f415fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.122+0000 7fb8fac15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f40cc810 0x7fb8f4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.122+0000 7fb8fac15640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 0x7fb8f4155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8fac15640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415eda0 con 0x7fb8f415d180 2023-11-08T19:27:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8fac15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415f410 con 0x7fb8f406a390 2023-11-08T19:27:05.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8fac15640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f415f6f0 con 0x7fb8f40cc810 2023-11-08T19:27:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 0x7fb8f4155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 0x7fb8f4155880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51934/0 (socket says 172.21.15.119:51934) 2023-11-08T19:27:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 -- 172.21.15.119:0/509105580 learned_addr learned my addr 172.21.15.119:0/509105580 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 -- 172.21.15.119:0/509105580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f40cc810 msgr2=0x7fb8f4155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f40cc810 0x7fb8f4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 -- 172.21.15.119:0/509105580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f406a390 msgr2=0x7fb8f415fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f406a390 0x7fb8f415fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 -- 172.21.15.119:0/509105580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40094d0 con 0x7fb8f415d180 2023-11-08T19:27:05.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.123+0000 7fb8f91b2640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 0x7fb8f4155880 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00cd50 tx=0x7fb8ec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.124+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00f680 con 0x7fb8f415d180 2023-11-08T19:27:05.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.124+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec00f800 con 0x7fb8f415d180 2023-11-08T19:27:05.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.124+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec00fbb0 con 0x7fb8f415d180 2023-11-08T19:27:05.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.124+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f415f9d0 con 0x7fb8f415d180 2023-11-08T19:27:05.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.124+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f40b1380 con 0x7fb8f415d180 2023-11-08T19:27:05.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.125+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f415fc10 con 0x7fb8f415d180 2023-11-08T19:27:05.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.126+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8ec00b040 con 0x7fb8f415d180 2023-11-08T19:27:05.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.127+0000 7fb8e97fa640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8e006ae70 0x7fb8e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.127+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb8ec08a240 con 0x7fb8f415d180 2023-11-08T19:27:05.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.127+0000 7fb8f89b1640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8e006ae70 0x7fb8e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.128+0000 7fb8f89b1640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8e006ae70 0x7fb8e006d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0003cc0 tx=0x7fb8f00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.131+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8f415fc10 con 0x7fb8f415d180 2023-11-08T19:27:05.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.321+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8f415fc10 con 0x7fb8f415d180 2023-11-08T19:27:05.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.323+0000 7fb8e97fa640 1 -- 172.21.15.119:0/509105580 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18536 (secure 0 0 0) 0x7fb8f415fc10 con 0x7fb8f415d180 2023-11-08T19:27:05.324 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:05.325 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":53,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:04.328147+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:05.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8e006ae70 msgr2=0x7fb8e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8e006ae70 0x7fb8e006d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0003cc0 tx=0x7fb8f00079b0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 msgr2=0x7fb8f4155880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 0x7fb8f4155880 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00cd50 tx=0x7fb8ec00a650 comp rx=0 tx=0).stop 2023-11-08T19:27:05.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 shutdown_connections 2023-11-08T19:27:05.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8e006ae70 0x7fb8e006d330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8f415d180 0x7fb8f4155880 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8f40cc810 0x7fb8f4155340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.325+0000 7fb8fac15640 1 --2- 172.21.15.119:0/509105580 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8f406a390 0x7fb8f415fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.326+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 >> 172.21.15.119:0/509105580 conn(0x7fb8f40cb610 msgr2=0x7fb8f415b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:05.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.326+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 shutdown_connections 2023-11-08T19:27:05.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.326+0000 7fb8fac15640 1 -- 172.21.15.119:0/509105580 wait complete. 2023-11-08T19:27:05.345 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:05.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.551+0000 7f85a359e640 1 Processor -- start 2023-11-08T19:27:05.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.551+0000 7f85a359e640 1 -- start start 2023-11-08T19:27:05.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85a415b0d0 0x7f85a40ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85a415b900 0x7f85a4070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a359e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a406cff0 con 0x7f85a40710a0 2023-11-08T19:27:05.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a359e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a406d2d0 con 0x7f85a415b900 2023-11-08T19:27:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a406bd20 con 0x7f85a415b0d0 2023-11-08T19:27:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a1d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85a415b900 0x7f85a4070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a1d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85a415b900 0x7f85a4070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51944/0 (socket says 172.21.15.119:51944) 2023-11-08T19:27:05.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.552+0000 7f85a1d9b640 1 -- 172.21.15.119:0/1178347722 learned_addr learned my addr 172.21.15.119:0/1178347722 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:05.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.553+0000 7f85a259c640 1 --2- 172.21.15.119:0/1178347722 >> v2:172.21.15.188:3300/0 conn(0x7f85a415b0d0 0x7f85a40ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.553+0000 7f85a1d9b640 1 -- 172.21.15.119:0/1178347722 >> v2:172.21.15.188:3300/0 conn(0x7f85a415b0d0 msgr2=0x7f85a40ad7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.553+0000 7f85a1d9b640 1 --2- 172.21.15.119:0/1178347722 >> v2:172.21.15.188:3300/0 conn(0x7f85a415b0d0 0x7f85a40ad7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.553+0000 7f85a1d9b640 1 -- 172.21.15.119:0/1178347722 >> v1:172.21.15.188:6789/0 conn(0x7f85a40710a0 legacy=0x7f85a415da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.553+0000 7f85a1d9b640 1 -- 172.21.15.119:0/1178347722 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a406d760 con 0x7f85a415b900 2023-11-08T19:27:05.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.554+0000 7f85a1d9b640 1 --2- 172.21.15.119:0/1178347722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a4070b60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f85a406f240 tx=0x7f859402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212b478ebc214003 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.554+0000 7f85a159a640 1 -- 172.21.15.119:0/1178347722 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859402c9f0 con 0x7f85a415b900 2023-11-08T19:27:05.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.554+0000 7f85a159a640 1 -- 172.21.15.119:0/1178347722 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859402cb50 con 0x7f85a415b900 2023-11-08T19:27:05.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.554+0000 7f85a159a640 1 -- 172.21.15.119:0/1178347722 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85940343f0 con 0x7f85a415b900 2023-11-08T19:27:05.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 -- 172.21.15.119:0/1178347722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 msgr2=0x7f85a4070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 --2- 172.21.15.119:0/1178347722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a4070b60 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f85a406f240 tx=0x7f859402be50 comp rx=0 tx=0).stop 2023-11-08T19:27:05.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 -- 172.21.15.119:0/1178347722 shutdown_connections 2023-11-08T19:27:05.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 --2- 172.21.15.119:0/1178347722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a4070b60 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 --2- 172.21.15.119:0/1178347722 >> v2:172.21.15.188:3300/0 conn(0x7f85a415b0d0 0x7f85a40ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 -- 172.21.15.119:0/1178347722 >> 172.21.15.119:0/1178347722 conn(0x7f85a40b0560 msgr2=0x7f85a40b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:05.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 -- 172.21.15.119:0/1178347722 shutdown_connections 2023-11-08T19:27:05.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.555+0000 7f85a359e640 1 -- 172.21.15.119:0/1178347722 wait complete. 2023-11-08T19:27:05.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.557+0000 7f85a359e640 1 Processor -- start 2023-11-08T19:27:05.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.557+0000 7f85a359e640 1 -- start start 2023-11-08T19:27:05.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a40710a0 0x7f85a40c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a415b0d0 0x7f85a40b1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a40c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c6f40 con 0x7f85a415b900 2023-11-08T19:27:05.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c5180 con 0x7f85a415b0d0 2023-11-08T19:27:05.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c78d0 con 0x7f85a40710a0 2023-11-08T19:27:05.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a40c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a40c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51954/0 (socket says 172.21.15.119:51954) 2023-11-08T19:27:05.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.558+0000 7f85a2d9d640 1 -- 172.21.15.119:0/3925460307 learned_addr learned my addr 172.21.15.119:0/3925460307 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:05.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a2d9d640 1 -- 172.21.15.119:0/3925460307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a40710a0 msgr2=0x7f85a40c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a2d9d640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a40710a0 0x7f85a40c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a2d9d640 1 -- 172.21.15.119:0/3925460307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a415b0d0 msgr2=0x7f85a40b1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a2d9d640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a415b0d0 0x7f85a40b1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a2d9d640 1 -- 172.21.15.119:0/3925460307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594009b30 con 0x7f85a415b900 2023-11-08T19:27:05.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a2d9d640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a40c8320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b440 tx=0x7f859c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c011830 con 0x7f85a415b900 2023-11-08T19:27:05.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c011990 con 0x7f85a415b900 2023-11-08T19:27:05.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a40c7e90 con 0x7f85a415b900 2023-11-08T19:27:05.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c011d40 con 0x7f85a415b900 2023-11-08T19:27:05.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.559+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a40c7120 con 0x7f85a415b900 2023-11-08T19:27:05.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.560+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a404ece0 con 0x7f85a415b900 2023-11-08T19:27:05.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.561+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f859c01a400 con 0x7f85a415b900 2023-11-08T19:27:05.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.562+0000 7f859affd640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f858406aea0 0x7f858406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.562+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f859c0895b0 con 0x7f85a415b900 2023-11-08T19:27:05.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.563+0000 7f85a259c640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f858406aea0 0x7f858406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.563+0000 7f85a259c640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f858406aea0 0x7f858406d360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f858c006fd0 tx=0x7f858c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.567+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f859c0544a0 con 0x7f85a415b900 2023-11-08T19:27:05.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.743+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85a40c8020 con 0x7f85a415b900 2023-11-08T19:27:05.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.745+0000 7f859affd640 1 -- 172.21.15.119:0/3925460307 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18536 (secure 0 0 0) 0x7f859c057ac0 con 0x7f85a415b900 2023-11-08T19:27:05.747 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:05.748 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":53,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:04.328147+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:05.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f858406aea0 msgr2=0x7f858406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f858406aea0 0x7f858406d360 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f858c006fd0 tx=0x7f858c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:05.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 msgr2=0x7f85a40c8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a40c8320 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f859c00b440 tx=0x7f859c00b900 comp rx=0 tx=0).stop 2023-11-08T19:27:05.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 shutdown_connections 2023-11-08T19:27:05.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f858406aea0 0x7f858406d360 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85a415b900 0x7f85a40c8320 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85a415b0d0 0x7f85a40b1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 --2- 172.21.15.119:0/3925460307 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85a40710a0 0x7f85a40c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 >> 172.21.15.119:0/3925460307 conn(0x7f85a40b0560 msgr2=0x7f85a40afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:05.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 shutdown_connections 2023-11-08T19:27:05.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.748+0000 7f85a359e640 1 -- 172.21.15.119:0/3925460307 wait complete. 2023-11-08T19:27:05.769 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:27:05.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.966+0000 7f118dca9640 1 Processor -- start 2023-11-08T19:27:05.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.966+0000 7f118dca9640 1 -- start start 2023-11-08T19:27:05.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f118dca9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1188144580 0x7f118815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f118dca9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f118815f1d0 0x7f11880c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f118dca9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f118815ea60 con 0x7f1188144960 2023-11-08T19:27:05.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f118dca9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880c9150 con 0x7f118815f1d0 2023-11-08T19:27:05.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f118dca9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11880c9360 con 0x7f1188144580 2023-11-08T19:27:05.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f1187fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f118815f1d0 0x7f11880c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f1187fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f118815f1d0 0x7f11880c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51964/0 (socket says 172.21.15.119:51964) 2023-11-08T19:27:05.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f1187fff640 1 -- 172.21.15.119:0/1057172583 learned_addr learned my addr 172.21.15.119:0/1057172583 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:05.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.968+0000 7f11877fe640 1 --2- 172.21.15.119:0/1057172583 >> v2:172.21.15.188:3300/0 conn(0x7f1188144580 0x7f118815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.969+0000 7f1187fff640 1 -- 172.21.15.119:0/1057172583 >> v2:172.21.15.188:3300/0 conn(0x7f1188144580 msgr2=0x7f118815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.969+0000 7f1187fff640 1 --2- 172.21.15.119:0/1057172583 >> v2:172.21.15.188:3300/0 conn(0x7f1188144580 0x7f118815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.969+0000 7f1187fff640 1 -- 172.21.15.119:0/1057172583 >> v1:172.21.15.188:6789/0 conn(0x7f1188144960 legacy=0x7f11880cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:05.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.969+0000 7f1187fff640 1 -- 172.21.15.119:0/1057172583 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11880c9640 con 0x7f118815f1d0 2023-11-08T19:27:05.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.970+0000 7f1187fff640 1 --2- 172.21.15.119:0/1057172583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f11880c3a50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f11880ceab0 tx=0x7f118002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f5553157840392 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.970+0000 7f11867fc640 1 -- 172.21.15.119:0/1057172583 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118002c9f0 con 0x7f118815f1d0 2023-11-08T19:27:05.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.970+0000 7f11867fc640 1 -- 172.21.15.119:0/1057172583 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118002cb50 con 0x7f118815f1d0 2023-11-08T19:27:05.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.970+0000 7f11867fc640 1 -- 172.21.15.119:0/1057172583 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11800343f0 con 0x7f118815f1d0 2023-11-08T19:27:05.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 -- 172.21.15.119:0/1057172583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 msgr2=0x7f11880c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 --2- 172.21.15.119:0/1057172583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f11880c3a50 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f11880ceab0 tx=0x7f118002be50 comp rx=0 tx=0).stop 2023-11-08T19:27:05.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 -- 172.21.15.119:0/1057172583 shutdown_connections 2023-11-08T19:27:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 --2- 172.21.15.119:0/1057172583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f11880c3a50 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 --2- 172.21.15.119:0/1057172583 >> v2:172.21.15.188:3300/0 conn(0x7f1188144580 0x7f118815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 -- 172.21.15.119:0/1057172583 >> 172.21.15.119:0/1057172583 conn(0x7f11880cb610 msgr2=0x7f11880cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 -- 172.21.15.119:0/1057172583 shutdown_connections 2023-11-08T19:27:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.971+0000 7f118dca9640 1 -- 172.21.15.119:0/1057172583 wait complete. 2023-11-08T19:27:05.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.973+0000 7f118dca9640 1 Processor -- start 2023-11-08T19:27:05.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.973+0000 7f118dca9640 1 -- start start 2023-11-08T19:27:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f118dca9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1188144580 0x7f11880b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f118dca9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1188144960 0x7f11880cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f118dca9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f1188139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f118dca9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880ce0e0 con 0x7f118815f1d0 2023-11-08T19:27:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f118dca9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11880b05f0 con 0x7f1188144960 2023-11-08T19:27:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f118dca9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11880cdc50 con 0x7f1188144580 2023-11-08T19:27:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f1187fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f1188139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f1187fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f1188139450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51974/0 (socket says 172.21.15.119:51974) 2023-11-08T19:27:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f1187fff640 1 -- 172.21.15.119:0/3612625559 learned_addr learned my addr 172.21.15.119:0/3612625559 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.974+0000 7f1186ffd640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1188144960 0x7f11880cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f1187fff640 1 -- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1188144580 msgr2=0x7f11880b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f1187fff640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1188144580 0x7f11880b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f1187fff640 1 -- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1188144960 msgr2=0x7f11880cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f1187fff640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1188144960 0x7f11880cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f1187fff640 1 -- 172.21.15.119:0/3612625559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1180009b30 con 0x7f118815f1d0 2023-11-08T19:27:05.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f1187fff640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f1188139450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f118002c460 tx=0x7f1180004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1180009510 con 0x7f118815f1d0 2023-11-08T19:27:05.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f118003e450 con 0x7f118815f1d0 2023-11-08T19:27:05.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f118003e7e0 con 0x7f118815f1d0 2023-11-08T19:27:05.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11880cd040 con 0x7f118815f1d0 2023-11-08T19:27:05.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.975+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11880cd600 con 0x7f118815f1d0 2023-11-08T19:27:05.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.976+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1188074430 con 0x7f118815f1d0 2023-11-08T19:27:05.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.977+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f118003e940 con 0x7f118815f1d0 2023-11-08T19:27:05.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.978+0000 7f116ffff640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f117406ae70 0x7f117406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:05.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.978+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f11800ac900 con 0x7f118815f1d0 2023-11-08T19:27:05.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.979+0000 7f11877fe640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f117406ae70 0x7f117406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:05.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.979+0000 7f11877fe640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f117406ae70 0x7f117406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1178006fd0 tx=0x7f1178008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:05.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:05.983+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11800778c0 con 0x7f118815f1d0 2023-11-08T19:27:06.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.211+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f11880cdd90 con 0x7f118815f1d0 2023-11-08T19:27:06.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.916+0000 7f116ffff640 1 -- 172.21.15.119:0/3612625559 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f118007aee0 con 0x7f118815f1d0 2023-11-08T19:27:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.919+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f117406ae70 msgr2=0x7f117406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.919+0000 7f118dca9640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f117406ae70 0x7f117406d330 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1178006fd0 tx=0x7f1178008040 comp rx=0 tx=0).stop 2023-11-08T19:27:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.919+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 msgr2=0x7f1188139450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.919+0000 7f118dca9640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f1188139450 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f118002c460 tx=0x7f1180004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 shutdown_connections 2023-11-08T19:27:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f117406ae70 0x7f117406d330 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f118815f1d0 0x7f1188139450 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1188144960 0x7f11880cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 --2- 172.21.15.119:0/3612625559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1188144580 0x7f11880b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 >> 172.21.15.119:0/3612625559 conn(0x7f11880cb610 msgr2=0x7f118815dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 shutdown_connections 2023-11-08T19:27:06.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:06.920+0000 7f118dca9640 1 -- 172.21.15.119:0/3612625559 wait complete. 2023-11-08T19:27:06.942 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:27:06.942 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:27:07.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.151+0000 7f1ea4200640 1 Processor -- start 2023-11-08T19:27:07.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.151+0000 7f1ea4200640 1 -- start start 2023-11-08T19:27:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea4200640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e9c144580 0x7f1e9c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea4200640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e9c144960 0x7f1e9c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea4200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea4200640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c0c9150 con 0x7f1e9c144580 2023-11-08T19:27:07.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea4200640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e9c15ea60 con 0x7f1e9c144960 2023-11-08T19:27:07.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea4200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c0cb270 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea1f9c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e9c144580 0x7f1e9c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea1f9c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e9c144580 0x7f1e9c15bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51976/0 (socket says 172.21.15.119:51976) 2023-11-08T19:27:07.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.153+0000 7f1ea1f9c640 1 -- 172.21.15.119:0/927894177 learned_addr learned my addr 172.21.15.119:0/927894177 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:07.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea1f9c640 1 -- 172.21.15.119:0/927894177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c15f1d0 msgr2=0x7f1e9c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:07.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea279d640 1 --2- 172.21.15.119:0/927894177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea179b640 1 --2- 172.21.15.119:0/927894177 >> v2:172.21.15.188:3300/0 conn(0x7f1e9c144960 0x7f1e9c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea1f9c640 1 --2- 172.21.15.119:0/927894177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea1f9c640 1 -- 172.21.15.119:0/927894177 >> v2:172.21.15.188:3300/0 conn(0x7f1e9c144960 msgr2=0x7f1e9c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea1f9c640 1 --2- 172.21.15.119:0/927894177 >> v2:172.21.15.188:3300/0 conn(0x7f1e9c144960 0x7f1e9c0cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.154+0000 7f1ea1f9c640 1 -- 172.21.15.119:0/927894177 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c0c9360 con 0x7f1e9c144580 2023-11-08T19:27:07.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.155+0000 7f1ea1f9c640 1 --2- 172.21.15.119:0/927894177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c144580 0x7f1e9c15bff0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0ce900 tx=0x7f1e9802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510b0e60958ee31f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.155+0000 7f1ea0f9a640 1 -- 172.21.15.119:0/927894177 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9802c9f0 con 0x7f1e9c144580 2023-11-08T19:27:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.155+0000 7f1ea0f9a640 1 -- 172.21.15.119:0/927894177 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9802cb50 con 0x7f1e9c144580 2023-11-08T19:27:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.155+0000 7f1ea0f9a640 1 -- 172.21.15.119:0/927894177 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980343f0 con 0x7f1e9c144580 2023-11-08T19:27:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 -- 172.21.15.119:0/927894177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c144580 msgr2=0x7f1e9c15bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 --2- 172.21.15.119:0/927894177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c144580 0x7f1e9c15bff0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c0ce900 tx=0x7f1e9802be50 comp rx=0 tx=0).stop 2023-11-08T19:27:07.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 -- 172.21.15.119:0/927894177 shutdown_connections 2023-11-08T19:27:07.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 --2- 172.21.15.119:0/927894177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 --2- 172.21.15.119:0/927894177 >> v2:172.21.15.188:3300/0 conn(0x7f1e9c144960 0x7f1e9c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 --2- 172.21.15.119:0/927894177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c144580 0x7f1e9c15bff0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 -- 172.21.15.119:0/927894177 >> 172.21.15.119:0/927894177 conn(0x7f1e9c0cb610 msgr2=0x7f1e9c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:07.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 -- 172.21.15.119:0/927894177 shutdown_connections 2023-11-08T19:27:07.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.156+0000 7f1ea4200640 1 -- 172.21.15.119:0/927894177 wait complete. 2023-11-08T19:27:07.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.158+0000 7f1ea4200640 1 Processor -- start 2023-11-08T19:27:07.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.158+0000 7f1ea4200640 1 -- start start 2023-11-08T19:27:07.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea4200640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c144580 0x7f1e9c156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea4200640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c144960 0x7f1e9c139e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea4200640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea4200640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c15a500 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea4200640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c159c30 con 0x7f1e9c144580 2023-11-08T19:27:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea4200640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c13a360 con 0x7f1e9c144960 2023-11-08T19:27:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea279d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea279d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51984/0 (socket says 172.21.15.119:51984) 2023-11-08T19:27:07.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.159+0000 7f1ea279d640 1 -- 172.21.15.119:0/2188129011 learned_addr learned my addr 172.21.15.119:0/2188129011 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:07.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea179b640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c144960 0x7f1e9c139e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea279d640 1 -- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c144960 msgr2=0x7f1e9c139e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea279d640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c144960 0x7f1e9c139e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea279d640 1 -- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c144580 msgr2=0x7f1e9c156120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea1f9c640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c144580 0x7f1e9c156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea279d640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c144580 0x7f1e9c156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea279d640 1 -- 172.21.15.119:0/2188129011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98009b30 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea179b640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c144960 0x7f1e9c139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:07.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea279d640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c7920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c4a0 tx=0x7f1e9400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:07.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94004220 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e94004380 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c0c7e60 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.160+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94003830 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.161+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c159e40 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.162+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c15a640 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.163+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e94003990 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.163+0000 7f1e927fc640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e8006ae70 0x7f1e8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.164+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1e940894a0 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.164+0000 7f1ea1f9c640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e8006ae70 0x7f1e8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.165+0000 7f1ea1f9c640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e8006ae70 0x7f1e8006d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1e98035000 tx=0x7f1e98030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:07.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.168+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e9c15a640 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.394+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1e9c15a640 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.919+0000 7f1e927fc640 1 -- 172.21.15.119:0/2188129011 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1e9c15a640 con 0x7f1e9c15f1d0 2023-11-08T19:27:07.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.922+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e8006ae70 msgr2=0x7f1e8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.922+0000 7f1ea4200640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e8006ae70 0x7f1e8006d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f1e98035000 tx=0x7f1e98030000 comp rx=0 tx=0).stop 2023-11-08T19:27:07.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.922+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 msgr2=0x7f1e9c0c7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:07.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.922+0000 7f1ea4200640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c7920 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c4a0 tx=0x7f1e9400c960 comp rx=0 tx=0).stop 2023-11-08T19:27:07.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 shutdown_connections 2023-11-08T19:27:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e8006ae70 0x7f1e8006d330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e9c15f1d0 0x7f1e9c0c7920 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e9c144960 0x7f1e9c139e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 --2- 172.21.15.119:0/2188129011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e9c144580 0x7f1e9c156120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:07.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 >> 172.21.15.119:0/2188129011 conn(0x7f1e9c0cb610 msgr2=0x7f1e9c13dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:07.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 shutdown_connections 2023-11-08T19:27:07.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:07.923+0000 7f1ea4200640 1 -- 172.21.15.119:0/2188129011 wait complete. 2023-11-08T19:27:07.945 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:27:07.946 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:08.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.152+0000 7f22ddb00640 1 Processor -- start 2023-11-08T19:27:08.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.152+0000 7f22ddb00640 1 -- start start 2023-11-08T19:27:08.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.154+0000 7f22ddb00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f22d8142f80 0x7f22d80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.154+0000 7f22ddb00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80c4960 con 0x7f22d815d9f0 2023-11-08T19:27:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.154+0000 7f22ddb00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d815df60 con 0x7f22d806cb40 2023-11-08T19:27:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.154+0000 7f22ddb00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d8143a40 con 0x7f22d8142f80 2023-11-08T19:27:08.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.154+0000 7f22d6ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f22d815d9f0 0x7f22d80c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37566/0 (socket says 172.21.15.119:37566) 2023-11-08T19:27:08.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.154+0000 7f22d6ffd640 1 -- 172.21.15.119:0/895166651 learned_addr learned my addr 172.21.15.119:0/895166651 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:08.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d77fe640 1 --2- 172.21.15.119:0/895166651 >> v2:172.21.15.188:3300/0 conn(0x7f22d8142f80 0x7f22d80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177251535 0 0) 0x7f22d80c4960 con 0x7f22d815d9f0 2023-11-08T19:27:08.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22c4005730 con 0x7f22d815d9f0 2023-11-08T19:27:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625695479 0 0) 0x7f22c4005730 con 0x7f22d815d9f0 2023-11-08T19:27:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 >> v2:172.21.15.188:3300/0 conn(0x7f22d8142f80 msgr2=0x7f22d80cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 --2- 172.21.15.119:0/895166651 >> v2:172.21.15.188:3300/0 conn(0x7f22d8142f80 0x7f22d80cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 >> v1:172.21.15.188:6789/0 conn(0x7f22d806cb40 legacy=0x7f22d80c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d815e240 con 0x7f22d815d9f0 2023-11-08T19:27:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f22c0001670 con 0x7f22d815d9f0 2023-11-08T19:27:08.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 >> v1:172.21.15.119:6789/0 conn(0x7f22d815d9f0 legacy=0x7f22d80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c4005b80 0x7f22c4005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22c40069d0 0x7f22c4006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c4007390 0x7f22c400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8143a40 con 0x7f22c40069d0 2023-11-08T19:27:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c400e2e0 con 0x7f22c4005b80 2023-11-08T19:27:08.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.155+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22c400e4a0 con 0x7f22c4007390 2023-11-08T19:27:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22c40069d0 0x7f22c4006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d77fe640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c4005b80 0x7f22c4005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d6ffd640 1 -- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c4007390 msgr2=0x7f22c400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d7fff640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c4007390 0x7f22c400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22c4007390 0x7f22c400db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d6ffd640 1 -- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c4005b80 msgr2=0x7f22c4005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c4005b80 0x7f22c4005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d6ffd640 1 -- 172.21.15.119:0/895166651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4005730 con 0x7f22c40069d0 2023-11-08T19:27:08.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d77fe640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c4005b80 0x7f22c4005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.156+0000 7f22d7fff640 1 -- 172.21.15.119:0/895166651 reap_dead start 2023-11-08T19:27:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.157+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22c40069d0 0x7f22c4006e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f22c000c240 tx=0x7f22c4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7cb103e0a3fad8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.157+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0005ad0 con 0x7f22c40069d0 2023-11-08T19:27:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.157+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c0005c30 con 0x7f22c40069d0 2023-11-08T19:27:08.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.157+0000 7f22d67fc640 1 -- 172.21.15.119:0/895166651 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c002caf0 con 0x7f22c40069d0 2023-11-08T19:27:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 -- 172.21.15.119:0/895166651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22c40069d0 msgr2=0x7f22c4006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22c40069d0 0x7f22c4006e50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f22c000c240 tx=0x7f22c4007a00 comp rx=0 tx=0).stop 2023-11-08T19:27:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 -- 172.21.15.119:0/895166651 shutdown_connections 2023-11-08T19:27:08.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 --2- 172.21.15.119:0/895166651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22c40069d0 0x7f22c4006e50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 -- 172.21.15.119:0/895166651 >> 172.21.15.119:0/895166651 conn(0x7f22d80cb610 msgr2=0x7f22d80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 -- 172.21.15.119:0/895166651 shutdown_connections 2023-11-08T19:27:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.158+0000 7f22ddb00640 1 -- 172.21.15.119:0/895166651 wait complete. 2023-11-08T19:27:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.160+0000 7f22ddb00640 1 Processor -- start 2023-11-08T19:27:08.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.160+0000 7f22ddb00640 1 -- start start 2023-11-08T19:27:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22ddb00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c40069d0 0x7f22d80b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22ddb00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 0x7f22d8159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22ddb00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d8070e10 0x7f22d80712c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22ddb00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8071800 con 0x7f22d815c1f0 2023-11-08T19:27:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22ddb00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8159790 con 0x7f22c40069d0 2023-11-08T19:27:08.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22ddb00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d80b2950 con 0x7f22d8070e10 2023-11-08T19:27:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22d6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 0x7f22d8159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22d6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 0x7f22d8159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51998/0 (socket says 172.21.15.119:51998) 2023-11-08T19:27:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22d6ffd640 1 -- 172.21.15.119:0/1031157243 learned_addr learned my addr 172.21.15.119:0/1031157243 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:08.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.161+0000 7f22d77fe640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c40069d0 0x7f22d80b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d7fff640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d8070e10 0x7f22d80712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d6ffd640 1 -- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d8070e10 msgr2=0x7f22d80712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d8070e10 0x7f22d80712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d6ffd640 1 -- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c40069d0 msgr2=0x7f22d80b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c40069d0 0x7f22d80b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d6ffd640 1 -- 172.21.15.119:0/1031157243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80b2c30 con 0x7f22d815c1f0 2023-11-08T19:27:08.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d77fe640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c40069d0 0x7f22d80b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:08.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d7fff640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d8070e10 0x7f22d80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:08.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22d6ffd640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 0x7f22d8159250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f22c0001930 tx=0x7f22c002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c003c040 con 0x7f22d815c1f0 2023-11-08T19:27:08.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c002cd80 con 0x7f22d815c1f0 2023-11-08T19:27:08.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c0007290 con 0x7f22d815c1f0 2023-11-08T19:27:08.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80b2f10 con 0x7f22d815c1f0 2023-11-08T19:27:08.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.162+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d815c650 con 0x7f22d815c1f0 2023-11-08T19:27:08.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.164+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d8074430 con 0x7f22d815c1f0 2023-11-08T19:27:08.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.164+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22c0032070 con 0x7f22d815c1f0 2023-11-08T19:27:08.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.165+0000 7f22cffff640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f22b806ae70 0x7f22b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.165+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f22c00aae40 con 0x7f22d815c1f0 2023-11-08T19:27:08.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.165+0000 7f22d77fe640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f22b806ae70 0x7f22b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.166+0000 7f22d77fe640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f22b806ae70 0x7f22b806d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22d804f700 tx=0x7f22c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.169+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22c0075bb0 con 0x7f22d815c1f0 2023-11-08T19:27:08.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.397+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22d80c5990 con 0x7f22d815c1f0 2023-11-08T19:27:08.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.398+0000 7f22cffff640 1 -- 172.21.15.119:0/1031157243 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f22c00791d0 con 0x7f22d815c1f0 2023-11-08T19:27:08.399 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:08.400 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:04.329003+0000","modified":"2023-11-08T19:27:07.395675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/976565354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":976565354},{"type":"v1","addr":"172.21.15.119:6837","nonce":976565354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:27:08.400 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-11-08T19:27:08.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f22b806ae70 msgr2=0x7f22b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f22b806ae70 0x7f22b806d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22d804f700 tx=0x7f22c8008040 comp rx=0 tx=0).stop 2023-11-08T19:27:08.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 msgr2=0x7f22d8159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 0x7f22d8159250 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f22c0001930 tx=0x7f22c002e070 comp rx=0 tx=0).stop 2023-11-08T19:27:08.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 shutdown_connections 2023-11-08T19:27:08.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f22b806ae70 0x7f22b806d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22d8070e10 0x7f22d80712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22d815c1f0 0x7f22d8159250 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 --2- 172.21.15.119:0/1031157243 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22c40069d0 0x7f22d80b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 >> 172.21.15.119:0/1031157243 conn(0x7f22d80cb610 msgr2=0x7f22d80cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.400+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 shutdown_connections 2023-11-08T19:27:08.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.401+0000 7f22ddb00640 1 -- 172.21.15.119:0/1031157243 wait complete. 2023-11-08T19:27:08.421 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:08.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.634+0000 7f4846660640 1 Processor -- start 2023-11-08T19:27:08.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.634+0000 7f4846660640 1 -- start start 2023-11-08T19:27:08.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.636+0000 7f4846660640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f484006d520 0x7f484006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.636+0000 7f4846660640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 0x7f48400ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.636+0000 7f4846660640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484015a870 con 0x7f484006bee0 2023-11-08T19:27:08.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.636+0000 7f4846660640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484006cbc0 con 0x7f484006d520 2023-11-08T19:27:08.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.636+0000 7f4846660640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484015cfa0 con 0x7f48400cc640 2023-11-08T19:27:08.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 0x7f48400ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 0x7f48400ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:34408/0 (socket says 172.21.15.119:34408) 2023-11-08T19:27:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 -- 172.21.15.119:0/4076106079 learned_addr learned my addr 172.21.15.119:0/4076106079 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f484565e640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f484006bee0 0x7f484006c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:50596/0 (socket says 172.21.15.119:50596) 2023-11-08T19:27:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4844e5d640 1 --2- 172.21.15.119:0/4076106079 >> v2:172.21.15.188:3300/0 conn(0x7f484006d520 0x7f484006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 -- 172.21.15.119:0/4076106079 >> v2:172.21.15.188:3300/0 conn(0x7f484006d520 msgr2=0x7f484006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 --2- 172.21.15.119:0/4076106079 >> v2:172.21.15.188:3300/0 conn(0x7f484006d520 0x7f484006cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 -- 172.21.15.119:0/4076106079 >> v1:172.21.15.188:6789/0 conn(0x7f484006bee0 legacy=0x7f484006c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4845e5f640 1 -- 172.21.15.119:0/4076106079 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48400c9ad0 con 0x7f48400cc640 2023-11-08T19:27:08.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.637+0000 7f4844e5d640 1 --2- 172.21.15.119:0/4076106079 >> v2:172.21.15.188:3300/0 conn(0x7f484006d520 0x7f484006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:08.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.639+0000 7f4845e5f640 1 --2- 172.21.15.119:0/4076106079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 0x7f48400ccaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f483800d150 tx=0x7f483802e160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91dd3911de29165e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4837fff640 1 -- 172.21.15.119:0/4076106079 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838038400 con 0x7f48400cc640 2023-11-08T19:27:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4837fff640 1 -- 172.21.15.119:0/4076106079 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838032070 con 0x7f48400cc640 2023-11-08T19:27:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4837fff640 1 -- 172.21.15.119:0/4076106079 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4838038780 con 0x7f48400cc640 2023-11-08T19:27:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4846660640 1 -- 172.21.15.119:0/4076106079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 msgr2=0x7f48400ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4846660640 1 --2- 172.21.15.119:0/4076106079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 0x7f48400ccaf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f483800d150 tx=0x7f483802e160 comp rx=0 tx=0).stop 2023-11-08T19:27:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4846660640 1 -- 172.21.15.119:0/4076106079 shutdown_connections 2023-11-08T19:27:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4846660640 1 --2- 172.21.15.119:0/4076106079 >> v2:172.21.15.188:3300/0 conn(0x7f484006d520 0x7f484006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4846660640 1 --2- 172.21.15.119:0/4076106079 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48400cc640 0x7f48400ccaf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.640+0000 7f4846660640 1 -- 172.21.15.119:0/4076106079 >> 172.21.15.119:0/4076106079 conn(0x7f48400b0560 msgr2=0x7f48400b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.641+0000 7f4846660640 1 -- 172.21.15.119:0/4076106079 shutdown_connections 2023-11-08T19:27:08.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.641+0000 7f4846660640 1 -- 172.21.15.119:0/4076106079 wait complete. 2023-11-08T19:27:08.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.642+0000 7f4846660640 1 Processor -- start 2023-11-08T19:27:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.642+0000 7f4846660640 1 -- start start 2023-11-08T19:27:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.643+0000 7f4846660640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484006bee0 0x7f48400b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.643+0000 7f4846660640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484006d520 0x7f4840070b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.643+0000 7f4846660640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 0x7f48400b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.643+0000 7f4846660640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400c5480 con 0x7f48400cc640 2023-11-08T19:27:08.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.643+0000 7f4846660640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48400c2d20 con 0x7f484006bee0 2023-11-08T19:27:08.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.643+0000 7f4846660640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48400b4580 con 0x7f484006d520 2023-11-08T19:27:08.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 0x7f48400b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 0x7f48400b2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49834/0 (socket says 172.21.15.119:49834) 2023-11-08T19:27:08.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 -- 172.21.15.119:0/2221020816 learned_addr learned my addr 172.21.15.119:0/2221020816 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:08.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 -- 172.21.15.119:0/2221020816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484006d520 msgr2=0x7f4840070b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484006d520 0x7f4840070b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 -- 172.21.15.119:0/2221020816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484006bee0 msgr2=0x7f48400b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484006bee0 0x7f48400b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 -- 172.21.15.119:0/2221020816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4830009400 con 0x7f48400cc640 2023-11-08T19:27:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.644+0000 7f4845e5f640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 0x7f48400b2210 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f483802e770 tx=0x7f4838004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.645+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483800b720 con 0x7f48400cc640 2023-11-08T19:27:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.645+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4838042430 con 0x7f48400cc640 2023-11-08T19:27:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.645+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48380427c0 con 0x7f48400cc640 2023-11-08T19:27:08.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.645+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483800bd40 con 0x7f48400cc640 2023-11-08T19:27:08.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.645+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48400c26d0 con 0x7f48400cc640 2023-11-08T19:27:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.646+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484004ece0 con 0x7f48400cc640 2023-11-08T19:27:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.647+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4838042920 con 0x7f48400cc640 2023-11-08T19:27:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.648+0000 7f4835ffb640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f482406ae70 0x7f482406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:08.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.648+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f48380ac300 con 0x7f48400cc640 2023-11-08T19:27:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.648+0000 7f484565e640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f482406ae70 0x7f482406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:08.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.648+0000 7f484565e640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f482406ae70 0x7f482406d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f483c0015e0 tx=0x7f483c0079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:08.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.653+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48380771a0 con 0x7f48400cc640 2023-11-08T19:27:08.835 INFO:tasks.ceph.mds.c.smithi119.stderr:2023-11-08T19:27:08.834+0000 7fcb362aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:08.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.844+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48400c2eb0 con 0x7f48400cc640 2023-11-08T19:27:08.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.847+0000 7f4835ffb640 1 -- 172.21.15.119:0/2221020816 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18523 (secure 0 0 0) 0x7f483807a7c0 con 0x7f48400cc640 2023-11-08T19:27:08.849 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:08.849 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:05.896221+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f482406ae70 msgr2=0x7f482406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f482406ae70 0x7f482406d330 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f483c0015e0 tx=0x7f483c0079c0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 msgr2=0x7f48400b2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 0x7f48400b2210 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f483802e770 tx=0x7f4838004ab0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 shutdown_connections 2023-11-08T19:27:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f482406ae70 0x7f482406d330 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48400cc640 0x7f48400b2210 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f484006d520 0x7f4840070b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 --2- 172.21.15.119:0/2221020816 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f484006bee0 0x7f48400b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:08.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 >> 172.21.15.119:0/2221020816 conn(0x7f48400b0560 msgr2=0x7f48400ae940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:08.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 shutdown_connections 2023-11-08T19:27:08.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:08.850+0000 7f4846660640 1 -- 172.21.15.119:0/2221020816 wait complete. 2023-11-08T19:27:08.869 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:27:08.870 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:09.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.078+0000 7f711a22e640 1 Processor -- start 2023-11-08T19:27:09.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.078+0000 7f711a22e640 1 -- start start 2023-11-08T19:27:09.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.079+0000 7f711a22e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71140cac30 0x7f711415f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.079+0000 7f711a22e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71140adf20 0x7f711415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.079+0000 7f711a22e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711415f8c0 con 0x7f71140ad9d0 2023-11-08T19:27:09.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.079+0000 7f711a22e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7114071c70 con 0x7f71140adf20 2023-11-08T19:27:09.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.079+0000 7f711a22e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71140ca2e0 con 0x7f71140cac30 2023-11-08T19:27:09.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71140adf20 0x7f711415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71140adf20 0x7f711415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49850/0 (socket says 172.21.15.119:49850) 2023-11-08T19:27:09.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 -- 172.21.15.119:0/515479379 learned_addr learned my addr 172.21.15.119:0/515479379 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:09.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 -- 172.21.15.119:0/515479379 >> v2:172.21.15.188:3300/0 conn(0x7f71140cac30 msgr2=0x7f711415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 --2- 172.21.15.119:0/515479379 >> v2:172.21.15.188:3300/0 conn(0x7f71140cac30 0x7f711415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 -- 172.21.15.119:0/515479379 >> v1:172.21.15.188:6789/0 conn(0x7f71140ad9d0 legacy=0x7f71140c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.080+0000 7f7118a2b640 1 -- 172.21.15.119:0/515479379 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71140ca5c0 con 0x7f71140adf20 2023-11-08T19:27:09.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.081+0000 7f7118a2b640 1 --2- 172.21.15.119:0/515479379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140adf20 0x7f711415a910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f711406b410 tx=0x7f710402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8032f315226b63 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f710bfff640 1 -- 172.21.15.119:0/515479379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104031890 con 0x7f71140adf20 2023-11-08T19:27:09.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f710bfff640 1 -- 172.21.15.119:0/515479379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71040319f0 con 0x7f71140adf20 2023-11-08T19:27:09.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f710bfff640 1 -- 172.21.15.119:0/515479379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7104031d80 con 0x7f71140adf20 2023-11-08T19:27:09.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f711a22e640 1 -- 172.21.15.119:0/515479379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140adf20 msgr2=0x7f711415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f711a22e640 1 --2- 172.21.15.119:0/515479379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140adf20 0x7f711415a910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f711406b410 tx=0x7f710402b930 comp rx=0 tx=0).stop 2023-11-08T19:27:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f711a22e640 1 -- 172.21.15.119:0/515479379 shutdown_connections 2023-11-08T19:27:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f711a22e640 1 --2- 172.21.15.119:0/515479379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140adf20 0x7f711415a910 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f711a22e640 1 --2- 172.21.15.119:0/515479379 >> v2:172.21.15.188:3300/0 conn(0x7f71140cac30 0x7f711415f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.082+0000 7f711a22e640 1 -- 172.21.15.119:0/515479379 >> 172.21.15.119:0/515479379 conn(0x7f71140b0560 msgr2=0x7f71140b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.083+0000 7f711a22e640 1 -- 172.21.15.119:0/515479379 shutdown_connections 2023-11-08T19:27:09.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.083+0000 7f711a22e640 1 -- 172.21.15.119:0/515479379 wait complete. 2023-11-08T19:27:09.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.084+0000 7f711a22e640 1 Processor -- start 2023-11-08T19:27:09.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.084+0000 7f711a22e640 1 -- start start 2023-11-08T19:27:09.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.085+0000 7f711a22e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71140ad9d0 0x7f71141f20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.085+0000 7f711a22e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71140adf20 0x7f71140b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.085+0000 7f711a22e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 0x7f71140b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.085+0000 7f711a22e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71141f37a0 con 0x7f71140cac30 2023-11-08T19:27:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.085+0000 7f711a22e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711414fcd0 con 0x7f71140adf20 2023-11-08T19:27:09.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.085+0000 7f711a22e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71141f2630 con 0x7f71140ad9d0 2023-11-08T19:27:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 0x7f71140b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 0x7f71140b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49862/0 (socket says 172.21.15.119:49862) 2023-11-08T19:27:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 -- 172.21.15.119:0/3743021056 learned_addr learned my addr 172.21.15.119:0/3743021056 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7118a2b640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71140adf20 0x7f71140b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f711922c640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71140ad9d0 0x7f71141f20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 -- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71140ad9d0 msgr2=0x7f71141f20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71140ad9d0 0x7f71141f20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 -- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71140adf20 msgr2=0x7f71140b2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71140adf20 0x7f71140b2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.086+0000 7f7119a2d640 1 -- 172.21.15.119:0/3743021056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040094d0 con 0x7f71140cac30 2023-11-08T19:27:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.087+0000 7f7119a2d640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 0x7f71140b33d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c7b0 tx=0x7f710c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.087+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c003d80 con 0x7f71140cac30 2023-11-08T19:27:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.087+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710c00ce60 con 0x7f71140cac30 2023-11-08T19:27:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.087+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c0052b0 con 0x7f71140cac30 2023-11-08T19:27:09.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.087+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711414ef10 con 0x7f71140cac30 2023-11-08T19:27:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.087+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711414f4d0 con 0x7f71140cac30 2023-11-08T19:27:09.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.088+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711404ece0 con 0x7f71140cac30 2023-11-08T19:27:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.089+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f710c006b10 con 0x7f71140cac30 2023-11-08T19:27:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.090+0000 7f7109ffb640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f70f806ae70 0x7f70f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.090+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f710c088c00 con 0x7f71140cac30 2023-11-08T19:27:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.090+0000 7f711922c640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f70f806ae70 0x7f70f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.091+0000 7f711922c640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f70f806ae70 0x7f70f806d330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f71140c8fa0 tx=0x7f7110008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.094+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f710c053ab0 con 0x7f71140cac30 2023-11-08T19:27:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.281+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f71140c9c10 con 0x7f70f806ae70 2023-11-08T19:27:09.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.304+0000 7f7109ffb640 1 -- 172.21.15.119:0/3743021056 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368090 (secure 0 0 0) 0x7f71140c9c10 con 0x7f70f806ae70 2023-11-08T19:27:09.311 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:09.313 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:27:09.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.311+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f70f806ae70 msgr2=0x7f70f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.311+0000 7f711a22e640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f70f806ae70 0x7f70f806d330 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f71140c8fa0 tx=0x7f7110008040 comp rx=0 tx=0).stop 2023-11-08T19:27:09.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.311+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 msgr2=0x7f71140b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.311+0000 7f711a22e640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 0x7f71140b33d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f710c00c7b0 tx=0x7f710c00cc70 comp rx=0 tx=0).stop 2023-11-08T19:27:09.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 shutdown_connections 2023-11-08T19:27:09.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f70f806ae70 0x7f70f806d330 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71140cac30 0x7f71140b33d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71140adf20 0x7f71140b2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 --2- 172.21.15.119:0/3743021056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71140ad9d0 0x7f71141f20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 >> 172.21.15.119:0/3743021056 conn(0x7f71140b0560 msgr2=0x7f71140b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 shutdown_connections 2023-11-08T19:27:09.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.312+0000 7f711a22e640 1 -- 172.21.15.119:0/3743021056 wait complete. 2023-11-08T19:27:09.333 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-11-08T19:27:08.703549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230688,"kb_used_data":6180,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744032,"statfs":{"total":773094113280,"available":772857888768,"internally_reserved":0,"allocated":6328320,"data_stored":3497421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.840035"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.341382+0000","last_change":"2023-11-08T19:27:04.927066+0000","last_active":"2023-11-08T19:27:06.341382+0000","last_peered":"2023-11-08T19:27:06.341382+0000","last_clean":"2023-11-08T19:27:06.341382+0000","last_became_active":"2023-11-08T19:27:04.926172+0000","last_became_peered":"2023-11-08T19:27:04.926172+0000","last_unstale":"2023-11-08T19:27:06.341382+0000","last_undegraded":"2023-11-08T19:27:06.341382+0000","last_fullsized":"2023-11-08T19:27:06.341382+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:33.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935359+0000","last_change":"2023-11-08T19:27:04.927178+0000","last_active":"2023-11-08T19:27:04.935359+0000","last_peered":"2023-11-08T19:27:04.935359+0000","last_clean":"2023-11-08T19:27:04.935359+0000","last_became_active":"2023-11-08T19:27:04.926914+0000","last_became_peered":"2023-11-08T19:27:04.926914+0000","last_unstale":"2023-11-08T19:27:04.935359+0000","last_undegraded":"2023-11-08T19:27:04.935359+0000","last_fullsized":"2023-11-08T19:27:04.935359+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:42.243726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922440+0000","last_change":"2023-11-08T19:27:04.919694+0000","last_active":"2023-11-08T19:27:04.922440+0000","last_peered":"2023-11-08T19:27:04.922440+0000","last_clean":"2023-11-08T19:27:04.922440+0000","last_became_active":"2023-11-08T19:27:04.919400+0000","last_became_peered":"2023-11-08T19:27:04.919400+0000","last_unstale":"2023-11-08T19:27:04.922440+0000","last_undegraded":"2023-11-08T19:27:04.922440+0000","last_fullsized":"2023-11-08T19:27:04.922440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:43.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917471+0000","last_change":"2023-11-08T19:27:04.919426+0000","last_active":"2023-11-08T19:27:05.917471+0000","last_peered":"2023-11-08T19:27:05.917471+0000","last_clean":"2023-11-08T19:27:05.917471+0000","last_became_active":"2023-11-08T19:27:04.919120+0000","last_became_peered":"2023-11-08T19:27:04.919120+0000","last_unstale":"2023-11-08T19:27:05.917471+0000","last_undegraded":"2023-11-08T19:27:05.917471+0000","last_fullsized":"2023-11-08T19:27:05.917471+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:47.306710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340675+0000","last_change":"2023-11-08T19:27:04.918284+0000","last_active":"2023-11-08T19:27:06.340675+0000","last_peered":"2023-11-08T19:27:06.340675+0000","last_clean":"2023-11-08T19:27:06.340675+0000","last_became_active":"2023-11-08T19:27:04.918036+0000","last_became_peered":"2023-11-08T19:27:04.918036+0000","last_unstale":"2023-11-08T19:27:06.340675+0000","last_undegraded":"2023-11-08T19:27:06.340675+0000","last_fullsized":"2023-11-08T19:27:06.340675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:23:09.377486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.933114+0000","last_change":"2023-11-08T19:27:04.929182+0000","last_active":"2023-11-08T19:27:04.933114+0000","last_peered":"2023-11-08T19:27:04.933114+0000","last_clean":"2023-11-08T19:27:04.933114+0000","last_became_active":"2023-11-08T19:27:04.928895+0000","last_became_peered":"2023-11-08T19:27:04.928895+0000","last_unstale":"2023-11-08T19:27:04.933114+0000","last_undegraded":"2023-11-08T19:27:04.933114+0000","last_fullsized":"2023-11-08T19:27:04.933114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:38.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.911608+0000","last_change":"2023-11-08T19:27:04.906410+0000","last_active":"2023-11-08T19:27:04.911608+0000","last_peered":"2023-11-08T19:27:04.911608+0000","last_clean":"2023-11-08T19:27:04.911608+0000","last_became_active":"2023-11-08T19:27:04.906107+0000","last_became_peered":"2023-11-08T19:27:04.906107+0000","last_unstale":"2023-11-08T19:27:04.911608+0000","last_undegraded":"2023-11-08T19:27:04.911608+0000","last_fullsized":"2023-11-08T19:27:04.911608+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:14.313670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922481+0000","last_change":"2023-11-08T19:27:04.907534+0000","last_active":"2023-11-08T19:27:04.922481+0000","last_peered":"2023-11-08T19:27:04.922481+0000","last_clean":"2023-11-08T19:27:04.922481+0000","last_became_active":"2023-11-08T19:27:04.907274+0000","last_became_peered":"2023-11-08T19:27:04.907274+0000","last_unstale":"2023-11-08T19:27:04.922481+0000","last_undegraded":"2023-11-08T19:27:04.922481+0000","last_fullsized":"2023-11-08T19:27:04.922481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:27.403294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.921980+0000","last_change":"2023-11-08T19:27:04.916608+0000","last_active":"2023-11-08T19:27:04.921980+0000","last_peered":"2023-11-08T19:27:04.921980+0000","last_clean":"2023-11-08T19:27:04.921980+0000","last_became_active":"2023-11-08T19:27:04.916141+0000","last_became_peered":"2023-11-08T19:27:04.916141+0000","last_unstale":"2023-11-08T19:27:04.921980+0000","last_undegraded":"2023-11-08T19:27:04.921980+0000","last_fullsized":"2023-11-08T19:27:04.921980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:27.487518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.910214+0000","last_change":"2023-11-08T19:27:04.909096+0000","last_active":"2023-11-08T19:27:05.910214+0000","last_peered":"2023-11-08T19:27:05.910214+0000","last_clean":"2023-11-08T19:27:05.910214+0000","last_became_active":"2023-11-08T19:27:04.908258+0000","last_became_peered":"2023-11-08T19:27:04.908258+0000","last_unstale":"2023-11-08T19:27:05.910214+0000","last_undegraded":"2023-11-08T19:27:05.910214+0000","last_fullsized":"2023-11-08T19:27:05.910214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:33.506251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909973+0000","last_change":"2023-11-08T19:27:04.912053+0000","last_active":"2023-11-08T19:27:05.909973+0000","last_peered":"2023-11-08T19:27:05.909973+0000","last_clean":"2023-11-08T19:27:05.909973+0000","last_became_active":"2023-11-08T19:27:04.911775+0000","last_became_peered":"2023-11-08T19:27:04.911775+0000","last_unstale":"2023-11-08T19:27:05.909973+0000","last_undegraded":"2023-11-08T19:27:05.909973+0000","last_fullsized":"2023-11-08T19:27:05.909973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:59:01.049984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917195+0000","last_change":"2023-11-08T19:27:04.909143+0000","last_active":"2023-11-08T19:27:05.917195+0000","last_peered":"2023-11-08T19:27:05.917195+0000","last_clean":"2023-11-08T19:27:05.917195+0000","last_became_active":"2023-11-08T19:27:04.907886+0000","last_became_peered":"2023-11-08T19:27:04.907886+0000","last_unstale":"2023-11-08T19:27:05.917195+0000","last_undegraded":"2023-11-08T19:27:05.917195+0000","last_fullsized":"2023-11-08T19:27:05.917195+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:37.168547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914165+0000","last_change":"2023-11-08T19:27:04.928932+0000","last_active":"2023-11-08T19:27:05.914165+0000","last_peered":"2023-11-08T19:27:05.914165+0000","last_clean":"2023-11-08T19:27:05.914165+0000","last_became_active":"2023-11-08T19:27:04.928660+0000","last_became_peered":"2023-11-08T19:27:04.928660+0000","last_unstale":"2023-11-08T19:27:05.914165+0000","last_undegraded":"2023-11-08T19:27:05.914165+0000","last_fullsized":"2023-11-08T19:27:05.914165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:24.476633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.925182+0000","last_change":"2023-11-08T19:27:04.920158+0000","last_active":"2023-11-08T19:27:04.925182+0000","last_peered":"2023-11-08T19:27:04.925182+0000","last_clean":"2023-11-08T19:27:04.925182+0000","last_became_active":"2023-11-08T19:27:04.919862+0000","last_became_peered":"2023-11-08T19:27:04.919862+0000","last_unstale":"2023-11-08T19:27:04.925182+0000","last_undegraded":"2023-11-08T19:27:04.925182+0000","last_fullsized":"2023-11-08T19:27:04.925182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:04.629985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.923965+0000","last_change":"2023-11-08T19:27:04.918153+0000","last_active":"2023-11-08T19:27:04.923965+0000","last_peered":"2023-11-08T19:27:04.923965+0000","last_clean":"2023-11-08T19:27:04.923965+0000","last_became_active":"2023-11-08T19:27:04.917885+0000","last_became_peered":"2023-11-08T19:27:04.917885+0000","last_unstale":"2023-11-08T19:27:04.923965+0000","last_undegraded":"2023-11-08T19:27:04.923965+0000","last_fullsized":"2023-11-08T19:27:04.923965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:52:22.451586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.934339+0000","last_change":"2023-11-08T19:27:04.931573+0000","last_active":"2023-11-08T19:27:04.934339+0000","last_peered":"2023-11-08T19:27:04.934339+0000","last_clean":"2023-11-08T19:27:04.934339+0000","last_became_active":"2023-11-08T19:27:04.931100+0000","last_became_peered":"2023-11-08T19:27:04.931100+0000","last_unstale":"2023-11-08T19:27:04.934339+0000","last_undegraded":"2023-11-08T19:27:04.934339+0000","last_fullsized":"2023-11-08T19:27:04.934339+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:05.247418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922595+0000","last_change":"2023-11-08T19:27:04.913083+0000","last_active":"2023-11-08T19:27:04.922595+0000","last_peered":"2023-11-08T19:27:04.922595+0000","last_clean":"2023-11-08T19:27:04.922595+0000","last_became_active":"2023-11-08T19:27:04.912789+0000","last_became_peered":"2023-11-08T19:27:04.912789+0000","last_unstale":"2023-11-08T19:27:04.922595+0000","last_undegraded":"2023-11-08T19:27:04.922595+0000","last_fullsized":"2023-11-08T19:27:04.922595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:00.218479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922264+0000","last_change":"2023-11-08T19:27:04.918068+0000","last_active":"2023-11-08T19:27:04.922264+0000","last_peered":"2023-11-08T19:27:04.922264+0000","last_clean":"2023-11-08T19:27:04.922264+0000","last_became_active":"2023-11-08T19:27:04.917742+0000","last_became_peered":"2023-11-08T19:27:04.917742+0000","last_unstale":"2023-11-08T19:27:04.922264+0000","last_undegraded":"2023-11-08T19:27:04.922264+0000","last_fullsized":"2023-11-08T19:27:04.922264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:27.487518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340821+0000","last_change":"2023-11-08T19:27:04.918112+0000","last_active":"2023-11-08T19:27:06.340821+0000","last_peered":"2023-11-08T19:27:06.340821+0000","last_clean":"2023-11-08T19:27:06.340821+0000","last_became_active":"2023-11-08T19:27:04.917823+0000","last_became_peered":"2023-11-08T19:27:04.917823+0000","last_unstale":"2023-11-08T19:27:06.340821+0000","last_undegraded":"2023-11-08T19:27:06.340821+0000","last_fullsized":"2023-11-08T19:27:06.340821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:37.539021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935074+0000","last_change":"2023-11-08T19:27:04.926456+0000","last_active":"2023-11-08T19:27:04.935074+0000","last_peered":"2023-11-08T19:27:04.935074+0000","last_clean":"2023-11-08T19:27:04.935074+0000","last_became_active":"2023-11-08T19:27:04.923368+0000","last_became_peered":"2023-11-08T19:27:04.923368+0000","last_unstale":"2023-11-08T19:27:04.935074+0000","last_undegraded":"2023-11-08T19:27:04.935074+0000","last_fullsized":"2023-11-08T19:27:04.935074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:41.490954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.916228+0000","last_change":"2023-11-08T19:27:04.911552+0000","last_active":"2023-11-08T19:27:04.916228+0000","last_peered":"2023-11-08T19:27:04.916228+0000","last_clean":"2023-11-08T19:27:04.916228+0000","last_became_active":"2023-11-08T19:27:04.911199+0000","last_became_peered":"2023-11-08T19:27:04.911199+0000","last_unstale":"2023-11-08T19:27:04.916228+0000","last_undegraded":"2023-11-08T19:27:04.916228+0000","last_fullsized":"2023-11-08T19:27:04.916228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:47.590564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.931670+0000","last_change":"2023-11-08T19:27:04.922120+0000","last_active":"2023-11-08T19:27:04.931670+0000","last_peered":"2023-11-08T19:27:04.931670+0000","last_clean":"2023-11-08T19:27:04.931670+0000","last_became_active":"2023-11-08T19:27:04.921852+0000","last_became_peered":"2023-11-08T19:27:04.921852+0000","last_unstale":"2023-11-08T19:27:04.931670+0000","last_undegraded":"2023-11-08T19:27:04.931670+0000","last_fullsized":"2023-11-08T19:27:04.931670+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:57.229596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.928431+0000","last_change":"2023-11-08T19:27:04.925637+0000","last_active":"2023-11-08T19:27:04.928431+0000","last_peered":"2023-11-08T19:27:04.928431+0000","last_clean":"2023-11-08T19:27:04.928431+0000","last_became_active":"2023-11-08T19:27:04.925319+0000","last_became_peered":"2023-11-08T19:27:04.925319+0000","last_unstale":"2023-11-08T19:27:04.928431+0000","last_undegraded":"2023-11-08T19:27:04.928431+0000","last_fullsized":"2023-11-08T19:27:04.928431+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:12:41.183786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.913022+0000","last_change":"2023-11-08T19:27:04.920182+0000","last_active":"2023-11-08T19:27:05.913022+0000","last_peered":"2023-11-08T19:27:05.913022+0000","last_clean":"2023-11-08T19:27:05.913022+0000","last_became_active":"2023-11-08T19:27:04.917978+0000","last_became_peered":"2023-11-08T19:27:04.917978+0000","last_unstale":"2023-11-08T19:27:05.913022+0000","last_undegraded":"2023-11-08T19:27:05.913022+0000","last_fullsized":"2023-11-08T19:27:05.913022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:13.185529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.341401+0000","last_change":"2023-11-08T19:27:04.919872+0000","last_active":"2023-11-08T19:27:06.341401+0000","last_peered":"2023-11-08T19:27:06.341401+0000","last_clean":"2023-11-08T19:27:06.341401+0000","last_became_active":"2023-11-08T19:27:04.919566+0000","last_became_peered":"2023-11-08T19:27:04.919566+0000","last_unstale":"2023-11-08T19:27:06.341401+0000","last_undegraded":"2023-11-08T19:27:06.341401+0000","last_fullsized":"2023-11-08T19:27:06.341401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:13:56.548842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916386+0000","last_change":"2023-11-08T19:27:04.904565+0000","last_active":"2023-11-08T19:27:05.916386+0000","last_peered":"2023-11-08T19:27:05.916386+0000","last_clean":"2023-11-08T19:27:05.916386+0000","last_became_active":"2023-11-08T19:27:04.904237+0000","last_became_peered":"2023-11-08T19:27:04.904237+0000","last_unstale":"2023-11-08T19:27:05.916386+0000","last_undegraded":"2023-11-08T19:27:05.916386+0000","last_fullsized":"2023-11-08T19:27:05.916386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:27.403294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.934040+0000","last_change":"2023-11-08T19:27:04.929199+0000","last_active":"2023-11-08T19:27:04.934040+0000","last_peered":"2023-11-08T19:27:04.934040+0000","last_clean":"2023-11-08T19:27:04.934040+0000","last_became_active":"2023-11-08T19:27:04.928847+0000","last_became_peered":"2023-11-08T19:27:04.928847+0000","last_unstale":"2023-11-08T19:27:04.934040+0000","last_undegraded":"2023-11-08T19:27:04.934040+0000","last_fullsized":"2023-11-08T19:27:04.934040+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:09:05.549788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340742+0000","last_change":"2023-11-08T19:27:04.915372+0000","last_active":"2023-11-08T19:27:06.340742+0000","last_peered":"2023-11-08T19:27:06.340742+0000","last_clean":"2023-11-08T19:27:06.340742+0000","last_became_active":"2023-11-08T19:27:04.915101+0000","last_became_peered":"2023-11-08T19:27:04.915101+0000","last_unstale":"2023-11-08T19:27:06.340742+0000","last_undegraded":"2023-11-08T19:27:06.340742+0000","last_fullsized":"2023-11-08T19:27:06.340742+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:39.250906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.921039+0000","last_change":"2023-11-08T19:27:04.912922+0000","last_active":"2023-11-08T19:27:04.921039+0000","last_peered":"2023-11-08T19:27:04.921039+0000","last_clean":"2023-11-08T19:27:04.921039+0000","last_became_active":"2023-11-08T19:27:04.912557+0000","last_became_peered":"2023-11-08T19:27:04.912557+0000","last_unstale":"2023-11-08T19:27:04.921039+0000","last_undegraded":"2023-11-08T19:27:04.921039+0000","last_fullsized":"2023-11-08T19:27:04.921039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:09:57.760385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.920564+0000","last_change":"2023-11-08T19:27:04.913834+0000","last_active":"2023-11-08T19:27:04.920564+0000","last_peered":"2023-11-08T19:27:04.920564+0000","last_clean":"2023-11-08T19:27:04.920564+0000","last_became_active":"2023-11-08T19:27:04.913084+0000","last_became_peered":"2023-11-08T19:27:04.913084+0000","last_unstale":"2023-11-08T19:27:04.920564+0000","last_undegraded":"2023-11-08T19:27:04.920564+0000","last_fullsized":"2023-11-08T19:27:04.920564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:29.856508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340291+0000","last_change":"2023-11-08T19:27:04.920753+0000","last_active":"2023-11-08T19:27:06.340291+0000","last_peered":"2023-11-08T19:27:06.340291+0000","last_clean":"2023-11-08T19:27:06.340291+0000","last_became_active":"2023-11-08T19:27:04.920360+0000","last_became_peered":"2023-11-08T19:27:04.920360+0000","last_unstale":"2023-11-08T19:27:06.340291+0000","last_undegraded":"2023-11-08T19:27:06.340291+0000","last_fullsized":"2023-11-08T19:27:06.340291+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:22:03.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.925926+0000","last_change":"2023-11-08T19:27:04.915874+0000","last_active":"2023-11-08T19:27:04.925926+0000","last_peered":"2023-11-08T19:27:04.925926+0000","last_clean":"2023-11-08T19:27:04.925926+0000","last_became_active":"2023-11-08T19:27:04.915597+0000","last_became_peered":"2023-11-08T19:27:04.915597+0000","last_unstale":"2023-11-08T19:27:04.925926+0000","last_undegraded":"2023-11-08T19:27:04.925926+0000","last_fullsized":"2023-11-08T19:27:04.925926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:59:01.049984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.920626+0000","last_change":"2023-11-08T19:27:04.915405+0000","last_active":"2023-11-08T19:27:04.920626+0000","last_peered":"2023-11-08T19:27:04.920626+0000","last_clean":"2023-11-08T19:27:04.920626+0000","last_became_active":"2023-11-08T19:27:04.915013+0000","last_became_peered":"2023-11-08T19:27:04.915013+0000","last_unstale":"2023-11-08T19:27:04.920626+0000","last_undegraded":"2023-11-08T19:27:04.920626+0000","last_fullsized":"2023-11-08T19:27:04.920626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:27.370958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.913382+0000","last_change":"2023-11-08T19:27:03.912055+0000","last_active":"2023-11-08T19:27:05.913382+0000","last_peered":"2023-11-08T19:27:05.913382+0000","last_clean":"2023-11-08T19:27:05.913382+0000","last_became_active":"2023-11-08T19:27:03.911771+0000","last_became_peered":"2023-11-08T19:27:03.911771+0000","last_unstale":"2023-11-08T19:27:05.913382+0000","last_undegraded":"2023-11-08T19:27:05.913382+0000","last_fullsized":"2023-11-08T19:27:05.913382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:46.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.888135+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:27:04.888135+0000","last_peered":"2023-11-08T19:27:04.888135+0000","last_clean":"2023-11-08T19:27:04.888135+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:27:04.888135+0000","last_undegraded":"2023-11-08T19:27:04.888135+0000","last_fullsized":"2023-11-08T19:27:04.888135+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.915878+0000","last_change":"2023-11-08T19:27:04.911411+0000","last_active":"2023-11-08T19:27:05.915878+0000","last_peered":"2023-11-08T19:27:05.915878+0000","last_clean":"2023-11-08T19:27:05.915878+0000","last_became_active":"2023-11-08T19:27:04.909022+0000","last_became_peered":"2023-11-08T19:27:04.909022+0000","last_unstale":"2023-11-08T19:27:05.915878+0000","last_undegraded":"2023-11-08T19:27:05.915878+0000","last_fullsized":"2023-11-08T19:27:05.915878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:43.705543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.940534+0000","last_change":"2023-11-08T19:27:03.908080+0000","last_active":"2023-11-08T19:27:04.940534+0000","last_peered":"2023-11-08T19:27:04.940534+0000","last_clean":"2023-11-08T19:27:04.940534+0000","last_became_active":"2023-11-08T19:27:03.907721+0000","last_became_peered":"2023-11-08T19:27:03.907721+0000","last_unstale":"2023-11-08T19:27:04.940534+0000","last_undegraded":"2023-11-08T19:27:04.940534+0000","last_fullsized":"2023-11-08T19:27:04.940534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:31:54.746884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.894291+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:27:04.894291+0000","last_peered":"2023-11-08T19:27:04.894291+0000","last_clean":"2023-11-08T19:27:04.894291+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:27:04.894291+0000","last_undegraded":"2023-11-08T19:27:04.894291+0000","last_fullsized":"2023-11-08T19:27:04.894291+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.928450+0000","last_change":"2023-11-08T19:27:04.925710+0000","last_active":"2023-11-08T19:27:04.928450+0000","last_peered":"2023-11-08T19:27:04.928450+0000","last_clean":"2023-11-08T19:27:04.928450+0000","last_became_active":"2023-11-08T19:27:04.925395+0000","last_became_peered":"2023-11-08T19:27:04.925395+0000","last_unstale":"2023-11-08T19:27:04.928450+0000","last_undegraded":"2023-11-08T19:27:04.928450+0000","last_fullsized":"2023-11-08T19:27:04.928450+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:58.532658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.937447+0000","last_change":"2023-11-08T19:27:03.907134+0000","last_active":"2023-11-08T19:27:04.937447+0000","last_peered":"2023-11-08T19:27:04.937447+0000","last_clean":"2023-11-08T19:27:04.937447+0000","last_became_active":"2023-11-08T19:27:03.906810+0000","last_became_peered":"2023-11-08T19:27:03.906810+0000","last_unstale":"2023-11-08T19:27:04.937447+0000","last_undegraded":"2023-11-08T19:27:04.937447+0000","last_fullsized":"2023-11-08T19:27:04.937447+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:03.616731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.899784+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:27:04.899784+0000","last_peered":"2023-11-08T19:27:04.899784+0000","last_clean":"2023-11-08T19:27:04.899784+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:27:04.899784+0000","last_undegraded":"2023-11-08T19:27:04.899784+0000","last_fullsized":"2023-11-08T19:27:04.899784+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.915221+0000","last_change":"2023-11-08T19:27:04.912592+0000","last_active":"2023-11-08T19:27:04.915221+0000","last_peered":"2023-11-08T19:27:04.915221+0000","last_clean":"2023-11-08T19:27:04.915221+0000","last_became_active":"2023-11-08T19:27:04.912250+0000","last_became_peered":"2023-11-08T19:27:04.912250+0000","last_unstale":"2023-11-08T19:27:04.915221+0000","last_undegraded":"2023-11-08T19:27:04.915221+0000","last_fullsized":"2023-11-08T19:27:04.915221+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:47.590564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.915175+0000","last_change":"2023-11-08T19:27:03.904545+0000","last_active":"2023-11-08T19:27:05.915175+0000","last_peered":"2023-11-08T19:27:05.915175+0000","last_clean":"2023-11-08T19:27:05.915175+0000","last_became_active":"2023-11-08T19:27:03.904242+0000","last_became_peered":"2023-11-08T19:27:03.904242+0000","last_unstale":"2023-11-08T19:27:05.915175+0000","last_undegraded":"2023-11-08T19:27:05.915175+0000","last_fullsized":"2023-11-08T19:27:05.915175+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:12.652246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.889374+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:27:04.889374+0000","last_peered":"2023-11-08T19:27:04.889374+0000","last_clean":"2023-11-08T19:27:04.889374+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:27:04.889374+0000","last_undegraded":"2023-11-08T19:27:04.889374+0000","last_fullsized":"2023-11-08T19:27:04.889374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.934218+0000","last_change":"2023-11-08T19:27:04.931559+0000","last_active":"2023-11-08T19:27:04.934218+0000","last_peered":"2023-11-08T19:27:04.934218+0000","last_clean":"2023-11-08T19:27:04.934218+0000","last_became_active":"2023-11-08T19:27:04.931294+0000","last_became_peered":"2023-11-08T19:27:04.931294+0000","last_unstale":"2023-11-08T19:27:04.934218+0000","last_undegraded":"2023-11-08T19:27:04.934218+0000","last_fullsized":"2023-11-08T19:27:04.934218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:04.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.902585+0000","last_change":"2023-11-08T19:27:03.907571+0000","last_active":"2023-11-08T19:27:04.902585+0000","last_peered":"2023-11-08T19:27:04.902585+0000","last_clean":"2023-11-08T19:27:04.902585+0000","last_became_active":"2023-11-08T19:27:03.907272+0000","last_became_peered":"2023-11-08T19:27:03.907272+0000","last_unstale":"2023-11-08T19:27:04.902585+0000","last_undegraded":"2023-11-08T19:27:04.902585+0000","last_fullsized":"2023-11-08T19:27:04.902585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:06.474193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.896550+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:27:04.896550+0000","last_peered":"2023-11-08T19:27:04.896550+0000","last_clean":"2023-11-08T19:27:04.896550+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:27:04.896550+0000","last_undegraded":"2023-11-08T19:27:04.896550+0000","last_fullsized":"2023-11-08T19:27:04.896550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.918820+0000","last_change":"2023-11-08T19:27:04.914786+0000","last_active":"2023-11-08T19:27:04.918820+0000","last_peered":"2023-11-08T19:27:04.918820+0000","last_clean":"2023-11-08T19:27:04.918820+0000","last_became_active":"2023-11-08T19:27:04.914428+0000","last_became_peered":"2023-11-08T19:27:04.914428+0000","last_unstale":"2023-11-08T19:27:04.918820+0000","last_undegraded":"2023-11-08T19:27:04.918820+0000","last_fullsized":"2023-11-08T19:27:04.918820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:12:41.183786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.904225+0000","last_change":"2023-11-08T19:27:03.908441+0000","last_active":"2023-11-08T19:27:04.904225+0000","last_peered":"2023-11-08T19:27:04.904225+0000","last_clean":"2023-11-08T19:27:04.904225+0000","last_became_active":"2023-11-08T19:27:03.908074+0000","last_became_peered":"2023-11-08T19:27:03.908074+0000","last_unstale":"2023-11-08T19:27:04.904225+0000","last_undegraded":"2023-11-08T19:27:04.904225+0000","last_fullsized":"2023-11-08T19:27:04.904225+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:59:00.036730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.881017+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:27:03.881017+0000","last_peered":"2023-11-08T19:27:03.881017+0000","last_clean":"2023-11-08T19:27:03.881017+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:27:03.881017+0000","last_undegraded":"2023-11-08T19:27:03.881017+0000","last_fullsized":"2023-11-08T19:27:03.881017+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938059+0000","last_change":"2023-11-08T19:27:04.937203+0000","last_active":"2023-11-08T19:27:04.938059+0000","last_peered":"2023-11-08T19:27:04.938059+0000","last_clean":"2023-11-08T19:27:04.938059+0000","last_became_active":"2023-11-08T19:27:04.936912+0000","last_became_peered":"2023-11-08T19:27:04.936912+0000","last_unstale":"2023-11-08T19:27:04.938059+0000","last_undegraded":"2023-11-08T19:27:04.938059+0000","last_fullsized":"2023-11-08T19:27:04.938059+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:10:10.152395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.940630+0000","last_change":"2023-11-08T19:27:03.899892+0000","last_active":"2023-11-08T19:27:04.940630+0000","last_peered":"2023-11-08T19:27:04.940630+0000","last_clean":"2023-11-08T19:27:04.940630+0000","last_became_active":"2023-11-08T19:27:03.899580+0000","last_became_peered":"2023-11-08T19:27:03.899580+0000","last_unstale":"2023-11-08T19:27:04.940630+0000","last_undegraded":"2023-11-08T19:27:04.940630+0000","last_fullsized":"2023-11-08T19:27:04.940630+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:03.036895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.882073+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:27:03.882073+0000","last_peered":"2023-11-08T19:27:03.882073+0000","last_clean":"2023-11-08T19:27:03.882073+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:27:03.882073+0000","last_undegraded":"2023-11-08T19:27:03.882073+0000","last_fullsized":"2023-11-08T19:27:03.882073+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909628+0000","last_change":"2023-11-08T19:27:04.901305+0000","last_active":"2023-11-08T19:27:05.909628+0000","last_peered":"2023-11-08T19:27:05.909628+0000","last_clean":"2023-11-08T19:27:05.909628+0000","last_became_active":"2023-11-08T19:27:04.900993+0000","last_became_peered":"2023-11-08T19:27:04.900993+0000","last_unstale":"2023-11-08T19:27:05.909628+0000","last_undegraded":"2023-11-08T19:27:05.909628+0000","last_fullsized":"2023-11-08T19:27:05.909628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:27.370958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936994+0000","last_change":"2023-11-08T19:27:03.902924+0000","last_active":"2023-11-08T19:27:04.936994+0000","last_peered":"2023-11-08T19:27:04.936994+0000","last_clean":"2023-11-08T19:27:04.936994+0000","last_became_active":"2023-11-08T19:27:03.902417+0000","last_became_peered":"2023-11-08T19:27:03.902417+0000","last_unstale":"2023-11-08T19:27:04.936994+0000","last_undegraded":"2023-11-08T19:27:04.936994+0000","last_fullsized":"2023-11-08T19:27:04.936994+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:56.923012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":27,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.895589+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:27:03.895589+0000","last_peered":"2023-11-08T19:27:03.895589+0000","last_clean":"2023-11-08T19:27:03.895589+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:27:03.895589+0000","last_undegraded":"2023-11-08T19:27:03.895589+0000","last_fullsized":"2023-11-08T19:27:03.895589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936773+0000","last_change":"2023-11-08T19:27:04.928702+0000","last_active":"2023-11-08T19:27:04.936773+0000","last_peered":"2023-11-08T19:27:04.936773+0000","last_clean":"2023-11-08T19:27:04.936773+0000","last_became_active":"2023-11-08T19:27:04.928408+0000","last_became_peered":"2023-11-08T19:27:04.928408+0000","last_unstale":"2023-11-08T19:27:04.936773+0000","last_undegraded":"2023-11-08T19:27:04.936773+0000","last_fullsized":"2023-11-08T19:27:04.936773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:17.846233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.932720+0000","last_change":"2023-11-08T19:27:03.909927+0000","last_active":"2023-11-08T19:27:04.932720+0000","last_peered":"2023-11-08T19:27:04.932720+0000","last_clean":"2023-11-08T19:27:04.932720+0000","last_became_active":"2023-11-08T19:27:03.909666+0000","last_became_peered":"2023-11-08T19:27:03.909666+0000","last_unstale":"2023-11-08T19:27:04.932720+0000","last_undegraded":"2023-11-08T19:27:04.932720+0000","last_fullsized":"2023-11-08T19:27:04.932720+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:12.652246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.898221+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:27:03.898221+0000","last_peered":"2023-11-08T19:27:03.898221+0000","last_clean":"2023-11-08T19:27:03.898221+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:27:03.898221+0000","last_undegraded":"2023-11-08T19:27:03.898221+0000","last_fullsized":"2023-11-08T19:27:03.898221+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935000+0000","last_change":"2023-11-08T19:27:04.924022+0000","last_active":"2023-11-08T19:27:04.935000+0000","last_peered":"2023-11-08T19:27:04.935000+0000","last_clean":"2023-11-08T19:27:04.935000+0000","last_became_active":"2023-11-08T19:27:04.923732+0000","last_became_peered":"2023-11-08T19:27:04.923732+0000","last_unstale":"2023-11-08T19:27:04.935000+0000","last_undegraded":"2023-11-08T19:27:04.935000+0000","last_fullsized":"2023-11-08T19:27:04.935000+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:37.168547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927129+0000","last_change":"2023-11-08T19:27:03.904899+0000","last_active":"2023-11-08T19:27:04.927129+0000","last_peered":"2023-11-08T19:27:04.927129+0000","last_clean":"2023-11-08T19:27:04.927129+0000","last_became_active":"2023-11-08T19:27:03.904141+0000","last_became_peered":"2023-11-08T19:27:03.904141+0000","last_unstale":"2023-11-08T19:27:04.927129+0000","last_undegraded":"2023-11-08T19:27:04.927129+0000","last_fullsized":"2023-11-08T19:27:04.927129+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:57.519404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.884400+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:27:03.884400+0000","last_peered":"2023-11-08T19:27:03.884400+0000","last_clean":"2023-11-08T19:27:03.884400+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:27:03.884400+0000","last_undegraded":"2023-11-08T19:27:03.884400+0000","last_fullsized":"2023-11-08T19:27:03.884400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.923229+0000","last_change":"2023-11-08T19:27:04.913816+0000","last_active":"2023-11-08T19:27:04.923229+0000","last_peered":"2023-11-08T19:27:04.923229+0000","last_clean":"2023-11-08T19:27:04.923229+0000","last_became_active":"2023-11-08T19:27:04.913569+0000","last_became_peered":"2023-11-08T19:27:04.913569+0000","last_unstale":"2023-11-08T19:27:04.923229+0000","last_undegraded":"2023-11-08T19:27:04.923229+0000","last_fullsized":"2023-11-08T19:27:04.923229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:41.017102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.889880+0000","last_change":"2023-11-08T19:27:03.904570+0000","last_active":"2023-11-08T19:27:04.889880+0000","last_peered":"2023-11-08T19:27:04.889880+0000","last_clean":"2023-11-08T19:27:04.889880+0000","last_became_active":"2023-11-08T19:27:03.904254+0000","last_became_peered":"2023-11-08T19:27:03.904254+0000","last_unstale":"2023-11-08T19:27:04.889880+0000","last_undegraded":"2023-11-08T19:27:04.889880+0000","last_fullsized":"2023-11-08T19:27:04.889880+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:28.843254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.897783+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:27:03.897783+0000","last_peered":"2023-11-08T19:27:03.897783+0000","last_clean":"2023-11-08T19:27:03.897783+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:27:03.897783+0000","last_undegraded":"2023-11-08T19:27:03.897783+0000","last_fullsized":"2023-11-08T19:27:03.897783+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.913393+0000","last_change":"2023-11-08T19:27:04.914624+0000","last_active":"2023-11-08T19:27:05.913393+0000","last_peered":"2023-11-08T19:27:05.913393+0000","last_clean":"2023-11-08T19:27:05.913393+0000","last_became_active":"2023-11-08T19:27:04.911514+0000","last_became_peered":"2023-11-08T19:27:04.911514+0000","last_unstale":"2023-11-08T19:27:05.913393+0000","last_undegraded":"2023-11-08T19:27:05.913393+0000","last_fullsized":"2023-11-08T19:27:05.913393+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:28:50.948226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914998+0000","last_change":"2023-11-08T19:27:03.900167+0000","last_active":"2023-11-08T19:27:05.914998+0000","last_peered":"2023-11-08T19:27:05.914998+0000","last_clean":"2023-11-08T19:27:05.914998+0000","last_became_active":"2023-11-08T19:27:03.899708+0000","last_became_peered":"2023-11-08T19:27:03.899708+0000","last_unstale":"2023-11-08T19:27:05.914998+0000","last_undegraded":"2023-11-08T19:27:05.914998+0000","last_fullsized":"2023-11-08T19:27:05.914998+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:32.492997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":27,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.895753+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:27:03.895753+0000","last_peered":"2023-11-08T19:27:03.895753+0000","last_clean":"2023-11-08T19:27:03.895753+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:27:03.895753+0000","last_undegraded":"2023-11-08T19:27:03.895753+0000","last_fullsized":"2023-11-08T19:27:03.895753+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.910397+0000","last_change":"2023-11-08T19:27:04.915733+0000","last_active":"2023-11-08T19:27:05.910397+0000","last_peered":"2023-11-08T19:27:05.910397+0000","last_clean":"2023-11-08T19:27:05.910397+0000","last_became_active":"2023-11-08T19:27:04.915478+0000","last_became_peered":"2023-11-08T19:27:04.915478+0000","last_unstale":"2023-11-08T19:27:05.910397+0000","last_undegraded":"2023-11-08T19:27:05.910397+0000","last_fullsized":"2023-11-08T19:27:05.910397+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:57.936266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938381+0000","last_change":"2023-11-08T19:27:03.910798+0000","last_active":"2023-11-08T19:27:04.938381+0000","last_peered":"2023-11-08T19:27:04.938381+0000","last_clean":"2023-11-08T19:27:04.938381+0000","last_became_active":"2023-11-08T19:27:03.910524+0000","last_became_peered":"2023-11-08T19:27:03.910524+0000","last_unstale":"2023-11-08T19:27:04.938381+0000","last_undegraded":"2023-11-08T19:27:04.938381+0000","last_fullsized":"2023-11-08T19:27:04.938381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:26.743339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.896665+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:27:03.896665+0000","last_peered":"2023-11-08T19:27:03.896665+0000","last_clean":"2023-11-08T19:27:03.896665+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:27:03.896665+0000","last_undegraded":"2023-11-08T19:27:03.896665+0000","last_fullsized":"2023-11-08T19:27:03.896665+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927262+0000","last_change":"2023-11-08T19:27:04.914804+0000","last_active":"2023-11-08T19:27:04.927262+0000","last_peered":"2023-11-08T19:27:04.927262+0000","last_clean":"2023-11-08T19:27:04.927262+0000","last_became_active":"2023-11-08T19:27:04.913952+0000","last_became_peered":"2023-11-08T19:27:04.913952+0000","last_unstale":"2023-11-08T19:27:04.927262+0000","last_undegraded":"2023-11-08T19:27:04.927262+0000","last_fullsized":"2023-11-08T19:27:04.927262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:29.856508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938081+0000","last_change":"2023-11-08T19:27:03.901576+0000","last_active":"2023-11-08T19:27:04.938081+0000","last_peered":"2023-11-08T19:27:04.938081+0000","last_clean":"2023-11-08T19:27:04.938081+0000","last_became_active":"2023-11-08T19:27:03.900686+0000","last_became_peered":"2023-11-08T19:27:03.900686+0000","last_unstale":"2023-11-08T19:27:04.938081+0000","last_undegraded":"2023-11-08T19:27:04.938081+0000","last_fullsized":"2023-11-08T19:27:04.938081+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:46.914427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.886338+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:27:03.886338+0000","last_peered":"2023-11-08T19:27:03.886338+0000","last_clean":"2023-11-08T19:27:03.886338+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:27:03.886338+0000","last_undegraded":"2023-11-08T19:27:03.886338+0000","last_fullsized":"2023-11-08T19:27:03.886338+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.932003+0000","last_change":"2023-11-08T19:27:04.922551+0000","last_active":"2023-11-08T19:27:04.932003+0000","last_peered":"2023-11-08T19:27:04.932003+0000","last_clean":"2023-11-08T19:27:04.932003+0000","last_became_active":"2023-11-08T19:27:04.920930+0000","last_became_peered":"2023-11-08T19:27:04.920930+0000","last_unstale":"2023-11-08T19:27:04.932003+0000","last_undegraded":"2023-11-08T19:27:04.932003+0000","last_fullsized":"2023-11-08T19:27:04.932003+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:34.302718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917277+0000","last_change":"2023-11-08T19:27:03.904810+0000","last_active":"2023-11-08T19:27:05.917277+0000","last_peered":"2023-11-08T19:27:05.917277+0000","last_clean":"2023-11-08T19:27:05.917277+0000","last_became_active":"2023-11-08T19:27:03.904328+0000","last_became_peered":"2023-11-08T19:27:03.904328+0000","last_unstale":"2023-11-08T19:27:05.917277+0000","last_undegraded":"2023-11-08T19:27:05.917277+0000","last_fullsized":"2023-11-08T19:27:05.917277+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:46.914427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.882286+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:27:03.882286+0000","last_peered":"2023-11-08T19:27:03.882286+0000","last_clean":"2023-11-08T19:27:03.882286+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:27:03.882286+0000","last_undegraded":"2023-11-08T19:27:03.882286+0000","last_fullsized":"2023-11-08T19:27:03.882286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.932902+0000","last_change":"2023-11-08T19:27:04.923408+0000","last_active":"2023-11-08T19:27:04.932902+0000","last_peered":"2023-11-08T19:27:04.932902+0000","last_clean":"2023-11-08T19:27:04.932902+0000","last_became_active":"2023-11-08T19:27:04.923068+0000","last_became_peered":"2023-11-08T19:27:04.923068+0000","last_unstale":"2023-11-08T19:27:04.932902+0000","last_undegraded":"2023-11-08T19:27:04.932902+0000","last_fullsized":"2023-11-08T19:27:04.932902+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:43.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936782+0000","last_change":"2023-11-08T19:27:03.894748+0000","last_active":"2023-11-08T19:27:04.936782+0000","last_peered":"2023-11-08T19:27:04.936782+0000","last_clean":"2023-11-08T19:27:04.936782+0000","last_became_active":"2023-11-08T19:27:03.889635+0000","last_became_peered":"2023-11-08T19:27:03.889635+0000","last_unstale":"2023-11-08T19:27:04.936782+0000","last_undegraded":"2023-11-08T19:27:04.936782+0000","last_fullsized":"2023-11-08T19:27:04.936782+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:42.692289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.900111+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:27:04.900111+0000","last_peered":"2023-11-08T19:27:04.900111+0000","last_clean":"2023-11-08T19:27:04.900111+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:27:04.900111+0000","last_undegraded":"2023-11-08T19:27:04.900111+0000","last_fullsized":"2023-11-08T19:27:04.900111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927983+0000","last_change":"2023-11-08T19:27:04.925637+0000","last_active":"2023-11-08T19:27:04.927983+0000","last_peered":"2023-11-08T19:27:04.927983+0000","last_clean":"2023-11-08T19:27:04.927983+0000","last_became_active":"2023-11-08T19:27:04.925083+0000","last_became_peered":"2023-11-08T19:27:04.925083+0000","last_unstale":"2023-11-08T19:27:04.927983+0000","last_undegraded":"2023-11-08T19:27:04.927983+0000","last_fullsized":"2023-11-08T19:27:04.927983+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:05.247418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.890479+0000","last_change":"2023-11-08T19:27:03.901613+0000","last_active":"2023-11-08T19:27:04.890479+0000","last_peered":"2023-11-08T19:27:04.890479+0000","last_clean":"2023-11-08T19:27:04.890479+0000","last_became_active":"2023-11-08T19:27:03.901279+0000","last_became_peered":"2023-11-08T19:27:03.901279+0000","last_unstale":"2023-11-08T19:27:04.890479+0000","last_undegraded":"2023-11-08T19:27:04.890479+0000","last_fullsized":"2023-11-08T19:27:04.890479+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:26.357704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.900813+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:27:04.900813+0000","last_peered":"2023-11-08T19:27:04.900813+0000","last_clean":"2023-11-08T19:27:04.900813+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:27:04.900813+0000","last_undegraded":"2023-11-08T19:27:04.900813+0000","last_fullsized":"2023-11-08T19:27:04.900813+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935314+0000","last_change":"2023-11-08T19:27:04.927030+0000","last_active":"2023-11-08T19:27:04.935314+0000","last_peered":"2023-11-08T19:27:04.935314+0000","last_clean":"2023-11-08T19:27:04.935314+0000","last_became_active":"2023-11-08T19:27:04.925015+0000","last_became_peered":"2023-11-08T19:27:04.925015+0000","last_unstale":"2023-11-08T19:27:04.935314+0000","last_undegraded":"2023-11-08T19:27:04.935314+0000","last_fullsized":"2023-11-08T19:27:04.935314+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:47.306710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909733+0000","last_change":"2023-11-08T19:27:03.893095+0000","last_active":"2023-11-08T19:27:05.909733+0000","last_peered":"2023-11-08T19:27:05.909733+0000","last_clean":"2023-11-08T19:27:05.909733+0000","last_became_active":"2023-11-08T19:27:03.892485+0000","last_became_peered":"2023-11-08T19:27:03.892485+0000","last_unstale":"2023-11-08T19:27:05.909733+0000","last_undegraded":"2023-11-08T19:27:05.909733+0000","last_fullsized":"2023-11-08T19:27:05.909733+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:13.300416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.900354+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:27:04.900354+0000","last_peered":"2023-11-08T19:27:04.900354+0000","last_clean":"2023-11-08T19:27:04.900354+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:27:04.900354+0000","last_undegraded":"2023-11-08T19:27:04.900354+0000","last_fullsized":"2023-11-08T19:27:04.900354+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.924823+0000","last_change":"2023-11-08T19:27:04.919386+0000","last_active":"2023-11-08T19:27:04.924823+0000","last_peered":"2023-11-08T19:27:04.924823+0000","last_clean":"2023-11-08T19:27:04.924823+0000","last_became_active":"2023-11-08T19:27:04.919083+0000","last_became_peered":"2023-11-08T19:27:04.919083+0000","last_unstale":"2023-11-08T19:27:04.924823+0000","last_undegraded":"2023-11-08T19:27:04.924823+0000","last_fullsized":"2023-11-08T19:27:04.924823+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:27.843034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.890834+0000","last_change":"2023-11-08T19:27:03.905592+0000","last_active":"2023-11-08T19:27:04.890834+0000","last_peered":"2023-11-08T19:27:04.890834+0000","last_clean":"2023-11-08T19:27:04.890834+0000","last_became_active":"2023-11-08T19:27:03.905323+0000","last_became_peered":"2023-11-08T19:27:03.905323+0000","last_unstale":"2023-11-08T19:27:04.890834+0000","last_undegraded":"2023-11-08T19:27:04.890834+0000","last_fullsized":"2023-11-08T19:27:04.890834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:42.692289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.887290+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:27:03.887290+0000","last_peered":"2023-11-08T19:27:03.887290+0000","last_clean":"2023-11-08T19:27:03.887290+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:27:03.887290+0000","last_undegraded":"2023-11-08T19:27:03.887290+0000","last_fullsized":"2023-11-08T19:27:03.887290+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909760+0000","last_change":"2023-11-08T19:27:04.910277+0000","last_active":"2023-11-08T19:27:05.909760+0000","last_peered":"2023-11-08T19:27:05.909760+0000","last_clean":"2023-11-08T19:27:05.909760+0000","last_became_active":"2023-11-08T19:27:04.909995+0000","last_became_peered":"2023-11-08T19:27:04.909995+0000","last_unstale":"2023-11-08T19:27:05.909760+0000","last_undegraded":"2023-11-08T19:27:05.909760+0000","last_fullsized":"2023-11-08T19:27:05.909760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:05.247418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909355+0000","last_change":"2023-11-08T19:27:03.885745+0000","last_active":"2023-11-08T19:27:05.909355+0000","last_peered":"2023-11-08T19:27:05.909355+0000","last_clean":"2023-11-08T19:27:05.909355+0000","last_became_active":"2023-11-08T19:27:03.885501+0000","last_became_peered":"2023-11-08T19:27:03.885501+0000","last_unstale":"2023-11-08T19:27:05.909355+0000","last_undegraded":"2023-11-08T19:27:05.909355+0000","last_fullsized":"2023-11-08T19:27:05.909355+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:28:49.934972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.902449+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:27:04.902449+0000","last_peered":"2023-11-08T19:27:04.902449+0000","last_clean":"2023-11-08T19:27:04.902449+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:27:04.902449+0000","last_undegraded":"2023-11-08T19:27:04.902449+0000","last_fullsized":"2023-11-08T19:27:04.902449+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916741+0000","last_change":"2023-11-08T19:27:04.908438+0000","last_active":"2023-11-08T19:27:05.916741+0000","last_peered":"2023-11-08T19:27:05.916741+0000","last_clean":"2023-11-08T19:27:05.916741+0000","last_became_active":"2023-11-08T19:27:04.908120+0000","last_became_peered":"2023-11-08T19:27:04.908120+0000","last_unstale":"2023-11-08T19:27:05.916741+0000","last_undegraded":"2023-11-08T19:27:05.916741+0000","last_fullsized":"2023-11-08T19:27:05.916741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:57.229596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.898500+0000","last_change":"2023-11-08T19:27:03.908637+0000","last_active":"2023-11-08T19:27:04.898500+0000","last_peered":"2023-11-08T19:27:04.898500+0000","last_clean":"2023-11-08T19:27:04.898500+0000","last_became_active":"2023-11-08T19:27:03.908202+0000","last_became_peered":"2023-11-08T19:27:03.908202+0000","last_unstale":"2023-11-08T19:27:04.898500+0000","last_undegraded":"2023-11-08T19:27:04.898500+0000","last_fullsized":"2023-11-08T19:27:04.898500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:03.036895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.881571+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:27:03.881571+0000","last_peered":"2023-11-08T19:27:03.881571+0000","last_clean":"2023-11-08T19:27:03.881571+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:27:03.881571+0000","last_undegraded":"2023-11-08T19:27:03.881571+0000","last_fullsized":"2023-11-08T19:27:03.881571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916426+0000","last_change":"2023-11-08T19:27:04.920366+0000","last_active":"2023-11-08T19:27:05.916426+0000","last_peered":"2023-11-08T19:27:05.916426+0000","last_clean":"2023-11-08T19:27:05.916426+0000","last_became_active":"2023-11-08T19:27:04.918576+0000","last_became_peered":"2023-11-08T19:27:04.918576+0000","last_unstale":"2023-11-08T19:27:05.916426+0000","last_undegraded":"2023-11-08T19:27:05.916426+0000","last_fullsized":"2023-11-08T19:27:05.916426+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:27.843034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.910457+0000","last_change":"2023-11-08T19:27:03.890631+0000","last_active":"2023-11-08T19:27:05.910457+0000","last_peered":"2023-11-08T19:27:05.910457+0000","last_clean":"2023-11-08T19:27:05.910457+0000","last_became_active":"2023-11-08T19:27:03.890346+0000","last_became_peered":"2023-11-08T19:27:03.890346+0000","last_unstale":"2023-11-08T19:27:05.910457+0000","last_undegraded":"2023-11-08T19:27:05.910457+0000","last_fullsized":"2023-11-08T19:27:05.910457+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:28.843254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.899330+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:27:04.899330+0000","last_peered":"2023-11-08T19:27:04.899330+0000","last_clean":"2023-11-08T19:27:04.899330+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:27:04.899330+0000","last_undegraded":"2023-11-08T19:27:04.899330+0000","last_fullsized":"2023-11-08T19:27:04.899330+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927816+0000","last_change":"2023-11-08T19:27:04.920912+0000","last_active":"2023-11-08T19:27:04.927816+0000","last_peered":"2023-11-08T19:27:04.927816+0000","last_clean":"2023-11-08T19:27:04.927816+0000","last_became_active":"2023-11-08T19:27:04.920579+0000","last_became_peered":"2023-11-08T19:27:04.920579+0000","last_unstale":"2023-11-08T19:27:04.927816+0000","last_undegraded":"2023-11-08T19:27:04.927816+0000","last_fullsized":"2023-11-08T19:27:04.927816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:02.402334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917414+0000","last_change":"2023-11-08T19:27:03.904629+0000","last_active":"2023-11-08T19:27:05.917414+0000","last_peered":"2023-11-08T19:27:05.917414+0000","last_clean":"2023-11-08T19:27:05.917414+0000","last_became_active":"2023-11-08T19:27:03.903936+0000","last_became_peered":"2023-11-08T19:27:03.903936+0000","last_unstale":"2023-11-08T19:27:05.917414+0000","last_undegraded":"2023-11-08T19:27:05.917414+0000","last_fullsized":"2023-11-08T19:27:05.917414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:33.289464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.887391+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:27:04.887391+0000","last_peered":"2023-11-08T19:27:04.887391+0000","last_clean":"2023-11-08T19:27:04.887391+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:27:04.887391+0000","last_undegraded":"2023-11-08T19:27:04.887391+0000","last_fullsized":"2023-11-08T19:27:04.887391+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909634+0000","last_change":"2023-11-08T19:27:04.900445+0000","last_active":"2023-11-08T19:27:05.909634+0000","last_peered":"2023-11-08T19:27:05.909634+0000","last_clean":"2023-11-08T19:27:05.909634+0000","last_became_active":"2023-11-08T19:27:04.899584+0000","last_became_peered":"2023-11-08T19:27:04.899584+0000","last_unstale":"2023-11-08T19:27:05.909634+0000","last_undegraded":"2023-11-08T19:27:05.909634+0000","last_fullsized":"2023-11-08T19:27:05.909634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:04.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914660+0000","last_change":"2023-11-08T19:27:03.912669+0000","last_active":"2023-11-08T19:27:05.914660+0000","last_peered":"2023-11-08T19:27:05.914660+0000","last_clean":"2023-11-08T19:27:05.914660+0000","last_became_active":"2023-11-08T19:27:03.912340+0000","last_became_peered":"2023-11-08T19:27:03.912340+0000","last_unstale":"2023-11-08T19:27:05.914660+0000","last_undegraded":"2023-11-08T19:27:05.914660+0000","last_fullsized":"2023-11-08T19:27:05.914660+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:22.048098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.885693+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:27:03.885693+0000","last_peered":"2023-11-08T19:27:03.885693+0000","last_clean":"2023-11-08T19:27:03.885693+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:27:03.885693+0000","last_undegraded":"2023-11-08T19:27:03.885693+0000","last_fullsized":"2023-11-08T19:27:03.885693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.933255+0000","last_change":"2023-11-08T19:27:04.930297+0000","last_active":"2023-11-08T19:27:04.933255+0000","last_peered":"2023-11-08T19:27:04.933255+0000","last_clean":"2023-11-08T19:27:04.933255+0000","last_became_active":"2023-11-08T19:27:04.929969+0000","last_became_peered":"2023-11-08T19:27:04.929969+0000","last_unstale":"2023-11-08T19:27:04.933255+0000","last_undegraded":"2023-11-08T19:27:04.933255+0000","last_fullsized":"2023-11-08T19:27:04.933255+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:57.936266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909591+0000","last_change":"2023-11-08T19:27:03.902190+0000","last_active":"2023-11-08T19:27:05.909591+0000","last_peered":"2023-11-08T19:27:05.909591+0000","last_clean":"2023-11-08T19:27:05.909591+0000","last_became_active":"2023-11-08T19:27:03.901899+0000","last_became_peered":"2023-11-08T19:27:03.901899+0000","last_unstale":"2023-11-08T19:27:05.909591+0000","last_undegraded":"2023-11-08T19:27:05.909591+0000","last_fullsized":"2023-11-08T19:27:05.909591+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:52:21.438332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.898304+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:27:03.898304+0000","last_peered":"2023-11-08T19:27:03.898304+0000","last_clean":"2023-11-08T19:27:03.898304+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:27:03.898304+0000","last_undegraded":"2023-11-08T19:27:03.898304+0000","last_fullsized":"2023-11-08T19:27:03.898304+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916195+0000","last_change":"2023-11-08T19:27:04.910936+0000","last_active":"2023-11-08T19:27:05.916195+0000","last_peered":"2023-11-08T19:27:05.916195+0000","last_clean":"2023-11-08T19:27:05.916195+0000","last_became_active":"2023-11-08T19:27:04.910659+0000","last_became_peered":"2023-11-08T19:27:04.910659+0000","last_unstale":"2023-11-08T19:27:05.916195+0000","last_undegraded":"2023-11-08T19:27:05.916195+0000","last_fullsized":"2023-11-08T19:27:05.916195+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:41.017102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938580+0000","last_change":"2023-11-08T19:27:03.904556+0000","last_active":"2023-11-08T19:27:04.938580+0000","last_peered":"2023-11-08T19:27:04.938580+0000","last_clean":"2023-11-08T19:27:04.938580+0000","last_became_active":"2023-11-08T19:27:03.904250+0000","last_became_peered":"2023-11-08T19:27:03.904250+0000","last_unstale":"2023-11-08T19:27:04.938580+0000","last_undegraded":"2023-11-08T19:27:04.938580+0000","last_fullsized":"2023-11-08T19:27:04.938580+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:17.602970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.882702+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:27:03.882702+0000","last_peered":"2023-11-08T19:27:03.882702+0000","last_clean":"2023-11-08T19:27:03.882702+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:27:03.882702+0000","last_undegraded":"2023-11-08T19:27:03.882702+0000","last_fullsized":"2023-11-08T19:27:03.882702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927964+0000","last_change":"2023-11-08T19:27:04.914484+0000","last_active":"2023-11-08T19:27:04.927964+0000","last_peered":"2023-11-08T19:27:04.927964+0000","last_clean":"2023-11-08T19:27:04.927964+0000","last_became_active":"2023-11-08T19:27:04.914215+0000","last_became_peered":"2023-11-08T19:27:04.914215+0000","last_unstale":"2023-11-08T19:27:04.927964+0000","last_undegraded":"2023-11-08T19:27:04.927964+0000","last_fullsized":"2023-11-08T19:27:04.927964+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:33.506251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916544+0000","last_change":"2023-11-08T19:27:03.897253+0000","last_active":"2023-11-08T19:27:05.916544+0000","last_peered":"2023-11-08T19:27:05.916544+0000","last_clean":"2023-11-08T19:27:05.916544+0000","last_became_active":"2023-11-08T19:27:03.896651+0000","last_became_peered":"2023-11-08T19:27:03.896651+0000","last_unstale":"2023-11-08T19:27:05.916544+0000","last_undegraded":"2023-11-08T19:27:05.916544+0000","last_fullsized":"2023-11-08T19:27:05.916544+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:37.598146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.899940+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:27:04.899940+0000","last_peered":"2023-11-08T19:27:04.899940+0000","last_clean":"2023-11-08T19:27:04.899940+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:27:04.899940+0000","last_undegraded":"2023-11-08T19:27:04.899940+0000","last_fullsized":"2023-11-08T19:27:04.899940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916721+0000","last_change":"2023-11-08T19:27:04.920650+0000","last_active":"2023-11-08T19:27:05.916721+0000","last_peered":"2023-11-08T19:27:05.916721+0000","last_clean":"2023-11-08T19:27:05.916721+0000","last_became_active":"2023-11-08T19:27:04.920343+0000","last_became_peered":"2023-11-08T19:27:04.920343+0000","last_unstale":"2023-11-08T19:27:05.916721+0000","last_undegraded":"2023-11-08T19:27:05.916721+0000","last_fullsized":"2023-11-08T19:27:05.916721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:00.218479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936171+0000","last_change":"2023-11-08T19:27:03.906001+0000","last_active":"2023-11-08T19:27:04.936171+0000","last_peered":"2023-11-08T19:27:04.936171+0000","last_clean":"2023-11-08T19:27:04.936171+0000","last_became_active":"2023-11-08T19:27:03.904816+0000","last_became_peered":"2023-11-08T19:27:03.904816+0000","last_unstale":"2023-11-08T19:27:04.936171+0000","last_undegraded":"2023-11-08T19:27:04.936171+0000","last_fullsized":"2023-11-08T19:27:04.936171+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:26.357704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":27,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.895444+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:27:03.895444+0000","last_peered":"2023-11-08T19:27:03.895444+0000","last_clean":"2023-11-08T19:27:03.895444+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:27:03.895444+0000","last_undegraded":"2023-11-08T19:27:03.895444+0000","last_fullsized":"2023-11-08T19:27:03.895444+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.921866+0000","last_change":"2023-11-08T19:27:04.916409+0000","last_active":"2023-11-08T19:27:04.921866+0000","last_peered":"2023-11-08T19:27:04.921866+0000","last_clean":"2023-11-08T19:27:04.921866+0000","last_became_active":"2023-11-08T19:27:04.916107+0000","last_became_peered":"2023-11-08T19:27:04.916107+0000","last_unstale":"2023-11-08T19:27:04.921866+0000","last_undegraded":"2023-11-08T19:27:04.921866+0000","last_fullsized":"2023-11-08T19:27:04.921866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:38.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938481+0000","last_change":"2023-11-08T19:27:03.908821+0000","last_active":"2023-11-08T19:27:04.938481+0000","last_peered":"2023-11-08T19:27:04.938481+0000","last_clean":"2023-11-08T19:27:04.938481+0000","last_became_active":"2023-11-08T19:27:03.907215+0000","last_became_peered":"2023-11-08T19:27:03.907215+0000","last_unstale":"2023-11-08T19:27:04.938481+0000","last_undegraded":"2023-11-08T19:27:04.938481+0000","last_fullsized":"2023-11-08T19:27:04.938481+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:04.741435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.891854+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:27:04.891854+0000","last_peered":"2023-11-08T19:27:04.891854+0000","last_clean":"2023-11-08T19:27:04.891854+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:27:04.891854+0000","last_undegraded":"2023-11-08T19:27:04.891854+0000","last_fullsized":"2023-11-08T19:27:04.891854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.926862+0000","last_change":"2023-11-08T19:27:04.923108+0000","last_active":"2023-11-08T19:27:04.926862+0000","last_peered":"2023-11-08T19:27:04.926862+0000","last_clean":"2023-11-08T19:27:04.926862+0000","last_became_active":"2023-11-08T19:27:04.922936+0000","last_became_peered":"2023-11-08T19:27:04.922936+0000","last_unstale":"2023-11-08T19:27:04.926862+0000","last_undegraded":"2023-11-08T19:27:04.926862+0000","last_fullsized":"2023-11-08T19:27:04.926862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:33.506251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.905793+0000","last_change":"2023-11-08T19:27:03.896581+0000","last_active":"2023-11-08T19:27:04.905793+0000","last_peered":"2023-11-08T19:27:04.905793+0000","last_clean":"2023-11-08T19:27:04.905793+0000","last_became_active":"2023-11-08T19:27:03.896291+0000","last_became_peered":"2023-11-08T19:27:03.896291+0000","last_unstale":"2023-11-08T19:27:04.905793+0000","last_undegraded":"2023-11-08T19:27:04.905793+0000","last_fullsized":"2023-11-08T19:27:04.905793+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:10:09.139141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.887232+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:27:03.887232+0000","last_peered":"2023-11-08T19:27:03.887232+0000","last_clean":"2023-11-08T19:27:03.887232+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:27:03.887232+0000","last_undegraded":"2023-11-08T19:27:03.887232+0000","last_fullsized":"2023-11-08T19:27:03.887232+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935054+0000","last_change":"2023-11-08T19:27:04.923610+0000","last_active":"2023-11-08T19:27:04.935054+0000","last_peered":"2023-11-08T19:27:04.935054+0000","last_clean":"2023-11-08T19:27:04.935054+0000","last_became_active":"2023-11-08T19:27:04.923023+0000","last_became_peered":"2023-11-08T19:27:04.923023+0000","last_unstale":"2023-11-08T19:27:04.935054+0000","last_undegraded":"2023-11-08T19:27:04.935054+0000","last_fullsized":"2023-11-08T19:27:04.935054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:33.083930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916800+0000","last_change":"2023-11-08T19:27:03.908831+0000","last_active":"2023-11-08T19:27:05.916800+0000","last_peered":"2023-11-08T19:27:05.916800+0000","last_clean":"2023-11-08T19:27:05.916800+0000","last_became_active":"2023-11-08T19:27:03.908545+0000","last_became_peered":"2023-11-08T19:27:03.908545+0000","last_unstale":"2023-11-08T19:27:05.916800+0000","last_undegraded":"2023-11-08T19:27:05.916800+0000","last_fullsized":"2023-11-08T19:27:05.916800+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:01.389080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.885750+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:27:03.885750+0000","last_peered":"2023-11-08T19:27:03.885750+0000","last_clean":"2023-11-08T19:27:03.885750+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:27:03.885750+0000","last_undegraded":"2023-11-08T19:27:03.885750+0000","last_fullsized":"2023-11-08T19:27:03.885750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927591+0000","last_change":"2023-11-08T19:27:04.918967+0000","last_active":"2023-11-08T19:27:04.927591+0000","last_peered":"2023-11-08T19:27:04.927591+0000","last_clean":"2023-11-08T19:27:04.927591+0000","last_became_active":"2023-11-08T19:27:04.918628+0000","last_became_peered":"2023-11-08T19:27:04.918628+0000","last_unstale":"2023-11-08T19:27:04.927591+0000","last_undegraded":"2023-11-08T19:27:04.927591+0000","last_fullsized":"2023-11-08T19:27:04.927591+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:09:05.549788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914399+0000","last_change":"2023-11-08T19:27:03.911427+0000","last_active":"2023-11-08T19:27:05.914399+0000","last_peered":"2023-11-08T19:27:05.914399+0000","last_clean":"2023-11-08T19:27:05.914399+0000","last_became_active":"2023-11-08T19:27:03.911152+0000","last_became_peered":"2023-11-08T19:27:03.911152+0000","last_unstale":"2023-11-08T19:27:05.914399+0000","last_undegraded":"2023-11-08T19:27:05.914399+0000","last_fullsized":"2023-11-08T19:27:05.914399+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:20.468848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.891475+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:27:03.891475+0000","last_peered":"2023-11-08T19:27:03.891475+0000","last_clean":"2023-11-08T19:27:03.891475+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:27:03.891475+0000","last_undegraded":"2023-11-08T19:27:03.891475+0000","last_fullsized":"2023-11-08T19:27:03.891475+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.889644+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:27:03.889644+0000","last_peered":"2023-11-08T19:27:03.889644+0000","last_clean":"2023-11-08T19:27:03.889644+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:27:03.889644+0000","last_undegraded":"2023-11-08T19:27:03.889644+0000","last_fullsized":"2023-11-08T19:27:03.889644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":877151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.087,"15min":1.1040000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4930000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.29,"15min":1.325},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4450000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.625}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.232,"15min":1.2689999999999999},"min":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6619999999999999,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.1060000000000001,"15min":1.1299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3160000000000001,"5min":1.681,"15min":1.681},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.466,"15min":1.5169999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.5089999999999999,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.349,"15min":1.385},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.772,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.76800000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.341,"15min":1.377},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.419,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.272,"5min":1.536,"15min":1.5800000000000001},"min":{"1min":0.95299999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6439999999999999,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":0.83799999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3560000000000001,"15min":1.3700000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.704,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.4690000000000001,"15min":1.5109999999999999},"min":{"1min":0.82799999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.5780000000000001,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4470000000000001,"15min":1.4650000000000001},"min":{"1min":1.004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.766,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.446,"5min":1.4850000000000001,"15min":1.4910000000000001},"min":{"1min":1.1599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":1.9590000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.81699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5700000000000001,"15min":1.587},"min":{"1min":1.036,"5min":1.036,"15min":1.036},"max":{"1min":1.8680000000000001,"5min":2.427,"15min":2.427},"last":0.878},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.6060000000000001,"15min":1.6439999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.746,"5min":2.468,"15min":2.468},"last":0.997}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":293596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":0.95399999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.83399999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":1.089},"min":{"1min":0.747,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.534,"5min":1.679,"15min":1.679},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.331,"15min":1.3540000000000001},"min":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.591,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.274,"15min":1.304},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4990000000000001,"5min":4.609,"15min":4.609},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2210000000000001,"15min":1.214},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.5609999999999999,"5min":1.861,"15min":1.861},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.877,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.179,"15min":1.1799999999999999},"min":{"1min":0.80300000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.663,"5min":1.784,"15min":1.784},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.383,"15min":1.381},"min":{"1min":0.89900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.024,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":1.04,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.89500000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.98199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7749999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.347,"5min":1.3320000000000001,"15min":1.3300000000000001},"min":{"1min":0.90200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.831,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5509999999999999,"15min":1.5449999999999999},"min":{"1min":1.097,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1240000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.96599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.724,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.81499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":294248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0900000000000001,"15min":1.073},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.712,"5min":1.76,"15min":1.76},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.246,"15min":1.2390000000000001},"min":{"1min":0.879,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.645,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.165}]},{"osd":1,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.167,"15min":1.179},"min":{"1min":0.82399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.401,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.405,"15min":1.462},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4970000000000001,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.345}]},{"osd":2,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1799999999999999,"15min":1.171},"min":{"1min":0.91500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8700000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.623},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.4650000000000001,"15min":1.5229999999999999},"min":{"1min":0.76100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.5900000000000001,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":0.96999999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8200000000000001,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.3149999999999999,"15min":1.3069999999999999},"min":{"1min":0.89900000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7609999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.502}]},{"osd":4,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.288,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.966,"5min":2.347,"15min":2.347},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.544,"15min":1.5629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0230000000000001,"5min":3.153,"15min":3.153},"last":1.4610000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.516,"15min":1.536},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9199999999999999,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.569},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.494,"15min":1.5069999999999999},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.869,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.407}]},{"osd":7,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.6830000000000001,"15min":1.7150000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1269999999999998,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.288},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.587,"15min":1.5920000000000001},"min":{"1min":0.99399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.994,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.673}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":288235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.036,"15min":1.0409999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.248,"15min":1.2070000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":7.0119999999999996,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.89400000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.103,"15min":1.1020000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.121,"15min":1.163},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2470000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.145}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3520000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.8719999999999999,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.431},{"interface":"front","average":{"1min":1.177,"5min":1.3959999999999999,"15min":1.4319999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.375,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.5589999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3080000000000001,"15min":1.302},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.91900000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0470000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.232}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.486,"15min":1.528},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.4630000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.298},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5469999999999999,"15min":1.5660000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.3799999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5569999999999999,"15min":1.573},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.2360000000000002,"5min":4.335,"15min":4.335},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.613,"15min":1.6339999999999999},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.47}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.637,"15min":1.6419999999999999},"min":{"1min":0.95499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.5490000000000004,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.7170000000000001,"15min":1.7430000000000001},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":3.1389999999999998,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.51}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":286679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.056,"15min":1.0960000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2909999999999999,"5min":3.972,"15min":3.972},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3999999999999999,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.1679999999999999,"15min":1.2070000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6839999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":1.0900000000000001,"15min":1.121},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3360000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.871}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.3089999999999999,"15min":1.3380000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.752,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.101},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.256,"15min":1.282},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.778,"5min":2.73,"15min":2.73},"last":0.79000000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.4350000000000001,"15min":1.4690000000000001},"min":{"1min":0.872,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.861,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.448,"15min":1.47},"min":{"1min":0.91400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5980000000000001,"5min":3.052,"15min":3.052},"last":1.028}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.4750000000000001,"15min":1.5109999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.149,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.331,"5min":1.5169999999999999,"15min":1.548},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.375,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":0.99199999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":1.054,"5min":0.878,"15min":0.878},"max":{"1min":1.8220000000000001,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.5740000000000001,"15min":1.6000000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.4860000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.6930000000000001,"15min":1.736},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.528,"5min":4.407,"15min":4.407},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.573,"15min":1.5960000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8740000000000001,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.3149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":285660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.222,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.90200000000000002,"15min":0.93400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.206,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.202,"5min":1.393,"15min":1.393},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.02},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.177,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.51300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.486,"5min":1.53,"15min":1.53},"last":1.71},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.099,"15min":1.139},"min":{"1min":0.48099999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3859999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.1699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.3600000000000001,"15min":1.421},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3100000000000001,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.607},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3360000000000001,"15min":1.3779999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.506,"5min":2.278,"15min":2.278},"last":1.329}]},{"osd":5,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.389,"15min":1.4350000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6180000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.77},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.456,"5min":2.536,"15min":2.536},"last":1.954}]},{"osd":6,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.363,"15min":1.395},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5409999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.4219999999999999,"15min":1.468},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8480000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.655}]},{"osd":7,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.5389999999999999,"15min":1.593},"min":{"1min":0.81399999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5329999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.861},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.5289999999999999,"15min":1.5680000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.5840000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.8140000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":293686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.753,"15min":0.754},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.0369999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.86499999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1339999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.88300000000000001,"15min":0.88600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.224,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.93700000000000006}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0580000000000001,"15min":1.0569999999999999},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.4970000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.499,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8540000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.669}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.542,"5min":2.165,"15min":2.165},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2410000000000001,"15min":1.2569999999999999},"min":{"1min":0.73799999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4910000000000001,"5min":1.885,"15min":1.885},"last":1.242}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.254,"15min":1.264},"min":{"1min":0.71099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7390000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.45},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.706,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.359,"15min":1.367},"min":{"1min":0.754,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.772,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.373,"15min":1.3859999999999999},"min":{"1min":0.92000000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.6930000000000001,"5min":2.282,"15min":2.282},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.329},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6559999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3740000000000001,"15min":1.3919999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.956,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":878166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0649999999999999,"5min":1.357,"15min":1.357},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2889999999999999,"5min":1.7,"15min":1.7},"last":0.97199999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2410000000000001,"5min":1.508,"15min":1.508},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.91400000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1,"15min":1.012},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.278,"5min":1.748,"15min":1.748},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0880000000000001,"15min":1.115},"min":{"1min":0.53500000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":2.496,"15min":2.496},"last":1.202}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2290000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5169999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.363},{"interface":"front","average":{"1min":1.169,"5min":1.4159999999999999,"15min":1.4570000000000001},"min":{"1min":0.81699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8700000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.169}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.351,"15min":1.3819999999999999},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9510000000000001,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.427,"15min":1.4570000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7070000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.516}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.516},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.8160000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.3999999999999999,"15min":1.4199999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6259999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.1220000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3899999999999999,"15min":1.4059999999999999},"min":{"1min":0.92600000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.746,"5min":2.173,"15min":2.173},"last":1.232},{"interface":"front","average":{"1min":1.24,"5min":1.4590000000000001,"15min":1.4950000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.0070000000000001,"5min":2.306,"15min":2.306},"last":1.4159999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:09.341 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:09.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.591+0000 7f851a0fc640 1 Processor -- start 2023-11-08T19:27:09.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.591+0000 7f851a0fc640 1 -- start start 2023-11-08T19:27:09.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f851a0fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85140c9720 0x7f85140c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f851a0fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f85141610c0 0x7f85141634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f851a0fc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85140c9360 con 0x7f8514159d90 2023-11-08T19:27:09.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f851a0fc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f851406d3c0 con 0x7f85141610c0 2023-11-08T19:27:09.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f851a0fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85140ca490 con 0x7f85140c9720 2023-11-08T19:27:09.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f85137fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8514159d90 0x7f851415a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46884/0 (socket says 172.21.15.119:46884) 2023-11-08T19:27:09.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f85137fe640 1 -- 172.21.15.119:0/3990474201 learned_addr learned my addr 172.21.15.119:0/3990474201 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:09.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f8513fff640 1 --2- 172.21.15.119:0/3990474201 >> v2:172.21.15.119:3300/0 conn(0x7f85141610c0 0x7f85141634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.593+0000 7f850bfff640 1 --2- 172.21.15.119:0/3990474201 >> v2:172.21.15.188:3300/0 conn(0x7f85140c9720 0x7f85140c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.594+0000 7f8512ffd640 1 -- 172.21.15.119:0/3990474201 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335298424 0 0) 0x7f85140c9360 con 0x7f8514159d90 2023-11-08T19:27:09.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.594+0000 7f8512ffd640 1 -- 172.21.15.119:0/3990474201 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8500003520 con 0x7f8514159d90 2023-11-08T19:27:09.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.594+0000 7f8513fff640 1 -- 172.21.15.119:0/3990474201 >> v2:172.21.15.188:3300/0 conn(0x7f85140c9720 msgr2=0x7f85140c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:09.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.594+0000 7f8513fff640 1 --2- 172.21.15.119:0/3990474201 >> v2:172.21.15.188:3300/0 conn(0x7f85140c9720 0x7f85140c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.594+0000 7f8513fff640 1 -- 172.21.15.119:0/3990474201 >> v1:172.21.15.119:6789/0 conn(0x7f8514159d90 legacy=0x7f851415a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.594+0000 7f8513fff640 1 -- 172.21.15.119:0/3990474201 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85140ca770 con 0x7f85141610c0 2023-11-08T19:27:09.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.595+0000 7f8513fff640 1 --2- 172.21.15.119:0/3990474201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f85141634b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f85140b18a0 tx=0x7f850c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abaf7bb9ed0caf10 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.595+0000 7f8512ffd640 1 -- 172.21.15.119:0/3990474201 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c02cc70 con 0x7f85141610c0 2023-11-08T19:27:09.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.595+0000 7f8512ffd640 1 -- 172.21.15.119:0/3990474201 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c02cdd0 con 0x7f85141610c0 2023-11-08T19:27:09.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.595+0000 7f8512ffd640 1 -- 172.21.15.119:0/3990474201 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c034660 con 0x7f85141610c0 2023-11-08T19:27:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 -- 172.21.15.119:0/3990474201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 msgr2=0x7f85141634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 --2- 172.21.15.119:0/3990474201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f85141634b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f85140b18a0 tx=0x7f850c02bec0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 -- 172.21.15.119:0/3990474201 shutdown_connections 2023-11-08T19:27:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 --2- 172.21.15.119:0/3990474201 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f85141634b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 --2- 172.21.15.119:0/3990474201 >> v2:172.21.15.188:3300/0 conn(0x7f85140c9720 0x7f85140c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 -- 172.21.15.119:0/3990474201 >> 172.21.15.119:0/3990474201 conn(0x7f85140cb610 msgr2=0x7f85140cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 -- 172.21.15.119:0/3990474201 shutdown_connections 2023-11-08T19:27:09.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.596+0000 7f851a0fc640 1 -- 172.21.15.119:0/3990474201 wait complete. 2023-11-08T19:27:09.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.598+0000 7f851a0fc640 1 Processor -- start 2023-11-08T19:27:09.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.598+0000 7f851a0fc640 1 -- start start 2023-11-08T19:27:09.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f851a0fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85140c9720 0x7f8514156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f851a0fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8514159d90 0x7f851414dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f851a0fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f851414cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f851a0fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514144ae0 con 0x7f85141610c0 2023-11-08T19:27:09.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f851a0fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c3a50 con 0x7f8514159d90 2023-11-08T19:27:09.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f851a0fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85140c42f0 con 0x7f85140c9720 2023-11-08T19:27:09.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f851414cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f851414cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49884/0 (socket says 172.21.15.119:49884) 2023-11-08T19:27:09.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 -- 172.21.15.119:0/2867216012 learned_addr learned my addr 172.21.15.119:0/2867216012 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:09.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f850bfff640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8514159d90 0x7f851414dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f85137fe640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85140c9720 0x7f8514156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 -- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85140c9720 msgr2=0x7f8514156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85140c9720 0x7f8514156e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 -- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8514159d90 msgr2=0x7f851414dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.599+0000 7f8513fff640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8514159d90 0x7f851414dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f8513fff640 1 -- 172.21.15.119:0/2867216012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c009a70 con 0x7f85141610c0 2023-11-08T19:27:09.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f850bfff640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8514159d90 0x7f851414dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:09.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f85137fe640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85140c9720 0x7f8514156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:09.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f8513fff640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f851414cb60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f850c037040 tx=0x7f850c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c040650 con 0x7f85141610c0 2023-11-08T19:27:09.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850c0407b0 con 0x7f85141610c0 2023-11-08T19:27:09.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8514158e70 con 0x7f85141610c0 2023-11-08T19:27:09.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850c040b60 con 0x7f85141610c0 2023-11-08T19:27:09.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.600+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85140c3f80 con 0x7f85141610c0 2023-11-08T19:27:09.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.602+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8514074430 con 0x7f85141610c0 2023-11-08T19:27:09.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.602+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f850c040cc0 con 0x7f85141610c0 2023-11-08T19:27:09.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.603+0000 7f8510ff9640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f84f406ae70 0x7f84f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:09.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.603+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f850c0ac500 con 0x7f85141610c0 2023-11-08T19:27:09.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.604+0000 7f85137fe640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f84f406ae70 0x7f84f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:09.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.604+0000 7f85137fe640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f84f406ae70 0x7f84f406d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f85040011e0 tx=0x7f8504009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:09.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.607+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f850c077300 con 0x7f85141610c0 2023-11-08T19:27:09.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.792+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f851406be90 con 0x7f84f406ae70 2023-11-08T19:27:09.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.809+0000 7f8510ff9640 1 -- 172.21.15.119:0/2867216012 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368090 (secure 0 0 0) 0x7f85040013f0 con 0x7f84f406ae70 2023-11-08T19:27:09.815 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:09.818 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:27:09.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.815+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f84f406ae70 msgr2=0x7f84f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.815+0000 7f851a0fc640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f84f406ae70 0x7f84f406d330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f85040011e0 tx=0x7f8504009040 comp rx=0 tx=0).stop 2023-11-08T19:27:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.815+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 msgr2=0x7f851414cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.815+0000 7f851a0fc640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f851414cb60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f850c037040 tx=0x7f850c004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 shutdown_connections 2023-11-08T19:27:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f84f406ae70 0x7f84f406d330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f85141610c0 0x7f851414cb60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8514159d90 0x7f851414dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 --2- 172.21.15.119:0/2867216012 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85140c9720 0x7f8514156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 >> 172.21.15.119:0/2867216012 conn(0x7f85140cb610 msgr2=0x7f85140c7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 shutdown_connections 2023-11-08T19:27:09.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:09.816+0000 7f851a0fc640 1 -- 172.21.15.119:0/2867216012 wait complete. 2023-11-08T19:27:09.836 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2023-11-08T19:27:08.703549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230688,"kb_used_data":6180,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744032,"statfs":{"total":773094113280,"available":772857888768,"internally_reserved":0,"allocated":6328320,"data_stored":3497421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.840035"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.341382+0000","last_change":"2023-11-08T19:27:04.927066+0000","last_active":"2023-11-08T19:27:06.341382+0000","last_peered":"2023-11-08T19:27:06.341382+0000","last_clean":"2023-11-08T19:27:06.341382+0000","last_became_active":"2023-11-08T19:27:04.926172+0000","last_became_peered":"2023-11-08T19:27:04.926172+0000","last_unstale":"2023-11-08T19:27:06.341382+0000","last_undegraded":"2023-11-08T19:27:06.341382+0000","last_fullsized":"2023-11-08T19:27:06.341382+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:33.316117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935359+0000","last_change":"2023-11-08T19:27:04.927178+0000","last_active":"2023-11-08T19:27:04.935359+0000","last_peered":"2023-11-08T19:27:04.935359+0000","last_clean":"2023-11-08T19:27:04.935359+0000","last_became_active":"2023-11-08T19:27:04.926914+0000","last_became_peered":"2023-11-08T19:27:04.926914+0000","last_unstale":"2023-11-08T19:27:04.935359+0000","last_undegraded":"2023-11-08T19:27:04.935359+0000","last_fullsized":"2023-11-08T19:27:04.935359+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:42.243726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922440+0000","last_change":"2023-11-08T19:27:04.919694+0000","last_active":"2023-11-08T19:27:04.922440+0000","last_peered":"2023-11-08T19:27:04.922440+0000","last_clean":"2023-11-08T19:27:04.922440+0000","last_became_active":"2023-11-08T19:27:04.919400+0000","last_became_peered":"2023-11-08T19:27:04.919400+0000","last_unstale":"2023-11-08T19:27:04.922440+0000","last_undegraded":"2023-11-08T19:27:04.922440+0000","last_fullsized":"2023-11-08T19:27:04.922440+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:43.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917471+0000","last_change":"2023-11-08T19:27:04.919426+0000","last_active":"2023-11-08T19:27:05.917471+0000","last_peered":"2023-11-08T19:27:05.917471+0000","last_clean":"2023-11-08T19:27:05.917471+0000","last_became_active":"2023-11-08T19:27:04.919120+0000","last_became_peered":"2023-11-08T19:27:04.919120+0000","last_unstale":"2023-11-08T19:27:05.917471+0000","last_undegraded":"2023-11-08T19:27:05.917471+0000","last_fullsized":"2023-11-08T19:27:05.917471+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:47.306710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340675+0000","last_change":"2023-11-08T19:27:04.918284+0000","last_active":"2023-11-08T19:27:06.340675+0000","last_peered":"2023-11-08T19:27:06.340675+0000","last_clean":"2023-11-08T19:27:06.340675+0000","last_became_active":"2023-11-08T19:27:04.918036+0000","last_became_peered":"2023-11-08T19:27:04.918036+0000","last_unstale":"2023-11-08T19:27:06.340675+0000","last_undegraded":"2023-11-08T19:27:06.340675+0000","last_fullsized":"2023-11-08T19:27:06.340675+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:23:09.377486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.933114+0000","last_change":"2023-11-08T19:27:04.929182+0000","last_active":"2023-11-08T19:27:04.933114+0000","last_peered":"2023-11-08T19:27:04.933114+0000","last_clean":"2023-11-08T19:27:04.933114+0000","last_became_active":"2023-11-08T19:27:04.928895+0000","last_became_peered":"2023-11-08T19:27:04.928895+0000","last_unstale":"2023-11-08T19:27:04.933114+0000","last_undegraded":"2023-11-08T19:27:04.933114+0000","last_fullsized":"2023-11-08T19:27:04.933114+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:38.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.911608+0000","last_change":"2023-11-08T19:27:04.906410+0000","last_active":"2023-11-08T19:27:04.911608+0000","last_peered":"2023-11-08T19:27:04.911608+0000","last_clean":"2023-11-08T19:27:04.911608+0000","last_became_active":"2023-11-08T19:27:04.906107+0000","last_became_peered":"2023-11-08T19:27:04.906107+0000","last_unstale":"2023-11-08T19:27:04.911608+0000","last_undegraded":"2023-11-08T19:27:04.911608+0000","last_fullsized":"2023-11-08T19:27:04.911608+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:14.313670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922481+0000","last_change":"2023-11-08T19:27:04.907534+0000","last_active":"2023-11-08T19:27:04.922481+0000","last_peered":"2023-11-08T19:27:04.922481+0000","last_clean":"2023-11-08T19:27:04.922481+0000","last_became_active":"2023-11-08T19:27:04.907274+0000","last_became_peered":"2023-11-08T19:27:04.907274+0000","last_unstale":"2023-11-08T19:27:04.922481+0000","last_undegraded":"2023-11-08T19:27:04.922481+0000","last_fullsized":"2023-11-08T19:27:04.922481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:27.403294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.921980+0000","last_change":"2023-11-08T19:27:04.916608+0000","last_active":"2023-11-08T19:27:04.921980+0000","last_peered":"2023-11-08T19:27:04.921980+0000","last_clean":"2023-11-08T19:27:04.921980+0000","last_became_active":"2023-11-08T19:27:04.916141+0000","last_became_peered":"2023-11-08T19:27:04.916141+0000","last_unstale":"2023-11-08T19:27:04.921980+0000","last_undegraded":"2023-11-08T19:27:04.921980+0000","last_fullsized":"2023-11-08T19:27:04.921980+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:27.487518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.910214+0000","last_change":"2023-11-08T19:27:04.909096+0000","last_active":"2023-11-08T19:27:05.910214+0000","last_peered":"2023-11-08T19:27:05.910214+0000","last_clean":"2023-11-08T19:27:05.910214+0000","last_became_active":"2023-11-08T19:27:04.908258+0000","last_became_peered":"2023-11-08T19:27:04.908258+0000","last_unstale":"2023-11-08T19:27:05.910214+0000","last_undegraded":"2023-11-08T19:27:05.910214+0000","last_fullsized":"2023-11-08T19:27:05.910214+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:33.506251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909973+0000","last_change":"2023-11-08T19:27:04.912053+0000","last_active":"2023-11-08T19:27:05.909973+0000","last_peered":"2023-11-08T19:27:05.909973+0000","last_clean":"2023-11-08T19:27:05.909973+0000","last_became_active":"2023-11-08T19:27:04.911775+0000","last_became_peered":"2023-11-08T19:27:04.911775+0000","last_unstale":"2023-11-08T19:27:05.909973+0000","last_undegraded":"2023-11-08T19:27:05.909973+0000","last_fullsized":"2023-11-08T19:27:05.909973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:59:01.049984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917195+0000","last_change":"2023-11-08T19:27:04.909143+0000","last_active":"2023-11-08T19:27:05.917195+0000","last_peered":"2023-11-08T19:27:05.917195+0000","last_clean":"2023-11-08T19:27:05.917195+0000","last_became_active":"2023-11-08T19:27:04.907886+0000","last_became_peered":"2023-11-08T19:27:04.907886+0000","last_unstale":"2023-11-08T19:27:05.917195+0000","last_undegraded":"2023-11-08T19:27:05.917195+0000","last_fullsized":"2023-11-08T19:27:05.917195+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:37.168547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914165+0000","last_change":"2023-11-08T19:27:04.928932+0000","last_active":"2023-11-08T19:27:05.914165+0000","last_peered":"2023-11-08T19:27:05.914165+0000","last_clean":"2023-11-08T19:27:05.914165+0000","last_became_active":"2023-11-08T19:27:04.928660+0000","last_became_peered":"2023-11-08T19:27:04.928660+0000","last_unstale":"2023-11-08T19:27:05.914165+0000","last_undegraded":"2023-11-08T19:27:05.914165+0000","last_fullsized":"2023-11-08T19:27:05.914165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:24.476633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.925182+0000","last_change":"2023-11-08T19:27:04.920158+0000","last_active":"2023-11-08T19:27:04.925182+0000","last_peered":"2023-11-08T19:27:04.925182+0000","last_clean":"2023-11-08T19:27:04.925182+0000","last_became_active":"2023-11-08T19:27:04.919862+0000","last_became_peered":"2023-11-08T19:27:04.919862+0000","last_unstale":"2023-11-08T19:27:04.925182+0000","last_undegraded":"2023-11-08T19:27:04.925182+0000","last_fullsized":"2023-11-08T19:27:04.925182+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:04.629985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.923965+0000","last_change":"2023-11-08T19:27:04.918153+0000","last_active":"2023-11-08T19:27:04.923965+0000","last_peered":"2023-11-08T19:27:04.923965+0000","last_clean":"2023-11-08T19:27:04.923965+0000","last_became_active":"2023-11-08T19:27:04.917885+0000","last_became_peered":"2023-11-08T19:27:04.917885+0000","last_unstale":"2023-11-08T19:27:04.923965+0000","last_undegraded":"2023-11-08T19:27:04.923965+0000","last_fullsized":"2023-11-08T19:27:04.923965+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:52:22.451586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.934339+0000","last_change":"2023-11-08T19:27:04.931573+0000","last_active":"2023-11-08T19:27:04.934339+0000","last_peered":"2023-11-08T19:27:04.934339+0000","last_clean":"2023-11-08T19:27:04.934339+0000","last_became_active":"2023-11-08T19:27:04.931100+0000","last_became_peered":"2023-11-08T19:27:04.931100+0000","last_unstale":"2023-11-08T19:27:04.934339+0000","last_undegraded":"2023-11-08T19:27:04.934339+0000","last_fullsized":"2023-11-08T19:27:04.934339+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:05.247418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922595+0000","last_change":"2023-11-08T19:27:04.913083+0000","last_active":"2023-11-08T19:27:04.922595+0000","last_peered":"2023-11-08T19:27:04.922595+0000","last_clean":"2023-11-08T19:27:04.922595+0000","last_became_active":"2023-11-08T19:27:04.912789+0000","last_became_peered":"2023-11-08T19:27:04.912789+0000","last_unstale":"2023-11-08T19:27:04.922595+0000","last_undegraded":"2023-11-08T19:27:04.922595+0000","last_fullsized":"2023-11-08T19:27:04.922595+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:00.218479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.922264+0000","last_change":"2023-11-08T19:27:04.918068+0000","last_active":"2023-11-08T19:27:04.922264+0000","last_peered":"2023-11-08T19:27:04.922264+0000","last_clean":"2023-11-08T19:27:04.922264+0000","last_became_active":"2023-11-08T19:27:04.917742+0000","last_became_peered":"2023-11-08T19:27:04.917742+0000","last_unstale":"2023-11-08T19:27:04.922264+0000","last_undegraded":"2023-11-08T19:27:04.922264+0000","last_fullsized":"2023-11-08T19:27:04.922264+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:27.487518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340821+0000","last_change":"2023-11-08T19:27:04.918112+0000","last_active":"2023-11-08T19:27:06.340821+0000","last_peered":"2023-11-08T19:27:06.340821+0000","last_clean":"2023-11-08T19:27:06.340821+0000","last_became_active":"2023-11-08T19:27:04.917823+0000","last_became_peered":"2023-11-08T19:27:04.917823+0000","last_unstale":"2023-11-08T19:27:06.340821+0000","last_undegraded":"2023-11-08T19:27:06.340821+0000","last_fullsized":"2023-11-08T19:27:06.340821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:37.539021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935074+0000","last_change":"2023-11-08T19:27:04.926456+0000","last_active":"2023-11-08T19:27:04.935074+0000","last_peered":"2023-11-08T19:27:04.935074+0000","last_clean":"2023-11-08T19:27:04.935074+0000","last_became_active":"2023-11-08T19:27:04.923368+0000","last_became_peered":"2023-11-08T19:27:04.923368+0000","last_unstale":"2023-11-08T19:27:04.935074+0000","last_undegraded":"2023-11-08T19:27:04.935074+0000","last_fullsized":"2023-11-08T19:27:04.935074+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:15:41.490954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.916228+0000","last_change":"2023-11-08T19:27:04.911552+0000","last_active":"2023-11-08T19:27:04.916228+0000","last_peered":"2023-11-08T19:27:04.916228+0000","last_clean":"2023-11-08T19:27:04.916228+0000","last_became_active":"2023-11-08T19:27:04.911199+0000","last_became_peered":"2023-11-08T19:27:04.911199+0000","last_unstale":"2023-11-08T19:27:04.916228+0000","last_undegraded":"2023-11-08T19:27:04.916228+0000","last_fullsized":"2023-11-08T19:27:04.916228+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:47.590564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.931670+0000","last_change":"2023-11-08T19:27:04.922120+0000","last_active":"2023-11-08T19:27:04.931670+0000","last_peered":"2023-11-08T19:27:04.931670+0000","last_clean":"2023-11-08T19:27:04.931670+0000","last_became_active":"2023-11-08T19:27:04.921852+0000","last_became_peered":"2023-11-08T19:27:04.921852+0000","last_unstale":"2023-11-08T19:27:04.931670+0000","last_undegraded":"2023-11-08T19:27:04.931670+0000","last_fullsized":"2023-11-08T19:27:04.931670+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:57.229596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.928431+0000","last_change":"2023-11-08T19:27:04.925637+0000","last_active":"2023-11-08T19:27:04.928431+0000","last_peered":"2023-11-08T19:27:04.928431+0000","last_clean":"2023-11-08T19:27:04.928431+0000","last_became_active":"2023-11-08T19:27:04.925319+0000","last_became_peered":"2023-11-08T19:27:04.925319+0000","last_unstale":"2023-11-08T19:27:04.928431+0000","last_undegraded":"2023-11-08T19:27:04.928431+0000","last_fullsized":"2023-11-08T19:27:04.928431+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:12:41.183786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.913022+0000","last_change":"2023-11-08T19:27:04.920182+0000","last_active":"2023-11-08T19:27:05.913022+0000","last_peered":"2023-11-08T19:27:05.913022+0000","last_clean":"2023-11-08T19:27:05.913022+0000","last_became_active":"2023-11-08T19:27:04.917978+0000","last_became_peered":"2023-11-08T19:27:04.917978+0000","last_unstale":"2023-11-08T19:27:05.913022+0000","last_undegraded":"2023-11-08T19:27:05.913022+0000","last_fullsized":"2023-11-08T19:27:05.913022+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:13.185529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.341401+0000","last_change":"2023-11-08T19:27:04.919872+0000","last_active":"2023-11-08T19:27:06.341401+0000","last_peered":"2023-11-08T19:27:06.341401+0000","last_clean":"2023-11-08T19:27:06.341401+0000","last_became_active":"2023-11-08T19:27:04.919566+0000","last_became_peered":"2023-11-08T19:27:04.919566+0000","last_unstale":"2023-11-08T19:27:06.341401+0000","last_undegraded":"2023-11-08T19:27:06.341401+0000","last_fullsized":"2023-11-08T19:27:06.341401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:13:56.548842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916386+0000","last_change":"2023-11-08T19:27:04.904565+0000","last_active":"2023-11-08T19:27:05.916386+0000","last_peered":"2023-11-08T19:27:05.916386+0000","last_clean":"2023-11-08T19:27:05.916386+0000","last_became_active":"2023-11-08T19:27:04.904237+0000","last_became_peered":"2023-11-08T19:27:04.904237+0000","last_unstale":"2023-11-08T19:27:05.916386+0000","last_undegraded":"2023-11-08T19:27:05.916386+0000","last_fullsized":"2023-11-08T19:27:05.916386+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:27.403294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.934040+0000","last_change":"2023-11-08T19:27:04.929199+0000","last_active":"2023-11-08T19:27:04.934040+0000","last_peered":"2023-11-08T19:27:04.934040+0000","last_clean":"2023-11-08T19:27:04.934040+0000","last_became_active":"2023-11-08T19:27:04.928847+0000","last_became_peered":"2023-11-08T19:27:04.928847+0000","last_unstale":"2023-11-08T19:27:04.934040+0000","last_undegraded":"2023-11-08T19:27:04.934040+0000","last_fullsized":"2023-11-08T19:27:04.934040+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:09:05.549788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340742+0000","last_change":"2023-11-08T19:27:04.915372+0000","last_active":"2023-11-08T19:27:06.340742+0000","last_peered":"2023-11-08T19:27:06.340742+0000","last_clean":"2023-11-08T19:27:06.340742+0000","last_became_active":"2023-11-08T19:27:04.915101+0000","last_became_peered":"2023-11-08T19:27:04.915101+0000","last_unstale":"2023-11-08T19:27:06.340742+0000","last_undegraded":"2023-11-08T19:27:06.340742+0000","last_fullsized":"2023-11-08T19:27:06.340742+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:39.250906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.921039+0000","last_change":"2023-11-08T19:27:04.912922+0000","last_active":"2023-11-08T19:27:04.921039+0000","last_peered":"2023-11-08T19:27:04.921039+0000","last_clean":"2023-11-08T19:27:04.921039+0000","last_became_active":"2023-11-08T19:27:04.912557+0000","last_became_peered":"2023-11-08T19:27:04.912557+0000","last_unstale":"2023-11-08T19:27:04.921039+0000","last_undegraded":"2023-11-08T19:27:04.921039+0000","last_fullsized":"2023-11-08T19:27:04.921039+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:09:57.760385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.920564+0000","last_change":"2023-11-08T19:27:04.913834+0000","last_active":"2023-11-08T19:27:04.920564+0000","last_peered":"2023-11-08T19:27:04.920564+0000","last_clean":"2023-11-08T19:27:04.920564+0000","last_became_active":"2023-11-08T19:27:04.913084+0000","last_became_peered":"2023-11-08T19:27:04.913084+0000","last_unstale":"2023-11-08T19:27:04.920564+0000","last_undegraded":"2023-11-08T19:27:04.920564+0000","last_fullsized":"2023-11-08T19:27:04.920564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:29.856508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:06.340291+0000","last_change":"2023-11-08T19:27:04.920753+0000","last_active":"2023-11-08T19:27:06.340291+0000","last_peered":"2023-11-08T19:27:06.340291+0000","last_clean":"2023-11-08T19:27:06.340291+0000","last_became_active":"2023-11-08T19:27:04.920360+0000","last_became_peered":"2023-11-08T19:27:04.920360+0000","last_unstale":"2023-11-08T19:27:06.340291+0000","last_undegraded":"2023-11-08T19:27:06.340291+0000","last_fullsized":"2023-11-08T19:27:06.340291+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:22:03.743456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.925926+0000","last_change":"2023-11-08T19:27:04.915874+0000","last_active":"2023-11-08T19:27:04.925926+0000","last_peered":"2023-11-08T19:27:04.925926+0000","last_clean":"2023-11-08T19:27:04.925926+0000","last_became_active":"2023-11-08T19:27:04.915597+0000","last_became_peered":"2023-11-08T19:27:04.915597+0000","last_unstale":"2023-11-08T19:27:04.925926+0000","last_undegraded":"2023-11-08T19:27:04.925926+0000","last_fullsized":"2023-11-08T19:27:04.925926+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:59:01.049984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.920626+0000","last_change":"2023-11-08T19:27:04.915405+0000","last_active":"2023-11-08T19:27:04.920626+0000","last_peered":"2023-11-08T19:27:04.920626+0000","last_clean":"2023-11-08T19:27:04.920626+0000","last_became_active":"2023-11-08T19:27:04.915013+0000","last_became_peered":"2023-11-08T19:27:04.915013+0000","last_unstale":"2023-11-08T19:27:04.920626+0000","last_undegraded":"2023-11-08T19:27:04.920626+0000","last_fullsized":"2023-11-08T19:27:04.920626+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:27.370958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.913382+0000","last_change":"2023-11-08T19:27:03.912055+0000","last_active":"2023-11-08T19:27:05.913382+0000","last_peered":"2023-11-08T19:27:05.913382+0000","last_clean":"2023-11-08T19:27:05.913382+0000","last_became_active":"2023-11-08T19:27:03.911771+0000","last_became_peered":"2023-11-08T19:27:03.911771+0000","last_unstale":"2023-11-08T19:27:05.913382+0000","last_undegraded":"2023-11-08T19:27:05.913382+0000","last_fullsized":"2023-11-08T19:27:05.913382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:46.984231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.888135+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:27:04.888135+0000","last_peered":"2023-11-08T19:27:04.888135+0000","last_clean":"2023-11-08T19:27:04.888135+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:27:04.888135+0000","last_undegraded":"2023-11-08T19:27:04.888135+0000","last_fullsized":"2023-11-08T19:27:04.888135+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.915878+0000","last_change":"2023-11-08T19:27:04.911411+0000","last_active":"2023-11-08T19:27:05.915878+0000","last_peered":"2023-11-08T19:27:05.915878+0000","last_clean":"2023-11-08T19:27:05.915878+0000","last_became_active":"2023-11-08T19:27:04.909022+0000","last_became_peered":"2023-11-08T19:27:04.909022+0000","last_unstale":"2023-11-08T19:27:05.915878+0000","last_undegraded":"2023-11-08T19:27:05.915878+0000","last_fullsized":"2023-11-08T19:27:05.915878+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:43.705543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.940534+0000","last_change":"2023-11-08T19:27:03.908080+0000","last_active":"2023-11-08T19:27:04.940534+0000","last_peered":"2023-11-08T19:27:04.940534+0000","last_clean":"2023-11-08T19:27:04.940534+0000","last_became_active":"2023-11-08T19:27:03.907721+0000","last_became_peered":"2023-11-08T19:27:03.907721+0000","last_unstale":"2023-11-08T19:27:04.940534+0000","last_undegraded":"2023-11-08T19:27:04.940534+0000","last_fullsized":"2023-11-08T19:27:04.940534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:31:54.746884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.894291+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:27:04.894291+0000","last_peered":"2023-11-08T19:27:04.894291+0000","last_clean":"2023-11-08T19:27:04.894291+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:27:04.894291+0000","last_undegraded":"2023-11-08T19:27:04.894291+0000","last_fullsized":"2023-11-08T19:27:04.894291+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.928450+0000","last_change":"2023-11-08T19:27:04.925710+0000","last_active":"2023-11-08T19:27:04.928450+0000","last_peered":"2023-11-08T19:27:04.928450+0000","last_clean":"2023-11-08T19:27:04.928450+0000","last_became_active":"2023-11-08T19:27:04.925395+0000","last_became_peered":"2023-11-08T19:27:04.925395+0000","last_unstale":"2023-11-08T19:27:04.928450+0000","last_undegraded":"2023-11-08T19:27:04.928450+0000","last_fullsized":"2023-11-08T19:27:04.928450+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:58.532658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.937447+0000","last_change":"2023-11-08T19:27:03.907134+0000","last_active":"2023-11-08T19:27:04.937447+0000","last_peered":"2023-11-08T19:27:04.937447+0000","last_clean":"2023-11-08T19:27:04.937447+0000","last_became_active":"2023-11-08T19:27:03.906810+0000","last_became_peered":"2023-11-08T19:27:03.906810+0000","last_unstale":"2023-11-08T19:27:04.937447+0000","last_undegraded":"2023-11-08T19:27:04.937447+0000","last_fullsized":"2023-11-08T19:27:04.937447+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:03.616731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.899784+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:27:04.899784+0000","last_peered":"2023-11-08T19:27:04.899784+0000","last_clean":"2023-11-08T19:27:04.899784+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:27:04.899784+0000","last_undegraded":"2023-11-08T19:27:04.899784+0000","last_fullsized":"2023-11-08T19:27:04.899784+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.915221+0000","last_change":"2023-11-08T19:27:04.912592+0000","last_active":"2023-11-08T19:27:04.915221+0000","last_peered":"2023-11-08T19:27:04.915221+0000","last_clean":"2023-11-08T19:27:04.915221+0000","last_became_active":"2023-11-08T19:27:04.912250+0000","last_became_peered":"2023-11-08T19:27:04.912250+0000","last_unstale":"2023-11-08T19:27:04.915221+0000","last_undegraded":"2023-11-08T19:27:04.915221+0000","last_fullsized":"2023-11-08T19:27:04.915221+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:47.590564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.915175+0000","last_change":"2023-11-08T19:27:03.904545+0000","last_active":"2023-11-08T19:27:05.915175+0000","last_peered":"2023-11-08T19:27:05.915175+0000","last_clean":"2023-11-08T19:27:05.915175+0000","last_became_active":"2023-11-08T19:27:03.904242+0000","last_became_peered":"2023-11-08T19:27:03.904242+0000","last_unstale":"2023-11-08T19:27:05.915175+0000","last_undegraded":"2023-11-08T19:27:05.915175+0000","last_fullsized":"2023-11-08T19:27:05.915175+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:12.652246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.889374+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:27:04.889374+0000","last_peered":"2023-11-08T19:27:04.889374+0000","last_clean":"2023-11-08T19:27:04.889374+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:27:04.889374+0000","last_undegraded":"2023-11-08T19:27:04.889374+0000","last_fullsized":"2023-11-08T19:27:04.889374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.934218+0000","last_change":"2023-11-08T19:27:04.931559+0000","last_active":"2023-11-08T19:27:04.934218+0000","last_peered":"2023-11-08T19:27:04.934218+0000","last_clean":"2023-11-08T19:27:04.934218+0000","last_became_active":"2023-11-08T19:27:04.931294+0000","last_became_peered":"2023-11-08T19:27:04.931294+0000","last_unstale":"2023-11-08T19:27:04.934218+0000","last_undegraded":"2023-11-08T19:27:04.934218+0000","last_fullsized":"2023-11-08T19:27:04.934218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:04.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.902585+0000","last_change":"2023-11-08T19:27:03.907571+0000","last_active":"2023-11-08T19:27:04.902585+0000","last_peered":"2023-11-08T19:27:04.902585+0000","last_clean":"2023-11-08T19:27:04.902585+0000","last_became_active":"2023-11-08T19:27:03.907272+0000","last_became_peered":"2023-11-08T19:27:03.907272+0000","last_unstale":"2023-11-08T19:27:04.902585+0000","last_undegraded":"2023-11-08T19:27:04.902585+0000","last_fullsized":"2023-11-08T19:27:04.902585+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:06.474193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.896550+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:27:04.896550+0000","last_peered":"2023-11-08T19:27:04.896550+0000","last_clean":"2023-11-08T19:27:04.896550+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:27:04.896550+0000","last_undegraded":"2023-11-08T19:27:04.896550+0000","last_fullsized":"2023-11-08T19:27:04.896550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.918820+0000","last_change":"2023-11-08T19:27:04.914786+0000","last_active":"2023-11-08T19:27:04.918820+0000","last_peered":"2023-11-08T19:27:04.918820+0000","last_clean":"2023-11-08T19:27:04.918820+0000","last_became_active":"2023-11-08T19:27:04.914428+0000","last_became_peered":"2023-11-08T19:27:04.914428+0000","last_unstale":"2023-11-08T19:27:04.918820+0000","last_undegraded":"2023-11-08T19:27:04.918820+0000","last_fullsized":"2023-11-08T19:27:04.918820+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:12:41.183786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.904225+0000","last_change":"2023-11-08T19:27:03.908441+0000","last_active":"2023-11-08T19:27:04.904225+0000","last_peered":"2023-11-08T19:27:04.904225+0000","last_clean":"2023-11-08T19:27:04.904225+0000","last_became_active":"2023-11-08T19:27:03.908074+0000","last_became_peered":"2023-11-08T19:27:03.908074+0000","last_unstale":"2023-11-08T19:27:04.904225+0000","last_undegraded":"2023-11-08T19:27:04.904225+0000","last_fullsized":"2023-11-08T19:27:04.904225+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:59:00.036730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.881017+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:27:03.881017+0000","last_peered":"2023-11-08T19:27:03.881017+0000","last_clean":"2023-11-08T19:27:03.881017+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:27:03.881017+0000","last_undegraded":"2023-11-08T19:27:03.881017+0000","last_fullsized":"2023-11-08T19:27:03.881017+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938059+0000","last_change":"2023-11-08T19:27:04.937203+0000","last_active":"2023-11-08T19:27:04.938059+0000","last_peered":"2023-11-08T19:27:04.938059+0000","last_clean":"2023-11-08T19:27:04.938059+0000","last_became_active":"2023-11-08T19:27:04.936912+0000","last_became_peered":"2023-11-08T19:27:04.936912+0000","last_unstale":"2023-11-08T19:27:04.938059+0000","last_undegraded":"2023-11-08T19:27:04.938059+0000","last_fullsized":"2023-11-08T19:27:04.938059+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:10:10.152395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.940630+0000","last_change":"2023-11-08T19:27:03.899892+0000","last_active":"2023-11-08T19:27:04.940630+0000","last_peered":"2023-11-08T19:27:04.940630+0000","last_clean":"2023-11-08T19:27:04.940630+0000","last_became_active":"2023-11-08T19:27:03.899580+0000","last_became_peered":"2023-11-08T19:27:03.899580+0000","last_unstale":"2023-11-08T19:27:04.940630+0000","last_undegraded":"2023-11-08T19:27:04.940630+0000","last_fullsized":"2023-11-08T19:27:04.940630+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:03.036895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.882073+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:27:03.882073+0000","last_peered":"2023-11-08T19:27:03.882073+0000","last_clean":"2023-11-08T19:27:03.882073+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:27:03.882073+0000","last_undegraded":"2023-11-08T19:27:03.882073+0000","last_fullsized":"2023-11-08T19:27:03.882073+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909628+0000","last_change":"2023-11-08T19:27:04.901305+0000","last_active":"2023-11-08T19:27:05.909628+0000","last_peered":"2023-11-08T19:27:05.909628+0000","last_clean":"2023-11-08T19:27:05.909628+0000","last_became_active":"2023-11-08T19:27:04.900993+0000","last_became_peered":"2023-11-08T19:27:04.900993+0000","last_unstale":"2023-11-08T19:27:05.909628+0000","last_undegraded":"2023-11-08T19:27:05.909628+0000","last_fullsized":"2023-11-08T19:27:05.909628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:27.370958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936994+0000","last_change":"2023-11-08T19:27:03.902924+0000","last_active":"2023-11-08T19:27:04.936994+0000","last_peered":"2023-11-08T19:27:04.936994+0000","last_clean":"2023-11-08T19:27:04.936994+0000","last_became_active":"2023-11-08T19:27:03.902417+0000","last_became_peered":"2023-11-08T19:27:03.902417+0000","last_unstale":"2023-11-08T19:27:04.936994+0000","last_undegraded":"2023-11-08T19:27:04.936994+0000","last_fullsized":"2023-11-08T19:27:04.936994+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:56.923012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":27,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.895589+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:27:03.895589+0000","last_peered":"2023-11-08T19:27:03.895589+0000","last_clean":"2023-11-08T19:27:03.895589+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:27:03.895589+0000","last_undegraded":"2023-11-08T19:27:03.895589+0000","last_fullsized":"2023-11-08T19:27:03.895589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936773+0000","last_change":"2023-11-08T19:27:04.928702+0000","last_active":"2023-11-08T19:27:04.936773+0000","last_peered":"2023-11-08T19:27:04.936773+0000","last_clean":"2023-11-08T19:27:04.936773+0000","last_became_active":"2023-11-08T19:27:04.928408+0000","last_became_peered":"2023-11-08T19:27:04.928408+0000","last_unstale":"2023-11-08T19:27:04.936773+0000","last_undegraded":"2023-11-08T19:27:04.936773+0000","last_fullsized":"2023-11-08T19:27:04.936773+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:17.846233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.932720+0000","last_change":"2023-11-08T19:27:03.909927+0000","last_active":"2023-11-08T19:27:04.932720+0000","last_peered":"2023-11-08T19:27:04.932720+0000","last_clean":"2023-11-08T19:27:04.932720+0000","last_became_active":"2023-11-08T19:27:03.909666+0000","last_became_peered":"2023-11-08T19:27:03.909666+0000","last_unstale":"2023-11-08T19:27:04.932720+0000","last_undegraded":"2023-11-08T19:27:04.932720+0000","last_fullsized":"2023-11-08T19:27:04.932720+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:12.652246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.898221+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:27:03.898221+0000","last_peered":"2023-11-08T19:27:03.898221+0000","last_clean":"2023-11-08T19:27:03.898221+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:27:03.898221+0000","last_undegraded":"2023-11-08T19:27:03.898221+0000","last_fullsized":"2023-11-08T19:27:03.898221+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935000+0000","last_change":"2023-11-08T19:27:04.924022+0000","last_active":"2023-11-08T19:27:04.935000+0000","last_peered":"2023-11-08T19:27:04.935000+0000","last_clean":"2023-11-08T19:27:04.935000+0000","last_became_active":"2023-11-08T19:27:04.923732+0000","last_became_peered":"2023-11-08T19:27:04.923732+0000","last_unstale":"2023-11-08T19:27:04.935000+0000","last_undegraded":"2023-11-08T19:27:04.935000+0000","last_fullsized":"2023-11-08T19:27:04.935000+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:37.168547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927129+0000","last_change":"2023-11-08T19:27:03.904899+0000","last_active":"2023-11-08T19:27:04.927129+0000","last_peered":"2023-11-08T19:27:04.927129+0000","last_clean":"2023-11-08T19:27:04.927129+0000","last_became_active":"2023-11-08T19:27:03.904141+0000","last_became_peered":"2023-11-08T19:27:03.904141+0000","last_unstale":"2023-11-08T19:27:04.927129+0000","last_undegraded":"2023-11-08T19:27:04.927129+0000","last_fullsized":"2023-11-08T19:27:04.927129+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:57.519404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.884400+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:27:03.884400+0000","last_peered":"2023-11-08T19:27:03.884400+0000","last_clean":"2023-11-08T19:27:03.884400+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:27:03.884400+0000","last_undegraded":"2023-11-08T19:27:03.884400+0000","last_fullsized":"2023-11-08T19:27:03.884400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.923229+0000","last_change":"2023-11-08T19:27:04.913816+0000","last_active":"2023-11-08T19:27:04.923229+0000","last_peered":"2023-11-08T19:27:04.923229+0000","last_clean":"2023-11-08T19:27:04.923229+0000","last_became_active":"2023-11-08T19:27:04.913569+0000","last_became_peered":"2023-11-08T19:27:04.913569+0000","last_unstale":"2023-11-08T19:27:04.923229+0000","last_undegraded":"2023-11-08T19:27:04.923229+0000","last_fullsized":"2023-11-08T19:27:04.923229+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:41.017102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.889880+0000","last_change":"2023-11-08T19:27:03.904570+0000","last_active":"2023-11-08T19:27:04.889880+0000","last_peered":"2023-11-08T19:27:04.889880+0000","last_clean":"2023-11-08T19:27:04.889880+0000","last_became_active":"2023-11-08T19:27:03.904254+0000","last_became_peered":"2023-11-08T19:27:03.904254+0000","last_unstale":"2023-11-08T19:27:04.889880+0000","last_undegraded":"2023-11-08T19:27:04.889880+0000","last_fullsized":"2023-11-08T19:27:04.889880+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:28.843254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.897783+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:27:03.897783+0000","last_peered":"2023-11-08T19:27:03.897783+0000","last_clean":"2023-11-08T19:27:03.897783+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:27:03.897783+0000","last_undegraded":"2023-11-08T19:27:03.897783+0000","last_fullsized":"2023-11-08T19:27:03.897783+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.913393+0000","last_change":"2023-11-08T19:27:04.914624+0000","last_active":"2023-11-08T19:27:05.913393+0000","last_peered":"2023-11-08T19:27:05.913393+0000","last_clean":"2023-11-08T19:27:05.913393+0000","last_became_active":"2023-11-08T19:27:04.911514+0000","last_became_peered":"2023-11-08T19:27:04.911514+0000","last_unstale":"2023-11-08T19:27:05.913393+0000","last_undegraded":"2023-11-08T19:27:05.913393+0000","last_fullsized":"2023-11-08T19:27:05.913393+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:28:50.948226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914998+0000","last_change":"2023-11-08T19:27:03.900167+0000","last_active":"2023-11-08T19:27:05.914998+0000","last_peered":"2023-11-08T19:27:05.914998+0000","last_clean":"2023-11-08T19:27:05.914998+0000","last_became_active":"2023-11-08T19:27:03.899708+0000","last_became_peered":"2023-11-08T19:27:03.899708+0000","last_unstale":"2023-11-08T19:27:05.914998+0000","last_undegraded":"2023-11-08T19:27:05.914998+0000","last_fullsized":"2023-11-08T19:27:05.914998+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:32.492997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":27,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.895753+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:27:03.895753+0000","last_peered":"2023-11-08T19:27:03.895753+0000","last_clean":"2023-11-08T19:27:03.895753+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:27:03.895753+0000","last_undegraded":"2023-11-08T19:27:03.895753+0000","last_fullsized":"2023-11-08T19:27:03.895753+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.910397+0000","last_change":"2023-11-08T19:27:04.915733+0000","last_active":"2023-11-08T19:27:05.910397+0000","last_peered":"2023-11-08T19:27:05.910397+0000","last_clean":"2023-11-08T19:27:05.910397+0000","last_became_active":"2023-11-08T19:27:04.915478+0000","last_became_peered":"2023-11-08T19:27:04.915478+0000","last_unstale":"2023-11-08T19:27:05.910397+0000","last_undegraded":"2023-11-08T19:27:05.910397+0000","last_fullsized":"2023-11-08T19:27:05.910397+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:57.936266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938381+0000","last_change":"2023-11-08T19:27:03.910798+0000","last_active":"2023-11-08T19:27:04.938381+0000","last_peered":"2023-11-08T19:27:04.938381+0000","last_clean":"2023-11-08T19:27:04.938381+0000","last_became_active":"2023-11-08T19:27:03.910524+0000","last_became_peered":"2023-11-08T19:27:03.910524+0000","last_unstale":"2023-11-08T19:27:04.938381+0000","last_undegraded":"2023-11-08T19:27:04.938381+0000","last_fullsized":"2023-11-08T19:27:04.938381+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:26.743339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.896665+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:27:03.896665+0000","last_peered":"2023-11-08T19:27:03.896665+0000","last_clean":"2023-11-08T19:27:03.896665+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:27:03.896665+0000","last_undegraded":"2023-11-08T19:27:03.896665+0000","last_fullsized":"2023-11-08T19:27:03.896665+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927262+0000","last_change":"2023-11-08T19:27:04.914804+0000","last_active":"2023-11-08T19:27:04.927262+0000","last_peered":"2023-11-08T19:27:04.927262+0000","last_clean":"2023-11-08T19:27:04.927262+0000","last_became_active":"2023-11-08T19:27:04.913952+0000","last_became_peered":"2023-11-08T19:27:04.913952+0000","last_unstale":"2023-11-08T19:27:04.927262+0000","last_undegraded":"2023-11-08T19:27:04.927262+0000","last_fullsized":"2023-11-08T19:27:04.927262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:29.856508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938081+0000","last_change":"2023-11-08T19:27:03.901576+0000","last_active":"2023-11-08T19:27:04.938081+0000","last_peered":"2023-11-08T19:27:04.938081+0000","last_clean":"2023-11-08T19:27:04.938081+0000","last_became_active":"2023-11-08T19:27:03.900686+0000","last_became_peered":"2023-11-08T19:27:03.900686+0000","last_unstale":"2023-11-08T19:27:04.938081+0000","last_undegraded":"2023-11-08T19:27:04.938081+0000","last_fullsized":"2023-11-08T19:27:04.938081+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:46.914427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.886338+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:27:03.886338+0000","last_peered":"2023-11-08T19:27:03.886338+0000","last_clean":"2023-11-08T19:27:03.886338+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:27:03.886338+0000","last_undegraded":"2023-11-08T19:27:03.886338+0000","last_fullsized":"2023-11-08T19:27:03.886338+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.932003+0000","last_change":"2023-11-08T19:27:04.922551+0000","last_active":"2023-11-08T19:27:04.932003+0000","last_peered":"2023-11-08T19:27:04.932003+0000","last_clean":"2023-11-08T19:27:04.932003+0000","last_became_active":"2023-11-08T19:27:04.920930+0000","last_became_peered":"2023-11-08T19:27:04.920930+0000","last_unstale":"2023-11-08T19:27:04.932003+0000","last_undegraded":"2023-11-08T19:27:04.932003+0000","last_fullsized":"2023-11-08T19:27:04.932003+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:34.302718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917277+0000","last_change":"2023-11-08T19:27:03.904810+0000","last_active":"2023-11-08T19:27:05.917277+0000","last_peered":"2023-11-08T19:27:05.917277+0000","last_clean":"2023-11-08T19:27:05.917277+0000","last_became_active":"2023-11-08T19:27:03.904328+0000","last_became_peered":"2023-11-08T19:27:03.904328+0000","last_unstale":"2023-11-08T19:27:05.917277+0000","last_undegraded":"2023-11-08T19:27:05.917277+0000","last_fullsized":"2023-11-08T19:27:05.917277+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:12:46.914427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.882286+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:27:03.882286+0000","last_peered":"2023-11-08T19:27:03.882286+0000","last_clean":"2023-11-08T19:27:03.882286+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:27:03.882286+0000","last_undegraded":"2023-11-08T19:27:03.882286+0000","last_fullsized":"2023-11-08T19:27:03.882286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.932902+0000","last_change":"2023-11-08T19:27:04.923408+0000","last_active":"2023-11-08T19:27:04.932902+0000","last_peered":"2023-11-08T19:27:04.932902+0000","last_clean":"2023-11-08T19:27:04.932902+0000","last_became_active":"2023-11-08T19:27:04.923068+0000","last_became_peered":"2023-11-08T19:27:04.923068+0000","last_unstale":"2023-11-08T19:27:04.932902+0000","last_undegraded":"2023-11-08T19:27:04.932902+0000","last_fullsized":"2023-11-08T19:27:04.932902+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:43.604779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936782+0000","last_change":"2023-11-08T19:27:03.894748+0000","last_active":"2023-11-08T19:27:04.936782+0000","last_peered":"2023-11-08T19:27:04.936782+0000","last_clean":"2023-11-08T19:27:04.936782+0000","last_became_active":"2023-11-08T19:27:03.889635+0000","last_became_peered":"2023-11-08T19:27:03.889635+0000","last_unstale":"2023-11-08T19:27:04.936782+0000","last_undegraded":"2023-11-08T19:27:04.936782+0000","last_fullsized":"2023-11-08T19:27:04.936782+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:42.692289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.900111+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:27:04.900111+0000","last_peered":"2023-11-08T19:27:04.900111+0000","last_clean":"2023-11-08T19:27:04.900111+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:27:04.900111+0000","last_undegraded":"2023-11-08T19:27:04.900111+0000","last_fullsized":"2023-11-08T19:27:04.900111+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927983+0000","last_change":"2023-11-08T19:27:04.925637+0000","last_active":"2023-11-08T19:27:04.927983+0000","last_peered":"2023-11-08T19:27:04.927983+0000","last_clean":"2023-11-08T19:27:04.927983+0000","last_became_active":"2023-11-08T19:27:04.925083+0000","last_became_peered":"2023-11-08T19:27:04.925083+0000","last_unstale":"2023-11-08T19:27:04.927983+0000","last_undegraded":"2023-11-08T19:27:04.927983+0000","last_fullsized":"2023-11-08T19:27:04.927983+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:05.247418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.890479+0000","last_change":"2023-11-08T19:27:03.901613+0000","last_active":"2023-11-08T19:27:04.890479+0000","last_peered":"2023-11-08T19:27:04.890479+0000","last_clean":"2023-11-08T19:27:04.890479+0000","last_became_active":"2023-11-08T19:27:03.901279+0000","last_became_peered":"2023-11-08T19:27:03.901279+0000","last_unstale":"2023-11-08T19:27:04.890479+0000","last_undegraded":"2023-11-08T19:27:04.890479+0000","last_fullsized":"2023-11-08T19:27:04.890479+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:26.357704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.900813+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:27:04.900813+0000","last_peered":"2023-11-08T19:27:04.900813+0000","last_clean":"2023-11-08T19:27:04.900813+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:27:04.900813+0000","last_undegraded":"2023-11-08T19:27:04.900813+0000","last_fullsized":"2023-11-08T19:27:04.900813+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935314+0000","last_change":"2023-11-08T19:27:04.927030+0000","last_active":"2023-11-08T19:27:04.935314+0000","last_peered":"2023-11-08T19:27:04.935314+0000","last_clean":"2023-11-08T19:27:04.935314+0000","last_became_active":"2023-11-08T19:27:04.925015+0000","last_became_peered":"2023-11-08T19:27:04.925015+0000","last_unstale":"2023-11-08T19:27:04.935314+0000","last_undegraded":"2023-11-08T19:27:04.935314+0000","last_fullsized":"2023-11-08T19:27:04.935314+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:47.306710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909733+0000","last_change":"2023-11-08T19:27:03.893095+0000","last_active":"2023-11-08T19:27:05.909733+0000","last_peered":"2023-11-08T19:27:05.909733+0000","last_clean":"2023-11-08T19:27:05.909733+0000","last_became_active":"2023-11-08T19:27:03.892485+0000","last_became_peered":"2023-11-08T19:27:03.892485+0000","last_unstale":"2023-11-08T19:27:05.909733+0000","last_undegraded":"2023-11-08T19:27:05.909733+0000","last_fullsized":"2023-11-08T19:27:05.909733+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:13.300416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.900354+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:27:04.900354+0000","last_peered":"2023-11-08T19:27:04.900354+0000","last_clean":"2023-11-08T19:27:04.900354+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:27:04.900354+0000","last_undegraded":"2023-11-08T19:27:04.900354+0000","last_fullsized":"2023-11-08T19:27:04.900354+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.924823+0000","last_change":"2023-11-08T19:27:04.919386+0000","last_active":"2023-11-08T19:27:04.924823+0000","last_peered":"2023-11-08T19:27:04.924823+0000","last_clean":"2023-11-08T19:27:04.924823+0000","last_became_active":"2023-11-08T19:27:04.919083+0000","last_became_peered":"2023-11-08T19:27:04.919083+0000","last_unstale":"2023-11-08T19:27:04.924823+0000","last_undegraded":"2023-11-08T19:27:04.924823+0000","last_fullsized":"2023-11-08T19:27:04.924823+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:27.843034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.890834+0000","last_change":"2023-11-08T19:27:03.905592+0000","last_active":"2023-11-08T19:27:04.890834+0000","last_peered":"2023-11-08T19:27:04.890834+0000","last_clean":"2023-11-08T19:27:04.890834+0000","last_became_active":"2023-11-08T19:27:03.905323+0000","last_became_peered":"2023-11-08T19:27:03.905323+0000","last_unstale":"2023-11-08T19:27:04.890834+0000","last_undegraded":"2023-11-08T19:27:04.890834+0000","last_fullsized":"2023-11-08T19:27:04.890834+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:42.692289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.887290+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:27:03.887290+0000","last_peered":"2023-11-08T19:27:03.887290+0000","last_clean":"2023-11-08T19:27:03.887290+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:27:03.887290+0000","last_undegraded":"2023-11-08T19:27:03.887290+0000","last_fullsized":"2023-11-08T19:27:03.887290+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909760+0000","last_change":"2023-11-08T19:27:04.910277+0000","last_active":"2023-11-08T19:27:05.909760+0000","last_peered":"2023-11-08T19:27:05.909760+0000","last_clean":"2023-11-08T19:27:05.909760+0000","last_became_active":"2023-11-08T19:27:04.909995+0000","last_became_peered":"2023-11-08T19:27:04.909995+0000","last_unstale":"2023-11-08T19:27:05.909760+0000","last_undegraded":"2023-11-08T19:27:05.909760+0000","last_fullsized":"2023-11-08T19:27:05.909760+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:05.247418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909355+0000","last_change":"2023-11-08T19:27:03.885745+0000","last_active":"2023-11-08T19:27:05.909355+0000","last_peered":"2023-11-08T19:27:05.909355+0000","last_clean":"2023-11-08T19:27:05.909355+0000","last_became_active":"2023-11-08T19:27:03.885501+0000","last_became_peered":"2023-11-08T19:27:03.885501+0000","last_unstale":"2023-11-08T19:27:05.909355+0000","last_undegraded":"2023-11-08T19:27:05.909355+0000","last_fullsized":"2023-11-08T19:27:05.909355+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:28:49.934972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.902449+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:27:04.902449+0000","last_peered":"2023-11-08T19:27:04.902449+0000","last_clean":"2023-11-08T19:27:04.902449+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:27:04.902449+0000","last_undegraded":"2023-11-08T19:27:04.902449+0000","last_fullsized":"2023-11-08T19:27:04.902449+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916741+0000","last_change":"2023-11-08T19:27:04.908438+0000","last_active":"2023-11-08T19:27:05.916741+0000","last_peered":"2023-11-08T19:27:05.916741+0000","last_clean":"2023-11-08T19:27:05.916741+0000","last_became_active":"2023-11-08T19:27:04.908120+0000","last_became_peered":"2023-11-08T19:27:04.908120+0000","last_unstale":"2023-11-08T19:27:05.916741+0000","last_undegraded":"2023-11-08T19:27:05.916741+0000","last_fullsized":"2023-11-08T19:27:05.916741+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:57.229596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.898500+0000","last_change":"2023-11-08T19:27:03.908637+0000","last_active":"2023-11-08T19:27:04.898500+0000","last_peered":"2023-11-08T19:27:04.898500+0000","last_clean":"2023-11-08T19:27:04.898500+0000","last_became_active":"2023-11-08T19:27:03.908202+0000","last_became_peered":"2023-11-08T19:27:03.908202+0000","last_unstale":"2023-11-08T19:27:04.898500+0000","last_undegraded":"2023-11-08T19:27:04.898500+0000","last_fullsized":"2023-11-08T19:27:04.898500+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:03.036895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.881571+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:27:03.881571+0000","last_peered":"2023-11-08T19:27:03.881571+0000","last_clean":"2023-11-08T19:27:03.881571+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:27:03.881571+0000","last_undegraded":"2023-11-08T19:27:03.881571+0000","last_fullsized":"2023-11-08T19:27:03.881571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916426+0000","last_change":"2023-11-08T19:27:04.920366+0000","last_active":"2023-11-08T19:27:05.916426+0000","last_peered":"2023-11-08T19:27:05.916426+0000","last_clean":"2023-11-08T19:27:05.916426+0000","last_became_active":"2023-11-08T19:27:04.918576+0000","last_became_peered":"2023-11-08T19:27:04.918576+0000","last_unstale":"2023-11-08T19:27:05.916426+0000","last_undegraded":"2023-11-08T19:27:05.916426+0000","last_fullsized":"2023-11-08T19:27:05.916426+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:27.843034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.910457+0000","last_change":"2023-11-08T19:27:03.890631+0000","last_active":"2023-11-08T19:27:05.910457+0000","last_peered":"2023-11-08T19:27:05.910457+0000","last_clean":"2023-11-08T19:27:05.910457+0000","last_became_active":"2023-11-08T19:27:03.890346+0000","last_became_peered":"2023-11-08T19:27:03.890346+0000","last_unstale":"2023-11-08T19:27:05.910457+0000","last_undegraded":"2023-11-08T19:27:05.910457+0000","last_fullsized":"2023-11-08T19:27:05.910457+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:28.843254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.899330+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:27:04.899330+0000","last_peered":"2023-11-08T19:27:04.899330+0000","last_clean":"2023-11-08T19:27:04.899330+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:27:04.899330+0000","last_undegraded":"2023-11-08T19:27:04.899330+0000","last_fullsized":"2023-11-08T19:27:04.899330+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927816+0000","last_change":"2023-11-08T19:27:04.920912+0000","last_active":"2023-11-08T19:27:04.927816+0000","last_peered":"2023-11-08T19:27:04.927816+0000","last_clean":"2023-11-08T19:27:04.927816+0000","last_became_active":"2023-11-08T19:27:04.920579+0000","last_became_peered":"2023-11-08T19:27:04.920579+0000","last_unstale":"2023-11-08T19:27:04.927816+0000","last_undegraded":"2023-11-08T19:27:04.927816+0000","last_fullsized":"2023-11-08T19:27:04.927816+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:02.402334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.917414+0000","last_change":"2023-11-08T19:27:03.904629+0000","last_active":"2023-11-08T19:27:05.917414+0000","last_peered":"2023-11-08T19:27:05.917414+0000","last_clean":"2023-11-08T19:27:05.917414+0000","last_became_active":"2023-11-08T19:27:03.903936+0000","last_became_peered":"2023-11-08T19:27:03.903936+0000","last_unstale":"2023-11-08T19:27:05.917414+0000","last_undegraded":"2023-11-08T19:27:05.917414+0000","last_fullsized":"2023-11-08T19:27:05.917414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:28:33.289464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.887391+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:27:04.887391+0000","last_peered":"2023-11-08T19:27:04.887391+0000","last_clean":"2023-11-08T19:27:04.887391+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:27:04.887391+0000","last_undegraded":"2023-11-08T19:27:04.887391+0000","last_fullsized":"2023-11-08T19:27:04.887391+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909634+0000","last_change":"2023-11-08T19:27:04.900445+0000","last_active":"2023-11-08T19:27:05.909634+0000","last_peered":"2023-11-08T19:27:05.909634+0000","last_clean":"2023-11-08T19:27:05.909634+0000","last_became_active":"2023-11-08T19:27:04.899584+0000","last_became_peered":"2023-11-08T19:27:04.899584+0000","last_unstale":"2023-11-08T19:27:05.909634+0000","last_undegraded":"2023-11-08T19:27:05.909634+0000","last_fullsized":"2023-11-08T19:27:05.909634+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:04.050149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914660+0000","last_change":"2023-11-08T19:27:03.912669+0000","last_active":"2023-11-08T19:27:05.914660+0000","last_peered":"2023-11-08T19:27:05.914660+0000","last_clean":"2023-11-08T19:27:05.914660+0000","last_became_active":"2023-11-08T19:27:03.912340+0000","last_became_peered":"2023-11-08T19:27:03.912340+0000","last_unstale":"2023-11-08T19:27:05.914660+0000","last_undegraded":"2023-11-08T19:27:05.914660+0000","last_fullsized":"2023-11-08T19:27:05.914660+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:31:22.048098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.885693+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:27:03.885693+0000","last_peered":"2023-11-08T19:27:03.885693+0000","last_clean":"2023-11-08T19:27:03.885693+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:27:03.885693+0000","last_undegraded":"2023-11-08T19:27:03.885693+0000","last_fullsized":"2023-11-08T19:27:03.885693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.933255+0000","last_change":"2023-11-08T19:27:04.930297+0000","last_active":"2023-11-08T19:27:04.933255+0000","last_peered":"2023-11-08T19:27:04.933255+0000","last_clean":"2023-11-08T19:27:04.933255+0000","last_became_active":"2023-11-08T19:27:04.929969+0000","last_became_peered":"2023-11-08T19:27:04.929969+0000","last_unstale":"2023-11-08T19:27:04.933255+0000","last_undegraded":"2023-11-08T19:27:04.933255+0000","last_fullsized":"2023-11-08T19:27:04.933255+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:53:57.936266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.909591+0000","last_change":"2023-11-08T19:27:03.902190+0000","last_active":"2023-11-08T19:27:05.909591+0000","last_peered":"2023-11-08T19:27:05.909591+0000","last_clean":"2023-11-08T19:27:05.909591+0000","last_became_active":"2023-11-08T19:27:03.901899+0000","last_became_peered":"2023-11-08T19:27:03.901899+0000","last_unstale":"2023-11-08T19:27:05.909591+0000","last_undegraded":"2023-11-08T19:27:05.909591+0000","last_fullsized":"2023-11-08T19:27:05.909591+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:52:21.438332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":26,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.898304+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:27:03.898304+0000","last_peered":"2023-11-08T19:27:03.898304+0000","last_clean":"2023-11-08T19:27:03.898304+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:27:03.898304+0000","last_undegraded":"2023-11-08T19:27:03.898304+0000","last_fullsized":"2023-11-08T19:27:03.898304+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916195+0000","last_change":"2023-11-08T19:27:04.910936+0000","last_active":"2023-11-08T19:27:05.916195+0000","last_peered":"2023-11-08T19:27:05.916195+0000","last_clean":"2023-11-08T19:27:05.916195+0000","last_became_active":"2023-11-08T19:27:04.910659+0000","last_became_peered":"2023-11-08T19:27:04.910659+0000","last_unstale":"2023-11-08T19:27:05.916195+0000","last_undegraded":"2023-11-08T19:27:05.916195+0000","last_fullsized":"2023-11-08T19:27:05.916195+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:10:41.017102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938580+0000","last_change":"2023-11-08T19:27:03.904556+0000","last_active":"2023-11-08T19:27:04.938580+0000","last_peered":"2023-11-08T19:27:04.938580+0000","last_clean":"2023-11-08T19:27:04.938580+0000","last_became_active":"2023-11-08T19:27:03.904250+0000","last_became_peered":"2023-11-08T19:27:03.904250+0000","last_unstale":"2023-11-08T19:27:04.938580+0000","last_undegraded":"2023-11-08T19:27:04.938580+0000","last_fullsized":"2023-11-08T19:27:04.938580+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:17.602970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.882702+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:27:03.882702+0000","last_peered":"2023-11-08T19:27:03.882702+0000","last_clean":"2023-11-08T19:27:03.882702+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:27:03.882702+0000","last_undegraded":"2023-11-08T19:27:03.882702+0000","last_fullsized":"2023-11-08T19:27:03.882702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927964+0000","last_change":"2023-11-08T19:27:04.914484+0000","last_active":"2023-11-08T19:27:04.927964+0000","last_peered":"2023-11-08T19:27:04.927964+0000","last_clean":"2023-11-08T19:27:04.927964+0000","last_became_active":"2023-11-08T19:27:04.914215+0000","last_became_peered":"2023-11-08T19:27:04.914215+0000","last_unstale":"2023-11-08T19:27:04.927964+0000","last_undegraded":"2023-11-08T19:27:04.927964+0000","last_fullsized":"2023-11-08T19:27:04.927964+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:33.506251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916544+0000","last_change":"2023-11-08T19:27:03.897253+0000","last_active":"2023-11-08T19:27:05.916544+0000","last_peered":"2023-11-08T19:27:05.916544+0000","last_clean":"2023-11-08T19:27:05.916544+0000","last_became_active":"2023-11-08T19:27:03.896651+0000","last_became_peered":"2023-11-08T19:27:03.896651+0000","last_unstale":"2023-11-08T19:27:05.916544+0000","last_undegraded":"2023-11-08T19:27:05.916544+0000","last_fullsized":"2023-11-08T19:27:05.916544+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:37.598146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":26,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.899940+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:27:04.899940+0000","last_peered":"2023-11-08T19:27:04.899940+0000","last_clean":"2023-11-08T19:27:04.899940+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:27:04.899940+0000","last_undegraded":"2023-11-08T19:27:04.899940+0000","last_fullsized":"2023-11-08T19:27:04.899940+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916721+0000","last_change":"2023-11-08T19:27:04.920650+0000","last_active":"2023-11-08T19:27:05.916721+0000","last_peered":"2023-11-08T19:27:05.916721+0000","last_clean":"2023-11-08T19:27:05.916721+0000","last_became_active":"2023-11-08T19:27:04.920343+0000","last_became_peered":"2023-11-08T19:27:04.920343+0000","last_unstale":"2023-11-08T19:27:05.916721+0000","last_undegraded":"2023-11-08T19:27:05.916721+0000","last_fullsized":"2023-11-08T19:27:05.916721+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:00.218479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.936171+0000","last_change":"2023-11-08T19:27:03.906001+0000","last_active":"2023-11-08T19:27:04.936171+0000","last_peered":"2023-11-08T19:27:04.936171+0000","last_clean":"2023-11-08T19:27:04.936171+0000","last_became_active":"2023-11-08T19:27:03.904816+0000","last_became_peered":"2023-11-08T19:27:03.904816+0000","last_unstale":"2023-11-08T19:27:04.936171+0000","last_undegraded":"2023-11-08T19:27:04.936171+0000","last_fullsized":"2023-11-08T19:27:04.936171+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:26.357704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":27,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.895444+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:27:03.895444+0000","last_peered":"2023-11-08T19:27:03.895444+0000","last_clean":"2023-11-08T19:27:03.895444+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:27:03.895444+0000","last_undegraded":"2023-11-08T19:27:03.895444+0000","last_fullsized":"2023-11-08T19:27:03.895444+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.921866+0000","last_change":"2023-11-08T19:27:04.916409+0000","last_active":"2023-11-08T19:27:04.921866+0000","last_peered":"2023-11-08T19:27:04.921866+0000","last_clean":"2023-11-08T19:27:04.921866+0000","last_became_active":"2023-11-08T19:27:04.916107+0000","last_became_peered":"2023-11-08T19:27:04.916107+0000","last_unstale":"2023-11-08T19:27:04.921866+0000","last_undegraded":"2023-11-08T19:27:04.921866+0000","last_fullsized":"2023-11-08T19:27:04.921866+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:38.611400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.938481+0000","last_change":"2023-11-08T19:27:03.908821+0000","last_active":"2023-11-08T19:27:04.938481+0000","last_peered":"2023-11-08T19:27:04.938481+0000","last_clean":"2023-11-08T19:27:04.938481+0000","last_became_active":"2023-11-08T19:27:03.907215+0000","last_became_peered":"2023-11-08T19:27:03.907215+0000","last_unstale":"2023-11-08T19:27:04.938481+0000","last_undegraded":"2023-11-08T19:27:04.938481+0000","last_fullsized":"2023-11-08T19:27:04.938481+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:04.741435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":27,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.891854+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:27:04.891854+0000","last_peered":"2023-11-08T19:27:04.891854+0000","last_clean":"2023-11-08T19:27:04.891854+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:27:04.891854+0000","last_undegraded":"2023-11-08T19:27:04.891854+0000","last_fullsized":"2023-11-08T19:27:04.891854+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.926862+0000","last_change":"2023-11-08T19:27:04.923108+0000","last_active":"2023-11-08T19:27:04.926862+0000","last_peered":"2023-11-08T19:27:04.926862+0000","last_clean":"2023-11-08T19:27:04.926862+0000","last_became_active":"2023-11-08T19:27:04.922936+0000","last_became_peered":"2023-11-08T19:27:04.922936+0000","last_unstale":"2023-11-08T19:27:04.926862+0000","last_undegraded":"2023-11-08T19:27:04.926862+0000","last_fullsized":"2023-11-08T19:27:04.926862+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:33.506251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.905793+0000","last_change":"2023-11-08T19:27:03.896581+0000","last_active":"2023-11-08T19:27:04.905793+0000","last_peered":"2023-11-08T19:27:04.905793+0000","last_clean":"2023-11-08T19:27:04.905793+0000","last_became_active":"2023-11-08T19:27:03.896291+0000","last_became_peered":"2023-11-08T19:27:03.896291+0000","last_unstale":"2023-11-08T19:27:04.905793+0000","last_undegraded":"2023-11-08T19:27:04.905793+0000","last_fullsized":"2023-11-08T19:27:04.905793+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:10:09.139141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.887232+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:27:03.887232+0000","last_peered":"2023-11-08T19:27:03.887232+0000","last_clean":"2023-11-08T19:27:03.887232+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:27:03.887232+0000","last_undegraded":"2023-11-08T19:27:03.887232+0000","last_fullsized":"2023-11-08T19:27:03.887232+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.935054+0000","last_change":"2023-11-08T19:27:04.923610+0000","last_active":"2023-11-08T19:27:04.935054+0000","last_peered":"2023-11-08T19:27:04.935054+0000","last_clean":"2023-11-08T19:27:04.935054+0000","last_became_active":"2023-11-08T19:27:04.923023+0000","last_became_peered":"2023-11-08T19:27:04.923023+0000","last_unstale":"2023-11-08T19:27:04.935054+0000","last_undegraded":"2023-11-08T19:27:04.935054+0000","last_fullsized":"2023-11-08T19:27:04.935054+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:33.083930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.916800+0000","last_change":"2023-11-08T19:27:03.908831+0000","last_active":"2023-11-08T19:27:05.916800+0000","last_peered":"2023-11-08T19:27:05.916800+0000","last_clean":"2023-11-08T19:27:05.916800+0000","last_became_active":"2023-11-08T19:27:03.908545+0000","last_became_peered":"2023-11-08T19:27:03.908545+0000","last_unstale":"2023-11-08T19:27:05.916800+0000","last_undegraded":"2023-11-08T19:27:05.916800+0000","last_fullsized":"2023-11-08T19:27:05.916800+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:01.389080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":25,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.885750+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:27:03.885750+0000","last_peered":"2023-11-08T19:27:03.885750+0000","last_clean":"2023-11-08T19:27:03.885750+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:27:03.885750+0000","last_undegraded":"2023-11-08T19:27:03.885750+0000","last_fullsized":"2023-11-08T19:27:03.885750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-08T19:27:04.927591+0000","last_change":"2023-11-08T19:27:04.918967+0000","last_active":"2023-11-08T19:27:04.927591+0000","last_peered":"2023-11-08T19:27:04.927591+0000","last_clean":"2023-11-08T19:27:04.927591+0000","last_became_active":"2023-11-08T19:27:04.918628+0000","last_became_peered":"2023-11-08T19:27:04.918628+0000","last_unstale":"2023-11-08T19:27:04.927591+0000","last_undegraded":"2023-11-08T19:27:04.927591+0000","last_fullsized":"2023-11-08T19:27:04.927591+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:03.859995+0000","last_clean_scrub_stamp":"2023-11-08T19:27:03.859995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:09:05.549788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-08T19:27:05.914399+0000","last_change":"2023-11-08T19:27:03.911427+0000","last_active":"2023-11-08T19:27:05.914399+0000","last_peered":"2023-11-08T19:27:05.914399+0000","last_clean":"2023-11-08T19:27:05.914399+0000","last_became_active":"2023-11-08T19:27:03.911152+0000","last_became_peered":"2023-11-08T19:27:03.911152+0000","last_unstale":"2023-11-08T19:27:05.914399+0000","last_undegraded":"2023-11-08T19:27:05.914399+0000","last_fullsized":"2023-11-08T19:27:05.914399+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:02.846741+0000","last_clean_scrub_stamp":"2023-11-08T19:27:02.846741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:23:20.468848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.891475+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:27:03.891475+0000","last_peered":"2023-11-08T19:27:03.891475+0000","last_clean":"2023-11-08T19:27:03.891475+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:27:03.891475+0000","last_undegraded":"2023-11-08T19:27:03.891475+0000","last_fullsized":"2023-11-08T19:27:03.891475+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":195,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T19:27:03.889644+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:27:03.889644+0000","last_peered":"2023-11-08T19:27:03.889644+0000","last_clean":"2023-11-08T19:27:03.889644+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:27:03.889644+0000","last_undegraded":"2023-11-08T19:27:03.889644+0000","last_fullsized":"2023-11-08T19:27:03.889644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32388,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339452,"statfs":{"total":96636764160,"available":96603598848,"internally_reserved":0,"allocated":1232896,"data_stored":877151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.087,"15min":1.1040000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4930000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.29,"15min":1.325},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4450000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.625}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.232,"15min":1.2689999999999999},"min":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6619999999999999,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.1060000000000001,"15min":1.1299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3160000000000001,"5min":1.681,"15min":1.681},"last":0.57399999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.466,"15min":1.5169999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.5089999999999999,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.133,"5min":1.349,"15min":1.385},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.772,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.76800000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.341,"15min":1.377},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.419,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.272,"5min":1.536,"15min":1.5800000000000001},"min":{"1min":0.95299999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6439999999999999,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":0.83799999999999997}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3560000000000001,"15min":1.3700000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.704,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.4690000000000001,"15min":1.5109999999999999},"min":{"1min":0.82799999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.5780000000000001,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4470000000000001,"15min":1.4650000000000001},"min":{"1min":1.004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.766,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":1.446,"5min":1.4850000000000001,"15min":1.4910000000000001},"min":{"1min":1.1599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":1.9590000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.81699999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5700000000000001,"15min":1.587},"min":{"1min":1.036,"5min":1.036,"15min":1.036},"max":{"1min":1.8680000000000001,"5min":2.427,"15min":2.427},"last":0.878},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.6060000000000001,"15min":1.6439999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.746,"5min":2.468,"15min":2.468},"last":0.997}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":643072,"data_stored":293596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":0.95399999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.83399999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.71299999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":1.089},"min":{"1min":0.747,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.534,"5min":1.679,"15min":1.679},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.331,"15min":1.3540000000000001},"min":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.591,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.274,"15min":1.304},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4990000000000001,"5min":4.609,"15min":4.609},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2210000000000001,"15min":1.214},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.5609999999999999,"5min":1.861,"15min":1.861},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.877,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.179,"15min":1.1799999999999999},"min":{"1min":0.80300000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.663,"5min":1.784,"15min":1.784},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.383,"15min":1.381},"min":{"1min":0.89900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.024,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":1.04,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.89500000000000002}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.98199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7749999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":1.347,"5min":1.3320000000000001,"15min":1.3300000000000001},"min":{"1min":0.90200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.831,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.83099999999999996}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5509999999999999,"15min":1.5449999999999999},"min":{"1min":1.097,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1240000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.96599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.724,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":0.81499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":651264,"data_stored":294248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0900000000000001,"15min":1.073},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.712,"5min":1.76,"15min":1.76},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.246,"15min":1.2390000000000001},"min":{"1min":0.879,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.645,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.165}]},{"osd":1,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.167,"15min":1.179},"min":{"1min":0.82399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.401,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.405,"15min":1.462},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4970000000000001,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.345}]},{"osd":2,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1799999999999999,"15min":1.171},"min":{"1min":0.91500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8700000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.623},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.4650000000000001,"15min":1.5229999999999999},"min":{"1min":0.76100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.5900000000000001,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":0.96999999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8200000000000001,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.3149999999999999,"15min":1.3069999999999999},"min":{"1min":0.89900000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7609999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.502}]},{"osd":4,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.288,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.966,"5min":2.347,"15min":2.347},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.544,"15min":1.5629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0230000000000001,"5min":3.153,"15min":3.153},"last":1.4610000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.516,"15min":1.536},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9199999999999999,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.569},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.494,"15min":1.5069999999999999},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.869,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.407}]},{"osd":7,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.6830000000000001,"15min":1.7150000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1269999999999998,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.288},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.587,"15min":1.5920000000000001},"min":{"1min":0.99399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.994,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.673}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":288235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.036,"15min":1.0409999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.492,"5min":1.248,"15min":1.2070000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":7.0119999999999996,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.89400000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.103,"15min":1.1020000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.121,"15min":1.163},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2470000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.145}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3520000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.8719999999999999,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.431},{"interface":"front","average":{"1min":1.177,"5min":1.3959999999999999,"15min":1.4319999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.375,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.5589999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3080000000000001,"15min":1.302},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.91900000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0470000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.232}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.486,"15min":1.528},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.4630000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.298},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5469999999999999,"15min":1.5660000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.3799999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5569999999999999,"15min":1.573},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.2360000000000002,"5min":4.335,"15min":4.335},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.613,"15min":1.6339999999999999},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.47}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.637,"15min":1.6419999999999999},"min":{"1min":0.95499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.5490000000000004,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.7170000000000001,"15min":1.7430000000000001},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":3.1389999999999998,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.51}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27652,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":638976,"data_stored":286679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.056,"15min":1.0960000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2909999999999999,"5min":3.972,"15min":3.972},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3999999999999999,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.56599999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.1679999999999999,"15min":1.2070000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6839999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.90200000000000002,"5min":1.0900000000000001,"15min":1.121},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3360000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.871}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.3089999999999999,"15min":1.3380000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.752,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.101},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.256,"15min":1.282},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.778,"5min":2.73,"15min":2.73},"last":0.79000000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.4350000000000001,"15min":1.4690000000000001},"min":{"1min":0.872,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.861,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.448,"15min":1.47},"min":{"1min":0.91400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5980000000000001,"5min":3.052,"15min":3.052},"last":1.028}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.4750000000000001,"15min":1.5109999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.149,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.331,"5min":1.5169999999999999,"15min":1.548},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.375,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":0.99199999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":1.054,"5min":0.878,"15min":0.878},"max":{"1min":1.8220000000000001,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.5740000000000001,"15min":1.6000000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.4860000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.1799999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.6930000000000001,"15min":1.736},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.528,"5min":4.407,"15min":4.407},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.573,"15min":1.5960000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8740000000000001,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.3149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27640,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":634880,"data_stored":285660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.222,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.90200000000000002,"15min":0.93400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.206,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.101}]},{"osd":1,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.202,"5min":1.393,"15min":1.393},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.02},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.177,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.4079999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.51300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.486,"5min":1.53,"15min":1.53},"last":1.71},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.099,"15min":1.139},"min":{"1min":0.48099999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3859999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.1699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.3600000000000001,"15min":1.421},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3100000000000001,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.607},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3360000000000001,"15min":1.3779999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.506,"5min":2.278,"15min":2.278},"last":1.329}]},{"osd":5,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.389,"15min":1.4350000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6180000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.77},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.456,"5min":2.536,"15min":2.536},"last":1.954}]},{"osd":6,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.363,"15min":1.395},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5409999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.4219999999999999,"15min":1.468},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8480000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.655}]},{"osd":7,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.5389999999999999,"15min":1.593},"min":{"1min":0.81399999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5329999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.861},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.5289999999999999,"15min":1.5680000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.5840000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.8140000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":647168,"data_stored":293686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.749,"5min":0.753,"15min":0.754},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.0369999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.86499999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1339999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.88300000000000001,"15min":0.88600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.224,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.93700000000000006}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0580000000000001,"15min":1.0569999999999999},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.4970000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.499,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8540000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.669}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.542,"5min":2.165,"15min":2.165},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2410000000000001,"15min":1.2569999999999999},"min":{"1min":0.73799999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4910000000000001,"5min":1.885,"15min":1.885},"last":1.242}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.254,"15min":1.264},"min":{"1min":0.71099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7390000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.45},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.706,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.375}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.359,"15min":1.367},"min":{"1min":0.754,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.772,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.373,"15min":1.3859999999999999},"min":{"1min":0.92000000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.6930000000000001,"5min":2.282,"15min":2.282},"last":1.5329999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.329},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6559999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3740000000000001,"15min":1.3919999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.956,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.3169999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32392,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339448,"statfs":{"total":96636764160,"available":96603594752,"internally_reserved":0,"allocated":1236992,"data_stored":878166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0649999999999999,"5min":1.357,"15min":1.357},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2889999999999999,"5min":1.7,"15min":1.7},"last":0.97199999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2410000000000001,"5min":1.508,"15min":1.508},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.91400000000000003}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1,"15min":1.012},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.278,"5min":1.748,"15min":1.748},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0880000000000001,"15min":1.115},"min":{"1min":0.53500000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":2.496,"15min":2.496},"last":1.202}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2290000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5169999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.363},{"interface":"front","average":{"1min":1.169,"5min":1.4159999999999999,"15min":1.4570000000000001},"min":{"1min":0.81699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8700000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.169}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.351,"15min":1.3819999999999999},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9510000000000001,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.427,"15min":1.4570000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7070000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.516}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.516},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.8160000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.3999999999999999,"15min":1.4199999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6259999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.1220000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3899999999999999,"15min":1.4059999999999999},"min":{"1min":0.92600000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.746,"5min":2.173,"15min":2.173},"last":1.232},{"interface":"front","average":{"1min":1.24,"5min":1.4590000000000001,"15min":1.4950000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.0070000000000001,"5min":2.306,"15min":2.306},"last":1.4159999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:09.843 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:27:09.844 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:27:10.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.095+0000 7f71ab325640 1 Processor -- start 2023-11-08T19:27:10.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.095+0000 7f71ab325640 1 -- start start 2023-11-08T19:27:10.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.097+0000 7f71ab325640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a4143600 0x7f71a40c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.097+0000 7f71ab325640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71a406d250 0x7f71a415de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.097+0000 7f71ab325640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71a415e440 con 0x7f71a415d9f0 2023-11-08T19:27:10.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.097+0000 7f71ab325640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a40c4f50 con 0x7f71a406d250 2023-11-08T19:27:10.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.097+0000 7f71ab325640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a40c4960 con 0x7f71a4143600 2023-11-08T19:27:10.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.098+0000 7f71a98c2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71a406d250 0x7f71a415de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.098+0000 7f71a88c0640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f71a415d9f0 0x7f71a406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:50614/0 (socket says 172.21.15.119:50614) 2023-11-08T19:27:10.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.098+0000 7f71a98c2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71a406d250 0x7f71a415de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:36430/0 (socket says 172.21.15.119:36430) 2023-11-08T19:27:10.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.098+0000 7f71a88c0640 1 -- 172.21.15.119:0/707481541 learned_addr learned my addr 172.21.15.119:0/707481541 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:10.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.098+0000 7f71a90c1640 1 --2- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a4143600 0x7f71a40c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.099+0000 7f71a98c2640 1 -- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a4143600 msgr2=0x7f71a40c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.099+0000 7f71a98c2640 1 --2- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a4143600 0x7f71a40c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.099+0000 7f71a98c2640 1 -- 172.21.15.119:0/707481541 >> v1:172.21.15.188:6789/0 conn(0x7f71a415d9f0 legacy=0x7f71a406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.099+0000 7f71a98c2640 1 -- 172.21.15.119:0/707481541 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a415e720 con 0x7f71a406d250 2023-11-08T19:27:10.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.100+0000 7f71a98c2640 1 --2- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a406d250 0x7f71a415de30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f719c009d90 tx=0x7f719c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=814c40ec6545d863 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.101+0000 7f719bfff640 1 -- 172.21.15.119:0/707481541 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c02cbc0 con 0x7f71a406d250 2023-11-08T19:27:10.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.101+0000 7f719bfff640 1 -- 172.21.15.119:0/707481541 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c02cd20 con 0x7f71a406d250 2023-11-08T19:27:10.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.101+0000 7f719bfff640 1 -- 172.21.15.119:0/707481541 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c034660 con 0x7f71a406d250 2023-11-08T19:27:10.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 -- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a406d250 msgr2=0x7f71a415de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 --2- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a406d250 0x7f71a415de30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f719c009d90 tx=0x7f719c02be10 comp rx=0 tx=0).stop 2023-11-08T19:27:10.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 -- 172.21.15.119:0/707481541 shutdown_connections 2023-11-08T19:27:10.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 --2- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a406d250 0x7f71a415de30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 --2- 172.21.15.119:0/707481541 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a4143600 0x7f71a40c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 -- 172.21.15.119:0/707481541 >> 172.21.15.119:0/707481541 conn(0x7f71a40cb610 msgr2=0x7f71a40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 -- 172.21.15.119:0/707481541 shutdown_connections 2023-11-08T19:27:10.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.102+0000 7f71ab325640 1 -- 172.21.15.119:0/707481541 wait complete. 2023-11-08T19:27:10.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.104+0000 7f71ab325640 1 Processor -- start 2023-11-08T19:27:10.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.104+0000 7f71ab325640 1 -- start start 2023-11-08T19:27:10.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71ab325640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a406d250 0x7f71a413f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71ab325640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a4143600 0x7f71a4140a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71ab325640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 0x7f71a40700a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71ab325640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4142310 con 0x7f71a415d9f0 2023-11-08T19:27:10.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71ab325640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a413f690 con 0x7f71a4143600 2023-11-08T19:27:10.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71ab325640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a4141270 con 0x7f71a406d250 2023-11-08T19:27:10.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71a98c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 0x7f71a40700a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71a98c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 0x7f71a40700a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49890/0 (socket says 172.21.15.119:49890) 2023-11-08T19:27:10.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71a98c2640 1 -- 172.21.15.119:0/334726351 learned_addr learned my addr 172.21.15.119:0/334726351 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:10.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71a88c0640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a4143600 0x7f71a4140a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71a90c1640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a406d250 0x7f71a413f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.105+0000 7f71a98c2640 1 -- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a406d250 msgr2=0x7f71a413f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71a98c2640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a406d250 0x7f71a413f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71a98c2640 1 -- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a4143600 msgr2=0x7f71a4140a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71a98c2640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a4143600 0x7f71a4140a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71a98c2640 1 -- 172.21.15.119:0/334726351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c009a70 con 0x7f71a415d9f0 2023-11-08T19:27:10.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71a98c2640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 0x7f71a40700a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f719c009eb0 tx=0x7f719c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c040650 con 0x7f71a415d9f0 2023-11-08T19:27:10.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c02ce90 con 0x7f71a415d9f0 2023-11-08T19:27:10.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a4141550 con 0x7f71a415d9f0 2023-11-08T19:27:10.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c0409f0 con 0x7f71a415d9f0 2023-11-08T19:27:10.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.106+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a4141b10 con 0x7f71a415d9f0 2023-11-08T19:27:10.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.108+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a4074430 con 0x7f71a415d9f0 2023-11-08T19:27:10.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.108+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f719c040b50 con 0x7f71a415d9f0 2023-11-08T19:27:10.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.109+0000 7f7199ffb640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f718806ae70 0x7f718806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.109+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f719c0ac4c0 con 0x7f71a415d9f0 2023-11-08T19:27:10.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.110+0000 7f71a90c1640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f718806ae70 0x7f718806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.110+0000 7f71a90c1640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f718806ae70 0x7f718806d330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009a10 tx=0x7f71a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.114+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f719c077230 con 0x7f71a415d9f0 2023-11-08T19:27:10.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.376+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f71a406d010 con 0x7f71a415d9f0 2023-11-08T19:27:10.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.380+0000 7f7199ffb640 1 -- 172.21.15.119:0/334726351 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f719c009450 con 0x7f71a415d9f0 2023-11-08T19:27:10.382 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-11-08T19:27:10.382 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:27:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2023-11-08T19:27:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2023-11-08T19:27:10.383 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:10.383 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-11-08T19:27:10.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.383+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f718806ae70 msgr2=0x7f718806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f718806ae70 0x7f718806d330 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009a10 tx=0x7f71a0008040 comp rx=0 tx=0).stop 2023-11-08T19:27:10.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 msgr2=0x7f71a40700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 0x7f71a40700a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f719c009eb0 tx=0x7f719c004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:10.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 shutdown_connections 2023-11-08T19:27:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f718806ae70 0x7f718806d330 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71a415d9f0 0x7f71a40700a0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a4143600 0x7f71a4140a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 --2- 172.21.15.119:0/334726351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a406d250 0x7f71a413f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 >> 172.21.15.119:0/334726351 conn(0x7f71a40cb610 msgr2=0x7f71a40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 shutdown_connections 2023-11-08T19:27:10.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.384+0000 7f71ab325640 1 -- 172.21.15.119:0/334726351 wait complete. 2023-11-08T19:27:10.404 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:10.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.604+0000 7f6ca15d2640 1 Processor -- start 2023-11-08T19:27:10.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.604+0000 7f6ca15d2640 1 -- start start 2023-11-08T19:27:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6ca15d2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6c9c0d1d50 0x7f6c9c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6ca15d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c0c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6ca15d2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c9c0c7100 con 0x7f6c9c15d0e0 2023-11-08T19:27:10.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6ca15d2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9c0d04f0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6ca15d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0d07d0 con 0x7f6c9c15d4c0 2023-11-08T19:27:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6c9bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6c9c15d0e0 0x7f6c9c0cda70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46896/0 (socket says 172.21.15.119:46896) 2023-11-08T19:27:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6c9bfff640 1 -- 172.21.15.119:0/3762913331 learned_addr learned my addr 172.21.15.119:0/3762913331 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:10.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.606+0000 7f6c93fff640 1 --2- 172.21.15.119:0/3762913331 >> v2:172.21.15.119:3300/0 conn(0x7f6c9c0d1d50 0x7f6c9c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.607+0000 7f6ca0dd1640 1 --2- 172.21.15.119:0/3762913331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c0c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.607+0000 7f6c93fff640 1 -- 172.21.15.119:0/3762913331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 msgr2=0x7f6c9c0c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.607+0000 7f6c93fff640 1 --2- 172.21.15.119:0/3762913331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c0c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.607+0000 7f6c93fff640 1 -- 172.21.15.119:0/3762913331 >> v1:172.21.15.119:6789/0 conn(0x7f6c9c15d0e0 legacy=0x7f6c9c0cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.607+0000 7f6c93fff640 1 -- 172.21.15.119:0/3762913331 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0c6ab0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.607+0000 7f6c9b7fe640 1 -- 172.21.15.119:0/3762913331 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2281275256 0 0) 0x7f6c9c0c7100 con 0x7f6c9c15d0e0 2023-11-08T19:27:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.608+0000 7f6c93fff640 1 --2- 172.21.15.119:0/3762913331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c0ce180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097a0 tx=0x7f6c8402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68287d60d3de527a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.608+0000 7f6c9b7fe640 1 -- 172.21.15.119:0/3762913331 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84030610 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.608+0000 7f6c9b7fe640 1 -- 172.21.15.119:0/3762913331 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c84030770 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.608+0000 7f6c9b7fe640 1 -- 172.21.15.119:0/3762913331 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84030b00 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3762913331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 msgr2=0x7f6c9c0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3762913331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c0ce180 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f6c840097a0 tx=0x7f6c8402b960 comp rx=0 tx=0).stop 2023-11-08T19:27:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3762913331 shutdown_connections 2023-11-08T19:27:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3762913331 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c0c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3762913331 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c0ce180 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3762913331 >> 172.21.15.119:0/3762913331 conn(0x7f6c9c0d2f60 msgr2=0x7f6c9c0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3762913331 shutdown_connections 2023-11-08T19:27:10.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.609+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3762913331 wait complete. 2023-11-08T19:27:10.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.611+0000 7f6ca15d2640 1 Processor -- start 2023-11-08T19:27:10.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.611+0000 7f6ca15d2640 1 -- start start 2023-11-08T19:27:10.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.612+0000 7f6ca15d2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.612+0000 7f6ca15d2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c9c15d0e0 0x7f6c9c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.612+0000 7f6ca15d2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.612+0000 7f6ca15d2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c1f2ed0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.612+0000 7f6ca15d2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c1f28c0 con 0x7f6c9c15d0e0 2023-11-08T19:27:10.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.612+0000 7f6ca15d2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c0cb6c0 con 0x7f6c9c15d4c0 2023-11-08T19:27:10.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c1f3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49898/0 (socket says 172.21.15.119:49898) 2023-11-08T19:27:10.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 -- 172.21.15.119:0/3329849853 learned_addr learned my addr 172.21.15.119:0/3329849853 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:10.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c93fff640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c9c15d0e0 0x7f6c9c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 -- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 msgr2=0x7f6c9c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 -- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c9c15d0e0 msgr2=0x7f6c9c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c9c15d0e0 0x7f6c9c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 -- 172.21.15.119:0/3329849853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c84009400 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.613+0000 7f6c9bfff640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c1f3360 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00c4d0 tx=0x7f6c8c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.614+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c0074e0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.614+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8c007660 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.614+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c1f2b60 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.614+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8c004a20 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.614+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c0cb9a0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.615+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c04ece0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.616+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c8c004b80 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.617+0000 7f6c997fa640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06aea0 0x7f6c7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:10.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.617+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6c8c088d90 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.617+0000 7f6c93fff640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06aea0 0x7f6c7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:10.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.618+0000 7f6c93fff640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06aea0 0x7f6c7c06d360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6c84036000 tx=0x7f6c84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:10.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.621+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c8c053b90 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.850+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c9c066ae0 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.851+0000 7f6c997fa640 1 -- 172.21.15.119:0/3329849853 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f6c8c018020 con 0x7f6c9c0d1d50 2023-11-08T19:27:10.852 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:10.853 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:04.329003+0000","modified":"2023-11-08T19:27:07.395675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/976565354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":976565354},{"type":"v1","addr":"172.21.15.119:6837","nonce":976565354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:27:10.853 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-11-08T19:27:10.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.853+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06aea0 msgr2=0x7f6c7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.853+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06aea0 0x7f6c7c06d360 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6c84036000 tx=0x7f6c84008030 comp rx=0 tx=0).stop 2023-11-08T19:27:10.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.853+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 msgr2=0x7f6c9c1f3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:10.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.853+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c1f3360 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c00c4d0 tx=0x7f6c8c00c990 comp rx=0 tx=0).stop 2023-11-08T19:27:10.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.853+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 shutdown_connections 2023-11-08T19:27:10.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.853+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6c7c06aea0 0x7f6c7c06d360 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.854+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6c9c15d4c0 0x7f6c9c154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.854+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6c9c15d0e0 0x7f6c9c153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.854+0000 7f6ca15d2640 1 --2- 172.21.15.119:0/3329849853 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6c9c0d1d50 0x7f6c9c1f3360 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:10.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.854+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 >> 172.21.15.119:0/3329849853 conn(0x7f6c9c0d2f60 msgr2=0x7f6c9c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:10.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.854+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 shutdown_connections 2023-11-08T19:27:10.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:10.854+0000 7f6ca15d2640 1 -- 172.21.15.119:0/3329849853 wait complete. 2023-11-08T19:27:11.276 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:27:04.329003+0000', 'modified': '2023-11-08T19:27:07.395675+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5739}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5739': {'gid': 5739, 'name': 'c', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/976565354', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 976565354}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 976565354}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:27:11.277 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:27:11.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:27:11.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.502+0000 7f5d53ec9640 1 Processor -- start 2023-11-08T19:27:11.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.502+0000 7f5d53ec9640 1 -- start start 2023-11-08T19:27:11.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d53ec9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d4c142f80 0x7f5d4c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d53ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15d9f0 0x7f5d4c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d53ec9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4c0c4960 con 0x7f5d4c06cb40 2023-11-08T19:27:11.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d53ec9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4c143a40 con 0x7f5d4c142f80 2023-11-08T19:27:11.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d53ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c15df60 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d51c65640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d4c142f80 0x7f5d4c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d51c65640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d4c142f80 0x7f5d4c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49908/0 (socket says 172.21.15.119:49908) 2023-11-08T19:27:11.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d51c65640 1 -- 172.21.15.119:0/3436594997 learned_addr learned my addr 172.21.15.119:0/3436594997 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.504+0000 7f5d51464640 1 --2- 172.21.15.119:0/3436594997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15d9f0 0x7f5d4c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.505+0000 7f5d51c65640 1 -- 172.21.15.119:0/3436594997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15d9f0 msgr2=0x7f5d4c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.505+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3436594997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15d9f0 0x7f5d4c0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.505+0000 7f5d51c65640 1 -- 172.21.15.119:0/3436594997 >> v1:172.21.15.188:6789/0 conn(0x7f5d4c06cb40 legacy=0x7f5d4c0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.505+0000 7f5d51c65640 1 -- 172.21.15.119:0/3436594997 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c15e240 con 0x7f5d4c142f80 2023-11-08T19:27:11.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.506+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3436594997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c142f80 0x7f5d4c0cc6d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00d150 tx=0x7f5d3c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9e355347069899 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.506+0000 7f5d50c63640 1 -- 172.21.15.119:0/3436594997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c02e9e0 con 0x7f5d4c142f80 2023-11-08T19:27:11.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.506+0000 7f5d50c63640 1 -- 172.21.15.119:0/3436594997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c02eb40 con 0x7f5d4c142f80 2023-11-08T19:27:11.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.506+0000 7f5d50c63640 1 -- 172.21.15.119:0/3436594997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c009600 con 0x7f5d4c142f80 2023-11-08T19:27:11.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 -- 172.21.15.119:0/3436594997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c142f80 msgr2=0x7f5d4c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/3436594997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c142f80 0x7f5d4c0cc6d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c00d150 tx=0x7f5d3c02e040 comp rx=0 tx=0).stop 2023-11-08T19:27:11.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 -- 172.21.15.119:0/3436594997 shutdown_connections 2023-11-08T19:27:11.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/3436594997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c15d9f0 0x7f5d4c0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/3436594997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c142f80 0x7f5d4c0cc6d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 -- 172.21.15.119:0/3436594997 >> 172.21.15.119:0/3436594997 conn(0x7f5d4c0cb610 msgr2=0x7f5d4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 -- 172.21.15.119:0/3436594997 shutdown_connections 2023-11-08T19:27:11.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.507+0000 7f5d53ec9640 1 -- 172.21.15.119:0/3436594997 wait complete. 2023-11-08T19:27:11.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.509+0000 7f5d53ec9640 1 Processor -- start 2023-11-08T19:27:11.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.509+0000 7f5d53ec9640 1 -- start start 2023-11-08T19:27:11.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d53ec9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c06cb40 0x7f5d4c0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d53ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c142f80 0x7f5d4c0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d53ec9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 0x7f5d4c13d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d53ec9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0c63a0 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d53ec9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c1f3b20 con 0x7f5d4c06cb40 2023-11-08T19:27:11.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d53ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4c0c5d50 con 0x7f5d4c142f80 2023-11-08T19:27:11.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d52466640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 0x7f5d4c13d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d52466640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 0x7f5d4c13d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49918/0 (socket says 172.21.15.119:49918) 2023-11-08T19:27:11.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.510+0000 7f5d52466640 1 -- 172.21.15.119:0/1708822234 learned_addr learned my addr 172.21.15.119:0/1708822234 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d51464640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c142f80 0x7f5d4c0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d51c65640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c06cb40 0x7f5d4c0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d52466640 1 -- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c142f80 msgr2=0x7f5d4c0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d52466640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c142f80 0x7f5d4c0b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d52466640 1 -- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c06cb40 msgr2=0x7f5d4c0b1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d52466640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c06cb40 0x7f5d4c0b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d52466640 1 -- 172.21.15.119:0/1708822234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c00bd40 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d52466640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 0x7f5d4c13d980 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400b410 tx=0x7f5d4400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d4a7fc640 1 -- 172.21.15.119:0/1708822234 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44004020 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d4a7fc640 1 -- 172.21.15.119:0/1708822234 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44004180 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d4a7fc640 1 -- 172.21.15.119:0/1708822234 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44011630 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4c0c6030 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.511+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c0b4670 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.515+0000 7f5d4a7fc640 1 -- 172.21.15.119:0/1708822234 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d44011790 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.515+0000 7f5d4a7fc640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806aea0 0x7f5d3806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.516+0000 7f5d4a7fc640 1 -- 172.21.15.119:0/1708822234 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5d440893a0 con 0x7f5d4c15d9f0 2023-11-08T19:27:11.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.516+0000 7f5d51c65640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806aea0 0x7f5d3806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.516+0000 7f5d56a57740 1 Processor -- start 2023-11-08T19:27:11.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.516+0000 7f5d56a57740 1 -- start start 2023-11-08T19:27:11.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.516+0000 7f5d51c65640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806aea0 0x7f5d3806d360 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c037000 tx=0x7f5d3c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0eec0 0x55c8c59e1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c5a11c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c59e2300 0x55c8c5a12190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d56a57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c59547e0 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5951ed0 con 0x55c8c59e2300 2023-11-08T19:27:11.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8c594d7c0 con 0x55c8c5a0eec0 2023-11-08T19:27:11.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d51464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c5a11c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d51464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c5a11c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49924/0 (socket says 172.21.15.119:49924) 2023-11-08T19:27:11.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.517+0000 7f5d51464640 1 -- 172.21.15.119:0/3297844288 learned_addr learned my addr 172.21.15.119:0/3297844288 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0eec0 0x55c8c59e1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51464640 1 -- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0eec0 msgr2=0x55c8c59e1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51464640 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0eec0 0x55c8c59e1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51464640 1 -- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c59e2300 msgr2=0x55c8c5a12190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51464640 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c59e2300 0x55c8c5a12190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51464640 1 -- 172.21.15.119:0/3297844288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c5920b10 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d51464640 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c5a11c50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009910 tx=0x7f5d40009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe83149025957a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d337fe640 1 -- 172.21.15.119:0/3297844288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000c070 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d337fe640 1 -- 172.21.15.119:0/3297844288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4000ea10 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.518+0000 7f5d337fe640 1 -- 172.21.15.119:0/3297844288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000edc0 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 -- 172.21.15.119:0/3297844288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 msgr2=0x55c8c5a11c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c5a11c50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009910 tx=0x7f5d40009dd0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 -- 172.21.15.119:0/3297844288 shutdown_connections 2023-11-08T19:27:11.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c59e2300 0x55c8c5a12190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c5a11c50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3297844288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0eec0 0x55c8c59e1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 -- 172.21.15.119:0/3297844288 >> 172.21.15.119:0/3297844288 conn(0x55c8c59e40d0 msgr2=0x55c8c5a2cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 -- 172.21.15.119:0/3297844288 shutdown_connections 2023-11-08T19:27:11.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.519+0000 7f5d56a57740 1 -- 172.21.15.119:0/3297844288 wait complete. 2023-11-08T19:27:11.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.520+0000 7f5d56a57740 1 -- start start 2023-11-08T19:27:11.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.520+0000 7f5d56a57740 1 Processor -- start 2023-11-08T19:27:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c59f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a15680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c58eac40 con 0x55c8c5a0eec0 2023-11-08T19:27:11.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c58ea8e0 con 0x55c8c5a09960 2023-11-08T19:27:11.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8c56b0f20 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d56a57740 10 client.0 _ll_register_callbacks cb 0x55c8c590ccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:11.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d51c65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c59f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d51c65640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c59f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49936/0 (socket says 172.21.15.119:49936) 2023-11-08T19:27:11.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.521+0000 7f5d51c65640 1 -- 172.21.15.119:0/3256603093 learned_addr learned my addr 172.21.15.119:0/3256603093 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51464640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a15680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51c65640 1 -- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 msgr2=0x55c8c5a15680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a15680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d52466640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51c65640 1 -- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 msgr2=0x55c8c5a07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a07a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51c65640 1 -- 172.21.15.119:0/3256603093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d40009510 con 0x55c8c5a0eec0 2023-11-08T19:27:11.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51464640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a15680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:11.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c59f9080 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c02e650 tx=0x7f5d3c030070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65dcb17b60cc531 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:27:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3256603093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c046050 con 0x55c8c5a0eec0 2023-11-08T19:27:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3256603093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5d3c04c660 con 0x55c8c5a0eec0 2023-11-08T19:27:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.522+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8c5921c80 con 0x55c8c5a0eec0 2023-11-08T19:27:11.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3256603093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d3c04c820 con 0x55c8c5a0eec0 2023-11-08T19:27:11.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 10 client.5877 fetch_fsmap learned FSMap version 26 2023-11-08T19:27:11.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55c8c594be40 con 0x55c8c5a0eec0 2023-11-08T19:27:11.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3256603093 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c8c5921c80 con 0x55c8c5a0eec0 2023-11-08T19:27:11.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3256603093 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d3c04c980 con 0x55c8c5a0eec0 2023-11-08T19:27:11.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 10 client.5877 fetch_fsmap finished waiting for FSMap version 26 2023-11-08T19:27:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 10 client.5877 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:27:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 20 client.5877 populate_metadata read hostname 'smithi119' 2023-11-08T19:27:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a402f0 0x55c8c5a406d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 4 client.5877 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:27:11.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 --> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8c59467a0 con 0x55c8c5a402f0 2023-11-08T19:27:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.523+0000 7f5d51464640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a402f0 0x55c8c5a406d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.524+0000 7f5d30ff9640 10 client.5877 ms_handle_connect on v2:172.21.15.119:6835/976565354 2023-11-08T19:27:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.524+0000 7f5d51464640 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a402f0 0x55c8c5a406d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e1595701dcb9ea server_cookie=13aac04e8fc41804 in_seq=0 out_seq=0 2023-11-08T19:27:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3256603093 <== mds.0 v2:172.21.15.119:6835/976565354 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c8c59467a0 con 0x55c8c5a402f0 2023-11-08T19:27:11.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d30ff9640 10 client.5877 handle_command_reply: tid=0 2023-11-08T19:27:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 client.5877 shutdown 2023-11-08T19:27:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 msgr2=0x55c8c59f9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c59f9080 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c02e650 tx=0x7f5d3c030070 comp rx=0 tx=0).stop 2023-11-08T19:27:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 shutdown_connections 2023-11-08T19:27:11.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a402f0 0x55c8c5a406d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a15680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3256603093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c59f9080 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d30ff9640 0 client.5877 ms_handle_reset on v2:172.21.15.119:6835/976565354 2023-11-08T19:27:11.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 >> 172.21.15.119:0/3256603093 conn(0x55c8c59e2300 msgr2=0x55c8c59e5d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 shutdown_connections 2023-11-08T19:27:11.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 1 -- 172.21.15.119:0/3256603093 wait complete. 2023-11-08T19:27:11.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.525+0000 7f5d56a57740 20 client.5877 trim_cache size 0 max 16384 2023-11-08T19:27:11.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.543+0000 7f5d56a57740 1 Processor -- start 2023-11-08T19:27:11.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.543+0000 7f5d56a57740 1 -- start start 2023-11-08T19:27:11.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a09960 0x55c8c5a30360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a91ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c59f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d56a57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5a2d9f0 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5a06e60 con 0x55c8c5a0eec0 2023-11-08T19:27:11.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5a0e390 con 0x55c8c5a09960 2023-11-08T19:27:11.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c59f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c59f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49940/0 (socket says 172.21.15.119:49940) 2023-11-08T19:27:11.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 -- 172.21.15.119:0/2380610351 learned_addr learned my addr 172.21.15.119:0/2380610351 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d51464640 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a91ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d51c65640 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a09960 0x55c8c5a30360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 -- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a09960 msgr2=0x55c8c5a30360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a09960 0x55c8c5a30360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 -- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a0eec0 msgr2=0x55c8c5a91ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a91ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d52466640 1 -- 172.21.15.119:0/2380610351 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8c58e9b90 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.544+0000 7f5d51c65640 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a09960 0x55c8c5a30360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:11.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.545+0000 7f5d52466640 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c59f4ee0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5d44057b60 tx=0x7f5d44057ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c6d155149abd59 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.545+0000 7f5d33fff640 1 -- 172.21.15.119:0/2380610351 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4405e0c0 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.545+0000 7f5d33fff640 1 -- 172.21.15.119:0/2380610351 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44060160 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.545+0000 7f5d33fff640 1 -- 172.21.15.119:0/2380610351 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44056200 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 -- 172.21.15.119:0/2380610351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 msgr2=0x55c8c59f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c59f4ee0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5d44057b60 tx=0x7f5d44057ad0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 -- 172.21.15.119:0/2380610351 shutdown_connections 2023-11-08T19:27:11.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0f2a0 0x55c8c59f4ee0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a91ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 --2- 172.21.15.119:0/2380610351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a09960 0x55c8c5a30360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 -- 172.21.15.119:0/2380610351 >> 172.21.15.119:0/2380610351 conn(0x55c8c59e2300 msgr2=0x55c8c5a47af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 -- 172.21.15.119:0/2380610351 shutdown_connections 2023-11-08T19:27:11.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 -- 172.21.15.119:0/2380610351 wait complete. 2023-11-08T19:27:11.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.546+0000 7f5d56a57740 1 -- start start 2023-11-08T19:27:11.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.547+0000 7f5d56a57740 1 Processor -- start 2023-11-08T19:27:11.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a90430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a47030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a07370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5a993e0 con 0x55c8c5a0eec0 2023-11-08T19:27:11.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5a476e0 con 0x55c8c5a09960 2023-11-08T19:27:11.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c8c5a47930 con 0x55c8c5a0f2a0 2023-11-08T19:27:11.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d56a57740 10 client.0 _ll_register_callbacks cb 0x7f5d3c00d050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:11.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a47030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a47030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49954/0 (socket says 172.21.15.119:49954) 2023-11-08T19:27:11.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 -- 172.21.15.119:0/3671577784 learned_addr learned my addr 172.21.15.119:0/3671577784 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 -- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 msgr2=0x55c8c5a07370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a90430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a07370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 -- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 msgr2=0x55c8c5a90430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a90430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.548+0000 7f5d51464640 1 -- 172.21.15.119:0/3671577784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c0c6030 con 0x55c8c5a0eec0 2023-11-08T19:27:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d51464640 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a47030 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5d40016040 tx=0x7f5d40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdafa586b67dc461 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:27:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3671577784 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40022040 con 0x55c8c5a0eec0 2023-11-08T19:27:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3671577784 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5d40021040 con 0x55c8c5a0eec0 2023-11-08T19:27:11.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5d4000ea10 con 0x55c8c5a0eec0 2023-11-08T19:27:11.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3671577784 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d4000ed90 con 0x55c8c5a0eec0 2023-11-08T19:27:11.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d56a57740 10 client.5883 fetch_fsmap learned FSMap version 26 2023-11-08T19:27:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55c8c590f180 con 0x55c8c5a0eec0 2023-11-08T19:27:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.549+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3671577784 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4000ea10 con 0x55c8c5a0eec0 2023-11-08T19:27:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3671577784 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 26) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5d40027440 con 0x55c8c5a0eec0 2023-11-08T19:27:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d56a57740 10 client.5883 fetch_fsmap finished waiting for FSMap version 26 2023-11-08T19:27:11.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d56a57740 10 client.5883 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:27:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d56a57740 20 client.5883 populate_metadata read hostname 'smithi119' 2023-11-08T19:27:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a98540 0x55c8c5a98920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d56a57740 4 client.5883 mds_command: new command op to 5739 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:27:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 --> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c8c59464c0 con 0x55c8c5a98540 2023-11-08T19:27:11.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a98540 0x55c8c5a98920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d51c65640 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a98540 0x55c8c5a98920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54c8036adb921b7e server_cookie=cc475fd992b16366 in_seq=0 out_seq=0 2023-11-08T19:27:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.550+0000 7f5d30ff9640 10 client.5883 ms_handle_connect on v2:172.21.15.119:6835/976565354 2023-11-08T19:27:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d30ff9640 1 -- 172.21.15.119:0/3671577784 <== mds.0 v2:172.21.15.119:6835/976565354 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55c8c59464c0 con 0x55c8c5a98540 2023-11-08T19:27:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d30ff9640 10 client.5883 handle_command_reply: tid=0 2023-11-08T19:27:11.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 client.5883 shutdown 2023-11-08T19:27:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 msgr2=0x55c8c5a47030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a47030 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f5d40016040 tx=0x7f5d40004730 comp rx=0 tx=0).stop 2023-11-08T19:27:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 shutdown_connections 2023-11-08T19:27:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:6835/976565354,v1:172.21.15.119:6837/976565354] conn(0x55c8c5a98540 0x55c8c5a98920 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55c8c5a0f2a0 0x55c8c5a07370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55c8c5a0eec0 0x55c8c5a47030 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d30ff9640 0 client.5883 ms_handle_reset on v2:172.21.15.119:6835/976565354 2023-11-08T19:27:11.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 --2- 172.21.15.119:0/3671577784 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55c8c5a09960 0x55c8c5a90430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 >> 172.21.15.119:0/3671577784 conn(0x55c8c59e2300 msgr2=0x55c8c59e7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 shutdown_connections 2023-11-08T19:27:11.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 1 -- 172.21.15.119:0/3671577784 wait complete. 2023-11-08T19:27:11.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.551+0000 7f5d56a57740 20 client.5883 trim_cache size 0 max 16384 2023-11-08T19:27:11.566 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:27:11.567 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:27:11.567 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:27:11.567 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5739, 2023-11-08T19:27:11.567 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:27:11.567 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:27:11.568 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:27:11.568 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.6512658350000002, 2023-11-08T19:27:11.568 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 26, 2023-11-08T19:27:11.568 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 54, 2023-11-08T19:27:11.568 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 54, 2023-11-08T19:27:11.569 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 13.069539354 2023-11-08T19:27:11.569 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:27:11.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806aea0 msgr2=0x7f5d3806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806aea0 0x7f5d3806d360 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c037000 tx=0x7f5d3c032000 comp rx=0 tx=0).stop 2023-11-08T19:27:11.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 msgr2=0x7f5d4c13d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 0x7f5d4c13d980 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400b410 tx=0x7f5d4400b8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 shutdown_connections 2023-11-08T19:27:11.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806aea0 0x7f5d3806d360 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d4c15d9f0 0x7f5d4c13d980 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d4c142f80 0x7f5d4c0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 --2- 172.21.15.119:0/1708822234 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d4c06cb40 0x7f5d4c0b1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 >> 172.21.15.119:0/1708822234 conn(0x7f5d4c0cb610 msgr2=0x7f5d4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 shutdown_connections 2023-11-08T19:27:11.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.552+0000 7f5d53ec9640 1 -- 172.21.15.119:0/1708822234 wait complete. 2023-11-08T19:27:11.573 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:27:11.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.777+0000 7fc3744e7640 1 Processor -- start 2023-11-08T19:27:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.777+0000 7fc3744e7640 1 -- start start 2023-11-08T19:27:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc3744e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc36c1447a0 0x7fc36c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc3744e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc36c0cad30 0x7fc36c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc3744e7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc36c0c3a50 con 0x7fc36c06a120 2023-11-08T19:27:11.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc3744e7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c144b80 con 0x7fc36c1447a0 2023-11-08T19:27:11.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc3744e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc36c0c9150 con 0x7fc36c0cad30 2023-11-08T19:27:11.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc372a84640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc36c06a120 0x7fc36c06a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46906/0 (socket says 172.21.15.119:46906) 2023-11-08T19:27:11.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc372a84640 1 -- 172.21.15.119:0/2529931647 learned_addr learned my addr 172.21.15.119:0/2529931647 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:11.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.779+0000 7fc372283640 1 --2- 172.21.15.119:0/2529931647 >> v2:172.21.15.119:3300/0 conn(0x7fc36c1447a0 0x7fc36c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.780+0000 7fc371a82640 1 --2- 172.21.15.119:0/2529931647 >> v2:172.21.15.188:3300/0 conn(0x7fc36c0cad30 0x7fc36c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.780+0000 7fc372283640 1 -- 172.21.15.119:0/2529931647 >> v2:172.21.15.188:3300/0 conn(0x7fc36c0cad30 msgr2=0x7fc36c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:11.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.780+0000 7fc372283640 1 --2- 172.21.15.119:0/2529931647 >> v2:172.21.15.188:3300/0 conn(0x7fc36c0cad30 0x7fc36c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.780+0000 7fc372283640 1 -- 172.21.15.119:0/2529931647 >> v1:172.21.15.119:6789/0 conn(0x7fc36c06a120 legacy=0x7fc36c06a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.780+0000 7fc372283640 1 -- 172.21.15.119:0/2529931647 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc36c0c9410 con 0x7fc36c1447a0 2023-11-08T19:27:11.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.780+0000 7fc371281640 1 -- 172.21.15.119:0/2529931647 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1909987504 0 0) 0x7fc36c0c3a50 con 0x7fc36c06a120 2023-11-08T19:27:11.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.781+0000 7fc372283640 1 --2- 172.21.15.119:0/2529931647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c1447a0 0x7fc36c15f4c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc36c06df20 tx=0x7fc36802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b41da7d13f9d9b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.781+0000 7fc371281640 1 -- 172.21.15.119:0/2529931647 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368031830 con 0x7fc36c1447a0 2023-11-08T19:27:11.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.781+0000 7fc371281640 1 -- 172.21.15.119:0/2529931647 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc368031990 con 0x7fc36c1447a0 2023-11-08T19:27:11.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.781+0000 7fc371281640 1 -- 172.21.15.119:0/2529931647 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc368031d20 con 0x7fc36c1447a0 2023-11-08T19:27:11.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 -- 172.21.15.119:0/2529931647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c1447a0 msgr2=0x7fc36c15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2529931647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c1447a0 0x7fc36c15f4c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc36c06df20 tx=0x7fc36802b8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 -- 172.21.15.119:0/2529931647 shutdown_connections 2023-11-08T19:27:11.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2529931647 >> v2:172.21.15.188:3300/0 conn(0x7fc36c0cad30 0x7fc36c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2529931647 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c1447a0 0x7fc36c15f4c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 -- 172.21.15.119:0/2529931647 >> 172.21.15.119:0/2529931647 conn(0x7fc36c0cb610 msgr2=0x7fc36c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:11.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 -- 172.21.15.119:0/2529931647 shutdown_connections 2023-11-08T19:27:11.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.782+0000 7fc3744e7640 1 -- 172.21.15.119:0/2529931647 wait complete. 2023-11-08T19:27:11.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.784+0000 7fc3744e7640 1 Processor -- start 2023-11-08T19:27:11.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.784+0000 7fc3744e7640 1 -- start start 2023-11-08T19:27:11.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc3744e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 0x7fc36c1334a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc3744e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc36c0cad30 0x7fc36c1339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc3744e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc36c1447a0 0x7fc36c13ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc3744e7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c142d30 con 0x7fc36c06a120 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc3744e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c143010 con 0x7fc36c0cad30 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc3744e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc36c0b2aa0 con 0x7fc36c1447a0 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc372283640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 0x7fc36c1334a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc372283640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 0x7fc36c1334a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49966/0 (socket says 172.21.15.119:49966) 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.785+0000 7fc372283640 1 -- 172.21.15.119:0/2615642584 learned_addr learned my addr 172.21.15.119:0/2615642584 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc371a82640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc36c0cad30 0x7fc36c1339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372283640 1 -- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc36c1447a0 msgr2=0x7fc36c13ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372a84640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc36c1447a0 0x7fc36c13ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372283640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc36c1447a0 0x7fc36c13ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372283640 1 -- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc36c0cad30 msgr2=0x7fc36c1339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372283640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc36c0cad30 0x7fc36c1339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:11.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372283640 1 -- 172.21.15.119:0/2615642584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368009400 con 0x7fc36c06a120 2023-11-08T19:27:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc371a82640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc36c0cad30 0x7fc36c1339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.786+0000 7fc372283640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 0x7fc36c1334a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc368000c00 tx=0x7fc3680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.787+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc36803aa60 con 0x7fc36c06a120 2023-11-08T19:27:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.787+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc36803abe0 con 0x7fc36c06a120 2023-11-08T19:27:11.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.787+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc36c0b2d80 con 0x7fc36c06a120 2023-11-08T19:27:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.787+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3680443f0 con 0x7fc36c06a120 2023-11-08T19:27:11.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.787+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc36c131470 con 0x7fc36c06a120 2023-11-08T19:27:11.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.788+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc36c074430 con 0x7fc36c06a120 2023-11-08T19:27:11.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.789+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc36804d450 con 0x7fc36c06a120 2023-11-08T19:27:11.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.790+0000 7fc362ffd640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc35006ae70 0x7fc35006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:11.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.790+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc3680ab9b0 con 0x7fc36c06a120 2023-11-08T19:27:11.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.790+0000 7fc371a82640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc35006ae70 0x7fc35006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:11.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.791+0000 7fc371a82640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc35006ae70 0x7fc35006d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc36c1f63a0 tx=0x7fc35c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:11.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:11.794+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc368076850 con 0x7fc36c06a120 2023-11-08T19:27:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.022+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc36c06caa0 con 0x7fc36c06a120 2023-11-08T19:27:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.023+0000 7fc362ffd640 1 -- 172.21.15.119:0/2615642584 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc368035070 con 0x7fc36c06a120 2023-11-08T19:27:12.025 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:27:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc35006ae70 msgr2=0x7fc35006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc35006ae70 0x7fc35006d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc36c1f63a0 tx=0x7fc35c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 msgr2=0x7fc36c1334a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:12.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 0x7fc36c1334a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc368000c00 tx=0x7fc3680046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 shutdown_connections 2023-11-08T19:27:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc35006ae70 0x7fc35006d330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc36c1447a0 0x7fc36c13ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc36c0cad30 0x7fc36c1339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 --2- 172.21.15.119:0/2615642584 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc36c06a120 0x7fc36c1334a0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.025+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 >> 172.21.15.119:0/2615642584 conn(0x7fc36c0cb610 msgr2=0x7fc36c0c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.026+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 shutdown_connections 2023-11-08T19:27:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:12.026+0000 7fc3744e7640 1 -- 172.21.15.119:0/2615642584 wait complete. 2023-11-08T19:27:12.045 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:27:12.045 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:27:12.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:27:12.046 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:27:12.046 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:12.047 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:27:12.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:27:12.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:27:12.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:12.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.048 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:27:12.099 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:27:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:27:12.099 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:27:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:27:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.100 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:12.100 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:27:12.101 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:12.101 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:27:12.102 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:12.102 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:27:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:12.102 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:27:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42238sec preferred_lft 42238sec 2023-11-08T19:27:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:27:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.103 INFO:teuthology.orchestra.run.smithi119.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:27:12.103 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:27:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::e06c:64ff:fe55:3f8b/64 scope link 2023-11-08T19:27:12.104 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.104 INFO:teuthology.orchestra.run.smithi119.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:27:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c086:39ff:fe1e:9142/64 scope link 2023-11-08T19:27:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.105 INFO:teuthology.orchestra.run.smithi119.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:12.105 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:27:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::494:cbff:fed6:d560/64 scope link 2023-11-08T19:27:12.106 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:12.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.107 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:27:12.194 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:27:12.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.198 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:27:12.254 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:27:12.255 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:27:12.255 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:27:12.255 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:27:12.255 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:27:12.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.256 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:27:12.257 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:27:12.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:27:12.257 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:27:12.258 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:27:12.258 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:27:12.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.441 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:27:12.496 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:27:12.496 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:27:12.498 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:12.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.499 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:27:12.593 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:12.594 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:12.650 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:12.650 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:12.651 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:12.703 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T19:27:12.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.704 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:27:12.836 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:27:12.837 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:27:12.872 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:27:12.872 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:27:12.872 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:27:12.873 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:27:12.873 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:27:12.874 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-08T19:27:12.874 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:27:12.874 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:27:12.875 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:27:12.875 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:27:12.879 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:27:12.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.880 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:12.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:12.965 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:27:13.029 INFO:teuthology.orchestra.run.smithi119.stdout:client5077 2023-11-08T19:27:13.029 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:13.030 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:13.083 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data mksnap snap1 2023-11-08T19:27:13.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.233+0000 7f48a0251f40 1 Processor -- start 2023-11-08T19:27:13.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.233+0000 7f48a0251f40 1 -- start start 2023-11-08T19:27:13.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.234+0000 7f48a0251f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec843f40 0x562dec90d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.234+0000 7f48a0251f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec872f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.234+0000 7f48a0251f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 0x562dec90f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.234+0000 7f48a0251f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec7f1340 con 0x562dec88a6b0 2023-11-08T19:27:13.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.234+0000 7f48a0251f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec78ba70 con 0x562dec843f40 2023-11-08T19:27:13.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.234+0000 7f48a0251f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dec78b710 con 0x562dec8895c0 2023-11-08T19:27:13.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 0x562dec90f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 0x562dec90f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49980/0 (socket says 172.21.15.119:49980) 2023-11-08T19:27:13.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 -- 172.21.15.119:0/18858413 learned_addr learned my addr 172.21.15.119:0/18858413 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:13.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489dd53640 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec843f40 0x562dec90d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489d552640 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec872f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 -- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 msgr2=0x562dec872f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec872f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 -- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec843f40 msgr2=0x562dec90d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec843f40 0x562dec90d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 -- 172.21.15.119:0/18858413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898005d20 con 0x562dec88a6b0 2023-11-08T19:27:13.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489d552640 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec872f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:13.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.235+0000 7f489e554640 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 0x562dec90f060 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4898009920 tx=0x7f4898005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e34d6eb854893e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f4897fff640 1 -- 172.21.15.119:0/18858413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800f040 con 0x562dec88a6b0 2023-11-08T19:27:13.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f4897fff640 1 -- 172.21.15.119:0/18858413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48980089b0 con 0x562dec88a6b0 2023-11-08T19:27:13.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f4897fff640 1 -- 172.21.15.119:0/18858413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898008d60 con 0x562dec88a6b0 2023-11-08T19:27:13.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 -- 172.21.15.119:0/18858413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 msgr2=0x562dec90f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 0x562dec90f060 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4898009920 tx=0x7f4898005a00 comp rx=0 tx=0).stop 2023-11-08T19:27:13.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 -- 172.21.15.119:0/18858413 shutdown_connections 2023-11-08T19:27:13.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec88a6b0 0x562dec90f060 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec872f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 --2- 172.21.15.119:0/18858413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec843f40 0x562dec90d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 -- 172.21.15.119:0/18858413 >> 172.21.15.119:0/18858413 conn(0x562dec840760 msgr2=0x562dec88dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 -- 172.21.15.119:0/18858413 shutdown_connections 2023-11-08T19:27:13.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.236+0000 7f48a0251f40 1 -- 172.21.15.119:0/18858413 wait complete. 2023-11-08T19:27:13.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.237+0000 7f48a0251f40 1 Processor -- start 2023-11-08T19:27:13.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.237+0000 7f48a0251f40 1 -- start start 2023-11-08T19:27:13.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.238+0000 7f48a0251f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 0x562dec887500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.238+0000 7f48a0251f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec9015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.238+0000 7f48a0251f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec88a6b0 0x562dec901b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.238+0000 7f48a0251f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec77f520 con 0x562dec843f40 2023-11-08T19:27:13.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.238+0000 7f48a0251f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x562dec902040 con 0x562dec88a6b0 2023-11-08T19:27:13.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.238+0000 7f48a0251f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x562dec919570 con 0x562dec8895c0 2023-11-08T19:27:13.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 0x562dec887500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 0x562dec887500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49990/0 (socket says 172.21.15.119:49990) 2023-11-08T19:27:13.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 -- 172.21.15.119:0/876351180 learned_addr learned my addr 172.21.15.119:0/876351180 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:13.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489e554640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec88a6b0 0x562dec901b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489d552640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec9015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 -- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 msgr2=0x562dec9015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec9015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 -- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec88a6b0 msgr2=0x562dec901b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec88a6b0 0x562dec901b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 -- 172.21.15.119:0/876351180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48980095a0 con 0x562dec843f40 2023-11-08T19:27:13.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489e554640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec88a6b0 0x562dec901b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:13.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.239+0000 7f489dd53640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 0x562dec887500 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f489000a4b0 tx=0x7f4890003330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.240+0000 7f4896ffd640 1 -- 172.21.15.119:0/876351180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489005d5f0 con 0x562dec843f40 2023-11-08T19:27:13.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.240+0000 7f4896ffd640 1 -- 172.21.15.119:0/876351180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489005d750 con 0x562dec843f40 2023-11-08T19:27:13.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.240+0000 7f4896ffd640 1 -- 172.21.15.119:0/876351180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489005db00 con 0x562dec843f40 2023-11-08T19:27:13.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.240+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489000a1c0 con 0x562dec843f40 2023-11-08T19:27:13.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.240+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562dec797680 con 0x562dec843f40 2023-11-08T19:27:13.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.242+0000 7f4896ffd640 1 -- 172.21.15.119:0/876351180 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f489005dc60 con 0x562dec843f40 2023-11-08T19:27:13.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.243+0000 7f4896ffd640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4880069820 0x7f488006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.243+0000 7f4896ffd640 1 -- 172.21.15.119:0/876351180 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f48900ca6b0 con 0x562dec843f40 2023-11-08T19:27:13.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.243+0000 7f489d552640 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4880069820 0x7f488006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.243+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- pool_op(create snap pool 9 tid 1 name snap1 v0) v4 -- 0x562dec7fb2c0 con 0x562dec843f40 2023-11-08T19:27:13.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.243+0000 7f4896ffd640 1 -- 172.21.15.119:0/876351180 <== mon.0 v2:172.21.15.119:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v54) v1 ==== 43+0+0 (secure 0 0 0) 0x7f48900955d0 con 0x562dec843f40 2023-11-08T19:27:13.254 INFO:teuthology.orchestra.run.smithi119.stderr:error creating pool cephfs_data snapshot snap1: (95) Operation not supported 2023-11-08T19:27:13.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4880069820 msgr2=0x7f488006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4880069820 0x7f488006bce0 secure :-1 s=COMPRESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x562dec9025e0 tx=0x7f4888001460 comp rx=0 tx=0).stop 2023-11-08T19:27:13.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 msgr2=0x562dec887500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 0x562dec887500 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f489000a4b0 tx=0x7f4890003330 comp rx=0 tx=0).stop 2023-11-08T19:27:13.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 shutdown_connections 2023-11-08T19:27:13.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4880069820 0x7f488006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x562dec88a6b0 0x562dec901b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x562dec8895c0 0x562dec9015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 --2- 172.21.15.119:0/876351180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x562dec843f40 0x562dec887500 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 >> 172.21.15.119:0/876351180 conn(0x562dec840760 msgr2=0x562dec91a840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 shutdown_connections 2023-11-08T19:27:13.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.244+0000 7f48a0251f40 1 -- 172.21.15.119:0/876351180 wait complete. 2023-11-08T19:27:13.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:13.258 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata mksnap snap2 2023-11-08T19:27:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.364+0000 7f3bc9b30f40 1 Processor -- start 2023-11-08T19:27:13.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.364+0000 7f3bc9b30f40 1 -- start start 2023-11-08T19:27:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bc9b30f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc672e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bc9b30f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc67820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bc9b30f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 0x55e4cfd057f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bc9b30f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cfbe8340 con 0x55e4cfd16950 2023-11-08T19:27:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bc9b30f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cfb82a70 con 0x55e4cfd16570 2023-11-08T19:27:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bc9b30f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4cfb82710 con 0x55e4cfc67d60 2023-11-08T19:27:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc67820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc67820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50014/0 (socket says 172.21.15.119:50014) 2023-11-08T19:27:13.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 -- 172.21.15.119:0/1775326894 learned_addr learned my addr 172.21.15.119:0/1775326894 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:13.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 -- 172.21.15.119:0/1775326894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 msgr2=0x55e4cfd057f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 0x55e4cfd057f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 -- 172.21.15.119:0/1775326894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 msgr2=0x55e4cfc672e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc672e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.365+0000 7f3bbffff640 1 -- 172.21.15.119:0/1775326894 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4cfb8e680 con 0x55e4cfd16950 2023-11-08T19:27:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bbffff640 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc67820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009840 tx=0x7f3bb0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f7d6f76ea63628 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc6630640 1 -- 172.21.15.119:0/1775326894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000f050 con 0x55e4cfd16950 2023-11-08T19:27:13.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc6630640 1 -- 172.21.15.119:0/1775326894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0010040 con 0x55e4cfd16950 2023-11-08T19:27:13.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc6630640 1 -- 172.21.15.119:0/1775326894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0008af0 con 0x55e4cfd16950 2023-11-08T19:27:13.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/1775326894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 msgr2=0x55e4cfc67820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc67820 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009840 tx=0x7f3bb0005a00 comp rx=0 tx=0).stop 2023-11-08T19:27:13.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/1775326894 shutdown_connections 2023-11-08T19:27:13.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 0x55e4cfd057f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc67820 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/1775326894 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc672e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.366+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/1775326894 >> 172.21.15.119:0/1775326894 conn(0x55e4cfc38a30 msgr2=0x55e4cfc7d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.367+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/1775326894 shutdown_connections 2023-11-08T19:27:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.367+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/1775326894 wait complete. 2023-11-08T19:27:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.367+0000 7f3bc9b30f40 1 Processor -- start 2023-11-08T19:27:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.368+0000 7f3bc9b30f40 1 -- start start 2023-11-08T19:27:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.368+0000 7f3bc9b30f40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 0x55e4cfd04d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.368+0000 7f3bc9b30f40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc69e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc9b30f40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc6a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc9b30f40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cfb76520 con 0x55e4cfd16950 2023-11-08T19:27:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc9b30f40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4cfd15d80 con 0x55e4cfd16570 2023-11-08T19:27:13.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc9b30f40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e4cfd15fd0 con 0x55e4cfc67d60 2023-11-08T19:27:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc6a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc6a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50026/0 (socket says 172.21.15.119:50026) 2023-11-08T19:27:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 -- 172.21.15.119:0/475893691 learned_addr learned my addr 172.21.15.119:0/475893691 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bbffff640 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc69e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 -- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 msgr2=0x55e4cfd04d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 0x55e4cfd04d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 -- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 msgr2=0x55e4cfc69e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc69e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.369+0000 7f3bc7e33640 1 -- 172.21.15.119:0/475893691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb0009410 con 0x55e4cfd16950 2023-11-08T19:27:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.370+0000 7f3bc7e33640 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc6a3b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00447a0 tx=0x7f3bc0055e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.370+0000 7f3bc562e640 1 -- 172.21.15.119:0/475893691 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc005a070 con 0x55e4cfd16950 2023-11-08T19:27:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.370+0000 7f3bc562e640 1 -- 172.21.15.119:0/475893691 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc0057e90 con 0x55e4cfd16950 2023-11-08T19:27:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.370+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e4cfb81830 con 0x55e4cfd16950 2023-11-08T19:27:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.370+0000 7f3bc562e640 1 -- 172.21.15.119:0/475893691 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0005ae0 con 0x55e4cfd16950 2023-11-08T19:27:13.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.370+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4cfb81290 con 0x55e4cfd16950 2023-11-08T19:27:13.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.372+0000 7f3bc562e640 1 -- 172.21.15.119:0/475893691 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3bc0055460 con 0x55e4cfd16950 2023-11-08T19:27:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.373+0000 7f3bc562e640 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3ba8069820 0x7f3ba806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.373+0000 7f3bc562e640 1 -- 172.21.15.119:0/475893691 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3bc00c85f0 con 0x55e4cfd16950 2023-11-08T19:27:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.373+0000 7f3bc7632640 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3ba8069820 0x7f3ba806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.373+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- pool_op(create snap pool 8 tid 1 name snap2 v0) v4 -- 0x55e4cfbf22c0 con 0x55e4cfd16950 2023-11-08T19:27:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc562e640 1 -- 172.21.15.119:0/475893691 <== mon.0 v2:172.21.15.119:3300/0 6 ==== pool_op_reply(tid 1 (95) Operation not supported v54) v1 ==== 43+0+0 (secure 0 0 0) 0x7f3bc0093490 con 0x55e4cfd16950 2023-11-08T19:27:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:error creating pool cephfs_metadata snapshot snap2: (95) Operation not supported 2023-11-08T19:27:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3ba8069820 msgr2=0x7f3ba806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3ba8069820 0x7f3ba806bce0 secure :-1 s=SESSION_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0x7f3bb8001aa0 tx=0x7f3bb8000ed0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 msgr2=0x55e4cfc6a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc6a3b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00447a0 tx=0x7f3bc0055e10 comp rx=0 tx=0).stop 2023-11-08T19:27:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 shutdown_connections 2023-11-08T19:27:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3ba8069820 0x7f3ba806bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55e4cfd16950 0x55e4cfc6a3b0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55e4cfd16570 0x55e4cfc69e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 --2- 172.21.15.119:0/475893691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55e4cfc67d60 0x55e4cfd04d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 >> 172.21.15.119:0/475893691 conn(0x55e4cfc38a30 msgr2=0x55e4cfc7d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 shutdown_connections 2023-11-08T19:27:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.374+0000 7f3bc9b30f40 1 -- 172.21.15.119:0/475893691 wait complete. 2023-11-08T19:27:13.389 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_data snap3 2023-11-08T19:27:13.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.603+0000 7f3fd4000640 1 Processor -- start 2023-11-08T19:27:13.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.603+0000 7f3fd4000640 1 -- start start 2023-11-08T19:27:13.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.604+0000 7f3fd4000640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3fcc15d340 0x7f3fcc0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.604+0000 7f3fd4000640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3fcc0d14f0 0x7f3fcc0cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.604+0000 7f3fd4000640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc0b2710 con 0x7f3fcc15d720 2023-11-08T19:27:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.604+0000 7f3fd4000640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc0cb4d0 con 0x7f3fcc15d340 2023-11-08T19:27:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.604+0000 7f3fd4000640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc06bbb0 con 0x7f3fcc0d14f0 2023-11-08T19:27:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.604+0000 7f3fd1d9c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3fcc15d340 0x7f3fcc0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd1d9c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3fcc15d340 0x7f3fcc0cd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50030/0 (socket says 172.21.15.119:50030) 2023-11-08T19:27:13.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd1d9c640 1 -- 172.21.15.119:0/2587019450 learned_addr learned my addr 172.21.15.119:0/2587019450 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd259d640 1 --2- 172.21.15.119:0/2587019450 >> v2:172.21.15.188:3300/0 conn(0x7f3fcc0d14f0 0x7f3fcc0cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd1d9c640 1 -- 172.21.15.119:0/2587019450 >> v2:172.21.15.188:3300/0 conn(0x7f3fcc0d14f0 msgr2=0x7f3fcc0cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd1d9c640 1 --2- 172.21.15.119:0/2587019450 >> v2:172.21.15.188:3300/0 conn(0x7f3fcc0d14f0 0x7f3fcc0cdba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd1d9c640 1 -- 172.21.15.119:0/2587019450 >> v1:172.21.15.188:6789/0 conn(0x7f3fcc15d720 legacy=0x7f3fcc0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:13.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.605+0000 7f3fd1d9c640 1 -- 172.21.15.119:0/2587019450 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc06be90 con 0x7f3fcc15d340 2023-11-08T19:27:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.606+0000 7f3fd1d9c640 1 --2- 172.21.15.119:0/2587019450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d340 0x7f3fcc0cd660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc06aea0 tx=0x7f3fbc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1446b3d60a77995 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.606+0000 7f3fd0d9a640 1 -- 172.21.15.119:0/2587019450 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc02ca80 con 0x7f3fcc15d340 2023-11-08T19:27:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.606+0000 7f3fd0d9a640 1 -- 172.21.15.119:0/2587019450 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc02cbe0 con 0x7f3fcc15d340 2023-11-08T19:27:13.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.606+0000 7f3fd0d9a640 1 -- 172.21.15.119:0/2587019450 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0343f0 con 0x7f3fcc15d340 2023-11-08T19:27:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 -- 172.21.15.119:0/2587019450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d340 msgr2=0x7f3fcc0cd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 --2- 172.21.15.119:0/2587019450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d340 0x7f3fcc0cd660 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc06aea0 tx=0x7f3fbc02bee0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 -- 172.21.15.119:0/2587019450 shutdown_connections 2023-11-08T19:27:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 --2- 172.21.15.119:0/2587019450 >> v2:172.21.15.188:3300/0 conn(0x7f3fcc0d14f0 0x7f3fcc0cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 --2- 172.21.15.119:0/2587019450 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d340 0x7f3fcc0cd660 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 -- 172.21.15.119:0/2587019450 >> 172.21.15.119:0/2587019450 conn(0x7f3fcc09a110 msgr2=0x7f3fcc0d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:13.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 -- 172.21.15.119:0/2587019450 shutdown_connections 2023-11-08T19:27:13.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.607+0000 7f3fd4000640 1 -- 172.21.15.119:0/2587019450 wait complete. 2023-11-08T19:27:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.609+0000 7f3fd4000640 1 Processor -- start 2023-11-08T19:27:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.609+0000 7f3fd4000640 1 -- start start 2023-11-08T19:27:13.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd4000640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fcc0d14f0 0x7f3fcc158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd4000640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fcc15d340 0x7f3fcc15e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd4000640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d720 0x7f3fcc0b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd4000640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c6060 con 0x7f3fcc15d720 2023-11-08T19:27:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd4000640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1f2740 con 0x7f3fcc15d340 2023-11-08T19:27:13.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd4000640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c53e0 con 0x7f3fcc0d14f0 2023-11-08T19:27:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd259d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d720 0x7f3fcc0b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd259d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d720 0x7f3fcc0b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50034/0 (socket says 172.21.15.119:50034) 2023-11-08T19:27:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.610+0000 7f3fd259d640 1 -- 172.21.15.119:0/1043708055 learned_addr learned my addr 172.21.15.119:0/1043708055 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd159b640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fcc15d340 0x7f3fcc15e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd1d9c640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fcc0d14f0 0x7f3fcc158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd259d640 1 -- 172.21.15.119:0/1043708055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fcc0d14f0 msgr2=0x7f3fcc158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd259d640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3fcc0d14f0 0x7f3fcc158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd259d640 1 -- 172.21.15.119:0/1043708055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fcc15d340 msgr2=0x7f3fcc15e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:13.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd259d640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3fcc15d340 0x7f3fcc15e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd259d640 1 -- 172.21.15.119:0/1043708055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc009b30 con 0x7f3fcc15d720 2023-11-08T19:27:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd259d640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3fcc15d720 0x7f3fcc0b4970 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3fc400c7f0 tx=0x7f3fc400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4004220 con 0x7f3fcc15d720 2023-11-08T19:27:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc4004380 con 0x7f3fcc15d720 2023-11-08T19:27:13.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4007860 con 0x7f3fcc15d720 2023-11-08T19:27:13.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.611+0000 7f3fd4000640 1 -- 172.21.15.119:0/1043708055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc0c56c0 con 0x7f3fcc15d720 2023-11-08T19:27:13.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.612+0000 7f3fd4000640 1 -- 172.21.15.119:0/1043708055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc0c5c80 con 0x7f3fcc15d720 2023-11-08T19:27:13.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.613+0000 7f3fd4000640 1 -- 172.21.15.119:0/1043708055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc159540 con 0x7f3fcc15d720 2023-11-08T19:27:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.614+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fc4007bd0 con 0x7f3fcc15d720 2023-11-08T19:27:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.615+0000 7f3fca7fc640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3fb806aea0 0x7f3fb806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:13.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.615+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3fc4089910 con 0x7f3fcc15d720 2023-11-08T19:27:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.615+0000 7f3fd1d9c640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3fb806aea0 0x7f3fb806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.616+0000 7f3fd1d9c640 1 --2- 172.21.15.119:0/1043708055 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3fb806aea0 0x7f3fb806d360 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc035000 tx=0x7f3fbc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:13.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.619+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fcc159540 con 0x7f3fcc15d720 2023-11-08T19:27:13.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.809+0000 7f3fd4000640 1 -- 172.21.15.119:0/1043708055 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"} v 0) v1 -- 0x7f3fcc159540 con 0x7f3fcc15d720 2023-11-08T19:27:13.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:13.810+0000 7f3fca7fc640 1 -- 172.21.15.119:0/1043708055 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_data", "snap": "snap3"}]=-95 (95) Operation not supported v54) v1 ==== 131+0+0 (secure 0 0 0) 0x7f3fcc159540 con 0x7f3fcc15d720 2023-11-08T19:27:13.812 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOTSUP: (95) Operation not supported 2023-11-08T19:27:13.831 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:27:13.833 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool mksnap cephfs_metadata snap4 2023-11-08T19:27:14.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.069+0000 7f6234b99640 1 Processor -- start 2023-11-08T19:27:14.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.069+0000 7f6234b99640 1 -- start start 2023-11-08T19:27:14.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f6234b99640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f623015deb0 0x7f62300cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f6234b99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623006b5e0 0x7f623006ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f6234b99640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6230071720 con 0x7f623015e290 2023-11-08T19:27:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f6234b99640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62300cf5e0 con 0x7f623015deb0 2023-11-08T19:27:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f6234b99640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300cf8c0 con 0x7f623006b5e0 2023-11-08T19:27:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f622f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f623015deb0 0x7f62300cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f622f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f623015deb0 0x7f62300cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50046/0 (socket says 172.21.15.119:50046) 2023-11-08T19:27:14.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.071+0000 7f622f7fe640 1 -- 172.21.15.119:0/4043860059 learned_addr learned my addr 172.21.15.119:0/4043860059 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:14.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.072+0000 7f622ffff640 1 --2- 172.21.15.119:0/4043860059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623006b5e0 0x7f623006ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.072+0000 7f622f7fe640 1 -- 172.21.15.119:0/4043860059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623006b5e0 msgr2=0x7f623006ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:14.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.072+0000 7f622f7fe640 1 --2- 172.21.15.119:0/4043860059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623006b5e0 0x7f623006ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.072+0000 7f622f7fe640 1 -- 172.21.15.119:0/4043860059 >> v1:172.21.15.188:6789/0 conn(0x7f623015e290 legacy=0x7f623015e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.072+0000 7f622f7fe640 1 -- 172.21.15.119:0/4043860059 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300710d0 con 0x7f623015deb0 2023-11-08T19:27:14.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.073+0000 7f622f7fe640 1 --2- 172.21.15.119:0/4043860059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623015deb0 0x7f62300cccf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6220009870 tx=0x7f622002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6c89c375df8fee server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f622e7fc640 1 -- 172.21.15.119:0/4043860059 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220031640 con 0x7f623015deb0 2023-11-08T19:27:14.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f622e7fc640 1 -- 172.21.15.119:0/4043860059 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62200317a0 con 0x7f623015deb0 2023-11-08T19:27:14.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f622e7fc640 1 -- 172.21.15.119:0/4043860059 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220031b30 con 0x7f623015deb0 2023-11-08T19:27:14.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f6234b99640 1 -- 172.21.15.119:0/4043860059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623015deb0 msgr2=0x7f62300cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f6234b99640 1 --2- 172.21.15.119:0/4043860059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623015deb0 0x7f62300cccf0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f6220009870 tx=0x7f622002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f6234b99640 1 -- 172.21.15.119:0/4043860059 shutdown_connections 2023-11-08T19:27:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f6234b99640 1 --2- 172.21.15.119:0/4043860059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623006b5e0 0x7f623006ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.074+0000 7f6234b99640 1 --2- 172.21.15.119:0/4043860059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623015deb0 0x7f62300cccf0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.075+0000 7f6234b99640 1 -- 172.21.15.119:0/4043860059 >> 172.21.15.119:0/4043860059 conn(0x7f62300ceac0 msgr2=0x7f62300ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.075+0000 7f6234b99640 1 -- 172.21.15.119:0/4043860059 shutdown_connections 2023-11-08T19:27:14.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.075+0000 7f6234b99640 1 -- 172.21.15.119:0/4043860059 wait complete. 2023-11-08T19:27:14.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.077+0000 7f6234b99640 1 Processor -- start 2023-11-08T19:27:14.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.078+0000 7f6234b99640 1 -- start start 2023-11-08T19:27:14.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f6234b99640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623006b5e0 0x7f62300c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f6234b99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623015deb0 0x7f6230156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f6234b99640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623015e290 0x7f6230147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f6234b99640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300c7f40 con 0x7f623006b5e0 2023-11-08T19:27:14.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f6234b99640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62301575a0 con 0x7f623015e290 2023-11-08T19:27:14.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f6234b99640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62301566c0 con 0x7f623015deb0 2023-11-08T19:27:14.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f622f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623006b5e0 0x7f62300c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f622f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623006b5e0 0x7f62300c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50052/0 (socket says 172.21.15.119:50052) 2023-11-08T19:27:14.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.079+0000 7f622f7fe640 1 -- 172.21.15.119:0/1549698904 learned_addr learned my addr 172.21.15.119:0/1549698904 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:14.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622ffff640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623015e290 0x7f6230147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622effd640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623015deb0 0x7f6230156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622f7fe640 1 -- 172.21.15.119:0/1549698904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623015deb0 msgr2=0x7f6230156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622f7fe640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f623015deb0 0x7f6230156180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622f7fe640 1 -- 172.21.15.119:0/1549698904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623015e290 msgr2=0x7f6230147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622f7fe640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f623015e290 0x7f6230147e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622f7fe640 1 -- 172.21.15.119:0/1549698904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62200094d0 con 0x7f623006b5e0 2023-11-08T19:27:14.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f622f7fe640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f623006b5e0 0x7f62300c7850 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6220000c00 tx=0x7f62200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.080+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622003ac60 con 0x7f623006b5e0 2023-11-08T19:27:14.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.081+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622003adc0 con 0x7f623006b5e0 2023-11-08T19:27:14.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.081+0000 7f6234b99640 1 -- 172.21.15.119:0/1549698904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6230156990 con 0x7f623006b5e0 2023-11-08T19:27:14.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.081+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220044630 con 0x7f623006b5e0 2023-11-08T19:27:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.081+0000 7f6234b99640 1 -- 172.21.15.119:0/1549698904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6230156f50 con 0x7f623006b5e0 2023-11-08T19:27:14.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.082+0000 7f6234b99640 1 -- 172.21.15.119:0/1549698904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623004ece0 con 0x7f623006b5e0 2023-11-08T19:27:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.083+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f622002f050 con 0x7f623006b5e0 2023-11-08T19:27:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.084+0000 7f6217fff640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f621c06ae70 0x7f621c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.084+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f62200ab800 con 0x7f623006b5e0 2023-11-08T19:27:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.084+0000 7f622effd640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f621c06ae70 0x7f621c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.085+0000 7f622effd640 1 --2- 172.21.15.119:0/1549698904 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f621c06ae70 0x7f621c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f62300cd7d0 tx=0x7f6224009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.089+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6220076570 con 0x7f623006b5e0 2023-11-08T19:27:14.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.288+0000 7f6234b99640 1 -- 172.21.15.119:0/1549698904 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"} v 0) v1 -- 0x7f62300cb1c0 con 0x7f623006b5e0 2023-11-08T19:27:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.289+0000 7f6217fff640 1 -- 172.21.15.119:0/1549698904 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool mksnap", "pool": "cephfs_metadata", "snap": "snap4"}]=-95 (95) Operation not supported v54) v1 ==== 135+0+0 (secure 0 0 0) 0x7f6220031e60 con 0x7f623006b5e0 2023-11-08T19:27:14.292 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOTSUP: (95) Operation not supported 2023-11-08T19:27:14.303 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:27:14.304 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:14.305 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:14.428 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:14.550 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:14.551 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:14.551 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:14.611 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:14.612 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:14.667 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:14.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.869+0000 7f3dce687640 1 Processor -- start 2023-11-08T19:27:14.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.869+0000 7f3dce687640 1 -- start start 2023-11-08T19:27:14.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dce687640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3dc81447a0 0x7f3dc815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dce687640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3dc80cad30 0x7f3dc806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dce687640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc80c3a50 con 0x7f3dc806a120 2023-11-08T19:27:14.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dce687640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc8144b80 con 0x7f3dc80cad30 2023-11-08T19:27:14.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dce687640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc80c9150 con 0x7f3dc81447a0 2023-11-08T19:27:14.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dc77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3dc80cad30 0x7f3dc806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dccc24640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3dc806a120 0x7f3dc806a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46928/0 (socket says 172.21.15.119:46928) 2023-11-08T19:27:14.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.871+0000 7f3dccc24640 1 -- 172.21.15.119:0/2843629803 learned_addr learned my addr 172.21.15.119:0/2843629803 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:14.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc7fff640 1 --2- 172.21.15.119:0/2843629803 >> v2:172.21.15.188:3300/0 conn(0x7f3dc81447a0 0x7f3dc815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc6ffd640 1 -- 172.21.15.119:0/2843629803 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669396933 0 0) 0x7f3dc80c3a50 con 0x7f3dc806a120 2023-11-08T19:27:14.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc6ffd640 1 -- 172.21.15.119:0/2843629803 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3db0003520 con 0x7f3dc806a120 2023-11-08T19:27:14.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc77fe640 1 -- 172.21.15.119:0/2843629803 >> v2:172.21.15.188:3300/0 conn(0x7f3dc81447a0 msgr2=0x7f3dc815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:14.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc77fe640 1 --2- 172.21.15.119:0/2843629803 >> v2:172.21.15.188:3300/0 conn(0x7f3dc81447a0 0x7f3dc815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc77fe640 1 -- 172.21.15.119:0/2843629803 >> v1:172.21.15.119:6789/0 conn(0x7f3dc806a120 legacy=0x7f3dc806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.872+0000 7f3dc77fe640 1 -- 172.21.15.119:0/2843629803 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80c9410 con 0x7f3dc80cad30 2023-11-08T19:27:14.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.873+0000 7f3dc77fe640 1 --2- 172.21.15.119:0/2843629803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc80cad30 0x7f3dc806f930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80cfbd0 tx=0x7f3dbc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb3208f1373a9cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.873+0000 7f3dc6ffd640 1 -- 172.21.15.119:0/2843629803 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc031830 con 0x7f3dc80cad30 2023-11-08T19:27:14.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.873+0000 7f3dc6ffd640 1 -- 172.21.15.119:0/2843629803 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dbc031990 con 0x7f3dc80cad30 2023-11-08T19:27:14.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.873+0000 7f3dc6ffd640 1 -- 172.21.15.119:0/2843629803 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dbc031d00 con 0x7f3dc80cad30 2023-11-08T19:27:14.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 -- 172.21.15.119:0/2843629803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc80cad30 msgr2=0x7f3dc806f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 --2- 172.21.15.119:0/2843629803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc80cad30 0x7f3dc806f930 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80cfbd0 tx=0x7f3dbc02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 -- 172.21.15.119:0/2843629803 shutdown_connections 2023-11-08T19:27:14.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 --2- 172.21.15.119:0/2843629803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc80cad30 0x7f3dc806f930 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 --2- 172.21.15.119:0/2843629803 >> v2:172.21.15.188:3300/0 conn(0x7f3dc81447a0 0x7f3dc815f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 -- 172.21.15.119:0/2843629803 >> 172.21.15.119:0/2843629803 conn(0x7f3dc80cb610 msgr2=0x7f3dc80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:14.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 -- 172.21.15.119:0/2843629803 shutdown_connections 2023-11-08T19:27:14.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.874+0000 7f3dce687640 1 -- 172.21.15.119:0/2843629803 wait complete. 2023-11-08T19:27:14.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.876+0000 7f3dce687640 1 Processor -- start 2023-11-08T19:27:14.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.876+0000 7f3dce687640 1 -- start start 2023-11-08T19:27:14.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dce687640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc806a120 0x7f3dc80b4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dce687640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc80cad30 0x7f3dc815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dce687640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 0x7f3dc80b18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dce687640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc815add0 con 0x7f3dc81447a0 2023-11-08T19:27:14.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dce687640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80b2080 con 0x7f3dc80cad30 2023-11-08T19:27:14.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dce687640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80b1de0 con 0x7f3dc806a120 2023-11-08T19:27:14.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dccc24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 0x7f3dc80b18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dccc24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 0x7f3dc80b18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50068/0 (socket says 172.21.15.119:50068) 2023-11-08T19:27:14.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.877+0000 7f3dccc24640 1 -- 172.21.15.119:0/975754921 learned_addr learned my addr 172.21.15.119:0/975754921 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:14.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dc7fff640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc806a120 0x7f3dc80b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dccc24640 1 -- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc806a120 msgr2=0x7f3dc80b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dc77fe640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc80cad30 0x7f3dc815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dccc24640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc806a120 0x7f3dc80b4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dccc24640 1 -- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc80cad30 msgr2=0x7f3dc815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:14.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dccc24640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc80cad30 0x7f3dc815a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:14.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dccc24640 1 -- 172.21.15.119:0/975754921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dbc009400 con 0x7f3dc81447a0 2023-11-08T19:27:14.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dccc24640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 0x7f3dc80b18a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000cb30 tx=0x7f3dc000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc000aca0 con 0x7f3dc81447a0 2023-11-08T19:27:14.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc000ae00 con 0x7f3dc81447a0 2023-11-08T19:27:14.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc0012600 con 0x7f3dc81447a0 2023-11-08T19:27:14.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc815ca70 con 0x7f3dc81447a0 2023-11-08T19:27:14.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.878+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc815d030 con 0x7f3dc81447a0 2023-11-08T19:27:14.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.880+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc8074430 con 0x7f3dc81447a0 2023-11-08T19:27:14.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.880+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3dc0012820 con 0x7f3dc81447a0 2023-11-08T19:27:14.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.881+0000 7f3dc4ff9640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3db406ae70 0x7f3db406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:14.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.881+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3dc00897f0 con 0x7f3dc81447a0 2023-11-08T19:27:14.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.882+0000 7f3dc7fff640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3db406ae70 0x7f3db406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:14.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.882+0000 7f3dc7fff640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3db406ae70 0x7f3db406d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:14.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:14.886+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3dc00546a0 con 0x7f3dc81447a0 2023-11-08T19:27:15.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.076+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dc80cc490 con 0x7f3dc81447a0 2023-11-08T19:27:15.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.078+0000 7f3dc4ff9640 1 -- 172.21.15.119:0/975754921 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18523 (secure 0 0 0) 0x7f3dc0057cc0 con 0x7f3dc81447a0 2023-11-08T19:27:15.081 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:15.081 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:05.896221+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.081+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3db406ae70 msgr2=0x7f3db406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.081+0000 7f3dce687640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3db406ae70 0x7f3db406d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3db8006fd0 tx=0x7f3db8008040 comp rx=0 tx=0).stop 2023-11-08T19:27:15.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.081+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 msgr2=0x7f3dc80b18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.081+0000 7f3dce687640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 0x7f3dc80b18a0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f3dc000cb30 tx=0x7f3dc000a060 comp rx=0 tx=0).stop 2023-11-08T19:27:15.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.081+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 shutdown_connections 2023-11-08T19:27:15.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3db406ae70 0x7f3db406d330 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3dc81447a0 0x7f3dc80b18a0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc80cad30 0x7f3dc815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 --2- 172.21.15.119:0/975754921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc806a120 0x7f3dc80b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 >> 172.21.15.119:0/975754921 conn(0x7f3dc80cb610 msgr2=0x7f3dc80c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 shutdown_connections 2023-11-08T19:27:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.082+0000 7f3dce687640 1 -- 172.21.15.119:0/975754921 wait complete. 2023-11-08T19:27:15.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:27:02.460658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:27:03.346355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:15.104 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.315+0000 7fd841173640 1 Processor -- start 2023-11-08T19:27:15.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.315+0000 7fd841173640 1 -- start start 2023-11-08T19:27:15.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.317+0000 7fd841173640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c15d000 0x7fd83c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.317+0000 7fd841173640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 0x7fd83c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.317+0000 7fd841173640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c06d220 con 0x7fd83c0ae250 2023-11-08T19:27:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.317+0000 7fd841173640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c0ccd50 con 0x7fd83c0c3a50 2023-11-08T19:27:15.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.317+0000 7fd841173640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0aca40 con 0x7fd83c15d000 2023-11-08T19:27:15.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.318+0000 7fd833fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 0x7fd83c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.318+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c15d000 0x7fd83c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.318+0000 7fd833fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 0x7fd83c06a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:36666/0 (socket says 172.21.15.119:36666) 2023-11-08T19:27:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.318+0000 7fd833fff640 1 -- 172.21.15.119:0/3833853730 learned_addr learned my addr 172.21.15.119:0/3833853730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.318+0000 7fd83b59e640 1 --1- 172.21.15.119:0/3833853730 >> v1:172.21.15.188:6789/0 conn(0x7fd83c0ae250 0x7fd83c160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:50646/0 (socket says 172.21.15.119:50646) 2023-11-08T19:27:15.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.319+0000 7fd83ad9d640 1 -- 172.21.15.119:0/3833853730 >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 msgr2=0x7fd83c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.319+0000 7fd83ad9d640 1 --2- 172.21.15.119:0/3833853730 >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 0x7fd83c06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.319+0000 7fd83ad9d640 1 -- 172.21.15.119:0/3833853730 >> v1:172.21.15.188:6789/0 conn(0x7fd83c0ae250 legacy=0x7fd83c160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.319+0000 7fd83ad9d640 1 -- 172.21.15.119:0/3833853730 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0acd20 con 0x7fd83c15d000 2023-11-08T19:27:15.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.319+0000 7fd833fff640 1 --2- 172.21.15.119:0/3833853730 >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 0x7fd83c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:15.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.321+0000 7fd83ad9d640 1 --2- 172.21.15.119:0/3833853730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c15d000 0x7fd83c0cc810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0097a0 tx=0x7fd82c02b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26dec17739647ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.321+0000 7fd83a59c640 1 -- 172.21.15.119:0/3833853730 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c031780 con 0x7fd83c15d000 2023-11-08T19:27:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.321+0000 7fd83a59c640 1 -- 172.21.15.119:0/3833853730 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c0318e0 con 0x7fd83c15d000 2023-11-08T19:27:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.321+0000 7fd83a59c640 1 -- 172.21.15.119:0/3833853730 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c031c70 con 0x7fd83c15d000 2023-11-08T19:27:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 -- 172.21.15.119:0/3833853730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c15d000 msgr2=0x7fd83c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 --2- 172.21.15.119:0/3833853730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c15d000 0x7fd83c0cc810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0097a0 tx=0x7fd82c02b820 comp rx=0 tx=0).stop 2023-11-08T19:27:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 -- 172.21.15.119:0/3833853730 shutdown_connections 2023-11-08T19:27:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 --2- 172.21.15.119:0/3833853730 >> v2:172.21.15.188:3300/0 conn(0x7fd83c0c3a50 0x7fd83c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 --2- 172.21.15.119:0/3833853730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c15d000 0x7fd83c0cc810 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 -- 172.21.15.119:0/3833853730 >> 172.21.15.119:0/3833853730 conn(0x7fd83c0cb610 msgr2=0x7fd83c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 -- 172.21.15.119:0/3833853730 shutdown_connections 2023-11-08T19:27:15.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.322+0000 7fd841173640 1 -- 172.21.15.119:0/3833853730 wait complete. 2023-11-08T19:27:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.324+0000 7fd841173640 1 Processor -- start 2023-11-08T19:27:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.324+0000 7fd841173640 1 -- start start 2023-11-08T19:27:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd841173640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 0x7fd83c070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd841173640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c0c3a50 0x7fd83c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd841173640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c15d000 0x7fd83c1f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd841173640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c15eea0 con 0x7fd83c0ae250 2023-11-08T19:27:15.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd841173640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c15d920 con 0x7fd83c15d000 2023-11-08T19:27:15.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd841173640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c157210 con 0x7fd83c0c3a50 2023-11-08T19:27:15.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 0x7fd83c070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd83ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 0x7fd83c070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50080/0 (socket says 172.21.15.119:50080) 2023-11-08T19:27:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.325+0000 7fd83ad9d640 1 -- 172.21.15.119:0/2693803796 learned_addr learned my addr 172.21.15.119:0/2693803796 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83ad9d640 1 -- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c0c3a50 msgr2=0x7fd83c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83ad9d640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c0c3a50 0x7fd83c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83ad9d640 1 -- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c15d000 msgr2=0x7fd83c1f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83b59e640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c15d000 0x7fd83c1f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83ad9d640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c15d000 0x7fd83c1f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83ad9d640 1 -- 172.21.15.119:0/2693803796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8240094d0 con 0x7fd83c0ae250 2023-11-08T19:27:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83b59e640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c15d000 0x7fd83c1f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:15.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd83ad9d640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 0x7fd83c070ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0098c0 tx=0x7fd82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c032820 con 0x7fd83c0ae250 2023-11-08T19:27:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.326+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c032980 con 0x7fd83c0ae250 2023-11-08T19:27:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.327+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c032d30 con 0x7fd83c0ae250 2023-11-08T19:27:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.327+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c009400 con 0x7fd83c0ae250 2023-11-08T19:27:15.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.327+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c071530 con 0x7fd83c0ae250 2023-11-08T19:27:15.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.328+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c074430 con 0x7fd83c0ae250 2023-11-08T19:27:15.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.329+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd82c04e440 con 0x7fd83c0ae250 2023-11-08T19:27:15.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.329+0000 7fd8337fe640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd81c06ae70 0x7fd81c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.330+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd82c0ab970 con 0x7fd83c0ae250 2023-11-08T19:27:15.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.330+0000 7fd833fff640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd81c06ae70 0x7fd81c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.330+0000 7fd833fff640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd81c06ae70 0x7fd81c06d330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd824006f40 tx=0x7fd824006e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.334+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd82c076780 con 0x7fd83c0ae250 2023-11-08T19:27:15.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.521+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd83c06cd50 con 0x7fd83c0ae250 2023-11-08T19:27:15.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.521+0000 7fd8337fe640 1 -- 172.21.15.119:0/2693803796 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd82c079da0 con 0x7fd83c0ae250 2023-11-08T19:27:15.524 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd81c06ae70 msgr2=0x7fd81c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd81c06ae70 0x7fd81c06d330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd824006f40 tx=0x7fd824006e90 comp rx=0 tx=0).stop 2023-11-08T19:27:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 msgr2=0x7fd83c070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 0x7fd83c070ea0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd82c0098c0 tx=0x7fd82c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 shutdown_connections 2023-11-08T19:27:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd81c06ae70 0x7fd81c06d330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd83c15d000 0x7fd83c1f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd83c0c3a50 0x7fd83c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 --2- 172.21.15.119:0/2693803796 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd83c0ae250 0x7fd83c070ea0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.524+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 >> 172.21.15.119:0/2693803796 conn(0x7fd83c0cb610 msgr2=0x7fd83c0afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.525+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 shutdown_connections 2023-11-08T19:27:15.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.525+0000 7fd841173640 1 -- 172.21.15.119:0/2693803796 wait complete. 2023-11-08T19:27:15.545 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.743+0000 7f88d359e640 1 Processor -- start 2023-11-08T19:27:15.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.744+0000 7f88d359e640 1 -- start start 2023-11-08T19:27:15.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.745+0000 7f88d359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88d40b3f40 0x7f88d4167de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.745+0000 7f88d359e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d40d0830 con 0x7f88d4160910 2023-11-08T19:27:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.745+0000 7f88d359e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d40b27c0 con 0x7f88d4160530 2023-11-08T19:27:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.745+0000 7f88d359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d40b2aa0 con 0x7f88d40b3f40 2023-11-08T19:27:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.745+0000 7f88d1d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f88d4160910 0x7f88d40ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46932/0 (socket says 172.21.15.119:46932) 2023-11-08T19:27:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.745+0000 7f88d1d9b640 1 -- 172.21.15.119:0/2331555808 learned_addr learned my addr 172.21.15.119:0/2331555808 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d2d9d640 1 --2- 172.21.15.119:0/2331555808 >> v2:172.21.15.188:3300/0 conn(0x7f88d40b3f40 0x7f88d4167de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2796531222 0 0) 0x7f88d40d0830 con 0x7f88d4160910 2023-11-08T19:27:15.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c0003520 con 0x7f88d4160910 2023-11-08T19:27:15.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1578347187 0 0) 0x7f88c0003520 con 0x7f88d4160910 2023-11-08T19:27:15.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 >> v2:172.21.15.188:3300/0 conn(0x7f88d40b3f40 msgr2=0x7f88d4167de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 --2- 172.21.15.119:0/2331555808 >> v2:172.21.15.188:3300/0 conn(0x7f88d40b3f40 0x7f88d4167de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 >> v1:172.21.15.188:6789/0 conn(0x7f88d4160530 legacy=0x7f88d40cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40d1640 con 0x7f88d4160910 2023-11-08T19:27:15.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f88c40019f0 con 0x7f88d4160910 2023-11-08T19:27:15.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 >> v1:172.21.15.119:6789/0 conn(0x7f88d4160910 legacy=0x7f88d40ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88c0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88c00047c0 0x7f88c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88c0005180 0x7f88c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40b2aa0 con 0x7f88c0003970 2023-11-08T19:27:15.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c000c020 con 0x7f88c00047c0 2023-11-08T19:27:15.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.746+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c000c1e0 con 0x7f88c0005180 2023-11-08T19:27:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d259c640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88c0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d1d9b640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88c00047c0 0x7f88c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d2d9d640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88c0005180 0x7f88c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d259c640 1 -- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88c0005180 msgr2=0x7f88c000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d259c640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88c0005180 0x7f88c000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d259c640 1 -- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88c00047c0 msgr2=0x7f88c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d259c640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88c00047c0 0x7f88c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d259c640 1 -- 172.21.15.119:0/2331555808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c0003520 con 0x7f88c0003970 2023-11-08T19:27:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d1d9b640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88c00047c0 0x7f88c0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:15.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.747+0000 7f88d2d9d640 1 -- 172.21.15.119:0/2331555808 reap_dead start 2023-11-08T19:27:15.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.748+0000 7f88d259c640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88c0003d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f88bc001770 tx=0x7f88c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12369561326dbdf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.748+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0052d0 con 0x7f88c0003970 2023-11-08T19:27:15.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.748+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc02e050 con 0x7f88c0003970 2023-11-08T19:27:15.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.748+0000 7f88d159a640 1 -- 172.21.15.119:0/2331555808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc003640 con 0x7f88c0003970 2023-11-08T19:27:15.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 -- 172.21.15.119:0/2331555808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 msgr2=0x7f88c0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88c0003d90 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f88bc001770 tx=0x7f88c00057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 -- 172.21.15.119:0/2331555808 shutdown_connections 2023-11-08T19:27:15.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 --2- 172.21.15.119:0/2331555808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88c0003d90 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 -- 172.21.15.119:0/2331555808 >> 172.21.15.119:0/2331555808 conn(0x7f88d40d2ec0 msgr2=0x7f88d40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 -- 172.21.15.119:0/2331555808 shutdown_connections 2023-11-08T19:27:15.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.749+0000 7f88d359e640 1 -- 172.21.15.119:0/2331555808 wait complete. 2023-11-08T19:27:15.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.751+0000 7f88d359e640 1 Processor -- start 2023-11-08T19:27:15.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.751+0000 7f88d359e640 1 -- start start 2023-11-08T19:27:15.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88d40c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d40c5ac0 0x7f88d40b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d41fa220 0x7f88d41fa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d41fabe0 con 0x7f88c0003970 2023-11-08T19:27:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40c8e00 con 0x7f88d41fa220 2023-11-08T19:27:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40c5f40 con 0x7f88d40c5ac0 2023-11-08T19:27:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88d40c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88d40c6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50092/0 (socket says 172.21.15.119:50092) 2023-11-08T19:27:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.752+0000 7f88d259c640 1 -- 172.21.15.119:0/2642057238 learned_addr learned my addr 172.21.15.119:0/2642057238 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d259c640 1 -- 172.21.15.119:0/2642057238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d40c5ac0 msgr2=0x7f88d40b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d259c640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d40c5ac0 0x7f88d40b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d259c640 1 -- 172.21.15.119:0/2642057238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d41fa220 msgr2=0x7f88d41fa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d259c640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d41fa220 0x7f88d41fa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d259c640 1 -- 172.21.15.119:0/2642057238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40c9050 con 0x7f88c0003970 2023-11-08T19:27:15.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d259c640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88d40c6890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f88bc034040 tx=0x7f88bc001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc006ea0 con 0x7f88c0003970 2023-11-08T19:27:15.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc007a80 con 0x7f88c0003970 2023-11-08T19:27:15.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc007470 con 0x7f88c0003970 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d40c9330 con 0x7f88c0003970 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.753+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d40c98f0 con 0x7f88c0003970 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.755+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d40afb50 con 0x7f88c0003970 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.755+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88bc0075d0 con 0x7f88c0003970 2023-11-08T19:27:15.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.756+0000 7f88caffd640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88b406aea0 0x7f88b406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:15.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.756+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f88bc0ab970 con 0x7f88c0003970 2023-11-08T19:27:15.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.756+0000 7f88d1d9b640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88b406aea0 0x7f88b406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:15.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.757+0000 7f88d1d9b640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88b406aea0 0x7f88b406d360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88c4001b20 tx=0x7f88c4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:15.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.761+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88bc0766e0 con 0x7f88c0003970 2023-11-08T19:27:15.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.935+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f88d404ece0 con 0x7f88c0003970 2023-11-08T19:27:15.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.937+0000 7f88caffd640 1 -- 172.21.15.119:0/2642057238 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v54) v1 ==== 103+0+11 (secure 0 0 0) 0x7f88bc04b020 con 0x7f88c0003970 2023-11-08T19:27:15.939 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88b406aea0 msgr2=0x7f88b406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88b406aea0 0x7f88b406d360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88c4001b20 tx=0x7f88c4009040 comp rx=0 tx=0).stop 2023-11-08T19:27:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 msgr2=0x7f88d40c6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:15.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88d40c6890 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f88bc034040 tx=0x7f88bc001170 comp rx=0 tx=0).stop 2023-11-08T19:27:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 shutdown_connections 2023-11-08T19:27:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88b406aea0 0x7f88b406d360 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d41fa220 0x7f88d41fa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d40c5ac0 0x7f88d40b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 --2- 172.21.15.119:0/2642057238 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88c0003970 0x7f88d40c6890 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 >> 172.21.15.119:0/2642057238 conn(0x7f88d40d2ec0 msgr2=0x7f88d41e7470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 shutdown_connections 2023-11-08T19:27:15.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:15.939+0000 7f88d359e640 1 -- 172.21.15.119:0/2642057238 wait complete. 2023-11-08T19:27:15.959 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.165+0000 7fb99ead4640 1 Processor -- start 2023-11-08T19:27:16.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.166+0000 7fb99ead4640 1 -- start start 2023-11-08T19:27:16.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb99ead4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 0x7fb9980d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb99ead4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb998161870 0x7fb9980cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb99ead4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99815df50 con 0x7fb9980cfa00 2023-11-08T19:27:16.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb99ead4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9980d1e60 con 0x7fb998161870 2023-11-08T19:27:16.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb99ead4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980d2830 con 0x7fb99815d4a0 2023-11-08T19:27:16.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb997fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb9980cfa00 0x7fb998161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46934/0 (socket says 172.21.15.119:46934) 2023-11-08T19:27:16.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.167+0000 7fb997fff640 1 -- 172.21.15.119:0/28025096 learned_addr learned my addr 172.21.15.119:0/28025096 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb99c870640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 0x7fb9980d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb99d071640 1 --2- 172.21.15.119:0/28025096 >> v2:172.21.15.188:3300/0 conn(0x7fb998161870 0x7fb9980cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067158738 0 0) 0x7fb99815df50 con 0x7fb9980cfa00 2023-11-08T19:27:16.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb984003520 con 0x7fb9980cfa00 2023-11-08T19:27:16.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2930980371 0 0) 0x7fb984003520 con 0x7fb9980cfa00 2023-11-08T19:27:16.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 msgr2=0x7fb9980d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 0x7fb9980d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 >> v2:172.21.15.188:3300/0 conn(0x7fb998161870 msgr2=0x7fb9980cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 --2- 172.21.15.119:0/28025096 >> v2:172.21.15.188:3300/0 conn(0x7fb998161870 0x7fb9980cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9980d03a0 con 0x7fb9980cfa00 2023-11-08T19:27:16.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fb980001670 con 0x7fb9980cfa00 2023-11-08T19:27:16.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 >> v1:172.21.15.119:6789/0 conn(0x7fb9980cfa00 legacy=0x7fb998161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb984003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 0x7fb9840047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb984004d00 0x7fb984005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980d1e60 con 0x7fb984003970 2023-11-08T19:27:16.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980d2830 con 0x7fb984004d00 2023-11-08T19:27:16.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.168+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98400b9b0 con 0x7fb99815d4a0 2023-11-08T19:27:16.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.169+0000 7fb99c870640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb984003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.169+0000 7fb99c870640 1 -- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 msgr2=0x7fb9840047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.169+0000 7fb99c870640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb99815d4a0 0x7fb9840047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.169+0000 7fb99c870640 1 -- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb984004d00 msgr2=0x7fb984005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.169+0000 7fb99c870640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb984004d00 0x7fb984005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.169+0000 7fb99c870640 1 -- 172.21.15.119:0/28025096 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb984003520 con 0x7fb984003970 2023-11-08T19:27:16.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.170+0000 7fb99c870640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb984003d90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb988009fc0 tx=0x7fb9840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7781643fc33d13 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.170+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98802cc60 con 0x7fb984003970 2023-11-08T19:27:16.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.170+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98800f040 con 0x7fb984003970 2023-11-08T19:27:16.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.170+0000 7fb9977fe640 1 -- 172.21.15.119:0/28025096 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988034440 con 0x7fb984003970 2023-11-08T19:27:16.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99ead4640 1 -- 172.21.15.119:0/28025096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 msgr2=0x7fb984003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99ead4640 1 --2- 172.21.15.119:0/28025096 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb984003d90 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb988009fc0 tx=0x7fb9840057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99d071640 1 -- 172.21.15.119:0/28025096 reap_dead start 2023-11-08T19:27:16.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99ead4640 1 -- 172.21.15.119:0/28025096 shutdown_connections 2023-11-08T19:27:16.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99ead4640 1 -- 172.21.15.119:0/28025096 >> 172.21.15.119:0/28025096 conn(0x7fb99806a9f0 msgr2=0x7fb99806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99ead4640 1 -- 172.21.15.119:0/28025096 shutdown_connections 2023-11-08T19:27:16.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.171+0000 7fb99ead4640 1 -- 172.21.15.119:0/28025096 wait complete. 2023-11-08T19:27:16.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.173+0000 7fb99ead4640 1 Processor -- start 2023-11-08T19:27:16.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.173+0000 7fb99ead4640 1 -- start start 2023-11-08T19:27:16.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99ead4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb9980a9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99ead4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9981f6420 0x7fb998160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99ead4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9980aa5e0 0x7fb9980aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99ead4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9980cb300 con 0x7fb984003970 2023-11-08T19:27:16.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99ead4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9981f6900 con 0x7fb9980aa5e0 2023-11-08T19:27:16.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99ead4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9981f6be0 con 0x7fb9981f6420 2023-11-08T19:27:16.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb9980a9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb9980a9aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50102/0 (socket says 172.21.15.119:50102) 2023-11-08T19:27:16.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 -- 172.21.15.119:0/666352683 learned_addr learned my addr 172.21.15.119:0/666352683 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:16.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 -- 172.21.15.119:0/666352683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9981f6420 msgr2=0x7fb998160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9981f6420 0x7fb998160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 -- 172.21.15.119:0/666352683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9980aa5e0 msgr2=0x7fb9980aaa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9980aa5e0 0x7fb9980aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.174+0000 7fb99c870640 1 -- 172.21.15.119:0/666352683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9981f6ec0 con 0x7fb984003970 2023-11-08T19:27:16.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.175+0000 7fb99c870640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb9980a9aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e230 tx=0x7fb988009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.175+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988034b50 con 0x7fb984003970 2023-11-08T19:27:16.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.175+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb988034cb0 con 0x7fb984003970 2023-11-08T19:27:16.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.175+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9981f71a0 con 0x7fb984003970 2023-11-08T19:27:16.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.175+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb988003bb0 con 0x7fb984003970 2023-11-08T19:27:16.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.175+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9980aafb0 con 0x7fb984003970 2023-11-08T19:27:16.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.176+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9981f73c0 con 0x7fb984003970 2023-11-08T19:27:16.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.177+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb988040450 con 0x7fb984003970 2023-11-08T19:27:16.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.179+0000 7fb9957fa640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb97806ae70 0x7fb97806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.179+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb9880ac070 con 0x7fb984003970 2023-11-08T19:27:16.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.179+0000 7fb997fff640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb97806ae70 0x7fb97806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.180+0000 7fb997fff640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb97806ae70 0x7fb97806d330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb9800017a0 tx=0x7fb980001550 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.183+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb988076fa0 con 0x7fb984003970 2023-11-08T19:27:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.376+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb9980d10f0 con 0x7fb984003970 2023-11-08T19:27:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.376+0000 7fb9957fa640 1 -- 172.21.15.119:0/666352683 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb98807a5c0 con 0x7fb984003970 2023-11-08T19:27:16.379 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb97806ae70 msgr2=0x7fb97806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb97806ae70 0x7fb97806d330 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb9800017a0 tx=0x7fb980001550 comp rx=0 tx=0).stop 2023-11-08T19:27:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 msgr2=0x7fb9980a9aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb9980a9aa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb98800e230 tx=0x7fb988009750 comp rx=0 tx=0).stop 2023-11-08T19:27:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 shutdown_connections 2023-11-08T19:27:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb97806ae70 0x7fb97806d330 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb9980aa5e0 0x7fb9980aaa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb9981f6420 0x7fb998160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 --2- 172.21.15.119:0/666352683 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb984003970 0x7fb9980a9aa0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.379+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 >> 172.21.15.119:0/666352683 conn(0x7fb99806a9f0 msgr2=0x7fb9980b8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.380+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 shutdown_connections 2023-11-08T19:27:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.380+0000 7fb99ead4640 1 -- 172.21.15.119:0/666352683 wait complete. 2023-11-08T19:27:16.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:16.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.608+0000 7fdd61f73640 1 Processor -- start 2023-11-08T19:27:16.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.609+0000 7fdd61f73640 1 -- start start 2023-11-08T19:27:16.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.610+0000 7fdd61f73640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd5c144690 0x7fdd5c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.610+0000 7fdd61f73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd61f73640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd5c162ae0 con 0x7fdd5c06ba30 2023-11-08T19:27:16.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd61f73640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5c165de0 con 0x7fdd5c144690 2023-11-08T19:27:16.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd61f73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0cd050 con 0x7fdd5c0cc6d0 2023-11-08T19:27:16.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd5c144690 0x7fdd5c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdd5c144690 0x7fdd5c06a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50118/0 (socket says 172.21.15.119:50118) 2023-11-08T19:27:16.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 -- 172.21.15.119:0/3288414403 learned_addr learned my addr 172.21.15.119:0/3288414403 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:16.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 -- 172.21.15.119:0/3288414403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c0cc6d0 msgr2=0x7fdd5c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:16.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5affd640 1 --2- 172.21.15.119:0/3288414403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 --2- 172.21.15.119:0/3288414403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 -- 172.21.15.119:0/3288414403 >> v1:172.21.15.119:6789/0 conn(0x7fdd5c06ba30 legacy=0x7fdd5c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.611+0000 7fdd5b7fe640 1 -- 172.21.15.119:0/3288414403 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5c165160 con 0x7fdd5c144690 2023-11-08T19:27:16.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.612+0000 7fdd5b7fe640 1 --2- 172.21.15.119:0/3288414403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c06a550 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009870 tx=0x7fdd4402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c699eac2aba2bfed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.613+0000 7fdd5a7fc640 1 -- 172.21.15.119:0/3288414403 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44030840 con 0x7fdd5c144690 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.613+0000 7fdd5a7fc640 1 -- 172.21.15.119:0/3288414403 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd440309a0 con 0x7fdd5c144690 2023-11-08T19:27:16.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.613+0000 7fdd5a7fc640 1 -- 172.21.15.119:0/3288414403 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44030d30 con 0x7fdd5c144690 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.613+0000 7fdd61f73640 1 -- 172.21.15.119:0/3288414403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 msgr2=0x7fdd5c06a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.613+0000 7fdd61f73640 1 --2- 172.21.15.119:0/3288414403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c06a550 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009870 tx=0x7fdd4402b980 comp rx=0 tx=0).stop 2023-11-08T19:27:16.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.613+0000 7fdd61f73640 1 -- 172.21.15.119:0/3288414403 shutdown_connections 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.614+0000 7fdd61f73640 1 --2- 172.21.15.119:0/3288414403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.614+0000 7fdd61f73640 1 --2- 172.21.15.119:0/3288414403 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c06a550 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.614+0000 7fdd61f73640 1 -- 172.21.15.119:0/3288414403 >> 172.21.15.119:0/3288414403 conn(0x7fdd5c0cb610 msgr2=0x7fdd5c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.614+0000 7fdd61f73640 1 -- 172.21.15.119:0/3288414403 shutdown_connections 2023-11-08T19:27:16.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.614+0000 7fdd61f73640 1 -- 172.21.15.119:0/3288414403 wait complete. 2023-11-08T19:27:16.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.615+0000 7fdd61f73640 1 Processor -- start 2023-11-08T19:27:16.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.615+0000 7fdd61f73640 1 -- start start 2023-11-08T19:27:16.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.616+0000 7fdd61f73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c06ba30 0x7fdd5c1406f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.616+0000 7fdd61f73640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0b4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.616+0000 7fdd61f73640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c0b1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.616+0000 7fdd61f73640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b3430 con 0x7fdd5c144690 2023-11-08T19:27:16.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.616+0000 7fdd61f73640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c140c30 con 0x7fdd5c0cc6d0 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.616+0000 7fdd61f73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c0b3660 con 0x7fdd5c06ba30 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c0b1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c0b1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50124/0 (socket says 172.21.15.119:50124) 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 -- 172.21.15.119:0/955115400 learned_addr learned my addr 172.21.15.119:0/955115400 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:16.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5affd640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5b7fe640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c06ba30 0x7fdd5c1406f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 -- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c06ba30 msgr2=0x7fdd5c1406f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c06ba30 0x7fdd5c1406f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 -- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd5c0cc6d0 msgr2=0x7fdd5c0b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 -- 172.21.15.119:0/955115400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd440094d0 con 0x7fdd5c144690 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5b7fe640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c06ba30 0x7fdd5c1406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:16.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.617+0000 7fdd5bfff640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c0b1890 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c3c0 tx=0x7fdd5400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.618+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540052d0 con 0x7fdd5c144690 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.618+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54005430 con 0x7fdd5c144690 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.618+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54003830 con 0x7fdd5c144690 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.618+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c0b3940 con 0x7fdd5c144690 2023-11-08T19:27:16.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.618+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c0b3f00 con 0x7fdd5c144690 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.619+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c074430 con 0x7fdd5c144690 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.620+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd54003990 con 0x7fdd5c144690 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.621+0000 7fdd53fff640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdd3c06ae70 0x7fdd3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.621+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdd54088c10 con 0x7fdd5c144690 2023-11-08T19:27:16.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.621+0000 7fdd5b7fe640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdd3c06ae70 0x7fdd3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:16.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.622+0000 7fdd5b7fe640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdd3c06ae70 0x7fdd3c06d330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd44036000 tx=0x7fdd4403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:16.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.625+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd54053ab0 con 0x7fdd5c144690 2023-11-08T19:27:16.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.818+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdd5c06ae90 con 0x7fdd5c144690 2023-11-08T19:27:16.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.819+0000 7fdd53fff640 1 -- 172.21.15.119:0/955115400 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdd540170a0 con 0x7fdd5c144690 2023-11-08T19:27:16.821 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:27:16.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdd3c06ae70 msgr2=0x7fdd3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdd3c06ae70 0x7fdd3c06d330 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdd44036000 tx=0x7fdd4403b040 comp rx=0 tx=0).stop 2023-11-08T19:27:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 msgr2=0x7fdd5c0b1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:16.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c0b1890 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdd5400c3c0 tx=0x7fdd5400c880 comp rx=0 tx=0).stop 2023-11-08T19:27:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 shutdown_connections 2023-11-08T19:27:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdd3c06ae70 0x7fdd3c06d330 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdd5c144690 0x7fdd5c0b1890 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdd5c0cc6d0 0x7fdd5c0b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 --2- 172.21.15.119:0/955115400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdd5c06ba30 0x7fdd5c1406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:16.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 >> 172.21.15.119:0/955115400 conn(0x7fdd5c0cb610 msgr2=0x7fdd5c06bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:16.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 shutdown_connections 2023-11-08T19:27:16.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:16.821+0000 7fdd61f73640 1 -- 172.21.15.119:0/955115400 wait complete. 2023-11-08T19:27:16.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:27:17.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.044+0000 7f44e23a9640 1 Processor -- start 2023-11-08T19:27:17.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.044+0000 7f44e23a9640 1 -- start start 2023-11-08T19:27:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.045+0000 7f44e23a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e23a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44dc0cad30 0x7f44dc06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e23a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44dc0cb170 0x7f44dc06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e23a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc0c3a50 con 0x7f44dc0cad30 2023-11-08T19:27:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e23a9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc144b80 con 0x7f44dc0cb170 2023-11-08T19:27:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e23a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c9150 con 0x7f44dc1447a0 2023-11-08T19:27:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44dc0cad30 0x7f44dc06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44dc0cad30 0x7f44dc06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50140/0 (socket says 172.21.15.119:50140) 2023-11-08T19:27:17.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 -- 172.21.15.119:0/756996650 learned_addr learned my addr 172.21.15.119:0/756996650 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:17.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e0946640 1 --2- 172.21.15.119:0/756996650 >> v2:172.21.15.188:3300/0 conn(0x7f44dc0cb170 0x7f44dc06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44dbfff640 1 --2- 172.21.15.119:0/756996650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 -- 172.21.15.119:0/756996650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 msgr2=0x7f44dc15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 --2- 172.21.15.119:0/756996650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 -- 172.21.15.119:0/756996650 >> v2:172.21.15.188:3300/0 conn(0x7f44dc0cb170 msgr2=0x7f44dc06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 --2- 172.21.15.119:0/756996650 >> v2:172.21.15.188:3300/0 conn(0x7f44dc0cb170 0x7f44dc06a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44d3fff640 1 -- 172.21.15.119:0/756996650 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc0c9410 con 0x7f44dc0cad30 2023-11-08T19:27:17.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.046+0000 7f44e0946640 1 --2- 172.21.15.119:0/756996650 >> v2:172.21.15.188:3300/0 conn(0x7f44dc0cb170 0x7f44dc06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:17.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.047+0000 7f44d3fff640 1 --2- 172.21.15.119:0/756996650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cad30 0x7f44dc06f930 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0ca7a0 tx=0x7f44c402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfae893af8d74c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.048+0000 7f44db7fe640 1 -- 172.21.15.119:0/756996650 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4031880 con 0x7f44dc0cad30 2023-11-08T19:27:17.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.048+0000 7f44db7fe640 1 -- 172.21.15.119:0/756996650 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c40319e0 con 0x7f44dc0cad30 2023-11-08T19:27:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.048+0000 7f44db7fe640 1 -- 172.21.15.119:0/756996650 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4031d70 con 0x7f44dc0cad30 2023-11-08T19:27:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.048+0000 7f44e23a9640 1 -- 172.21.15.119:0/756996650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cad30 msgr2=0x7f44dc06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.048+0000 7f44e23a9640 1 --2- 172.21.15.119:0/756996650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cad30 0x7f44dc06f930 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f44dc0ca7a0 tx=0x7f44c402b920 comp rx=0 tx=0).stop 2023-11-08T19:27:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 -- 172.21.15.119:0/756996650 shutdown_connections 2023-11-08T19:27:17.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 --2- 172.21.15.119:0/756996650 >> v2:172.21.15.188:3300/0 conn(0x7f44dc0cb170 0x7f44dc06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 --2- 172.21.15.119:0/756996650 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cad30 0x7f44dc06f930 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 --2- 172.21.15.119:0/756996650 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 -- 172.21.15.119:0/756996650 >> 172.21.15.119:0/756996650 conn(0x7f44dc0cb610 msgr2=0x7f44dc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 -- 172.21.15.119:0/756996650 shutdown_connections 2023-11-08T19:27:17.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.049+0000 7f44e23a9640 1 -- 172.21.15.119:0/756996650 wait complete. 2023-11-08T19:27:17.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.050+0000 7f44e23a9640 1 Processor -- start 2023-11-08T19:27:17.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 -- start start 2023-11-08T19:27:17.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44dc0cad30 0x7f44dc1f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 0x7f44dc0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc1f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1f4d60 con 0x7f44dc0cb170 2023-11-08T19:27:17.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1f11f0 con 0x7f44dc0cad30 2023-11-08T19:27:17.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.051+0000 7f44e23a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1f08c0 con 0x7f44dc1447a0 2023-11-08T19:27:17.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 0x7f44dc0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 0x7f44dc0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50148/0 (socket says 172.21.15.119:50148) 2023-11-08T19:27:17.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 -- 172.21.15.119:0/241806997 learned_addr learned my addr 172.21.15.119:0/241806997 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:17.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 -- 172.21.15.119:0/241806997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 msgr2=0x7f44dc1f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc1f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 -- 172.21.15.119:0/241806997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44dc0cad30 msgr2=0x7f44dc1f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44dc0cad30 0x7f44dc1f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.052+0000 7f44d3fff640 1 -- 172.21.15.119:0/241806997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c40094d0 con 0x7f44dc0cb170 2023-11-08T19:27:17.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.053+0000 7f44d3fff640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 0x7f44dc0b4170 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f44c402b960 tx=0x7f44c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.053+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c4031870 con 0x7f44dc0cb170 2023-11-08T19:27:17.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.053+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c40443f0 con 0x7f44dc0cb170 2023-11-08T19:27:17.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.053+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44dc1f0ba0 con 0x7f44dc0cb170 2023-11-08T19:27:17.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.053+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c40447a0 con 0x7f44dc0cb170 2023-11-08T19:27:17.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.053+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc1f1680 con 0x7f44dc0cb170 2023-11-08T19:27:17.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.054+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc074430 con 0x7f44dc0cb170 2023-11-08T19:27:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.055+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44c404d450 con 0x7f44dc0cb170 2023-11-08T19:27:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.056+0000 7f44d97fa640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.056+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f44c40ac070 con 0x7f44dc0cb170 2023-11-08T19:27:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.056+0000 7f44dbfff640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.057+0000 7f44dbfff640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44bc06ae70 0x7f44bc06d330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f44cc006b10 tx=0x7f44cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.062+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44c40deca0 con 0x7f44dc0cb170 2023-11-08T19:27:17.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.263+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f44dc1f1330 con 0x7f44dc0cb170 2023-11-08T19:27:17.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.264+0000 7f44d97fa640 1 -- 172.21.15.119:0/241806997 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f44c4035070 con 0x7f44dc0cb170 2023-11-08T19:27:17.267 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:27:17.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44bc06ae70 msgr2=0x7f44bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44bc06ae70 0x7f44bc06d330 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f44cc006b10 tx=0x7f44cc008040 comp rx=0 tx=0).stop 2023-11-08T19:27:17.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 msgr2=0x7f44dc0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 0x7f44dc0b4170 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f44c402b960 tx=0x7f44c40046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 shutdown_connections 2023-11-08T19:27:17.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44dc1447a0 0x7f44dc1f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44dc0cb170 0x7f44dc0b4170 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 --2- 172.21.15.119:0/241806997 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44dc0cad30 0x7f44dc1f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 >> 172.21.15.119:0/241806997 conn(0x7f44dc0cb610 msgr2=0x7f44dc06a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 shutdown_connections 2023-11-08T19:27:17.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.266+0000 7f44e23a9640 1 -- 172.21.15.119:0/241806997 wait complete. 2023-11-08T19:27:17.288 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:27:17.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:17.289 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:17.389 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:27:17.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:17.390 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:27:17.445 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:27:17.445 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:17.505 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:27:17.505 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:17.506 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:17.565 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:27:17.565 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:27:17.565 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:17.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:27:17.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.844+0000 7f6d35607640 1 Processor -- start 2023-11-08T19:27:17.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.845+0000 7f6d35607640 1 -- start start 2023-11-08T19:27:17.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d35607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d35607640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d3015d9f0 0x7f6d300c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d35607640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d300c4960 con 0x7f6d3006cb40 2023-11-08T19:27:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d35607640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d30143a40 con 0x7f6d3015d9f0 2023-11-08T19:27:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d35607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3015df60 con 0x7f6d30142f80 2023-11-08T19:27:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d2f7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6d3006cb40 0x7f6d300c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46950/0 (socket says 172.21.15.119:46950) 2023-11-08T19:27:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d2f7fe640 1 -- 172.21.15.119:0/3699917385 learned_addr learned my addr 172.21.15.119:0/3699917385 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:17.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d267fc640 1 --2- 172.21.15.119:0/3699917385 >> v2:172.21.15.119:3300/0 conn(0x7f6d3015d9f0 0x7f6d300c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.847+0000 7f6d2effd640 1 --2- 172.21.15.119:0/3699917385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.848+0000 7f6d2e7fc640 1 -- 172.21.15.119:0/3699917385 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162789609 0 0) 0x7f6d300c4960 con 0x7f6d3006cb40 2023-11-08T19:27:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.848+0000 7f6d2e7fc640 1 -- 172.21.15.119:0/3699917385 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d1c003520 con 0x7f6d3006cb40 2023-11-08T19:27:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.848+0000 7f6d267fc640 1 -- 172.21.15.119:0/3699917385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 msgr2=0x7f6d300cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.848+0000 7f6d267fc640 1 --2- 172.21.15.119:0/3699917385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.848+0000 7f6d267fc640 1 -- 172.21.15.119:0/3699917385 >> v1:172.21.15.119:6789/0 conn(0x7f6d3006cb40 legacy=0x7f6d300c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.848+0000 7f6d267fc640 1 -- 172.21.15.119:0/3699917385 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3015e240 con 0x7f6d3015d9f0 2023-11-08T19:27:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.849+0000 7f6d267fc640 1 --2- 172.21.15.119:0/3699917385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300c9cd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d180097a0 tx=0x7f6d1802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45959853aa5a537 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.849+0000 7f6d2e7fc640 1 -- 172.21.15.119:0/3699917385 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18031780 con 0x7f6d3015d9f0 2023-11-08T19:27:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.849+0000 7f6d2e7fc640 1 -- 172.21.15.119:0/3699917385 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d180318e0 con 0x7f6d3015d9f0 2023-11-08T19:27:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.849+0000 7f6d2e7fc640 1 -- 172.21.15.119:0/3699917385 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18031c70 con 0x7f6d3015d9f0 2023-11-08T19:27:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 -- 172.21.15.119:0/3699917385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 msgr2=0x7f6d300c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 --2- 172.21.15.119:0/3699917385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300c9cd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d180097a0 tx=0x7f6d1802b820 comp rx=0 tx=0).stop 2023-11-08T19:27:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 -- 172.21.15.119:0/3699917385 shutdown_connections 2023-11-08T19:27:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 --2- 172.21.15.119:0/3699917385 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300c9cd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 --2- 172.21.15.119:0/3699917385 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 -- 172.21.15.119:0/3699917385 >> 172.21.15.119:0/3699917385 conn(0x7f6d300cb610 msgr2=0x7f6d300cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:17.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 -- 172.21.15.119:0/3699917385 shutdown_connections 2023-11-08T19:27:17.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.850+0000 7f6d35607640 1 -- 172.21.15.119:0/3699917385 wait complete. 2023-11-08T19:27:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.851+0000 7f6d35607640 1 Processor -- start 2023-11-08T19:27:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.852+0000 7f6d35607640 1 -- start start 2023-11-08T19:27:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d35607640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d3006cb40 0x7f6d300b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d35607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d35607640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d35607640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3006f930 con 0x7f6d3015d9f0 2023-11-08T19:27:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d35607640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3006fb40 con 0x7f6d3006cb40 2023-11-08T19:27:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d35607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3006fe20 con 0x7f6d30142f80 2023-11-08T19:27:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50166/0 (socket says 172.21.15.119:50166) 2023-11-08T19:27:17.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 -- 172.21.15.119:0/279895830 learned_addr learned my addr 172.21.15.119:0/279895830 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:17.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 -- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 msgr2=0x7f6d300b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2effd640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d3006cb40 0x7f6d300b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 -- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d3006cb40 msgr2=0x7f6d300b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:17.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d3006cb40 0x7f6d300b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:17.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.853+0000 7f6d2f7fe640 1 -- 172.21.15.119:0/279895830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d18009400 con 0x7f6d3015d9f0 2023-11-08T19:27:17.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.854+0000 7f6d2f7fe640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300b21a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800caa0 tx=0x7f6d2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.854+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d2800ac00 con 0x7f6d3015d9f0 2023-11-08T19:27:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.854+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d2800ad60 con 0x7f6d3015d9f0 2023-11-08T19:27:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.854+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28012620 con 0x7f6d3015d9f0 2023-11-08T19:27:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.854+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d30070100 con 0x7f6d3015d9f0 2023-11-08T19:27:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.854+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d300b4580 con 0x7f6d3015d9f0 2023-11-08T19:27:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.855+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d30070620 con 0x7f6d3015d9f0 2023-11-08T19:27:17.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.856+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d28007070 con 0x7f6d3015d9f0 2023-11-08T19:27:17.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.857+0000 7f6d27fff640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d1006ae70 0x7f6d1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.857+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6d280899d0 con 0x7f6d3015d9f0 2023-11-08T19:27:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.857+0000 7f6d2effd640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d1006ae70 0x7f6d1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.858+0000 7f6d2effd640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d1006ae70 0x7f6d1006d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:17.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:17.862+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d30070620 con 0x7f6d3015d9f0 2023-11-08T19:27:18.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.087+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d30070620 con 0x7f6d3015d9f0 2023-11-08T19:27:18.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.089+0000 7f6d27fff640 1 -- 172.21.15.119:0/279895830 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f6d30070620 con 0x7f6d3015d9f0 2023-11-08T19:27:18.091 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:18.091 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:04.329003+0000","modified":"2023-11-08T19:27:07.395675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/976565354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":976565354},{"type":"v1","addr":"172.21.15.119:6837","nonce":976565354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:27:18.092 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-11-08T19:27:18.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d1006ae70 msgr2=0x7f6d1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d1006ae70 0x7f6d1006d330 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d20006fd0 tx=0x7f6d20008040 comp rx=0 tx=0).stop 2023-11-08T19:27:18.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 msgr2=0x7f6d300b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300b21a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800caa0 tx=0x7f6d2800cf60 comp rx=0 tx=0).stop 2023-11-08T19:27:18.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 shutdown_connections 2023-11-08T19:27:18.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d1006ae70 0x7f6d1006d330 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d3015d9f0 0x7f6d300b21a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d30142f80 0x7f6d300b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 --2- 172.21.15.119:0/279895830 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d3006cb40 0x7f6d300b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 >> 172.21.15.119:0/279895830 conn(0x7f6d300cb610 msgr2=0x7f6d3015c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.091+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 shutdown_connections 2023-11-08T19:27:18.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.092+0000 7f6d35607640 1 -- 172.21.15.119:0/279895830 wait complete. 2023-11-08T19:27:18.114 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:18.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.332+0000 7fe5cc26f640 1 Processor -- start 2023-11-08T19:27:18.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.333+0000 7fe5cc26f640 1 -- start start 2023-11-08T19:27:18.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.334+0000 7fe5cc26f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 0x7fe5c415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.334+0000 7fe5cc26f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5c40cad30 0x7fe5c406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.334+0000 7fe5cc26f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40c3a50 con 0x7fe5c40cb170 2023-11-08T19:27:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.334+0000 7fe5cc26f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c4144b80 con 0x7fe5c40cad30 2023-11-08T19:27:18.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.334+0000 7fe5cc26f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40c9150 con 0x7fe5c41447a0 2023-11-08T19:27:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.335+0000 7fe5ca00b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 0x7fe5c415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.335+0000 7fe5ca00b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 0x7fe5c415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:34770/0 (socket says 172.21.15.119:34770) 2023-11-08T19:27:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.335+0000 7fe5ca00b640 1 -- 172.21.15.119:0/2690474146 learned_addr learned my addr 172.21.15.119:0/2690474146 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:18.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.335+0000 7fe5c980a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe5c40cad30 0x7fe5c406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.336+0000 7fe5ca00b640 1 -- 172.21.15.119:0/2690474146 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40cad30 msgr2=0x7fe5c406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.336+0000 7fe5ca00b640 1 --2- 172.21.15.119:0/2690474146 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40cad30 0x7fe5c406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.336+0000 7fe5ca00b640 1 -- 172.21.15.119:0/2690474146 >> v1:172.21.15.188:6789/0 conn(0x7fe5c40cb170 legacy=0x7fe5c406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.336+0000 7fe5ca00b640 1 -- 172.21.15.119:0/2690474146 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40c9410 con 0x7fe5c41447a0 2023-11-08T19:27:18.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.336+0000 7fe5c9009640 1 -- 172.21.15.119:0/2690474146 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144863739 0 0) 0x7fe5c40c3a50 con 0x7fe5c40cb170 2023-11-08T19:27:18.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.337+0000 7fe5ca00b640 1 --2- 172.21.15.119:0/2690474146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 0x7fe5c415f4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009870 tx=0x7fe5c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da2b5c5b40ceb3a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.338+0000 7fe5c9009640 1 -- 172.21.15.119:0/2690474146 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0030840 con 0x7fe5c41447a0 2023-11-08T19:27:18.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.338+0000 7fe5c9009640 1 -- 172.21.15.119:0/2690474146 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c00309a0 con 0x7fe5c41447a0 2023-11-08T19:27:18.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.338+0000 7fe5c9009640 1 -- 172.21.15.119:0/2690474146 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0030d30 con 0x7fe5c41447a0 2023-11-08T19:27:18.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.338+0000 7fe5cc26f640 1 -- 172.21.15.119:0/2690474146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 msgr2=0x7fe5c415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.338+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/2690474146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 0x7fe5c415f4c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009870 tx=0x7fe5c002b980 comp rx=0 tx=0).stop 2023-11-08T19:27:18.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.339+0000 7fe5cc26f640 1 -- 172.21.15.119:0/2690474146 shutdown_connections 2023-11-08T19:27:18.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.339+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/2690474146 >> v2:172.21.15.188:3300/0 conn(0x7fe5c40cad30 0x7fe5c406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.339+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/2690474146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c41447a0 0x7fe5c415f4c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.339+0000 7fe5cc26f640 1 -- 172.21.15.119:0/2690474146 >> 172.21.15.119:0/2690474146 conn(0x7fe5c40cb610 msgr2=0x7fe5c40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:18.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.339+0000 7fe5cc26f640 1 -- 172.21.15.119:0/2690474146 shutdown_connections 2023-11-08T19:27:18.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.339+0000 7fe5cc26f640 1 -- 172.21.15.119:0/2690474146 wait complete. 2023-11-08T19:27:18.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.340+0000 7fe5cc26f640 1 Processor -- start 2023-11-08T19:27:18.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.340+0000 7fe5cc26f640 1 -- start start 2023-11-08T19:27:18.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.341+0000 7fe5cc26f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 0x7fe5c415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.341+0000 7fe5cc26f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40cb170 0x7fe5c40b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.341+0000 7fe5cc26f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c41447a0 0x7fe5c40c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.341+0000 7fe5cc26f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40ce090 con 0x7fe5c40cad30 2023-11-08T19:27:18.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.341+0000 7fe5cc26f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40b3df0 con 0x7fe5c41447a0 2023-11-08T19:27:18.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.341+0000 7fe5cc26f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40b47c0 con 0x7fe5c40cb170 2023-11-08T19:27:18.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 0x7fe5c415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 0x7fe5c415a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50172/0 (socket says 172.21.15.119:50172) 2023-11-08T19:27:18.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 -- 172.21.15.119:0/1361672833 learned_addr learned my addr 172.21.15.119:0/1361672833 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:18.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca80c640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c41447a0 0x7fe5c40c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5c980a640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40cb170 0x7fe5c40b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 -- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40cb170 msgr2=0x7fe5c40b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40cb170 0x7fe5c40b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 -- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c41447a0 msgr2=0x7fe5c40c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c41447a0 0x7fe5c40c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca00b640 1 -- 172.21.15.119:0/1361672833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00094d0 con 0x7fe5c40cad30 2023-11-08T19:27:18.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5ca80c640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c41447a0 0x7fe5c40c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:18.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.342+0000 7fe5c980a640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40cb170 0x7fe5c40b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.343+0000 7fe5ca00b640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 0x7fe5c415a080 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00090b0 tx=0x7fe5c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.343+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c003e070 con 0x7fe5c40cad30 2023-11-08T19:27:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.343+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c0039800 con 0x7fe5c40cad30 2023-11-08T19:27:18.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.343+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c40b31e0 con 0x7fe5c40cad30 2023-11-08T19:27:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.343+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0039bb0 con 0x7fe5c40cad30 2023-11-08T19:27:18.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.343+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c40b37a0 con 0x7fe5c40cad30 2023-11-08T19:27:18.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.344+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c4074430 con 0x7fe5c40cad30 2023-11-08T19:27:18.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.345+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5c0042070 con 0x7fe5c40cad30 2023-11-08T19:27:18.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.346+0000 7fe5baffd640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe5a806ae70 0x7fe5a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.346+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe5c00ab500 con 0x7fe5c40cad30 2023-11-08T19:27:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.346+0000 7fe5c980a640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe5a806ae70 0x7fe5a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.347+0000 7fe5c980a640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe5a806ae70 0x7fe5a806d330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40ce8b0 tx=0x7fe5b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.350+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5c0076270 con 0x7fe5c40cad30 2023-11-08T19:27:18.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.546+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5c406caa0 con 0x7fe5c40cad30 2023-11-08T19:27:18.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.549+0000 7fe5baffd640 1 -- 172.21.15.119:0/1361672833 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18523 (secure 0 0 0) 0x7fe5c0079890 con 0x7fe5c40cad30 2023-11-08T19:27:18.552 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:18.552 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:05.896221+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:18.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe5a806ae70 msgr2=0x7fe5a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe5a806ae70 0x7fe5a806d330 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40ce8b0 tx=0x7fe5b4008040 comp rx=0 tx=0).stop 2023-11-08T19:27:18.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 msgr2=0x7fe5c415a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 0x7fe5c415a080 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00090b0 tx=0x7fe5c00046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 shutdown_connections 2023-11-08T19:27:18.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe5a806ae70 0x7fe5a806d330 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe5c41447a0 0x7fe5c40c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe5c40cb170 0x7fe5c40b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 --2- 172.21.15.119:0/1361672833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe5c40cad30 0x7fe5c415a080 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 >> 172.21.15.119:0/1361672833 conn(0x7fe5c40cb610 msgr2=0x7fe5c40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:18.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 shutdown_connections 2023-11-08T19:27:18.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.552+0000 7fe5cc26f640 1 -- 172.21.15.119:0/1361672833 wait complete. 2023-11-08T19:27:18.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:27:02.460658+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:27:03.346355+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:18.573 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:18.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.789+0000 7fc5bf31d640 1 Processor -- start 2023-11-08T19:27:18.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.789+0000 7fc5bf31d640 1 -- start start 2023-11-08T19:27:18.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bf31d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5b8143600 0x7fc5b80c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bf31d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 0x7fc5b815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bf31d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b815e440 con 0x7fc5b815d9f0 2023-11-08T19:27:18.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bf31d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b80c4f50 con 0x7fc5b8143600 2023-11-08T19:27:18.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bf31d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80c4960 con 0x7fc5b806d250 2023-11-08T19:27:18.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bd8ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 0x7fc5b815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bd8ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 0x7fc5b815de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:37232/0 (socket says 172.21.15.119:37232) 2023-11-08T19:27:18.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bd8ba640 1 -- 172.21.15.119:0/1942685456 learned_addr learned my addr 172.21.15.119:0/1942685456 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:18.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bc8b8640 1 --1- 172.21.15.119:0/1942685456 >> v1:172.21.15.188:6789/0 conn(0x7fc5b815d9f0 0x7fc5b806cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:40442/0 (socket says 172.21.15.119:40442) 2023-11-08T19:27:18.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.791+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/1942685456 >> v2:172.21.15.188:3300/0 conn(0x7fc5b8143600 0x7fc5b80c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.792+0000 7fc5bd0b9640 1 -- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 msgr2=0x7fc5b815de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:18.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.792+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 0x7fc5b815de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.792+0000 7fc5bd0b9640 1 -- 172.21.15.119:0/1942685456 >> v1:172.21.15.188:6789/0 conn(0x7fc5b815d9f0 legacy=0x7fc5b806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.792+0000 7fc5bd0b9640 1 -- 172.21.15.119:0/1942685456 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b815e720 con 0x7fc5b8143600 2023-11-08T19:27:18.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.792+0000 7fc5bd8ba640 1 --2- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 0x7fc5b815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:27:18.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.793+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8143600 0x7fc5b80c5370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40097a0 tx=0x7fc5b402da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81708d9602feffad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.794+0000 7fc5affff640 1 -- 172.21.15.119:0/1942685456 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4031850 con 0x7fc5b8143600 2023-11-08T19:27:18.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.794+0000 7fc5affff640 1 -- 172.21.15.119:0/1942685456 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b40319b0 con 0x7fc5b8143600 2023-11-08T19:27:18.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.794+0000 7fc5affff640 1 -- 172.21.15.119:0/1942685456 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b4031d40 con 0x7fc5b8143600 2023-11-08T19:27:18.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.794+0000 7fc5bf31d640 1 -- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8143600 msgr2=0x7fc5b80c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.794+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8143600 0x7fc5b80c5370 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc5b40097a0 tx=0x7fc5b402da30 comp rx=0 tx=0).stop 2023-11-08T19:27:18.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.795+0000 7fc5bf31d640 1 -- 172.21.15.119:0/1942685456 shutdown_connections 2023-11-08T19:27:18.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.795+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b806d250 0x7fc5b815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.795+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/1942685456 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b8143600 0x7fc5b80c5370 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.795+0000 7fc5bf31d640 1 -- 172.21.15.119:0/1942685456 >> 172.21.15.119:0/1942685456 conn(0x7fc5b80cb610 msgr2=0x7fc5b80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:18.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.795+0000 7fc5bf31d640 1 -- 172.21.15.119:0/1942685456 shutdown_connections 2023-11-08T19:27:18.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.795+0000 7fc5bf31d640 1 -- 172.21.15.119:0/1942685456 wait complete. 2023-11-08T19:27:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.796+0000 7fc5bf31d640 1 Processor -- start 2023-11-08T19:27:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.797+0000 7fc5bf31d640 1 -- start start 2023-11-08T19:27:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bf31d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b806d250 0x7fc5b80ac9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bf31d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 0x7fc5b80af560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bf31d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b815d9f0 0x7fc5b813f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bf31d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80acf20 con 0x7fc5b8143600 2023-11-08T19:27:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bf31d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80afaa0 con 0x7fc5b806d250 2023-11-08T19:27:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bf31d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b80afd60 con 0x7fc5b815d9f0 2023-11-08T19:27:18.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bc8b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 0x7fc5b80af560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bc8b8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 0x7fc5b80af560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44334/0 (socket says 172.21.15.119:44334) 2023-11-08T19:27:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bc8b8640 1 -- 172.21.15.119:0/903481326 learned_addr learned my addr 172.21.15.119:0/903481326 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b806d250 0x7fc5b80ac9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bd8ba640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b815d9f0 0x7fc5b813f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.798+0000 7fc5bc8b8640 1 -- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b815d9f0 msgr2=0x7fc5b813f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bc8b8640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b815d9f0 0x7fc5b813f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bc8b8640 1 -- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b806d250 msgr2=0x7fc5b80ac9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bc8b8640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b806d250 0x7fc5b80ac9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:18.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bc8b8640 1 -- 172.21.15.119:0/903481326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b0009b30 con 0x7fc5b8143600 2023-11-08T19:27:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b806d250 0x7fc5b80ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bc8b8640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 0x7fc5b80af560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800c3e0 tx=0x7fc5a800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80074e0 con 0x7fc5b8143600 2023-11-08T19:27:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a8007640 con 0x7fc5b8143600 2023-11-08T19:27:18.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a80049e0 con 0x7fc5b8143600 2023-11-08T19:27:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b4009400 con 0x7fc5b8143600 2023-11-08T19:27:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.799+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b80b0320 con 0x7fc5b8143600 2023-11-08T19:27:18.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.801+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b8074430 con 0x7fc5b8143600 2023-11-08T19:27:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.802+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc5a8004b40 con 0x7fc5b8143600 2023-11-08T19:27:18.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.803+0000 7fc5adffb640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc59c06ae70 0x7fc59c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.803+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc5a8089350 con 0x7fc5b8143600 2023-11-08T19:27:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.803+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc59c06ae70 0x7fc59c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.804+0000 7fc5bd0b9640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc59c06ae70 0x7fc59c06d330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4038000 tx=0x7fc5b403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:18.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:18.808+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc5a80540c0 con 0x7fc5b8143600 2023-11-08T19:27:19.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.000+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5b80c4230 con 0x7fc5b8143600 2023-11-08T19:27:19.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.001+0000 7fc5adffb640 1 -- 172.21.15.119:0/903481326 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5a80576e0 con 0x7fc5b8143600 2023-11-08T19:27:19.004 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc59c06ae70 msgr2=0x7fc59c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc59c06ae70 0x7fc59c06d330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4038000 tx=0x7fc5b403b040 comp rx=0 tx=0).stop 2023-11-08T19:27:19.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 msgr2=0x7fc5b80af560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 0x7fc5b80af560 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc5a800c3e0 tx=0x7fc5a800c8a0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 shutdown_connections 2023-11-08T19:27:19.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc59c06ae70 0x7fc59c06d330 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5b815d9f0 0x7fc5b813f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc5b8143600 0x7fc5b80af560 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 --2- 172.21.15.119:0/903481326 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5b806d250 0x7fc5b80ac9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 >> 172.21.15.119:0/903481326 conn(0x7fc5b80cb610 msgr2=0x7fc5b8142810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:19.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 shutdown_connections 2023-11-08T19:27:19.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.005+0000 7fc5bf31d640 1 -- 172.21.15.119:0/903481326 wait complete. 2023-11-08T19:27:19.026 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:19.240 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:19.246 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.242+0000 7f4250365640 1 Processor -- start 2023-11-08T19:27:19.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.242+0000 7f4250365640 1 -- start start 2023-11-08T19:27:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.243+0000 7f4250365640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4248143600 0x7f42480c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.243+0000 7f4250365640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f424815d9f0 0x7f424815de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.243+0000 7f4250365640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f424815e440 con 0x7f424806cb40 2023-11-08T19:27:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.243+0000 7f4250365640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42480c4f50 con 0x7f4248143600 2023-11-08T19:27:19.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.243+0000 7f4250365640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42480c4960 con 0x7f424815d9f0 2023-11-08T19:27:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.244+0000 7f424e902640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f424806cb40 0x7f424806cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53486/0 (socket says 172.21.15.119:53486) 2023-11-08T19:27:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.244+0000 7f424e902640 1 -- 172.21.15.119:0/4162902415 learned_addr learned my addr 172.21.15.119:0/4162902415 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.244+0000 7f424e101640 1 --2- 172.21.15.119:0/4162902415 >> v2:172.21.15.119:3300/0 conn(0x7f4248143600 0x7f42480c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.244+0000 7f424d900640 1 --2- 172.21.15.119:0/4162902415 >> v2:172.21.15.188:3300/0 conn(0x7f424815d9f0 0x7f424815de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.244+0000 7f424d0ff640 1 -- 172.21.15.119:0/4162902415 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267101737 0 0) 0x7f424815e440 con 0x7f424806cb40 2023-11-08T19:27:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.244+0000 7f424d0ff640 1 -- 172.21.15.119:0/4162902415 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4234003520 con 0x7f424806cb40 2023-11-08T19:27:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.245+0000 7f424e101640 1 -- 172.21.15.119:0/4162902415 >> v2:172.21.15.188:3300/0 conn(0x7f424815d9f0 msgr2=0x7f424815de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.245+0000 7f424e101640 1 --2- 172.21.15.119:0/4162902415 >> v2:172.21.15.188:3300/0 conn(0x7f424815d9f0 0x7f424815de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.245+0000 7f424e101640 1 -- 172.21.15.119:0/4162902415 >> v1:172.21.15.119:6789/0 conn(0x7f424806cb40 legacy=0x7f424806cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.245+0000 7f424e101640 1 -- 172.21.15.119:0/4162902415 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424815e720 con 0x7f4248143600 2023-11-08T19:27:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.246+0000 7f424e101640 1 --2- 172.21.15.119:0/4162902415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4248143600 0x7f42480c5370 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f42480c91e0 tx=0x7f424402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9ad2554a65abe1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.246+0000 7f424d0ff640 1 -- 172.21.15.119:0/4162902415 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424402e070 con 0x7f4248143600 2023-11-08T19:27:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.246+0000 7f424d0ff640 1 -- 172.21.15.119:0/4162902415 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42440315a0 con 0x7f4248143600 2023-11-08T19:27:19.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.246+0000 7f424d0ff640 1 -- 172.21.15.119:0/4162902415 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4244031910 con 0x7f4248143600 2023-11-08T19:27:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 -- 172.21.15.119:0/4162902415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4248143600 msgr2=0x7f42480c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 --2- 172.21.15.119:0/4162902415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4248143600 0x7f42480c5370 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f42480c91e0 tx=0x7f424402b840 comp rx=0 tx=0).stop 2023-11-08T19:27:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 -- 172.21.15.119:0/4162902415 shutdown_connections 2023-11-08T19:27:19.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 --2- 172.21.15.119:0/4162902415 >> v2:172.21.15.188:3300/0 conn(0x7f424815d9f0 0x7f424815de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 --2- 172.21.15.119:0/4162902415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4248143600 0x7f42480c5370 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 -- 172.21.15.119:0/4162902415 >> 172.21.15.119:0/4162902415 conn(0x7f42480cb610 msgr2=0x7f42480cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 -- 172.21.15.119:0/4162902415 shutdown_connections 2023-11-08T19:27:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.247+0000 7f4250365640 1 -- 172.21.15.119:0/4162902415 wait complete. 2023-11-08T19:27:19.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.249+0000 7f4250365640 1 Processor -- start 2023-11-08T19:27:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.249+0000 7f4250365640 1 -- start start 2023-11-08T19:27:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f4250365640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f424806cb40 0x7f42481428f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f4250365640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4248143600 0x7f42481383b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f4250365640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 0x7f424806ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f4250365640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480c8a10 con 0x7f424815d9f0 2023-11-08T19:27:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f4250365640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42480c6710 con 0x7f4248143600 2023-11-08T19:27:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f4250365640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4248142e30 con 0x7f424806cb40 2023-11-08T19:27:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 0x7f424806ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 0x7f424806ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44356/0 (socket says 172.21.15.119:44356) 2023-11-08T19:27:19.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 -- 172.21.15.119:0/565474680 learned_addr learned my addr 172.21.15.119:0/565474680 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424d900640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4248143600 0x7f42481383b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e101640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f424806cb40 0x7f42481428f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 -- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f424806cb40 msgr2=0x7f42481428f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f424806cb40 0x7f42481428f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 -- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4248143600 msgr2=0x7f42481383b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4248143600 0x7f42481383b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.250+0000 7f424e902640 1 -- 172.21.15.119:0/565474680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4244009400 con 0x7f424815d9f0 2023-11-08T19:27:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f424d900640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4248143600 0x7f42481383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:19.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f424e902640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 0x7f424806ba30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f424000caa0 tx=0x7f424000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424000ac00 con 0x7f424815d9f0 2023-11-08T19:27:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424000ad60 con 0x7f424815d9f0 2023-11-08T19:27:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42480c8730 con 0x7f424815d9f0 2023-11-08T19:27:19.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240012620 con 0x7f424815d9f0 2023-11-08T19:27:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.251+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42480c6c90 con 0x7f424815d9f0 2023-11-08T19:27:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.252+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4248074430 con 0x7f424815d9f0 2023-11-08T19:27:19.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.254+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4240007070 con 0x7f424815d9f0 2023-11-08T19:27:19.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.254+0000 7f423affd640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f422c06ae70 0x7f422c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.255+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4240089870 con 0x7f424815d9f0 2023-11-08T19:27:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.255+0000 7f424e101640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f422c06ae70 0x7f422c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.256+0000 7f424e101640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f422c06ae70 0x7f422c06d330 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4244036000 tx=0x7f424403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.259+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4240054670 con 0x7f424815d9f0 2023-11-08T19:27:19.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.454+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f42480b4b60 con 0x7f424815d9f0 2023-11-08T19:27:19.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.454+0000 7f423affd640 1 -- 172.21.15.119:0/565474680 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v54) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4240017020 con 0x7f424815d9f0 2023-11-08T19:27:19.456 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f422c06ae70 msgr2=0x7f422c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f422c06ae70 0x7f422c06d330 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f4244036000 tx=0x7f424403d040 comp rx=0 tx=0).stop 2023-11-08T19:27:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 msgr2=0x7f424806ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 0x7f424806ba30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f424000caa0 tx=0x7f424000cf60 comp rx=0 tx=0).stop 2023-11-08T19:27:19.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 shutdown_connections 2023-11-08T19:27:19.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f422c06ae70 0x7f422c06d330 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f424815d9f0 0x7f424806ba30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4248143600 0x7f42481383b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 --2- 172.21.15.119:0/565474680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f424806cb40 0x7f42481428f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 >> 172.21.15.119:0/565474680 conn(0x7f42480cb610 msgr2=0x7f42480c5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:19.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 shutdown_connections 2023-11-08T19:27:19.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.457+0000 7f4250365640 1 -- 172.21.15.119:0/565474680 wait complete. 2023-11-08T19:27:19.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:19.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.686+0000 7f0902e7c640 1 Processor -- start 2023-11-08T19:27:19.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.686+0000 7f0902e7c640 1 -- start start 2023-11-08T19:27:19.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0902e7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d0f90 0x7f08fc0d0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0902e7c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f08fc0cec10 0x7f08fc0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0902e7c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08fc0d1370 0x7f08fc0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0902e7c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0d0260 con 0x7f08fc0d1370 2023-11-08T19:27:19.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0902e7c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08fc0cf3d0 con 0x7f08fc0cec10 2023-11-08T19:27:19.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0902e7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0cf050 con 0x7f08fc0d0f90 2023-11-08T19:27:19.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.687+0000 7f0901419640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08fc0d1370 0x7f08fc0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f08fc0d1370 0x7f08fc0d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44364/0 (socket says 172.21.15.119:44364) 2023-11-08T19:27:19.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 -- 172.21.15.119:0/1408392303 learned_addr learned my addr 172.21.15.119:0/1408392303 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:19.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0900c18640 1 --2- 172.21.15.119:0/1408392303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d0f90 0x7f08fc0d0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f08fbfff640 1 --2- 172.21.15.119:0/1408392303 >> v2:172.21.15.188:3300/0 conn(0x7f08fc0cec10 0x7f08fc0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 -- 172.21.15.119:0/1408392303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d0f90 msgr2=0x7f08fc0d0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:19.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 --2- 172.21.15.119:0/1408392303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d0f90 0x7f08fc0d0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 -- 172.21.15.119:0/1408392303 >> v2:172.21.15.188:3300/0 conn(0x7f08fc0cec10 msgr2=0x7f08fc0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:19.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 --2- 172.21.15.119:0/1408392303 >> v2:172.21.15.188:3300/0 conn(0x7f08fc0cec10 0x7f08fc0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f0901419640 1 -- 172.21.15.119:0/1408392303 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08fc160c60 con 0x7f08fc0d1370 2023-11-08T19:27:19.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.688+0000 7f08fbfff640 1 --2- 172.21.15.119:0/1408392303 >> v2:172.21.15.188:3300/0 conn(0x7f08fc0cec10 0x7f08fc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:19.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.689+0000 7f0901419640 1 --2- 172.21.15.119:0/1408392303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0d1370 0x7f08fc0d17a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f08fc06d0f0 tx=0x7f08f402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e6e16c081ad3fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.689+0000 7f08fb7fe640 1 -- 172.21.15.119:0/1408392303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f402c950 con 0x7f08fc0d1370 2023-11-08T19:27:19.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.689+0000 7f08fb7fe640 1 -- 172.21.15.119:0/1408392303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f402cab0 con 0x7f08fc0d1370 2023-11-08T19:27:19.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.689+0000 7f08fb7fe640 1 -- 172.21.15.119:0/1408392303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f402ce40 con 0x7f08fc0d1370 2023-11-08T19:27:19.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 -- 172.21.15.119:0/1408392303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0d1370 msgr2=0x7f08fc0d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 --2- 172.21.15.119:0/1408392303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0d1370 0x7f08fc0d17a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f08fc06d0f0 tx=0x7f08f402bdb0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 -- 172.21.15.119:0/1408392303 shutdown_connections 2023-11-08T19:27:19.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 --2- 172.21.15.119:0/1408392303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0d1370 0x7f08fc0d17a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 --2- 172.21.15.119:0/1408392303 >> v2:172.21.15.188:3300/0 conn(0x7f08fc0cec10 0x7f08fc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 --2- 172.21.15.119:0/1408392303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d0f90 0x7f08fc0d0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 -- 172.21.15.119:0/1408392303 >> 172.21.15.119:0/1408392303 conn(0x7f08fc0d3090 msgr2=0x7f08fc0d34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:19.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 -- 172.21.15.119:0/1408392303 shutdown_connections 2023-11-08T19:27:19.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.690+0000 7f0902e7c640 1 -- 172.21.15.119:0/1408392303 wait complete. 2023-11-08T19:27:19.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.692+0000 7f0902e7c640 1 Processor -- start 2023-11-08T19:27:19.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.692+0000 7f0902e7c640 1 -- start start 2023-11-08T19:27:19.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.693+0000 7f0902e7c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 0x7f08fc1f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.693+0000 7f0902e7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08fc0d0f90 0x7f08fc0c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.693+0000 7f0902e7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d1370 0x7f08fc0cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.693+0000 7f0902e7c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0caed0 con 0x7f08fc0cec10 2023-11-08T19:27:19.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.693+0000 7f0902e7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc14f320 con 0x7f08fc0d0f90 2023-11-08T19:27:19.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.693+0000 7f0902e7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08fc0c6c80 con 0x7f08fc0d1370 2023-11-08T19:27:19.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 0x7f08fc1f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 0x7f08fc1f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44366/0 (socket says 172.21.15.119:44366) 2023-11-08T19:27:19.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 -- 172.21.15.119:0/658945381 learned_addr learned my addr 172.21.15.119:0/658945381 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:19.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 -- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d1370 msgr2=0x7f08fc0cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d1370 0x7f08fc0cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 -- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08fc0d0f90 msgr2=0x7f08fc0c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f08fbfff640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08fc0d0f90 0x7f08fc0c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08fc0d0f90 0x7f08fc0c7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 -- 172.21.15.119:0/658945381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4009b30 con 0x7f08fc0cec10 2023-11-08T19:27:19.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.694+0000 7f0900c18640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 0x7f08fc1f5d90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c7f0 tx=0x7f08f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.695+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0004220 con 0x7f08fc0cec10 2023-11-08T19:27:19.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.695+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f0004380 con 0x7f08fc0cec10 2023-11-08T19:27:19.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.695+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0007840 con 0x7f08fc0cec10 2023-11-08T19:27:19.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.695+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08fc0c6f60 con 0x7f08fc0cec10 2023-11-08T19:27:19.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.695+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08fc0c7520 con 0x7f08fc0cec10 2023-11-08T19:27:19.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.696+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08fc074430 con 0x7f08fc0cec10 2023-11-08T19:27:19.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.697+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08f0007bb0 con 0x7f08fc0cec10 2023-11-08T19:27:19.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.698+0000 7f08f97fa640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f08e006ae70 0x7f08e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:19.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.698+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f08f0089780 con 0x7f08fc0cec10 2023-11-08T19:27:19.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.698+0000 7f08fbfff640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f08e006ae70 0x7f08e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:19.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.699+0000 7f08fbfff640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f08e006ae70 0x7f08e006d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f08fc14ea60 tx=0x7f08e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:19.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.702+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08f0054620 con 0x7f08fc0cec10 2023-11-08T19:27:19.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.891+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f08fc0b8e50 con 0x7f08fc0cec10 2023-11-08T19:27:19.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.892+0000 7f08f97fa640 1 -- 172.21.15.119:0/658945381 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f08f005e070 con 0x7f08fc0cec10 2023-11-08T19:27:19.893 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:19.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f08e006ae70 msgr2=0x7f08e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f08e006ae70 0x7f08e006d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f08fc14ea60 tx=0x7f08e8008040 comp rx=0 tx=0).stop 2023-11-08T19:27:19.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 msgr2=0x7f08fc1f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:19.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 0x7f08fc1f5d90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c7f0 tx=0x7f08f000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 shutdown_connections 2023-11-08T19:27:19.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f08e006ae70 0x7f08e006d330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f08fc0d1370 0x7f08fc0cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f08fc0d0f90 0x7f08fc0c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 --2- 172.21.15.119:0/658945381 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f08fc0cec10 0x7f08fc1f5d90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:19.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 >> 172.21.15.119:0/658945381 conn(0x7f08fc0d3090 msgr2=0x7f08fc15e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:19.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 shutdown_connections 2023-11-08T19:27:19.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:19.895+0000 7f0902e7c640 1 -- 172.21.15.119:0/658945381 wait complete. 2023-11-08T19:27:19.916 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:20.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.130+0000 7f1e7629f640 1 Processor -- start 2023-11-08T19:27:20.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.131+0000 7f1e7629f640 1 -- start start 2023-11-08T19:27:20.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.132+0000 7f1e7629f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e70160530 0x7f1e700cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.132+0000 7f1e7629f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1e70160910 0x7f1e700cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.132+0000 7f1e7629f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e700b27c0 con 0x7f1e700d3ac0 2023-11-08T19:27:20.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.132+0000 7f1e7629f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e700b2aa0 con 0x7f1e70160530 2023-11-08T19:27:20.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.132+0000 7f1e7629f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e700d1640 con 0x7f1e70160910 2023-11-08T19:27:20.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e70160530 0x7f1e700cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1e70160530 0x7f1e700cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44382/0 (socket says 172.21.15.119:44382) 2023-11-08T19:27:20.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 -- 172.21.15.119:0/2788380286 learned_addr learned my addr 172.21.15.119:0/2788380286 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:20.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e75a9e640 1 --1- 172.21.15.119:0/2788380286 >> v1:172.21.15.119:6789/0 conn(0x7f1e700d3ac0 0x7f1e700cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53498/0 (socket says 172.21.15.119:53498) 2023-11-08T19:27:20.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e74a9c640 1 --2- 172.21.15.119:0/2788380286 >> v2:172.21.15.188:3300/0 conn(0x7f1e70160910 0x7f1e700cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 -- 172.21.15.119:0/2788380286 >> v2:172.21.15.188:3300/0 conn(0x7f1e70160910 msgr2=0x7f1e700cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 --2- 172.21.15.119:0/2788380286 >> v2:172.21.15.188:3300/0 conn(0x7f1e70160910 0x7f1e700cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 -- 172.21.15.119:0/2788380286 >> v1:172.21.15.119:6789/0 conn(0x7f1e700d3ac0 legacy=0x7f1e700cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.133+0000 7f1e7529d640 1 -- 172.21.15.119:0/2788380286 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e700d1920 con 0x7f1e70160530 2023-11-08T19:27:20.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.134+0000 7f1e7529d640 1 --2- 172.21.15.119:0/2788380286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e70160530 0x7f1e700cd960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1e700d2e90 tx=0x7f1e6002bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2beb0fc97b075d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.134+0000 7f1e6ffff640 1 -- 172.21.15.119:0/2788380286 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6002cb60 con 0x7f1e70160530 2023-11-08T19:27:20.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e6ffff640 1 -- 172.21.15.119:0/2788380286 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6002ccc0 con 0x7f1e70160530 2023-11-08T19:27:20.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e6ffff640 1 -- 172.21.15.119:0/2788380286 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60034610 con 0x7f1e70160530 2023-11-08T19:27:20.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 -- 172.21.15.119:0/2788380286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e70160530 msgr2=0x7f1e700cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 --2- 172.21.15.119:0/2788380286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e70160530 0x7f1e700cd960 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1e700d2e90 tx=0x7f1e6002bfc0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 -- 172.21.15.119:0/2788380286 shutdown_connections 2023-11-08T19:27:20.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 --2- 172.21.15.119:0/2788380286 >> v2:172.21.15.188:3300/0 conn(0x7f1e70160910 0x7f1e700cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 --2- 172.21.15.119:0/2788380286 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e70160530 0x7f1e700cd960 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 -- 172.21.15.119:0/2788380286 >> 172.21.15.119:0/2788380286 conn(0x7f1e700d2ec0 msgr2=0x7f1e700d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:20.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 -- 172.21.15.119:0/2788380286 shutdown_connections 2023-11-08T19:27:20.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.135+0000 7f1e7629f640 1 -- 172.21.15.119:0/2788380286 wait complete. 2023-11-08T19:27:20.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.137+0000 7f1e7629f640 1 Processor -- start 2023-11-08T19:27:20.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.137+0000 7f1e7629f640 1 -- start start 2023-11-08T19:27:20.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7629f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 0x7f1e700b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7629f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70160530 0x7f1e700c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7629f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70160910 0x7f1e700c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7629f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700c6de0 con 0x7f1e700d3ac0 2023-11-08T19:27:20.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7629f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700afb50 con 0x7f1e70160910 2023-11-08T19:27:20.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7629f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e700b0690 con 0x7f1e70160530 2023-11-08T19:27:20.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7529d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 0x7f1e700b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7529d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 0x7f1e700b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44394/0 (socket says 172.21.15.119:44394) 2023-11-08T19:27:20.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.138+0000 7f1e7529d640 1 -- 172.21.15.119:0/249261964 learned_addr learned my addr 172.21.15.119:0/249261964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:20.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e75a9e640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70160910 0x7f1e700c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e7529d640 1 -- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70160530 msgr2=0x7f1e700c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e74a9c640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70160530 0x7f1e700c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e7529d640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70160530 0x7f1e700c9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e7529d640 1 -- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70160910 msgr2=0x7f1e700c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e7529d640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70160910 0x7f1e700c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e7529d640 1 -- 172.21.15.119:0/249261964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e60009b30 con 0x7f1e700d3ac0 2023-11-08T19:27:20.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e7529d640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 0x7f1e700b19f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1e60000f80 tx=0x7f1e60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.139+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e60009510 con 0x7f1e700d3ac0 2023-11-08T19:27:20.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.140+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e6003e450 con 0x7f1e700d3ac0 2023-11-08T19:27:20.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.140+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e6003e800 con 0x7f1e700d3ac0 2023-11-08T19:27:20.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.140+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e700c77e0 con 0x7f1e700d3ac0 2023-11-08T19:27:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.140+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e700b0040 con 0x7f1e700d3ac0 2023-11-08T19:27:20.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.141+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7004ece0 con 0x7f1e700d3ac0 2023-11-08T19:27:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.142+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e6003e960 con 0x7f1e700d3ac0 2023-11-08T19:27:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.143+0000 7f1e6dffb640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e5c06aea0 0x7f1e5c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.143+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1e600ac4d0 con 0x7f1e700d3ac0 2023-11-08T19:27:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.143+0000 7f1e74a9c640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e5c06aea0 0x7f1e5c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.144+0000 7f1e74a9c640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e5c06aea0 0x7f1e5c06d360 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006fd0 tx=0x7f1e64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.147+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e60077240 con 0x7f1e700d3ac0 2023-11-08T19:27:20.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.333+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1e700c7080 con 0x7f1e700d3ac0 2023-11-08T19:27:20.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.334+0000 7f1e6dffb640 1 -- 172.21.15.119:0/249261964 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1e6007a860 con 0x7f1e700d3ac0 2023-11-08T19:27:20.335 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:27:20.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e5c06aea0 msgr2=0x7f1e5c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e5c06aea0 0x7f1e5c06d360 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1e64006fd0 tx=0x7f1e64008040 comp rx=0 tx=0).stop 2023-11-08T19:27:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 msgr2=0x7f1e700b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 0x7f1e700b19f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f1e60000f80 tx=0x7f1e60004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 shutdown_connections 2023-11-08T19:27:20.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1e5c06aea0 0x7f1e5c06d360 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1e70160910 0x7f1e700c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1e70160530 0x7f1e700c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 --2- 172.21.15.119:0/249261964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1e700d3ac0 0x7f1e700b19f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 >> 172.21.15.119:0/249261964 conn(0x7f1e700d2ec0 msgr2=0x7f1e70155300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:20.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 shutdown_connections 2023-11-08T19:27:20.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.336+0000 7f1e7629f640 1 -- 172.21.15.119:0/249261964 wait complete. 2023-11-08T19:27:20.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:20.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.566+0000 7f8236ded640 1 Processor -- start 2023-11-08T19:27:20.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.566+0000 7f8236ded640 1 -- start start 2023-11-08T19:27:20.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f8236ded640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300adf20 0x7f823015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f8236ded640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82300ad9d0 0x7f823015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f8236ded640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82300ca2e0 con 0x7f82300cac30 2023-11-08T19:27:20.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f8236ded640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8230071c70 con 0x7f82300ad9d0 2023-11-08T19:27:20.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f8236ded640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823015b390 con 0x7f82300adf20 2023-11-08T19:27:20.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f82365ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82300ad9d0 0x7f823015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f82365ec640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f82300ad9d0 0x7f823015ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44402/0 (socket says 172.21.15.119:44402) 2023-11-08T19:27:20.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.567+0000 7f82365ec640 1 -- 172.21.15.119:0/2395319329 learned_addr learned my addr 172.21.15.119:0/2395319329 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:20.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.568+0000 7f82355ea640 1 --2- 172.21.15.119:0/2395319329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300adf20 0x7f823015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.568+0000 7f82365ec640 1 -- 172.21.15.119:0/2395319329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300adf20 msgr2=0x7f823015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.568+0000 7f82365ec640 1 --2- 172.21.15.119:0/2395319329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300adf20 0x7f823015a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.568+0000 7f82365ec640 1 -- 172.21.15.119:0/2395319329 >> v1:172.21.15.188:6789/0 conn(0x7f82300cac30 legacy=0x7f823015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:20.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.568+0000 7f82365ec640 1 -- 172.21.15.119:0/2395319329 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82300ca5c0 con 0x7f82300ad9d0 2023-11-08T19:27:20.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.569+0000 7f82365ec640 1 --2- 172.21.15.119:0/2395319329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300ad9d0 0x7f823015ae50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8228009e50 tx=0x7f822802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f74afe12de59b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.569+0000 7f8234de9640 1 -- 172.21.15.119:0/2395319329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822802cb00 con 0x7f82300ad9d0 2023-11-08T19:27:20.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.569+0000 7f8234de9640 1 -- 172.21.15.119:0/2395319329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822802cc60 con 0x7f82300ad9d0 2023-11-08T19:27:20.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.569+0000 7f8234de9640 1 -- 172.21.15.119:0/2395319329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82280343f0 con 0x7f82300ad9d0 2023-11-08T19:27:20.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 -- 172.21.15.119:0/2395319329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300ad9d0 msgr2=0x7f823015ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 --2- 172.21.15.119:0/2395319329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300ad9d0 0x7f823015ae50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f8228009e50 tx=0x7f822802bf60 comp rx=0 tx=0).stop 2023-11-08T19:27:20.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 -- 172.21.15.119:0/2395319329 shutdown_connections 2023-11-08T19:27:20.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 --2- 172.21.15.119:0/2395319329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300ad9d0 0x7f823015ae50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 --2- 172.21.15.119:0/2395319329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300adf20 0x7f823015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 -- 172.21.15.119:0/2395319329 >> 172.21.15.119:0/2395319329 conn(0x7f82300b0560 msgr2=0x7f82300b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:20.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 -- 172.21.15.119:0/2395319329 shutdown_connections 2023-11-08T19:27:20.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.570+0000 7f8236ded640 1 -- 172.21.15.119:0/2395319329 wait complete. 2023-11-08T19:27:20.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.572+0000 7f8236ded640 1 Processor -- start 2023-11-08T19:27:20.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.572+0000 7f8236ded640 1 -- start start 2023-11-08T19:27:20.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8236ded640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300ad9d0 0x7f82301f13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8236ded640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82300adf20 0x7f8230154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8236ded640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 0x7f823014fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8236ded640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82301547d0 con 0x7f82300cac30 2023-11-08T19:27:20.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8236ded640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8230150130 con 0x7f82300adf20 2023-11-08T19:27:20.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8236ded640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8230154a30 con 0x7f82300ad9d0 2023-11-08T19:27:20.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f82365ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 0x7f823014fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f82365ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 0x7f823014fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44418/0 (socket says 172.21.15.119:44418) 2023-11-08T19:27:20.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f82365ec640 1 -- 172.21.15.119:0/1799215821 learned_addr learned my addr 172.21.15.119:0/1799215821 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:20.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f82355ea640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82300adf20 0x7f8230154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f8235deb640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300ad9d0 0x7f82301f13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f82365ec640 1 -- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300ad9d0 msgr2=0x7f82301f13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.573+0000 7f82365ec640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300ad9d0 0x7f82301f13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82365ec640 1 -- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82300adf20 msgr2=0x7f8230154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82365ec640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82300adf20 0x7f8230154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82365ec640 1 -- 172.21.15.119:0/1799215821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8228009b30 con 0x7f82300cac30 2023-11-08T19:27:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82355ea640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82300adf20 0x7f8230154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82365ec640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 0x7f823014fbf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8228009e50 tx=0x7f8228004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8228009510 con 0x7f82300cac30 2023-11-08T19:27:20.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822803e450 con 0x7f82300cac30 2023-11-08T19:27:20.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8230154d10 con 0x7f82300cac30 2023-11-08T19:27:20.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822803e800 con 0x7f82300cac30 2023-11-08T19:27:20.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.574+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f823015cba0 con 0x7f82300cac30 2023-11-08T19:27:20.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.575+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f823004ece0 con 0x7f82300cac30 2023-11-08T19:27:20.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.576+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f822803e960 con 0x7f82300cac30 2023-11-08T19:27:20.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.577+0000 7f82267fc640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f821406aea0 0x7f821406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:20.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.577+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f82280ac7a0 con 0x7f82300cac30 2023-11-08T19:27:20.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.578+0000 7f8235deb640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f821406aea0 0x7f821406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:20.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.578+0000 7f8235deb640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f821406aea0 0x7f821406d360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82300b21d0 tx=0x7f822c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:20.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.582+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82280775c0 con 0x7f82300cac30 2023-11-08T19:27:20.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.799+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82300b1110 con 0x7f82300cac30 2023-11-08T19:27:20.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.800+0000 7f82267fc640 1 -- 172.21.15.119:0/1799215821 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f822807abe0 con 0x7f82300cac30 2023-11-08T19:27:20.802 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:20.802 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:04.329003+0000","modified":"2023-11-08T19:27:07.395675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/976565354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":976565354},{"type":"v1","addr":"172.21.15.119:6837","nonce":976565354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:27:20.802 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-11-08T19:27:20.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f821406aea0 msgr2=0x7f821406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f821406aea0 0x7f821406d360 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82300b21d0 tx=0x7f822c009040 comp rx=0 tx=0).stop 2023-11-08T19:27:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 msgr2=0x7f823014fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 0x7f823014fbf0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f8228009e50 tx=0x7f8228004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 shutdown_connections 2023-11-08T19:27:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f821406aea0 0x7f821406d360 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f82300cac30 0x7f823014fbf0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82300adf20 0x7f8230154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 --2- 172.21.15.119:0/1799215821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82300ad9d0 0x7f82301f13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 >> 172.21.15.119:0/1799215821 conn(0x7f82300b0560 msgr2=0x7f8230159a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 shutdown_connections 2023-11-08T19:27:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:20.803+0000 7f8236ded640 1 -- 172.21.15.119:0/1799215821 wait complete. 2023-11-08T19:27:20.823 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.020+0000 7f0f3412c640 1 Processor -- start 2023-11-08T19:27:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.020+0000 7f0f3412c640 1 -- start start 2023-11-08T19:27:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.021+0000 7f0f3412c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0f2c161900 0x7f0f2c070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.021+0000 7f0f3412c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0f2c06a500 0x7f0f2c06a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.021+0000 7f0f3412c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f2c0cf3e0 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.021+0000 7f0f3412c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c0cfbd0 con 0x7f0f2c06a500 2023-11-08T19:27:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.021+0000 7f0f3412c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2c0cd8f0 con 0x7f0f2c161900 2023-11-08T19:27:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f31ec8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0f2c06ebd0 0x7f0f2c06efb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53512/0 (socket says 172.21.15.119:53512) 2023-11-08T19:27:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f31ec8640 1 -- 172.21.15.119:0/2218037907 learned_addr learned my addr 172.21.15.119:0/2218037907 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f326c9640 1 --2- 172.21.15.119:0/2218037907 >> v2:172.21.15.119:3300/0 conn(0x7f0f2c06a500 0x7f0f2c06a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f316c7640 1 --2- 172.21.15.119:0/2218037907 >> v2:172.21.15.188:3300/0 conn(0x7f0f2c161900 0x7f0f2c070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f30ec6640 1 -- 172.21.15.119:0/2218037907 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312590459 0 0) 0x7f0f2c0cf3e0 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f30ec6640 1 -- 172.21.15.119:0/2218037907 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f14003520 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f326c9640 1 -- 172.21.15.119:0/2218037907 >> v2:172.21.15.188:3300/0 conn(0x7f0f2c161900 msgr2=0x7f0f2c070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f326c9640 1 --2- 172.21.15.119:0/2218037907 >> v2:172.21.15.188:3300/0 conn(0x7f0f2c161900 0x7f0f2c070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f326c9640 1 -- 172.21.15.119:0/2218037907 >> v1:172.21.15.119:6789/0 conn(0x7f0f2c06ebd0 legacy=0x7f0f2c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.022+0000 7f0f326c9640 1 -- 172.21.15.119:0/2218037907 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0cdbd0 con 0x7f0f2c06a500 2023-11-08T19:27:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.023+0000 7f0f326c9640 1 --2- 172.21.15.119:0/2218037907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06a500 0x7f0f2c06a990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c066e00 tx=0x7f0f1c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d0a616e74f8acb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.023+0000 7f0f30ec6640 1 -- 172.21.15.119:0/2218037907 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c02e070 con 0x7f0f2c06a500 2023-11-08T19:27:21.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.023+0000 7f0f30ec6640 1 -- 172.21.15.119:0/2218037907 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1c0315a0 con 0x7f0f2c06a500 2023-11-08T19:27:21.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.023+0000 7f0f30ec6640 1 -- 172.21.15.119:0/2218037907 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1c031930 con 0x7f0f2c06a500 2023-11-08T19:27:21.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 -- 172.21.15.119:0/2218037907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06a500 msgr2=0x7f0f2c06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 --2- 172.21.15.119:0/2218037907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06a500 0x7f0f2c06a990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c066e00 tx=0x7f0f1c02b840 comp rx=0 tx=0).stop 2023-11-08T19:27:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 -- 172.21.15.119:0/2218037907 shutdown_connections 2023-11-08T19:27:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 --2- 172.21.15.119:0/2218037907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06a500 0x7f0f2c06a990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 --2- 172.21.15.119:0/2218037907 >> v2:172.21.15.188:3300/0 conn(0x7f0f2c161900 0x7f0f2c070b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 -- 172.21.15.119:0/2218037907 >> 172.21.15.119:0/2218037907 conn(0x7f0f2c0cfd20 msgr2=0x7f0f2c0d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 -- 172.21.15.119:0/2218037907 shutdown_connections 2023-11-08T19:27:21.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.024+0000 7f0f3412c640 1 -- 172.21.15.119:0/2218037907 wait complete. 2023-11-08T19:27:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.026+0000 7f0f3412c640 1 Processor -- start 2023-11-08T19:27:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.026+0000 7f0f3412c640 1 -- start start 2023-11-08T19:27:21.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f3412c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f2c06a500 0x7f0f2c1f2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f3412c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 0x7f0f2c15efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f3412c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f2c161900 0x7f0f2c15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f3412c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b4d80 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f3412c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b5060 con 0x7f0f2c161900 2023-11-08T19:27:21.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f3412c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2c0b5340 con 0x7f0f2c06a500 2023-11-08T19:27:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 0x7f0f2c15efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 0x7f0f2c15efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44436/0 (socket says 172.21.15.119:44436) 2023-11-08T19:27:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 -- 172.21.15.119:0/984598175 learned_addr learned my addr 172.21.15.119:0/984598175 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 -- 172.21.15.119:0/984598175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f2c06a500 msgr2=0x7f0f2c1f2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f2c06a500 0x7f0f2c1f2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 -- 172.21.15.119:0/984598175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f2c161900 msgr2=0x7f0f2c15f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f2c161900 0x7f0f2c15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.027+0000 7f0f316c7640 1 -- 172.21.15.119:0/984598175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1c009400 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.028+0000 7f0f316c7640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 0x7f0f2c15efd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0f20007e40 tx=0x7f0f2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.028+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20014620 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.028+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2000ee90 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.028+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f200149c0 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.028+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f2c0b5620 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.028+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f2c1f27a0 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.029+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f2c074430 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.030+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0f20014b20 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.031+0000 7f0f2a7fc640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f1806ae70 0x7f0f1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.031+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0f2008a280 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.031+0000 7f0f31ec8640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f1806ae70 0x7f0f1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.032+0000 7f0f31ec8640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f1806ae70 0x7f0f1806d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f24001610 tx=0x7f0f24007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.036+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0f20054ff0 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.224+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f2c0af540 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.227+0000 7f0f2a7fc640 1 -- 172.21.15.119:0/984598175 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18523 (secure 0 0 0) 0x7f0f20058610 con 0x7f0f2c06ebd0 2023-11-08T19:27:21.229 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:21.229 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:05.896221+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:21.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f1806ae70 msgr2=0x7f0f1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f1806ae70 0x7f0f1806d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f0f24001610 tx=0x7f0f24007930 comp rx=0 tx=0).stop 2023-11-08T19:27:21.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 msgr2=0x7f0f2c15efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 0x7f0f2c15efd0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0f20007e40 tx=0x7f0f2000e4b0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 shutdown_connections 2023-11-08T19:27:21.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0f1806ae70 0x7f0f1806d330 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0f2c161900 0x7f0f2c15f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0f2c06ebd0 0x7f0f2c15efd0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 --2- 172.21.15.119:0/984598175 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0f2c06a500 0x7f0f2c1f2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 >> 172.21.15.119:0/984598175 conn(0x7f0f2c0cfd20 msgr2=0x7f0f2c1602f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 shutdown_connections 2023-11-08T19:27:21.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.230+0000 7f0f3412c640 1 -- 172.21.15.119:0/984598175 wait complete. 2023-11-08T19:27:21.249 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:27:21.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:21.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.460+0000 7f4266067640 1 Processor -- start 2023-11-08T19:27:21.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.460+0000 7f4266067640 1 -- start start 2023-11-08T19:27:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.461+0000 7f4266067640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f426015b0d0 0x7f42600ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.461+0000 7f4266067640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f426015b900 0x7f4260070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.461+0000 7f4266067640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42600710a0 0x7f426015da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.461+0000 7f4266067640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426006cff0 con 0x7f426015b0d0 2023-11-08T19:27:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.461+0000 7f4266067640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426006d2d0 con 0x7f426015b900 2023-11-08T19:27:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.461+0000 7f4266067640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426006bd20 con 0x7f42600710a0 2023-11-08T19:27:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f426015b0d0 0x7f42600ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f426015b0d0 0x7f42600ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44438/0 (socket says 172.21.15.119:44438) 2023-11-08T19:27:21.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 -- 172.21.15.119:0/3164640781 learned_addr learned my addr 172.21.15.119:0/3164640781 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 -- 172.21.15.119:0/3164640781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42600710a0 msgr2=0x7f426015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 --2- 172.21.15.119:0/3164640781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42600710a0 0x7f426015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 -- 172.21.15.119:0/3164640781 >> v2:172.21.15.188:3300/0 conn(0x7f426015b900 msgr2=0x7f4260070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 --2- 172.21.15.119:0/3164640781 >> v2:172.21.15.188:3300/0 conn(0x7f426015b900 0x7f4260070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.462+0000 7f4265065640 1 -- 172.21.15.119:0/3164640781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426006d760 con 0x7f426015b0d0 2023-11-08T19:27:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.463+0000 7f4265065640 1 --2- 172.21.15.119:0/3164640781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b0d0 0x7f42600ad7d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f42600c2570 tx=0x7f425c02da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf5c6bdf9c063e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4257fff640 1 -- 172.21.15.119:0/3164640781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c02f850 con 0x7f426015b0d0 2023-11-08T19:27:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4257fff640 1 -- 172.21.15.119:0/3164640781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c02f9b0 con 0x7f426015b0d0 2023-11-08T19:27:21.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4257fff640 1 -- 172.21.15.119:0/3164640781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c02fd40 con 0x7f426015b0d0 2023-11-08T19:27:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 -- 172.21.15.119:0/3164640781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b0d0 msgr2=0x7f42600ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 --2- 172.21.15.119:0/3164640781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b0d0 0x7f42600ad7d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f42600c2570 tx=0x7f425c02da90 comp rx=0 tx=0).stop 2023-11-08T19:27:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 -- 172.21.15.119:0/3164640781 shutdown_connections 2023-11-08T19:27:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 --2- 172.21.15.119:0/3164640781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42600710a0 0x7f426015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 --2- 172.21.15.119:0/3164640781 >> v2:172.21.15.188:3300/0 conn(0x7f426015b900 0x7f4260070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 --2- 172.21.15.119:0/3164640781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b0d0 0x7f42600ad7d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.464+0000 7f4266067640 1 -- 172.21.15.119:0/3164640781 >> 172.21.15.119:0/3164640781 conn(0x7f42600b0560 msgr2=0x7f42600b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.465+0000 7f4266067640 1 -- 172.21.15.119:0/3164640781 shutdown_connections 2023-11-08T19:27:21.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.465+0000 7f4266067640 1 -- 172.21.15.119:0/3164640781 wait complete. 2023-11-08T19:27:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.466+0000 7f4266067640 1 Processor -- start 2023-11-08T19:27:21.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.466+0000 7f4266067640 1 -- start start 2023-11-08T19:27:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.467+0000 7f4266067640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42600710a0 0x7f426014df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.467+0000 7f4266067640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f426015b0d0 0x7f42600c7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.467+0000 7f4266067640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 0x7f42600ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.467+0000 7f4266067640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42600c7970 con 0x7f426015b900 2023-11-08T19:27:21.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.467+0000 7f4266067640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f426014e440 con 0x7f42600710a0 2023-11-08T19:27:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.467+0000 7f4266067640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42600b3d80 con 0x7f426015b0d0 2023-11-08T19:27:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 0x7f42600ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 0x7f42600ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44442/0 (socket says 172.21.15.119:44442) 2023-11-08T19:27:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 -- 172.21.15.119:0/71128365 learned_addr learned my addr 172.21.15.119:0/71128365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:21.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265065640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42600710a0 0x7f426014df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4264864640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f426015b0d0 0x7f42600c7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 -- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f426015b0d0 msgr2=0x7f42600c7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f426015b0d0 0x7f42600c7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 -- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42600710a0 msgr2=0x7f426014df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42600710a0 0x7f426014df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 -- 172.21.15.119:0/71128365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c0094d0 con 0x7f426015b900 2023-11-08T19:27:21.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265065640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42600710a0 0x7f426014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:21.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.468+0000 7f4265866640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 0x7f42600ac8d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f425800b9a0 tx=0x7f425800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.469+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425800cba0 con 0x7f426015b900 2023-11-08T19:27:21.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.469+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425800cd20 con 0x7f426015b900 2023-11-08T19:27:21.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.469+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4258014680 con 0x7f426015b900 2023-11-08T19:27:21.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.469+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42600b4060 con 0x7f426015b900 2023-11-08T19:27:21.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.469+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42600b4340 con 0x7f426015b900 2023-11-08T19:27:21.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.470+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42600c7b20 con 0x7f426015b900 2023-11-08T19:27:21.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.471+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f425801d3b0 con 0x7f426015b900 2023-11-08T19:27:21.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.472+0000 7f4255ffb640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f424406ae70 0x7f424406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.472+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f42580898f0 con 0x7f426015b900 2023-11-08T19:27:21.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.472+0000 7f4265065640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f424406ae70 0x7f424406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.473+0000 7f4265065640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f424406ae70 0x7f424406d330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f425c038000 tx=0x7f425c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.476+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42600c7b20 con 0x7f426015b900 2023-11-08T19:27:21.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.704+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42600c7b20 con 0x7f426015b900 2023-11-08T19:27:21.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.709+0000 7f4255ffb640 1 -- 172.21.15.119:0/71128365 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f42600c7b20 con 0x7f426015b900 2023-11-08T19:27:21.710 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:21.711 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:04.329003+0000","modified":"2023-11-08T19:27:07.395675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/976565354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":976565354},{"type":"v1","addr":"172.21.15.119:6837","nonce":976565354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:27:21.711 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-11-08T19:27:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.711+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f424406ae70 msgr2=0x7f424406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.711+0000 7f4266067640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f424406ae70 0x7f424406d330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f425c038000 tx=0x7f425c008030 comp rx=0 tx=0).stop 2023-11-08T19:27:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.711+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 msgr2=0x7f42600ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.711+0000 7f4266067640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 0x7f42600ac8d0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f425800b9a0 tx=0x7f425800be60 comp rx=0 tx=0).stop 2023-11-08T19:27:21.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 shutdown_connections 2023-11-08T19:27:21.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f424406ae70 0x7f424406d330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f426015b900 0x7f42600ac8d0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f426015b0d0 0x7f42600c7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 --2- 172.21.15.119:0/71128365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f42600710a0 0x7f426014df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 >> 172.21.15.119:0/71128365 conn(0x7f42600b0560 msgr2=0x7f42600cda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 shutdown_connections 2023-11-08T19:27:21.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.712+0000 7f4266067640 1 -- 172.21.15.119:0/71128365 wait complete. 2023-11-08T19:27:21.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.934+0000 7fce343c9640 1 Processor -- start 2023-11-08T19:27:21.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.934+0000 7fce343c9640 1 -- start start 2023-11-08T19:27:21.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce343c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce2c1447a0 0x7fce2c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce343c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce2c0cad30 0x7fce2c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce343c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c0c3a50 con 0x7fce2c06a120 2023-11-08T19:27:21.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce343c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c144b80 con 0x7fce2c0cad30 2023-11-08T19:27:21.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce343c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0c9150 con 0x7fce2c1447a0 2023-11-08T19:27:21.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce2c0cad30 0x7fce2c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce2c0cad30 0x7fce2c06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44446/0 (socket says 172.21.15.119:44446) 2023-11-08T19:27:21.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 -- 172.21.15.119:0/3864530479 learned_addr learned my addr 172.21.15.119:0/3864530479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:21.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 -- 172.21.15.119:0/3864530479 >> v2:172.21.15.188:3300/0 conn(0x7fce2c1447a0 msgr2=0x7fce2c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 --2- 172.21.15.119:0/3864530479 >> v2:172.21.15.188:3300/0 conn(0x7fce2c1447a0 0x7fce2c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 -- 172.21.15.119:0/3864530479 >> v1:172.21.15.188:6789/0 conn(0x7fce2c06a120 legacy=0x7fce2c06a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:21.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.936+0000 7fce31964640 1 -- 172.21.15.119:0/3864530479 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0c9410 con 0x7fce2c0cad30 2023-11-08T19:27:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.938+0000 7fce31964640 1 --2- 172.21.15.119:0/3864530479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c06f930 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06df20 tx=0x7fce1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34d02771c8956b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.938+0000 7fce31163640 1 -- 172.21.15.119:0/3864530479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c031870 con 0x7fce2c0cad30 2023-11-08T19:27:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.938+0000 7fce31163640 1 -- 172.21.15.119:0/3864530479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c0319d0 con 0x7fce2c0cad30 2023-11-08T19:27:21.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.938+0000 7fce31163640 1 -- 172.21.15.119:0/3864530479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c031d60 con 0x7fce2c0cad30 2023-11-08T19:27:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 -- 172.21.15.119:0/3864530479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 msgr2=0x7fce2c06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 --2- 172.21.15.119:0/3864530479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c06f930 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06df20 tx=0x7fce1c02b910 comp rx=0 tx=0).stop 2023-11-08T19:27:21.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 -- 172.21.15.119:0/3864530479 shutdown_connections 2023-11-08T19:27:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 --2- 172.21.15.119:0/3864530479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c06f930 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 --2- 172.21.15.119:0/3864530479 >> v2:172.21.15.188:3300/0 conn(0x7fce2c1447a0 0x7fce2c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 -- 172.21.15.119:0/3864530479 >> 172.21.15.119:0/3864530479 conn(0x7fce2c0cb610 msgr2=0x7fce2c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 -- 172.21.15.119:0/3864530479 shutdown_connections 2023-11-08T19:27:21.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.939+0000 7fce343c9640 1 -- 172.21.15.119:0/3864530479 wait complete. 2023-11-08T19:27:21.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.941+0000 7fce343c9640 1 Processor -- start 2023-11-08T19:27:21.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.941+0000 7fce343c9640 1 -- start start 2023-11-08T19:27:21.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce343c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce2c06a120 0x7fce2c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce343c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c15bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce343c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce2c1447a0 0x7fce2c15a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce343c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0c6f10 con 0x7fce2c0cad30 2023-11-08T19:27:21.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce343c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0c6290 con 0x7fce2c1447a0 2023-11-08T19:27:21.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce343c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0c6570 con 0x7fce2c06a120 2023-11-08T19:27:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce31964640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c15bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce31964640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c15bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44458/0 (socket says 172.21.15.119:44458) 2023-11-08T19:27:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.942+0000 7fce31964640 1 -- 172.21.15.119:0/130756181 learned_addr learned my addr 172.21.15.119:0/130756181 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce31964640 1 -- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce2c06a120 msgr2=0x7fce2c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce32165640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce2c06a120 0x7fce2c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce32966640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce2c1447a0 0x7fce2c15a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce31964640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce2c06a120 0x7fce2c0c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce31964640 1 -- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce2c1447a0 msgr2=0x7fce2c15a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce31964640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce2c1447a0 0x7fce2c15a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:21.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce31964640 1 -- 172.21.15.119:0/130756181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c0094d0 con 0x7fce2c0cad30 2023-11-08T19:27:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce32966640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce2c1447a0 0x7fce2c15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce31964640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c15bd50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fce1c000c00 tx=0x7fce1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.943+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c03ad30 con 0x7fce2c0cad30 2023-11-08T19:27:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.944+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c0443f0 con 0x7fce2c0cad30 2023-11-08T19:27:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.944+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0447a0 con 0x7fce2c0cad30 2023-11-08T19:27:21.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.944+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c0c6850 con 0x7fce2c0cad30 2023-11-08T19:27:21.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.944+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c15ae60 con 0x7fce2c0cad30 2023-11-08T19:27:21.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.945+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c0c7050 con 0x7fce2c0cad30 2023-11-08T19:27:21.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.946+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce1c04d450 con 0x7fce2c0cad30 2023-11-08T19:27:21.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.946+0000 7fce22ffd640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce1006ae70 0x7fce1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.947+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fce1c02d040 con 0x7fce2c0cad30 2023-11-08T19:27:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.947+0000 7fce32165640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce1006ae70 0x7fce1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.948+0000 7fce32165640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce1006ae70 0x7fce1006d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce2c143e90 tx=0x7fce28007310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:21.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:21.951+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce2c0c7050 con 0x7fce2c0cad30 2023-11-08T19:27:22.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.180+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce2c0c7050 con 0x7fce2c0cad30 2023-11-08T19:27:22.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.181+0000 7fce22ffd640 1 -- 172.21.15.119:0/130756181 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fce2c0c7050 con 0x7fce2c0cad30 2023-11-08T19:27:22.183 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:22.183 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:04.329003+0000","modified":"2023-11-08T19:27:07.395675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5739},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5739":{"gid":5739,"name":"c","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/976565354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":976565354},{"type":"v1","addr":"172.21.15.119:6837","nonce":976565354}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T19:27:22.184 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 26 2023-11-08T19:27:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce1006ae70 msgr2=0x7fce1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce1006ae70 0x7fce1006d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fce2c143e90 tx=0x7fce28007310 comp rx=0 tx=0).stop 2023-11-08T19:27:22.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 msgr2=0x7fce2c15bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c15bd50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fce1c000c00 tx=0x7fce1c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 shutdown_connections 2023-11-08T19:27:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce1006ae70 0x7fce1006d330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce2c1447a0 0x7fce2c15a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce2c0cad30 0x7fce2c15bd50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 --2- 172.21.15.119:0/130756181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce2c06a120 0x7fce2c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.184+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 >> 172.21.15.119:0/130756181 conn(0x7fce2c0cb610 msgr2=0x7fce2c0cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.185+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 shutdown_connections 2023-11-08T19:27:22.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.185+0000 7fce343c9640 1 -- 172.21.15.119:0/130756181 wait complete. 2023-11-08T19:27:22.205 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.422+0000 7f39911f5640 1 Processor -- start 2023-11-08T19:27:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.422+0000 7f39911f5640 1 -- start start 2023-11-08T19:27:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.423+0000 7f39911f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f398c1447a0 0x7f398c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.423+0000 7f39911f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 0x7f398c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.423+0000 7f39911f5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c06aa90 con 0x7f398c0cad30 2023-11-08T19:27:22.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.423+0000 7f39911f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c070040 con 0x7f398c1447a0 2023-11-08T19:27:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.423+0000 7f39911f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c3a50 con 0x7f398c06a120 2023-11-08T19:27:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.424+0000 7f3983fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f398c0cad30 0x7f398c06f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53520/0 (socket says 172.21.15.119:53520) 2023-11-08T19:27:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.424+0000 7f3983fff640 1 -- 172.21.15.119:0/1801380481 learned_addr learned my addr 172.21.15.119:0/1801380481 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.424+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1320833785 0 0) 0x7f398c06aa90 con 0x7f398c0cad30 2023-11-08T19:27:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.424+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3978003520 con 0x7f398c0cad30 2023-11-08T19:27:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.424+0000 7f398ad9d640 1 --2- 172.21.15.119:0/1801380481 >> v2:172.21.15.188:3300/0 conn(0x7f398c1447a0 0x7f398c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.424+0000 7f398b59e640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 0x7f398c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884005628 0 0) 0x7f3978003520 con 0x7f398c0cad30 2023-11-08T19:27:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 msgr2=0x7f398c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 0x7f398c06a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 >> v2:172.21.15.188:3300/0 conn(0x7f398c1447a0 msgr2=0x7f398c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 --2- 172.21.15.119:0/1801380481 >> v2:172.21.15.188:3300/0 conn(0x7f398c1447a0 0x7f398c15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c144b80 con 0x7f398c0cad30 2023-11-08T19:27:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f3974001670 con 0x7f398c0cad30 2023-11-08T19:27:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 >> v1:172.21.15.119:6789/0 conn(0x7f398c0cad30 legacy=0x7f398c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 0x7f39780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f3978005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c070040 con 0x7f3978004d00 2023-11-08T19:27:22.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c3a50 con 0x7f3978003970 2023-11-08T19:27:22.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f397800bab0 con 0x7f398c06a120 2023-11-08T19:27:22.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398b59e640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f3978005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398ad9d640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f3983fff640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 0x7f39780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398b59e640 1 -- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 msgr2=0x7f39780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398b59e640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c06a120 0x7f39780047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398b59e640 1 -- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3978003970 msgr2=0x7f3978003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398b59e640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398b59e640 1 -- 172.21.15.119:0/1801380481 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3978003520 con 0x7f3978004d00 2023-11-08T19:27:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.425+0000 7f398ad9d640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3978003970 0x7f3978003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f398b59e640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f3978005180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3984007360 tx=0x7f39780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c3f259e7aa7d14 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398402ccd0 con 0x7f3978004d00 2023-11-08T19:27:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398402ce30 con 0x7f3978004d00 2023-11-08T19:27:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f398a59c640 1 -- 172.21.15.119:0/1801380481 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984034660 con 0x7f3978004d00 2023-11-08T19:27:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f39911f5640 1 -- 172.21.15.119:0/1801380481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 msgr2=0x7f3978005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f39911f5640 1 --2- 172.21.15.119:0/1801380481 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f3978005180 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f3984007360 tx=0x7f39780057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f398b59e640 1 -- 172.21.15.119:0/1801380481 reap_dead start 2023-11-08T19:27:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f39911f5640 1 -- 172.21.15.119:0/1801380481 shutdown_connections 2023-11-08T19:27:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.427+0000 7f39911f5640 1 -- 172.21.15.119:0/1801380481 >> 172.21.15.119:0/1801380481 conn(0x7f398c0cb610 msgr2=0x7f398c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.428+0000 7f39911f5640 1 -- 172.21.15.119:0/1801380481 shutdown_connections 2023-11-08T19:27:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.428+0000 7f39911f5640 1 -- 172.21.15.119:0/1801380481 wait complete. 2023-11-08T19:27:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.429+0000 7f39911f5640 1 Processor -- start 2023-11-08T19:27:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.429+0000 7f39911f5640 1 -- start start 2023-11-08T19:27:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.430+0000 7f39911f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f398c15a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.430+0000 7f39911f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0c5e50 0x7f398c0cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.430+0000 7f39911f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cdde0 0x7f398c0c6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.430+0000 7f39911f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0cd410 con 0x7f3978004d00 2023-11-08T19:27:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.430+0000 7f39911f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0cd0e0 con 0x7f398c0cdde0 2023-11-08T19:27:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.430+0000 7f39911f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0b15e0 con 0x7f398c0c5e50 2023-11-08T19:27:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f398c15a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f398c15a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44484/0 (socket says 172.21.15.119:44484) 2023-11-08T19:27:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 -- 172.21.15.119:0/398993256 learned_addr learned my addr 172.21.15.119:0/398993256 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 -- 172.21.15.119:0/398993256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0c5e50 msgr2=0x7f398c0cd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0c5e50 0x7f398c0cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 -- 172.21.15.119:0/398993256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cdde0 msgr2=0x7f398c0c6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cdde0 0x7f398c0c6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 -- 172.21.15.119:0/398993256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0b18a0 con 0x7f3978004d00 2023-11-08T19:27:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.431+0000 7f398ad9d640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f398c15a750 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e7e0 tx=0x7f397c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.432+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00c850 con 0x7f3978004d00 2023-11-08T19:27:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.432+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c00c9b0 con 0x7f3978004d00 2023-11-08T19:27:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.432+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00cd20 con 0x7f3978004d00 2023-11-08T19:27:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.432+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c0b1b80 con 0x7f3978004d00 2023-11-08T19:27:22.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.432+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c15ac90 con 0x7f3978004d00 2023-11-08T19:27:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.433+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c074430 con 0x7f3978004d00 2023-11-08T19:27:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.434+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f397c013410 con 0x7f3978004d00 2023-11-08T19:27:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.435+0000 7f39837fe640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f396c06aea0 0x7f396c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.435+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f397c089790 con 0x7f3978004d00 2023-11-08T19:27:22.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.435+0000 7f3983fff640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f396c06aea0 0x7f396c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.436+0000 7f3983fff640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f396c06aea0 0x7f396c06d360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3974000ff0 tx=0x7f3974009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.439+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f397c054500 con 0x7f3978004d00 2023-11-08T19:27:22.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.632+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f398c06caa0 con 0x7f3978004d00 2023-11-08T19:27:22.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.635+0000 7f39837fe640 1 -- 172.21.15.119:0/398993256 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18523 (secure 0 0 0) 0x7f397c057b20 con 0x7f3978004d00 2023-11-08T19:27:22.637 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:22.637 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":54,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:05.896221+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:02.460658+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:03.346355+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f396c06aea0 msgr2=0x7f396c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f396c06aea0 0x7f396c06d360 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3974000ff0 tx=0x7f3974009040 comp rx=0 tx=0).stop 2023-11-08T19:27:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 msgr2=0x7f398c15a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f398c15a750 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f397c00e7e0 tx=0x7f397c00eca0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 shutdown_connections 2023-11-08T19:27:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f396c06aea0 0x7f396c06d360 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f398c0cdde0 0x7f398c0c6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f398c0c5e50 0x7f398c0cd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 --2- 172.21.15.119:0/398993256 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3978004d00 0x7f398c15a750 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 >> 172.21.15.119:0/398993256 conn(0x7f398c0cb610 msgr2=0x7f398c0c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 shutdown_connections 2023-11-08T19:27:22.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.638+0000 7f39911f5640 1 -- 172.21.15.119:0/398993256 wait complete. 2023-11-08T19:27:22.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:27:22.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.873+0000 7fad402b0640 1 Processor -- start 2023-11-08T19:27:22.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.873+0000 7fad402b0640 1 -- start start 2023-11-08T19:27:22.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad402b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad380cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad402b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad3815d9f0 0x7fad380c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad402b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad380c4960 con 0x7fad3806cb40 2023-11-08T19:27:22.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad402b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad38143a40 con 0x7fad3815d9f0 2023-11-08T19:27:22.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad402b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3815df60 con 0x7fad38142f80 2023-11-08T19:27:22.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad3e84d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fad3806cb40 0x7fad380c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53522/0 (socket says 172.21.15.119:53522) 2023-11-08T19:27:22.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.875+0000 7fad3e84d640 1 -- 172.21.15.119:0/1953612941 learned_addr learned my addr 172.21.15.119:0/1953612941 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:22.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d84b640 1 --2- 172.21.15.119:0/1953612941 >> v2:172.21.15.188:3300/0 conn(0x7fad3815d9f0 0x7fad380c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3e04c640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad380cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800911434 0 0) 0x7fad380c4960 con 0x7fad3806cb40 2023-11-08T19:27:22.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad24003520 con 0x7fad3806cb40 2023-11-08T19:27:22.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1210770466 0 0) 0x7fad24003520 con 0x7fad3806cb40 2023-11-08T19:27:22.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 msgr2=0x7fad380cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad380cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 >> v2:172.21.15.188:3300/0 conn(0x7fad3815d9f0 msgr2=0x7fad380c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 --2- 172.21.15.119:0/1953612941 >> v2:172.21.15.188:3300/0 conn(0x7fad3815d9f0 0x7fad380c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3815e240 con 0x7fad3806cb40 2023-11-08T19:27:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fad300019f0 con 0x7fad3806cb40 2023-11-08T19:27:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3e04c640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad380cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 >> v1:172.21.15.119:6789/0 conn(0x7fad3806cb40 legacy=0x7fad380c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad24003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24003fb0 0x7fad240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad24004d00 0x7fad24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad38143a40 con 0x7fad24003fb0 2023-11-08T19:27:22.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3815df60 con 0x7fad24004d00 2023-11-08T19:27:22.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.876+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2400bb40 con 0x7fad38142f80 2023-11-08T19:27:22.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3d84b640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24003fb0 0x7fad240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3e04c640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad24003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3e84d640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad24004d00 0x7fad24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3d84b640 1 -- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 msgr2=0x7fad24003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3d84b640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad38142f80 0x7fad24003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3d84b640 1 -- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad24004d00 msgr2=0x7fad24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3d84b640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad24004d00 0x7fad24005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.877+0000 7fad3d84b640 1 -- 172.21.15.119:0/1953612941 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24003520 con 0x7fad24003fb0 2023-11-08T19:27:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.878+0000 7fad3d84b640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24003fb0 0x7fad240047c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad2800ba30 tx=0x7fad240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667c252026e3cc9e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28033650 con 0x7fad24003fb0 2023-11-08T19:27:22.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad280337b0 con 0x7fad24003fb0 2023-11-08T19:27:22.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad3d04a640 1 -- 172.21.15.119:0/1953612941 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad28033b40 con 0x7fad24003fb0 2023-11-08T19:27:22.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad402b0640 1 -- 172.21.15.119:0/1953612941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24003fb0 msgr2=0x7fad240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad402b0640 1 --2- 172.21.15.119:0/1953612941 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad24003fb0 0x7fad240047c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fad2800ba30 tx=0x7fad240057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad3e84d640 1 -- 172.21.15.119:0/1953612941 reap_dead start 2023-11-08T19:27:22.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad402b0640 1 -- 172.21.15.119:0/1953612941 shutdown_connections 2023-11-08T19:27:22.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad402b0640 1 -- 172.21.15.119:0/1953612941 >> 172.21.15.119:0/1953612941 conn(0x7fad380cb610 msgr2=0x7fad380cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:22.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad402b0640 1 -- 172.21.15.119:0/1953612941 shutdown_connections 2023-11-08T19:27:22.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.879+0000 7fad402b0640 1 -- 172.21.15.119:0/1953612941 wait complete. 2023-11-08T19:27:22.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.881+0000 7fad402b0640 1 Processor -- start 2023-11-08T19:27:22.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.881+0000 7fad402b0640 1 -- start start 2023-11-08T19:27:22.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad402b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad24003fb0 0x7fad38157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad402b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad380c8370 0x7fad3813f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad402b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 0x7fad380c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad402b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3813fb80 con 0x7fad3813f6f0 2023-11-08T19:27:22.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad402b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380c8b50 con 0x7fad380c8370 2023-11-08T19:27:22.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad402b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380c8870 con 0x7fad24003fb0 2023-11-08T19:27:22.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad3e84d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 0x7fad380c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad3e84d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 0x7fad380c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44496/0 (socket says 172.21.15.119:44496) 2023-11-08T19:27:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.882+0000 7fad3e84d640 1 -- 172.21.15.119:0/1760641271 learned_addr learned my addr 172.21.15.119:0/1760641271 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3d84b640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad380c8370 0x7fad3813f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e04c640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad24003fb0 0x7fad38157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e84d640 1 -- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad24003fb0 msgr2=0x7fad38157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e84d640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad24003fb0 0x7fad38157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e84d640 1 -- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad380c8370 msgr2=0x7fad3813f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e84d640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad380c8370 0x7fad3813f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e84d640 1 -- 172.21.15.119:0/1760641271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad38157bb0 con 0x7fad3813f6f0 2023-11-08T19:27:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad3e84d640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 0x7fad380c5e30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fad300013d0 tx=0x7fad3000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3000ec10 con 0x7fad3813f6f0 2023-11-08T19:27:22.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3000ed90 con 0x7fad3813f6f0 2023-11-08T19:27:22.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad30013660 con 0x7fad3813f6f0 2023-11-08T19:27:22.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.883+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad38157e90 con 0x7fad3813f6f0 2023-11-08T19:27:22.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.884+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad381577c0 con 0x7fad3813f6f0 2023-11-08T19:27:22.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.885+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38074430 con 0x7fad3813f6f0 2023-11-08T19:27:22.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.886+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad300137c0 con 0x7fad3813f6f0 2023-11-08T19:27:22.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.887+0000 7fad2effd640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad1c06ae70 0x7fad1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:22.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.887+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fad30089d40 con 0x7fad3813f6f0 2023-11-08T19:27:22.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.887+0000 7fad3e04c640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad1c06ae70 0x7fad1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:22.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.887+0000 7fad3e04c640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad1c06ae70 0x7fad1c06d330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fad34006fd0 tx=0x7fad34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:22.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:22.891+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad30054b40 con 0x7fad3813f6f0 2023-11-08T19:27:23.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.119+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad380c4230 con 0x7fad3813f6f0 2023-11-08T19:27:23.122 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-11-08T19:27:23.120+0000 7f8fd7f7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:27:23.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.123+0000 7fad2effd640 1 -- 172.21.15.119:0/1760641271 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad30058160 con 0x7fad3813f6f0 2023-11-08T19:27:23.125 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:27:23.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.127+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad1c06ae70 msgr2=0x7fad1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.127+0000 7fad402b0640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad1c06ae70 0x7fad1c06d330 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fad34006fd0 tx=0x7fad34008040 comp rx=0 tx=0).stop 2023-11-08T19:27:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.127+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 msgr2=0x7fad380c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.127+0000 7fad402b0640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 0x7fad380c5e30 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fad300013d0 tx=0x7fad3000e530 comp rx=0 tx=0).stop 2023-11-08T19:27:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 shutdown_connections 2023-11-08T19:27:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad1c06ae70 0x7fad1c06d330 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad3813f6f0 0x7fad380c5e30 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad380c8370 0x7fad3813f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 --2- 172.21.15.119:0/1760641271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad24003fb0 0x7fad38157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 >> 172.21.15.119:0/1760641271 conn(0x7fad380cb610 msgr2=0x7fad380cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.128+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 shutdown_connections 2023-11-08T19:27:23.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.129+0000 7fad402b0640 1 -- 172.21.15.119:0/1760641271 wait complete. 2023-11-08T19:27:23.136 INFO:tasks.ceph.mds.c.smithi119.stderr: -1700> 2023-11-08T19:27:08.834+0000 7fcb362aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:23.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:27:23.199 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:27:23.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.345+0000 7fd1519c9640 1 Processor -- start 2023-11-08T19:27:23.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.345+0000 7fd1519c9640 1 -- start start 2023-11-08T19:27:23.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd1519c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd14c143520 0x7fd14c0caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd1519c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd14c143900 0x7fd14c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd1519c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0c9d50 con 0x7fd14c0c5030 2023-11-08T19:27:23.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd1519c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c0c54e0 con 0x7fd14c143900 2023-11-08T19:27:23.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd1519c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c15e5e0 con 0x7fd14c143520 2023-11-08T19:27:23.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd1427fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd14c143900 0x7fd14c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd14b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd14c0c5030 0x7fd14c15d870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53534/0 (socket says 172.21.15.119:53534) 2023-11-08T19:27:23.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd14b7fe640 1 -- 172.21.15.119:0/3264503240 learned_addr learned my addr 172.21.15.119:0/3264503240 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:23.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.347+0000 7fd14affd640 1 --2- 172.21.15.119:0/3264503240 >> v2:172.21.15.188:3300/0 conn(0x7fd14c143520 0x7fd14c0caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.348+0000 7fd1427fc640 1 -- 172.21.15.119:0/3264503240 >> v2:172.21.15.188:3300/0 conn(0x7fd14c143520 msgr2=0x7fd14c0caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:23.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.348+0000 7fd1427fc640 1 --2- 172.21.15.119:0/3264503240 >> v2:172.21.15.188:3300/0 conn(0x7fd14c143520 0x7fd14c0caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.348+0000 7fd1427fc640 1 -- 172.21.15.119:0/3264503240 >> v1:172.21.15.119:6789/0 conn(0x7fd14c0c5030 legacy=0x7fd14c15d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.348+0000 7fd1427fc640 1 -- 172.21.15.119:0/3264503240 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0c9fa0 con 0x7fd14c143900 2023-11-08T19:27:23.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.349+0000 7fd1427fc640 1 --2- 172.21.15.119:0/3264503240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c143900 0x7fd14c0c4af0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cc6e0 tx=0x7fd13402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aa98fc259c8710 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.349+0000 7fd14a7fc640 1 -- 172.21.15.119:0/3264503240 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134031830 con 0x7fd14c143900 2023-11-08T19:27:23.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.349+0000 7fd14a7fc640 1 -- 172.21.15.119:0/3264503240 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd134031990 con 0x7fd14c143900 2023-11-08T19:27:23.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.349+0000 7fd14a7fc640 1 -- 172.21.15.119:0/3264503240 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd134031d20 con 0x7fd14c143900 2023-11-08T19:27:23.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 -- 172.21.15.119:0/3264503240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c143900 msgr2=0x7fd14c0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 --2- 172.21.15.119:0/3264503240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c143900 0x7fd14c0c4af0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fd14c0cc6e0 tx=0x7fd13402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 -- 172.21.15.119:0/3264503240 shutdown_connections 2023-11-08T19:27:23.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 --2- 172.21.15.119:0/3264503240 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c143900 0x7fd14c0c4af0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 --2- 172.21.15.119:0/3264503240 >> v2:172.21.15.188:3300/0 conn(0x7fd14c143520 0x7fd14c0caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 -- 172.21.15.119:0/3264503240 >> 172.21.15.119:0/3264503240 conn(0x7fd14c0b0950 msgr2=0x7fd14c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:23.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 -- 172.21.15.119:0/3264503240 shutdown_connections 2023-11-08T19:27:23.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.350+0000 7fd1519c9640 1 -- 172.21.15.119:0/3264503240 wait complete. 2023-11-08T19:27:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.352+0000 7fd1519c9640 1 Processor -- start 2023-11-08T19:27:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.352+0000 7fd1519c9640 1 -- start start 2023-11-08T19:27:23.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd1519c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14c0c5030 0x7fd14c142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd1519c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14c143520 0x7fd14c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd1519c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 0x7fd14c0c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd1519c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c142b90 con 0x7fd14c0b4030 2023-11-08T19:27:23.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd1519c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0c8d40 con 0x7fd14c143520 2023-11-08T19:27:23.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd1519c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0b2f90 con 0x7fd14c0c5030 2023-11-08T19:27:23.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd14b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 0x7fd14c0c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd14b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 0x7fd14c0c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44534/0 (socket says 172.21.15.119:44534) 2023-11-08T19:27:23.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.353+0000 7fd14b7fe640 1 -- 172.21.15.119:0/2529075660 learned_addr learned my addr 172.21.15.119:0/2529075660 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:23.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14affd640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14c0c5030 0x7fd14c142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd1427fc640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14c143520 0x7fd14c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14b7fe640 1 -- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14c0c5030 msgr2=0x7fd14c142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14b7fe640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14c0c5030 0x7fd14c142650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14b7fe640 1 -- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14c143520 msgr2=0x7fd14c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:23.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14b7fe640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14c143520 0x7fd14c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14b7fe640 1 -- 172.21.15.119:0/2529075660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd134009400 con 0x7fd14c0b4030 2023-11-08T19:27:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14affd640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14c0c5030 0x7fd14c142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd14b7fe640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 0x7fd14c0c6920 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd14400cbc0 tx=0x7fd14400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400ad40 con 0x7fd14c0b4030 2023-11-08T19:27:23.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144012400 con 0x7fd14c0b4030 2023-11-08T19:27:23.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14400aea0 con 0x7fd14c0b4030 2023-11-08T19:27:23.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.354+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c0b3270 con 0x7fd14c0b4030 2023-11-08T19:27:23.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.355+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c0b3830 con 0x7fd14c0b4030 2023-11-08T19:27:23.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.356+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c04ece0 con 0x7fd14c0b4030 2023-11-08T19:27:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.357+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd144007070 con 0x7fd14c0b4030 2023-11-08T19:27:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.358+0000 7fd148ff9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.358+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fd144089770 con 0x7fd14c0b4030 2023-11-08T19:27:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.358+0000 7fd14affd640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:23.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.359+0000 7fd14affd640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:23.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.362+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1440545d0 con 0x7fd14c0b4030 2023-11-08T19:27:23.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:23.583+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd14c0cd2f0 con 0x7fd14c0b4030 2023-11-08T19:27:24.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.127+0000 7fd148ff9640 1 -- 172.21.15.119:0/2529075660 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd144057bf0 con 0x7fd14c0b4030 2023-11-08T19:27:24.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 msgr2=0x7fd12c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd13c006fd0 tx=0x7fd13c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:24.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 msgr2=0x7fd14c0c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 0x7fd14c0c6920 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd14400cbc0 tx=0x7fd14400a430 comp rx=0 tx=0).stop 2023-11-08T19:27:24.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 shutdown_connections 2023-11-08T19:27:24.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd12c06ae70 0x7fd12c06d330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd14c0b4030 0x7fd14c0c6920 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd14c143520 0x7fd14c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 --2- 172.21.15.119:0/2529075660 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd14c0c5030 0x7fd14c142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 >> 172.21.15.119:0/2529075660 conn(0x7fd14c0b0950 msgr2=0x7fd14c0c39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:24.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 shutdown_connections 2023-11-08T19:27:24.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.131+0000 7fd1519c9640 1 -- 172.21.15.119:0/2529075660 wait complete. 2023-11-08T19:27:24.153 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:27:24.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.417+0000 7f59d441a640 1 Processor -- start 2023-11-08T19:27:24.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.417+0000 7f59d441a640 1 -- start start 2023-11-08T19:27:24.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.418+0000 7f59d441a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59cc143600 0x7f59cc0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.418+0000 7f59d441a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59cc15d9f0 0x7f59cc15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.418+0000 7f59d441a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59cc15e440 con 0x7f59cc06cb40 2023-11-08T19:27:24.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.418+0000 7f59d441a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc0c4f50 con 0x7f59cc143600 2023-11-08T19:27:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.418+0000 7f59d441a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59cc0c4960 con 0x7f59cc15d9f0 2023-11-08T19:27:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59cc143600 0x7f59cc0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f59cc143600 0x7f59cc0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44540/0 (socket says 172.21.15.119:44540) 2023-11-08T19:27:24.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 -- 172.21.15.119:0/3094354097 learned_addr learned my addr 172.21.15.119:0/3094354097 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 -- 172.21.15.119:0/3094354097 >> v2:172.21.15.188:3300/0 conn(0x7f59cc15d9f0 msgr2=0x7f59cc15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 --2- 172.21.15.119:0/3094354097 >> v2:172.21.15.188:3300/0 conn(0x7f59cc15d9f0 0x7f59cc15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 -- 172.21.15.119:0/3094354097 >> v1:172.21.15.188:6789/0 conn(0x7f59cc06cb40 legacy=0x7f59cc06cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.419+0000 7f59d21b6640 1 -- 172.21.15.119:0/3094354097 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc15e720 con 0x7f59cc143600 2023-11-08T19:27:24.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.420+0000 7f59d21b6640 1 --2- 172.21.15.119:0/3094354097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc143600 0x7f59cc0c5370 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0c91e0 tx=0x7f59c802da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee811d5403583d10 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.420+0000 7f59d11b4640 1 -- 172.21.15.119:0/3094354097 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c802f850 con 0x7f59cc143600 2023-11-08T19:27:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.420+0000 7f59d11b4640 1 -- 172.21.15.119:0/3094354097 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c802f9b0 con 0x7f59cc143600 2023-11-08T19:27:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.420+0000 7f59d11b4640 1 -- 172.21.15.119:0/3094354097 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c802fd40 con 0x7f59cc143600 2023-11-08T19:27:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 -- 172.21.15.119:0/3094354097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc143600 msgr2=0x7f59cc0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 --2- 172.21.15.119:0/3094354097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc143600 0x7f59cc0c5370 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0c91e0 tx=0x7f59c802da90 comp rx=0 tx=0).stop 2023-11-08T19:27:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 -- 172.21.15.119:0/3094354097 shutdown_connections 2023-11-08T19:27:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 --2- 172.21.15.119:0/3094354097 >> v2:172.21.15.188:3300/0 conn(0x7f59cc15d9f0 0x7f59cc15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 --2- 172.21.15.119:0/3094354097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc143600 0x7f59cc0c5370 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 -- 172.21.15.119:0/3094354097 >> 172.21.15.119:0/3094354097 conn(0x7f59cc0cb610 msgr2=0x7f59cc0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:24.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 -- 172.21.15.119:0/3094354097 shutdown_connections 2023-11-08T19:27:24.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.421+0000 7f59d441a640 1 -- 172.21.15.119:0/3094354097 wait complete. 2023-11-08T19:27:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.423+0000 7f59d441a640 1 Processor -- start 2023-11-08T19:27:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.423+0000 7f59d441a640 1 -- start start 2023-11-08T19:27:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d441a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 0x7f59cc15ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d441a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59cc143600 0x7f59cc0b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d441a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59cc15d9f0 0x7f59cc0c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d441a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc139740 con 0x7f59cc06cb40 2023-11-08T19:27:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d441a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc1399e0 con 0x7f59cc143600 2023-11-08T19:27:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d441a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59cc139cc0 con 0x7f59cc15d9f0 2023-11-08T19:27:24.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 0x7f59cc15ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 0x7f59cc15ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44550/0 (socket says 172.21.15.119:44550) 2023-11-08T19:27:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 -- 172.21.15.119:0/805255223 learned_addr learned my addr 172.21.15.119:0/805255223 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 -- 172.21.15.119:0/805255223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59cc15d9f0 msgr2=0x7f59cc0c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59cc15d9f0 0x7f59cc0c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 -- 172.21.15.119:0/805255223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59cc143600 msgr2=0x7f59cc0b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59cc143600 0x7f59cc0b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.424+0000 7f59d21b6640 1 -- 172.21.15.119:0/805255223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c80094d0 con 0x7f59cc06cb40 2023-11-08T19:27:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.425+0000 7f59d21b6640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 0x7f59cc15ccc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009960 tx=0x7f59c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.425+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c803e040 con 0x7f59cc06cb40 2023-11-08T19:27:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.425+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c8039c40 con 0x7f59cc06cb40 2023-11-08T19:27:24.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.425+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c802fe20 con 0x7f59cc06cb40 2023-11-08T19:27:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.425+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59cc139fa0 con 0x7f59cc06cb40 2023-11-08T19:27:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.425+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59cc137350 con 0x7f59cc06cb40 2023-11-08T19:27:24.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.426+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59cc13a4c0 con 0x7f59cc06cb40 2023-11-08T19:27:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.427+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59c8043390 con 0x7f59cc06cb40 2023-11-08T19:27:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.428+0000 7f59c2ffd640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f59b006ae70 0x7f59b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:24.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.428+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f59c8034070 con 0x7f59cc06cb40 2023-11-08T19:27:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.428+0000 7f59d19b5640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f59b006ae70 0x7f59b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.429+0000 7f59d19b5640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f59b006ae70 0x7f59b006d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0c8990 tx=0x7f59bc006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:24.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.433+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59cc13a4c0 con 0x7f59cc06cb40 2023-11-08T19:27:24.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:24.609+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f59cc13a4c0 con 0x7f59cc06cb40 2023-11-08T19:27:25.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.138+0000 7f59c2ffd640 1 -- 172.21.15.119:0/805255223 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f59cc13a4c0 con 0x7f59cc06cb40 2023-11-08T19:27:25.140 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:27:25.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f59b006ae70 msgr2=0x7f59b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f59b006ae70 0x7f59b006d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0c8990 tx=0x7f59bc006bf0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 msgr2=0x7f59cc15ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 0x7f59cc15ccc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009960 tx=0x7f59c8004730 comp rx=0 tx=0).stop 2023-11-08T19:27:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 shutdown_connections 2023-11-08T19:27:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f59b006ae70 0x7f59b006d330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59cc15d9f0 0x7f59cc0c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59cc143600 0x7f59cc0b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 --2- 172.21.15.119:0/805255223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f59cc06cb40 0x7f59cc15ccc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.141+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 >> 172.21.15.119:0/805255223 conn(0x7f59cc0cb610 msgr2=0x7f59cc1405d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:25.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.142+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 shutdown_connections 2023-11-08T19:27:25.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.142+0000 7f59d441a640 1 -- 172.21.15.119:0/805255223 wait complete. 2023-11-08T19:27:25.165 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:27:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.379+0000 7fa7010c2640 1 Processor -- start 2023-11-08T19:27:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.379+0000 7fa7010c2640 1 -- start start 2023-11-08T19:27:25.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.380+0000 7fa7010c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 0x7fa6fc0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.380+0000 7fa7010c2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc15b6a0 con 0x7fa6fc0cdea0 2023-11-08T19:27:25.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.380+0000 7fa7010c2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6fc0ce330 con 0x7fa6fc0ccbc0 2023-11-08T19:27:25.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.380+0000 7fa7010c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0ce610 con 0x7fa6fc15b2c0 2023-11-08T19:27:25.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6f259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa6fc0cdea0 0x7fa6fc0cf770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53548/0 (socket says 172.21.15.119:53548) 2023-11-08T19:27:25.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6f259c640 1 -- 172.21.15.119:0/4283373087 learned_addr learned my addr 172.21.15.119:0/4283373087 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:25.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fad9d640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 0x7fa6fc0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70797315 0 0) 0x7fa6e4001110 con 0x7fa6fc0cdea0 2023-11-08T19:27:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6e8003520 con 0x7fa6fc0cdea0 2023-11-08T19:27:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172339577 0 0) 0x7fa6e8003520 con 0x7fa6fc0cdea0 2023-11-08T19:27:25.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 msgr2=0x7fa6fc0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fa59c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 0x7fa6fc0cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.381+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 >> v1:172.21.15.188:6789/0 conn(0x7fa6fc0ccbc0 legacy=0x7fa6fc0d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc0cbba0 con 0x7fa6fc0cdea0 2023-11-08T19:27:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fa6e4001800 con 0x7fa6fc0cdea0 2023-11-08T19:27:25.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 >> v1:172.21.15.119:6789/0 conn(0x7fa6fc0cdea0 legacy=0x7fa6fc0cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80039a0 0x7fa6e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 0x7fa6e8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc0ce610 con 0x7fa6e80047c0 2023-11-08T19:27:25.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800bb10 con 0x7fa6e80039a0 2023-11-08T19:27:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e800bcd0 con 0x7fa6fc15b2c0 2023-11-08T19:27:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6f259c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6fad9d640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80039a0 0x7fa6e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6f259c640 1 -- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 msgr2=0x7fa6e8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:25.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6f259c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc15b2c0 0x7fa6e8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6f259c640 1 -- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80039a0 msgr2=0x7fa6e8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:25.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6f259c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6e80039a0 0x7fa6e8003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.382+0000 7fa6f259c640 1 -- 172.21.15.119:0/4283373087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e8003520 con 0x7fa6e80047c0 2023-11-08T19:27:25.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.383+0000 7fa6f259c640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6e8004c40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007fc0 tx=0x7fa6e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68daa7a4b2ff56d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4005ad0 con 0x7fa6e80047c0 2023-11-08T19:27:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e4005c50 con 0x7fa6e80047c0 2023-11-08T19:27:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa6fa59c640 1 -- 172.21.15.119:0/4283373087 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e402c7e0 con 0x7fa6e80047c0 2023-11-08T19:27:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa7010c2640 1 -- 172.21.15.119:0/4283373087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 msgr2=0x7fa6e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa7010c2640 1 --2- 172.21.15.119:0/4283373087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6e8004c40 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007fc0 tx=0x7fa6e80057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa6fb59e640 1 -- 172.21.15.119:0/4283373087 reap_dead start 2023-11-08T19:27:25.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa7010c2640 1 -- 172.21.15.119:0/4283373087 shutdown_connections 2023-11-08T19:27:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa7010c2640 1 -- 172.21.15.119:0/4283373087 >> 172.21.15.119:0/4283373087 conn(0x7fa6fc08afe0 msgr2=0x7fa6fc09a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa7010c2640 1 -- 172.21.15.119:0/4283373087 shutdown_connections 2023-11-08T19:27:25.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.384+0000 7fa7010c2640 1 -- 172.21.15.119:0/4283373087 wait complete. 2023-11-08T19:27:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.386+0000 7fa7010c2640 1 Processor -- start 2023-11-08T19:27:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.386+0000 7fa7010c2640 1 -- start start 2023-11-08T19:27:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa7010c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6fc15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa7010c2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6fc0b5650 0x7fa6fc0b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa7010c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0b5ab0 0x7fa6fc0b1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa7010c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc15c8a0 con 0x7fa6e80047c0 2023-11-08T19:27:25.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa7010c2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc15bc90 con 0x7fa6fc0b5650 2023-11-08T19:27:25.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa7010c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6fc15bf70 con 0x7fa6fc0b5ab0 2023-11-08T19:27:25.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa6fad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6fc15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa6fad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6fc15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44582/0 (socket says 172.21.15.119:44582) 2023-11-08T19:27:25.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.387+0000 7fa6fad9d640 1 -- 172.21.15.119:0/2566096589 learned_addr learned my addr 172.21.15.119:0/2566096589 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:25.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fb59e640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0b5ab0 0x7fa6fc0b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6f259c640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6fc0b5650 0x7fa6fc0b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fad9d640 1 -- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0b5ab0 msgr2=0x7fa6fc0b1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fad9d640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0b5ab0 0x7fa6fc0b1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fad9d640 1 -- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6fc0b5650 msgr2=0x7fa6fc0b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:25.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fad9d640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6fc0b5650 0x7fa6fc0b0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:25.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fad9d640 1 -- 172.21.15.119:0/2566096589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6fc15c250 con 0x7fa6e80047c0 2023-11-08T19:27:25.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fb59e640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0b5ab0 0x7fa6fc0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:25.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6fad9d640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6fc15a180 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00e7e0 tx=0x7fa6ec00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00c6d0 con 0x7fa6e80047c0 2023-11-08T19:27:25.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6ec00c830 con 0x7fa6e80047c0 2023-11-08T19:27:25.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6fc15c530 con 0x7fa6e80047c0 2023-11-08T19:27:25.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec00cbe0 con 0x7fa6e80047c0 2023-11-08T19:27:25.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.388+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6fc15cd30 con 0x7fa6e80047c0 2023-11-08T19:27:25.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.389+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6fc04ece0 con 0x7fa6e80047c0 2023-11-08T19:27:25.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.390+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa6ec00cd40 con 0x7fa6e80047c0 2023-11-08T19:27:25.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.392+0000 7fa6f3fff640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa6dc06aea0 0x7fa6dc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:25.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.392+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6030+0+0 (secure 0 0 0) 0x7fa6ec089510 con 0x7fa6e80047c0 2023-11-08T19:27:25.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.392+0000 7fa6f259c640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa6dc06aea0 0x7fa6dc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:25.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.393+0000 7fa6f259c640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa6dc06aea0 0x7fa6dc06d360 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0cc8b0 tx=0x7fa6e4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:25.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.396+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa6ec0543e0 con 0x7fa6e80047c0 2023-11-08T19:27:25.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:25.584+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa6fc0cf4e0 con 0x7fa6e80047c0 2023-11-08T19:27:26.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.141+0000 7fa6f3fff640 1 -- 172.21.15.119:0/2566096589 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa6ec0093a0 con 0x7fa6e80047c0 2023-11-08T19:27:26.143 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-11-08T19:27:26.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa6dc06aea0 msgr2=0x7fa6dc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa6dc06aea0 0x7fa6dc06d360 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc0cc8b0 tx=0x7fa6e4009030 comp rx=0 tx=0).stop 2023-11-08T19:27:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 msgr2=0x7fa6fc15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6fc15a180 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00e7e0 tx=0x7fa6ec00eca0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 shutdown_connections 2023-11-08T19:27:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa6dc06aea0 0x7fa6dc06d360 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6fc0b5ab0 0x7fa6fc0b1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6fc0b5650 0x7fa6fc0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 --2- 172.21.15.119:0/2566096589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6e80047c0 0x7fa6fc15a180 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.145+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 >> 172.21.15.119:0/2566096589 conn(0x7fa6fc08afe0 msgr2=0x7fa6fc0af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.146+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 shutdown_connections 2023-11-08T19:27:26.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.147+0000 7fa7010c2640 1 -- 172.21.15.119:0/2566096589 wait complete. 2023-11-08T19:27:26.170 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:27:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.375+0000 7f2b5d4c9640 1 Processor -- start 2023-11-08T19:27:26.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.375+0000 7f2b5d4c9640 1 -- start start 2023-11-08T19:27:26.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.377+0000 7f2b5d4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b58144550 0x7f2b580cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.377+0000 7f2b5d4c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b580cb8c0 0x7f2b5815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.377+0000 7f2b5d4c9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2b5815bb10 0x7f2b5815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.377+0000 7f2b5d4c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580cbd20 con 0x7f2b580cb8c0 2023-11-08T19:27:26.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.377+0000 7f2b5d4c9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b580c43b0 con 0x7f2b5815bb10 2023-11-08T19:27:26.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.377+0000 7f2b5d4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5815c340 con 0x7f2b58144550 2023-11-08T19:27:26.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b580cb8c0 0x7f2b5815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2b580cb8c0 0x7f2b5815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44592/0 (socket says 172.21.15.119:44592) 2023-11-08T19:27:26.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 -- 172.21.15.119:0/3230706058 learned_addr learned my addr 172.21.15.119:0/3230706058 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:26.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b56ffd640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b58144550 0x7f2b580cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 -- 172.21.15.119:0/3230706058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b58144550 msgr2=0x7f2b580cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b577fe640 1 --2- 172.21.15.119:0/3230706058 >> v2:172.21.15.188:3300/0 conn(0x7f2b5815bb10 0x7f2b5815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b58144550 0x7f2b580cb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 -- 172.21.15.119:0/3230706058 >> v2:172.21.15.188:3300/0 conn(0x7f2b5815bb10 msgr2=0x7f2b5815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 --2- 172.21.15.119:0/3230706058 >> v2:172.21.15.188:3300/0 conn(0x7f2b5815bb10 0x7f2b5815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b567fc640 1 -- 172.21.15.119:0/3230706058 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b580c3a80 con 0x7f2b580cb8c0 2023-11-08T19:27:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b56ffd640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b58144550 0x7f2b580cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:26.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.378+0000 7f2b577fe640 1 --2- 172.21.15.119:0/3230706058 >> v2:172.21.15.188:3300/0 conn(0x7f2b5815bb10 0x7f2b5815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:26.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.379+0000 7f2b567fc640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b580cb8c0 0x7f2b5815b5d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2b580cb000 tx=0x7f2b4002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b5383f562ab859e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b55ffb640 1 -- 172.21.15.119:0/3230706058 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031880 con 0x7f2b580cb8c0 2023-11-08T19:27:26.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b55ffb640 1 -- 172.21.15.119:0/3230706058 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b400319e0 con 0x7f2b580cb8c0 2023-11-08T19:27:26.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b55ffb640 1 -- 172.21.15.119:0/3230706058 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031d70 con 0x7f2b580cb8c0 2023-11-08T19:27:26.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/3230706058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b580cb8c0 msgr2=0x7f2b5815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b580cb8c0 0x7f2b5815b5d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2b580cb000 tx=0x7f2b4002b920 comp rx=0 tx=0).stop 2023-11-08T19:27:26.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/3230706058 shutdown_connections 2023-11-08T19:27:26.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/3230706058 >> v2:172.21.15.188:3300/0 conn(0x7f2b5815bb10 0x7f2b5815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b580cb8c0 0x7f2b5815b5d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/3230706058 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b58144550 0x7f2b580cb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/3230706058 >> 172.21.15.119:0/3230706058 conn(0x7f2b580b0950 msgr2=0x7f2b580b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:26.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/3230706058 shutdown_connections 2023-11-08T19:27:26.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.380+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/3230706058 wait complete. 2023-11-08T19:27:26.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.382+0000 7f2b5d4c9640 1 Processor -- start 2023-11-08T19:27:26.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.382+0000 7f2b5d4c9640 1 -- start start 2023-11-08T19:27:26.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.383+0000 7f2b5d4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b580cb8c0 0x7f2b58135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.383+0000 7f2b5d4c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 0x7f2b580ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.383+0000 7f2b5d4c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5815bb10 0x7f2b580ca580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.383+0000 7f2b5d4c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5815b290 con 0x7f2b58144550 2023-11-08T19:27:26.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.383+0000 7f2b5d4c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58134c80 con 0x7f2b5815bb10 2023-11-08T19:27:26.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.383+0000 7f2b5d4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b58135110 con 0x7f2b580cb8c0 2023-11-08T19:27:26.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 0x7f2b580ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 0x7f2b580ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44606/0 (socket says 172.21.15.119:44606) 2023-11-08T19:27:26.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 -- 172.21.15.119:0/2468063110 learned_addr learned my addr 172.21.15.119:0/2468063110 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:26.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b56ffd640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b580cb8c0 0x7f2b58135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b577fe640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5815bb10 0x7f2b580ca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 -- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b580cb8c0 msgr2=0x7f2b58135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b580cb8c0 0x7f2b58135dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 -- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5815bb10 msgr2=0x7f2b580ca580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5815bb10 0x7f2b580ca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.384+0000 7f2b567fc640 1 -- 172.21.15.119:0/2468063110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b400094d0 con 0x7f2b58144550 2023-11-08T19:27:26.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.385+0000 7f2b567fc640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 0x7f2b580ca040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2b40000c00 tx=0x7f2b400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.385+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40031850 con 0x7f2b58144550 2023-11-08T19:27:26.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.385+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b400443f0 con 0x7f2b58144550 2023-11-08T19:27:26.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.385+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b58134070 con 0x7f2b58144550 2023-11-08T19:27:26.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.385+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b400447a0 con 0x7f2b58144550 2023-11-08T19:27:26.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.385+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b58134630 con 0x7f2b58144550 2023-11-08T19:27:26.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.386+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b58134e10 con 0x7f2b58144550 2023-11-08T19:27:26.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.387+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b40044900 con 0x7f2b58144550 2023-11-08T19:27:26.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.388+0000 7f2b4f7fe640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b3806aea0 0x7f2b3806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.388+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2b400abf70 con 0x7f2b58144550 2023-11-08T19:27:26.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.389+0000 7f2b56ffd640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b3806aea0 0x7f2b3806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.389+0000 7f2b56ffd640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b3806aea0 0x7f2b3806d360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b48008f10 tx=0x7f2b48008460 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.393+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b40077000 con 0x7f2b58144550 2023-11-08T19:27:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.582+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2b5804ece0 con 0x7f2b58144550 2023-11-08T19:27:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.589+0000 7f2b4f7fe640 1 -- 172.21.15.119:0/2468063110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2b4007a620 con 0x7f2b58144550 2023-11-08T19:27:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b3806aea0 msgr2=0x7f2b3806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b3806aea0 0x7f2b3806d360 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2b48008f10 tx=0x7f2b48008460 comp rx=0 tx=0).stop 2023-11-08T19:27:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 msgr2=0x7f2b580ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 0x7f2b580ca040 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2b40000c00 tx=0x7f2b400046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 shutdown_connections 2023-11-08T19:27:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2b3806aea0 0x7f2b3806d360 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2b5815bb10 0x7f2b580ca580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2b58144550 0x7f2b580ca040 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 --2- 172.21.15.119:0/2468063110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2b580cb8c0 0x7f2b58135dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 >> 172.21.15.119:0/2468063110 conn(0x7f2b580b0950 msgr2=0x7f2b580c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 shutdown_connections 2023-11-08T19:27:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.592+0000 7f2b5d4c9640 1 -- 172.21.15.119:0/2468063110 wait complete. 2023-11-08T19:27:26.611 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools' 2023-11-08T19:27:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.819+0000 7fb5ef76c640 1 Processor -- start 2023-11-08T19:27:26.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.820+0000 7fb5ef76c640 1 -- start start 2023-11-08T19:27:26.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ef76c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5e80ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ef76c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5e8161250 0x7fb5e81616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ef76c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e80cda70 con 0x7fb5e815b080 2023-11-08T19:27:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ef76c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e8161fc0 con 0x7fb5e8161250 2023-11-08T19:27:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ef76c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80cdcc0 con 0x7fb5e80cffd0 2023-11-08T19:27:26.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ed508640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb5e815b080 0x7fb5e80cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53558/0 (socket says 172.21.15.119:53558) 2023-11-08T19:27:26.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.821+0000 7fb5ed508640 1 -- 172.21.15.119:0/1199642272 learned_addr learned my addr 172.21.15.119:0/1199642272 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:26.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2019890411 0 0) 0x7fb5e80cda70 con 0x7fb5e815b080 2023-11-08T19:27:26.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5edd09640 1 --2- 172.21.15.119:0/1199642272 >> v2:172.21.15.188:3300/0 conn(0x7fb5e8161250 0x7fb5e81616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d0003520 con 0x7fb5e815b080 2023-11-08T19:27:26.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5ecd07640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5e80ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 80293635 0 0) 0x7fb5d0003520 con 0x7fb5e815b080 2023-11-08T19:27:26.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 msgr2=0x7fb5e80ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:26.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5e80ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 >> v2:172.21.15.188:3300/0 conn(0x7fb5e8161250 msgr2=0x7fb5e81616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:26.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/1199642272 >> v2:172.21.15.188:3300/0 conn(0x7fb5e8161250 0x7fb5e81616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80cdfa0 con 0x7fb5e815b080 2023-11-08T19:27:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fb5d80019f0 con 0x7fb5e815b080 2023-11-08T19:27:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5ecd07640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5e80ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 >> v1:172.21.15.119:6789/0 conn(0x7fb5e815b080 legacy=0x7fb5e80cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0003970 0x7fb5d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8161fc0 con 0x7fb5d0004d00 2023-11-08T19:27:26.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80cdcc0 con 0x7fb5d0003970 2023-11-08T19:27:26.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.822+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d000ba60 con 0x7fb5e80cffd0 2023-11-08T19:27:26.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5edd09640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5ed508640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0003970 0x7fb5d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5ecd07640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5edd09640 1 -- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 msgr2=0x7fb5d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:26.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5edd09640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80cffd0 0x7fb5d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5edd09640 1 -- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0003970 msgr2=0x7fb5d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5edd09640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d0003970 0x7fb5d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.823+0000 7fb5edd09640 1 -- 172.21.15.119:0/1199642272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d0003520 con 0x7fb5d0004d00 2023-11-08T19:27:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.824+0000 7fb5edd09640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5d0005180 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009a90 tx=0x7fb5d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3af908967b93bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.824+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e002c6b0 con 0x7fb5d0004d00 2023-11-08T19:27:26.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.824+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e002c810 con 0x7fb5d0004d00 2023-11-08T19:27:26.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.824+0000 7fb5e7fff640 1 -- 172.21.15.119:0/1199642272 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e002cba0 con 0x7fb5d0004d00 2023-11-08T19:27:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5ef76c640 1 -- 172.21.15.119:0/1199642272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 msgr2=0x7fb5d0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/1199642272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5d0005180 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009a90 tx=0x7fb5d00057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5edd09640 1 -- 172.21.15.119:0/1199642272 reap_dead start 2023-11-08T19:27:26.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5ef76c640 1 -- 172.21.15.119:0/1199642272 shutdown_connections 2023-11-08T19:27:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5ef76c640 1 -- 172.21.15.119:0/1199642272 >> 172.21.15.119:0/1199642272 conn(0x7fb5e806a9f0 msgr2=0x7fb5e806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5ef76c640 1 -- 172.21.15.119:0/1199642272 shutdown_connections 2023-11-08T19:27:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.825+0000 7fb5ef76c640 1 -- 172.21.15.119:0/1199642272 wait complete. 2023-11-08T19:27:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.827+0000 7fb5ef76c640 1 Processor -- start 2023-11-08T19:27:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.827+0000 7fb5ef76c640 1 -- start start 2023-11-08T19:27:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ef76c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5e80c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ef76c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80ca9d0 0x7fb5e80cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ef76c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80cc840 0x7fb5e80cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ef76c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80cb390 con 0x7fb5d0004d00 2023-11-08T19:27:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ef76c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e815eb40 con 0x7fb5e80cc840 2023-11-08T19:27:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ef76c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e815f6c0 con 0x7fb5e80ca9d0 2023-11-08T19:27:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ed508640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5e80c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ed508640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5e80c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44630/0 (socket says 172.21.15.119:44630) 2023-11-08T19:27:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.828+0000 7fb5ed508640 1 -- 172.21.15.119:0/4074395969 learned_addr learned my addr 172.21.15.119:0/4074395969 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5edd09640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80cc840 0x7fb5e80cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ecd07640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80ca9d0 0x7fb5e80cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ed508640 1 -- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80ca9d0 msgr2=0x7fb5e80cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ed508640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80ca9d0 0x7fb5e80cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ed508640 1 -- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80cc840 msgr2=0x7fb5e80cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ed508640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80cc840 0x7fb5e80cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ed508640 1 -- 172.21.15.119:0/4074395969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e814fe00 con 0x7fb5d0004d00 2023-11-08T19:27:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ed508640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5e80c5e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8007a80 tx=0x7fb5d800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d800eeb0 con 0x7fb5d0004d00 2023-11-08T19:27:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d8013420 con 0x7fb5d0004d00 2023-11-08T19:27:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80137f0 con 0x7fb5d0004d00 2023-11-08T19:27:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.829+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e815ed90 con 0x7fb5d0004d00 2023-11-08T19:27:26.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.830+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e815f350 con 0x7fb5d0004d00 2023-11-08T19:27:26.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.831+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e8074430 con 0x7fb5d0004d00 2023-11-08T19:27:26.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.832+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb5d8013950 con 0x7fb5d0004d00 2023-11-08T19:27:26.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.833+0000 7fb5e5ffb640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5d406aea0 0x7fb5d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:26.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.833+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb5d8089dd0 con 0x7fb5d0004d00 2023-11-08T19:27:26.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.833+0000 7fb5ecd07640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5d406aea0 0x7fb5d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:26.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.834+0000 7fb5ecd07640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5d406aea0 0x7fb5d406d360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb5e815e560 tx=0x7fb5dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:26.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:26.837+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb5d8054e60 con 0x7fb5d0004d00 2023-11-08T19:27:27.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.100+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]} v 0) v1 -- 0x7fb5e80d1120 con 0x7fb5d0004d00 2023-11-08T19:27:27.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.145+0000 7fb5e5ffb640 1 -- 172.21.15.119:0/4074395969 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_disallow_monitor_managed_snaps_for_fs_pools"]}]=0 v166) v1 ==== 178+0+0 (secure 0 0 0) 0x7fb5d8058480 con 0x7fb5d0004d00 2023-11-08T19:27:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.149+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5d406aea0 msgr2=0x7fb5d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.149+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5d406aea0 0x7fb5d406d360 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb5e815e560 tx=0x7fb5dc008040 comp rx=0 tx=0).stop 2023-11-08T19:27:27.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.149+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 msgr2=0x7fb5e80c5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.149+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5e80c5e70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8007a80 tx=0x7fb5d800e530 comp rx=0 tx=0).stop 2023-11-08T19:27:27.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.151+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 shutdown_connections 2023-11-08T19:27:27.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.151+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb5d406aea0 0x7fb5d406d360 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.151+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5e80cc840 0x7fb5e80cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.151+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5e80ca9d0 0x7fb5e80cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.151+0000 7fb5ef76c640 1 --2- 172.21.15.119:0/4074395969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb5d0004d00 0x7fb5e80c5e70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.151+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 >> 172.21.15.119:0/4074395969 conn(0x7fb5e806a9f0 msgr2=0x7fb5e80d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.152+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 shutdown_connections 2023-11-08T19:27:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.153+0000 7fb5ef76c640 1 -- 172.21.15.119:0/4074395969 wait complete. 2023-11-08T19:27:27.177 INFO:tasks.cephfs_test_runner:Test that creation of monitor managed snaps fails for pools attached ... ok 2023-11-08T19:27:27.178 INFO:tasks.cephfs_test_runner:Starting test: test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) Test that using a pool with snaps for fs creation fails 2023-11-08T19:27:27.178 INFO:tasks.cephfs_test_runner:test_using_pool_with_snap_fails_fs_creation (tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools) 2023-11-08T19:27:27.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.362+0000 7f6566487640 1 Processor -- start 2023-11-08T19:27:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.363+0000 7f6566487640 1 -- start start 2023-11-08T19:27:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.364+0000 7f6566487640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f656006a390 0x7f65600c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.364+0000 7f6566487640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f656015d180 0x7f656006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.364+0000 7f6566487640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65600ae410 con 0x7f65600cc810 2023-11-08T19:27:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.364+0000 7f6566487640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656006ba30 con 0x7f656015d180 2023-11-08T19:27:27.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.364+0000 7f6566487640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f656006bcf0 con 0x7f656006a390 2023-11-08T19:27:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6557fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f65600cc810 0x7f656015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53566/0 (socket says 172.21.15.119:53566) 2023-11-08T19:27:27.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6557fff640 1 -- 172.21.15.119:0/3860373554 learned_addr learned my addr 172.21.15.119:0/3860373554 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6564a24640 1 --2- 172.21.15.119:0/3860373554 >> v2:172.21.15.119:3300/0 conn(0x7f656015d180 0x7f656006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f655ffff640 1 --2- 172.21.15.119:0/3860373554 >> v2:172.21.15.188:3300/0 conn(0x7f656006a390 0x7f65600c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f655f7fe640 1 -- 172.21.15.119:0/3860373554 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227626647 0 0) 0x7f65600ae410 con 0x7f65600cc810 2023-11-08T19:27:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6564a24640 1 -- 172.21.15.119:0/3860373554 >> v2:172.21.15.188:3300/0 conn(0x7f656006a390 msgr2=0x7f65600c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6564a24640 1 --2- 172.21.15.119:0/3860373554 >> v2:172.21.15.188:3300/0 conn(0x7f656006a390 0x7f65600c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6564a24640 1 -- 172.21.15.119:0/3860373554 >> v1:172.21.15.119:6789/0 conn(0x7f65600cc810 legacy=0x7f656015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f6564a24640 1 -- 172.21.15.119:0/3860373554 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f656006bfd0 con 0x7f656015d180 2023-11-08T19:27:27.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.365+0000 7f655ffff640 1 --2- 172.21.15.119:0/3860373554 >> v2:172.21.15.188:3300/0 conn(0x7f656006a390 0x7f65600c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.367+0000 7f6564a24640 1 --2- 172.21.15.119:0/3860373554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f656015d180 0x7f656006cf70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f65600cfbd0 tx=0x7f655802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=768484a1122e7b5a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.367+0000 7f655f7fe640 1 -- 172.21.15.119:0/3860373554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655802cc70 con 0x7f656015d180 2023-11-08T19:27:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.367+0000 7f655f7fe640 1 -- 172.21.15.119:0/3860373554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655802cdd0 con 0x7f656015d180 2023-11-08T19:27:27.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.367+0000 7f655f7fe640 1 -- 172.21.15.119:0/3860373554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558034660 con 0x7f656015d180 2023-11-08T19:27:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.367+0000 7f6566487640 1 -- 172.21.15.119:0/3860373554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f656015d180 msgr2=0x7f656006cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.367+0000 7f6566487640 1 --2- 172.21.15.119:0/3860373554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f656015d180 0x7f656006cf70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f65600cfbd0 tx=0x7f655802bec0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.368+0000 7f6566487640 1 -- 172.21.15.119:0/3860373554 shutdown_connections 2023-11-08T19:27:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.368+0000 7f6566487640 1 --2- 172.21.15.119:0/3860373554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f656015d180 0x7f656006cf70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.368+0000 7f6566487640 1 --2- 172.21.15.119:0/3860373554 >> v2:172.21.15.188:3300/0 conn(0x7f656006a390 0x7f65600c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.368+0000 7f6566487640 1 -- 172.21.15.119:0/3860373554 >> 172.21.15.119:0/3860373554 conn(0x7f65600cb610 msgr2=0x7f65600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.368+0000 7f6566487640 1 -- 172.21.15.119:0/3860373554 shutdown_connections 2023-11-08T19:27:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.368+0000 7f6566487640 1 -- 172.21.15.119:0/3860373554 wait complete. 2023-11-08T19:27:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.369+0000 7f6566487640 1 Processor -- start 2023-11-08T19:27:27.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 -- start start 2023-11-08T19:27:27.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656006a390 0x7f6560157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 0x7f65600b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656015d180 0x7f656015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600b3b50 con 0x7f65600cc810 2023-11-08T19:27:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65600b25c0 con 0x7f656006a390 2023-11-08T19:27:27.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.370+0000 7f6566487640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65600b28a0 con 0x7f656015d180 2023-11-08T19:27:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 0x7f65600b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 0x7f65600b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44644/0 (socket says 172.21.15.119:44644) 2023-11-08T19:27:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 -- 172.21.15.119:0/4238424802 learned_addr learned my addr 172.21.15.119:0/4238424802 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f655ffff640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656006a390 0x7f6560157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6564a24640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656015d180 0x7f656015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 -- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656015d180 msgr2=0x7f656015f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656015d180 0x7f656015f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 -- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656006a390 msgr2=0x7f6560157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656006a390 0x7f6560157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.371+0000 7f6557fff640 1 -- 172.21.15.119:0/4238424802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6558009a70 con 0x7f65600cc810 2023-11-08T19:27:27.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.372+0000 7f6557fff640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 0x7f65600b3fe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f654800c4b0 tx=0x7f654800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.372+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65480074e0 con 0x7f65600cc810 2023-11-08T19:27:27.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.372+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6548007660 con 0x7f65600cc810 2023-11-08T19:27:27.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.372+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6548004a20 con 0x7f65600cc810 2023-11-08T19:27:27.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.372+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65600b3500 con 0x7f65600cc810 2023-11-08T19:27:27.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.372+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65600b4520 con 0x7f65600cc810 2023-11-08T19:27:27.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.373+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65600b4710 con 0x7f65600cc810 2023-11-08T19:27:27.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.374+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6548004b80 con 0x7f65600cc810 2023-11-08T19:27:27.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.375+0000 7f655d7fa640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f654006ae70 0x7f654006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.375+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f65480887b0 con 0x7f65600cc810 2023-11-08T19:27:27.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.375+0000 7f655ffff640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f654006ae70 0x7f654006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.376+0000 7f655ffff640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f654006ae70 0x7f654006d330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f65600c4050 tx=0x7f6550008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.380+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6548053980 con 0x7f65600cc810 2023-11-08T19:27:27.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.572+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6560074430 con 0x7f65600cc810 2023-11-08T19:27:27.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.575+0000 7f655d7fa640 1 -- 172.21.15.119:0/4238424802 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+14944 (secure 0 0 0) 0x7f6548018090 con 0x7f65600cc810 2023-11-08T19:27:27.577 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:27.577 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":57,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:26.138357+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/976565354":"2023-11-09T19:27:23.121290+0000","172.21.15.119:6835/976565354":"2023-11-09T19:27:23.121290+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:27.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f654006ae70 msgr2=0x7f654006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f654006ae70 0x7f654006d330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f65600c4050 tx=0x7f6550008040 comp rx=0 tx=0).stop 2023-11-08T19:27:27.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 msgr2=0x7f65600b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 0x7f65600b3fe0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f654800c4b0 tx=0x7f654800c970 comp rx=0 tx=0).stop 2023-11-08T19:27:27.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 shutdown_connections 2023-11-08T19:27:27.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f654006ae70 0x7f654006d330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f656015d180 0x7f656015f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f65600cc810 0x7f65600b3fe0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 --2- 172.21.15.119:0/4238424802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f656006a390 0x7f6560157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 >> 172.21.15.119:0/4238424802 conn(0x7f65600cb610 msgr2=0x7f656015b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 shutdown_connections 2023-11-08T19:27:27.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.578+0000 7f6566487640 1 -- 172.21.15.119:0/4238424802 wait complete. 2023-11-08T19:27:27.599 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:27.600 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:27.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.817+0000 7f6d601a9640 1 Processor -- start 2023-11-08T19:27:27.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.817+0000 7f6d601a9640 1 -- start start 2023-11-08T19:27:27.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d601a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d5815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d601a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5815e440 con 0x7f6d5806cb40 2023-11-08T19:27:27.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d601a9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5815e720 con 0x7f6d58143600 2023-11-08T19:27:27.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d601a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580c4f50 con 0x7f6d5815d9f0 2023-11-08T19:27:27.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d5e746640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6d5806cb40 0x7f6d580c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:53576/0 (socket says 172.21.15.119:53576) 2023-11-08T19:27:27.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d5e746640 1 -- 172.21.15.119:0/3155269985 learned_addr learned my addr 172.21.15.119:0/3155269985 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:27.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.819+0000 7f6d5d744640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d5815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821920277 0 0) 0x7f6d5815e440 con 0x7f6d5806cb40 2023-11-08T19:27:27.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d44005730 con 0x7f6d5806cb40 2023-11-08T19:27:27.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350102126 0 0) 0x7f6d44005730 con 0x7f6d5806cb40 2023-11-08T19:27:27.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 msgr2=0x7f6d5815de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d5815de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 >> v1:172.21.15.188:6789/0 conn(0x7f6d58143600 legacy=0x7f6d580c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d580c4b70 con 0x7f6d5806cb40 2023-11-08T19:27:27.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f6d500019f0 con 0x7f6d5806cb40 2023-11-08T19:27:27.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 >> v1:172.21.15.119:6789/0 conn(0x7f6d5806cb40 legacy=0x7f6d580c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d44005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44006230 0x7f6d440069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44006f10 0x7f6d44007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580c4f50 con 0x7f6d44006f10 2023-11-08T19:27:27.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4400de70 con 0x7f6d44006230 2023-11-08T19:27:27.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4400e030 con 0x7f6d5815d9f0 2023-11-08T19:27:27.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.820+0000 7f6d5e746640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44006f10 0x7f6d44007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5d744640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44006230 0x7f6d440069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5df45640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d44005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5e746640 1 -- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 msgr2=0x7f6d44005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5e746640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d44005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5e746640 1 -- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44006230 msgr2=0x7f6d440069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:27.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5e746640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44006230 0x7f6d440069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5e746640 1 -- 172.21.15.119:0/3155269985 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d44005730 con 0x7f6d44006f10 2023-11-08T19:27:27.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5d744640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d44006230 0x7f6d440069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:27.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.821+0000 7f6d5df45640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5815d9f0 0x7f6d44005bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.822+0000 7f6d5e746640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44006f10 0x7f6d44007390 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000a760 tx=0x7f6d44007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cbe07b9ece3bf15 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.822+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50034630 con 0x7f6d44006f10 2023-11-08T19:27:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.822+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5002f070 con 0x7f6d44006f10 2023-11-08T19:27:27.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.822+0000 7f6d5cf43640 1 -- 172.21.15.119:0/3155269985 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50034a50 con 0x7f6d44006f10 2023-11-08T19:27:27.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d601a9640 1 -- 172.21.15.119:0/3155269985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44006f10 msgr2=0x7f6d44007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d601a9640 1 --2- 172.21.15.119:0/3155269985 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d44006f10 0x7f6d44007390 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000a760 tx=0x7f6d44007a00 comp rx=0 tx=0).stop 2023-11-08T19:27:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d5e746640 1 -- 172.21.15.119:0/3155269985 reap_dead start 2023-11-08T19:27:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d601a9640 1 -- 172.21.15.119:0/3155269985 shutdown_connections 2023-11-08T19:27:27.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d601a9640 1 -- 172.21.15.119:0/3155269985 >> 172.21.15.119:0/3155269985 conn(0x7f6d580cb610 msgr2=0x7f6d580cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:27.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d601a9640 1 -- 172.21.15.119:0/3155269985 shutdown_connections 2023-11-08T19:27:27.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.823+0000 7f6d601a9640 1 -- 172.21.15.119:0/3155269985 wait complete. 2023-11-08T19:27:27.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.824+0000 7f6d601a9640 1 Processor -- start 2023-11-08T19:27:27.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.825+0000 7f6d601a9640 1 -- start start 2023-11-08T19:27:27.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d601a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44006f10 0x7f6d581379f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d601a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5815c870 0x7f6d580c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d601a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 0x7f6d580c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d601a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d580b1870 con 0x7f6d580c7e30 2023-11-08T19:27:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d601a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d58137f30 con 0x7f6d5815c870 2023-11-08T19:27:27.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d601a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d581f6430 con 0x7f6d44006f10 2023-11-08T19:27:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 0x7f6d580c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 0x7f6d580c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44664/0 (socket says 172.21.15.119:44664) 2023-11-08T19:27:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 -- 172.21.15.119:0/106876288 learned_addr learned my addr 172.21.15.119:0/106876288 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5d744640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5815c870 0x7f6d580c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 -- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44006f10 msgr2=0x7f6d581379f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44006f10 0x7f6d581379f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 -- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5815c870 msgr2=0x7f6d580c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5815c870 0x7f6d580c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5e746640 1 -- 172.21.15.119:0/106876288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d581f6710 con 0x7f6d580c7e30 2023-11-08T19:27:27.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.826+0000 7f6d5d744640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5815c870 0x7f6d580c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.827+0000 7f6d5e746640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 0x7f6d580c9d60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6d5002cab0 tx=0x7f6d50001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.827+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50034630 con 0x7f6d580c7e30 2023-11-08T19:27:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.827+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d50034790 con 0x7f6d580c7e30 2023-11-08T19:27:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.827+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50034b40 con 0x7f6d580c7e30 2023-11-08T19:27:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.827+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d581f69f0 con 0x7f6d580c7e30 2023-11-08T19:27:27.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.827+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5815ccf0 con 0x7f6d580c7e30 2023-11-08T19:27:27.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.828+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d580c4230 con 0x7f6d580c7e30 2023-11-08T19:27:27.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.829+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d50031030 con 0x7f6d580c7e30 2023-11-08T19:27:27.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.830+0000 7f6d4e7fc640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:27.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.830+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6d500ab5b0 con 0x7f6d580c7e30 2023-11-08T19:27:27.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.830+0000 7f6d5df45640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:27.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.831+0000 7f6d5df45640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d5813fc90 tx=0x7f6d54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:27.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:27.834+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d500767f0 con 0x7f6d580c7e30 2023-11-08T19:27:28.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.010+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d580c4230 con 0x7f6d580c7e30 2023-11-08T19:27:28.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.011+0000 7f6d4e7fc640 1 -- 172.21.15.119:0/106876288 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d50079e10 con 0x7f6d580c7e30 2023-11-08T19:27:28.012 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:28.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 msgr2=0x7f6d3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d5813fc90 tx=0x7f6d54009040 comp rx=0 tx=0).stop 2023-11-08T19:27:28.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 msgr2=0x7f6d580c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 0x7f6d580c9d60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6d5002cab0 tx=0x7f6d50001cb0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 shutdown_connections 2023-11-08T19:27:28.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d580c7e30 0x7f6d580c9d60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5815c870 0x7f6d580c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 --2- 172.21.15.119:0/106876288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d44006f10 0x7f6d581379f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 >> 172.21.15.119:0/106876288 conn(0x7f6d580cb610 msgr2=0x7f6d581407b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 shutdown_connections 2023-11-08T19:27:28.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.013+0000 7f6d601a9640 1 -- 172.21.15.119:0/106876288 wait complete. 2023-11-08T19:27:28.033 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:28.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.247+0000 7fa854ac9640 1 Processor -- start 2023-11-08T19:27:28.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.247+0000 7fa854ac9640 1 -- start start 2023-11-08T19:27:28.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.248+0000 7fa854ac9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa850144580 0x7fa85015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa854ac9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa854ac9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85015ea60 con 0x7fa85015f1d0 2023-11-08T19:27:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa854ac9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8500c9150 con 0x7fa850144580 2023-11-08T19:27:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa854ac9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500cb270 con 0x7fa850144960 2023-11-08T19:27:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa850144580 0x7fa85015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa850144580 0x7fa85015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44678/0 (socket says 172.21.15.119:44678) 2023-11-08T19:27:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 -- 172.21.15.119:0/1804659992 learned_addr learned my addr 172.21.15.119:0/1804659992 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 -- 172.21.15.119:0/1804659992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 msgr2=0x7fa8500cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 --2- 172.21.15.119:0/1804659992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 -- 172.21.15.119:0/1804659992 >> v1:172.21.15.188:6789/0 conn(0x7fa85015f1d0 legacy=0x7fa8500c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.249+0000 7fa84e59c640 1 -- 172.21.15.119:0/1804659992 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8500c9360 con 0x7fa850144580 2023-11-08T19:27:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.250+0000 7fa84e59c640 1 --2- 172.21.15.119:0/1804659992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa85015bff0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa840009870 tx=0x7fa84002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5541016d6886d033 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa84d59a640 1 -- 172.21.15.119:0/1804659992 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8400317c0 con 0x7fa850144580 2023-11-08T19:27:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa84d59a640 1 -- 172.21.15.119:0/1804659992 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa840031920 con 0x7fa850144580 2023-11-08T19:27:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa84d59a640 1 -- 172.21.15.119:0/1804659992 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840031cb0 con 0x7fa850144580 2023-11-08T19:27:28.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa854ac9640 1 -- 172.21.15.119:0/1804659992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 msgr2=0x7fa85015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa854ac9640 1 --2- 172.21.15.119:0/1804659992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa85015bff0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa840009870 tx=0x7fa84002b860 comp rx=0 tx=0).stop 2023-11-08T19:27:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa854ac9640 1 -- 172.21.15.119:0/1804659992 shutdown_connections 2023-11-08T19:27:28.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa854ac9640 1 --2- 172.21.15.119:0/1804659992 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa854ac9640 1 --2- 172.21.15.119:0/1804659992 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa85015bff0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.251+0000 7fa854ac9640 1 -- 172.21.15.119:0/1804659992 >> 172.21.15.119:0/1804659992 conn(0x7fa8500cb610 msgr2=0x7fa8500cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.252+0000 7fa854ac9640 1 -- 172.21.15.119:0/1804659992 shutdown_connections 2023-11-08T19:27:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.252+0000 7fa854ac9640 1 -- 172.21.15.119:0/1804659992 wait complete. 2023-11-08T19:27:28.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.253+0000 7fa854ac9640 1 Processor -- start 2023-11-08T19:27:28.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.253+0000 7fa854ac9640 1 -- start start 2023-11-08T19:27:28.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.254+0000 7fa854ac9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa8500c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.254+0000 7fa854ac9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.254+0000 7fa854ac9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85015f1d0 0x7fa8501393d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.254+0000 7fa854ac9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa850071a70 con 0x7fa850144580 2023-11-08T19:27:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.254+0000 7fa854ac9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa850070df0 con 0x7fa85015f1d0 2023-11-08T19:27:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.254+0000 7fa854ac9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8500710d0 con 0x7fa850144960 2023-11-08T19:27:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa8500c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa8500c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:44686/0 (socket says 172.21.15.119:44686) 2023-11-08T19:27:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 -- 172.21.15.119:0/421841909 learned_addr learned my addr 172.21.15.119:0/421841909 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84ed9d640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85015f1d0 0x7fa8501393d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84dd9b640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 -- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 msgr2=0x7fa8500cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 -- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85015f1d0 msgr2=0x7fa8501393d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85015f1d0 0x7fa8501393d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 -- 172.21.15.119:0/421841909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400094d0 con 0x7fa850144580 2023-11-08T19:27:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84ed9d640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85015f1d0 0x7fa8501393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84dd9b640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.255+0000 7fa84e59c640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa8500c6770 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa8400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.256+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840032860 con 0x7fa850144580 2023-11-08T19:27:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.256+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8400329e0 con 0x7fa850144580 2023-11-08T19:27:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.256+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa840032d90 con 0x7fa850144580 2023-11-08T19:27:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.256+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8500713b0 con 0x7fa850144580 2023-11-08T19:27:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.256+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa850070ac0 con 0x7fa850144580 2023-11-08T19:27:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.257+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa850071bb0 con 0x7fa850144580 2023-11-08T19:27:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.258+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa84004d3f0 con 0x7fa850144580 2023-11-08T19:27:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.259+0000 7fa83effd640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa83006ae70 0x7fa83006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.260+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa8400ab470 con 0x7fa850144580 2023-11-08T19:27:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.260+0000 7fa84dd9b640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa83006ae70 0x7fa83006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.260+0000 7fa84dd9b640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa83006ae70 0x7fa83006d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa844006fd0 tx=0x7fa844008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.264+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa850071bb0 con 0x7fa850144580 2023-11-08T19:27:28.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.452+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fa850071210 con 0x7fa850144580 2023-11-08T19:27:28.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.453+0000 7fa83effd640 1 -- 172.21.15.119:0/421841909 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v57) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa850071210 con 0x7fa850144580 2023-11-08T19:27:28.454 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:28.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa83006ae70 msgr2=0x7fa83006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa83006ae70 0x7fa83006d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa844006fd0 tx=0x7fa844008040 comp rx=0 tx=0).stop 2023-11-08T19:27:28.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 msgr2=0x7fa8500c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa8500c6770 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa840000c00 tx=0x7fa8400046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 shutdown_connections 2023-11-08T19:27:28.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa83006ae70 0x7fa83006d330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85015f1d0 0x7fa8501393d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa850144960 0x7fa8500cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 --2- 172.21.15.119:0/421841909 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa850144580 0x7fa8500c6770 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.455+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 >> 172.21.15.119:0/421841909 conn(0x7fa8500cb610 msgr2=0x7fa85015bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.456+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 shutdown_connections 2023-11-08T19:27:28.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.456+0000 7fa854ac9640 1 -- 172.21.15.119:0/421841909 wait complete. 2023-11-08T19:27:28.474 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2023-11-08T19:27:28.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.670+0000 7fce76955640 1 Processor -- start 2023-11-08T19:27:28.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.670+0000 7fce76955640 1 -- start start 2023-11-08T19:27:28.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce76955640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce7006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce76955640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce700c9150 0x7fce700cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce76955640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce700ccc10 0x7fce701653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce76955640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce701619b0 con 0x7fce700c9150 2023-11-08T19:27:28.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce76955640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce700cd0a0 con 0x7fce700ccc10 2023-11-08T19:27:28.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce76955640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7006d4b0 con 0x7fce7006a970 2023-11-08T19:27:28.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce6f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce700c9150 0x7fce700cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce6f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fce700c9150 0x7fce700cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33902/0 (socket says 172.21.15.119:33902) 2023-11-08T19:27:28.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.672+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2736997018 learned_addr learned my addr 172.21.15.119:0/2736997018 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:28.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce74ef2640 1 --2- 172.21.15.119:0/2736997018 >> v2:172.21.15.188:3300/0 conn(0x7fce700ccc10 0x7fce701653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6ffff640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce7006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2736997018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 msgr2=0x7fce7006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6f7fe640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce7006cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2736997018 >> v2:172.21.15.188:3300/0 conn(0x7fce700ccc10 msgr2=0x7fce701653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:28.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6f7fe640 1 --2- 172.21.15.119:0/2736997018 >> v2:172.21.15.188:3300/0 conn(0x7fce700ccc10 0x7fce701653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2736997018 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce70147b80 con 0x7fce700c9150 2023-11-08T19:27:28.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce74ef2640 1 --2- 172.21.15.119:0/2736997018 >> v2:172.21.15.188:3300/0 conn(0x7fce700ccc10 0x7fce701653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:28.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.673+0000 7fce6ffff640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce7006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:28.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.674+0000 7fce6f7fe640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce700cc6d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fce7006b1c0 tx=0x7fce6402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c7b5b41db85945 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.674+0000 7fce6effd640 1 -- 172.21.15.119:0/2736997018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640317e0 con 0x7fce700c9150 2023-11-08T19:27:28.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.674+0000 7fce6effd640 1 -- 172.21.15.119:0/2736997018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64031940 con 0x7fce700c9150 2023-11-08T19:27:28.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.674+0000 7fce6effd640 1 -- 172.21.15.119:0/2736997018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64031cd0 con 0x7fce700c9150 2023-11-08T19:27:28.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 -- 172.21.15.119:0/2736997018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 msgr2=0x7fce700cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce700cc6d0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fce7006b1c0 tx=0x7fce6402b880 comp rx=0 tx=0).stop 2023-11-08T19:27:28.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 -- 172.21.15.119:0/2736997018 shutdown_connections 2023-11-08T19:27:28.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 --2- 172.21.15.119:0/2736997018 >> v2:172.21.15.188:3300/0 conn(0x7fce700ccc10 0x7fce701653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce700cc6d0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 --2- 172.21.15.119:0/2736997018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce7006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 -- 172.21.15.119:0/2736997018 >> 172.21.15.119:0/2736997018 conn(0x7fce700cb610 msgr2=0x7fce700cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:28.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 -- 172.21.15.119:0/2736997018 shutdown_connections 2023-11-08T19:27:28.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.675+0000 7fce76955640 1 -- 172.21.15.119:0/2736997018 wait complete. 2023-11-08T19:27:28.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.677+0000 7fce76955640 1 Processor -- start 2023-11-08T19:27:28.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.677+0000 7fce76955640 1 -- start start 2023-11-08T19:27:28.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce76955640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce701f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce76955640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce7014a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce76955640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce700ccc10 0x7fce700702d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce76955640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70163160 con 0x7fce700c9150 2023-11-08T19:27:28.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce76955640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70163440 con 0x7fce700ccc10 2023-11-08T19:27:28.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce76955640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce70163720 con 0x7fce7006a970 2023-11-08T19:27:28.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce7014a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce7014a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33914/0 (socket says 172.21.15.119:33914) 2023-11-08T19:27:28.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2312156864 learned_addr learned my addr 172.21.15.119:0/2312156864 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:28.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2312156864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 msgr2=0x7fce701f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce701f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2312156864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce700ccc10 msgr2=0x7fce700702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:28.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce700ccc10 0x7fce700702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:28.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.678+0000 7fce6f7fe640 1 -- 172.21.15.119:0/2312156864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640094d0 con 0x7fce700c9150 2023-11-08T19:27:28.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.679+0000 7fce6f7fe640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce7014a4a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fce6402b8c0 tx=0x7fce640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.679+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640329c0 con 0x7fce700c9150 2023-11-08T19:27:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.679+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64032b40 con 0x7fce700c9150 2023-11-08T19:27:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.679+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce640453f0 con 0x7fce700c9150 2023-11-08T19:27:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.679+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce70163a00 con 0x7fce700c9150 2023-11-08T19:27:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.679+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7014a9e0 con 0x7fce700c9150 2023-11-08T19:27:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.680+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce70163f20 con 0x7fce700c9150 2023-11-08T19:27:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.681+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce6404d3f0 con 0x7fce700c9150 2023-11-08T19:27:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.682+0000 7fce6cff9640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce5c06aea0 0x7fce5c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.682+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fce640ab440 con 0x7fce700c9150 2023-11-08T19:27:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.683+0000 7fce6ffff640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce5c06aea0 0x7fce5c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:28.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.683+0000 7fce6ffff640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce5c06aea0 0x7fce5c06d360 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fce60006fd0 tx=0x7fce60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:28.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.687+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce70163f20 con 0x7fce700c9150 2023-11-08T19:27:28.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:28.950+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) v1 -- 0x7fce70163f20 con 0x7fce700c9150 2023-11-08T19:27:29.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.150+0000 7fce6cff9640 1 -- 172.21.15.119:0/2312156864 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v168) v1 ==== 176+0+0 (secure 0 0 0) 0x7fce70163f20 con 0x7fce700c9150 2023-11-08T19:27:29.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce5c06aea0 msgr2=0x7fce5c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce5c06aea0 0x7fce5c06d360 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fce60006fd0 tx=0x7fce60008040 comp rx=0 tx=0).stop 2023-11-08T19:27:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 msgr2=0x7fce7014a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce7014a4a0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fce6402b8c0 tx=0x7fce640046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 shutdown_connections 2023-11-08T19:27:29.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fce5c06aea0 0x7fce5c06d360 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce700ccc10 0x7fce700702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fce700c9150 0x7fce7014a4a0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 --2- 172.21.15.119:0/2312156864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce7006a970 0x7fce701f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 >> 172.21.15.119:0/2312156864 conn(0x7fce700cb610 msgr2=0x7fce700cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 shutdown_connections 2023-11-08T19:27:29.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.154+0000 7fce76955640 1 -- 172.21.15.119:0/2312156864 wait complete. 2023-11-08T19:27:29.175 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:27:29.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.390+0000 7f2afd51d640 1 Processor -- start 2023-11-08T19:27:29.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.390+0000 7f2afd51d640 1 -- start start 2023-11-08T19:27:29.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.392+0000 7f2afd51d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2af815d000 0x7f2af80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.392+0000 7f2afd51d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af806d220 con 0x7f2af806a830 2023-11-08T19:27:29.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.392+0000 7f2afd51d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af80aca40 con 0x7f2af80c3a50 2023-11-08T19:27:29.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.392+0000 7f2afd51d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af80ccd50 con 0x7f2af815d000 2023-11-08T19:27:29.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.392+0000 7f2af77fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2af806a830 0x7f2af8160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45458/0 (socket says 172.21.15.119:45458) 2023-11-08T19:27:29.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.392+0000 7f2af77fe640 1 -- 172.21.15.119:0/3168184399 learned_addr learned my addr 172.21.15.119:0/3168184399 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:29.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af6ffd640 1 --2- 172.21.15.119:0/3168184399 >> v2:172.21.15.188:3300/0 conn(0x7f2af815d000 0x7f2af80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 665106691 0 0) 0x7f2af806d220 con 0x7f2af806a830 2023-11-08T19:27:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae4003520 con 0x7f2af806a830 2023-11-08T19:27:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 900880666 0 0) 0x7f2ae4003520 con 0x7f2af806a830 2023-11-08T19:27:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 >> v2:172.21.15.188:3300/0 conn(0x7f2af815d000 msgr2=0x7f2af80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 --2- 172.21.15.119:0/3168184399 >> v2:172.21.15.188:3300/0 conn(0x7f2af815d000 0x7f2af80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 >> v1:172.21.15.188:6789/0 conn(0x7f2af80c3a50 legacy=0x7f2af806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80acd20 con 0x7f2af806a830 2023-11-08T19:27:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2af0001a10 con 0x7f2af806a830 2023-11-08T19:27:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 >> v1:172.21.15.119:6789/0 conn(0x7f2af806a830 legacy=0x7f2af8160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ae4003970 0x7f2ae4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae40047c0 0x7f2ae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80ccd50 con 0x7f2ae4005180 2023-11-08T19:27:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae400c120 con 0x7f2ae40047c0 2023-11-08T19:27:29.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.393+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae400c2e0 con 0x7f2ae4003970 2023-11-08T19:27:29.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af77fe640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af6ffd640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ae4003970 0x7f2ae4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2aee7fc640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae40047c0 0x7f2ae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af77fe640 1 -- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ae4003970 msgr2=0x7f2ae4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af77fe640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2ae4003970 0x7f2ae4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af77fe640 1 -- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae40047c0 msgr2=0x7f2ae4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:29.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af77fe640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae40047c0 0x7f2ae4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2af77fe640 1 -- 172.21.15.119:0/3168184399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae4003520 con 0x7f2ae4005180 2023-11-08T19:27:29.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.394+0000 7f2aee7fc640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2ae40047c0 0x7f2ae4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:29.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.395+0000 7f2af77fe640 1 -- 172.21.15.119:0/3168184399 reap_dead start 2023-11-08T19:27:29.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.395+0000 7f2af77fe640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2ae400b980 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2af8071ee0 tx=0x7f2ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50915b4babf5055 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.395+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0034630 con 0x7f2ae4005180 2023-11-08T19:27:29.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af002f070 con 0x7f2ae4005180 2023-11-08T19:27:29.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2af67fc640 1 -- 172.21.15.119:0/3168184399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af00349b0 con 0x7f2ae4005180 2023-11-08T19:27:29.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 -- 172.21.15.119:0/3168184399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 msgr2=0x7f2ae400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2ae400b980 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2af8071ee0 tx=0x7f2ae40057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 -- 172.21.15.119:0/3168184399 shutdown_connections 2023-11-08T19:27:29.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 --2- 172.21.15.119:0/3168184399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2ae400b980 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 -- 172.21.15.119:0/3168184399 >> 172.21.15.119:0/3168184399 conn(0x7f2af80cb610 msgr2=0x7f2af80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 -- 172.21.15.119:0/3168184399 shutdown_connections 2023-11-08T19:27:29.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.396+0000 7f2afd51d640 1 -- 172.21.15.119:0/3168184399 wait complete. 2023-11-08T19:27:29.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.398+0000 7f2afd51d640 1 Processor -- start 2023-11-08T19:27:29.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.398+0000 7f2afd51d640 1 -- start start 2023-11-08T19:27:29.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2afd51d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2af81f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2afd51d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2af81f3830 0x7f2af815f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2afd51d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af815fe40 0x7f2af80b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2afd51d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af81602d0 con 0x7f2ae4005180 2023-11-08T19:27:29.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2afd51d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b2510 con 0x7f2af815fe40 2023-11-08T19:27:29.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2afd51d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af80b27f0 con 0x7f2af81f3830 2023-11-08T19:27:29.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2af81f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2af6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2af81f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33928/0 (socket says 172.21.15.119:33928) 2023-11-08T19:27:29.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.399+0000 7f2af6ffd640 1 -- 172.21.15.119:0/671854737 learned_addr learned my addr 172.21.15.119:0/671854737 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:29.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2aee7fc640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2af81f3830 0x7f2af815f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af77fe640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af815fe40 0x7f2af80b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af6ffd640 1 -- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2af81f3830 msgr2=0x7f2af815f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af6ffd640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2af81f3830 0x7f2af815f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af6ffd640 1 -- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af815fe40 msgr2=0x7f2af80b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af6ffd640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af815fe40 0x7f2af80b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af6ffd640 1 -- 172.21.15.119:0/671854737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80b2ad0 con 0x7f2ae4005180 2023-11-08T19:27:29.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2af6ffd640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2af81f32f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8011540 tx=0x7f2ae8011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.400+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8015930 con 0x7f2ae4005180 2023-11-08T19:27:29.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.401+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8015a90 con 0x7f2ae4005180 2023-11-08T19:27:29.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.401+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8015e60 con 0x7f2ae4005180 2023-11-08T19:27:29.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.401+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af80b2db0 con 0x7f2ae4005180 2023-11-08T19:27:29.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.401+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af80b1db0 con 0x7f2ae4005180 2023-11-08T19:27:29.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.402+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af806cd50 con 0x7f2ae4005180 2023-11-08T19:27:29.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.403+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ae800f460 con 0x7f2ae4005180 2023-11-08T19:27:29.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.404+0000 7f2aeffff640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ad806ae70 0x7f2ad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:29.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.404+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2ae8089db0 con 0x7f2ae4005180 2023-11-08T19:27:29.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.404+0000 7f2aee7fc640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ad806ae70 0x7f2ad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:29.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.405+0000 7f2aee7fc640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ad806ae70 0x7f2ad806d330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2af80c5bf0 tx=0x7f2ae0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:29.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.408+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2ae808e070 con 0x7f2ae4005180 2023-11-08T19:27:29.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.597+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2af80c4fa0 con 0x7f2ae4005180 2023-11-08T19:27:29.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.604+0000 7f2aeffff640 1 -- 172.21.15.119:0/671854737 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v15)=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2ae8054ed0 con 0x7f2ae4005180 2023-11-08T19:27:29.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ad806ae70 msgr2=0x7f2ad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ad806ae70 0x7f2ad806d330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2af80c5bf0 tx=0x7f2ae0009040 comp rx=0 tx=0).stop 2023-11-08T19:27:29.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 msgr2=0x7f2af81f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:29.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2af81f32f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8011540 tx=0x7f2ae8011a00 comp rx=0 tx=0).stop 2023-11-08T19:27:29.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 shutdown_connections 2023-11-08T19:27:29.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ad806ae70 0x7f2ad806d330 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2af815fe40 0x7f2af80b1870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2af81f3830 0x7f2af815f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 --2- 172.21.15.119:0/671854737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2ae4005180 0x7f2af81f32f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:29.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 >> 172.21.15.119:0/671854737 conn(0x7f2af80cb610 msgr2=0x7f2af80712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:29.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 shutdown_connections 2023-11-08T19:27:29.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:29.607+0000 7f2afd51d640 1 -- 172.21.15.119:0/671854737 wait complete. 2023-11-08T19:27:29.626 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:29.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:29.738 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:29.863 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:29.864 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:29.864 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:29.924 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:27:29.925 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:27:29.925 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:29.986 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:27:29.987 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:29.987 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:30.048 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:27:30.048 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:27:30.049 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:30.109 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:27:30.109 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:27:30.110 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:30.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.328+0000 7f913a058640 1 Processor -- start 2023-11-08T19:27:30.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.328+0000 7f913a058640 1 -- start start 2023-11-08T19:27:30.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.329+0000 7f913a058640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.329+0000 7f913a058640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f913415d9f0 0x7f91340c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.329+0000 7f913a058640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f913406cb40 0x7f913406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.329+0000 7f913a058640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9134143a40 con 0x7f913415d9f0 2023-11-08T19:27:30.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.329+0000 7f913a058640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913406d510 con 0x7f913406cb40 2023-11-08T19:27:30.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.329+0000 7f913a058640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91340c4960 con 0x7f9134142f80 2023-11-08T19:27:30.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f913415d9f0 0x7f91340c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f913415d9f0 0x7f91340c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33940/0 (socket says 172.21.15.119:33940) 2023-11-08T19:27:30.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 -- 172.21.15.119:0/274351082 learned_addr learned my addr 172.21.15.119:0/274351082 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:30.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f91337fe640 1 --2- 172.21.15.119:0/274351082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9133fff640 1 --2- 172.21.15.119:0/274351082 >> v2:172.21.15.188:3300/0 conn(0x7f913406cb40 0x7f913406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 -- 172.21.15.119:0/274351082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 msgr2=0x7f91340cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:30.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 --2- 172.21.15.119:0/274351082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 -- 172.21.15.119:0/274351082 >> v2:172.21.15.188:3300/0 conn(0x7f913406cb40 msgr2=0x7f913406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:30.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 --2- 172.21.15.119:0/274351082 >> v2:172.21.15.188:3300/0 conn(0x7f913406cb40 0x7f913406cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.330+0000 7f9132ffd640 1 -- 172.21.15.119:0/274351082 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913415df60 con 0x7f913415d9f0 2023-11-08T19:27:30.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.331+0000 7f9132ffd640 1 --2- 172.21.15.119:0/274351082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91340c9cd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f91340cb550 tx=0x7f911c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f281d5e98a021ce2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.332+0000 7f91327fc640 1 -- 172.21.15.119:0/274351082 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0317d0 con 0x7f913415d9f0 2023-11-08T19:27:30.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.332+0000 7f91327fc640 1 -- 172.21.15.119:0/274351082 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c031930 con 0x7f913415d9f0 2023-11-08T19:27:30.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.332+0000 7f91327fc640 1 -- 172.21.15.119:0/274351082 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c031cc0 con 0x7f913415d9f0 2023-11-08T19:27:30.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.332+0000 7f913a058640 1 -- 172.21.15.119:0/274351082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 msgr2=0x7f91340c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.332+0000 7f913a058640 1 --2- 172.21.15.119:0/274351082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91340c9cd0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f91340cb550 tx=0x7f911c02b870 comp rx=0 tx=0).stop 2023-11-08T19:27:30.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 -- 172.21.15.119:0/274351082 shutdown_connections 2023-11-08T19:27:30.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 --2- 172.21.15.119:0/274351082 >> v2:172.21.15.188:3300/0 conn(0x7f913406cb40 0x7f913406cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 --2- 172.21.15.119:0/274351082 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91340c9cd0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 --2- 172.21.15.119:0/274351082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 -- 172.21.15.119:0/274351082 >> 172.21.15.119:0/274351082 conn(0x7f91340cb610 msgr2=0x7f91340cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:30.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 -- 172.21.15.119:0/274351082 shutdown_connections 2023-11-08T19:27:30.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.333+0000 7f913a058640 1 -- 172.21.15.119:0/274351082 wait complete. 2023-11-08T19:27:30.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.335+0000 7f913a058640 1 Processor -- start 2023-11-08T19:27:30.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.335+0000 7f913a058640 1 -- start start 2023-11-08T19:27:30.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.336+0000 7f913a058640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913406cb40 0x7f91340b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.336+0000 7f913a058640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.336+0000 7f913a058640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91341f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.336+0000 7f913a058640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340b48d0 con 0x7f913415d9f0 2023-11-08T19:27:30.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.336+0000 7f913a058640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91340b45f0 con 0x7f913406cb40 2023-11-08T19:27:30.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.336+0000 7f913a058640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913406f9d0 con 0x7f9134142f80 2023-11-08T19:27:30.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91341f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91341f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33952/0 (socket says 172.21.15.119:33952) 2023-11-08T19:27:30.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 -- 172.21.15.119:0/425105850 learned_addr learned my addr 172.21.15.119:0/425105850 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:30.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f91337fe640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913406cb40 0x7f91340b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9132ffd640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 -- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 msgr2=0x7f91340b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 -- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913406cb40 msgr2=0x7f91340b19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913406cb40 0x7f91340b19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 -- 172.21.15.119:0/425105850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0094d0 con 0x7f913415d9f0 2023-11-08T19:27:30.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.337+0000 7f9133fff640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91341f63f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c4c0 tx=0x7f912c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.338+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c004220 con 0x7f913415d9f0 2023-11-08T19:27:30.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.338+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912c004380 con 0x7f913415d9f0 2023-11-08T19:27:30.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.338+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912c003810 con 0x7f913415d9f0 2023-11-08T19:27:30.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.338+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913406fcb0 con 0x7f913415d9f0 2023-11-08T19:27:30.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.338+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91340b41f0 con 0x7f913415d9f0 2023-11-08T19:27:30.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.339+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9134074430 con 0x7f913415d9f0 2023-11-08T19:27:30.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.340+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f912c003970 con 0x7f913415d9f0 2023-11-08T19:27:30.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.341+0000 7f912bfff640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f911406ae70 0x7f911406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.341+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f912c089220 con 0x7f913415d9f0 2023-11-08T19:27:30.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.341+0000 7f91337fe640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f911406ae70 0x7f911406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.342+0000 7f91337fe640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f911406ae70 0x7f911406d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f91240099c0 tx=0x7f9124003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.346+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f912c054330 con 0x7f913415d9f0 2023-11-08T19:27:30.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.571+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91340c4230 con 0x7f913415d9f0 2023-11-08T19:27:30.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.572+0000 7f912bfff640 1 -- 172.21.15.119:0/425105850 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f912c01ee90 con 0x7f913415d9f0 2023-11-08T19:27:30.574 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:30.575 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4673,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2030798279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2030798279},{"type":"v1","addr":"172.21.15.188:6833","nonce":2030798279}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":4981,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/386871148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":386871148},{"type":"v1","addr":"172.21.15.188:6835","nonce":386871148}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":5730,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3828662266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3828662266},{"type":"v1","addr":"172.21.15.119:6836","nonce":3828662266}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":21},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/605361247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":605361247},{"type":"v1","addr":"172.21.15.119:6837","nonce":605361247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-11-08T19:27:30.575 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 28 2023-11-08T19:27:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.574+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f911406ae70 msgr2=0x7f911406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.574+0000 7f913a058640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f911406ae70 0x7f911406d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f91240099c0 tx=0x7f9124003990 comp rx=0 tx=0).stop 2023-11-08T19:27:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.574+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 msgr2=0x7f91341f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.574+0000 7f913a058640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91341f63f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c4c0 tx=0x7f912c00c980 comp rx=0 tx=0).stop 2023-11-08T19:27:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 shutdown_connections 2023-11-08T19:27:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f911406ae70 0x7f911406d330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f913415d9f0 0x7f91341f63f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9134142f80 0x7f91340b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 --2- 172.21.15.119:0/425105850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f913406cb40 0x7f91340b19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 >> 172.21.15.119:0/425105850 conn(0x7f91340cb610 msgr2=0x7f91340b0ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 shutdown_connections 2023-11-08T19:27:30.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.575+0000 7f913a058640 1 -- 172.21.15.119:0/425105850 wait complete. 2023-11-08T19:27:30.594 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:27:30.595 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:27:30.595 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:27:30.596 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:30.596 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:27:30.596 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:27:30.597 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:27:30.597 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:30.597 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:27:30.597 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:27:30.598 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:27:30.598 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:30.598 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:27:30.599 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:27:30.599 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:27:30.599 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:27:30.617 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:27:30.617 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:27:30.618 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:27:30.619 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:27:30.624 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:27:30.624 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:27:30.625 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:27:30.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:27:30.667 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:27:30.667 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:30.673 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:27:30.674 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:30.675 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:27:30.676 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:30.681 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:27:30.681 DEBUG:teuthology.parallel:result is None 2023-11-08T19:27:30.682 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:27:30.827 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:27:30.830 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:27:30.838 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:27:30.841 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:27:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.870+0000 7efede07c640 1 Processor -- start 2023-11-08T19:27:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.870+0000 7efede07c640 1 -- start start 2023-11-08T19:27:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efede07c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efed8142f80 0x7efed80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efede07c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efede07c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efed806cb40 0x7efed806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efede07c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed8143a40 con 0x7efed8142f80 2023-11-08T19:27:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efede07c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efed806d510 con 0x7efed806cb40 2023-11-08T19:27:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efede07c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed80c4960 con 0x7efed815d9f0 2023-11-08T19:27:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efed77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efed8142f80 0x7efed80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efed77fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efed8142f80 0x7efed80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33994/0 (socket says 172.21.15.119:33994) 2023-11-08T19:27:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efed77fe640 1 -- 172.21.15.119:0/1460010527 learned_addr learned my addr 172.21.15.119:0/1460010527 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.871+0000 7efed7fff640 1 --2- 172.21.15.119:0/1460010527 >> v2:172.21.15.188:3300/0 conn(0x7efed806cb40 0x7efed806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efecffff640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efed77fe640 1 -- 172.21.15.119:0/1460010527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 msgr2=0x7efed80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efed77fe640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efed77fe640 1 -- 172.21.15.119:0/1460010527 >> v2:172.21.15.188:3300/0 conn(0x7efed806cb40 msgr2=0x7efed806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efed77fe640 1 --2- 172.21.15.119:0/1460010527 >> v2:172.21.15.188:3300/0 conn(0x7efed806cb40 0x7efed806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efed77fe640 1 -- 172.21.15.119:0/1460010527 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed815df60 con 0x7efed8142f80 2023-11-08T19:27:30.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efecffff640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:30.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.872+0000 7efed7fff640 1 --2- 172.21.15.119:0/1460010527 >> v2:172.21.15.188:3300/0 conn(0x7efed806cb40 0x7efed806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:30.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.873+0000 7efed77fe640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed80cc6d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efed80cb550 tx=0x7efec802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604441224110e4d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.873+0000 7efed6ffd640 1 -- 172.21.15.119:0/1460010527 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec80317e0 con 0x7efed8142f80 2023-11-08T19:27:30.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.873+0000 7efed6ffd640 1 -- 172.21.15.119:0/1460010527 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec8031940 con 0x7efed8142f80 2023-11-08T19:27:30.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.873+0000 7efed6ffd640 1 -- 172.21.15.119:0/1460010527 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec8031cd0 con 0x7efed8142f80 2023-11-08T19:27:30.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 -- 172.21.15.119:0/1460010527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 msgr2=0x7efed80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed80cc6d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efed80cb550 tx=0x7efec802b880 comp rx=0 tx=0).stop 2023-11-08T19:27:30.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 -- 172.21.15.119:0/1460010527 shutdown_connections 2023-11-08T19:27:30.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 --2- 172.21.15.119:0/1460010527 >> v2:172.21.15.188:3300/0 conn(0x7efed806cb40 0x7efed806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 --2- 172.21.15.119:0/1460010527 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed80cc6d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 -- 172.21.15.119:0/1460010527 >> 172.21.15.119:0/1460010527 conn(0x7efed80cb610 msgr2=0x7efed80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:30.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 -- 172.21.15.119:0/1460010527 shutdown_connections 2023-11-08T19:27:30.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.874+0000 7efede07c640 1 -- 172.21.15.119:0/1460010527 wait complete. 2023-11-08T19:27:30.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.876+0000 7efede07c640 1 Processor -- start 2023-11-08T19:27:30.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.876+0000 7efede07c640 1 -- start start 2023-11-08T19:27:30.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efede07c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efed806cb40 0x7efed80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efede07c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed81f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efede07c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efede07c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed81f6e60 con 0x7efed8142f80 2023-11-08T19:27:30.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efede07c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efed8159f10 con 0x7efed806cb40 2023-11-08T19:27:30.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efede07c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efed8137270 con 0x7efed815d9f0 2023-11-08T19:27:30.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efecffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed81f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efecffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed81f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34008/0 (socket says 172.21.15.119:34008) 2023-11-08T19:27:30.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efecffff640 1 -- 172.21.15.119:0/2996081772 learned_addr learned my addr 172.21.15.119:0/2996081772 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:30.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efed7fff640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.877+0000 7efed77fe640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efed806cb40 0x7efed80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efecffff640 1 -- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 msgr2=0x7efed80c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efecffff640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efecffff640 1 -- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efed806cb40 msgr2=0x7efed80b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:30.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efecffff640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efed806cb40 0x7efed80b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:30.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efecffff640 1 -- 172.21.15.119:0/2996081772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec80094d0 con 0x7efed8142f80 2023-11-08T19:27:30.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efed77fe640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efed806cb40 0x7efed80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:30.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efecffff640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed81f6920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efec000c7f0 tx=0x7efec000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00079d0 con 0x7efed8142f80 2023-11-08T19:27:30.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec0007b30 con 0x7efed8142f80 2023-11-08T19:27:30.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec000ce40 con 0x7efed8142f80 2023-11-08T19:27:30.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efed81366f0 con 0x7efed8142f80 2023-11-08T19:27:30.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.878+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efed8136cb0 con 0x7efed8142f80 2023-11-08T19:27:30.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.879+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efed80cafb0 con 0x7efed8142f80 2023-11-08T19:27:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.881+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efec00113f0 con 0x7efed8142f80 2023-11-08T19:27:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.882+0000 7efed4ff9640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efeb806aea0 0x7efeb806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.882+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7efec00895b0 con 0x7efed8142f80 2023-11-08T19:27:30.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.882+0000 7efed77fe640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efeb806aea0 0x7efeb806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.883+0000 7efed77fe640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efeb806aea0 0x7efeb806d360 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efec8036000 tx=0x7efec803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:30.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:30.886+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efec0054800 con 0x7efed8142f80 2023-11-08T19:27:31.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.080+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7efed81342e0 con 0x7efed8142f80 2023-11-08T19:27:31.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.176+0000 7efed4ff9640 1 -- 172.21.15.119:0/2996081772 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v59) v1 ==== 97+0+0 (secure 0 0 0) 0x7efed81342e0 con 0x7efed8142f80 2023-11-08T19:27:31.178 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-11-08T19:27:31.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efeb806aea0 msgr2=0x7efeb806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efeb806aea0 0x7efeb806d360 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efec8036000 tx=0x7efec803d040 comp rx=0 tx=0).stop 2023-11-08T19:27:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 msgr2=0x7efed81f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed81f6920 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efec000c7f0 tx=0x7efec000cbf0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 shutdown_connections 2023-11-08T19:27:31.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efeb806aea0 0x7efeb806d360 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efed815d9f0 0x7efed80c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efed8142f80 0x7efed81f6920 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 --2- 172.21.15.119:0/2996081772 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efed806cb40 0x7efed80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 >> 172.21.15.119:0/2996081772 conn(0x7efed80cb610 msgr2=0x7efed80cd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 shutdown_connections 2023-11-08T19:27:31.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.179+0000 7efede07c640 1 -- 172.21.15.119:0/2996081772 wait complete. 2023-11-08T19:27:31.201 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:27:31.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.392+0000 7f5d5b800640 1 Processor -- start 2023-11-08T19:27:31.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.393+0000 7f5d5b800640 1 -- start start 2023-11-08T19:27:31.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d5b800640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5d5415d380 0x7f5d540cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d5b800640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d540cf6c0 0x7f5d540d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d5b800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d540d1380 0x7f5d540b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d5b800640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540d1810 con 0x7f5d540cf6c0 2023-11-08T19:27:31.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d5b800640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d5415eb30 con 0x7f5d5415d380 2023-11-08T19:27:31.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d5b800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5415e570 con 0x7f5d540d1380 2023-11-08T19:27:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d58d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d540cf6c0 0x7f5d540d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d58d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5d540cf6c0 0x7f5d540d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34016/0 (socket says 172.21.15.119:34016) 2023-11-08T19:27:31.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.394+0000 7f5d58d9b640 1 -- 172.21.15.119:0/2413197383 learned_addr learned my addr 172.21.15.119:0/2413197383 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:31.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d59d9d640 1 --2- 172.21.15.119:0/2413197383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d540d1380 0x7f5d540b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d5959c640 1 --2- 172.21.15.119:0/2413197383 >> v2:172.21.15.188:3300/0 conn(0x7f5d5415d380 0x7f5d540cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d58d9b640 1 -- 172.21.15.119:0/2413197383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d540d1380 msgr2=0x7f5d540b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:31.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d58d9b640 1 --2- 172.21.15.119:0/2413197383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d540d1380 0x7f5d540b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d58d9b640 1 -- 172.21.15.119:0/2413197383 >> v2:172.21.15.188:3300/0 conn(0x7f5d5415d380 msgr2=0x7f5d540cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:31.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d58d9b640 1 --2- 172.21.15.119:0/2413197383 >> v2:172.21.15.188:3300/0 conn(0x7f5d5415d380 0x7f5d540cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d58d9b640 1 -- 172.21.15.119:0/2413197383 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5415e850 con 0x7f5d540cf6c0 2023-11-08T19:27:31.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.395+0000 7f5d5959c640 1 --2- 172.21.15.119:0/2413197383 >> v2:172.21.15.188:3300/0 conn(0x7f5d5415d380 0x7f5d540cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:31.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.396+0000 7f5d58d9b640 1 --2- 172.21.15.119:0/2413197383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d540d0e40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5d54070200 tx=0x7f5d4402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c6981b133f2b7d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.396+0000 7f5d4bfff640 1 -- 172.21.15.119:0/2413197383 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44031860 con 0x7f5d540cf6c0 2023-11-08T19:27:31.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.396+0000 7f5d4bfff640 1 -- 172.21.15.119:0/2413197383 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440319c0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.396+0000 7f5d4bfff640 1 -- 172.21.15.119:0/2413197383 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44031d50 con 0x7f5d540cf6c0 2023-11-08T19:27:31.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 -- 172.21.15.119:0/2413197383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 msgr2=0x7f5d540d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 --2- 172.21.15.119:0/2413197383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d540d0e40 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5d54070200 tx=0x7f5d4402b900 comp rx=0 tx=0).stop 2023-11-08T19:27:31.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 -- 172.21.15.119:0/2413197383 shutdown_connections 2023-11-08T19:27:31.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 --2- 172.21.15.119:0/2413197383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d540d1380 0x7f5d540b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 --2- 172.21.15.119:0/2413197383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d540d0e40 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 --2- 172.21.15.119:0/2413197383 >> v2:172.21.15.188:3300/0 conn(0x7f5d5415d380 0x7f5d540cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 -- 172.21.15.119:0/2413197383 >> 172.21.15.119:0/2413197383 conn(0x7f5d5408afe0 msgr2=0x7f5d5409a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:31.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 -- 172.21.15.119:0/2413197383 shutdown_connections 2023-11-08T19:27:31.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.397+0000 7f5d5b800640 1 -- 172.21.15.119:0/2413197383 wait complete. 2023-11-08T19:27:31.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.399+0000 7f5d5b800640 1 Processor -- start 2023-11-08T19:27:31.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.399+0000 7f5d5b800640 1 -- start start 2023-11-08T19:27:31.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.400+0000 7f5d5b800640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d5414e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5b800640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d540d1380 0x7f5d5406b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5b800640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5415d380 0x7f5d5406b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5b800640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5414e860 con 0x7f5d540cf6c0 2023-11-08T19:27:31.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5b800640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c7790 con 0x7f5d540d1380 2023-11-08T19:27:31.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5b800640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540b3ad0 con 0x7f5d5415d380 2023-11-08T19:27:31.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d5414e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d5414e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34024/0 (socket says 172.21.15.119:34024) 2023-11-08T19:27:31.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 -- 172.21.15.119:0/563910145 learned_addr learned my addr 172.21.15.119:0/563910145 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 -- 172.21.15.119:0/563910145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5415d380 msgr2=0x7f5d5406b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5415d380 0x7f5d5406b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 -- 172.21.15.119:0/563910145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d540d1380 msgr2=0x7f5d5406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d540d1380 0x7f5d5406b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.401+0000 7f5d5959c640 1 -- 172.21.15.119:0/563910145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d440094d0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.402+0000 7f5d5959c640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d5414e320 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c7f0 tx=0x7f5d5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.402+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50007820 con 0x7f5d540cf6c0 2023-11-08T19:27:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.402+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d50007980 con 0x7f5d540cf6c0 2023-11-08T19:27:31.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.402+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d50007cf0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.402+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d540b3db0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.402+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d540b4370 con 0x7f5d540cf6c0 2023-11-08T19:27:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.403+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d54074430 con 0x7f5d540cf6c0 2023-11-08T19:27:31.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.404+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d500113f0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.405+0000 7f5d49ffb640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.405+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5d500899a0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.405+0000 7f5d58d9b640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.406+0000 7f5d58d9b640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d44036000 tx=0x7f5d44030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.410+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5d500549d0 con 0x7f5d540cf6c0 2023-11-08T19:27:31.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.652+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5d54073770 con 0x7f5d540cf6c0 2023-11-08T19:27:31.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.653+0000 7f5d49ffb640 1 -- 172.21.15.119:0/563910145 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5d5001f020 con 0x7f5d540cf6c0 2023-11-08T19:27:31.655 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:31.655 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:27:31.655 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-11-08T19:27:31.656 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.656 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-11-08T19:27:31.656 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg==", 2023-11-08T19:27:31.656 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.656 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:31.657 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:31.657 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.657 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.657 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.658 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.658 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-11-08T19:27:31.658 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ==", 2023-11-08T19:27:31.658 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.658 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:31.659 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:31.659 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.659 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.659 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.659 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.660 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-11-08T19:27:31.660 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw==", 2023-11-08T19:27:31.660 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.660 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.660 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:31.661 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:31.661 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.661 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.661 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.661 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.662 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-11-08T19:27:31.662 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ==", 2023-11-08T19:27:31.662 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.662 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:31.663 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:31.663 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.663 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.663 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.663 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.663 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-11-08T19:27:31.664 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA==", 2023-11-08T19:27:31.664 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.664 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.664 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.665 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.665 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.665 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.665 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-11-08T19:27:31.665 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A==", 2023-11-08T19:27:31.666 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.666 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.666 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.666 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.667 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.667 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.667 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-11-08T19:27:31.667 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA==", 2023-11-08T19:27:31.667 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.668 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.668 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.668 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.668 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.668 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.669 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-11-08T19:27:31.669 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ==", 2023-11-08T19:27:31.669 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.669 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.669 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.670 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.670 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.670 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.670 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.670 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-11-08T19:27:31.671 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ==", 2023-11-08T19:27:31.671 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.671 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.671 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.671 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.672 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.672 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.672 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.672 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-11-08T19:27:31.672 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw==", 2023-11-08T19:27:31.673 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.673 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.673 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.673 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.673 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.673 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.674 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.674 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-11-08T19:27:31.674 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g==", 2023-11-08T19:27:31.674 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.674 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.675 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.675 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.675 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.675 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.675 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.676 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-11-08T19:27:31.676 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ==", 2023-11-08T19:27:31.676 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:27:31.676 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:27:31.677 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.677 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.677 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.677 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.677 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-11-08T19:27:31.678 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw==", 2023-11-08T19:27:31.678 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.678 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-11-08T19:27:31.678 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:31.679 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.679 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.679 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.679 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-11-08T19:27:31.679 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ==", 2023-11-08T19:27:31.679 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:27:31.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:27:31.680 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:27:31.680 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.681 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.681 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.681 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-11-08T19:27:31.681 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q==", 2023-11-08T19:27:31.681 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.682 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.682 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:27:31.682 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:27:31.682 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:27:31.682 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.683 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.683 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.683 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-11-08T19:27:31.683 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ==", 2023-11-08T19:27:31.683 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.684 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:27:31.684 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:27:31.684 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:27:31.684 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:27:31.684 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.684 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.685 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.685 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-11-08T19:27:31.685 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBH4Etl5+oDFBAAMV8QbfI7L35tUuiNz9bvdw==", 2023-11-08T19:27:31.685 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.685 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:27:31.686 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:27:31.686 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:27:31.686 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.686 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.686 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.687 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.687 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-11-08T19:27:31.687 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA==", 2023-11-08T19:27:31.687 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.687 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:27:31.688 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:31.688 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.688 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.688 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:31.688 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:31.689 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-11-08T19:27:31.689 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ==", 2023-11-08T19:27:31.689 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:27:31.689 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:27:31.689 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:27:31.689 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:27:31.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.690 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:31.690 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-11-08T19:27:31.690 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:27:31.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 msgr2=0x7f5d3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5d44036000 tx=0x7f5d44030000 comp rx=0 tx=0).stop 2023-11-08T19:27:31.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 msgr2=0x7f5d5414e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d5414e320 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f5d5000c7f0 tx=0x7f5d5000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 shutdown_connections 2023-11-08T19:27:31.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5d3806ae70 0x7f5d3806d330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5d5415d380 0x7f5d5406b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5d540d1380 0x7f5d5406b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 --2- 172.21.15.119:0/563910145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5d540cf6c0 0x7f5d5414e320 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 >> 172.21.15.119:0/563910145 conn(0x7f5d5408afe0 msgr2=0x7f5d5415c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:31.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 shutdown_connections 2023-11-08T19:27:31.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.656+0000 7f5d5b800640 1 -- 172.21.15.119:0/563910145 wait complete. 2023-11-08T19:27:31.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:31.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.908+0000 7f2a8e639640 1 Processor -- start 2023-11-08T19:27:31.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.909+0000 7f2a8e639640 1 -- start start 2023-11-08T19:27:31.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.910+0000 7f2a8e639640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a8806a390 0x7f2a880c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.910+0000 7f2a8e639640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a8815d180 0x7f2a8806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.910+0000 7f2a8e639640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a880ae410 con 0x7f2a880cc810 2023-11-08T19:27:31.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.910+0000 7f2a8e639640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8806ba30 con 0x7f2a8815d180 2023-11-08T19:27:31.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.910+0000 7f2a8e639640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8806bcf0 con 0x7f2a8806a390 2023-11-08T19:27:31.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.910+0000 7f2a8cbd6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a8815d180 0x7f2a8806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a8cbd6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2a8815d180 0x7f2a8806cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34034/0 (socket says 172.21.15.119:34034) 2023-11-08T19:27:31.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/670182220 learned_addr learned my addr 172.21.15.119:0/670182220 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:31.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a87fff640 1 --2- 172.21.15.119:0/670182220 >> v2:172.21.15.188:3300/0 conn(0x7f2a8806a390 0x7f2a880c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/670182220 >> v2:172.21.15.188:3300/0 conn(0x7f2a8806a390 msgr2=0x7f2a880c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:31.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a8cbd6640 1 --2- 172.21.15.119:0/670182220 >> v2:172.21.15.188:3300/0 conn(0x7f2a8806a390 0x7f2a880c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/670182220 >> v1:172.21.15.119:6789/0 conn(0x7f2a880cc810 legacy=0x7f2a8815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/670182220 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8806bfd0 con 0x7f2a8815d180 2023-11-08T19:27:31.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.911+0000 7f2a87fff640 1 --2- 172.21.15.119:0/670182220 >> v2:172.21.15.188:3300/0 conn(0x7f2a8806a390 0x7f2a880c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:31.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.912+0000 7f2a8cbd6640 1 --2- 172.21.15.119:0/670182220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8806cf70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2a880cfbd0 tx=0x7f2a8002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2603e3b4faa61b9d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.912+0000 7f2a877fe640 1 -- 172.21.15.119:0/670182220 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8002cb20 con 0x7f2a8815d180 2023-11-08T19:27:31.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.912+0000 7f2a877fe640 1 -- 172.21.15.119:0/670182220 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8002cc80 con 0x7f2a8815d180 2023-11-08T19:27:31.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.912+0000 7f2a877fe640 1 -- 172.21.15.119:0/670182220 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a800343f0 con 0x7f2a8815d180 2023-11-08T19:27:31.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 -- 172.21.15.119:0/670182220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 msgr2=0x7f2a8806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 --2- 172.21.15.119:0/670182220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8806cf70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2a880cfbd0 tx=0x7f2a8002bf80 comp rx=0 tx=0).stop 2023-11-08T19:27:31.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 -- 172.21.15.119:0/670182220 shutdown_connections 2023-11-08T19:27:31.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 --2- 172.21.15.119:0/670182220 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8806cf70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 --2- 172.21.15.119:0/670182220 >> v2:172.21.15.188:3300/0 conn(0x7f2a8806a390 0x7f2a880c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 -- 172.21.15.119:0/670182220 >> 172.21.15.119:0/670182220 conn(0x7f2a880cb610 msgr2=0x7f2a880cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:31.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 -- 172.21.15.119:0/670182220 shutdown_connections 2023-11-08T19:27:31.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.913+0000 7f2a8e639640 1 -- 172.21.15.119:0/670182220 wait complete. 2023-11-08T19:27:31.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.915+0000 7f2a8e639640 1 Processor -- start 2023-11-08T19:27:31.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.915+0000 7f2a8e639640 1 -- start start 2023-11-08T19:27:31.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8e639640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8806a390 0x7f2a881f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8e639640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880cc810 0x7f2a881f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8e639640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8815fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8e639640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881f6b90 con 0x7f2a8815d180 2023-11-08T19:27:31.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8e639640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881f2c60 con 0x7f2a880cc810 2023-11-08T19:27:31.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8e639640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a881f2f40 con 0x7f2a8806a390 2023-11-08T19:27:31.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8815fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8815fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34038/0 (socket says 172.21.15.119:34038) 2023-11-08T19:27:31.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/1061836253 learned_addr learned my addr 172.21.15.119:0/1061836253 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:31.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a87fff640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8806a390 0x7f2a881f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8806a390 msgr2=0x7f2a881f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8806a390 0x7f2a881f25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a7ffff640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880cc810 0x7f2a881f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880cc810 msgr2=0x7f2a881f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:31.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.916+0000 7f2a8cbd6640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880cc810 0x7f2a881f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:31.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a8cbd6640 1 -- 172.21.15.119:0/1061836253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80009b30 con 0x7f2a8815d180 2023-11-08T19:27:31.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a8cbd6640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8815fdd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2a80036040 tx=0x7f2a80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80007e70 con 0x7f2a8815d180 2023-11-08T19:27:31.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a80009510 con 0x7f2a8815d180 2023-11-08T19:27:31.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8003e690 con 0x7f2a8815d180 2023-11-08T19:27:31.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a881f3220 con 0x7f2a8815d180 2023-11-08T19:27:31.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.917+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a881f3810 con 0x7f2a8815d180 2023-11-08T19:27:31.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.919+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a88074430 con 0x7f2a8815d180 2023-11-08T19:27:31.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.919+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a8003e7f0 con 0x7f2a8815d180 2023-11-08T19:27:31.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.920+0000 7f2a857fa640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a6806ae70 0x7f2a6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:31.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.920+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2a800ac190 con 0x7f2a8815d180 2023-11-08T19:27:31.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.921+0000 7f2a87fff640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a6806ae70 0x7f2a6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:31.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.921+0000 7f2a87fff640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a6806ae70 0x7f2a6806d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a8815c6f0 tx=0x7f2a78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:31.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:31.925+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a80077380 con 0x7f2a8815d180 2023-11-08T19:27:32.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.117+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a8806cd50 con 0x7f2a8815d180 2023-11-08T19:27:32.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.119+0000 7f2a857fa640 1 -- 172.21.15.119:0/1061836253 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+14815 (secure 0 0 0) 0x7f2a8007a9a0 con 0x7f2a8815d180 2023-11-08T19:27:32.120 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:32.121 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":61,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:31.187801+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:32.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a6806ae70 msgr2=0x7f2a6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a6806ae70 0x7f2a6806d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2a8815c6f0 tx=0x7f2a78008040 comp rx=0 tx=0).stop 2023-11-08T19:27:32.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 msgr2=0x7f2a8815fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8815fdd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2a80036040 tx=0x7f2a80004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:32.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 shutdown_connections 2023-11-08T19:27:32.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a6806ae70 0x7f2a6806d330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a8815d180 0x7f2a8815fdd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a880cc810 0x7f2a881f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 --2- 172.21.15.119:0/1061836253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a8806a390 0x7f2a881f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 >> 172.21.15.119:0/1061836253 conn(0x7f2a880cb610 msgr2=0x7f2a880cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 shutdown_connections 2023-11-08T19:27:32.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.121+0000 7f2a8e639640 1 -- 172.21.15.119:0/1061836253 wait complete. 2023-11-08T19:27:32.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:32.142 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:32.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.349+0000 7fd510bee640 1 Processor -- start 2023-11-08T19:27:32.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.349+0000 7fd510bee640 1 -- start start 2023-11-08T19:27:32.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.351+0000 7fd510bee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd50c15bff0 0x7fd50c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.351+0000 7fd510bee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c15ea60 con 0x7fd50c144580 2023-11-08T19:27:32.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.351+0000 7fd510bee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50c15c460 con 0x7fd50c144960 2023-11-08T19:27:32.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.351+0000 7fd510bee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd50c0c9150 con 0x7fd50c15bff0 2023-11-08T19:27:32.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.351+0000 7fd50a59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd50c144580 0x7fd50c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45502/0 (socket says 172.21.15.119:45502) 2023-11-08T19:27:32.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.351+0000 7fd50a59c640 1 -- 172.21.15.119:0/1859666154 learned_addr learned my addr 172.21.15.119:0/1859666154 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:32.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd50ad9d640 1 --2- 172.21.15.119:0/1859666154 >> v2:172.21.15.188:3300/0 conn(0x7fd50c15bff0 0x7fd50c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409842206 0 0) 0x7fd50c15ea60 con 0x7fd50c144580 2023-11-08T19:27:32.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4f8003520 con 0x7fd50c144580 2023-11-08T19:27:32.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3813291284 0 0) 0x7fd4f8003520 con 0x7fd50c144580 2023-11-08T19:27:32.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 >> v2:172.21.15.188:3300/0 conn(0x7fd50c15bff0 msgr2=0x7fd50c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 --2- 172.21.15.119:0/1859666154 >> v2:172.21.15.188:3300/0 conn(0x7fd50c15bff0 0x7fd50c0c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 >> v1:172.21.15.188:6789/0 conn(0x7fd50c144960 legacy=0x7fd50c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c15f8e0 con 0x7fd50c144580 2023-11-08T19:27:32.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd4fc001670 con 0x7fd50c144580 2023-11-08T19:27:32.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 >> v1:172.21.15.119:6789/0 conn(0x7fd50c144580 legacy=0x7fd50c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.352+0000 7fd509d9b640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd509d9b640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f80047c0 0x7fd4f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd509d9b640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f80051d0 0x7fd4f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0c9150 con 0x7fd4f80039c0 2023-11-08T19:27:32.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800c120 con 0x7fd4f80051d0 2023-11-08T19:27:32.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f800c2e0 con 0x7fd4f80047c0 2023-11-08T19:27:32.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50a59c640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd501d9b640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f80047c0 0x7fd4f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50a59c640 1 -- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f80047c0 msgr2=0x7fd4f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50a59c640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd4f80047c0 0x7fd4f8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50a59c640 1 -- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f80051d0 msgr2=0x7fd4f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50a59c640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd4f80051d0 0x7fd4f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50a59c640 1 -- 172.21.15.119:0/1859666154 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8003520 con 0x7fd4f80039c0 2023-11-08T19:27:32.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.353+0000 7fd50ad9d640 1 -- 172.21.15.119:0/1859666154 reap_dead start 2023-11-08T19:27:32.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.354+0000 7fd50a59c640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc001eb0 tx=0x7fd4f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0dd89a83d7e6af1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc005ad0 con 0x7fd4f80039c0 2023-11-08T19:27:32.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc005c30 con 0x7fd4f80039c0 2023-11-08T19:27:32.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd509d9b640 1 -- 172.21.15.119:0/1859666154 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc02ca00 con 0x7fd4f80039c0 2023-11-08T19:27:32.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 -- 172.21.15.119:0/1859666154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 msgr2=0x7fd4f8003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc001eb0 tx=0x7fd4f80057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 -- 172.21.15.119:0/1859666154 shutdown_connections 2023-11-08T19:27:32.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 --2- 172.21.15.119:0/1859666154 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd4f8003de0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 -- 172.21.15.119:0/1859666154 >> 172.21.15.119:0/1859666154 conn(0x7fd50c0cb610 msgr2=0x7fd50c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 -- 172.21.15.119:0/1859666154 shutdown_connections 2023-11-08T19:27:32.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.355+0000 7fd510bee640 1 -- 172.21.15.119:0/1859666154 wait complete. 2023-11-08T19:27:32.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.357+0000 7fd510bee640 1 Processor -- start 2023-11-08T19:27:32.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.357+0000 7fd510bee640 1 -- start start 2023-11-08T19:27:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd510bee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd50c1f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd510bee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50c1f6b00 0x7fd50c1efea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd510bee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50c06f930 0x7fd50c06fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd510bee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0cdbf0 con 0x7fd4f80039c0 2023-11-08T19:27:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd510bee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c0702f0 con 0x7fd50c1f6b00 2023-11-08T19:27:32.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd510bee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd50c1f3110 con 0x7fd50c06f930 2023-11-08T19:27:32.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.358+0000 7fd50a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd50c1f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd50c1f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34052/0 (socket says 172.21.15.119:34052) 2023-11-08T19:27:32.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 -- 172.21.15.119:0/2435865896 learned_addr learned my addr 172.21.15.119:0/2435865896 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:32.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd501d9b640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50c1f6b00 0x7fd50c1efea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 -- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50c06f930 msgr2=0x7fd50c06fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50ad9d640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50c06f930 0x7fd50c06fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50c06f930 0x7fd50c06fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 -- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50c1f6b00 msgr2=0x7fd50c1efea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50c1f6b00 0x7fd50c1efea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 -- 172.21.15.119:0/2435865896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd50c1f33f0 con 0x7fd4f80039c0 2023-11-08T19:27:32.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd501d9b640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50c1f6b00 0x7fd50c1efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:32.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.359+0000 7fd50a59c640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd50c1f2aa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc000c00 tx=0x7fd4fc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.360+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc03c040 con 0x7fd4f80039c0 2023-11-08T19:27:32.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.360+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4fc02cbf0 con 0x7fd4f80039c0 2023-11-08T19:27:32.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.360+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4fc007070 con 0x7fd4f80039c0 2023-11-08T19:27:32.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.360+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50c1f36d0 con 0x7fd4f80039c0 2023-11-08T19:27:32.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.360+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50c1f3cc0 con 0x7fd4f80039c0 2023-11-08T19:27:32.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.361+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd50c1f3250 con 0x7fd4f80039c0 2023-11-08T19:27:32.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.362+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4fc032070 con 0x7fd4f80039c0 2023-11-08T19:27:32.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.363+0000 7fd5037fe640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4ec06aea0 0x7fd4ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.363+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd4fc0aa910 con 0x7fd4f80039c0 2023-11-08T19:27:32.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.363+0000 7fd501d9b640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4ec06aea0 0x7fd4ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.364+0000 7fd501d9b640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4ec06aea0 0x7fd4ec06d360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd50c070ad0 tx=0x7fd4f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.367+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd50c1f3250 con 0x7fd4f80039c0 2023-11-08T19:27:32.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.553+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd50c1f3250 con 0x7fd4f80039c0 2023-11-08T19:27:32.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.554+0000 7fd5037fe640 1 -- 172.21.15.119:0/2435865896 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd50c1f3250 con 0x7fd4f80039c0 2023-11-08T19:27:32.555 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:32.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4ec06aea0 msgr2=0x7fd4ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4ec06aea0 0x7fd4ec06d360 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd50c070ad0 tx=0x7fd4f4009040 comp rx=0 tx=0).stop 2023-11-08T19:27:32.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 msgr2=0x7fd50c1f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd50c1f2aa0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc000c00 tx=0x7fd4fc02e070 comp rx=0 tx=0).stop 2023-11-08T19:27:32.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 shutdown_connections 2023-11-08T19:27:32.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4ec06aea0 0x7fd4ec06d360 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50c06f930 0x7fd50c06fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd50c1f6b00 0x7fd50c1efea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.556+0000 7fd510bee640 1 --2- 172.21.15.119:0/2435865896 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd4f80039c0 0x7fd50c1f2aa0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.557+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 >> 172.21.15.119:0/2435865896 conn(0x7fd50c0cb610 msgr2=0x7fd50c15cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.557+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 shutdown_connections 2023-11-08T19:27:32.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.557+0000 7fd510bee640 1 -- 172.21.15.119:0/2435865896 wait complete. 2023-11-08T19:27:32.576 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.789+0000 7fd9b9656640 1 Processor -- start 2023-11-08T19:27:32.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.790+0000 7fd9b9656640 1 -- start start 2023-11-08T19:27:32.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b9656640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9b415d4a0 0x7fd9b40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b9656640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b415df50 con 0x7fd9b40cfa00 2023-11-08T19:27:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b9656640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b40d1e60 con 0x7fd9b4161870 2023-11-08T19:27:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b9656640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b40d2830 con 0x7fd9b415d4a0 2023-11-08T19:27:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b2ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9b415d4a0 0x7fd9b40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b2ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd9b415d4a0 0x7fd9b40d22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34064/0 (socket says 172.21.15.119:34064) 2023-11-08T19:27:32.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.791+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/2152612321 learned_addr learned my addr 172.21.15.119:0/2152612321 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:32.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.792+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/2152612321 >> v1:172.21.15.188:6789/0 conn(0x7fd9b4161870 legacy=0x7fd9b40cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:32.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.792+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/2152612321 >> v1:172.21.15.119:6789/0 conn(0x7fd9b40cfa00 legacy=0x7fd9b4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.792+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/2152612321 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4161d40 con 0x7fd9b415d4a0 2023-11-08T19:27:32.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.793+0000 7fd9b2ffd640 1 --2- 172.21.15.119:0/2152612321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b415d4a0 0x7fd9b40d22f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009870 tx=0x7fd9a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a897f5c67eb72b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.793+0000 7fd9b27fc640 1 -- 172.21.15.119:0/2152612321 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4031870 con 0x7fd9b415d4a0 2023-11-08T19:27:32.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.793+0000 7fd9b27fc640 1 -- 172.21.15.119:0/2152612321 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40319d0 con 0x7fd9b415d4a0 2023-11-08T19:27:32.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.793+0000 7fd9b27fc640 1 -- 172.21.15.119:0/2152612321 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4031d60 con 0x7fd9b415d4a0 2023-11-08T19:27:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 -- 172.21.15.119:0/2152612321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b415d4a0 msgr2=0x7fd9b40d22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 --2- 172.21.15.119:0/2152612321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b415d4a0 0x7fd9b40d22f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009870 tx=0x7fd9a402b910 comp rx=0 tx=0).stop 2023-11-08T19:27:32.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 -- 172.21.15.119:0/2152612321 shutdown_connections 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 --2- 172.21.15.119:0/2152612321 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b415d4a0 0x7fd9b40d22f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 -- 172.21.15.119:0/2152612321 >> 172.21.15.119:0/2152612321 conn(0x7fd9b406a9f0 msgr2=0x7fd9b406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 -- 172.21.15.119:0/2152612321 shutdown_connections 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.794+0000 7fd9b9656640 1 -- 172.21.15.119:0/2152612321 wait complete. 2023-11-08T19:27:32.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.796+0000 7fd9b9656640 1 Processor -- start 2023-11-08T19:27:32.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.796+0000 7fd9b9656640 1 -- start start 2023-11-08T19:27:32.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b9656640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 0x7fd9b40c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b9656640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b415d4a0 0x7fd9b40c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b9656640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b4161870 0x7fd9b41f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b9656640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b414aac0 con 0x7fd9b40cfa00 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b9656640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40c7dd0 con 0x7fd9b415d4a0 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b9656640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40c80b0 con 0x7fd9b4161870 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 0x7fd9b40c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 0x7fd9b40c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34074/0 (socket says 172.21.15.119:34074) 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.797+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/3944006967 learned_addr learned my addr 172.21.15.119:0/3944006967 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/3944006967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b4161870 msgr2=0x7fd9b41f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b2ffd640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b4161870 0x7fd9b41f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/3944006967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b415d4a0 msgr2=0x7fd9b40c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b2ffd640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b415d4a0 0x7fd9b40c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b2ffd640 1 -- 172.21.15.119:0/3944006967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a40094d0 con 0x7fd9b40cfa00 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b2ffd640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 0x7fd9b40c77f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009870 tx=0x7fd9a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4031820 con 0x7fd9b40cfa00 2023-11-08T19:27:32.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a40443f0 con 0x7fd9b40cfa00 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a4044780 con 0x7fd9b40cfa00 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b40c8390 con 0x7fd9b40cfa00 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.798+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b414aee0 con 0x7fd9b40cfa00 2023-11-08T19:27:32.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.799+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b40cf4b0 con 0x7fd9b40cfa00 2023-11-08T19:27:32.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.800+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd9a404d450 con 0x7fd9b40cfa00 2023-11-08T19:27:32.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.801+0000 7fd9ab7fe640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd99806aea0 0x7fd99806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.801+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fd9a402d040 con 0x7fd9b40cfa00 2023-11-08T19:27:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.801+0000 7fd9abfff640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd99806aea0 0x7fd99806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.802+0000 7fd9abfff640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd99806aea0 0x7fd99806d360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd99c000f60 tx=0x7fd99c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:32.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.805+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9a4076e40 con 0x7fd9b40cfa00 2023-11-08T19:27:32.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.994+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fd9b40d10f0 con 0x7fd9b40cfa00 2023-11-08T19:27:32.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.995+0000 7fd9ab7fe640 1 -- 172.21.15.119:0/3944006967 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v61) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd9a4035020 con 0x7fd9b40cfa00 2023-11-08T19:27:32.997 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:32.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd99806aea0 msgr2=0x7fd99806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd99806aea0 0x7fd99806d360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd99c000f60 tx=0x7fd99c0079b0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 msgr2=0x7fd9b40c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 0x7fd9b40c77f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4009870 tx=0x7fd9a40046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 shutdown_connections 2023-11-08T19:27:33.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd99806aea0 0x7fd99806d360 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd9b4161870 0x7fd9b41f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd9b415d4a0 0x7fd9b40c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 --2- 172.21.15.119:0/3944006967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd9b40cfa00 0x7fd9b40c77f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 >> 172.21.15.119:0/3944006967 conn(0x7fd9b406a9f0 msgr2=0x7fd9b415b050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:33.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 shutdown_connections 2023-11-08T19:27:33.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:32.998+0000 7fd9b9656640 1 -- 172.21.15.119:0/3944006967 wait complete. 2023-11-08T19:27:33.017 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:27:33.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.228+0000 7fe2702e5640 1 Processor -- start 2023-11-08T19:27:33.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.228+0000 7fe2702e5640 1 -- start start 2023-11-08T19:27:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe2702e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2680c9720 0x7fe2680c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe2702e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 0x7fe2681634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe2702e5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2680c9360 con 0x7fe268159d90 2023-11-08T19:27:33.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe2702e5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26806d3c0 con 0x7fe2680c9720 2023-11-08T19:27:33.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe2702e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680ca490 con 0x7fe2681610c0 2023-11-08T19:27:33.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe26e081640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe268159d90 0x7fe26815a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45526/0 (socket says 172.21.15.119:45526) 2023-11-08T19:27:33.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.230+0000 7fe26e081640 1 -- 172.21.15.119:0/1947196975 learned_addr learned my addr 172.21.15.119:0/1947196975 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:33.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2643105663 0 0) 0x7fe2680c9360 con 0x7fe268159d90 2023-11-08T19:27:33.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe254003440 con 0x7fe268159d90 2023-11-08T19:27:33.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26e882640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 0x7fe2681634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d880640 1 --2- 172.21.15.119:0/1947196975 >> v2:172.21.15.188:3300/0 conn(0x7fe2680c9720 0x7fe2680c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 426240593 0 0) 0x7fe254003440 con 0x7fe268159d90 2023-11-08T19:27:33.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 msgr2=0x7fe2681634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:33.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 0x7fe2681634b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 >> v2:172.21.15.188:3300/0 conn(0x7fe2680c9720 msgr2=0x7fe2680c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:33.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 --2- 172.21.15.119:0/1947196975 >> v2:172.21.15.188:3300/0 conn(0x7fe2680c9720 0x7fe2680c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680ca770 con 0x7fe268159d90 2023-11-08T19:27:33.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe264001670 con 0x7fe268159d90 2023-11-08T19:27:33.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 >> v1:172.21.15.119:6789/0 conn(0x7fe268159d90 legacy=0x7fe26815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 0x7fe254003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254004090 0x7fe25400a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25400af40 0x7fe25400d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26806d3c0 con 0x7fe254004090 2023-11-08T19:27:33.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680ca490 con 0x7fe25400af40 2023-11-08T19:27:33.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25400dd20 con 0x7fe2681610c0 2023-11-08T19:27:33.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.231+0000 7fe26d880640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254004090 0x7fe25400a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.232+0000 7fe26d880640 1 -- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 msgr2=0x7fe254003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:33.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.232+0000 7fe26d880640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2681610c0 0x7fe254003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.232+0000 7fe26d880640 1 -- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25400af40 msgr2=0x7fe25400d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.232+0000 7fe26d880640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe25400af40 0x7fe25400d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.232+0000 7fe26d880640 1 -- 172.21.15.119:0/1947196975 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe254003440 con 0x7fe254004090 2023-11-08T19:27:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.233+0000 7fe26d880640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254004090 0x7fe25400a860 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe25800ba30 tx=0x7fe2540036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddbc15d15791dcf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.233+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258032680 con 0x7fe254004090 2023-11-08T19:27:33.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2580327e0 con 0x7fe254004090 2023-11-08T19:27:33.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe26d07f640 1 -- 172.21.15.119:0/1947196975 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258032b70 con 0x7fe254004090 2023-11-08T19:27:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe2702e5640 1 -- 172.21.15.119:0/1947196975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254004090 msgr2=0x7fe25400a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe2702e5640 1 --2- 172.21.15.119:0/1947196975 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe254004090 0x7fe25400a860 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe25800ba30 tx=0x7fe2540036f0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe26e882640 1 -- 172.21.15.119:0/1947196975 reap_dead start 2023-11-08T19:27:33.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe2702e5640 1 -- 172.21.15.119:0/1947196975 shutdown_connections 2023-11-08T19:27:33.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe2702e5640 1 -- 172.21.15.119:0/1947196975 >> 172.21.15.119:0/1947196975 conn(0x7fe2680cb610 msgr2=0x7fe2680cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:33.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe2702e5640 1 -- 172.21.15.119:0/1947196975 shutdown_connections 2023-11-08T19:27:33.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.234+0000 7fe2702e5640 1 -- 172.21.15.119:0/1947196975 wait complete. 2023-11-08T19:27:33.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.236+0000 7fe2702e5640 1 Processor -- start 2023-11-08T19:27:33.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.236+0000 7fe2702e5640 1 -- start start 2023-11-08T19:27:33.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe2702e5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe254004090 0x7fe2681548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe2702e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 0x7fe26814a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe2702e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe268158950 0x7fe268158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe2702e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe268159340 con 0x7fe2680c4500 2023-11-08T19:27:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe2702e5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680c49c0 con 0x7fe254004090 2023-11-08T19:27:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe2702e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26815acd0 con 0x7fe268158950 2023-11-08T19:27:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe26d880640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 0x7fe26814a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe26d880640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 0x7fe26814a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34098/0 (socket says 172.21.15.119:34098) 2023-11-08T19:27:33.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.237+0000 7fe26d880640 1 -- 172.21.15.119:0/2492924466 learned_addr learned my addr 172.21.15.119:0/2492924466 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26e081640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe254004090 0x7fe2681548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26e882640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe268158950 0x7fe268158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26d880640 1 -- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe268158950 msgr2=0x7fe268158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26d880640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe268158950 0x7fe268158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26d880640 1 -- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe254004090 msgr2=0x7fe2681548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26d880640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe254004090 0x7fe2681548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26d880640 1 -- 172.21.15.119:0/2492924466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26815afb0 con 0x7fe2680c4500 2023-11-08T19:27:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26e882640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe268158950 0x7fe268158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe26d880640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 0x7fe26814a670 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe25802bf00 tx=0x7fe258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258004220 con 0x7fe2680c4500 2023-11-08T19:27:33.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2580043a0 con 0x7fe2680c4500 2023-11-08T19:27:33.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.238+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26815b290 con 0x7fe2680c4500 2023-11-08T19:27:33.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.239+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258030aa0 con 0x7fe2680c4500 2023-11-08T19:27:33.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.239+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26814abb0 con 0x7fe2680c4500 2023-11-08T19:27:33.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.240+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2680c8f10 con 0x7fe2680c4500 2023-11-08T19:27:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.241+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe258030c00 con 0x7fe2680c4500 2023-11-08T19:27:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.242+0000 7fe25effd640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe24c06ae70 0x7fe24c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.242+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe2580ac110 con 0x7fe2680c4500 2023-11-08T19:27:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.242+0000 7fe26e081640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe24c06ae70 0x7fe24c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.243+0000 7fe26e081640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe24c06ae70 0x7fe24c06d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe264000ff0 tx=0x7fe2640016e0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.246+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe2580772f0 con 0x7fe2680c4500 2023-11-08T19:27:33.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.435+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe2680c8f10 con 0x7fe2680c4500 2023-11-08T19:27:33.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.436+0000 7fe25effd640 1 -- 172.21.15.119:0/2492924466 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+137 (secure 0 0 0) 0x7fe258030eb0 con 0x7fe2680c4500 2023-11-08T19:27:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:33.438 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-11-08T19:27:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:33.438 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-11-08T19:27:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-11-08T19:27:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:27:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:27:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 7, 2023-11-08T19:27:33.439 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "snap-test-pool" 2023-11-08T19:27:33.440 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:27:33.440 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-11-08T19:27:33.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe24c06ae70 msgr2=0x7fe24c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe24c06ae70 0x7fe24c06d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe264000ff0 tx=0x7fe2640016e0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 msgr2=0x7fe26814a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 0x7fe26814a670 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe25802bf00 tx=0x7fe258004730 comp rx=0 tx=0).stop 2023-11-08T19:27:33.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 shutdown_connections 2023-11-08T19:27:33.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe24c06ae70 0x7fe24c06d330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe268158950 0x7fe268158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2680c4500 0x7fe26814a670 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 --2- 172.21.15.119:0/2492924466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe254004090 0x7fe2681548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 >> 172.21.15.119:0/2492924466 conn(0x7fe2680cb610 msgr2=0x7fe26806a330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:33.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 shutdown_connections 2023-11-08T19:27:33.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.439+0000 7fe2702e5640 1 -- 172.21.15.119:0/2492924466 wait complete. 2023-11-08T19:27:33.456 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:27:33.457 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:27:33.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.669+0000 7f2dc3c4b640 1 Processor -- start 2023-11-08T19:27:33.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.669+0000 7f2dc3c4b640 1 -- start start 2023-11-08T19:27:33.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc3c4b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2dbc143600 0x7f2dbc0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc3c4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc15d9f0 0x7f2dbc15de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc3c4b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc15e440 con 0x7f2dbc06cb40 2023-11-08T19:27:33.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc3c4b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c4f50 con 0x7f2dbc143600 2023-11-08T19:27:33.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc3c4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c4960 con 0x7f2dbc15d9f0 2023-11-08T19:27:33.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc19e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2dbc143600 0x7f2dbc0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc21e8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2dbc06cb40 0x7f2dbc0c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45528/0 (socket says 172.21.15.119:45528) 2023-11-08T19:27:33.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.671+0000 7f2dc21e8640 1 -- 172.21.15.119:0/3456165622 learned_addr learned my addr 172.21.15.119:0/3456165622 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:33.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.672+0000 7f2dc11e6640 1 --2- 172.21.15.119:0/3456165622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc15d9f0 0x7f2dbc15de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.672+0000 7f2dc19e7640 1 -- 172.21.15.119:0/3456165622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc15d9f0 msgr2=0x7f2dbc15de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:33.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.672+0000 7f2dc19e7640 1 --2- 172.21.15.119:0/3456165622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc15d9f0 0x7f2dbc15de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.672+0000 7f2dc19e7640 1 -- 172.21.15.119:0/3456165622 >> v1:172.21.15.119:6789/0 conn(0x7f2dbc06cb40 legacy=0x7f2dbc0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.672+0000 7f2dc19e7640 1 -- 172.21.15.119:0/3456165622 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc15e720 con 0x7f2dbc143600 2023-11-08T19:27:33.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.673+0000 7f2dc19e7640 1 --2- 172.21.15.119:0/3456165622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0c5370 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac02be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddff76d1e60c40 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.673+0000 7f2dc09e5640 1 -- 172.21.15.119:0/3456165622 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac02cbc0 con 0x7f2dbc143600 2023-11-08T19:27:33.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.673+0000 7f2dc09e5640 1 -- 172.21.15.119:0/3456165622 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dac02cd20 con 0x7f2dbc143600 2023-11-08T19:27:33.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.673+0000 7f2dc09e5640 1 -- 172.21.15.119:0/3456165622 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dac034660 con 0x7f2dbc143600 2023-11-08T19:27:33.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/3456165622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 msgr2=0x7f2dbc0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/3456165622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0c5370 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009d90 tx=0x7f2dac02be10 comp rx=0 tx=0).stop 2023-11-08T19:27:33.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/3456165622 shutdown_connections 2023-11-08T19:27:33.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/3456165622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc15d9f0 0x7f2dbc15de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/3456165622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0c5370 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/3456165622 >> 172.21.15.119:0/3456165622 conn(0x7f2dbc0cb610 msgr2=0x7f2dbc0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:33.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/3456165622 shutdown_connections 2023-11-08T19:27:33.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.674+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/3456165622 wait complete. 2023-11-08T19:27:33.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.676+0000 7f2dc3c4b640 1 Processor -- start 2023-11-08T19:27:33.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.676+0000 7f2dc3c4b640 1 -- start start 2023-11-08T19:27:33.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc3c4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc06cb40 0x7f2dbc15c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc3c4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0aca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc3c4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dbc15d9f0 0x7f2dbc1319c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc3c4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc12e960 con 0x7f2dbc143600 2023-11-08T19:27:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc3c4b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc12ec40 con 0x7f2dbc15d9f0 2023-11-08T19:27:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc3c4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc12ef20 con 0x7f2dbc06cb40 2023-11-08T19:27:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc11e6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0aca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc11e6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0aca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34116/0 (socket says 172.21.15.119:34116) 2023-11-08T19:27:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.677+0000 7f2dc11e6640 1 -- 172.21.15.119:0/78657148 learned_addr learned my addr 172.21.15.119:0/78657148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:33.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc19e7640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc06cb40 0x7f2dbc15c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc11e6640 1 -- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc06cb40 msgr2=0x7f2dbc15c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc21e8640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dbc15d9f0 0x7f2dbc1319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc11e6640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc06cb40 0x7f2dbc15c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc11e6640 1 -- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dbc15d9f0 msgr2=0x7f2dbc1319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc11e6640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dbc15d9f0 0x7f2dbc1319c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc11e6640 1 -- 172.21.15.119:0/78657148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dac009a70 con 0x7f2dbc143600 2023-11-08T19:27:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc11e6640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0aca50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2db000d5d0 tx=0x7f2db000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0004240 con 0x7f2dbc143600 2023-11-08T19:27:33.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db00043a0 con 0x7f2dbc143600 2023-11-08T19:27:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db0003830 con 0x7f2dbc143600 2023-11-08T19:27:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.678+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc12f200 con 0x7f2dbc143600 2023-11-08T19:27:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.679+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc15ca90 con 0x7f2dbc143600 2023-11-08T19:27:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.680+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc074430 con 0x7f2dbc143600 2023-11-08T19:27:33.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.681+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2db0003990 con 0x7f2dbc143600 2023-11-08T19:27:33.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.681+0000 7f2dba7fc640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2da806ae70 0x7f2da806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:33.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.681+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f2db00897a0 con 0x7f2dbc143600 2023-11-08T19:27:33.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.682+0000 7f2dc19e7640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2da806ae70 0x7f2da806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:33.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.682+0000 7f2dc19e7640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2da806ae70 0x7f2da806d330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2dac036000 tx=0x7f2dac030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:33.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.686+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2db0054900 con 0x7f2dbc143600 2023-11-08T19:27:33.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:33.877+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2dbc0c4230 con 0x7f2dbc143600 2023-11-08T19:27:34.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.644+0000 7f2dba7fc640 1 -- 172.21.15.119:0/78657148 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2db0057f20 con 0x7f2dbc143600 2023-11-08T19:27:34.647 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-11-08T19:27:34.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2da806ae70 msgr2=0x7f2da806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2da806ae70 0x7f2da806d330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2dac036000 tx=0x7f2dac030000 comp rx=0 tx=0).stop 2023-11-08T19:27:34.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 msgr2=0x7f2dbc0aca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0aca50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2db000d5d0 tx=0x7f2db000da90 comp rx=0 tx=0).stop 2023-11-08T19:27:34.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 shutdown_connections 2023-11-08T19:27:34.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2da806ae70 0x7f2da806d330 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2dbc15d9f0 0x7f2dbc1319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2dbc143600 0x7f2dbc0aca50 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 --2- 172.21.15.119:0/78657148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2dbc06cb40 0x7f2dbc15c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.648+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 >> 172.21.15.119:0/78657148 conn(0x7f2dbc0cb610 msgr2=0x7f2dbc0ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:34.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.649+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 shutdown_connections 2023-11-08T19:27:34.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.649+0000 7f2dc3c4b640 1 -- 172.21.15.119:0/78657148 wait complete. 2023-11-08T19:27:34.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:27:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.877+0000 7f3aa44d4640 1 Processor -- start 2023-11-08T19:27:34.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.878+0000 7f3aa44d4640 1 -- start start 2023-11-08T19:27:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.879+0000 7f3aa44d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c06f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.879+0000 7f3aa44d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3a9c0cd020 0x7f3a9c0ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.879+0000 7f3aa44d4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a9c0cb890 0x7f3a9c0ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa44d4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c06feb0 con 0x7f3a9c0cb890 2023-11-08T19:27:34.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa44d4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0c9400 con 0x7f3a9c0cd020 2023-11-08T19:27:34.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa44d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0cd8c0 con 0x7f3a9c0cb4b0 2023-11-08T19:27:34.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a9c0cb890 0x7f3a9c0ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a9c0cb890 0x7f3a9c0ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34128/0 (socket says 172.21.15.119:34128) 2023-11-08T19:27:34.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 -- 172.21.15.119:0/2674820139 learned_addr learned my addr 172.21.15.119:0/2674820139 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:34.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 -- 172.21.15.119:0/2674820139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 msgr2=0x7f3a9c06f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:34.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2270640 1 --2- 172.21.15.119:0/2674820139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c06f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 --2- 172.21.15.119:0/2674820139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c06f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 -- 172.21.15.119:0/2674820139 >> v2:172.21.15.188:3300/0 conn(0x7f3a9c0cd020 msgr2=0x7f3a9c0ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 --2- 172.21.15.119:0/2674820139 >> v2:172.21.15.188:3300/0 conn(0x7f3a9c0cd020 0x7f3a9c0ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.880+0000 7f3aa2a71640 1 -- 172.21.15.119:0/2674820139 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0ad810 con 0x7f3a9c0cb890 2023-11-08T19:27:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.882+0000 7f3aa2a71640 1 --2- 172.21.15.119:0/2674820139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cb890 0x7f3a9c0ca160 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0cc110 tx=0x7f3a9402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e01a4184373c214 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:34.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.882+0000 7f3aa126e640 1 -- 172.21.15.119:0/2674820139 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9402c9f0 con 0x7f3a9c0cb890 2023-11-08T19:27:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.882+0000 7f3aa126e640 1 -- 172.21.15.119:0/2674820139 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9402cb50 con 0x7f3a9c0cb890 2023-11-08T19:27:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.882+0000 7f3aa126e640 1 -- 172.21.15.119:0/2674820139 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940343f0 con 0x7f3a9c0cb890 2023-11-08T19:27:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 -- 172.21.15.119:0/2674820139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cb890 msgr2=0x7f3a9c0ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/2674820139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cb890 0x7f3a9c0ca160 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0cc110 tx=0x7f3a9402be50 comp rx=0 tx=0).stop 2023-11-08T19:27:34.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 -- 172.21.15.119:0/2674820139 shutdown_connections 2023-11-08T19:27:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/2674820139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cb890 0x7f3a9c0ca160 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/2674820139 >> v2:172.21.15.188:3300/0 conn(0x7f3a9c0cd020 0x7f3a9c0ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/2674820139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c06f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 -- 172.21.15.119:0/2674820139 >> 172.21.15.119:0/2674820139 conn(0x7f3a9c0b0950 msgr2=0x7f3a9c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:34.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 -- 172.21.15.119:0/2674820139 shutdown_connections 2023-11-08T19:27:34.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.883+0000 7f3aa44d4640 1 -- 172.21.15.119:0/2674820139 wait complete. 2023-11-08T19:27:34.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.885+0000 7f3aa44d4640 1 Processor -- start 2023-11-08T19:27:34.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.885+0000 7f3aa44d4640 1 -- start start 2023-11-08T19:27:34.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa44d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c070e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa44d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c0cb890 0x7f3a9c1f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa44d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 0x7f3a9c1f1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa44d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15acf0 con 0x7f3a9c0cd020 2023-11-08T19:27:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa44d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c1f2330 con 0x7f3a9c0cb890 2023-11-08T19:27:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa44d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c15af70 con 0x7f3a9c0cb4b0 2023-11-08T19:27:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 0x7f3a9c1f1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 0x7f3a9c1f1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34136/0 (socket says 172.21.15.119:34136) 2023-11-08T19:27:34.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 -- 172.21.15.119:0/926956851 learned_addr learned my addr 172.21.15.119:0/926956851 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:34.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 -- 172.21.15.119:0/926956851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 msgr2=0x7f3a9c070e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c070e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 -- 172.21.15.119:0/926956851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c0cb890 msgr2=0x7f3a9c1f18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:34.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c0cb890 0x7f3a9c1f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:34.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.886+0000 7f3aa2a71640 1 -- 172.21.15.119:0/926956851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94009b30 con 0x7f3a9c0cd020 2023-11-08T19:27:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.887+0000 7f3aa2a71640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 0x7f3a9c1f1df0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3a94036040 tx=0x7f3a94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.887+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9403e450 con 0x7f3a9c0cd020 2023-11-08T19:27:34.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.887+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9403e5d0 con 0x7f3a9c0cd020 2023-11-08T19:27:34.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.887+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c15b250 con 0x7f3a9c0cd020 2023-11-08T19:27:34.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.887+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9403e980 con 0x7f3a9c0cd020 2023-11-08T19:27:34.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.887+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c15b810 con 0x7f3a9c0cd020 2023-11-08T19:27:34.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.888+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c04ece0 con 0x7f3a9c0cd020 2023-11-08T19:27:34.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.889+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a9403eae0 con 0x7f3a9c0cd020 2023-11-08T19:27:34.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.890+0000 7f3a92ffd640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a8006ae70 0x7f3a8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:34.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.890+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f3a940ac6b0 con 0x7f3a9c0cd020 2023-11-08T19:27:34.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.890+0000 7f3aa2270640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a8006ae70 0x7f3a8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:34.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.891+0000 7f3aa2270640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a8006ae70 0x7f3a8006d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:34.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:34.895+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a94077600 con 0x7f3a9c0cd020 2023-11-08T19:27:35.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.083+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3a9c15ba00 con 0x7f3a9c0cd020 2023-11-08T19:27:35.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.660+0000 7f3a92ffd640 1 -- 172.21.15.119:0/926956851 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3a9407ac20 con 0x7f3a9c0cd020 2023-11-08T19:27:35.663 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-11-08T19:27:35.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.665+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a8006ae70 msgr2=0x7f3a8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.665+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a8006ae70 0x7f3a8006d330 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98008040 comp rx=0 tx=0).stop 2023-11-08T19:27:35.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.665+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 msgr2=0x7f3a9c1f1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.665+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 0x7f3a9c1f1df0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3a94036040 tx=0x7f3a94004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:35.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 shutdown_connections 2023-11-08T19:27:35.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a8006ae70 0x7f3a8006d330 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a9c0cd020 0x7f3a9c1f1df0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a9c0cb890 0x7f3a9c1f18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 --2- 172.21.15.119:0/926956851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a9c0cb4b0 0x7f3a9c070e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 >> 172.21.15.119:0/926956851 conn(0x7f3a9c0b0950 msgr2=0x7f3a9c0ceb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:35.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 shutdown_connections 2023-11-08T19:27:35.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.666+0000 7f3aa44d4640 1 -- 172.21.15.119:0/926956851 wait complete. 2023-11-08T19:27:35.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:27:35.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.884+0000 7fb0543f5640 1 Processor -- start 2023-11-08T19:27:35.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.885+0000 7fb0543f5640 1 -- start start 2023-11-08T19:27:35.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.886+0000 7fb0543f5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb04c15bff0 0x7fb04c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.886+0000 7fb0543f5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c15ea60 con 0x7fb04c144960 2023-11-08T19:27:35.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.886+0000 7fb0543f5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb04c15c460 con 0x7fb04c144580 2023-11-08T19:27:35.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.886+0000 7fb0543f5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c0c9150 con 0x7fb04c15bff0 2023-11-08T19:27:35.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.887+0000 7fb051990640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb04c144960 0x7fb04c15f1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45542/0 (socket says 172.21.15.119:45542) 2023-11-08T19:27:35.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.887+0000 7fb051990640 1 -- 172.21.15.119:0/3588711076 learned_addr learned my addr 172.21.15.119:0/3588711076 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.887+0000 7fb052992640 1 --2- 172.21.15.119:0/3588711076 >> v2:172.21.15.188:3300/0 conn(0x7fb04c15bff0 0x7fb04c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.887+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3688383586 0 0) 0x7fb04c15ea60 con 0x7fb04c144960 2023-11-08T19:27:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.887+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb038003520 con 0x7fb04c144960 2023-11-08T19:27:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 833013639 0 0) 0x7fb038003520 con 0x7fb04c144960 2023-11-08T19:27:35.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 >> v2:172.21.15.188:3300/0 conn(0x7fb04c15bff0 msgr2=0x7fb04c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 --2- 172.21.15.119:0/3588711076 >> v2:172.21.15.188:3300/0 conn(0x7fb04c15bff0 0x7fb04c0c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 >> v1:172.21.15.188:6789/0 conn(0x7fb04c144580 legacy=0x7fb04c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c15f8e0 con 0x7fb04c144960 2023-11-08T19:27:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fb040001670 con 0x7fb04c144960 2023-11-08T19:27:35.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 >> v1:172.21.15.119:6789/0 conn(0x7fb04c144960 legacy=0x7fb04c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb038003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0380047c0 0x7fb038004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb038005180 0x7fb03800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0c9150 con 0x7fb038003970 2023-11-08T19:27:35.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03800c120 con 0x7fb038005180 2023-11-08T19:27:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03800c2e0 con 0x7fb0380047c0 2023-11-08T19:27:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb052191640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb038003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.888+0000 7fb051990640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0380047c0 0x7fb038004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052992640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb038005180 0x7fb03800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052191640 1 -- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0380047c0 msgr2=0x7fb038004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052191640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb0380047c0 0x7fb038004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052191640 1 -- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb038005180 msgr2=0x7fb03800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052191640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb038005180 0x7fb03800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052191640 1 -- 172.21.15.119:0/3588711076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb038003520 con 0x7fb038003970 2023-11-08T19:27:35.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052992640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb038005180 0x7fb03800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:35.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.889+0000 7fb052992640 1 -- 172.21.15.119:0/3588711076 reap_dead start 2023-11-08T19:27:35.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.890+0000 7fb052191640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb038003d90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb048001750 tx=0x7fb0380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11ff35c8cc7711d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.890+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0480052d0 con 0x7fb038003970 2023-11-08T19:27:35.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.890+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04802e050 con 0x7fb038003970 2023-11-08T19:27:35.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.890+0000 7fb05118f640 1 -- 172.21.15.119:0/3588711076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048003830 con 0x7fb038003970 2023-11-08T19:27:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.890+0000 7fb0543f5640 1 -- 172.21.15.119:0/3588711076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 msgr2=0x7fb038003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.890+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb038003d90 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb048001750 tx=0x7fb0380057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.891+0000 7fb0543f5640 1 -- 172.21.15.119:0/3588711076 shutdown_connections 2023-11-08T19:27:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.891+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3588711076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb038003d90 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.891+0000 7fb0543f5640 1 -- 172.21.15.119:0/3588711076 >> 172.21.15.119:0/3588711076 conn(0x7fb04c0cb610 msgr2=0x7fb04c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:35.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.891+0000 7fb0543f5640 1 -- 172.21.15.119:0/3588711076 shutdown_connections 2023-11-08T19:27:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.891+0000 7fb0543f5640 1 -- 172.21.15.119:0/3588711076 wait complete. 2023-11-08T19:27:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.892+0000 7fb0543f5640 1 Processor -- start 2023-11-08T19:27:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.893+0000 7fb0543f5640 1 -- start start 2023-11-08T19:27:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb0543f5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb04c15ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb0543f5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04c1428b0 0x7fb04c0b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb0543f5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04c0cce90 0x7fb04c0cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb0543f5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0cd850 con 0x7fb038003970 2023-11-08T19:27:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb0543f5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0b4680 con 0x7fb04c1428b0 2023-11-08T19:27:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb0543f5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c0b35e0 con 0x7fb04c0cce90 2023-11-08T19:27:35.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb04c15ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb04c15ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34148/0 (socket says 172.21.15.119:34148) 2023-11-08T19:27:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 -- 172.21.15.119:0/3771366158 learned_addr learned my addr 172.21.15.119:0/3771366158 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 -- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04c0cce90 msgr2=0x7fb04c0cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04c0cce90 0x7fb04c0cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb051990640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04c1428b0 0x7fb04c0b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 -- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04c1428b0 msgr2=0x7fb04c0b2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:35.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04c1428b0 0x7fb04c0b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:35.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.894+0000 7fb052191640 1 -- 172.21.15.119:0/3771366158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c0b38c0 con 0x7fb038003970 2023-11-08T19:27:35.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.895+0000 7fb052191640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb04c15ba10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb04802bc00 tx=0x7fb048001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.895+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048007a30 con 0x7fb038003970 2023-11-08T19:27:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.895+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb048007640 con 0x7fb038003970 2023-11-08T19:27:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.895+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c0b3ba0 con 0x7fb038003970 2023-11-08T19:27:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.895+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb048006be0 con 0x7fb038003970 2023-11-08T19:27:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.895+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c0b4160 con 0x7fb038003970 2023-11-08T19:27:35.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.896+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c0b42f0 con 0x7fb038003970 2023-11-08T19:27:35.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.898+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb048006d40 con 0x7fb038003970 2023-11-08T19:27:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.899+0000 7fb03effd640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb03006ae70 0x7fb03006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.899+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fb0480ab7d0 con 0x7fb038003970 2023-11-08T19:27:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.899+0000 7fb051990640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb03006ae70 0x7fb03006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.900+0000 7fb051990640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb03006ae70 0x7fb03006d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb040001eb0 tx=0x7fb040000f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:35.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:35.902+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb048076840 con 0x7fb038003970 2023-11-08T19:27:36.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.119+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb04c06cb10 con 0x7fb038003970 2023-11-08T19:27:36.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.688+0000 7fb03effd640 1 -- 172.21.15.119:0/3771366158 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v31) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb048079e60 con 0x7fb038003970 2023-11-08T19:27:36.692 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 10 and data pool 11 2023-11-08T19:27:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.699+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb03006ae70 msgr2=0x7fb03006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.699+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb03006ae70 0x7fb03006d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb040001eb0 tx=0x7fb040000f10 comp rx=0 tx=0).stop 2023-11-08T19:27:36.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.699+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 msgr2=0x7fb04c15ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.699+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb04c15ba10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb04802bc00 tx=0x7fb048001120 comp rx=0 tx=0).stop 2023-11-08T19:27:36.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.700+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 shutdown_connections 2023-11-08T19:27:36.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.700+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb03006ae70 0x7fb03006d330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.700+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb04c0cce90 0x7fb04c0cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.700+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb04c1428b0 0x7fb04c0b2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.700+0000 7fb0543f5640 1 --2- 172.21.15.119:0/3771366158 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb038003970 0x7fb04c15ba10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.700+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 >> 172.21.15.119:0/3771366158 conn(0x7fb04c0cb610 msgr2=0x7fb04c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.701+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 shutdown_connections 2023-11-08T19:27:36.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.701+0000 7fb0543f5640 1 -- 172.21.15.119:0/3771366158 wait complete. 2023-11-08T19:27:36.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:36.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.916+0000 7f8206f75640 1 Processor -- start 2023-11-08T19:27:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.916+0000 7f8206f75640 1 -- start start 2023-11-08T19:27:36.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8206f75640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8200162d20 0x7f8200165110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8206f75640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820006cff0 con 0x7f820015b0d0 2023-11-08T19:27:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8206f75640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820006d2d0 con 0x7f82000ad7d0 2023-11-08T19:27:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8206f75640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f820006bd20 con 0x7f8200162d20 2023-11-08T19:27:36.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8205f73640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f820015b0d0 0x7f820015b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45544/0 (socket says 172.21.15.119:45544) 2023-11-08T19:27:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8205f73640 1 -- 172.21.15.119:0/3261998837 learned_addr learned my addr 172.21.15.119:0/3261998837 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:36.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.917+0000 7f8206774640 1 --2- 172.21.15.119:0/3261998837 >> v2:172.21.15.119:3300/0 conn(0x7f8200162d20 0x7f8200165110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.918+0000 7f8204f71640 1 -- 172.21.15.119:0/3261998837 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3116092019 0 0) 0x7f820006cff0 con 0x7f820015b0d0 2023-11-08T19:27:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.918+0000 7f8204f71640 1 -- 172.21.15.119:0/3261998837 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f0003520 con 0x7f820015b0d0 2023-11-08T19:27:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.918+0000 7f8206774640 1 -- 172.21.15.119:0/3261998837 >> v1:172.21.15.188:6789/0 conn(0x7f82000ad7d0 legacy=0x7f8200070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.918+0000 7f8206774640 1 -- 172.21.15.119:0/3261998837 >> v1:172.21.15.119:6789/0 conn(0x7f820015b0d0 legacy=0x7f820015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.918+0000 7f8206774640 1 -- 172.21.15.119:0/3261998837 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820006be60 con 0x7f8200162d20 2023-11-08T19:27:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.919+0000 7f8206774640 1 --2- 172.21.15.119:0/3261998837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8200162d20 0x7f8200165110 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f81f80097a0 tx=0x7f81f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9864965b06066f2a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8204f71640 1 -- 172.21.15.119:0/3261998837 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f802e070 con 0x7f8200162d20 2023-11-08T19:27:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8204f71640 1 -- 172.21.15.119:0/3261998837 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f80315a0 con 0x7f8200162d20 2023-11-08T19:27:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8204f71640 1 -- 172.21.15.119:0/3261998837 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8031930 con 0x7f8200162d20 2023-11-08T19:27:36.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 -- 172.21.15.119:0/3261998837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8200162d20 msgr2=0x7f8200165110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 --2- 172.21.15.119:0/3261998837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8200162d20 0x7f8200165110 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f81f80097a0 tx=0x7f81f802b840 comp rx=0 tx=0).stop 2023-11-08T19:27:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 -- 172.21.15.119:0/3261998837 shutdown_connections 2023-11-08T19:27:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 --2- 172.21.15.119:0/3261998837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8200162d20 0x7f8200165110 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 -- 172.21.15.119:0/3261998837 >> 172.21.15.119:0/3261998837 conn(0x7f82000b0560 msgr2=0x7f82000b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:36.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 -- 172.21.15.119:0/3261998837 shutdown_connections 2023-11-08T19:27:36.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.920+0000 7f8206f75640 1 -- 172.21.15.119:0/3261998837 wait complete. 2023-11-08T19:27:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.922+0000 7f8206f75640 1 Processor -- start 2023-11-08T19:27:36.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.922+0000 7f8206f75640 1 -- start start 2023-11-08T19:27:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8206f75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82000ad7d0 0x7f82000c8340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8206f75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 0x7f82000b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8206f75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8200162d20 0x7f82001f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8206f75640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82001510c0 con 0x7f820015b0d0 2023-11-08T19:27:36.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8206f75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82000b44d0 con 0x7f82000ad7d0 2023-11-08T19:27:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8206f75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8200151360 con 0x7f8200162d20 2023-11-08T19:27:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8205772640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 0x7f82000b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8205772640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 0x7f82000b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34168/0 (socket says 172.21.15.119:34168) 2023-11-08T19:27:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.923+0000 7f8205772640 1 -- 172.21.15.119:0/2451403104 learned_addr learned my addr 172.21.15.119:0/2451403104 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:36.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205f73640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82000ad7d0 0x7f82000c8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205772640 1 -- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8200162d20 msgr2=0x7f82001f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205772640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8200162d20 0x7f82001f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205772640 1 -- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82000ad7d0 msgr2=0x7f82000c8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:36.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205772640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82000ad7d0 0x7f82000c8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:36.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205772640 1 -- 172.21.15.119:0/2451403104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8009400 con 0x7f820015b0d0 2023-11-08T19:27:36.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8205772640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 0x7f82000b32e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00c430 tx=0x7f81ec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec011790 con 0x7f820015b0d0 2023-11-08T19:27:36.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec0118f0 con 0x7f820015b0d0 2023-11-08T19:27:36.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec011ce0 con 0x7f820015b0d0 2023-11-08T19:27:36.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820014d9d0 con 0x7f820015b0d0 2023-11-08T19:27:36.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.924+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82000b3f10 con 0x7f820015b0d0 2023-11-08T19:27:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.925+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82000c8880 con 0x7f820015b0d0 2023-11-08T19:27:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.926+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81ec01c460 con 0x7f820015b0d0 2023-11-08T19:27:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.927+0000 7f81fe7fc640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406aea0 0x7f81e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.927+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f81ec0898e0 con 0x7f820015b0d0 2023-11-08T19:27:36.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.928+0000 7f8205f73640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406aea0 0x7f81e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.928+0000 7f8205f73640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406aea0 0x7f81e406d360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f81f4001560 tx=0x7f81f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:36.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:36.932+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81ec054650 con 0x7f820015b0d0 2023-11-08T19:27:37.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.125+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f82001514d0 con 0x7f820015b0d0 2023-11-08T19:27:37.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.127+0000 7f81fe7fc640 1 -- 172.21.15.119:0/2451403104 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18539 (secure 0 0 0) 0x7f81ec057c70 con 0x7f820015b0d0 2023-11-08T19:27:37.129 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:37.129 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":64,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:36.121600+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:37.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.129+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406aea0 msgr2=0x7f81e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.129+0000 7f8206f75640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406aea0 0x7f81e406d360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f81f4001560 tx=0x7f81f4009040 comp rx=0 tx=0).stop 2023-11-08T19:27:37.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.129+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 msgr2=0x7f82000b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.129+0000 7f8206f75640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 0x7f82000b32e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f81ec00c430 tx=0x7f81ec00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 shutdown_connections 2023-11-08T19:27:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81e406aea0 0x7f81e406d360 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8200162d20 0x7f82001f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f820015b0d0 0x7f82000b32e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 --2- 172.21.15.119:0/2451403104 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f82000ad7d0 0x7f82000c8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 >> 172.21.15.119:0/2451403104 conn(0x7f82000b0560 msgr2=0x7f82000ae460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 shutdown_connections 2023-11-08T19:27:37.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.130+0000 7f8206f75640 1 -- 172.21.15.119:0/2451403104 wait complete. 2023-11-08T19:27:37.149 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:37.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.350+0000 7f146e261640 1 Processor -- start 2023-11-08T19:27:37.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.350+0000 7f146e261640 1 -- start start 2023-11-08T19:27:37.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f146e261640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f146815d180 0x7f14680c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f146e261640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14680ae410 con 0x7f14680cc810 2023-11-08T19:27:37.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f146e261640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f146806ba30 con 0x7f146806a390 2023-11-08T19:27:37.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f146e261640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f146806bcf0 con 0x7f146815d180 2023-11-08T19:27:37.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f145ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f14680cc810 0x7f1468160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45550/0 (socket says 172.21.15.119:45550) 2023-11-08T19:27:37.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f145ffff640 1 -- 172.21.15.119:0/1840295134 learned_addr learned my addr 172.21.15.119:0/1840295134 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:37.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f1467fff640 1 --2- 172.21.15.119:0/1840295134 >> v2:172.21.15.188:3300/0 conn(0x7f146815d180 0x7f14680c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200528793 0 0) 0x7f14680ae410 con 0x7f14680cc810 2023-11-08T19:27:37.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.352+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1454003520 con 0x7f14680cc810 2023-11-08T19:27:37.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893165366 0 0) 0x7f1454003520 con 0x7f14680cc810 2023-11-08T19:27:37.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 >> v2:172.21.15.188:3300/0 conn(0x7f146815d180 msgr2=0x7f14680c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:37.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 --2- 172.21.15.119:0/1840295134 >> v2:172.21.15.188:3300/0 conn(0x7f146815d180 0x7f14680c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 >> v1:172.21.15.188:6789/0 conn(0x7f146806a390 legacy=0x7f146815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f146806bfd0 con 0x7f14680cc810 2023-11-08T19:27:37.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f1450001670 con 0x7f14680cc810 2023-11-08T19:27:37.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 >> v1:172.21.15.119:6789/0 conn(0x7f14680cc810 legacy=0x7f1468160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14540039c0 0x7f1454003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f1454004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1454005180 0x7f145400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146806bcf0 con 0x7f14540047c0 2023-11-08T19:27:37.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145400c080 con 0x7f14540039c0 2023-11-08T19:27:37.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f145400c240 con 0x7f1454005180 2023-11-08T19:27:37.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.353+0000 7f145ffff640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f1454004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f145ffff640 1 -- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1454005180 msgr2=0x7f145400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:37.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f145ffff640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1454005180 0x7f145400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f145ffff640 1 -- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14540039c0 msgr2=0x7f1454003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:37.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f14677fe640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14540039c0 0x7f1454003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f145ffff640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14540039c0 0x7f1454003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f1467fff640 1 -- 172.21.15.119:0/1840295134 reap_dead start 2023-11-08T19:27:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f145ffff640 1 -- 172.21.15.119:0/1840295134 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1454003520 con 0x7f14540047c0 2023-11-08T19:27:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.354+0000 7f14677fe640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14540039c0 0x7f1454003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:37.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.355+0000 7f145ffff640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f1454004c40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f14680ccc60 tx=0x7f14540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48bdd4ed2ad624b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.355+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1450005ad0 con 0x7f14540047c0 2023-11-08T19:27:37.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.355+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1450005c30 con 0x7f14540047c0 2023-11-08T19:27:37.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.355+0000 7f1466ffd640 1 -- 172.21.15.119:0/1840295134 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145002c9d0 con 0x7f14540047c0 2023-11-08T19:27:37.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 -- 172.21.15.119:0/1840295134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 msgr2=0x7f1454004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f1454004c40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f14680ccc60 tx=0x7f14540057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 -- 172.21.15.119:0/1840295134 shutdown_connections 2023-11-08T19:27:37.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 --2- 172.21.15.119:0/1840295134 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f1454004c40 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 -- 172.21.15.119:0/1840295134 >> 172.21.15.119:0/1840295134 conn(0x7f14680cb610 msgr2=0x7f14680cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 -- 172.21.15.119:0/1840295134 shutdown_connections 2023-11-08T19:27:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.356+0000 7f146e261640 1 -- 172.21.15.119:0/1840295134 wait complete. 2023-11-08T19:27:37.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.357+0000 7f146e261640 1 Processor -- start 2023-11-08T19:27:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 -- start start 2023-11-08T19:27:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f14680ae240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14681466c0 0x7f146815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14680ccf00 0x7f14680cd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14680cd8c0 con 0x7f14540047c0 2023-11-08T19:27:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146815d6c0 con 0x7f14681466c0 2023-11-08T19:27:37.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.358+0000 7f146e261640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1468146b40 con 0x7f14680ccf00 2023-11-08T19:27:37.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f14680ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f14680ae240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34188/0 (socket says 172.21.15.119:34188) 2023-11-08T19:27:37.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 -- 172.21.15.119:0/844403204 learned_addr learned my addr 172.21.15.119:0/844403204 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:37.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 -- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14680ccf00 msgr2=0x7f14680cd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14680ccf00 0x7f14680cd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 -- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14681466c0 msgr2=0x7f146815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f145ffff640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14681466c0 0x7f146815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14681466c0 0x7f146815fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 -- 172.21.15.119:0/844403204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680b0640 con 0x7f14540047c0 2023-11-08T19:27:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.359+0000 7f14677fe640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f14680ae240 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f145800d380 tx=0x7f145800d840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.360+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145800b070 con 0x7f14540047c0 2023-11-08T19:27:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.360+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14580054e0 con 0x7f14540047c0 2023-11-08T19:27:37.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.360+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14680b0920 con 0x7f14540047c0 2023-11-08T19:27:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.360+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458003850 con 0x7f14540047c0 2023-11-08T19:27:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.360+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14680b0ee0 con 0x7f14540047c0 2023-11-08T19:27:37.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.361+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14680b10a0 con 0x7f14540047c0 2023-11-08T19:27:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.362+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1458003a70 con 0x7f14540047c0 2023-11-08T19:27:37.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.363+0000 7f1464ff9640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f144806ae70 0x7f144806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.363+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1458088ff0 con 0x7f14540047c0 2023-11-08T19:27:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.363+0000 7f145ffff640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f144806ae70 0x7f144806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.363+0000 7f145ffff640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f144806ae70 0x7f144806d330 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1450034000 tx=0x7f1450009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.367+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1458053f00 con 0x7f14540047c0 2023-11-08T19:27:37.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.555+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1468074430 con 0x7f14540047c0 2023-11-08T19:27:37.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.558+0000 7f1464ff9640 1 -- 172.21.15.119:0/844403204 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+18539 (secure 0 0 0) 0x7f1458018090 con 0x7f14540047c0 2023-11-08T19:27:37.560 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:37.560 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":64,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:36.121600+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:37.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f144806ae70 msgr2=0x7f144806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f144806ae70 0x7f144806d330 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1450034000 tx=0x7f1450009030 comp rx=0 tx=0).stop 2023-11-08T19:27:37.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 msgr2=0x7f14680ae240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f14680ae240 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f145800d380 tx=0x7f145800d840 comp rx=0 tx=0).stop 2023-11-08T19:27:37.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 shutdown_connections 2023-11-08T19:27:37.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f144806ae70 0x7f144806d330 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14680ccf00 0x7f14680cd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14681466c0 0x7f146815fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 --2- 172.21.15.119:0/844403204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f14540047c0 0x7f14680ae240 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 >> 172.21.15.119:0/844403204 conn(0x7f14680cb610 msgr2=0x7f14680cab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 shutdown_connections 2023-11-08T19:27:37.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.561+0000 7f146e261640 1 -- 172.21.15.119:0/844403204 wait complete. 2023-11-08T19:27:37.583 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:27:37.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.803+0000 7ff3df64b640 1 Processor -- start 2023-11-08T19:27:37.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.803+0000 7ff3df64b640 1 -- start start 2023-11-08T19:27:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3df64b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3d8144580 0x7ff3d815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3df64b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144960 0x7ff3d80cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3df64b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d815ea60 con 0x7ff3d815f1d0 2023-11-08T19:27:37.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3df64b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80c9150 con 0x7ff3d8144580 2023-11-08T19:27:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3df64b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80cb270 con 0x7ff3d8144960 2023-11-08T19:27:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3dd3e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff3d8144580 0x7ff3d815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3ddbe8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff3d815f1d0 0x7ff3d80c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45560/0 (socket says 172.21.15.119:45560) 2023-11-08T19:27:37.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3ddbe8640 1 -- 172.21.15.119:0/3218115196 learned_addr learned my addr 172.21.15.119:0/3218115196 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.805+0000 7ff3dcbe6640 1 --2- 172.21.15.119:0/3218115196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144960 0x7ff3d80cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.806+0000 7ff3dd3e7640 1 -- 172.21.15.119:0/3218115196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144960 msgr2=0x7ff3d80cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:37.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.806+0000 7ff3dd3e7640 1 --2- 172.21.15.119:0/3218115196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144960 0x7ff3d80cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.806+0000 7ff3dd3e7640 1 -- 172.21.15.119:0/3218115196 >> v1:172.21.15.119:6789/0 conn(0x7ff3d815f1d0 legacy=0x7ff3d80c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.806+0000 7ff3dd3e7640 1 -- 172.21.15.119:0/3218115196 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80c9360 con 0x7ff3d8144580 2023-11-08T19:27:37.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.806+0000 7ff3cbfff640 1 -- 172.21.15.119:0/3218115196 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3827019112 0 0) 0x7ff3d815ea60 con 0x7ff3d815f1d0 2023-11-08T19:27:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.807+0000 7ff3dd3e7640 1 --2- 172.21.15.119:0/3218115196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144580 0x7ff3d815bff0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40097a0 tx=0x7ff3d402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a96469315bca87 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.807+0000 7ff3cbfff640 1 -- 172.21.15.119:0/3218115196 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d402e070 con 0x7ff3d8144580 2023-11-08T19:27:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.807+0000 7ff3cbfff640 1 -- 172.21.15.119:0/3218115196 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d40315a0 con 0x7ff3d8144580 2023-11-08T19:27:37.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.807+0000 7ff3cbfff640 1 -- 172.21.15.119:0/3218115196 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d4031930 con 0x7ff3d8144580 2023-11-08T19:27:37.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 -- 172.21.15.119:0/3218115196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144580 msgr2=0x7ff3d815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 --2- 172.21.15.119:0/3218115196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144580 0x7ff3d815bff0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40097a0 tx=0x7ff3d402b840 comp rx=0 tx=0).stop 2023-11-08T19:27:37.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 -- 172.21.15.119:0/3218115196 shutdown_connections 2023-11-08T19:27:37.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 --2- 172.21.15.119:0/3218115196 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144960 0x7ff3d80cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 --2- 172.21.15.119:0/3218115196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144580 0x7ff3d815bff0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 -- 172.21.15.119:0/3218115196 >> 172.21.15.119:0/3218115196 conn(0x7ff3d80cb610 msgr2=0x7ff3d80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:37.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 -- 172.21.15.119:0/3218115196 shutdown_connections 2023-11-08T19:27:37.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.808+0000 7ff3df64b640 1 -- 172.21.15.119:0/3218115196 wait complete. 2023-11-08T19:27:37.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.809+0000 7ff3df64b640 1 Processor -- start 2023-11-08T19:27:37.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.810+0000 7ff3df64b640 1 -- start start 2023-11-08T19:27:37.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3df64b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144580 0x7ff3d80cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3df64b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 0x7ff3d813d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3df64b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d815f1d0 0x7ff3d813d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3df64b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d813e8e0 con 0x7ff3d8144960 2023-11-08T19:27:37.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3df64b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d813dc80 con 0x7ff3d815f1d0 2023-11-08T19:27:37.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3df64b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d813df60 con 0x7ff3d8144580 2023-11-08T19:27:37.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 0x7ff3d813d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 0x7ff3d813d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34212/0 (socket says 172.21.15.119:34212) 2023-11-08T19:27:37.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 -- 172.21.15.119:0/2721928093 learned_addr learned my addr 172.21.15.119:0/2721928093 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:37.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dd3e7640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144580 0x7ff3d80cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 -- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144580 msgr2=0x7ff3d80cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3ddbe8640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d815f1d0 0x7ff3d813d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144580 0x7ff3d80cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 -- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d815f1d0 msgr2=0x7ff3d813d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:37.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d815f1d0 0x7ff3d813d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:37.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.811+0000 7ff3dcbe6640 1 -- 172.21.15.119:0/2721928093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d4009400 con 0x7ff3d8144960 2023-11-08T19:27:37.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.812+0000 7ff3dcbe6640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 0x7ff3d813d200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00d800 tx=0x7ff3cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.812+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00b6c0 con 0x7ff3d8144960 2023-11-08T19:27:37.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.812+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3cc00b820 con 0x7ff3d8144960 2023-11-08T19:27:37.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.812+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d813e220 con 0x7ff3d8144960 2023-11-08T19:27:37.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.812+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3cc00bbb0 con 0x7ff3d8144960 2023-11-08T19:27:37.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.812+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d813ed00 con 0x7ff3d8144960 2023-11-08T19:27:37.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.813+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d8074430 con 0x7ff3d8144960 2023-11-08T19:27:37.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.814+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3cc00bd10 con 0x7ff3d8144960 2023-11-08T19:27:37.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.816+0000 7ff3c9ffb640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3bc06ae70 0x7ff3bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:37.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.816+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff3cc089dc0 con 0x7ff3d8144960 2023-11-08T19:27:37.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.816+0000 7ff3dd3e7640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3bc06ae70 0x7ff3bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:37.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.817+0000 7ff3dd3e7640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3bc06ae70 0x7ff3bc06d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4036000 tx=0x7ff3d403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:37.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:37.820+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff3cc054c60 con 0x7ff3d8144960 2023-11-08T19:27:38.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.047+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff3d806cb10 con 0x7ff3d8144960 2023-11-08T19:27:38.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.701+0000 7ff3c9ffb640 1 -- 172.21.15.119:0/2721928093 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff3cc058280 con 0x7ff3d8144960 2023-11-08T19:27:38.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.705+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3bc06ae70 msgr2=0x7ff3bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.705+0000 7ff3df64b640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3bc06ae70 0x7ff3bc06d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4036000 tx=0x7ff3d403d040 comp rx=0 tx=0).stop 2023-11-08T19:27:38.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.705+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 msgr2=0x7ff3d813d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.705+0000 7ff3df64b640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 0x7ff3d813d200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc00d800 tx=0x7ff3cc00dcc0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 shutdown_connections 2023-11-08T19:27:38.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff3bc06ae70 0x7ff3bc06d330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff3d815f1d0 0x7ff3d813d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff3d8144960 0x7ff3d813d200 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 --2- 172.21.15.119:0/2721928093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff3d8144580 0x7ff3d80cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 >> 172.21.15.119:0/2721928093 conn(0x7ff3d80cb610 msgr2=0x7ff3d8072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 shutdown_connections 2023-11-08T19:27:38.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.706+0000 7ff3df64b640 1 -- 172.21.15.119:0/2721928093 wait complete. 2023-11-08T19:27:38.729 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:27:38.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:27:38.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.946+0000 7f7ea6a1a640 1 Processor -- start 2023-11-08T19:27:38.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.946+0000 7f7ea6a1a640 1 -- start start 2023-11-08T19:27:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.947+0000 7f7ea6a1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ea015b2b0 0x7f7ea0071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.947+0000 7f7ea6a1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 0x7f7ea00ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea6a1a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea00c9440 con 0x7f7ea00cf5e0 2023-11-08T19:27:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea6a1a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea00c9690 con 0x7f7ea015b2b0 2023-11-08T19:27:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea6a1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00c9970 con 0x7f7ea015b690 2023-11-08T19:27:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea6219640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7ea00cf5e0 0x7f7ea00ca990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41778/0 (socket says 172.21.15.119:41778) 2023-11-08T19:27:38.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea6219640 1 -- 172.21.15.119:0/904122684 learned_addr learned my addr 172.21.15.119:0/904122684 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:38.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea5217640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 0x7f7ea00ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140669837 0 0) 0x7f7ea00c9440 con 0x7f7ea00cf5e0 2023-11-08T19:27:38.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea5a18640 1 --2- 172.21.15.119:0/904122684 >> v2:172.21.15.188:3300/0 conn(0x7f7ea015b2b0 0x7f7ea0071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.948+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e8c003520 con 0x7f7ea00cf5e0 2023-11-08T19:27:38.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102828004 0 0) 0x7f7e8c003520 con 0x7f7ea00cf5e0 2023-11-08T19:27:38.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 msgr2=0x7f7ea00ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 0x7f7ea00ca450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 >> v2:172.21.15.188:3300/0 conn(0x7f7ea015b2b0 msgr2=0x7f7ea0071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 --2- 172.21.15.119:0/904122684 >> v2:172.21.15.188:3300/0 conn(0x7f7ea015b2b0 0x7f7ea0071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00c9c50 con 0x7f7ea00cf5e0 2023-11-08T19:27:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f7e980019f0 con 0x7f7ea00cf5e0 2023-11-08T19:27:38.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 >> v1:172.21.15.119:6789/0 conn(0x7f7ea00cf5e0 legacy=0x7f7ea00ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e8c003970 0x7f7e8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7e8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 0x7f7e8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00c9690 con 0x7f7e8c0047c0 2023-11-08T19:27:38.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea00c9970 con 0x7f7e8c003970 2023-11-08T19:27:38.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8c00ba30 con 0x7f7ea015b690 2023-11-08T19:27:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea5a18640 1 --2- 172.21.15.119:0/904122684 >> v2:172.21.15.188:3300/0 conn(0x7f7ea015b2b0 0x7f7ea0071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:27:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.949+0000 7f7ea5217640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7e8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.950+0000 7f7ea5217640 1 -- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 msgr2=0x7f7e8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.950+0000 7f7ea6219640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 0x7f7e8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.950+0000 7f7ea5217640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea015b690 0x7f7e8c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.950+0000 7f7ea5217640 1 -- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e8c003970 msgr2=0x7f7e8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.950+0000 7f7ea5217640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e8c003970 0x7f7e8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.950+0000 7f7ea5217640 1 -- 172.21.15.119:0/904122684 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e8c003520 con 0x7f7e8c0047c0 2023-11-08T19:27:38.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.951+0000 7f7ea5217640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7e8c004c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7ea015ba70 tx=0x7f7e8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88058d1228cd5445 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.951+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9002c580 con 0x7f7e8c0047c0 2023-11-08T19:27:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.951+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9002c6e0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.951+0000 7f7ea4a16640 1 -- 172.21.15.119:0/904122684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9002ca70 con 0x7f7e8c0047c0 2023-11-08T19:27:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/904122684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 msgr2=0x7f7e8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/904122684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7e8c004c40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7ea015ba70 tx=0x7f7e8c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6219640 1 -- 172.21.15.119:0/904122684 reap_dead start 2023-11-08T19:27:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/904122684 shutdown_connections 2023-11-08T19:27:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/904122684 >> 172.21.15.119:0/904122684 conn(0x7f7ea00ceac0 msgr2=0x7f7ea00ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/904122684 shutdown_connections 2023-11-08T19:27:38.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.952+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/904122684 wait complete. 2023-11-08T19:27:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.953+0000 7f7ea6a1a640 1 Processor -- start 2023-11-08T19:27:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.954+0000 7f7ea6a1a640 1 -- start start 2023-11-08T19:27:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea6a1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7ea014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea6a1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea006c630 0x7f7ea006ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea6a1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea015d500 0x7f7ea015d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea6a1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea006cfb0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea6a1a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea015cbb0 con 0x7f7ea015d500 2023-11-08T19:27:38.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea6a1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea015bb10 con 0x7f7ea006c630 2023-11-08T19:27:38.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7ea014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7ea014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54758/0 (socket says 172.21.15.119:54758) 2023-11-08T19:27:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 -- 172.21.15.119:0/2988448968 learned_addr learned my addr 172.21.15.119:0/2988448968 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 -- 172.21.15.119:0/2988448968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea006c630 msgr2=0x7f7ea006ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea006c630 0x7f7ea006ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 -- 172.21.15.119:0/2988448968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea015d500 msgr2=0x7f7ea015d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:38.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea015d500 0x7f7ea015d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:38.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.955+0000 7f7ea5a18640 1 -- 172.21.15.119:0/2988448968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea015bdf0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.956+0000 7f7ea5a18640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7ea014ee30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e7e0 tx=0x7f7e9c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.956+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c004230 con 0x7f7e8c0047c0 2023-11-08T19:27:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.956+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00ee90 con 0x7f7e8c0047c0 2023-11-08T19:27:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.956+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea015c0d0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.956+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00c6d0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.956+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea015c690 con 0x7f7e8c0047c0 2023-11-08T19:27:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.957+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea004ece0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.958+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e9c00c830 con 0x7f7e8c0047c0 2023-11-08T19:27:38.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.959+0000 7f7e967fc640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7e8406aea0 0x7f7e8406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:38.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.959+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7e9c0896a0 con 0x7f7e8c0047c0 2023-11-08T19:27:38.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.959+0000 7f7ea5217640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7e8406aea0 0x7f7e8406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:38.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.960+0000 7f7ea5217640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7e8406aea0 0x7f7e8406d360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036000 tx=0x7f7e9002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:38.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:38.963+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e9c054380 con 0x7f7e8c0047c0 2023-11-08T19:27:39.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.194+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7ea0070860 con 0x7f7e8c0047c0 2023-11-08T19:27:39.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.713+0000 7f7e967fc640 1 -- 172.21.15.119:0/2988448968 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7e9c0579a0 con 0x7f7e8c0047c0 2023-11-08T19:27:39.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7e8406aea0 msgr2=0x7f7e8406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7e8406aea0 0x7f7e8406d360 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7e90036000 tx=0x7f7e9002e000 comp rx=0 tx=0).stop 2023-11-08T19:27:39.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 msgr2=0x7f7ea014ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7ea014ee30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c00e7e0 tx=0x7f7e9c00eca0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 shutdown_connections 2023-11-08T19:27:39.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7e8406aea0 0x7f7e8406d360 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ea015d500 0x7f7ea015d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ea006c630 0x7f7ea006ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 --2- 172.21.15.119:0/2988448968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e8c0047c0 0x7f7ea014ee30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 >> 172.21.15.119:0/2988448968 conn(0x7f7ea00ceac0 msgr2=0x7f7ea015a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:39.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 shutdown_connections 2023-11-08T19:27:39.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.718+0000 7f7ea6a1a640 1 -- 172.21.15.119:0/2988448968 wait complete. 2023-11-08T19:27:39.741 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:27:39.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:39.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.955+0000 7fc98972c640 1 Processor -- start 2023-11-08T19:27:39.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.955+0000 7fc98972c640 1 -- start start 2023-11-08T19:27:39.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc98972c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc9841447a0 0x7fc98415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc98972c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9840cad30 0x7fc98406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc98972c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc98406a120 0x7fc98406a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc98972c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc98406aa90 con 0x7fc9841447a0 2023-11-08T19:27:39.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc98972c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9840c3a50 con 0x7fc9840cad30 2023-11-08T19:27:39.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc98972c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc984144b80 con 0x7fc98406a120 2023-11-08T19:27:39.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc982ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc9841447a0 0x7fc98415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc982ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc9841447a0 0x7fc98415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54760/0 (socket says 172.21.15.119:54760) 2023-11-08T19:27:39.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.957+0000 7fc982ffd640 1 -- 172.21.15.119:0/3897130394 learned_addr learned my addr 172.21.15.119:0/3897130394 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:39.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.958+0000 7fc982ffd640 1 -- 172.21.15.119:0/3897130394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc98406a120 msgr2=0x7fc98406a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:39.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.958+0000 7fc982ffd640 1 --2- 172.21.15.119:0/3897130394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc98406a120 0x7fc98406a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.958+0000 7fc97bfff640 1 --2- 172.21.15.119:0/3897130394 >> v2:172.21.15.188:3300/0 conn(0x7fc9840cad30 0x7fc98406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.958+0000 7fc982ffd640 1 -- 172.21.15.119:0/3897130394 >> v2:172.21.15.188:3300/0 conn(0x7fc9840cad30 msgr2=0x7fc98406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:39.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.958+0000 7fc982ffd640 1 --2- 172.21.15.119:0/3897130394 >> v2:172.21.15.188:3300/0 conn(0x7fc9840cad30 0x7fc98406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.958+0000 7fc982ffd640 1 -- 172.21.15.119:0/3897130394 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9840c9150 con 0x7fc9841447a0 2023-11-08T19:27:39.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.959+0000 7fc982ffd640 1 --2- 172.21.15.119:0/3897130394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9841447a0 0x7fc98415f4c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc98406df20 tx=0x7fc97402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6bcba78a61f186 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:39.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.959+0000 7fc9827fc640 1 -- 172.21.15.119:0/3897130394 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974031860 con 0x7fc9841447a0 2023-11-08T19:27:39.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.959+0000 7fc9827fc640 1 -- 172.21.15.119:0/3897130394 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9740319c0 con 0x7fc9841447a0 2023-11-08T19:27:39.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.959+0000 7fc9827fc640 1 -- 172.21.15.119:0/3897130394 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974031d50 con 0x7fc9841447a0 2023-11-08T19:27:39.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 -- 172.21.15.119:0/3897130394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9841447a0 msgr2=0x7fc98415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 --2- 172.21.15.119:0/3897130394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9841447a0 0x7fc98415f4c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fc98406df20 tx=0x7fc97402b900 comp rx=0 tx=0).stop 2023-11-08T19:27:39.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 -- 172.21.15.119:0/3897130394 shutdown_connections 2023-11-08T19:27:39.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 --2- 172.21.15.119:0/3897130394 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc98406a120 0x7fc98406a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 --2- 172.21.15.119:0/3897130394 >> v2:172.21.15.188:3300/0 conn(0x7fc9840cad30 0x7fc98406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 --2- 172.21.15.119:0/3897130394 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9841447a0 0x7fc98415f4c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 -- 172.21.15.119:0/3897130394 >> 172.21.15.119:0/3897130394 conn(0x7fc9840cb610 msgr2=0x7fc9840cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:39.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 -- 172.21.15.119:0/3897130394 shutdown_connections 2023-11-08T19:27:39.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.960+0000 7fc98972c640 1 -- 172.21.15.119:0/3897130394 wait complete. 2023-11-08T19:27:39.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.962+0000 7fc98972c640 1 Processor -- start 2023-11-08T19:27:39.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.963+0000 7fc98972c640 1 -- start start 2023-11-08T19:27:39.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc98972c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 0x7fc98415a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc98972c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9840cad30 0x7fc9840cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc98972c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9841447a0 0x7fc9840c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc98972c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840b25b0 con 0x7fc98406a120 2023-11-08T19:27:39.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc98972c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840b2820 con 0x7fc9840cad30 2023-11-08T19:27:39.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc98972c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9840b2b00 con 0x7fc9841447a0 2023-11-08T19:27:39.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc982ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 0x7fc98415a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc982ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 0x7fc98415a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54774/0 (socket says 172.21.15.119:54774) 2023-11-08T19:27:39.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc982ffd640 1 -- 172.21.15.119:0/691710458 learned_addr learned my addr 172.21.15.119:0/691710458 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:39.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.964+0000 7fc97bfff640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9840cad30 0x7fc9840cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc9837fe640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9841447a0 0x7fc9840c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc982ffd640 1 -- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9841447a0 msgr2=0x7fc9840c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc982ffd640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9841447a0 0x7fc9840c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc982ffd640 1 -- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9840cad30 msgr2=0x7fc9840cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:39.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc982ffd640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9840cad30 0x7fc9840cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:39.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc982ffd640 1 -- 172.21.15.119:0/691710458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9740094d0 con 0x7fc98406a120 2023-11-08T19:27:39.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc97bfff640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9840cad30 0x7fc9840cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:39.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc9837fe640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9841447a0 0x7fc9840c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:39.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc982ffd640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 0x7fc98415a340 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc974000c00 tx=0x7fc9740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:39.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc974031830 con 0x7fc98406a120 2023-11-08T19:27:39.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9740443f0 con 0x7fc98406a120 2023-11-08T19:27:39.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9840b2de0 con 0x7fc98406a120 2023-11-08T19:27:39.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.965+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9740447a0 con 0x7fc98406a120 2023-11-08T19:27:39.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.966+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9840cdfb0 con 0x7fc98406a120 2023-11-08T19:27:39.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.967+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc984074430 con 0x7fc98406a120 2023-11-08T19:27:39.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.967+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc97404d450 con 0x7fc98406a120 2023-11-08T19:27:39.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.969+0000 7fc97b7fe640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc96406ae70 0x7fc96406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.969+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc9740ac040 con 0x7fc98406a120 2023-11-08T19:27:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.969+0000 7fc97bfff640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc96406ae70 0x7fc96406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.970+0000 7fc97bfff640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc96406ae70 0x7fc96406d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ce640 tx=0x7fc96c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:39.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:39.973+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc974076e50 con 0x7fc98406a120 2023-11-08T19:27:40.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.204+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc98406caa0 con 0x7fc98406a120 2023-11-08T19:27:40.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.205+0000 7fc97b7fe640 1 -- 172.21.15.119:0/691710458 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fc974035090 con 0x7fc98406a120 2023-11-08T19:27:40.206 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:40.207 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:36.122432+0000","modified":"2023-11-08T19:27:39.195905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/913059196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":913059196},{"type":"v1","addr":"172.21.15.188:6835","nonce":913059196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:40.207 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2023-11-08T19:27:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.207+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc96406ae70 msgr2=0x7fc96406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.207+0000 7fc98972c640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc96406ae70 0x7fc96406d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc9840ce640 tx=0x7fc96c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.207+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 msgr2=0x7fc98415a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.207+0000 7fc98972c640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 0x7fc98415a340 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc974000c00 tx=0x7fc9740046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 shutdown_connections 2023-11-08T19:27:40.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc96406ae70 0x7fc96406d330 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9841447a0 0x7fc9840c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9840cad30 0x7fc9840cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 --2- 172.21.15.119:0/691710458 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc98406a120 0x7fc98415a340 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 >> 172.21.15.119:0/691710458 conn(0x7fc9840cb610 msgr2=0x7fc984143ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:40.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 shutdown_connections 2023-11-08T19:27:40.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.208+0000 7fc98972c640 1 -- 172.21.15.119:0/691710458 wait complete. 2023-11-08T19:27:40.229 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:40.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.439+0000 7fab1c8c2640 1 Processor -- start 2023-11-08T19:27:40.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.439+0000 7fab1c8c2640 1 -- start start 2023-11-08T19:27:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab1c8c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 0x7fab180cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab1c8c2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab180c4960 con 0x7fab1806cb40 2023-11-08T19:27:40.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab1c8c2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab1815df60 con 0x7fab18142f80 2023-11-08T19:27:40.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab1c8c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab18143a40 con 0x7fab1815d9f0 2023-11-08T19:27:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab16d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fab1806cb40 0x7fab180c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41792/0 (socket says 172.21.15.119:41792) 2023-11-08T19:27:40.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab16d9d640 1 -- 172.21.15.119:0/3483229612 learned_addr learned my addr 172.21.15.119:0/3483229612 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.441+0000 7fab0ffff640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 0x7fab180cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025268097 0 0) 0x7fab180c4960 con 0x7fab1806cb40 2023-11-08T19:27:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab04003520 con 0x7fab1806cb40 2023-11-08T19:27:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668422306 0 0) 0x7fab04003520 con 0x7fab1806cb40 2023-11-08T19:27:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 msgr2=0x7fab180cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 0x7fab180cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 >> v1:172.21.15.188:6789/0 conn(0x7fab18142f80 legacy=0x7fab180c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1815e240 con 0x7fab1806cb40 2023-11-08T19:27:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fab100019f0 con 0x7fab1806cb40 2023-11-08T19:27:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 >> v1:172.21.15.119:6789/0 conn(0x7fab1806cb40 legacy=0x7fab180c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab040039a0 0x7fab04003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 0x7fab04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18143a40 con 0x7fab040047c0 2023-11-08T19:27:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400ba80 con 0x7fab040039a0 2023-11-08T19:27:40.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0400bc40 con 0x7fab1815d9f0 2023-11-08T19:27:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.442+0000 7fab0ffff640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab040047c0 0x7fab04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.443+0000 7fab1659c640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab040039a0 0x7fab04003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.443+0000 7fab0ffff640 1 -- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 msgr2=0x7fab04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.443+0000 7fab0ffff640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab1815d9f0 0x7fab04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.443+0000 7fab0ffff640 1 -- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab040039a0 msgr2=0x7fab04003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.443+0000 7fab0ffff640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab040039a0 0x7fab04003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.443+0000 7fab0ffff640 1 -- 172.21.15.119:0/3483229612 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab04003520 con 0x7fab040047c0 2023-11-08T19:27:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.444+0000 7fab0ffff640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab040047c0 0x7fab04004c40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fab00009ac0 tx=0x7fab040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367041c8ae1d9a3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.444+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0002c710 con 0x7fab040047c0 2023-11-08T19:27:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.444+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0002c870 con 0x7fab040047c0 2023-11-08T19:27:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.444+0000 7fab15d9b640 1 -- 172.21.15.119:0/3483229612 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0002cc00 con 0x7fab040047c0 2023-11-08T19:27:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab1c8c2640 1 -- 172.21.15.119:0/3483229612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab040047c0 msgr2=0x7fab04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/3483229612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab040047c0 0x7fab04004c40 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fab00009ac0 tx=0x7fab040057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab16d9d640 1 -- 172.21.15.119:0/3483229612 reap_dead start 2023-11-08T19:27:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab1c8c2640 1 -- 172.21.15.119:0/3483229612 shutdown_connections 2023-11-08T19:27:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab1c8c2640 1 -- 172.21.15.119:0/3483229612 >> 172.21.15.119:0/3483229612 conn(0x7fab180cb610 msgr2=0x7fab180cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab1c8c2640 1 -- 172.21.15.119:0/3483229612 shutdown_connections 2023-11-08T19:27:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.445+0000 7fab1c8c2640 1 -- 172.21.15.119:0/3483229612 wait complete. 2023-11-08T19:27:40.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.447+0000 7fab1c8c2640 1 Processor -- start 2023-11-08T19:27:40.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.448+0000 7fab1c8c2640 1 -- start start 2023-11-08T19:27:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1c8c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab040047c0 0x7fab18141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1c8c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 0x7fab181593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1c8c2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1813f2a0 0x7fab181367e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1c8c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18158f10 con 0x7fab18135d90 2023-11-08T19:27:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1c8c2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab18142020 con 0x7fab1813f2a0 2023-11-08T19:27:40.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1c8c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1813fc10 con 0x7fab040047c0 2023-11-08T19:27:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 0x7fab181593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 0x7fab181593a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54786/0 (socket says 172.21.15.119:54786) 2023-11-08T19:27:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 -- 172.21.15.119:0/1976674400 learned_addr learned my addr 172.21.15.119:0/1976674400 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab1659c640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab040047c0 0x7fab18141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 -- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab040047c0 msgr2=0x7fab18141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab040047c0 0x7fab18141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab16d9d640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1813f2a0 0x7fab181367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 -- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1813f2a0 msgr2=0x7fab181367e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1813f2a0 0x7fab181367e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.449+0000 7fab0ffff640 1 -- 172.21.15.119:0/1976674400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1813fef0 con 0x7fab18135d90 2023-11-08T19:27:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.450+0000 7fab0ffff640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 0x7fab181593a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab00037040 tx=0x7fab00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.450+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00003810 con 0x7fab18135d90 2023-11-08T19:27:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.450+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0002de90 con 0x7fab18135d90 2023-11-08T19:27:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.450+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab181588c0 con 0x7fab18135d90 2023-11-08T19:27:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.450+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab00003bb0 con 0x7fab18135d90 2023-11-08T19:27:40.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.450+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab181598e0 con 0x7fab18135d90 2023-11-08T19:27:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.451+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab18158720 con 0x7fab18135d90 2023-11-08T19:27:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.452+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab00003d10 con 0x7fab18135d90 2023-11-08T19:27:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.453+0000 7fab0effd640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf806aea0 0x7faaf806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.453+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fab000ab850 con 0x7fab18135d90 2023-11-08T19:27:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.454+0000 7fab1659c640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf806aea0 0x7faaf806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.454+0000 7fab1659c640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf806aea0 0x7faaf806d360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fab08001720 tx=0x7fab08007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.457+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab18158720 con 0x7fab18135d90 2023-11-08T19:27:40.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.652+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab18158720 con 0x7fab18135d90 2023-11-08T19:27:40.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.654+0000 7fab0effd640 1 -- 172.21.15.119:0/1976674400 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18526 (secure 0 0 0) 0x7fab18158720 con 0x7fab18135d90 2023-11-08T19:27:40.656 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:40.656 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":65,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:37.688093+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:40.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf806aea0 msgr2=0x7faaf806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf806aea0 0x7faaf806d360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fab08001720 tx=0x7fab08007b70 comp rx=0 tx=0).stop 2023-11-08T19:27:40.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 msgr2=0x7fab181593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 0x7fab181593a0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fab00037040 tx=0x7fab00004730 comp rx=0 tx=0).stop 2023-11-08T19:27:40.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 shutdown_connections 2023-11-08T19:27:40.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf806aea0 0x7faaf806d360 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab1813f2a0 0x7fab181367e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab18135d90 0x7fab181593a0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 --2- 172.21.15.119:0/1976674400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab040047c0 0x7fab18141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 >> 172.21.15.119:0/1976674400 conn(0x7fab180cb610 msgr2=0x7fab180ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:40.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 shutdown_connections 2023-11-08T19:27:40.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.657+0000 7fab1c8c2640 1 -- 172.21.15.119:0/1976674400 wait complete. 2023-11-08T19:27:40.677 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:27:40.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:40.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.886+0000 7ff865ec9640 1 Processor -- start 2023-11-08T19:27:40.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.886+0000 7ff865ec9640 1 -- start start 2023-11-08T19:27:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff865ec9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff86015e5f0 0x7ff86015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff865ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 0x7ff86006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff865ec9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8600ce750 con 0x7ff86015d410 2023-11-08T19:27:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff865ec9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff86006c1f0 con 0x7ff86015e5f0 2023-11-08T19:27:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff865ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86006b570 con 0x7ff86015dd30 2023-11-08T19:27:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff85effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff86015e5f0 0x7ff86015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff85f7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff86015d410 0x7ff8600ce040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41802/0 (socket says 172.21.15.119:41802) 2023-11-08T19:27:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.888+0000 7ff85f7fe640 1 -- 172.21.15.119:0/11079085 learned_addr learned my addr 172.21.15.119:0/11079085 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.889+0000 7ff85effd640 1 -- 172.21.15.119:0/11079085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 msgr2=0x7ff86006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.889+0000 7ff85effd640 1 --2- 172.21.15.119:0/11079085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 0x7ff86006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.889+0000 7ff85effd640 1 -- 172.21.15.119:0/11079085 >> v1:172.21.15.119:6789/0 conn(0x7ff86015d410 legacy=0x7ff8600ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.889+0000 7ff85e7fc640 1 -- 172.21.15.119:0/11079085 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228215825 0 0) 0x7ff8600ce750 con 0x7ff86015d410 2023-11-08T19:27:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.889+0000 7ff85effd640 1 -- 172.21.15.119:0/11079085 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff86006b850 con 0x7ff86015e5f0 2023-11-08T19:27:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.890+0000 7ff85effd640 1 --2- 172.21.15.119:0/11079085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015e5f0 0x7ff86015d7f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff8540097a0 tx=0x7ff85402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1609772536e73cc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.890+0000 7ff85e7fc640 1 -- 172.21.15.119:0/11079085 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff85402e070 con 0x7ff86015e5f0 2023-11-08T19:27:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.890+0000 7ff85e7fc640 1 -- 172.21.15.119:0/11079085 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8540315a0 con 0x7ff86015e5f0 2023-11-08T19:27:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.890+0000 7ff85e7fc640 1 -- 172.21.15.119:0/11079085 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff854031930 con 0x7ff86015e5f0 2023-11-08T19:27:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 -- 172.21.15.119:0/11079085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015e5f0 msgr2=0x7ff86015d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 --2- 172.21.15.119:0/11079085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015e5f0 0x7ff86015d7f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff8540097a0 tx=0x7ff85402b840 comp rx=0 tx=0).stop 2023-11-08T19:27:40.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 -- 172.21.15.119:0/11079085 shutdown_connections 2023-11-08T19:27:40.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 --2- 172.21.15.119:0/11079085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 0x7ff86006dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 --2- 172.21.15.119:0/11079085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015e5f0 0x7ff86015d7f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 -- 172.21.15.119:0/11079085 >> 172.21.15.119:0/11079085 conn(0x7ff8600d0db0 msgr2=0x7ff8600d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:40.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 -- 172.21.15.119:0/11079085 shutdown_connections 2023-11-08T19:27:40.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.891+0000 7ff865ec9640 1 -- 172.21.15.119:0/11079085 wait complete. 2023-11-08T19:27:40.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.893+0000 7ff865ec9640 1 Processor -- start 2023-11-08T19:27:40.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.893+0000 7ff865ec9640 1 -- start start 2023-11-08T19:27:40.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff865ec9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 0x7ff86015ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff865ec9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 0x7ff8600c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff865ec9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff86015e5f0 0x7ff8601f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff865ec9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8600c6750 con 0x7ff86015d410 2023-11-08T19:27:40.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff865ec9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8600c6400 con 0x7ff86015e5f0 2023-11-08T19:27:40.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff865ec9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff86015f380 con 0x7ff86015dd30 2023-11-08T19:27:40.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 0x7ff86015ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 0x7ff86015ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54804/0 (socket says 172.21.15.119:54804) 2023-11-08T19:27:40.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 -- 172.21.15.119:0/3893440585 learned_addr learned my addr 172.21.15.119:0/3893440585 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:40.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85ffff640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff86015e5f0 0x7ff8601f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 -- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 msgr2=0x7ff8600c4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 0x7ff8600c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 -- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff86015e5f0 msgr2=0x7ff8601f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:40.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff86015e5f0 0x7ff8601f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:40.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.894+0000 7ff85f7fe640 1 -- 172.21.15.119:0/3893440585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff854009400 con 0x7ff86015d410 2023-11-08T19:27:40.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.895+0000 7ff85f7fe640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 0x7ff86015ea10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff8500010e0 tx=0x7ff85000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.895+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850007480 con 0x7ff86015d410 2023-11-08T19:27:40.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.895+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8500075e0 con 0x7ff86015d410 2023-11-08T19:27:40.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.895+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8500059c0 con 0x7ff86015d410 2023-11-08T19:27:40.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.895+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff86015f660 con 0x7ff86015d410 2023-11-08T19:27:40.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.895+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff86015ef50 con 0x7ff86015d410 2023-11-08T19:27:40.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.896+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff86015f140 con 0x7ff86015d410 2023-11-08T19:27:40.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.897+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff850005b20 con 0x7ff86015d410 2023-11-08T19:27:40.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.898+0000 7ff847fff640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff84c06aea0 0x7ff84c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:40.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.899+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff850088e70 con 0x7ff86015d410 2023-11-08T19:27:40.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.899+0000 7ff85effd640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff84c06aea0 0x7ff84c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:40.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.899+0000 7ff85effd640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff84c06aea0 0x7ff84c06d360 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff854036000 tx=0x7ff854030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:40.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:40.903+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff86015f140 con 0x7ff86015d410 2023-11-08T19:27:41.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.087+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8600b00d0 con 0x7ff84c06aea0 2023-11-08T19:27:41.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.114+0000 7ff847fff640 1 -- 172.21.15.119:0/3893440585 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368173 (secure 0 0 0) 0x7ff8600b00d0 con 0x7ff84c06aea0 2023-11-08T19:27:41.123 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:41.125 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:27:41.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.120+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff84c06aea0 msgr2=0x7ff84c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.120+0000 7ff865ec9640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff84c06aea0 0x7ff84c06d360 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff854036000 tx=0x7ff854030000 comp rx=0 tx=0).stop 2023-11-08T19:27:41.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.120+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 msgr2=0x7ff86015ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.120+0000 7ff865ec9640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 0x7ff86015ea10 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff8500010e0 tx=0x7ff85000c9b0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.120+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 shutdown_connections 2023-11-08T19:27:41.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.120+0000 7ff865ec9640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff84c06aea0 0x7ff84c06d360 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.121+0000 7ff865ec9640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff86015e5f0 0x7ff8601f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.121+0000 7ff865ec9640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff86015dd30 0x7ff8600c4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.121+0000 7ff865ec9640 1 --2- 172.21.15.119:0/3893440585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff86015d410 0x7ff86015ea10 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.121+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 >> 172.21.15.119:0/3893440585 conn(0x7ff8600d0db0 msgr2=0x7ff86006ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:41.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.121+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 shutdown_connections 2023-11-08T19:27:41.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.121+0000 7ff865ec9640 1 -- 172.21.15.119:0/3893440585 wait complete. 2023-11-08T19:27:41.141 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-08T19:27:40.727429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231592,"kb_used_data":7104,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743128,"statfs":{"total":773094113280,"available":772856963072,"internally_reserved":0,"allocated":7274496,"data_stored":3983338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.062603"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723901+0000","last_change":"2023-11-08T19:27:36.716672+0000","last_active":"2023-11-08T19:27:36.723901+0000","last_peered":"2023-11-08T19:27:36.723901+0000","last_clean":"2023-11-08T19:27:36.723901+0000","last_became_active":"2023-11-08T19:27:36.716111+0000","last_became_peered":"2023-11-08T19:27:36.716111+0000","last_unstale":"2023-11-08T19:27:36.723901+0000","last_undegraded":"2023-11-08T19:27:36.723901+0000","last_fullsized":"2023-11-08T19:27:36.723901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:39:05.587346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.715410+0000","last_change":"2023-11-08T19:27:36.709840+0000","last_active":"2023-11-08T19:27:36.715410+0000","last_peered":"2023-11-08T19:27:36.715410+0000","last_clean":"2023-11-08T19:27:36.715410+0000","last_became_active":"2023-11-08T19:27:36.709572+0000","last_became_peered":"2023-11-08T19:27:36.709572+0000","last_unstale":"2023-11-08T19:27:36.715410+0000","last_undegraded":"2023-11-08T19:27:36.715410+0000","last_fullsized":"2023-11-08T19:27:36.715410+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:19.093728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.702479+0000","last_change":"2023-11-08T19:27:36.698576+0000","last_active":"2023-11-08T19:27:36.702479+0000","last_peered":"2023-11-08T19:27:36.702479+0000","last_clean":"2023-11-08T19:27:36.702479+0000","last_became_active":"2023-11-08T19:27:36.698300+0000","last_became_peered":"2023-11-08T19:27:36.698300+0000","last_unstale":"2023-11-08T19:27:36.702479+0000","last_undegraded":"2023-11-08T19:27:36.702479+0000","last_fullsized":"2023-11-08T19:27:36.702479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:27.236861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707821+0000","last_change":"2023-11-08T19:27:36.706145+0000","last_active":"2023-11-08T19:27:36.707821+0000","last_peered":"2023-11-08T19:27:36.707821+0000","last_clean":"2023-11-08T19:27:36.707821+0000","last_became_active":"2023-11-08T19:27:36.705843+0000","last_became_peered":"2023-11-08T19:27:36.705843+0000","last_unstale":"2023-11-08T19:27:36.707821+0000","last_undegraded":"2023-11-08T19:27:36.707821+0000","last_fullsized":"2023-11-08T19:27:36.707821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:59.733745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.714874+0000","last_change":"2023-11-08T19:27:36.712170+0000","last_active":"2023-11-08T19:27:36.714874+0000","last_peered":"2023-11-08T19:27:36.714874+0000","last_clean":"2023-11-08T19:27:36.714874+0000","last_became_active":"2023-11-08T19:27:36.711786+0000","last_became_peered":"2023-11-08T19:27:36.711786+0000","last_unstale":"2023-11-08T19:27:36.714874+0000","last_undegraded":"2023-11-08T19:27:36.714874+0000","last_fullsized":"2023-11-08T19:27:36.714874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:59.157976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.711502+0000","last_change":"2023-11-08T19:27:36.707650+0000","last_active":"2023-11-08T19:27:36.711502+0000","last_peered":"2023-11-08T19:27:36.711502+0000","last_clean":"2023-11-08T19:27:36.711502+0000","last_became_active":"2023-11-08T19:27:36.707354+0000","last_became_peered":"2023-11-08T19:27:36.707354+0000","last_unstale":"2023-11-08T19:27:36.711502+0000","last_undegraded":"2023-11-08T19:27:36.711502+0000","last_fullsized":"2023-11-08T19:27:36.711502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:29.016613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707911+0000","last_change":"2023-11-08T19:27:36.704297+0000","last_active":"2023-11-08T19:27:36.707911+0000","last_peered":"2023-11-08T19:27:36.707911+0000","last_clean":"2023-11-08T19:27:36.707911+0000","last_became_active":"2023-11-08T19:27:36.701042+0000","last_became_peered":"2023-11-08T19:27:36.701042+0000","last_unstale":"2023-11-08T19:27:36.707911+0000","last_undegraded":"2023-11-08T19:27:36.707911+0000","last_fullsized":"2023-11-08T19:27:36.707911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:45.169002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708812+0000","last_change":"2023-11-08T19:27:36.705605+0000","last_active":"2023-11-08T19:27:36.708812+0000","last_peered":"2023-11-08T19:27:36.708812+0000","last_clean":"2023-11-08T19:27:36.708812+0000","last_became_active":"2023-11-08T19:27:36.705070+0000","last_became_peered":"2023-11-08T19:27:36.705070+0000","last_unstale":"2023-11-08T19:27:36.708812+0000","last_undegraded":"2023-11-08T19:27:36.708812+0000","last_fullsized":"2023-11-08T19:27:36.708812+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:32.005497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710639+0000","last_change":"2023-11-08T19:27:36.702917+0000","last_active":"2023-11-08T19:27:36.710639+0000","last_peered":"2023-11-08T19:27:36.710639+0000","last_clean":"2023-11-08T19:27:36.710639+0000","last_became_active":"2023-11-08T19:27:36.702439+0000","last_became_peered":"2023-11-08T19:27:36.702439+0000","last_unstale":"2023-11-08T19:27:36.710639+0000","last_undegraded":"2023-11-08T19:27:36.710639+0000","last_fullsized":"2023-11-08T19:27:36.710639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:59.274535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701764+0000","last_change":"2023-11-08T19:27:36.696574+0000","last_active":"2023-11-08T19:27:36.701764+0000","last_peered":"2023-11-08T19:27:36.701764+0000","last_clean":"2023-11-08T19:27:36.701764+0000","last_became_active":"2023-11-08T19:27:36.694474+0000","last_became_peered":"2023-11-08T19:27:36.694474+0000","last_unstale":"2023-11-08T19:27:36.701764+0000","last_undegraded":"2023-11-08T19:27:36.701764+0000","last_fullsized":"2023-11-08T19:27:36.701764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:14.476460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723261+0000","last_change":"2023-11-08T19:27:36.716582+0000","last_active":"2023-11-08T19:27:36.723261+0000","last_peered":"2023-11-08T19:27:36.723261+0000","last_clean":"2023-11-08T19:27:36.723261+0000","last_became_active":"2023-11-08T19:27:36.716285+0000","last_became_peered":"2023-11-08T19:27:36.716285+0000","last_unstale":"2023-11-08T19:27:36.723261+0000","last_undegraded":"2023-11-08T19:27:36.723261+0000","last_fullsized":"2023-11-08T19:27:36.723261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:52.328999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706937+0000","last_change":"2023-11-08T19:27:36.705159+0000","last_active":"2023-11-08T19:27:36.706937+0000","last_peered":"2023-11-08T19:27:36.706937+0000","last_clean":"2023-11-08T19:27:36.706937+0000","last_became_active":"2023-11-08T19:27:36.704848+0000","last_became_peered":"2023-11-08T19:27:36.704848+0000","last_unstale":"2023-11-08T19:27:36.706937+0000","last_undegraded":"2023-11-08T19:27:36.706937+0000","last_fullsized":"2023-11-08T19:27:36.706937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:59.543611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707886+0000","last_change":"2023-11-08T19:27:36.704388+0000","last_active":"2023-11-08T19:27:36.707886+0000","last_peered":"2023-11-08T19:27:36.707886+0000","last_clean":"2023-11-08T19:27:36.707886+0000","last_became_active":"2023-11-08T19:27:36.704118+0000","last_became_peered":"2023-11-08T19:27:36.704118+0000","last_unstale":"2023-11-08T19:27:36.707886+0000","last_undegraded":"2023-11-08T19:27:36.707886+0000","last_fullsized":"2023-11-08T19:27:36.707886+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:37.069165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701160+0000","last_change":"2023-11-08T19:27:36.693889+0000","last_active":"2023-11-08T19:27:36.701160+0000","last_peered":"2023-11-08T19:27:36.701160+0000","last_clean":"2023-11-08T19:27:36.701160+0000","last_became_active":"2023-11-08T19:27:36.693530+0000","last_became_peered":"2023-11-08T19:27:36.693530+0000","last_unstale":"2023-11-08T19:27:36.701160+0000","last_undegraded":"2023-11-08T19:27:36.701160+0000","last_fullsized":"2023-11-08T19:27:36.701160+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:36.417003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708104+0000","last_change":"2023-11-08T19:27:36.701647+0000","last_active":"2023-11-08T19:27:36.708104+0000","last_peered":"2023-11-08T19:27:36.708104+0000","last_clean":"2023-11-08T19:27:36.708104+0000","last_became_active":"2023-11-08T19:27:36.701390+0000","last_became_peered":"2023-11-08T19:27:36.701390+0000","last_unstale":"2023-11-08T19:27:36.708104+0000","last_undegraded":"2023-11-08T19:27:36.708104+0000","last_fullsized":"2023-11-08T19:27:36.708104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:06.293100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710758+0000","last_change":"2023-11-08T19:27:36.705645+0000","last_active":"2023-11-08T19:27:36.710758+0000","last_peered":"2023-11-08T19:27:36.710758+0000","last_clean":"2023-11-08T19:27:36.710758+0000","last_became_active":"2023-11-08T19:27:36.705355+0000","last_became_peered":"2023-11-08T19:27:36.705355+0000","last_unstale":"2023-11-08T19:27:36.710758+0000","last_undegraded":"2023-11-08T19:27:36.710758+0000","last_fullsized":"2023-11-08T19:27:36.710758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:09.489563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710621+0000","last_change":"2023-11-08T19:27:36.702691+0000","last_active":"2023-11-08T19:27:36.710621+0000","last_peered":"2023-11-08T19:27:36.710621+0000","last_clean":"2023-11-08T19:27:36.710621+0000","last_became_active":"2023-11-08T19:27:36.702077+0000","last_became_peered":"2023-11-08T19:27:36.702077+0000","last_unstale":"2023-11-08T19:27:36.710621+0000","last_undegraded":"2023-11-08T19:27:36.710621+0000","last_fullsized":"2023-11-08T19:27:36.710621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:33:19.377581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710799+0000","last_change":"2023-11-08T19:27:36.706954+0000","last_active":"2023-11-08T19:27:36.710799+0000","last_peered":"2023-11-08T19:27:36.710799+0000","last_clean":"2023-11-08T19:27:36.710799+0000","last_became_active":"2023-11-08T19:27:36.705938+0000","last_became_peered":"2023-11-08T19:27:36.705938+0000","last_unstale":"2023-11-08T19:27:36.710799+0000","last_undegraded":"2023-11-08T19:27:36.710799+0000","last_fullsized":"2023-11-08T19:27:36.710799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:37.541707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708743+0000","last_change":"2023-11-08T19:27:36.705629+0000","last_active":"2023-11-08T19:27:36.708743+0000","last_peered":"2023-11-08T19:27:36.708743+0000","last_clean":"2023-11-08T19:27:36.708743+0000","last_became_active":"2023-11-08T19:27:36.705287+0000","last_became_peered":"2023-11-08T19:27:36.705287+0000","last_unstale":"2023-11-08T19:27:36.708743+0000","last_undegraded":"2023-11-08T19:27:36.708743+0000","last_fullsized":"2023-11-08T19:27:36.708743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:32:14.030744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701804+0000","last_change":"2023-11-08T19:27:36.698208+0000","last_active":"2023-11-08T19:27:36.701804+0000","last_peered":"2023-11-08T19:27:36.701804+0000","last_clean":"2023-11-08T19:27:36.701804+0000","last_became_active":"2023-11-08T19:27:36.697880+0000","last_became_peered":"2023-11-08T19:27:36.697880+0000","last_unstale":"2023-11-08T19:27:36.701804+0000","last_undegraded":"2023-11-08T19:27:36.701804+0000","last_fullsized":"2023-11-08T19:27:36.701804+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:52.328999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.702157+0000","last_change":"2023-11-08T19:27:36.697725+0000","last_active":"2023-11-08T19:27:36.702157+0000","last_peered":"2023-11-08T19:27:36.702157+0000","last_clean":"2023-11-08T19:27:36.702157+0000","last_became_active":"2023-11-08T19:27:36.697431+0000","last_became_peered":"2023-11-08T19:27:36.697431+0000","last_unstale":"2023-11-08T19:27:36.702157+0000","last_undegraded":"2023-11-08T19:27:36.702157+0000","last_fullsized":"2023-11-08T19:27:36.702157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:34.738158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.703783+0000","last_change":"2023-11-08T19:27:36.694772+0000","last_active":"2023-11-08T19:27:36.703783+0000","last_peered":"2023-11-08T19:27:36.703783+0000","last_clean":"2023-11-08T19:27:36.703783+0000","last_became_active":"2023-11-08T19:27:36.694463+0000","last_became_peered":"2023-11-08T19:27:36.694463+0000","last_unstale":"2023-11-08T19:27:36.703783+0000","last_undegraded":"2023-11-08T19:27:36.703783+0000","last_fullsized":"2023-11-08T19:27:36.703783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:29.380001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.714192+0000","last_change":"2023-11-08T19:27:36.710127+0000","last_active":"2023-11-08T19:27:36.714192+0000","last_peered":"2023-11-08T19:27:36.714192+0000","last_clean":"2023-11-08T19:27:36.714192+0000","last_became_active":"2023-11-08T19:27:36.709802+0000","last_became_peered":"2023-11-08T19:27:36.709802+0000","last_unstale":"2023-11-08T19:27:36.714192+0000","last_undegraded":"2023-11-08T19:27:36.714192+0000","last_fullsized":"2023-11-08T19:27:36.714192+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:11.860449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725140+0000","last_change":"2023-11-08T19:27:36.716966+0000","last_active":"2023-11-08T19:27:36.725140+0000","last_peered":"2023-11-08T19:27:36.725140+0000","last_clean":"2023-11-08T19:27:36.725140+0000","last_became_active":"2023-11-08T19:27:36.716636+0000","last_became_peered":"2023-11-08T19:27:36.716636+0000","last_unstale":"2023-11-08T19:27:36.725140+0000","last_undegraded":"2023-11-08T19:27:36.725140+0000","last_fullsized":"2023-11-08T19:27:36.725140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:38.342781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.720472+0000","last_change":"2023-11-08T19:27:36.710658+0000","last_active":"2023-11-08T19:27:36.720472+0000","last_peered":"2023-11-08T19:27:36.720472+0000","last_clean":"2023-11-08T19:27:36.720472+0000","last_became_active":"2023-11-08T19:27:36.710370+0000","last_became_peered":"2023-11-08T19:27:36.710370+0000","last_unstale":"2023-11-08T19:27:36.720472+0000","last_undegraded":"2023-11-08T19:27:36.720472+0000","last_fullsized":"2023-11-08T19:27:36.720472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:08.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708059+0000","last_change":"2023-11-08T19:27:36.704975+0000","last_active":"2023-11-08T19:27:36.708059+0000","last_peered":"2023-11-08T19:27:36.708059+0000","last_clean":"2023-11-08T19:27:36.708059+0000","last_became_active":"2023-11-08T19:27:36.704689+0000","last_became_peered":"2023-11-08T19:27:36.704689+0000","last_unstale":"2023-11-08T19:27:36.708059+0000","last_undegraded":"2023-11-08T19:27:36.708059+0000","last_fullsized":"2023-11-08T19:27:36.708059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:07.085248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.714389+0000","last_change":"2023-11-08T19:27:36.712698+0000","last_active":"2023-11-08T19:27:36.714389+0000","last_peered":"2023-11-08T19:27:36.714389+0000","last_clean":"2023-11-08T19:27:36.714389+0000","last_became_active":"2023-11-08T19:27:36.712363+0000","last_became_peered":"2023-11-08T19:27:36.712363+0000","last_unstale":"2023-11-08T19:27:36.714389+0000","last_undegraded":"2023-11-08T19:27:36.714389+0000","last_fullsized":"2023-11-08T19:27:36.714389+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:55.145043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706917+0000","last_change":"2023-11-08T19:27:36.696146+0000","last_active":"2023-11-08T19:27:36.706917+0000","last_peered":"2023-11-08T19:27:36.706917+0000","last_clean":"2023-11-08T19:27:36.706917+0000","last_became_active":"2023-11-08T19:27:36.695868+0000","last_became_peered":"2023-11-08T19:27:36.695868+0000","last_unstale":"2023-11-08T19:27:36.706917+0000","last_undegraded":"2023-11-08T19:27:36.706917+0000","last_fullsized":"2023-11-08T19:27:36.706917+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:35.672402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.696023+0000","last_change":"2023-11-08T19:27:36.691121+0000","last_active":"2023-11-08T19:27:36.696023+0000","last_peered":"2023-11-08T19:27:36.696023+0000","last_clean":"2023-11-08T19:27:36.696023+0000","last_became_active":"2023-11-08T19:27:36.686832+0000","last_became_peered":"2023-11-08T19:27:36.686832+0000","last_unstale":"2023-11-08T19:27:36.696023+0000","last_undegraded":"2023-11-08T19:27:36.696023+0000","last_fullsized":"2023-11-08T19:27:36.696023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:09.409597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707593+0000","last_change":"2023-11-08T19:27:36.702872+0000","last_active":"2023-11-08T19:27:36.707593+0000","last_peered":"2023-11-08T19:27:36.707593+0000","last_clean":"2023-11-08T19:27:36.707593+0000","last_became_active":"2023-11-08T19:27:36.702478+0000","last_became_peered":"2023-11-08T19:27:36.702478+0000","last_unstale":"2023-11-08T19:27:36.707593+0000","last_undegraded":"2023-11-08T19:27:36.707593+0000","last_fullsized":"2023-11-08T19:27:36.707593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:52.328999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.703769+0000","last_change":"2023-11-08T19:27:36.694811+0000","last_active":"2023-11-08T19:27:36.703769+0000","last_peered":"2023-11-08T19:27:36.703769+0000","last_clean":"2023-11-08T19:27:36.703769+0000","last_became_active":"2023-11-08T19:27:36.694473+0000","last_became_peered":"2023-11-08T19:27:36.694473+0000","last_unstale":"2023-11-08T19:27:36.703769+0000","last_undegraded":"2023-11-08T19:27:36.703769+0000","last_fullsized":"2023-11-08T19:27:36.703769+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:28.240261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708736+0000","last_change":"2023-11-08T19:27:36.704105+0000","last_active":"2023-11-08T19:27:36.708736+0000","last_peered":"2023-11-08T19:27:36.708736+0000","last_clean":"2023-11-08T19:27:36.708736+0000","last_became_active":"2023-11-08T19:27:36.703770+0000","last_became_peered":"2023-11-08T19:27:36.703770+0000","last_unstale":"2023-11-08T19:27:36.708736+0000","last_undegraded":"2023-11-08T19:27:36.708736+0000","last_fullsized":"2023-11-08T19:27:36.708736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:39.274464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.705590+0000","last_change":"2023-11-08T19:27:36.697782+0000","last_active":"2023-11-08T19:27:36.705590+0000","last_peered":"2023-11-08T19:27:36.705590+0000","last_clean":"2023-11-08T19:27:36.705590+0000","last_became_active":"2023-11-08T19:27:36.697278+0000","last_became_peered":"2023-11-08T19:27:36.697278+0000","last_unstale":"2023-11-08T19:27:36.705590+0000","last_undegraded":"2023-11-08T19:27:36.705590+0000","last_fullsized":"2023-11-08T19:27:36.705590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:52:54.238604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"64'8","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726463+0000","last_change":"2023-11-08T19:27:35.700276+0000","last_active":"2023-11-08T19:27:36.726463+0000","last_peered":"2023-11-08T19:27:36.726463+0000","last_clean":"2023-11-08T19:27:36.726463+0000","last_became_active":"2023-11-08T19:27:35.700002+0000","last_became_peered":"2023-11-08T19:27:35.700002+0000","last_unstale":"2023-11-08T19:27:36.726463+0000","last_undegraded":"2023-11-08T19:27:36.726463+0000","last_fullsized":"2023-11-08T19:27:36.726463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:18.733671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.672777+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:27:35.672777+0000","last_peered":"2023-11-08T19:27:35.672777+0000","last_clean":"2023-11-08T19:27:35.672777+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:27:35.672777+0000","last_undegraded":"2023-11-08T19:27:35.672777+0000","last_fullsized":"2023-11-08T19:27:35.672777+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.713136+0000","last_change":"2023-11-08T19:27:36.709698+0000","last_active":"2023-11-08T19:27:36.713136+0000","last_peered":"2023-11-08T19:27:36.713136+0000","last_clean":"2023-11-08T19:27:36.713136+0000","last_became_active":"2023-11-08T19:27:36.709423+0000","last_became_peered":"2023-11-08T19:27:36.709423+0000","last_unstale":"2023-11-08T19:27:36.713136+0000","last_undegraded":"2023-11-08T19:27:36.713136+0000","last_fullsized":"2023-11-08T19:27:36.713136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:39:05.587346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.677366+0000","last_change":"2023-11-08T19:27:35.698052+0000","last_active":"2023-11-08T19:27:36.677366+0000","last_peered":"2023-11-08T19:27:36.677366+0000","last_clean":"2023-11-08T19:27:36.677366+0000","last_became_active":"2023-11-08T19:27:35.697671+0000","last_became_peered":"2023-11-08T19:27:35.697671+0000","last_unstale":"2023-11-08T19:27:36.677366+0000","last_undegraded":"2023-11-08T19:27:36.677366+0000","last_fullsized":"2023-11-08T19:27:36.677366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:10.680560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.699222+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:27:35.699222+0000","last_peered":"2023-11-08T19:27:35.699222+0000","last_clean":"2023-11-08T19:27:35.699222+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:27:35.699222+0000","last_undegraded":"2023-11-08T19:27:35.699222+0000","last_fullsized":"2023-11-08T19:27:35.699222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.697376+0000","last_change":"2023-11-08T19:27:36.687160+0000","last_active":"2023-11-08T19:27:36.697376+0000","last_peered":"2023-11-08T19:27:36.697376+0000","last_clean":"2023-11-08T19:27:36.697376+0000","last_became_active":"2023-11-08T19:27:36.686463+0000","last_became_peered":"2023-11-08T19:27:36.686463+0000","last_unstale":"2023-11-08T19:27:36.697376+0000","last_undegraded":"2023-11-08T19:27:36.697376+0000","last_fullsized":"2023-11-08T19:27:36.697376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:43.218285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.689763+0000","last_change":"2023-11-08T19:27:35.703028+0000","last_active":"2023-11-08T19:27:36.689763+0000","last_peered":"2023-11-08T19:27:36.689763+0000","last_clean":"2023-11-08T19:27:36.689763+0000","last_became_active":"2023-11-08T19:27:35.702765+0000","last_became_peered":"2023-11-08T19:27:35.702765+0000","last_unstale":"2023-11-08T19:27:36.689763+0000","last_undegraded":"2023-11-08T19:27:36.689763+0000","last_fullsized":"2023-11-08T19:27:36.689763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:03.864531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690306+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:27:35.690306+0000","last_peered":"2023-11-08T19:27:35.690306+0000","last_clean":"2023-11-08T19:27:35.690306+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:27:35.690306+0000","last_undegraded":"2023-11-08T19:27:35.690306+0000","last_fullsized":"2023-11-08T19:27:35.690306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723506+0000","last_change":"2023-11-08T19:27:36.718152+0000","last_active":"2023-11-08T19:27:36.723506+0000","last_peered":"2023-11-08T19:27:36.723506+0000","last_clean":"2023-11-08T19:27:36.723506+0000","last_became_active":"2023-11-08T19:27:36.717884+0000","last_became_peered":"2023-11-08T19:27:36.717884+0000","last_unstale":"2023-11-08T19:27:36.723506+0000","last_undegraded":"2023-11-08T19:27:36.723506+0000","last_fullsized":"2023-11-08T19:27:36.723506+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:45.169002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.728117+0000","last_change":"2023-11-08T19:27:35.691767+0000","last_active":"2023-11-08T19:27:36.728117+0000","last_peered":"2023-11-08T19:27:36.728117+0000","last_clean":"2023-11-08T19:27:36.728117+0000","last_became_active":"2023-11-08T19:27:35.691274+0000","last_became_peered":"2023-11-08T19:27:35.691274+0000","last_unstale":"2023-11-08T19:27:36.728117+0000","last_undegraded":"2023-11-08T19:27:36.728117+0000","last_fullsized":"2023-11-08T19:27:36.728117+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:33.725280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.672696+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:27:35.672696+0000","last_peered":"2023-11-08T19:27:35.672696+0000","last_clean":"2023-11-08T19:27:35.672696+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:27:35.672696+0000","last_undegraded":"2023-11-08T19:27:35.672696+0000","last_fullsized":"2023-11-08T19:27:35.672696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.713222+0000","last_change":"2023-11-08T19:27:36.705366+0000","last_active":"2023-11-08T19:27:36.713222+0000","last_peered":"2023-11-08T19:27:36.713222+0000","last_clean":"2023-11-08T19:27:36.713222+0000","last_became_active":"2023-11-08T19:27:36.701244+0000","last_became_peered":"2023-11-08T19:27:36.701244+0000","last_unstale":"2023-11-08T19:27:36.713222+0000","last_undegraded":"2023-11-08T19:27:36.713222+0000","last_fullsized":"2023-11-08T19:27:36.713222+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:01.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"64'6","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.728657+0000","last_change":"2023-11-08T19:27:35.688971+0000","last_active":"2023-11-08T19:27:36.728657+0000","last_peered":"2023-11-08T19:27:36.728657+0000","last_clean":"2023-11-08T19:27:36.728657+0000","last_became_active":"2023-11-08T19:27:35.688242+0000","last_became_peered":"2023-11-08T19:27:35.688242+0000","last_unstale":"2023-11-08T19:27:36.728657+0000","last_undegraded":"2023-11-08T19:27:36.728657+0000","last_fullsized":"2023-11-08T19:27:36.728657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:48:39.655471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.685131+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:27:35.685131+0000","last_peered":"2023-11-08T19:27:35.685131+0000","last_clean":"2023-11-08T19:27:35.685131+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:27:35.685131+0000","last_undegraded":"2023-11-08T19:27:35.685131+0000","last_fullsized":"2023-11-08T19:27:35.685131+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.709229+0000","last_change":"2023-11-08T19:27:36.706871+0000","last_active":"2023-11-08T19:27:36.709229+0000","last_peered":"2023-11-08T19:27:36.709229+0000","last_clean":"2023-11-08T19:27:36.709229+0000","last_became_active":"2023-11-08T19:27:36.706611+0000","last_became_peered":"2023-11-08T19:27:36.706611+0000","last_unstale":"2023-11-08T19:27:36.709229+0000","last_undegraded":"2023-11-08T19:27:36.709229+0000","last_fullsized":"2023-11-08T19:27:36.709229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:27.547156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"64'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725227+0000","last_change":"2023-11-08T19:27:35.699732+0000","last_active":"2023-11-08T19:27:36.725227+0000","last_peered":"2023-11-08T19:27:36.725227+0000","last_clean":"2023-11-08T19:27:36.725227+0000","last_became_active":"2023-11-08T19:27:35.699247+0000","last_became_peered":"2023-11-08T19:27:35.699247+0000","last_unstale":"2023-11-08T19:27:36.725227+0000","last_undegraded":"2023-11-08T19:27:36.725227+0000","last_fullsized":"2023-11-08T19:27:36.725227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:28.926193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.682074+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:27:35.682074+0000","last_peered":"2023-11-08T19:27:35.682074+0000","last_clean":"2023-11-08T19:27:35.682074+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:27:35.682074+0000","last_undegraded":"2023-11-08T19:27:35.682074+0000","last_fullsized":"2023-11-08T19:27:35.682074+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.727992+0000","last_change":"2023-11-08T19:27:36.723911+0000","last_active":"2023-11-08T19:27:36.727992+0000","last_peered":"2023-11-08T19:27:36.727992+0000","last_clean":"2023-11-08T19:27:36.727992+0000","last_became_active":"2023-11-08T19:27:36.722714+0000","last_became_peered":"2023-11-08T19:27:36.722714+0000","last_unstale":"2023-11-08T19:27:36.727992+0000","last_undegraded":"2023-11-08T19:27:36.727992+0000","last_fullsized":"2023-11-08T19:27:36.727992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:07.085248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.688955+0000","last_change":"2023-11-08T19:27:35.701294+0000","last_active":"2023-11-08T19:27:36.688955+0000","last_peered":"2023-11-08T19:27:36.688955+0000","last_clean":"2023-11-08T19:27:36.688955+0000","last_became_active":"2023-11-08T19:27:35.700929+0000","last_became_peered":"2023-11-08T19:27:35.700929+0000","last_unstale":"2023-11-08T19:27:36.688955+0000","last_undegraded":"2023-11-08T19:27:36.688955+0000","last_fullsized":"2023-11-08T19:27:36.688955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:48.626834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683207+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:27:35.683207+0000","last_peered":"2023-11-08T19:27:35.683207+0000","last_clean":"2023-11-08T19:27:35.683207+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:27:35.683207+0000","last_undegraded":"2023-11-08T19:27:35.683207+0000","last_fullsized":"2023-11-08T19:27:35.683207+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.715640+0000","last_change":"2023-11-08T19:27:36.712174+0000","last_active":"2023-11-08T19:27:36.715640+0000","last_peered":"2023-11-08T19:27:36.715640+0000","last_clean":"2023-11-08T19:27:36.715640+0000","last_became_active":"2023-11-08T19:27:36.711595+0000","last_became_peered":"2023-11-08T19:27:36.711595+0000","last_unstale":"2023-11-08T19:27:36.715640+0000","last_undegraded":"2023-11-08T19:27:36.715640+0000","last_fullsized":"2023-11-08T19:27:36.715640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:39:05.587346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.681264+0000","last_change":"2023-11-08T19:27:35.699287+0000","last_active":"2023-11-08T19:27:36.681264+0000","last_peered":"2023-11-08T19:27:36.681264+0000","last_clean":"2023-11-08T19:27:36.681264+0000","last_became_active":"2023-11-08T19:27:35.698905+0000","last_became_peered":"2023-11-08T19:27:35.698905+0000","last_unstale":"2023-11-08T19:27:36.681264+0000","last_undegraded":"2023-11-08T19:27:36.681264+0000","last_fullsized":"2023-11-08T19:27:36.681264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:53.718272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":37,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.695797+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:27:35.695797+0000","last_peered":"2023-11-08T19:27:35.695797+0000","last_clean":"2023-11-08T19:27:35.695797+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:27:35.695797+0000","last_undegraded":"2023-11-08T19:27:35.695797+0000","last_fullsized":"2023-11-08T19:27:35.695797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.717038+0000","last_change":"2023-11-08T19:27:36.713266+0000","last_active":"2023-11-08T19:27:36.717038+0000","last_peered":"2023-11-08T19:27:36.717038+0000","last_clean":"2023-11-08T19:27:36.717038+0000","last_became_active":"2023-11-08T19:27:36.711706+0000","last_became_peered":"2023-11-08T19:27:36.711706+0000","last_unstale":"2023-11-08T19:27:36.717038+0000","last_undegraded":"2023-11-08T19:27:36.717038+0000","last_fullsized":"2023-11-08T19:27:36.717038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:11.686976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.680978+0000","last_change":"2023-11-08T19:27:35.684433+0000","last_active":"2023-11-08T19:27:36.680978+0000","last_peered":"2023-11-08T19:27:36.680978+0000","last_clean":"2023-11-08T19:27:36.680978+0000","last_became_active":"2023-11-08T19:27:35.683972+0000","last_became_peered":"2023-11-08T19:27:35.683972+0000","last_unstale":"2023-11-08T19:27:36.680978+0000","last_undegraded":"2023-11-08T19:27:36.680978+0000","last_fullsized":"2023-11-08T19:27:36.680978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:55.257234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.692555+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:27:35.692555+0000","last_peered":"2023-11-08T19:27:35.692555+0000","last_clean":"2023-11-08T19:27:35.692555+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:27:35.692555+0000","last_undegraded":"2023-11-08T19:27:35.692555+0000","last_fullsized":"2023-11-08T19:27:35.692555+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710665+0000","last_change":"2023-11-08T19:27:36.701688+0000","last_active":"2023-11-08T19:27:36.710665+0000","last_peered":"2023-11-08T19:27:36.710665+0000","last_clean":"2023-11-08T19:27:36.710665+0000","last_became_active":"2023-11-08T19:27:36.700932+0000","last_became_peered":"2023-11-08T19:27:36.700932+0000","last_unstale":"2023-11-08T19:27:36.710665+0000","last_undegraded":"2023-11-08T19:27:36.710665+0000","last_fullsized":"2023-11-08T19:27:36.710665+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:22.735243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.685511+0000","last_change":"2023-11-08T19:27:35.698924+0000","last_active":"2023-11-08T19:27:36.685511+0000","last_peered":"2023-11-08T19:27:36.685511+0000","last_clean":"2023-11-08T19:27:36.685511+0000","last_became_active":"2023-11-08T19:27:35.698598+0000","last_became_peered":"2023-11-08T19:27:35.698598+0000","last_unstale":"2023-11-08T19:27:36.685511+0000","last_undegraded":"2023-11-08T19:27:36.685511+0000","last_fullsized":"2023-11-08T19:27:36.685511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:12.271555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.684086+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:27:35.684086+0000","last_peered":"2023-11-08T19:27:35.684086+0000","last_clean":"2023-11-08T19:27:35.684086+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:27:35.684086+0000","last_undegraded":"2023-11-08T19:27:35.684086+0000","last_fullsized":"2023-11-08T19:27:35.684086+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.720811+0000","last_change":"2023-11-08T19:27:36.716610+0000","last_active":"2023-11-08T19:27:36.720811+0000","last_peered":"2023-11-08T19:27:36.720811+0000","last_clean":"2023-11-08T19:27:36.720811+0000","last_became_active":"2023-11-08T19:27:36.710960+0000","last_became_peered":"2023-11-08T19:27:36.710960+0000","last_unstale":"2023-11-08T19:27:36.720811+0000","last_undegraded":"2023-11-08T19:27:36.720811+0000","last_fullsized":"2023-11-08T19:27:36.720811+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:48:40.661888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.724281+0000","last_change":"2023-11-08T19:27:35.691410+0000","last_active":"2023-11-08T19:27:36.724281+0000","last_peered":"2023-11-08T19:27:36.724281+0000","last_clean":"2023-11-08T19:27:36.724281+0000","last_became_active":"2023-11-08T19:27:35.686591+0000","last_became_peered":"2023-11-08T19:27:35.686591+0000","last_unstale":"2023-11-08T19:27:36.724281+0000","last_undegraded":"2023-11-08T19:27:36.724281+0000","last_fullsized":"2023-11-08T19:27:36.724281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:58.537194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.694242+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:27:35.694242+0000","last_peered":"2023-11-08T19:27:35.694242+0000","last_clean":"2023-11-08T19:27:35.694242+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:27:35.694242+0000","last_undegraded":"2023-11-08T19:27:35.694242+0000","last_fullsized":"2023-11-08T19:27:35.694242+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.702961+0000","last_change":"2023-11-08T19:27:36.700688+0000","last_active":"2023-11-08T19:27:36.702961+0000","last_peered":"2023-11-08T19:27:36.702961+0000","last_clean":"2023-11-08T19:27:36.702961+0000","last_became_active":"2023-11-08T19:27:36.698691+0000","last_became_peered":"2023-11-08T19:27:36.698691+0000","last_unstale":"2023-11-08T19:27:36.702961+0000","last_undegraded":"2023-11-08T19:27:36.702961+0000","last_fullsized":"2023-11-08T19:27:36.702961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:38.342781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723217+0000","last_change":"2023-11-08T19:27:35.702083+0000","last_active":"2023-11-08T19:27:36.723217+0000","last_peered":"2023-11-08T19:27:36.723217+0000","last_clean":"2023-11-08T19:27:36.723217+0000","last_became_active":"2023-11-08T19:27:35.701931+0000","last_became_peered":"2023-11-08T19:27:35.701931+0000","last_unstale":"2023-11-08T19:27:36.723217+0000","last_undegraded":"2023-11-08T19:27:36.723217+0000","last_fullsized":"2023-11-08T19:27:36.723217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:56.851993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":37,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.695624+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:27:35.695624+0000","last_peered":"2023-11-08T19:27:35.695624+0000","last_clean":"2023-11-08T19:27:35.695624+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:27:35.695624+0000","last_undegraded":"2023-11-08T19:27:35.695624+0000","last_fullsized":"2023-11-08T19:27:35.695624+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.721874+0000","last_change":"2023-11-08T19:27:36.714283+0000","last_active":"2023-11-08T19:27:36.721874+0000","last_peered":"2023-11-08T19:27:36.721874+0000","last_clean":"2023-11-08T19:27:36.721874+0000","last_became_active":"2023-11-08T19:27:36.713973+0000","last_became_peered":"2023-11-08T19:27:36.713973+0000","last_unstale":"2023-11-08T19:27:36.721874+0000","last_undegraded":"2023-11-08T19:27:36.721874+0000","last_fullsized":"2023-11-08T19:27:36.721874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:30.189699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726778+0000","last_change":"2023-11-08T19:27:35.701825+0000","last_active":"2023-11-08T19:27:36.726778+0000","last_peered":"2023-11-08T19:27:36.726778+0000","last_clean":"2023-11-08T19:27:36.726778+0000","last_became_active":"2023-11-08T19:27:35.698585+0000","last_became_peered":"2023-11-08T19:27:35.698585+0000","last_unstale":"2023-11-08T19:27:36.726778+0000","last_undegraded":"2023-11-08T19:27:36.726778+0000","last_fullsized":"2023-11-08T19:27:36.726778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:28.373585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690469+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:27:35.690469+0000","last_peered":"2023-11-08T19:27:35.690469+0000","last_clean":"2023-11-08T19:27:35.690469+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:27:35.690469+0000","last_undegraded":"2023-11-08T19:27:35.690469+0000","last_fullsized":"2023-11-08T19:27:35.690469+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.720262+0000","last_change":"2023-11-08T19:27:36.712235+0000","last_active":"2023-11-08T19:27:36.720262+0000","last_peered":"2023-11-08T19:27:36.720262+0000","last_clean":"2023-11-08T19:27:36.720262+0000","last_became_active":"2023-11-08T19:27:36.711918+0000","last_became_peered":"2023-11-08T19:27:36.711918+0000","last_unstale":"2023-11-08T19:27:36.720262+0000","last_undegraded":"2023-11-08T19:27:36.720262+0000","last_fullsized":"2023-11-08T19:27:36.720262+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:34.731696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726328+0000","last_change":"2023-11-08T19:27:35.703104+0000","last_active":"2023-11-08T19:27:36.726328+0000","last_peered":"2023-11-08T19:27:36.726328+0000","last_clean":"2023-11-08T19:27:36.726328+0000","last_became_active":"2023-11-08T19:27:35.702369+0000","last_became_peered":"2023-11-08T19:27:35.702369+0000","last_unstale":"2023-11-08T19:27:36.726328+0000","last_undegraded":"2023-11-08T19:27:36.726328+0000","last_fullsized":"2023-11-08T19:27:36.726328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:23.275569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683506+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:27:35.683506+0000","last_peered":"2023-11-08T19:27:35.683506+0000","last_clean":"2023-11-08T19:27:35.683506+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:27:35.683506+0000","last_undegraded":"2023-11-08T19:27:35.683506+0000","last_fullsized":"2023-11-08T19:27:35.683506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706901+0000","last_change":"2023-11-08T19:27:36.698650+0000","last_active":"2023-11-08T19:27:36.706901+0000","last_peered":"2023-11-08T19:27:36.706901+0000","last_clean":"2023-11-08T19:27:36.706901+0000","last_became_active":"2023-11-08T19:27:36.698399+0000","last_became_peered":"2023-11-08T19:27:36.698399+0000","last_unstale":"2023-11-08T19:27:36.706901+0000","last_undegraded":"2023-11-08T19:27:36.706901+0000","last_fullsized":"2023-11-08T19:27:36.706901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:46.100687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.724731+0000","last_change":"2023-11-08T19:27:35.689917+0000","last_active":"2023-11-08T19:27:36.724731+0000","last_peered":"2023-11-08T19:27:36.724731+0000","last_clean":"2023-11-08T19:27:36.724731+0000","last_became_active":"2023-11-08T19:27:35.689120+0000","last_became_peered":"2023-11-08T19:27:35.689120+0000","last_unstale":"2023-11-08T19:27:36.724731+0000","last_undegraded":"2023-11-08T19:27:36.724731+0000","last_fullsized":"2023-11-08T19:27:36.724731+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:29.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683316+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:27:35.683316+0000","last_peered":"2023-11-08T19:27:35.683316+0000","last_clean":"2023-11-08T19:27:35.683316+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:27:35.683316+0000","last_undegraded":"2023-11-08T19:27:35.683316+0000","last_fullsized":"2023-11-08T19:27:35.683316+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.727793+0000","last_change":"2023-11-08T19:27:36.722335+0000","last_active":"2023-11-08T19:27:36.727793+0000","last_peered":"2023-11-08T19:27:36.727793+0000","last_clean":"2023-11-08T19:27:36.727793+0000","last_became_active":"2023-11-08T19:27:36.721130+0000","last_became_peered":"2023-11-08T19:27:36.721130+0000","last_unstale":"2023-11-08T19:27:36.727793+0000","last_undegraded":"2023-11-08T19:27:36.727793+0000","last_fullsized":"2023-11-08T19:27:36.727793+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:45.468959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"64'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.728398+0000","last_change":"2023-11-08T19:27:35.703330+0000","last_active":"2023-11-08T19:27:36.728398+0000","last_peered":"2023-11-08T19:27:36.728398+0000","last_clean":"2023-11-08T19:27:36.728398+0000","last_became_active":"2023-11-08T19:27:35.701922+0000","last_became_peered":"2023-11-08T19:27:35.701922+0000","last_unstale":"2023-11-08T19:27:36.728398+0000","last_undegraded":"2023-11-08T19:27:36.728398+0000","last_fullsized":"2023-11-08T19:27:36.728398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:42.211869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.689896+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:27:35.689896+0000","last_peered":"2023-11-08T19:27:35.689896+0000","last_clean":"2023-11-08T19:27:35.689896+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:27:35.689896+0000","last_undegraded":"2023-11-08T19:27:35.689896+0000","last_fullsized":"2023-11-08T19:27:35.689896+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708167+0000","last_change":"2023-11-08T19:27:36.703458+0000","last_active":"2023-11-08T19:27:36.708167+0000","last_peered":"2023-11-08T19:27:36.708167+0000","last_clean":"2023-11-08T19:27:36.708167+0000","last_became_active":"2023-11-08T19:27:36.702815+0000","last_became_peered":"2023-11-08T19:27:36.702815+0000","last_unstale":"2023-11-08T19:27:36.708167+0000","last_undegraded":"2023-11-08T19:27:36.708167+0000","last_fullsized":"2023-11-08T19:27:36.708167+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:48:40.661888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.690762+0000","last_change":"2023-11-08T19:27:35.694565+0000","last_active":"2023-11-08T19:27:36.690762+0000","last_peered":"2023-11-08T19:27:36.690762+0000","last_clean":"2023-11-08T19:27:36.690762+0000","last_became_active":"2023-11-08T19:27:35.694131+0000","last_became_peered":"2023-11-08T19:27:35.694131+0000","last_unstale":"2023-11-08T19:27:36.690762+0000","last_undegraded":"2023-11-08T19:27:36.690762+0000","last_fullsized":"2023-11-08T19:27:36.690762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:58.727329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690360+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:27:35.690360+0000","last_peered":"2023-11-08T19:27:35.690360+0000","last_clean":"2023-11-08T19:27:35.690360+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:27:35.690360+0000","last_undegraded":"2023-11-08T19:27:35.690360+0000","last_fullsized":"2023-11-08T19:27:35.690360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701812+0000","last_change":"2023-11-08T19:27:36.697531+0000","last_active":"2023-11-08T19:27:36.701812+0000","last_peered":"2023-11-08T19:27:36.701812+0000","last_clean":"2023-11-08T19:27:36.701812+0000","last_became_active":"2023-11-08T19:27:36.697220+0000","last_became_peered":"2023-11-08T19:27:36.697220+0000","last_unstale":"2023-11-08T19:27:36.701812+0000","last_undegraded":"2023-11-08T19:27:36.701812+0000","last_fullsized":"2023-11-08T19:27:36.701812+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:04.870947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.676019+0000","last_change":"2023-11-08T19:27:35.700686+0000","last_active":"2023-11-08T19:27:36.676019+0000","last_peered":"2023-11-08T19:27:36.676019+0000","last_clean":"2023-11-08T19:27:36.676019+0000","last_became_active":"2023-11-08T19:27:35.700348+0000","last_became_peered":"2023-11-08T19:27:35.700348+0000","last_unstale":"2023-11-08T19:27:36.676019+0000","last_undegraded":"2023-11-08T19:27:36.676019+0000","last_fullsized":"2023-11-08T19:27:36.676019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:58.727329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.687800+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:27:35.687800+0000","last_peered":"2023-11-08T19:27:35.687800+0000","last_clean":"2023-11-08T19:27:35.687800+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:27:35.687800+0000","last_undegraded":"2023-11-08T19:27:35.687800+0000","last_fullsized":"2023-11-08T19:27:35.687800+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.718690+0000","last_change":"2023-11-08T19:27:36.712627+0000","last_active":"2023-11-08T19:27:36.718690+0000","last_peered":"2023-11-08T19:27:36.718690+0000","last_clean":"2023-11-08T19:27:36.718690+0000","last_became_active":"2023-11-08T19:27:36.712267+0000","last_became_peered":"2023-11-08T19:27:36.712267+0000","last_unstale":"2023-11-08T19:27:36.718690+0000","last_undegraded":"2023-11-08T19:27:36.718690+0000","last_fullsized":"2023-11-08T19:27:36.718690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:59.630051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.687130+0000","last_change":"2023-11-08T19:27:35.700402+0000","last_active":"2023-11-08T19:27:36.687130+0000","last_peered":"2023-11-08T19:27:36.687130+0000","last_clean":"2023-11-08T19:27:36.687130+0000","last_became_active":"2023-11-08T19:27:35.699757+0000","last_became_peered":"2023-11-08T19:27:35.699757+0000","last_unstale":"2023-11-08T19:27:36.687130+0000","last_undegraded":"2023-11-08T19:27:36.687130+0000","last_fullsized":"2023-11-08T19:27:36.687130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:38.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.687018+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:27:35.687018+0000","last_peered":"2023-11-08T19:27:35.687018+0000","last_clean":"2023-11-08T19:27:35.687018+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:27:35.687018+0000","last_undegraded":"2023-11-08T19:27:35.687018+0000","last_fullsized":"2023-11-08T19:27:35.687018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.705551+0000","last_change":"2023-11-08T19:27:36.698372+0000","last_active":"2023-11-08T19:27:36.705551+0000","last_peered":"2023-11-08T19:27:36.705551+0000","last_clean":"2023-11-08T19:27:36.705551+0000","last_became_active":"2023-11-08T19:27:36.698102+0000","last_became_peered":"2023-11-08T19:27:36.698102+0000","last_unstale":"2023-11-08T19:27:36.705551+0000","last_undegraded":"2023-11-08T19:27:36.705551+0000","last_fullsized":"2023-11-08T19:27:36.705551+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:13:12.970804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.686108+0000","last_change":"2023-11-08T19:27:35.698393+0000","last_active":"2023-11-08T19:27:36.686108+0000","last_peered":"2023-11-08T19:27:36.686108+0000","last_clean":"2023-11-08T19:27:36.686108+0000","last_became_active":"2023-11-08T19:27:35.697894+0000","last_became_peered":"2023-11-08T19:27:35.697894+0000","last_unstale":"2023-11-08T19:27:36.686108+0000","last_undegraded":"2023-11-08T19:27:36.686108+0000","last_fullsized":"2023-11-08T19:27:36.686108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:36.028019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.666335+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:27:35.666335+0000","last_peered":"2023-11-08T19:27:35.666335+0000","last_clean":"2023-11-08T19:27:35.666335+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:27:35.666335+0000","last_undegraded":"2023-11-08T19:27:35.666335+0000","last_fullsized":"2023-11-08T19:27:35.666335+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725111+0000","last_change":"2023-11-08T19:27:36.720325+0000","last_active":"2023-11-08T19:27:36.725111+0000","last_peered":"2023-11-08T19:27:36.725111+0000","last_clean":"2023-11-08T19:27:36.725111+0000","last_became_active":"2023-11-08T19:27:36.718845+0000","last_became_peered":"2023-11-08T19:27:36.718845+0000","last_unstale":"2023-11-08T19:27:36.725111+0000","last_undegraded":"2023-11-08T19:27:36.725111+0000","last_fullsized":"2023-11-08T19:27:36.725111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:37.069165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.689448+0000","last_change":"2023-11-08T19:27:35.699119+0000","last_active":"2023-11-08T19:27:36.689448+0000","last_peered":"2023-11-08T19:27:36.689448+0000","last_clean":"2023-11-08T19:27:36.689448+0000","last_became_active":"2023-11-08T19:27:35.698779+0000","last_became_peered":"2023-11-08T19:27:35.698779+0000","last_unstale":"2023-11-08T19:27:36.689448+0000","last_undegraded":"2023-11-08T19:27:36.689448+0000","last_fullsized":"2023-11-08T19:27:36.689448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:44.462543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683018+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:27:35.683018+0000","last_peered":"2023-11-08T19:27:35.683018+0000","last_clean":"2023-11-08T19:27:35.683018+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:27:35.683018+0000","last_undegraded":"2023-11-08T19:27:35.683018+0000","last_fullsized":"2023-11-08T19:27:35.683018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.715166+0000","last_change":"2023-11-08T19:27:36.709926+0000","last_active":"2023-11-08T19:27:36.715166+0000","last_peered":"2023-11-08T19:27:36.715166+0000","last_clean":"2023-11-08T19:27:36.715166+0000","last_became_active":"2023-11-08T19:27:36.709606+0000","last_became_peered":"2023-11-08T19:27:36.709606+0000","last_unstale":"2023-11-08T19:27:36.715166+0000","last_undegraded":"2023-11-08T19:27:36.715166+0000","last_fullsized":"2023-11-08T19:27:36.715166+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:28.240261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.685757+0000","last_change":"2023-11-08T19:27:35.685323+0000","last_active":"2023-11-08T19:27:36.685757+0000","last_peered":"2023-11-08T19:27:36.685757+0000","last_clean":"2023-11-08T19:27:36.685757+0000","last_became_active":"2023-11-08T19:27:35.684519+0000","last_became_peered":"2023-11-08T19:27:35.684519+0000","last_unstale":"2023-11-08T19:27:36.685757+0000","last_undegraded":"2023-11-08T19:27:36.685757+0000","last_fullsized":"2023-11-08T19:27:36.685757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:34.830750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.666056+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:27:35.666056+0000","last_peered":"2023-11-08T19:27:35.666056+0000","last_clean":"2023-11-08T19:27:35.666056+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:27:35.666056+0000","last_undegraded":"2023-11-08T19:27:35.666056+0000","last_fullsized":"2023-11-08T19:27:35.666056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701947+0000","last_change":"2023-11-08T19:27:36.699999+0000","last_active":"2023-11-08T19:27:36.701947+0000","last_peered":"2023-11-08T19:27:36.701947+0000","last_clean":"2023-11-08T19:27:36.701947+0000","last_became_active":"2023-11-08T19:27:36.699709+0000","last_became_peered":"2023-11-08T19:27:36.699709+0000","last_unstale":"2023-11-08T19:27:36.701947+0000","last_undegraded":"2023-11-08T19:27:36.701947+0000","last_fullsized":"2023-11-08T19:27:36.701947+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:06.293100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.683282+0000","last_change":"2023-11-08T19:27:35.704135+0000","last_active":"2023-11-08T19:27:36.683282+0000","last_peered":"2023-11-08T19:27:36.683282+0000","last_clean":"2023-11-08T19:27:36.683282+0000","last_became_active":"2023-11-08T19:27:35.700389+0000","last_became_peered":"2023-11-08T19:27:35.700389+0000","last_unstale":"2023-11-08T19:27:36.683282+0000","last_undegraded":"2023-11-08T19:27:36.683282+0000","last_fullsized":"2023-11-08T19:27:36.683282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:44.462543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.673083+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:27:35.673083+0000","last_peered":"2023-11-08T19:27:35.673083+0000","last_clean":"2023-11-08T19:27:35.673083+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:27:35.673083+0000","last_undegraded":"2023-11-08T19:27:35.673083+0000","last_fullsized":"2023-11-08T19:27:35.673083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708141+0000","last_change":"2023-11-08T19:27:36.703957+0000","last_active":"2023-11-08T19:27:36.708141+0000","last_peered":"2023-11-08T19:27:36.708141+0000","last_clean":"2023-11-08T19:27:36.708141+0000","last_became_active":"2023-11-08T19:27:36.701505+0000","last_became_peered":"2023-11-08T19:27:36.701505+0000","last_unstale":"2023-11-08T19:27:36.708141+0000","last_undegraded":"2023-11-08T19:27:36.708141+0000","last_fullsized":"2023-11-08T19:27:36.708141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:43.218285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.724170+0000","last_change":"2023-11-08T19:27:35.695122+0000","last_active":"2023-11-08T19:27:36.724170+0000","last_peered":"2023-11-08T19:27:36.724170+0000","last_clean":"2023-11-08T19:27:36.724170+0000","last_became_active":"2023-11-08T19:27:35.694273+0000","last_became_peered":"2023-11-08T19:27:35.694273+0000","last_unstale":"2023-11-08T19:27:36.724170+0000","last_undegraded":"2023-11-08T19:27:36.724170+0000","last_fullsized":"2023-11-08T19:27:36.724170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:49.396825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.685308+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:27:35.685308+0000","last_peered":"2023-11-08T19:27:35.685308+0000","last_clean":"2023-11-08T19:27:35.685308+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:27:35.685308+0000","last_undegraded":"2023-11-08T19:27:35.685308+0000","last_fullsized":"2023-11-08T19:27:35.685308+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.711744+0000","last_change":"2023-11-08T19:27:36.708435+0000","last_active":"2023-11-08T19:27:36.711744+0000","last_peered":"2023-11-08T19:27:36.711744+0000","last_clean":"2023-11-08T19:27:36.711744+0000","last_became_active":"2023-11-08T19:27:36.707772+0000","last_became_peered":"2023-11-08T19:27:36.707772+0000","last_unstale":"2023-11-08T19:27:36.711744+0000","last_undegraded":"2023-11-08T19:27:36.711744+0000","last_fullsized":"2023-11-08T19:27:36.711744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:17.910713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.687317+0000","last_change":"2023-11-08T19:27:35.687556+0000","last_active":"2023-11-08T19:27:36.687317+0000","last_peered":"2023-11-08T19:27:36.687317+0000","last_clean":"2023-11-08T19:27:36.687317+0000","last_became_active":"2023-11-08T19:27:35.687309+0000","last_became_peered":"2023-11-08T19:27:35.687309+0000","last_unstale":"2023-11-08T19:27:36.687317+0000","last_undegraded":"2023-11-08T19:27:36.687317+0000","last_fullsized":"2023-11-08T19:27:36.687317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:06.078832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.692643+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:27:35.692643+0000","last_peered":"2023-11-08T19:27:35.692643+0000","last_clean":"2023-11-08T19:27:35.692643+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:27:35.692643+0000","last_undegraded":"2023-11-08T19:27:35.692643+0000","last_fullsized":"2023-11-08T19:27:35.692643+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707316+0000","last_change":"2023-11-08T19:27:36.702623+0000","last_active":"2023-11-08T19:27:36.707316+0000","last_peered":"2023-11-08T19:27:36.707316+0000","last_clean":"2023-11-08T19:27:36.707316+0000","last_became_active":"2023-11-08T19:27:36.702192+0000","last_became_peered":"2023-11-08T19:27:36.702192+0000","last_unstale":"2023-11-08T19:27:36.707316+0000","last_undegraded":"2023-11-08T19:27:36.707316+0000","last_fullsized":"2023-11-08T19:27:36.707316+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:21.818446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.677767+0000","last_change":"2023-11-08T19:27:35.698475+0000","last_active":"2023-11-08T19:27:36.677767+0000","last_peered":"2023-11-08T19:27:36.677767+0000","last_clean":"2023-11-08T19:27:36.677767+0000","last_became_active":"2023-11-08T19:27:35.697213+0000","last_became_peered":"2023-11-08T19:27:35.697213+0000","last_unstale":"2023-11-08T19:27:36.677767+0000","last_undegraded":"2023-11-08T19:27:36.677767+0000","last_fullsized":"2023-11-08T19:27:36.677767+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:36.062749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.686441+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:27:35.686441+0000","last_peered":"2023-11-08T19:27:35.686441+0000","last_clean":"2023-11-08T19:27:35.686441+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:27:35.686441+0000","last_undegraded":"2023-11-08T19:27:35.686441+0000","last_fullsized":"2023-11-08T19:27:35.686441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701615+0000","last_change":"2023-11-08T19:27:36.695934+0000","last_active":"2023-11-08T19:27:36.701615+0000","last_peered":"2023-11-08T19:27:36.701615+0000","last_clean":"2023-11-08T19:27:36.701615+0000","last_became_active":"2023-11-08T19:27:36.695599+0000","last_became_peered":"2023-11-08T19:27:36.695599+0000","last_unstale":"2023-11-08T19:27:36.701615+0000","last_undegraded":"2023-11-08T19:27:36.701615+0000","last_fullsized":"2023-11-08T19:27:36.701615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:45.623034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.689911+0000","last_change":"2023-11-08T19:27:35.692549+0000","last_active":"2023-11-08T19:27:36.689911+0000","last_peered":"2023-11-08T19:27:36.689911+0000","last_clean":"2023-11-08T19:27:36.689911+0000","last_became_active":"2023-11-08T19:27:35.692142+0000","last_became_peered":"2023-11-08T19:27:35.692142+0000","last_unstale":"2023-11-08T19:27:36.689911+0000","last_undegraded":"2023-11-08T19:27:36.689911+0000","last_fullsized":"2023-11-08T19:27:36.689911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:12.271555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690611+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:27:35.690611+0000","last_peered":"2023-11-08T19:27:35.690611+0000","last_clean":"2023-11-08T19:27:35.690611+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:27:35.690611+0000","last_undegraded":"2023-11-08T19:27:35.690611+0000","last_fullsized":"2023-11-08T19:27:35.690611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.713202+0000","last_change":"2023-11-08T19:27:36.704909+0000","last_active":"2023-11-08T19:27:36.713202+0000","last_peered":"2023-11-08T19:27:36.713202+0000","last_clean":"2023-11-08T19:27:36.713202+0000","last_became_active":"2023-11-08T19:27:36.701608+0000","last_became_peered":"2023-11-08T19:27:36.701608+0000","last_unstale":"2023-11-08T19:27:36.713202+0000","last_undegraded":"2023-11-08T19:27:36.713202+0000","last_fullsized":"2023-11-08T19:27:36.713202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:44:30.319676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.700766+0000","last_change":"2023-11-08T19:27:35.702904+0000","last_active":"2023-11-08T19:27:36.700766+0000","last_peered":"2023-11-08T19:27:36.700766+0000","last_clean":"2023-11-08T19:27:36.700766+0000","last_became_active":"2023-11-08T19:27:35.702628+0000","last_became_peered":"2023-11-08T19:27:35.702628+0000","last_unstale":"2023-11-08T19:27:36.700766+0000","last_undegraded":"2023-11-08T19:27:36.700766+0000","last_fullsized":"2023-11-08T19:27:36.700766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:10.680560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":37,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.695315+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:27:35.695315+0000","last_peered":"2023-11-08T19:27:35.695315+0000","last_clean":"2023-11-08T19:27:35.695315+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:27:35.695315+0000","last_undegraded":"2023-11-08T19:27:35.695315+0000","last_fullsized":"2023-11-08T19:27:35.695315+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706584+0000","last_change":"2023-11-08T19:27:36.703037+0000","last_active":"2023-11-08T19:27:36.706584+0000","last_peered":"2023-11-08T19:27:36.706584+0000","last_clean":"2023-11-08T19:27:36.706584+0000","last_became_active":"2023-11-08T19:27:36.702729+0000","last_became_peered":"2023-11-08T19:27:36.702729+0000","last_unstale":"2023-11-08T19:27:36.706584+0000","last_undegraded":"2023-11-08T19:27:36.706584+0000","last_fullsized":"2023-11-08T19:27:36.706584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:32:14.030744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.727651+0000","last_change":"2023-11-08T19:27:35.701273+0000","last_active":"2023-11-08T19:27:36.727651+0000","last_peered":"2023-11-08T19:27:36.727651+0000","last_clean":"2023-11-08T19:27:36.727651+0000","last_became_active":"2023-11-08T19:27:35.700942+0000","last_became_peered":"2023-11-08T19:27:35.700942+0000","last_unstale":"2023-11-08T19:27:36.727651+0000","last_undegraded":"2023-11-08T19:27:36.727651+0000","last_fullsized":"2023-11-08T19:27:36.727651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:36.535291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.698342+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:27:35.698342+0000","last_peered":"2023-11-08T19:27:35.698342+0000","last_clean":"2023-11-08T19:27:35.698342+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:27:35.698342+0000","last_undegraded":"2023-11-08T19:27:35.698342+0000","last_fullsized":"2023-11-08T19:27:35.698342+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723336+0000","last_change":"2023-11-08T19:27:36.715148+0000","last_active":"2023-11-08T19:27:36.723336+0000","last_peered":"2023-11-08T19:27:36.723336+0000","last_clean":"2023-11-08T19:27:36.723336+0000","last_became_active":"2023-11-08T19:27:36.714653+0000","last_became_peered":"2023-11-08T19:27:36.714653+0000","last_unstale":"2023-11-08T19:27:36.723336+0000","last_undegraded":"2023-11-08T19:27:36.723336+0000","last_fullsized":"2023-11-08T19:27:36.723336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:06.293100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726283+0000","last_change":"2023-11-08T19:27:35.702803+0000","last_active":"2023-11-08T19:27:36.726283+0000","last_peered":"2023-11-08T19:27:36.726283+0000","last_clean":"2023-11-08T19:27:36.726283+0000","last_became_active":"2023-11-08T19:27:35.701996+0000","last_became_peered":"2023-11-08T19:27:35.701996+0000","last_unstale":"2023-11-08T19:27:36.726283+0000","last_undegraded":"2023-11-08T19:27:36.726283+0000","last_fullsized":"2023-11-08T19:27:36.726283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:20.423465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683637+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:27:35.683637+0000","last_peered":"2023-11-08T19:27:35.683637+0000","last_clean":"2023-11-08T19:27:35.683637+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:27:35.683637+0000","last_undegraded":"2023-11-08T19:27:35.683637+0000","last_fullsized":"2023-11-08T19:27:35.683637+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.703854+0000","last_change":"2023-11-08T19:27:36.701630+0000","last_active":"2023-11-08T19:27:36.703854+0000","last_peered":"2023-11-08T19:27:36.703854+0000","last_clean":"2023-11-08T19:27:36.703854+0000","last_became_active":"2023-11-08T19:27:36.697730+0000","last_became_peered":"2023-11-08T19:27:36.697730+0000","last_unstale":"2023-11-08T19:27:36.703854+0000","last_undegraded":"2023-11-08T19:27:36.703854+0000","last_fullsized":"2023-11-08T19:27:36.703854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:34.731696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725710+0000","last_change":"2023-11-08T19:27:35.695294+0000","last_active":"2023-11-08T19:27:36.725710+0000","last_peered":"2023-11-08T19:27:36.725710+0000","last_clean":"2023-11-08T19:27:36.725710+0000","last_became_active":"2023-11-08T19:27:35.694816+0000","last_became_peered":"2023-11-08T19:27:35.694816+0000","last_unstale":"2023-11-08T19:27:36.725710+0000","last_undegraded":"2023-11-08T19:27:36.725710+0000","last_fullsized":"2023-11-08T19:27:36.725710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:28.540986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.685387+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:27:35.685387+0000","last_peered":"2023-11-08T19:27:35.685387+0000","last_clean":"2023-11-08T19:27:35.685387+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:27:35.685387+0000","last_undegraded":"2023-11-08T19:27:35.685387+0000","last_fullsized":"2023-11-08T19:27:35.685387+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707277+0000","last_change":"2023-11-08T19:27:36.702426+0000","last_active":"2023-11-08T19:27:36.707277+0000","last_peered":"2023-11-08T19:27:36.707277+0000","last_clean":"2023-11-08T19:27:36.707277+0000","last_became_active":"2023-11-08T19:27:36.702158+0000","last_became_peered":"2023-11-08T19:27:36.702158+0000","last_unstale":"2023-11-08T19:27:36.707277+0000","last_undegraded":"2023-11-08T19:27:36.707277+0000","last_fullsized":"2023-11-08T19:27:36.707277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:29.225198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725281+0000","last_change":"2023-11-08T19:27:35.694909+0000","last_active":"2023-11-08T19:27:36.725281+0000","last_peered":"2023-11-08T19:27:36.725281+0000","last_clean":"2023-11-08T19:27:36.725281+0000","last_became_active":"2023-11-08T19:27:35.694644+0000","last_became_peered":"2023-11-08T19:27:35.694644+0000","last_unstale":"2023-11-08T19:27:36.725281+0000","last_undegraded":"2023-11-08T19:27:36.725281+0000","last_fullsized":"2023-11-08T19:27:36.725281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:07.949148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.686174+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:27:35.686174+0000","last_peered":"2023-11-08T19:27:35.686174+0000","last_clean":"2023-11-08T19:27:35.686174+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:27:35.686174+0000","last_undegraded":"2023-11-08T19:27:35.686174+0000","last_fullsized":"2023-11-08T19:27:35.686174+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.689040+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:27:35.689040+0000","last_peered":"2023-11-08T19:27:35.689040+0000","last_clean":"2023-11-08T19:27:35.689040+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:27:35.689040+0000","last_undegraded":"2023-11-08T19:27:35.689040+0000","last_fullsized":"2023-11-08T19:27:35.689040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":940948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.087,"15min":1.1040000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4930000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.29,"15min":1.325},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4450000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":1.2949999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.232,"15min":1.2689999999999999},"min":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6619999999999999,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.1060000000000001,"15min":1.1299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3160000000000001,"5min":1.681,"15min":1.681},"last":1.5469999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.466,"15min":1.5169999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.5089999999999999,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.349,"15min":1.385},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.772,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.751}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.341,"15min":1.377},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.419,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.536,"15min":1.5800000000000001},"min":{"1min":0.95299999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6439999999999999,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.6359999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3560000000000001,"15min":1.3700000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.704,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.4690000000000001,"15min":1.5109999999999999},"min":{"1min":0.82799999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.5780000000000001,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.6539999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4470000000000001,"15min":1.4650000000000001},"min":{"1min":1.004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.766,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4850000000000001,"15min":1.4910000000000001},"min":{"1min":1.1599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":1.9590000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.99399999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5700000000000001,"15min":1.587},"min":{"1min":1.036,"5min":1.036,"15min":1.036},"max":{"1min":1.8680000000000001,"5min":2.427,"15min":2.427},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.6060000000000001,"15min":1.6439999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.746,"5min":2.468,"15min":2.468},"last":1.569}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":0.95399999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.83399999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.3580000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":1.089},"min":{"1min":0.747,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.534,"5min":1.679,"15min":1.679},"last":1.218},{"interface":"front","average":{"1min":1.194,"5min":1.331,"15min":1.3540000000000001},"min":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.591,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.89300000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.274,"15min":1.304},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4990000000000001,"5min":4.609,"15min":4.609},"last":1.411},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2210000000000001,"15min":1.214},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.5609999999999999,"5min":1.861,"15min":1.861},"last":1.024}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.877,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.496},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.179,"15min":1.1799999999999999},"min":{"1min":0.80300000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.663,"5min":1.784,"15min":1.784},"last":1.3220000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.383,"15min":1.381},"min":{"1min":0.89900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.024,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":1.04,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.577}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.98199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7749999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3320000000000001,"15min":1.3300000000000001},"min":{"1min":0.90200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.831,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1579999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5509999999999999,"15min":1.5449999999999999},"min":{"1min":1.097,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1240000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.96599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.724,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.611}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0900000000000001,"15min":1.073},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.712,"5min":1.76,"15min":1.76},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.246,"15min":1.2390000000000001},"min":{"1min":0.879,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.645,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4510000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.167,"15min":1.179},"min":{"1min":0.82399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.401,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.405,"15min":1.462},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4970000000000001,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.4890000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1799999999999999,"15min":1.171},"min":{"1min":0.91500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8700000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.4650000000000001,"15min":1.5229999999999999},"min":{"1min":0.76100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.5900000000000001,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.6220000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8200000000000001,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.3149999999999999,"15min":1.3069999999999999},"min":{"1min":0.89900000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7609999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2589999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.288,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.966,"5min":2.347,"15min":2.347},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.544,"15min":1.5629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0230000000000001,"5min":3.153,"15min":3.153},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.516,"15min":1.536},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9199999999999999,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.494,"15min":1.5069999999999999},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.869,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.518}]},{"osd":7,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.6830000000000001,"15min":1.7150000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1269999999999998,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.587,"15min":1.5920000000000001},"min":{"1min":0.99399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.994,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.4259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.036,"15min":1.0409999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.331},{"interface":"front","average":{"1min":1.492,"5min":1.248,"15min":1.2070000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":7.0119999999999996,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.256}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.103,"15min":1.1020000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.121,"15min":1.163},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2470000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.0900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3520000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.8719999999999999,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.3959999999999999,"15min":1.4319999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.375,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3080000000000001,"15min":1.302},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.91900000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0470000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.2889999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.486,"15min":1.528},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.4630000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5469999999999999,"15min":1.5660000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.1679999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5569999999999999,"15min":1.573},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.2360000000000002,"5min":4.335,"15min":4.335},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.613,"15min":1.6339999999999999},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.492}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.637,"15min":1.6419999999999999},"min":{"1min":0.95499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.5490000000000004,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.7170000000000001,"15min":1.7430000000000001},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":3.1389999999999998,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.625}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.056,"15min":1.0960000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2909999999999999,"5min":3.972,"15min":3.972},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3999999999999999,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.88500000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.1679999999999999,"15min":1.2070000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6839999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":1.0900000000000001,"15min":1.121},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3360000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":1.073}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.3089999999999999,"15min":1.3380000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.752,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.256,"15min":1.282},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.778,"5min":2.73,"15min":2.73},"last":1.48}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.4350000000000001,"15min":1.4690000000000001},"min":{"1min":0.872,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.861,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.052},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.448,"15min":1.47},"min":{"1min":0.91400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5980000000000001,"5min":3.052,"15min":3.052},"last":1.3959999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.4750000000000001,"15min":1.5109999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.149,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.5169999999999999,"15min":1.548},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.375,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.167}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":1.054,"5min":0.878,"15min":0.878},"max":{"1min":1.8220000000000001,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.5740000000000001,"15min":1.6000000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.4860000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.131}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.6930000000000001,"15min":1.736},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.528,"5min":4.407,"15min":4.407},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.573,"15min":1.5960000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8740000000000001,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.2390000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":349561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.222,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.052},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.90200000000000002,"15min":0.93400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.206,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.94599999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.202,"5min":1.393,"15min":1.393},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.02},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.177,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.51300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.486,"5min":1.53,"15min":1.53},"last":1.3029999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.099,"15min":1.139},"min":{"1min":0.48099999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3859999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.4450000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.3600000000000001,"15min":1.421},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3100000000000001,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.8},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3360000000000001,"15min":1.3779999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.506,"5min":2.278,"15min":2.278},"last":1.2050000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.389,"15min":1.4350000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6180000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":2.1600000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.456,"5min":2.536,"15min":2.536},"last":1.2629999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.363,"15min":1.395},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5409999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.4219999999999999,"15min":1.468},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8480000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":2.036}]},{"osd":7,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.5389999999999999,"15min":1.593},"min":{"1min":0.81399999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5329999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.5289999999999999,"15min":1.5680000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.5840000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.8839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.626},{"interface":"front","average":{"1min":0.749,"5min":0.753,"15min":0.754},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.0369999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79100000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1339999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.88300000000000001,"15min":0.88600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.224,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0580000000000001,"15min":1.0569999999999999},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.276},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.499,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8540000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.323}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.542,"5min":2.165,"15min":2.165},"last":1.248},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2410000000000001,"15min":1.2569999999999999},"min":{"1min":0.73799999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4910000000000001,"5min":1.885,"15min":1.885},"last":1.069}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.254,"15min":1.264},"min":{"1min":0.71099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7390000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.706,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.1719999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.359,"15min":1.367},"min":{"1min":0.754,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.772,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.425},{"interface":"front","average":{"1min":1.294,"5min":1.373,"15min":1.3859999999999999},"min":{"1min":0.92000000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.6930000000000001,"5min":2.282,"15min":2.282},"last":1.018}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.329},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6559999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.371},{"interface":"front","average":{"1min":1.262,"5min":1.3740000000000001,"15min":1.3919999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.956,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.2070000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":939929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0649999999999999,"5min":1.357,"15min":1.357},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2889999999999999,"5min":1.7,"15min":1.7},"last":0.995}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2410000000000001,"5min":1.508,"15min":1.508},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.05}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1,"15min":1.012},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.278,"5min":1.748,"15min":1.748},"last":0.875},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0880000000000001,"15min":1.115},"min":{"1min":0.53500000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":2.496,"15min":2.496},"last":1.3120000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2290000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5169999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.4159999999999999,"15min":1.4570000000000001},"min":{"1min":0.81699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8700000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.2170000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.351,"15min":1.3819999999999999},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9510000000000001,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.427,"15min":1.4570000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7070000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.522}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.516},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.8160000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.3999999999999999,"15min":1.4199999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6259999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4810000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3899999999999999,"15min":1.4059999999999999},"min":{"1min":0.92600000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.746,"5min":2.173,"15min":2.173},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.4590000000000001,"15min":1.4950000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.0070000000000001,"5min":2.306,"15min":2.306},"last":1.419}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:41.156 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:27:41.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.400+0000 7f6a9ffff640 1 Processor -- start 2023-11-08T19:27:41.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.400+0000 7f6a9ffff640 1 -- start start 2023-11-08T19:27:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.401+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa0070a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.401+0000 7f6a9ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6aa006ee80 0x7f6aa006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.401+0000 7f6a9ffff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aa0071980 con 0x7f6aa006a660 2023-11-08T19:27:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9ffff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aa015d330 con 0x7f6aa006ee80 2023-11-08T19:27:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9ffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa006aaf0 con 0x7f6aa015cf50 2023-11-08T19:27:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9e7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6aa006ee80 0x7f6aa006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9f7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6aa006a660 0x7f6aa01609a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41806/0 (socket says 172.21.15.119:41806) 2023-11-08T19:27:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9f7fe640 1 -- 172.21.15.119:0/2770443228 learned_addr learned my addr 172.21.15.119:0/2770443228 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9effd640 1 --2- 172.21.15.119:0/2770443228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa0070a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9dffb640 1 -- 172.21.15.119:0/2770443228 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773826336 0 0) 0x7f6aa0071980 con 0x7f6aa006a660 2023-11-08T19:27:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9dffb640 1 -- 172.21.15.119:0/2770443228 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a8c003520 con 0x7f6aa006a660 2023-11-08T19:27:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2770443228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 msgr2=0x7f6aa0070a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9e7fc640 1 --2- 172.21.15.119:0/2770443228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa0070a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2770443228 >> v1:172.21.15.119:6789/0 conn(0x7f6aa006a660 legacy=0x7f6aa01609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.402+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2770443228 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa00713c0 con 0x7f6aa006ee80 2023-11-08T19:27:41.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9e7fc640 1 --2- 172.21.15.119:0/2770443228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa006a120 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009d90 tx=0x7f6a8802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49d76e4e95ea5b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9dffb640 1 -- 172.21.15.119:0/2770443228 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a8802cc50 con 0x7f6aa006ee80 2023-11-08T19:27:41.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9dffb640 1 -- 172.21.15.119:0/2770443228 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a8802cdb0 con 0x7f6aa006ee80 2023-11-08T19:27:41.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9dffb640 1 -- 172.21.15.119:0/2770443228 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88034660 con 0x7f6aa006ee80 2023-11-08T19:27:41.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2770443228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 msgr2=0x7f6aa006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2770443228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa006a120 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6a88009d90 tx=0x7f6a8802bea0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.404+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2770443228 shutdown_connections 2023-11-08T19:27:41.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.405+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2770443228 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa006a120 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.405+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2770443228 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa0070a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.405+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2770443228 >> 172.21.15.119:0/2770443228 conn(0x7f6aa00cfae0 msgr2=0x7f6aa00cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:41.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.405+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2770443228 shutdown_connections 2023-11-08T19:27:41.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.405+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2770443228 wait complete. 2023-11-08T19:27:41.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.406+0000 7f6a9ffff640 1 Processor -- start 2023-11-08T19:27:41.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.406+0000 7f6a9ffff640 1 -- start start 2023-11-08T19:27:41.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.407+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aa006a660 0x7f6aa00c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.407+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa014df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.407+0000 7f6a9ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa014ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.407+0000 7f6a9ffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa00c6730 con 0x7f6aa006ee80 2023-11-08T19:27:41.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.407+0000 7f6a9ffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa014e4a0 con 0x7f6aa006a660 2023-11-08T19:27:41.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.407+0000 7f6a9ffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aa014b1d0 con 0x7f6aa015cf50 2023-11-08T19:27:41.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa014df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa014df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54814/0 (socket says 172.21.15.119:54814) 2023-11-08T19:27:41.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2968586838 learned_addr learned my addr 172.21.15.119:0/2968586838 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:41.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9effd640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aa006a660 0x7f6aa00c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 msgr2=0x7f6aa014ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa014ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aa006a660 msgr2=0x7f6aa00c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aa006a660 0x7f6aa00c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 -- 172.21.15.119:0/2968586838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a88009a70 con 0x7f6aa006ee80 2023-11-08T19:27:41.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.408+0000 7f6a9e7fc640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa014df60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6a88037040 tx=0x7f6a88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.409+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88040650 con 0x7f6aa006ee80 2023-11-08T19:27:41.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.409+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a880407b0 con 0x7f6aa006ee80 2023-11-08T19:27:41.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.409+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a88040b60 con 0x7f6aa006ee80 2023-11-08T19:27:41.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.409+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aa00c5b20 con 0x7f6aa006ee80 2023-11-08T19:27:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.409+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aa00c60e0 con 0x7f6aa006ee80 2023-11-08T19:27:41.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.410+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aa004ece0 con 0x7f6aa006ee80 2023-11-08T19:27:41.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.411+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6a88040cc0 con 0x7f6aa006ee80 2023-11-08T19:27:41.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.412+0000 7f6a977fe640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6a8006ae70 0x7f6a8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.412+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f6a880ac670 con 0x7f6aa006ee80 2023-11-08T19:27:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.412+0000 7f6a9effd640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6a8006ae70 0x7f6a8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.413+0000 7f6a9effd640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6a8006ae70 0x7f6a8006d330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6aa01f5f00 tx=0x7f6a90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.416+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6a880774d0 con 0x7f6aa006ee80 2023-11-08T19:27:41.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.604+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6aa00cd960 con 0x7f6a8006ae70 2023-11-08T19:27:41.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.631+0000 7f6a977fe640 1 -- 172.21.15.119:0/2968586838 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+368173 (secure 0 0 0) 0x7f6a9000e4c0 con 0x7f6a8006ae70 2023-11-08T19:27:41.635 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:41.639 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:27:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6a8006ae70 msgr2=0x7f6a8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6a8006ae70 0x7f6a8006d330 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f6aa01f5f00 tx=0x7f6a90008040 comp rx=0 tx=0).stop 2023-11-08T19:27:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 msgr2=0x7f6aa014df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa014df60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6a88037040 tx=0x7f6a88004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 shutdown_connections 2023-11-08T19:27:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6a8006ae70 0x7f6a8006d330 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6aa015cf50 0x7f6aa014ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6aa006ee80 0x7f6aa014df60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 --2- 172.21.15.119:0/2968586838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6aa006a660 0x7f6aa00c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.639+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 >> 172.21.15.119:0/2968586838 conn(0x7f6aa00cfae0 msgr2=0x7f6aa00ae8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:41.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.640+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 shutdown_connections 2023-11-08T19:27:41.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.640+0000 7f6a9ffff640 1 -- 172.21.15.119:0/2968586838 wait complete. 2023-11-08T19:27:41.663 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2023-11-08T19:27:40.727429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":231592,"kb_used_data":7104,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754743128,"statfs":{"total":773094113280,"available":772856963072,"internally_reserved":0,"allocated":7274496,"data_stored":3983338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.062603"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723901+0000","last_change":"2023-11-08T19:27:36.716672+0000","last_active":"2023-11-08T19:27:36.723901+0000","last_peered":"2023-11-08T19:27:36.723901+0000","last_clean":"2023-11-08T19:27:36.723901+0000","last_became_active":"2023-11-08T19:27:36.716111+0000","last_became_peered":"2023-11-08T19:27:36.716111+0000","last_unstale":"2023-11-08T19:27:36.723901+0000","last_undegraded":"2023-11-08T19:27:36.723901+0000","last_fullsized":"2023-11-08T19:27:36.723901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:39:05.587346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.715410+0000","last_change":"2023-11-08T19:27:36.709840+0000","last_active":"2023-11-08T19:27:36.715410+0000","last_peered":"2023-11-08T19:27:36.715410+0000","last_clean":"2023-11-08T19:27:36.715410+0000","last_became_active":"2023-11-08T19:27:36.709572+0000","last_became_peered":"2023-11-08T19:27:36.709572+0000","last_unstale":"2023-11-08T19:27:36.715410+0000","last_undegraded":"2023-11-08T19:27:36.715410+0000","last_fullsized":"2023-11-08T19:27:36.715410+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:19.093728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.702479+0000","last_change":"2023-11-08T19:27:36.698576+0000","last_active":"2023-11-08T19:27:36.702479+0000","last_peered":"2023-11-08T19:27:36.702479+0000","last_clean":"2023-11-08T19:27:36.702479+0000","last_became_active":"2023-11-08T19:27:36.698300+0000","last_became_peered":"2023-11-08T19:27:36.698300+0000","last_unstale":"2023-11-08T19:27:36.702479+0000","last_undegraded":"2023-11-08T19:27:36.702479+0000","last_fullsized":"2023-11-08T19:27:36.702479+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:27.236861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707821+0000","last_change":"2023-11-08T19:27:36.706145+0000","last_active":"2023-11-08T19:27:36.707821+0000","last_peered":"2023-11-08T19:27:36.707821+0000","last_clean":"2023-11-08T19:27:36.707821+0000","last_became_active":"2023-11-08T19:27:36.705843+0000","last_became_peered":"2023-11-08T19:27:36.705843+0000","last_unstale":"2023-11-08T19:27:36.707821+0000","last_undegraded":"2023-11-08T19:27:36.707821+0000","last_fullsized":"2023-11-08T19:27:36.707821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:59.733745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.714874+0000","last_change":"2023-11-08T19:27:36.712170+0000","last_active":"2023-11-08T19:27:36.714874+0000","last_peered":"2023-11-08T19:27:36.714874+0000","last_clean":"2023-11-08T19:27:36.714874+0000","last_became_active":"2023-11-08T19:27:36.711786+0000","last_became_peered":"2023-11-08T19:27:36.711786+0000","last_unstale":"2023-11-08T19:27:36.714874+0000","last_undegraded":"2023-11-08T19:27:36.714874+0000","last_fullsized":"2023-11-08T19:27:36.714874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:59.157976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.711502+0000","last_change":"2023-11-08T19:27:36.707650+0000","last_active":"2023-11-08T19:27:36.711502+0000","last_peered":"2023-11-08T19:27:36.711502+0000","last_clean":"2023-11-08T19:27:36.711502+0000","last_became_active":"2023-11-08T19:27:36.707354+0000","last_became_peered":"2023-11-08T19:27:36.707354+0000","last_unstale":"2023-11-08T19:27:36.711502+0000","last_undegraded":"2023-11-08T19:27:36.711502+0000","last_fullsized":"2023-11-08T19:27:36.711502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:29.016613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707911+0000","last_change":"2023-11-08T19:27:36.704297+0000","last_active":"2023-11-08T19:27:36.707911+0000","last_peered":"2023-11-08T19:27:36.707911+0000","last_clean":"2023-11-08T19:27:36.707911+0000","last_became_active":"2023-11-08T19:27:36.701042+0000","last_became_peered":"2023-11-08T19:27:36.701042+0000","last_unstale":"2023-11-08T19:27:36.707911+0000","last_undegraded":"2023-11-08T19:27:36.707911+0000","last_fullsized":"2023-11-08T19:27:36.707911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:45.169002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708812+0000","last_change":"2023-11-08T19:27:36.705605+0000","last_active":"2023-11-08T19:27:36.708812+0000","last_peered":"2023-11-08T19:27:36.708812+0000","last_clean":"2023-11-08T19:27:36.708812+0000","last_became_active":"2023-11-08T19:27:36.705070+0000","last_became_peered":"2023-11-08T19:27:36.705070+0000","last_unstale":"2023-11-08T19:27:36.708812+0000","last_undegraded":"2023-11-08T19:27:36.708812+0000","last_fullsized":"2023-11-08T19:27:36.708812+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:15:32.005497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710639+0000","last_change":"2023-11-08T19:27:36.702917+0000","last_active":"2023-11-08T19:27:36.710639+0000","last_peered":"2023-11-08T19:27:36.710639+0000","last_clean":"2023-11-08T19:27:36.710639+0000","last_became_active":"2023-11-08T19:27:36.702439+0000","last_became_peered":"2023-11-08T19:27:36.702439+0000","last_unstale":"2023-11-08T19:27:36.710639+0000","last_undegraded":"2023-11-08T19:27:36.710639+0000","last_fullsized":"2023-11-08T19:27:36.710639+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:59.274535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701764+0000","last_change":"2023-11-08T19:27:36.696574+0000","last_active":"2023-11-08T19:27:36.701764+0000","last_peered":"2023-11-08T19:27:36.701764+0000","last_clean":"2023-11-08T19:27:36.701764+0000","last_became_active":"2023-11-08T19:27:36.694474+0000","last_became_peered":"2023-11-08T19:27:36.694474+0000","last_unstale":"2023-11-08T19:27:36.701764+0000","last_undegraded":"2023-11-08T19:27:36.701764+0000","last_fullsized":"2023-11-08T19:27:36.701764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:14.476460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723261+0000","last_change":"2023-11-08T19:27:36.716582+0000","last_active":"2023-11-08T19:27:36.723261+0000","last_peered":"2023-11-08T19:27:36.723261+0000","last_clean":"2023-11-08T19:27:36.723261+0000","last_became_active":"2023-11-08T19:27:36.716285+0000","last_became_peered":"2023-11-08T19:27:36.716285+0000","last_unstale":"2023-11-08T19:27:36.723261+0000","last_undegraded":"2023-11-08T19:27:36.723261+0000","last_fullsized":"2023-11-08T19:27:36.723261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:52.328999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706937+0000","last_change":"2023-11-08T19:27:36.705159+0000","last_active":"2023-11-08T19:27:36.706937+0000","last_peered":"2023-11-08T19:27:36.706937+0000","last_clean":"2023-11-08T19:27:36.706937+0000","last_became_active":"2023-11-08T19:27:36.704848+0000","last_became_peered":"2023-11-08T19:27:36.704848+0000","last_unstale":"2023-11-08T19:27:36.706937+0000","last_undegraded":"2023-11-08T19:27:36.706937+0000","last_fullsized":"2023-11-08T19:27:36.706937+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:59.543611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707886+0000","last_change":"2023-11-08T19:27:36.704388+0000","last_active":"2023-11-08T19:27:36.707886+0000","last_peered":"2023-11-08T19:27:36.707886+0000","last_clean":"2023-11-08T19:27:36.707886+0000","last_became_active":"2023-11-08T19:27:36.704118+0000","last_became_peered":"2023-11-08T19:27:36.704118+0000","last_unstale":"2023-11-08T19:27:36.707886+0000","last_undegraded":"2023-11-08T19:27:36.707886+0000","last_fullsized":"2023-11-08T19:27:36.707886+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:37.069165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701160+0000","last_change":"2023-11-08T19:27:36.693889+0000","last_active":"2023-11-08T19:27:36.701160+0000","last_peered":"2023-11-08T19:27:36.701160+0000","last_clean":"2023-11-08T19:27:36.701160+0000","last_became_active":"2023-11-08T19:27:36.693530+0000","last_became_peered":"2023-11-08T19:27:36.693530+0000","last_unstale":"2023-11-08T19:27:36.701160+0000","last_undegraded":"2023-11-08T19:27:36.701160+0000","last_fullsized":"2023-11-08T19:27:36.701160+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:36.417003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708104+0000","last_change":"2023-11-08T19:27:36.701647+0000","last_active":"2023-11-08T19:27:36.708104+0000","last_peered":"2023-11-08T19:27:36.708104+0000","last_clean":"2023-11-08T19:27:36.708104+0000","last_became_active":"2023-11-08T19:27:36.701390+0000","last_became_peered":"2023-11-08T19:27:36.701390+0000","last_unstale":"2023-11-08T19:27:36.708104+0000","last_undegraded":"2023-11-08T19:27:36.708104+0000","last_fullsized":"2023-11-08T19:27:36.708104+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:06.293100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710758+0000","last_change":"2023-11-08T19:27:36.705645+0000","last_active":"2023-11-08T19:27:36.710758+0000","last_peered":"2023-11-08T19:27:36.710758+0000","last_clean":"2023-11-08T19:27:36.710758+0000","last_became_active":"2023-11-08T19:27:36.705355+0000","last_became_peered":"2023-11-08T19:27:36.705355+0000","last_unstale":"2023-11-08T19:27:36.710758+0000","last_undegraded":"2023-11-08T19:27:36.710758+0000","last_fullsized":"2023-11-08T19:27:36.710758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:09.489563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710621+0000","last_change":"2023-11-08T19:27:36.702691+0000","last_active":"2023-11-08T19:27:36.710621+0000","last_peered":"2023-11-08T19:27:36.710621+0000","last_clean":"2023-11-08T19:27:36.710621+0000","last_became_active":"2023-11-08T19:27:36.702077+0000","last_became_peered":"2023-11-08T19:27:36.702077+0000","last_unstale":"2023-11-08T19:27:36.710621+0000","last_undegraded":"2023-11-08T19:27:36.710621+0000","last_fullsized":"2023-11-08T19:27:36.710621+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:33:19.377581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710799+0000","last_change":"2023-11-08T19:27:36.706954+0000","last_active":"2023-11-08T19:27:36.710799+0000","last_peered":"2023-11-08T19:27:36.710799+0000","last_clean":"2023-11-08T19:27:36.710799+0000","last_became_active":"2023-11-08T19:27:36.705938+0000","last_became_peered":"2023-11-08T19:27:36.705938+0000","last_unstale":"2023-11-08T19:27:36.710799+0000","last_undegraded":"2023-11-08T19:27:36.710799+0000","last_fullsized":"2023-11-08T19:27:36.710799+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:37.541707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708743+0000","last_change":"2023-11-08T19:27:36.705629+0000","last_active":"2023-11-08T19:27:36.708743+0000","last_peered":"2023-11-08T19:27:36.708743+0000","last_clean":"2023-11-08T19:27:36.708743+0000","last_became_active":"2023-11-08T19:27:36.705287+0000","last_became_peered":"2023-11-08T19:27:36.705287+0000","last_unstale":"2023-11-08T19:27:36.708743+0000","last_undegraded":"2023-11-08T19:27:36.708743+0000","last_fullsized":"2023-11-08T19:27:36.708743+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:32:14.030744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701804+0000","last_change":"2023-11-08T19:27:36.698208+0000","last_active":"2023-11-08T19:27:36.701804+0000","last_peered":"2023-11-08T19:27:36.701804+0000","last_clean":"2023-11-08T19:27:36.701804+0000","last_became_active":"2023-11-08T19:27:36.697880+0000","last_became_peered":"2023-11-08T19:27:36.697880+0000","last_unstale":"2023-11-08T19:27:36.701804+0000","last_undegraded":"2023-11-08T19:27:36.701804+0000","last_fullsized":"2023-11-08T19:27:36.701804+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:52.328999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.702157+0000","last_change":"2023-11-08T19:27:36.697725+0000","last_active":"2023-11-08T19:27:36.702157+0000","last_peered":"2023-11-08T19:27:36.702157+0000","last_clean":"2023-11-08T19:27:36.702157+0000","last_became_active":"2023-11-08T19:27:36.697431+0000","last_became_peered":"2023-11-08T19:27:36.697431+0000","last_unstale":"2023-11-08T19:27:36.702157+0000","last_undegraded":"2023-11-08T19:27:36.702157+0000","last_fullsized":"2023-11-08T19:27:36.702157+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:34.738158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.703783+0000","last_change":"2023-11-08T19:27:36.694772+0000","last_active":"2023-11-08T19:27:36.703783+0000","last_peered":"2023-11-08T19:27:36.703783+0000","last_clean":"2023-11-08T19:27:36.703783+0000","last_became_active":"2023-11-08T19:27:36.694463+0000","last_became_peered":"2023-11-08T19:27:36.694463+0000","last_unstale":"2023-11-08T19:27:36.703783+0000","last_undegraded":"2023-11-08T19:27:36.703783+0000","last_fullsized":"2023-11-08T19:27:36.703783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:29.380001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.714192+0000","last_change":"2023-11-08T19:27:36.710127+0000","last_active":"2023-11-08T19:27:36.714192+0000","last_peered":"2023-11-08T19:27:36.714192+0000","last_clean":"2023-11-08T19:27:36.714192+0000","last_became_active":"2023-11-08T19:27:36.709802+0000","last_became_peered":"2023-11-08T19:27:36.709802+0000","last_unstale":"2023-11-08T19:27:36.714192+0000","last_undegraded":"2023-11-08T19:27:36.714192+0000","last_fullsized":"2023-11-08T19:27:36.714192+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:11.860449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725140+0000","last_change":"2023-11-08T19:27:36.716966+0000","last_active":"2023-11-08T19:27:36.725140+0000","last_peered":"2023-11-08T19:27:36.725140+0000","last_clean":"2023-11-08T19:27:36.725140+0000","last_became_active":"2023-11-08T19:27:36.716636+0000","last_became_peered":"2023-11-08T19:27:36.716636+0000","last_unstale":"2023-11-08T19:27:36.725140+0000","last_undegraded":"2023-11-08T19:27:36.725140+0000","last_fullsized":"2023-11-08T19:27:36.725140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:38.342781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.720472+0000","last_change":"2023-11-08T19:27:36.710658+0000","last_active":"2023-11-08T19:27:36.720472+0000","last_peered":"2023-11-08T19:27:36.720472+0000","last_clean":"2023-11-08T19:27:36.720472+0000","last_became_active":"2023-11-08T19:27:36.710370+0000","last_became_peered":"2023-11-08T19:27:36.710370+0000","last_unstale":"2023-11-08T19:27:36.720472+0000","last_undegraded":"2023-11-08T19:27:36.720472+0000","last_fullsized":"2023-11-08T19:27:36.720472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:08.031883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708059+0000","last_change":"2023-11-08T19:27:36.704975+0000","last_active":"2023-11-08T19:27:36.708059+0000","last_peered":"2023-11-08T19:27:36.708059+0000","last_clean":"2023-11-08T19:27:36.708059+0000","last_became_active":"2023-11-08T19:27:36.704689+0000","last_became_peered":"2023-11-08T19:27:36.704689+0000","last_unstale":"2023-11-08T19:27:36.708059+0000","last_undegraded":"2023-11-08T19:27:36.708059+0000","last_fullsized":"2023-11-08T19:27:36.708059+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:07.085248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.714389+0000","last_change":"2023-11-08T19:27:36.712698+0000","last_active":"2023-11-08T19:27:36.714389+0000","last_peered":"2023-11-08T19:27:36.714389+0000","last_clean":"2023-11-08T19:27:36.714389+0000","last_became_active":"2023-11-08T19:27:36.712363+0000","last_became_peered":"2023-11-08T19:27:36.712363+0000","last_unstale":"2023-11-08T19:27:36.714389+0000","last_undegraded":"2023-11-08T19:27:36.714389+0000","last_fullsized":"2023-11-08T19:27:36.714389+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:55.145043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706917+0000","last_change":"2023-11-08T19:27:36.696146+0000","last_active":"2023-11-08T19:27:36.706917+0000","last_peered":"2023-11-08T19:27:36.706917+0000","last_clean":"2023-11-08T19:27:36.706917+0000","last_became_active":"2023-11-08T19:27:36.695868+0000","last_became_peered":"2023-11-08T19:27:36.695868+0000","last_unstale":"2023-11-08T19:27:36.706917+0000","last_undegraded":"2023-11-08T19:27:36.706917+0000","last_fullsized":"2023-11-08T19:27:36.706917+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:35.672402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.696023+0000","last_change":"2023-11-08T19:27:36.691121+0000","last_active":"2023-11-08T19:27:36.696023+0000","last_peered":"2023-11-08T19:27:36.696023+0000","last_clean":"2023-11-08T19:27:36.696023+0000","last_became_active":"2023-11-08T19:27:36.686832+0000","last_became_peered":"2023-11-08T19:27:36.686832+0000","last_unstale":"2023-11-08T19:27:36.696023+0000","last_undegraded":"2023-11-08T19:27:36.696023+0000","last_fullsized":"2023-11-08T19:27:36.696023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:09.409597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707593+0000","last_change":"2023-11-08T19:27:36.702872+0000","last_active":"2023-11-08T19:27:36.707593+0000","last_peered":"2023-11-08T19:27:36.707593+0000","last_clean":"2023-11-08T19:27:36.707593+0000","last_became_active":"2023-11-08T19:27:36.702478+0000","last_became_peered":"2023-11-08T19:27:36.702478+0000","last_unstale":"2023-11-08T19:27:36.707593+0000","last_undegraded":"2023-11-08T19:27:36.707593+0000","last_fullsized":"2023-11-08T19:27:36.707593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:52.328999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.703769+0000","last_change":"2023-11-08T19:27:36.694811+0000","last_active":"2023-11-08T19:27:36.703769+0000","last_peered":"2023-11-08T19:27:36.703769+0000","last_clean":"2023-11-08T19:27:36.703769+0000","last_became_active":"2023-11-08T19:27:36.694473+0000","last_became_peered":"2023-11-08T19:27:36.694473+0000","last_unstale":"2023-11-08T19:27:36.703769+0000","last_undegraded":"2023-11-08T19:27:36.703769+0000","last_fullsized":"2023-11-08T19:27:36.703769+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:28.240261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708736+0000","last_change":"2023-11-08T19:27:36.704105+0000","last_active":"2023-11-08T19:27:36.708736+0000","last_peered":"2023-11-08T19:27:36.708736+0000","last_clean":"2023-11-08T19:27:36.708736+0000","last_became_active":"2023-11-08T19:27:36.703770+0000","last_became_peered":"2023-11-08T19:27:36.703770+0000","last_unstale":"2023-11-08T19:27:36.708736+0000","last_undegraded":"2023-11-08T19:27:36.708736+0000","last_fullsized":"2023-11-08T19:27:36.708736+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:39.274464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.705590+0000","last_change":"2023-11-08T19:27:36.697782+0000","last_active":"2023-11-08T19:27:36.705590+0000","last_peered":"2023-11-08T19:27:36.705590+0000","last_clean":"2023-11-08T19:27:36.705590+0000","last_became_active":"2023-11-08T19:27:36.697278+0000","last_became_peered":"2023-11-08T19:27:36.697278+0000","last_unstale":"2023-11-08T19:27:36.705590+0000","last_undegraded":"2023-11-08T19:27:36.705590+0000","last_fullsized":"2023-11-08T19:27:36.705590+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:52:54.238604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"64'8","reported_seq":19,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726463+0000","last_change":"2023-11-08T19:27:35.700276+0000","last_active":"2023-11-08T19:27:36.726463+0000","last_peered":"2023-11-08T19:27:36.726463+0000","last_clean":"2023-11-08T19:27:36.726463+0000","last_became_active":"2023-11-08T19:27:35.700002+0000","last_became_peered":"2023-11-08T19:27:35.700002+0000","last_unstale":"2023-11-08T19:27:36.726463+0000","last_undegraded":"2023-11-08T19:27:36.726463+0000","last_fullsized":"2023-11-08T19:27:36.726463+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:18.733671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.672777+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:27:35.672777+0000","last_peered":"2023-11-08T19:27:35.672777+0000","last_clean":"2023-11-08T19:27:35.672777+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:27:35.672777+0000","last_undegraded":"2023-11-08T19:27:35.672777+0000","last_fullsized":"2023-11-08T19:27:35.672777+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.713136+0000","last_change":"2023-11-08T19:27:36.709698+0000","last_active":"2023-11-08T19:27:36.713136+0000","last_peered":"2023-11-08T19:27:36.713136+0000","last_clean":"2023-11-08T19:27:36.713136+0000","last_became_active":"2023-11-08T19:27:36.709423+0000","last_became_peered":"2023-11-08T19:27:36.709423+0000","last_unstale":"2023-11-08T19:27:36.713136+0000","last_undegraded":"2023-11-08T19:27:36.713136+0000","last_fullsized":"2023-11-08T19:27:36.713136+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:39:05.587346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.677366+0000","last_change":"2023-11-08T19:27:35.698052+0000","last_active":"2023-11-08T19:27:36.677366+0000","last_peered":"2023-11-08T19:27:36.677366+0000","last_clean":"2023-11-08T19:27:36.677366+0000","last_became_active":"2023-11-08T19:27:35.697671+0000","last_became_peered":"2023-11-08T19:27:35.697671+0000","last_unstale":"2023-11-08T19:27:36.677366+0000","last_undegraded":"2023-11-08T19:27:36.677366+0000","last_fullsized":"2023-11-08T19:27:36.677366+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:10.680560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.699222+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:27:35.699222+0000","last_peered":"2023-11-08T19:27:35.699222+0000","last_clean":"2023-11-08T19:27:35.699222+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:27:35.699222+0000","last_undegraded":"2023-11-08T19:27:35.699222+0000","last_fullsized":"2023-11-08T19:27:35.699222+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.697376+0000","last_change":"2023-11-08T19:27:36.687160+0000","last_active":"2023-11-08T19:27:36.697376+0000","last_peered":"2023-11-08T19:27:36.697376+0000","last_clean":"2023-11-08T19:27:36.697376+0000","last_became_active":"2023-11-08T19:27:36.686463+0000","last_became_peered":"2023-11-08T19:27:36.686463+0000","last_unstale":"2023-11-08T19:27:36.697376+0000","last_undegraded":"2023-11-08T19:27:36.697376+0000","last_fullsized":"2023-11-08T19:27:36.697376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:43.218285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.689763+0000","last_change":"2023-11-08T19:27:35.703028+0000","last_active":"2023-11-08T19:27:36.689763+0000","last_peered":"2023-11-08T19:27:36.689763+0000","last_clean":"2023-11-08T19:27:36.689763+0000","last_became_active":"2023-11-08T19:27:35.702765+0000","last_became_peered":"2023-11-08T19:27:35.702765+0000","last_unstale":"2023-11-08T19:27:36.689763+0000","last_undegraded":"2023-11-08T19:27:36.689763+0000","last_fullsized":"2023-11-08T19:27:36.689763+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:03.864531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690306+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:27:35.690306+0000","last_peered":"2023-11-08T19:27:35.690306+0000","last_clean":"2023-11-08T19:27:35.690306+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:27:35.690306+0000","last_undegraded":"2023-11-08T19:27:35.690306+0000","last_fullsized":"2023-11-08T19:27:35.690306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723506+0000","last_change":"2023-11-08T19:27:36.718152+0000","last_active":"2023-11-08T19:27:36.723506+0000","last_peered":"2023-11-08T19:27:36.723506+0000","last_clean":"2023-11-08T19:27:36.723506+0000","last_became_active":"2023-11-08T19:27:36.717884+0000","last_became_peered":"2023-11-08T19:27:36.717884+0000","last_unstale":"2023-11-08T19:27:36.723506+0000","last_undegraded":"2023-11-08T19:27:36.723506+0000","last_fullsized":"2023-11-08T19:27:36.723506+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:45.169002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.728117+0000","last_change":"2023-11-08T19:27:35.691767+0000","last_active":"2023-11-08T19:27:36.728117+0000","last_peered":"2023-11-08T19:27:36.728117+0000","last_clean":"2023-11-08T19:27:36.728117+0000","last_became_active":"2023-11-08T19:27:35.691274+0000","last_became_peered":"2023-11-08T19:27:35.691274+0000","last_unstale":"2023-11-08T19:27:36.728117+0000","last_undegraded":"2023-11-08T19:27:36.728117+0000","last_fullsized":"2023-11-08T19:27:36.728117+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:33.725280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.672696+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:27:35.672696+0000","last_peered":"2023-11-08T19:27:35.672696+0000","last_clean":"2023-11-08T19:27:35.672696+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:27:35.672696+0000","last_undegraded":"2023-11-08T19:27:35.672696+0000","last_fullsized":"2023-11-08T19:27:35.672696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.713222+0000","last_change":"2023-11-08T19:27:36.705366+0000","last_active":"2023-11-08T19:27:36.713222+0000","last_peered":"2023-11-08T19:27:36.713222+0000","last_clean":"2023-11-08T19:27:36.713222+0000","last_became_active":"2023-11-08T19:27:36.701244+0000","last_became_peered":"2023-11-08T19:27:36.701244+0000","last_unstale":"2023-11-08T19:27:36.713222+0000","last_undegraded":"2023-11-08T19:27:36.713222+0000","last_fullsized":"2023-11-08T19:27:36.713222+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:01.643525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"64'6","reported_seq":17,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.728657+0000","last_change":"2023-11-08T19:27:35.688971+0000","last_active":"2023-11-08T19:27:36.728657+0000","last_peered":"2023-11-08T19:27:36.728657+0000","last_clean":"2023-11-08T19:27:36.728657+0000","last_became_active":"2023-11-08T19:27:35.688242+0000","last_became_peered":"2023-11-08T19:27:35.688242+0000","last_unstale":"2023-11-08T19:27:36.728657+0000","last_undegraded":"2023-11-08T19:27:36.728657+0000","last_fullsized":"2023-11-08T19:27:36.728657+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:48:39.655471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.685131+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:27:35.685131+0000","last_peered":"2023-11-08T19:27:35.685131+0000","last_clean":"2023-11-08T19:27:35.685131+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:27:35.685131+0000","last_undegraded":"2023-11-08T19:27:35.685131+0000","last_fullsized":"2023-11-08T19:27:35.685131+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.709229+0000","last_change":"2023-11-08T19:27:36.706871+0000","last_active":"2023-11-08T19:27:36.709229+0000","last_peered":"2023-11-08T19:27:36.709229+0000","last_clean":"2023-11-08T19:27:36.709229+0000","last_became_active":"2023-11-08T19:27:36.706611+0000","last_became_peered":"2023-11-08T19:27:36.706611+0000","last_unstale":"2023-11-08T19:27:36.709229+0000","last_undegraded":"2023-11-08T19:27:36.709229+0000","last_fullsized":"2023-11-08T19:27:36.709229+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:27.547156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"64'3","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725227+0000","last_change":"2023-11-08T19:27:35.699732+0000","last_active":"2023-11-08T19:27:36.725227+0000","last_peered":"2023-11-08T19:27:36.725227+0000","last_clean":"2023-11-08T19:27:36.725227+0000","last_became_active":"2023-11-08T19:27:35.699247+0000","last_became_peered":"2023-11-08T19:27:35.699247+0000","last_unstale":"2023-11-08T19:27:36.725227+0000","last_undegraded":"2023-11-08T19:27:36.725227+0000","last_fullsized":"2023-11-08T19:27:36.725227+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:28.926193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.682074+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:27:35.682074+0000","last_peered":"2023-11-08T19:27:35.682074+0000","last_clean":"2023-11-08T19:27:35.682074+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:27:35.682074+0000","last_undegraded":"2023-11-08T19:27:35.682074+0000","last_fullsized":"2023-11-08T19:27:35.682074+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.727992+0000","last_change":"2023-11-08T19:27:36.723911+0000","last_active":"2023-11-08T19:27:36.727992+0000","last_peered":"2023-11-08T19:27:36.727992+0000","last_clean":"2023-11-08T19:27:36.727992+0000","last_became_active":"2023-11-08T19:27:36.722714+0000","last_became_peered":"2023-11-08T19:27:36.722714+0000","last_unstale":"2023-11-08T19:27:36.727992+0000","last_undegraded":"2023-11-08T19:27:36.727992+0000","last_fullsized":"2023-11-08T19:27:36.727992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:07.085248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.688955+0000","last_change":"2023-11-08T19:27:35.701294+0000","last_active":"2023-11-08T19:27:36.688955+0000","last_peered":"2023-11-08T19:27:36.688955+0000","last_clean":"2023-11-08T19:27:36.688955+0000","last_became_active":"2023-11-08T19:27:35.700929+0000","last_became_peered":"2023-11-08T19:27:35.700929+0000","last_unstale":"2023-11-08T19:27:36.688955+0000","last_undegraded":"2023-11-08T19:27:36.688955+0000","last_fullsized":"2023-11-08T19:27:36.688955+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:43:48.626834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683207+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:27:35.683207+0000","last_peered":"2023-11-08T19:27:35.683207+0000","last_clean":"2023-11-08T19:27:35.683207+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:27:35.683207+0000","last_undegraded":"2023-11-08T19:27:35.683207+0000","last_fullsized":"2023-11-08T19:27:35.683207+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.715640+0000","last_change":"2023-11-08T19:27:36.712174+0000","last_active":"2023-11-08T19:27:36.715640+0000","last_peered":"2023-11-08T19:27:36.715640+0000","last_clean":"2023-11-08T19:27:36.715640+0000","last_became_active":"2023-11-08T19:27:36.711595+0000","last_became_peered":"2023-11-08T19:27:36.711595+0000","last_unstale":"2023-11-08T19:27:36.715640+0000","last_undegraded":"2023-11-08T19:27:36.715640+0000","last_fullsized":"2023-11-08T19:27:36.715640+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:39:05.587346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.681264+0000","last_change":"2023-11-08T19:27:35.699287+0000","last_active":"2023-11-08T19:27:36.681264+0000","last_peered":"2023-11-08T19:27:36.681264+0000","last_clean":"2023-11-08T19:27:36.681264+0000","last_became_active":"2023-11-08T19:27:35.698905+0000","last_became_peered":"2023-11-08T19:27:35.698905+0000","last_unstale":"2023-11-08T19:27:36.681264+0000","last_undegraded":"2023-11-08T19:27:36.681264+0000","last_fullsized":"2023-11-08T19:27:36.681264+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:53.718272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":37,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.695797+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:27:35.695797+0000","last_peered":"2023-11-08T19:27:35.695797+0000","last_clean":"2023-11-08T19:27:35.695797+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:27:35.695797+0000","last_undegraded":"2023-11-08T19:27:35.695797+0000","last_fullsized":"2023-11-08T19:27:35.695797+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.717038+0000","last_change":"2023-11-08T19:27:36.713266+0000","last_active":"2023-11-08T19:27:36.717038+0000","last_peered":"2023-11-08T19:27:36.717038+0000","last_clean":"2023-11-08T19:27:36.717038+0000","last_became_active":"2023-11-08T19:27:36.711706+0000","last_became_peered":"2023-11-08T19:27:36.711706+0000","last_unstale":"2023-11-08T19:27:36.717038+0000","last_undegraded":"2023-11-08T19:27:36.717038+0000","last_fullsized":"2023-11-08T19:27:36.717038+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:11.686976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.680978+0000","last_change":"2023-11-08T19:27:35.684433+0000","last_active":"2023-11-08T19:27:36.680978+0000","last_peered":"2023-11-08T19:27:36.680978+0000","last_clean":"2023-11-08T19:27:36.680978+0000","last_became_active":"2023-11-08T19:27:35.683972+0000","last_became_peered":"2023-11-08T19:27:35.683972+0000","last_unstale":"2023-11-08T19:27:36.680978+0000","last_undegraded":"2023-11-08T19:27:36.680978+0000","last_fullsized":"2023-11-08T19:27:36.680978+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:55.257234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.692555+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:27:35.692555+0000","last_peered":"2023-11-08T19:27:35.692555+0000","last_clean":"2023-11-08T19:27:35.692555+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:27:35.692555+0000","last_undegraded":"2023-11-08T19:27:35.692555+0000","last_fullsized":"2023-11-08T19:27:35.692555+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.710665+0000","last_change":"2023-11-08T19:27:36.701688+0000","last_active":"2023-11-08T19:27:36.710665+0000","last_peered":"2023-11-08T19:27:36.710665+0000","last_clean":"2023-11-08T19:27:36.710665+0000","last_became_active":"2023-11-08T19:27:36.700932+0000","last_became_peered":"2023-11-08T19:27:36.700932+0000","last_unstale":"2023-11-08T19:27:36.710665+0000","last_undegraded":"2023-11-08T19:27:36.710665+0000","last_fullsized":"2023-11-08T19:27:36.710665+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:22.735243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.685511+0000","last_change":"2023-11-08T19:27:35.698924+0000","last_active":"2023-11-08T19:27:36.685511+0000","last_peered":"2023-11-08T19:27:36.685511+0000","last_clean":"2023-11-08T19:27:36.685511+0000","last_became_active":"2023-11-08T19:27:35.698598+0000","last_became_peered":"2023-11-08T19:27:35.698598+0000","last_unstale":"2023-11-08T19:27:36.685511+0000","last_undegraded":"2023-11-08T19:27:36.685511+0000","last_fullsized":"2023-11-08T19:27:36.685511+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:12.271555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.684086+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:27:35.684086+0000","last_peered":"2023-11-08T19:27:35.684086+0000","last_clean":"2023-11-08T19:27:35.684086+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:27:35.684086+0000","last_undegraded":"2023-11-08T19:27:35.684086+0000","last_fullsized":"2023-11-08T19:27:35.684086+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.720811+0000","last_change":"2023-11-08T19:27:36.716610+0000","last_active":"2023-11-08T19:27:36.720811+0000","last_peered":"2023-11-08T19:27:36.720811+0000","last_clean":"2023-11-08T19:27:36.720811+0000","last_became_active":"2023-11-08T19:27:36.710960+0000","last_became_peered":"2023-11-08T19:27:36.710960+0000","last_unstale":"2023-11-08T19:27:36.720811+0000","last_undegraded":"2023-11-08T19:27:36.720811+0000","last_fullsized":"2023-11-08T19:27:36.720811+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:48:40.661888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.724281+0000","last_change":"2023-11-08T19:27:35.691410+0000","last_active":"2023-11-08T19:27:36.724281+0000","last_peered":"2023-11-08T19:27:36.724281+0000","last_clean":"2023-11-08T19:27:36.724281+0000","last_became_active":"2023-11-08T19:27:35.686591+0000","last_became_peered":"2023-11-08T19:27:35.686591+0000","last_unstale":"2023-11-08T19:27:36.724281+0000","last_undegraded":"2023-11-08T19:27:36.724281+0000","last_fullsized":"2023-11-08T19:27:36.724281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:34:58.537194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.694242+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:27:35.694242+0000","last_peered":"2023-11-08T19:27:35.694242+0000","last_clean":"2023-11-08T19:27:35.694242+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:27:35.694242+0000","last_undegraded":"2023-11-08T19:27:35.694242+0000","last_fullsized":"2023-11-08T19:27:35.694242+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.702961+0000","last_change":"2023-11-08T19:27:36.700688+0000","last_active":"2023-11-08T19:27:36.702961+0000","last_peered":"2023-11-08T19:27:36.702961+0000","last_clean":"2023-11-08T19:27:36.702961+0000","last_became_active":"2023-11-08T19:27:36.698691+0000","last_became_peered":"2023-11-08T19:27:36.698691+0000","last_unstale":"2023-11-08T19:27:36.702961+0000","last_undegraded":"2023-11-08T19:27:36.702961+0000","last_fullsized":"2023-11-08T19:27:36.702961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:30:38.342781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723217+0000","last_change":"2023-11-08T19:27:35.702083+0000","last_active":"2023-11-08T19:27:36.723217+0000","last_peered":"2023-11-08T19:27:36.723217+0000","last_clean":"2023-11-08T19:27:36.723217+0000","last_became_active":"2023-11-08T19:27:35.701931+0000","last_became_peered":"2023-11-08T19:27:35.701931+0000","last_unstale":"2023-11-08T19:27:36.723217+0000","last_undegraded":"2023-11-08T19:27:36.723217+0000","last_fullsized":"2023-11-08T19:27:36.723217+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:05:56.851993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":37,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.695624+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:27:35.695624+0000","last_peered":"2023-11-08T19:27:35.695624+0000","last_clean":"2023-11-08T19:27:35.695624+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:27:35.695624+0000","last_undegraded":"2023-11-08T19:27:35.695624+0000","last_fullsized":"2023-11-08T19:27:35.695624+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.721874+0000","last_change":"2023-11-08T19:27:36.714283+0000","last_active":"2023-11-08T19:27:36.721874+0000","last_peered":"2023-11-08T19:27:36.721874+0000","last_clean":"2023-11-08T19:27:36.721874+0000","last_became_active":"2023-11-08T19:27:36.713973+0000","last_became_peered":"2023-11-08T19:27:36.713973+0000","last_unstale":"2023-11-08T19:27:36.721874+0000","last_undegraded":"2023-11-08T19:27:36.721874+0000","last_fullsized":"2023-11-08T19:27:36.721874+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:30.189699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726778+0000","last_change":"2023-11-08T19:27:35.701825+0000","last_active":"2023-11-08T19:27:36.726778+0000","last_peered":"2023-11-08T19:27:36.726778+0000","last_clean":"2023-11-08T19:27:36.726778+0000","last_became_active":"2023-11-08T19:27:35.698585+0000","last_became_peered":"2023-11-08T19:27:35.698585+0000","last_unstale":"2023-11-08T19:27:36.726778+0000","last_undegraded":"2023-11-08T19:27:36.726778+0000","last_fullsized":"2023-11-08T19:27:36.726778+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:51:28.373585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690469+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:27:35.690469+0000","last_peered":"2023-11-08T19:27:35.690469+0000","last_clean":"2023-11-08T19:27:35.690469+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:27:35.690469+0000","last_undegraded":"2023-11-08T19:27:35.690469+0000","last_fullsized":"2023-11-08T19:27:35.690469+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.720262+0000","last_change":"2023-11-08T19:27:36.712235+0000","last_active":"2023-11-08T19:27:36.720262+0000","last_peered":"2023-11-08T19:27:36.720262+0000","last_clean":"2023-11-08T19:27:36.720262+0000","last_became_active":"2023-11-08T19:27:36.711918+0000","last_became_peered":"2023-11-08T19:27:36.711918+0000","last_unstale":"2023-11-08T19:27:36.720262+0000","last_undegraded":"2023-11-08T19:27:36.720262+0000","last_fullsized":"2023-11-08T19:27:36.720262+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:34.731696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726328+0000","last_change":"2023-11-08T19:27:35.703104+0000","last_active":"2023-11-08T19:27:36.726328+0000","last_peered":"2023-11-08T19:27:36.726328+0000","last_clean":"2023-11-08T19:27:36.726328+0000","last_became_active":"2023-11-08T19:27:35.702369+0000","last_became_peered":"2023-11-08T19:27:35.702369+0000","last_unstale":"2023-11-08T19:27:36.726328+0000","last_undegraded":"2023-11-08T19:27:36.726328+0000","last_fullsized":"2023-11-08T19:27:36.726328+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:33:23.275569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683506+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:27:35.683506+0000","last_peered":"2023-11-08T19:27:35.683506+0000","last_clean":"2023-11-08T19:27:35.683506+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:27:35.683506+0000","last_undegraded":"2023-11-08T19:27:35.683506+0000","last_fullsized":"2023-11-08T19:27:35.683506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706901+0000","last_change":"2023-11-08T19:27:36.698650+0000","last_active":"2023-11-08T19:27:36.706901+0000","last_peered":"2023-11-08T19:27:36.706901+0000","last_clean":"2023-11-08T19:27:36.706901+0000","last_became_active":"2023-11-08T19:27:36.698399+0000","last_became_peered":"2023-11-08T19:27:36.698399+0000","last_unstale":"2023-11-08T19:27:36.706901+0000","last_undegraded":"2023-11-08T19:27:36.706901+0000","last_fullsized":"2023-11-08T19:27:36.706901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:46.100687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.724731+0000","last_change":"2023-11-08T19:27:35.689917+0000","last_active":"2023-11-08T19:27:36.724731+0000","last_peered":"2023-11-08T19:27:36.724731+0000","last_clean":"2023-11-08T19:27:36.724731+0000","last_became_active":"2023-11-08T19:27:35.689120+0000","last_became_peered":"2023-11-08T19:27:35.689120+0000","last_unstale":"2023-11-08T19:27:36.724731+0000","last_undegraded":"2023-11-08T19:27:36.724731+0000","last_fullsized":"2023-11-08T19:27:36.724731+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:29.183282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683316+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:27:35.683316+0000","last_peered":"2023-11-08T19:27:35.683316+0000","last_clean":"2023-11-08T19:27:35.683316+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:27:35.683316+0000","last_undegraded":"2023-11-08T19:27:35.683316+0000","last_fullsized":"2023-11-08T19:27:35.683316+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.727793+0000","last_change":"2023-11-08T19:27:36.722335+0000","last_active":"2023-11-08T19:27:36.727793+0000","last_peered":"2023-11-08T19:27:36.727793+0000","last_clean":"2023-11-08T19:27:36.727793+0000","last_became_active":"2023-11-08T19:27:36.721130+0000","last_became_peered":"2023-11-08T19:27:36.721130+0000","last_unstale":"2023-11-08T19:27:36.727793+0000","last_undegraded":"2023-11-08T19:27:36.727793+0000","last_fullsized":"2023-11-08T19:27:36.727793+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:45.468959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"64'4","reported_seq":15,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.728398+0000","last_change":"2023-11-08T19:27:35.703330+0000","last_active":"2023-11-08T19:27:36.728398+0000","last_peered":"2023-11-08T19:27:36.728398+0000","last_clean":"2023-11-08T19:27:36.728398+0000","last_became_active":"2023-11-08T19:27:35.701922+0000","last_became_peered":"2023-11-08T19:27:35.701922+0000","last_unstale":"2023-11-08T19:27:36.728398+0000","last_undegraded":"2023-11-08T19:27:36.728398+0000","last_fullsized":"2023-11-08T19:27:36.728398+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:42.211869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.689896+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:27:35.689896+0000","last_peered":"2023-11-08T19:27:35.689896+0000","last_clean":"2023-11-08T19:27:35.689896+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:27:35.689896+0000","last_undegraded":"2023-11-08T19:27:35.689896+0000","last_fullsized":"2023-11-08T19:27:35.689896+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708167+0000","last_change":"2023-11-08T19:27:36.703458+0000","last_active":"2023-11-08T19:27:36.708167+0000","last_peered":"2023-11-08T19:27:36.708167+0000","last_clean":"2023-11-08T19:27:36.708167+0000","last_became_active":"2023-11-08T19:27:36.702815+0000","last_became_peered":"2023-11-08T19:27:36.702815+0000","last_unstale":"2023-11-08T19:27:36.708167+0000","last_undegraded":"2023-11-08T19:27:36.708167+0000","last_fullsized":"2023-11-08T19:27:36.708167+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:48:40.661888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.690762+0000","last_change":"2023-11-08T19:27:35.694565+0000","last_active":"2023-11-08T19:27:36.690762+0000","last_peered":"2023-11-08T19:27:36.690762+0000","last_clean":"2023-11-08T19:27:36.690762+0000","last_became_active":"2023-11-08T19:27:35.694131+0000","last_became_peered":"2023-11-08T19:27:35.694131+0000","last_unstale":"2023-11-08T19:27:36.690762+0000","last_undegraded":"2023-11-08T19:27:36.690762+0000","last_fullsized":"2023-11-08T19:27:36.690762+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:58.727329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690360+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:27:35.690360+0000","last_peered":"2023-11-08T19:27:35.690360+0000","last_clean":"2023-11-08T19:27:35.690360+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:27:35.690360+0000","last_undegraded":"2023-11-08T19:27:35.690360+0000","last_fullsized":"2023-11-08T19:27:35.690360+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701812+0000","last_change":"2023-11-08T19:27:36.697531+0000","last_active":"2023-11-08T19:27:36.701812+0000","last_peered":"2023-11-08T19:27:36.701812+0000","last_clean":"2023-11-08T19:27:36.701812+0000","last_became_active":"2023-11-08T19:27:36.697220+0000","last_became_peered":"2023-11-08T19:27:36.697220+0000","last_unstale":"2023-11-08T19:27:36.701812+0000","last_undegraded":"2023-11-08T19:27:36.701812+0000","last_fullsized":"2023-11-08T19:27:36.701812+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:04.870947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.676019+0000","last_change":"2023-11-08T19:27:35.700686+0000","last_active":"2023-11-08T19:27:36.676019+0000","last_peered":"2023-11-08T19:27:36.676019+0000","last_clean":"2023-11-08T19:27:36.676019+0000","last_became_active":"2023-11-08T19:27:35.700348+0000","last_became_peered":"2023-11-08T19:27:35.700348+0000","last_unstale":"2023-11-08T19:27:36.676019+0000","last_undegraded":"2023-11-08T19:27:36.676019+0000","last_fullsized":"2023-11-08T19:27:36.676019+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:58.727329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.687800+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:27:35.687800+0000","last_peered":"2023-11-08T19:27:35.687800+0000","last_clean":"2023-11-08T19:27:35.687800+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:27:35.687800+0000","last_undegraded":"2023-11-08T19:27:35.687800+0000","last_fullsized":"2023-11-08T19:27:35.687800+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.718690+0000","last_change":"2023-11-08T19:27:36.712627+0000","last_active":"2023-11-08T19:27:36.718690+0000","last_peered":"2023-11-08T19:27:36.718690+0000","last_clean":"2023-11-08T19:27:36.718690+0000","last_became_active":"2023-11-08T19:27:36.712267+0000","last_became_peered":"2023-11-08T19:27:36.712267+0000","last_unstale":"2023-11-08T19:27:36.718690+0000","last_undegraded":"2023-11-08T19:27:36.718690+0000","last_fullsized":"2023-11-08T19:27:36.718690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:59.630051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.687130+0000","last_change":"2023-11-08T19:27:35.700402+0000","last_active":"2023-11-08T19:27:36.687130+0000","last_peered":"2023-11-08T19:27:36.687130+0000","last_clean":"2023-11-08T19:27:36.687130+0000","last_became_active":"2023-11-08T19:27:35.699757+0000","last_became_peered":"2023-11-08T19:27:35.699757+0000","last_unstale":"2023-11-08T19:27:36.687130+0000","last_undegraded":"2023-11-08T19:27:36.687130+0000","last_fullsized":"2023-11-08T19:27:36.687130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:38.268048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.687018+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:27:35.687018+0000","last_peered":"2023-11-08T19:27:35.687018+0000","last_clean":"2023-11-08T19:27:35.687018+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:27:35.687018+0000","last_undegraded":"2023-11-08T19:27:35.687018+0000","last_fullsized":"2023-11-08T19:27:35.687018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.705551+0000","last_change":"2023-11-08T19:27:36.698372+0000","last_active":"2023-11-08T19:27:36.705551+0000","last_peered":"2023-11-08T19:27:36.705551+0000","last_clean":"2023-11-08T19:27:36.705551+0000","last_became_active":"2023-11-08T19:27:36.698102+0000","last_became_peered":"2023-11-08T19:27:36.698102+0000","last_unstale":"2023-11-08T19:27:36.705551+0000","last_undegraded":"2023-11-08T19:27:36.705551+0000","last_fullsized":"2023-11-08T19:27:36.705551+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:13:12.970804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.686108+0000","last_change":"2023-11-08T19:27:35.698393+0000","last_active":"2023-11-08T19:27:36.686108+0000","last_peered":"2023-11-08T19:27:36.686108+0000","last_clean":"2023-11-08T19:27:36.686108+0000","last_became_active":"2023-11-08T19:27:35.697894+0000","last_became_peered":"2023-11-08T19:27:35.697894+0000","last_unstale":"2023-11-08T19:27:36.686108+0000","last_undegraded":"2023-11-08T19:27:36.686108+0000","last_fullsized":"2023-11-08T19:27:36.686108+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:36.028019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.666335+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:27:35.666335+0000","last_peered":"2023-11-08T19:27:35.666335+0000","last_clean":"2023-11-08T19:27:35.666335+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:27:35.666335+0000","last_undegraded":"2023-11-08T19:27:35.666335+0000","last_fullsized":"2023-11-08T19:27:35.666335+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725111+0000","last_change":"2023-11-08T19:27:36.720325+0000","last_active":"2023-11-08T19:27:36.725111+0000","last_peered":"2023-11-08T19:27:36.725111+0000","last_clean":"2023-11-08T19:27:36.725111+0000","last_became_active":"2023-11-08T19:27:36.718845+0000","last_became_peered":"2023-11-08T19:27:36.718845+0000","last_unstale":"2023-11-08T19:27:36.725111+0000","last_undegraded":"2023-11-08T19:27:36.725111+0000","last_fullsized":"2023-11-08T19:27:36.725111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:37.069165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.689448+0000","last_change":"2023-11-08T19:27:35.699119+0000","last_active":"2023-11-08T19:27:36.689448+0000","last_peered":"2023-11-08T19:27:36.689448+0000","last_clean":"2023-11-08T19:27:36.689448+0000","last_became_active":"2023-11-08T19:27:35.698779+0000","last_became_peered":"2023-11-08T19:27:35.698779+0000","last_unstale":"2023-11-08T19:27:36.689448+0000","last_undegraded":"2023-11-08T19:27:36.689448+0000","last_fullsized":"2023-11-08T19:27:36.689448+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:44.462543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683018+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:27:35.683018+0000","last_peered":"2023-11-08T19:27:35.683018+0000","last_clean":"2023-11-08T19:27:35.683018+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:27:35.683018+0000","last_undegraded":"2023-11-08T19:27:35.683018+0000","last_fullsized":"2023-11-08T19:27:35.683018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.715166+0000","last_change":"2023-11-08T19:27:36.709926+0000","last_active":"2023-11-08T19:27:36.715166+0000","last_peered":"2023-11-08T19:27:36.715166+0000","last_clean":"2023-11-08T19:27:36.715166+0000","last_became_active":"2023-11-08T19:27:36.709606+0000","last_became_peered":"2023-11-08T19:27:36.709606+0000","last_unstale":"2023-11-08T19:27:36.715166+0000","last_undegraded":"2023-11-08T19:27:36.715166+0000","last_fullsized":"2023-11-08T19:27:36.715166+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:28.240261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.685757+0000","last_change":"2023-11-08T19:27:35.685323+0000","last_active":"2023-11-08T19:27:36.685757+0000","last_peered":"2023-11-08T19:27:36.685757+0000","last_clean":"2023-11-08T19:27:36.685757+0000","last_became_active":"2023-11-08T19:27:35.684519+0000","last_became_peered":"2023-11-08T19:27:35.684519+0000","last_unstale":"2023-11-08T19:27:36.685757+0000","last_undegraded":"2023-11-08T19:27:36.685757+0000","last_fullsized":"2023-11-08T19:27:36.685757+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:34.830750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.666056+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:27:35.666056+0000","last_peered":"2023-11-08T19:27:35.666056+0000","last_clean":"2023-11-08T19:27:35.666056+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:27:35.666056+0000","last_undegraded":"2023-11-08T19:27:35.666056+0000","last_fullsized":"2023-11-08T19:27:35.666056+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701947+0000","last_change":"2023-11-08T19:27:36.699999+0000","last_active":"2023-11-08T19:27:36.701947+0000","last_peered":"2023-11-08T19:27:36.701947+0000","last_clean":"2023-11-08T19:27:36.701947+0000","last_became_active":"2023-11-08T19:27:36.699709+0000","last_became_peered":"2023-11-08T19:27:36.699709+0000","last_unstale":"2023-11-08T19:27:36.701947+0000","last_undegraded":"2023-11-08T19:27:36.701947+0000","last_fullsized":"2023-11-08T19:27:36.701947+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:06.293100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.683282+0000","last_change":"2023-11-08T19:27:35.704135+0000","last_active":"2023-11-08T19:27:36.683282+0000","last_peered":"2023-11-08T19:27:36.683282+0000","last_clean":"2023-11-08T19:27:36.683282+0000","last_became_active":"2023-11-08T19:27:35.700389+0000","last_became_peered":"2023-11-08T19:27:35.700389+0000","last_unstale":"2023-11-08T19:27:36.683282+0000","last_undegraded":"2023-11-08T19:27:36.683282+0000","last_fullsized":"2023-11-08T19:27:36.683282+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:44.462543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.673083+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:27:35.673083+0000","last_peered":"2023-11-08T19:27:35.673083+0000","last_clean":"2023-11-08T19:27:35.673083+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:27:35.673083+0000","last_undegraded":"2023-11-08T19:27:35.673083+0000","last_fullsized":"2023-11-08T19:27:35.673083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.708141+0000","last_change":"2023-11-08T19:27:36.703957+0000","last_active":"2023-11-08T19:27:36.708141+0000","last_peered":"2023-11-08T19:27:36.708141+0000","last_clean":"2023-11-08T19:27:36.708141+0000","last_became_active":"2023-11-08T19:27:36.701505+0000","last_became_peered":"2023-11-08T19:27:36.701505+0000","last_unstale":"2023-11-08T19:27:36.708141+0000","last_undegraded":"2023-11-08T19:27:36.708141+0000","last_fullsized":"2023-11-08T19:27:36.708141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:43.218285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.724170+0000","last_change":"2023-11-08T19:27:35.695122+0000","last_active":"2023-11-08T19:27:36.724170+0000","last_peered":"2023-11-08T19:27:36.724170+0000","last_clean":"2023-11-08T19:27:36.724170+0000","last_became_active":"2023-11-08T19:27:35.694273+0000","last_became_peered":"2023-11-08T19:27:35.694273+0000","last_unstale":"2023-11-08T19:27:36.724170+0000","last_undegraded":"2023-11-08T19:27:36.724170+0000","last_fullsized":"2023-11-08T19:27:36.724170+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:42:49.396825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.685308+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:27:35.685308+0000","last_peered":"2023-11-08T19:27:35.685308+0000","last_clean":"2023-11-08T19:27:35.685308+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:27:35.685308+0000","last_undegraded":"2023-11-08T19:27:35.685308+0000","last_fullsized":"2023-11-08T19:27:35.685308+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.711744+0000","last_change":"2023-11-08T19:27:36.708435+0000","last_active":"2023-11-08T19:27:36.711744+0000","last_peered":"2023-11-08T19:27:36.711744+0000","last_clean":"2023-11-08T19:27:36.711744+0000","last_became_active":"2023-11-08T19:27:36.707772+0000","last_became_peered":"2023-11-08T19:27:36.707772+0000","last_unstale":"2023-11-08T19:27:36.711744+0000","last_undegraded":"2023-11-08T19:27:36.711744+0000","last_fullsized":"2023-11-08T19:27:36.711744+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:17.910713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.687317+0000","last_change":"2023-11-08T19:27:35.687556+0000","last_active":"2023-11-08T19:27:36.687317+0000","last_peered":"2023-11-08T19:27:36.687317+0000","last_clean":"2023-11-08T19:27:36.687317+0000","last_became_active":"2023-11-08T19:27:35.687309+0000","last_became_peered":"2023-11-08T19:27:35.687309+0000","last_unstale":"2023-11-08T19:27:36.687317+0000","last_undegraded":"2023-11-08T19:27:36.687317+0000","last_fullsized":"2023-11-08T19:27:36.687317+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:06.078832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":35,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.692643+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:27:35.692643+0000","last_peered":"2023-11-08T19:27:35.692643+0000","last_clean":"2023-11-08T19:27:35.692643+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:27:35.692643+0000","last_undegraded":"2023-11-08T19:27:35.692643+0000","last_fullsized":"2023-11-08T19:27:35.692643+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707316+0000","last_change":"2023-11-08T19:27:36.702623+0000","last_active":"2023-11-08T19:27:36.707316+0000","last_peered":"2023-11-08T19:27:36.707316+0000","last_clean":"2023-11-08T19:27:36.707316+0000","last_became_active":"2023-11-08T19:27:36.702192+0000","last_became_peered":"2023-11-08T19:27:36.702192+0000","last_unstale":"2023-11-08T19:27:36.707316+0000","last_undegraded":"2023-11-08T19:27:36.707316+0000","last_fullsized":"2023-11-08T19:27:36.707316+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:21.818446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.677767+0000","last_change":"2023-11-08T19:27:35.698475+0000","last_active":"2023-11-08T19:27:36.677767+0000","last_peered":"2023-11-08T19:27:36.677767+0000","last_clean":"2023-11-08T19:27:36.677767+0000","last_became_active":"2023-11-08T19:27:35.697213+0000","last_became_peered":"2023-11-08T19:27:35.697213+0000","last_unstale":"2023-11-08T19:27:36.677767+0000","last_undegraded":"2023-11-08T19:27:36.677767+0000","last_fullsized":"2023-11-08T19:27:36.677767+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:36.062749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.686441+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:27:35.686441+0000","last_peered":"2023-11-08T19:27:35.686441+0000","last_clean":"2023-11-08T19:27:35.686441+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:27:35.686441+0000","last_undegraded":"2023-11-08T19:27:35.686441+0000","last_fullsized":"2023-11-08T19:27:35.686441+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.701615+0000","last_change":"2023-11-08T19:27:36.695934+0000","last_active":"2023-11-08T19:27:36.701615+0000","last_peered":"2023-11-08T19:27:36.701615+0000","last_clean":"2023-11-08T19:27:36.701615+0000","last_became_active":"2023-11-08T19:27:36.695599+0000","last_became_peered":"2023-11-08T19:27:36.695599+0000","last_unstale":"2023-11-08T19:27:36.701615+0000","last_undegraded":"2023-11-08T19:27:36.701615+0000","last_fullsized":"2023-11-08T19:27:36.701615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:45.623034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.689911+0000","last_change":"2023-11-08T19:27:35.692549+0000","last_active":"2023-11-08T19:27:36.689911+0000","last_peered":"2023-11-08T19:27:36.689911+0000","last_clean":"2023-11-08T19:27:36.689911+0000","last_became_active":"2023-11-08T19:27:35.692142+0000","last_became_peered":"2023-11-08T19:27:35.692142+0000","last_unstale":"2023-11-08T19:27:36.689911+0000","last_undegraded":"2023-11-08T19:27:36.689911+0000","last_fullsized":"2023-11-08T19:27:36.689911+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:16:12.271555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.690611+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:27:35.690611+0000","last_peered":"2023-11-08T19:27:35.690611+0000","last_clean":"2023-11-08T19:27:35.690611+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:27:35.690611+0000","last_undegraded":"2023-11-08T19:27:35.690611+0000","last_fullsized":"2023-11-08T19:27:35.690611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.713202+0000","last_change":"2023-11-08T19:27:36.704909+0000","last_active":"2023-11-08T19:27:36.713202+0000","last_peered":"2023-11-08T19:27:36.713202+0000","last_clean":"2023-11-08T19:27:36.713202+0000","last_became_active":"2023-11-08T19:27:36.701608+0000","last_became_peered":"2023-11-08T19:27:36.701608+0000","last_unstale":"2023-11-08T19:27:36.713202+0000","last_undegraded":"2023-11-08T19:27:36.713202+0000","last_fullsized":"2023-11-08T19:27:36.713202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:44:30.319676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.700766+0000","last_change":"2023-11-08T19:27:35.702904+0000","last_active":"2023-11-08T19:27:36.700766+0000","last_peered":"2023-11-08T19:27:36.700766+0000","last_clean":"2023-11-08T19:27:36.700766+0000","last_became_active":"2023-11-08T19:27:35.702628+0000","last_became_peered":"2023-11-08T19:27:35.702628+0000","last_unstale":"2023-11-08T19:27:36.700766+0000","last_undegraded":"2023-11-08T19:27:36.700766+0000","last_fullsized":"2023-11-08T19:27:36.700766+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:10.680560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":37,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.695315+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:27:35.695315+0000","last_peered":"2023-11-08T19:27:35.695315+0000","last_clean":"2023-11-08T19:27:35.695315+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:27:35.695315+0000","last_undegraded":"2023-11-08T19:27:35.695315+0000","last_fullsized":"2023-11-08T19:27:35.695315+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.706584+0000","last_change":"2023-11-08T19:27:36.703037+0000","last_active":"2023-11-08T19:27:36.706584+0000","last_peered":"2023-11-08T19:27:36.706584+0000","last_clean":"2023-11-08T19:27:36.706584+0000","last_became_active":"2023-11-08T19:27:36.702729+0000","last_became_peered":"2023-11-08T19:27:36.702729+0000","last_unstale":"2023-11-08T19:27:36.706584+0000","last_undegraded":"2023-11-08T19:27:36.706584+0000","last_fullsized":"2023-11-08T19:27:36.706584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:32:14.030744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.727651+0000","last_change":"2023-11-08T19:27:35.701273+0000","last_active":"2023-11-08T19:27:36.727651+0000","last_peered":"2023-11-08T19:27:36.727651+0000","last_clean":"2023-11-08T19:27:36.727651+0000","last_became_active":"2023-11-08T19:27:35.700942+0000","last_became_peered":"2023-11-08T19:27:35.700942+0000","last_unstale":"2023-11-08T19:27:36.727651+0000","last_undegraded":"2023-11-08T19:27:36.727651+0000","last_fullsized":"2023-11-08T19:27:36.727651+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:16:36.535291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.698342+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:27:35.698342+0000","last_peered":"2023-11-08T19:27:35.698342+0000","last_clean":"2023-11-08T19:27:35.698342+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:27:35.698342+0000","last_undegraded":"2023-11-08T19:27:35.698342+0000","last_fullsized":"2023-11-08T19:27:35.698342+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.723336+0000","last_change":"2023-11-08T19:27:36.715148+0000","last_active":"2023-11-08T19:27:36.723336+0000","last_peered":"2023-11-08T19:27:36.723336+0000","last_clean":"2023-11-08T19:27:36.723336+0000","last_became_active":"2023-11-08T19:27:36.714653+0000","last_became_peered":"2023-11-08T19:27:36.714653+0000","last_unstale":"2023-11-08T19:27:36.723336+0000","last_undegraded":"2023-11-08T19:27:36.723336+0000","last_fullsized":"2023-11-08T19:27:36.723336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:06.293100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.726283+0000","last_change":"2023-11-08T19:27:35.702803+0000","last_active":"2023-11-08T19:27:36.726283+0000","last_peered":"2023-11-08T19:27:36.726283+0000","last_clean":"2023-11-08T19:27:36.726283+0000","last_became_active":"2023-11-08T19:27:35.701996+0000","last_became_peered":"2023-11-08T19:27:35.701996+0000","last_unstale":"2023-11-08T19:27:36.726283+0000","last_undegraded":"2023-11-08T19:27:36.726283+0000","last_fullsized":"2023-11-08T19:27:36.726283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:02:20.423465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.683637+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:27:35.683637+0000","last_peered":"2023-11-08T19:27:35.683637+0000","last_clean":"2023-11-08T19:27:35.683637+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:27:35.683637+0000","last_undegraded":"2023-11-08T19:27:35.683637+0000","last_fullsized":"2023-11-08T19:27:35.683637+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.703854+0000","last_change":"2023-11-08T19:27:36.701630+0000","last_active":"2023-11-08T19:27:36.703854+0000","last_peered":"2023-11-08T19:27:36.703854+0000","last_clean":"2023-11-08T19:27:36.703854+0000","last_became_active":"2023-11-08T19:27:36.697730+0000","last_became_peered":"2023-11-08T19:27:36.697730+0000","last_unstale":"2023-11-08T19:27:36.703854+0000","last_undegraded":"2023-11-08T19:27:36.703854+0000","last_fullsized":"2023-11-08T19:27:36.703854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:34.731696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"64'1","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725710+0000","last_change":"2023-11-08T19:27:35.695294+0000","last_active":"2023-11-08T19:27:36.725710+0000","last_peered":"2023-11-08T19:27:36.725710+0000","last_clean":"2023-11-08T19:27:36.725710+0000","last_became_active":"2023-11-08T19:27:35.694816+0000","last_became_peered":"2023-11-08T19:27:35.694816+0000","last_unstale":"2023-11-08T19:27:36.725710+0000","last_undegraded":"2023-11-08T19:27:36.725710+0000","last_fullsized":"2023-11-08T19:27:36.725710+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:28.540986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":34,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.685387+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:27:35.685387+0000","last_peered":"2023-11-08T19:27:35.685387+0000","last_clean":"2023-11-08T19:27:35.685387+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:27:35.685387+0000","last_undegraded":"2023-11-08T19:27:35.685387+0000","last_fullsized":"2023-11-08T19:27:35.685387+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.707277+0000","last_change":"2023-11-08T19:27:36.702426+0000","last_active":"2023-11-08T19:27:36.707277+0000","last_peered":"2023-11-08T19:27:36.707277+0000","last_clean":"2023-11-08T19:27:36.707277+0000","last_became_active":"2023-11-08T19:27:36.702158+0000","last_became_peered":"2023-11-08T19:27:36.702158+0000","last_unstale":"2023-11-08T19:27:36.707277+0000","last_undegraded":"2023-11-08T19:27:36.707277+0000","last_fullsized":"2023-11-08T19:27:36.707277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:35.647012+0000","last_clean_scrub_stamp":"2023-11-08T19:27:35.647012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:29.225198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"64'2","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-08T19:27:36.725281+0000","last_change":"2023-11-08T19:27:35.694909+0000","last_active":"2023-11-08T19:27:36.725281+0000","last_peered":"2023-11-08T19:27:36.725281+0000","last_clean":"2023-11-08T19:27:36.725281+0000","last_became_active":"2023-11-08T19:27:35.694644+0000","last_became_peered":"2023-11-08T19:27:35.694644+0000","last_unstale":"2023-11-08T19:27:36.725281+0000","last_undegraded":"2023-11-08T19:27:36.725281+0000","last_fullsized":"2023-11-08T19:27:36.725281+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:34.640596+0000","last_clean_scrub_stamp":"2023-11-08T19:27:34.640596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:07:07.949148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.686174+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:27:35.686174+0000","last_peered":"2023-11-08T19:27:35.686174+0000","last_clean":"2023-11-08T19:27:35.686174+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:27:35.686174+0000","last_undegraded":"2023-11-08T19:27:35.686174+0000","last_fullsized":"2023-11-08T19:27:35.686174+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":63,"state":"active+clean","last_fresh":"2023-11-08T19:27:35.689040+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:27:35.689040+0000","last_peered":"2023-11-08T19:27:35.689040+0000","last_clean":"2023-11-08T19:27:35.689040+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:27:35.689040+0000","last_undegraded":"2023-11-08T19:27:35.689040+0000","last_fullsized":"2023-11-08T19:27:35.689040+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1355776,"data_stored":940948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.087,"15min":1.1040000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4930000000000001,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.29,"15min":1.325},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.4450000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":1.2949999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.232,"15min":1.2689999999999999},"min":{"1min":0.69099999999999995,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.6619999999999999,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.1060000000000001,"15min":1.1299999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3160000000000001,"5min":1.681,"15min":1.681},"last":1.5469999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.466,"15min":1.5169999999999999},"min":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"max":{"1min":1.5089999999999999,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.133,"5min":1.349,"15min":1.385},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.772,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.751}]},{"osd":3,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.341,"15min":1.377},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.419,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.536,"15min":1.5800000000000001},"min":{"1min":0.95299999999999996,"5min":0.90300000000000002,"15min":0.90300000000000002},"max":{"1min":1.6439999999999999,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.6359999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.3560000000000001,"15min":1.3700000000000001},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":1.704,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.4690000000000001,"15min":1.5109999999999999},"min":{"1min":0.82799999999999996,"5min":0.82099999999999995,"15min":0.82099999999999995},"max":{"1min":1.5780000000000001,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.6539999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.4470000000000001,"15min":1.4650000000000001},"min":{"1min":1.004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.766,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4850000000000001,"15min":1.4910000000000001},"min":{"1min":1.1599999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"max":{"1min":1.9590000000000001,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":0.99399999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.5700000000000001,"15min":1.587},"min":{"1min":1.036,"5min":1.036,"15min":1.036},"max":{"1min":1.8680000000000001,"5min":2.427,"15min":2.427},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.6060000000000001,"15min":1.6439999999999999},"min":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.746,"5min":2.468,"15min":2.468},"last":1.569}]}]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":0.95399999999999996,"15min":0.92900000000000005},"min":{"1min":0.63400000000000001,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1180000000000001,"15min":1.107},"min":{"1min":0.83399999999999996,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.3580000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.0900000000000001,"15min":1.089},"min":{"1min":0.747,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.534,"5min":1.679,"15min":1.679},"last":1.218},{"interface":"front","average":{"1min":1.194,"5min":1.331,"15min":1.3540000000000001},"min":{"1min":0.76400000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.591,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.89300000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.274,"15min":1.304},"min":{"1min":0.625,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4990000000000001,"5min":4.609,"15min":4.609},"last":1.411},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2210000000000001,"15min":1.214},"min":{"1min":0.751,"5min":0.751,"15min":0.751},"max":{"1min":1.5609999999999999,"5min":1.861,"15min":1.861},"last":1.024}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3540000000000001,"5min":1.482,"15min":1.5029999999999999},"min":{"1min":0.877,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.8959999999999999,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.496},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.179,"15min":1.1799999999999999},"min":{"1min":0.80300000000000005,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.663,"5min":1.784,"15min":1.784},"last":1.3220000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.383,"15min":1.381},"min":{"1min":0.89900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":2.024,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.5009999999999999,"15min":1.4970000000000001},"min":{"1min":1.04,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.577}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.2450000000000001,"15min":1.24},"min":{"1min":0.98199999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7749999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.347,"5min":1.3320000000000001,"15min":1.3300000000000001},"min":{"1min":0.90200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.831,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.1579999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.587,"5min":1.5509999999999999,"15min":1.5449999999999999},"min":{"1min":1.097,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.1240000000000001,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.6339999999999999},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4419999999999999,"15min":1.45},"min":{"1min":0.96599999999999997,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.724,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.611}]}]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.0900000000000001,"15min":1.073},"min":{"1min":0.72299999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.712,"5min":1.76,"15min":1.76},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.246,"15min":1.2390000000000001},"min":{"1min":0.879,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.645,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4510000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.167,"15min":1.179},"min":{"1min":0.82399999999999995,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.401,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.405,"15min":1.462},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.4970000000000001,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.4890000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1799999999999999,"15min":1.171},"min":{"1min":0.91500000000000004,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.8700000000000001,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.4650000000000001,"15min":1.5229999999999999},"min":{"1min":0.76100000000000001,"5min":0.747,"15min":0.747},"max":{"1min":1.5900000000000001,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.6220000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2969999999999999,"15min":1.294},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8200000000000001,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.367,"5min":1.3149999999999999,"15min":1.3069999999999999},"min":{"1min":0.89900000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7609999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2589999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.288,"15min":1.2909999999999999},"min":{"1min":0.76600000000000001,"5min":0.755,"15min":0.755},"max":{"1min":1.966,"5min":2.347,"15min":2.347},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.544,"15min":1.5629999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.0230000000000001,"5min":3.153,"15min":3.153},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.516,"15min":1.536},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.9199999999999999,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.4139999999999999,"5min":1.494,"15min":1.5069999999999999},"min":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"max":{"1min":1.869,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.518}]},{"osd":7,"last update":"Wed Nov 8 19:26:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.6830000000000001,"15min":1.7150000000000001},"min":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.1269999999999998,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.5589999999999999,"5min":1.587,"15min":1.5920000000000001},"min":{"1min":0.99399999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.994,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.4259999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574887,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.036,"15min":1.0409999999999999},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.5219999999999998,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.331},{"interface":"front","average":{"1min":1.492,"5min":1.248,"15min":1.2070000000000001},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":7.0119999999999996,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.256}]},{"osd":1,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.103,"15min":1.1020000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":2.9790000000000001,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.1279999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":1.121,"15min":1.163},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.2470000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.0900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3400000000000001,"15min":1.3520000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.8719999999999999,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.177,"5min":1.3959999999999999,"15min":1.4319999999999999},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.375,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.2150000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.3080000000000001,"15min":1.302},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.2909999999999999,"5min":1.379,"15min":1.393},"min":{"1min":0.91900000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":3.0470000000000002,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.2889999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.486,"15min":1.528},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":2.4630000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.5469999999999999,"15min":1.5660000000000001},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":4.5880000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.1679999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.458,"5min":1.5569999999999999,"15min":1.573},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":3.2360000000000002,"5min":4.335,"15min":4.335},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.4850000000000001,"5min":1.613,"15min":1.6339999999999999},"min":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":4.4130000000000003,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.492}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.609,"5min":1.637,"15min":1.6419999999999999},"min":{"1min":0.95499999999999996,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":4.5490000000000004,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.7170000000000001,"15min":1.7430000000000001},"min":{"1min":1.095,"5min":1.095,"15min":1.095},"max":{"1min":3.1389999999999998,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.625}]}]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":1.056,"15min":1.0960000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.2909999999999999,"5min":3.972,"15min":3.972},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":1.0649999999999999,"15min":1.1060000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.3999999999999999,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.88500000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":1.1679999999999999,"15min":1.2070000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.6839999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":1.0900000000000001,"15min":1.121},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.3360000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":1.073}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.3089999999999999,"15min":1.3380000000000001},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.752,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.256,"15min":1.282},"min":{"1min":0.46100000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.778,"5min":2.73,"15min":2.73},"last":1.48}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.4350000000000001,"15min":1.4690000000000001},"min":{"1min":0.872,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.861,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.052},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.448,"15min":1.47},"min":{"1min":0.91400000000000003,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5980000000000001,"5min":3.052,"15min":3.052},"last":1.3959999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.4750000000000001,"15min":1.5109999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.149,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.331,"5min":1.5169999999999999,"15min":1.548},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.375,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.167}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3400000000000001,"5min":1.4239999999999999,"15min":1.4379999999999999},"min":{"1min":1.054,"5min":0.878,"15min":0.878},"max":{"1min":1.8220000000000001,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.5740000000000001,"15min":1.6000000000000001},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.4860000000000002,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.131}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4359999999999999,"5min":1.6930000000000001,"15min":1.736},"min":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"max":{"1min":2.528,"5min":4.407,"15min":4.407},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.573,"15min":1.5960000000000001},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8740000000000001,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.2390000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574887,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":757760,"data_stored":349561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":0.86899999999999999},"min":{"1min":0.41299999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.222,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.052},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.90200000000000002,"15min":0.93400000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.206,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.94599999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.36399999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.202,"5min":1.393,"15min":1.393},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.02},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.177,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.1479999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89700000000000002,"5min":1.03,"15min":1.052},"min":{"1min":0.51300000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.486,"5min":1.53,"15min":1.53},"last":1.3029999999999999},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.099,"15min":1.139},"min":{"1min":0.48099999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.3859999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.4450000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.3600000000000001,"15min":1.421},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.3100000000000001,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.8},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.3360000000000001,"15min":1.3779999999999999},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.506,"5min":2.278,"15min":2.278},"last":1.2050000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.389,"15min":1.4350000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.6180000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":2.1600000000000001},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.4219999999999999,"15min":1.4770000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.456,"5min":2.536,"15min":2.536},"last":1.2629999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.363,"15min":1.395},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.5409999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.1499999999999999,"5min":1.4219999999999999,"15min":1.468},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.8480000000000001,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":2.036}]},{"osd":7,"last update":"Wed Nov 8 19:26:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.5389999999999999,"15min":1.593},"min":{"1min":0.81399999999999995,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.5329999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.5289999999999999,"15min":1.5680000000000001},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.5840000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.8839999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344076,"statfs":{"total":96636764160,"available":96608333824,"internally_reserved":0,"allocated":761856,"data_stored":350580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.745,"15min":0.74199999999999999},"min":{"1min":0.50600000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.626},{"interface":"front","average":{"1min":0.749,"5min":0.753,"15min":0.754},"min":{"1min":0.38400000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.0369999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79100000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.1339999999999999,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.88300000000000001,"15min":0.88600000000000001},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.224,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0580000000000001,"15min":1.0569999999999999},"min":{"1min":0.60499999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":1.276},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0680000000000001,"15min":1.0860000000000001},"min":{"1min":0.499,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.8540000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.323}]},{"osd":4,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1559999999999999,"15min":1.157},"min":{"1min":0.66700000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.542,"5min":2.165,"15min":2.165},"last":1.248},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.2410000000000001,"15min":1.2569999999999999},"min":{"1min":0.73799999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4910000000000001,"5min":1.885,"15min":1.885},"last":1.069}]},{"osd":5,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.254,"15min":1.264},"min":{"1min":0.71099999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7390000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1599999999999999,"5min":1.3080000000000001,"15min":1.3320000000000001},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.706,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.1719999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3129999999999999,"5min":1.359,"15min":1.367},"min":{"1min":0.754,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.772,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.425},{"interface":"front","average":{"1min":1.294,"5min":1.373,"15min":1.3859999999999999},"min":{"1min":0.92000000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.6930000000000001,"5min":2.282,"15min":2.282},"last":1.018}]},{"osd":7,"last update":"Wed Nov 8 19:26:49 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.3169999999999999,"15min":1.329},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.6559999999999999,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.371},{"interface":"front","average":{"1min":1.262,"5min":1.3740000000000001,"15min":1.3919999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.956,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.2070000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":939929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.76800000000000002,"15min":0.77900000000000003},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.0649999999999999,"5min":1.357,"15min":1.357},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.92600000000000005,"15min":0.94599999999999995},"min":{"1min":0.55400000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.2889999999999999,"5min":1.7,"15min":1.7},"last":0.995}]},{"osd":2,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.85099999999999998,"15min":0.873},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.2410000000000001,"5min":1.508,"15min":1.508},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.98199999999999998},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.5660000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.05}]},{"osd":3,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":1,"15min":1.012},"min":{"1min":0.55900000000000005,"5min":0.44,"15min":0.44},"max":{"1min":1.278,"5min":1.748,"15min":1.748},"last":0.875},{"interface":"front","average":{"1min":0.92400000000000004,"5min":1.0880000000000001,"15min":1.115},"min":{"1min":0.53500000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.4470000000000001,"5min":2.496,"15min":2.496},"last":1.3120000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2290000000000001,"15min":1.2310000000000001},"min":{"1min":1.022,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.5169999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.169,"5min":1.4159999999999999,"15min":1.4570000000000001},"min":{"1min":0.81699999999999995,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.8700000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.2170000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.351,"15min":1.3819999999999999},"min":{"1min":0.74099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.9510000000000001,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.427,"15min":1.4570000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.7070000000000001,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.522}]},{"osd":6,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.4830000000000001,"15min":1.516},"min":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"max":{"1min":1.8160000000000001,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.3999999999999999,"15min":1.4199999999999999},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.6259999999999999,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4810000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:26:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3899999999999999,"15min":1.4059999999999999},"min":{"1min":0.92600000000000005,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.746,"5min":2.173,"15min":2.173},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.4590000000000001,"15min":1.4950000000000001},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.0070000000000001,"5min":2.306,"15min":2.306},"last":1.419}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:27:41.674 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:27:41.675 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.930+0000 7f679740e640 1 Processor -- start 2023-11-08T19:27:41.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.931+0000 7f679740e640 1 -- start start 2023-11-08T19:27:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.932+0000 7f679740e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790144580 0x7f679015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.932+0000 7f679740e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6790144960 0x7f67900cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.932+0000 7f679740e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679015ea60 con 0x7f679015f1d0 2023-11-08T19:27:41.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.932+0000 7f679740e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67900c9150 con 0x7f6790144960 2023-11-08T19:27:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.932+0000 7f679740e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900cb270 con 0x7f6790144580 2023-11-08T19:27:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67959ab640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f679015f1d0 0x7f67900c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41818/0 (socket says 172.21.15.119:41818) 2023-11-08T19:27:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67959ab640 1 -- 172.21.15.119:0/3064144365 learned_addr learned my addr 172.21.15.119:0/3064144365 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:41.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67949a9640 1 --2- 172.21.15.119:0/3064144365 >> v2:172.21.15.119:3300/0 conn(0x7f6790144960 0x7f67900cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67951aa640 1 --2- 172.21.15.119:0/3064144365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790144580 0x7f679015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f6787fff640 1 -- 172.21.15.119:0/3064144365 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389060207 0 0) 0x7f679015ea60 con 0x7f679015f1d0 2023-11-08T19:27:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67949a9640 1 -- 172.21.15.119:0/3064144365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790144580 msgr2=0x7f679015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:41.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67949a9640 1 --2- 172.21.15.119:0/3064144365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790144580 0x7f679015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67949a9640 1 -- 172.21.15.119:0/3064144365 >> v1:172.21.15.119:6789/0 conn(0x7f679015f1d0 legacy=0x7f67900c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.933+0000 7f67949a9640 1 -- 172.21.15.119:0/3064144365 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900c9360 con 0x7f6790144960 2023-11-08T19:27:41.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.934+0000 7f67949a9640 1 --2- 172.21.15.119:0/3064144365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f67900cad30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f67800097a0 tx=0x7f678002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b7fdf9d629a75 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.934+0000 7f6787fff640 1 -- 172.21.15.119:0/3064144365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780031780 con 0x7f6790144960 2023-11-08T19:27:41.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f6787fff640 1 -- 172.21.15.119:0/3064144365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67800318e0 con 0x7f6790144960 2023-11-08T19:27:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f6787fff640 1 -- 172.21.15.119:0/3064144365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780031c70 con 0x7f6790144960 2023-11-08T19:27:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 -- 172.21.15.119:0/3064144365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 msgr2=0x7f67900cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 --2- 172.21.15.119:0/3064144365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f67900cad30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f67800097a0 tx=0x7f678002b820 comp rx=0 tx=0).stop 2023-11-08T19:27:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 -- 172.21.15.119:0/3064144365 shutdown_connections 2023-11-08T19:27:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 --2- 172.21.15.119:0/3064144365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f67900cad30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 --2- 172.21.15.119:0/3064144365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6790144580 0x7f679015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 -- 172.21.15.119:0/3064144365 >> 172.21.15.119:0/3064144365 conn(0x7f67900cb610 msgr2=0x7f67900cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 -- 172.21.15.119:0/3064144365 shutdown_connections 2023-11-08T19:27:41.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.935+0000 7f679740e640 1 -- 172.21.15.119:0/3064144365 wait complete. 2023-11-08T19:27:41.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.937+0000 7f679740e640 1 Processor -- start 2023-11-08T19:27:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.937+0000 7f679740e640 1 -- start start 2023-11-08T19:27:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f679740e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790144580 0x7f679015ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f679740e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f679013d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f679740e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015f1d0 0x7f67900c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f679740e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900c5cd0 con 0x7f6790144960 2023-11-08T19:27:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f679740e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679013e990 con 0x7f6790144580 2023-11-08T19:27:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f679740e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679013ec70 con 0x7f679015f1d0 2023-11-08T19:27:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f67949a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f679013d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f67949a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f679013d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54828/0 (socket says 172.21.15.119:54828) 2023-11-08T19:27:41.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.938+0000 7f67949a9640 1 -- 172.21.15.119:0/1265825617 learned_addr learned my addr 172.21.15.119:0/1265825617 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67951aa640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790144580 0x7f679015ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67959ab640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015f1d0 0x7f67900c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67949a9640 1 -- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015f1d0 msgr2=0x7f67900c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67949a9640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015f1d0 0x7f67900c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67949a9640 1 -- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790144580 msgr2=0x7f679015ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67949a9640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790144580 0x7f679015ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67949a9640 1 -- 172.21.15.119:0/1265825617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6780009400 con 0x7f6790144960 2023-11-08T19:27:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67959ab640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015f1d0 0x7f67900c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67951aa640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790144580 0x7f679015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:41.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.939+0000 7f67949a9640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f679013d6a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6780034040 tx=0x7f67800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.940+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780032820 con 0x7f6790144960 2023-11-08T19:27:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.940+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780032980 con 0x7f6790144960 2023-11-08T19:27:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.940+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780032d30 con 0x7f6790144960 2023-11-08T19:27:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.940+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679013ef50 con 0x7f6790144960 2023-11-08T19:27:41.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.940+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679013dbe0 con 0x7f6790144960 2023-11-08T19:27:41.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.941+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67900c5e10 con 0x7f6790144960 2023-11-08T19:27:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.942+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f678004e440 con 0x7f6790144960 2023-11-08T19:27:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.943+0000 7f6785ffb640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f677406ae70 0x7f677406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.943+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f67800aba30 con 0x7f6790144960 2023-11-08T19:27:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.943+0000 7f67951aa640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f677406ae70 0x7f677406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:41.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.944+0000 7f67951aa640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f677406ae70 0x7f677406d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f678c006fd0 tx=0x7f678c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:41.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:41.948+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f67900c5e10 con 0x7f6790144960 2023-11-08T19:27:42.183 INFO:tasks.ceph.mds.d.smithi188.stderr:2023-11-08T19:27:42.181+0000 7f832698e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:27:42.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.206+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f67900c5e10 con 0x7f6790144960 2023-11-08T19:27:42.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.209+0000 7f6785ffb640 1 -- 172.21.15.119:0/1265825617 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f67900c5e10 con 0x7f6790144960 2023-11-08T19:27:42.211 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-11-08T19:27:42.211 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:27:42.211 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2023-11-08T19:27:42.212 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2023-11-08T19:27:42.212 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:27:42.212 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-11-08T19:27:42.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f677406ae70 msgr2=0x7f677406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f677406ae70 0x7f677406d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f678c006fd0 tx=0x7f678c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:42.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 msgr2=0x7f679013d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f679013d6a0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f6780034040 tx=0x7f67800046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 shutdown_connections 2023-11-08T19:27:42.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f677406ae70 0x7f677406d330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f679015f1d0 0x7f67900c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6790144960 0x7f679013d6a0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 --2- 172.21.15.119:0/1265825617 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6790144580 0x7f679015ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.212+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 >> 172.21.15.119:0/1265825617 conn(0x7f67900cb610 msgr2=0x7f679006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.213+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 shutdown_connections 2023-11-08T19:27:42.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.213+0000 7f679740e640 1 -- 172.21.15.119:0/1265825617 wait complete. 2023-11-08T19:27:42.232 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:42.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.447+0000 7f60f8e90640 1 Processor -- start 2023-11-08T19:27:42.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.447+0000 7f60f8e90640 1 -- start start 2023-11-08T19:27:42.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f8e90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f41447a0 0x7f60f415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f8e90640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60f40cb170 0x7f60f406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f8e90640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f40c3a50 con 0x7f60f40cad30 2023-11-08T19:27:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f8e90640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f4070040 con 0x7f60f40cb170 2023-11-08T19:27:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f8e90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4144b80 con 0x7f60f41447a0 2023-11-08T19:27:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f2d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f60f40cb170 0x7f60f406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f1d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f60f40cad30 0x7f60f406f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41820/0 (socket says 172.21.15.119:41820) 2023-11-08T19:27:42.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.449+0000 7f60f1d9b640 1 -- 172.21.15.119:0/93847018 learned_addr learned my addr 172.21.15.119:0/93847018 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.450+0000 7f60f259c640 1 --2- 172.21.15.119:0/93847018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f41447a0 0x7f60f415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.450+0000 7f60f2d9d640 1 -- 172.21.15.119:0/93847018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f41447a0 msgr2=0x7f60f415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.450+0000 7f60f2d9d640 1 --2- 172.21.15.119:0/93847018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f41447a0 0x7f60f415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.450+0000 7f60f2d9d640 1 -- 172.21.15.119:0/93847018 >> v1:172.21.15.119:6789/0 conn(0x7f60f40cad30 legacy=0x7f60f406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.450+0000 7f60f2d9d640 1 -- 172.21.15.119:0/93847018 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f40c9150 con 0x7f60f40cb170 2023-11-08T19:27:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.450+0000 7f60f159a640 1 -- 172.21.15.119:0/93847018 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2439643762 0 0) 0x7f60f40c3a50 con 0x7f60f40cad30 2023-11-08T19:27:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.451+0000 7f60f2d9d640 1 --2- 172.21.15.119:0/93847018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f406a120 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009d90 tx=0x7f60ec02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fccfbfd1661143 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.451+0000 7f60f159a640 1 -- 172.21.15.119:0/93847018 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec02cc50 con 0x7f60f40cb170 2023-11-08T19:27:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.451+0000 7f60f159a640 1 -- 172.21.15.119:0/93847018 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec02cdb0 con 0x7f60f40cb170 2023-11-08T19:27:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.451+0000 7f60f159a640 1 -- 172.21.15.119:0/93847018 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec034660 con 0x7f60f40cb170 2023-11-08T19:27:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 -- 172.21.15.119:0/93847018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 msgr2=0x7f60f406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 --2- 172.21.15.119:0/93847018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f406a120 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009d90 tx=0x7f60ec02bea0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 -- 172.21.15.119:0/93847018 shutdown_connections 2023-11-08T19:27:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 --2- 172.21.15.119:0/93847018 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f406a120 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 --2- 172.21.15.119:0/93847018 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f41447a0 0x7f60f415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 -- 172.21.15.119:0/93847018 >> 172.21.15.119:0/93847018 conn(0x7f60f40cb610 msgr2=0x7f60f40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 -- 172.21.15.119:0/93847018 shutdown_connections 2023-11-08T19:27:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.452+0000 7f60f8e90640 1 -- 172.21.15.119:0/93847018 wait complete. 2023-11-08T19:27:42.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.454+0000 7f60f8e90640 1 Processor -- start 2023-11-08T19:27:42.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.455+0000 7f60f8e90640 1 -- start start 2023-11-08T19:27:42.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f8e90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f40cad30 0x7f60f40c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f8e90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f41f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f8e90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f41447a0 0x7f60f40c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f8e90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40b2e20 con 0x7f60f40cb170 2023-11-08T19:27:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f8e90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40b3100 con 0x7f60f41447a0 2023-11-08T19:27:42.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f8e90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40b33e0 con 0x7f60f40cad30 2023-11-08T19:27:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f41f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f41f2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54840/0 (socket says 172.21.15.119:54840) 2023-11-08T19:27:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f1d9b640 1 -- 172.21.15.119:0/3162137858 learned_addr learned my addr 172.21.15.119:0/3162137858 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f259c640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f40cad30 0x7f60f40c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f2d9d640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f41447a0 0x7f60f40c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f1d9b640 1 -- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f40cad30 msgr2=0x7f60f40c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.456+0000 7f60f1d9b640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f40cad30 0x7f60f40c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60f1d9b640 1 -- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f41447a0 msgr2=0x7f60f40c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60f1d9b640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f41447a0 0x7f60f40c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60f1d9b640 1 -- 172.21.15.119:0/3162137858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec009a70 con 0x7f60f40cb170 2023-11-08T19:27:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60f1d9b640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f41f2db0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c4d0 tx=0x7f60dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0074e0 con 0x7f60f40cb170 2023-11-08T19:27:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc007660 con 0x7f60f40cb170 2023-11-08T19:27:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc004a20 con 0x7f60f40cb170 2023-11-08T19:27:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40b36c0 con 0x7f60f40cb170 2023-11-08T19:27:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.457+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f41f32f0 con 0x7f60f40cb170 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.458+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f4074430 con 0x7f60f40cb170 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.459+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f60dc004b80 con 0x7f60f40cb170 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.460+0000 7f60eaffd640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f60d406ae70 0x7f60d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.460+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f60dc088a90 con 0x7f60f40cb170 2023-11-08T19:27:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.461+0000 7f60f259c640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f60d406ae70 0x7f60d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:42.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.461+0000 7f60f259c640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f60d406ae70 0x7f60d406d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f60e4006fd0 tx=0x7f60e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:42.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.465+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f60dc053860 con 0x7f60f40cb170 2023-11-08T19:27:42.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.689+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60f406caa0 con 0x7f60f40cb170 2023-11-08T19:27:42.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.690+0000 7f60eaffd640 1 -- 172.21.15.119:0/3162137858 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f60dc018020 con 0x7f60f40cb170 2023-11-08T19:27:42.694 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:42.694 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:36.122432+0000","modified":"2023-11-08T19:27:39.195905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/913059196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":913059196},{"type":"v1","addr":"172.21.15.188:6835","nonce":913059196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2023-11-08T19:27:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f60d406ae70 msgr2=0x7f60d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f60d406ae70 0x7f60d406d330 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f60e4006fd0 tx=0x7f60e4008040 comp rx=0 tx=0).stop 2023-11-08T19:27:42.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 msgr2=0x7f60f41f2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f41f2db0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f60dc00c4d0 tx=0x7f60dc00c990 comp rx=0 tx=0).stop 2023-11-08T19:27:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 shutdown_connections 2023-11-08T19:27:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f60d406ae70 0x7f60d406d330 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f60f41447a0 0x7f60f40c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f60f40cb170 0x7f60f41f2db0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 --2- 172.21.15.119:0/3162137858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f60f40cad30 0x7f60f40c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 >> 172.21.15.119:0/3162137858 conn(0x7f60f40cb610 msgr2=0x7f60f40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 shutdown_connections 2023-11-08T19:27:42.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:42.693+0000 7f60f8e90640 1 -- 172.21.15.119:0/3162137858 wait complete. 2023-11-08T19:27:43.116 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:27:36.122432+0000', 'modified': '2023-11-08T19:27:39.195905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6147}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6147': {'gid': 6147, 'name': 'd', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/913059196', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6834', 'nonce': 913059196}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 913059196}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:27:43.117 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:27:43.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:27:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.336+0000 7f2c33839640 1 Processor -- start 2023-11-08T19:27:43.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.336+0000 7f2c33839640 1 -- start start 2023-11-08T19:27:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.337+0000 7f2c33839640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2c2c15f4c0 0x7f2c2c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.337+0000 7f2c33839640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 0x7f2c2c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.337+0000 7f2c33839640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c2c06aa90 con 0x7f2c2c1447a0 2023-11-08T19:27:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.337+0000 7f2c33839640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c2c15f930 con 0x7f2c2c15f4c0 2023-11-08T19:27:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.337+0000 7f2c33839640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c3a50 con 0x7f2c2c06a120 2023-11-08T19:27:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.338+0000 7f2c315d5640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2c2c1447a0 0x7f2c2c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41836/0 (socket says 172.21.15.119:41836) 2023-11-08T19:27:43.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.338+0000 7f2c315d5640 1 -- 172.21.15.119:0/3491611585 learned_addr learned my addr 172.21.15.119:0/3491611585 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.338+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251406250 0 0) 0x7f2c2c06aa90 con 0x7f2c2c1447a0 2023-11-08T19:27:43.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.338+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c18003520 con 0x7f2c2c1447a0 2023-11-08T19:27:43.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.338+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 0x7f2c2c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.338+0000 7f2c30dd4640 1 --2- 172.21.15.119:0/3491611585 >> v2:172.21.15.188:3300/0 conn(0x7f2c2c15f4c0 0x7f2c2c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3470111061 0 0) 0x7f2c18003520 con 0x7f2c2c1447a0 2023-11-08T19:27:43.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 msgr2=0x7f2c2c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 0x7f2c2c06a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 >> v2:172.21.15.188:3300/0 conn(0x7f2c2c15f4c0 msgr2=0x7f2c2c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 --2- 172.21.15.119:0/3491611585 >> v2:172.21.15.188:3300/0 conn(0x7f2c2c15f4c0 0x7f2c2c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c144b80 con 0x7f2c2c1447a0 2023-11-08T19:27:43.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2c28001670 con 0x7f2c2c1447a0 2023-11-08T19:27:43.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 >> v1:172.21.15.119:6789/0 conn(0x7f2c2c1447a0 legacy=0x7f2c2c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c18003970 0x7f2c18003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 0x7f2c180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c18004d00 0x7f2c18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c15f930 con 0x7f2c18004d00 2023-11-08T19:27:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0c3a50 con 0x7f2c18003970 2023-11-08T19:27:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1800bab0 con 0x7f2c2c06a120 2023-11-08T19:27:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c18004d00 0x7f2c18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c30dd4640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 0x7f2c180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.339+0000 7f2c315d5640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c18003970 0x7f2c18003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.340+0000 7f2c31dd6640 1 -- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 msgr2=0x7f2c180047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.340+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c06a120 0x7f2c180047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.340+0000 7f2c31dd6640 1 -- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c18003970 msgr2=0x7f2c18003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.340+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c18003970 0x7f2c18003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.340+0000 7f2c31dd6640 1 -- 172.21.15.119:0/3491611585 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18003520 con 0x7f2c18004d00 2023-11-08T19:27:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.341+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c18004d00 0x7f2c18005180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007360 tx=0x7f2c180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc8d1311318d4cc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.341+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2402ccd0 con 0x7f2c18004d00 2023-11-08T19:27:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.341+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2402ce30 con 0x7f2c18004d00 2023-11-08T19:27:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.341+0000 7f2c23fff640 1 -- 172.21.15.119:0/3491611585 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24034660 con 0x7f2c18004d00 2023-11-08T19:27:43.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c33839640 1 -- 172.21.15.119:0/3491611585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c18004d00 msgr2=0x7f2c18005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c33839640 1 --2- 172.21.15.119:0/3491611585 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c18004d00 0x7f2c18005180 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007360 tx=0x7f2c180057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c31dd6640 1 -- 172.21.15.119:0/3491611585 reap_dead start 2023-11-08T19:27:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c33839640 1 -- 172.21.15.119:0/3491611585 shutdown_connections 2023-11-08T19:27:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c33839640 1 -- 172.21.15.119:0/3491611585 >> 172.21.15.119:0/3491611585 conn(0x7f2c2c0cb610 msgr2=0x7f2c2c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c33839640 1 -- 172.21.15.119:0/3491611585 shutdown_connections 2023-11-08T19:27:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.342+0000 7f2c33839640 1 -- 172.21.15.119:0/3491611585 wait complete. 2023-11-08T19:27:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.344+0000 7f2c33839640 1 Processor -- start 2023-11-08T19:27:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.344+0000 7f2c33839640 1 -- start start 2023-11-08T19:27:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c33839640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c15a770 0x7f2c2c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c33839640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c2c15ab50 0x7f2c2c0c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c33839640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 0x7f2c2c0c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c33839640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c0cde20 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c33839640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c13d900 con 0x7f2c2c15ab50 2023-11-08T19:27:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c33839640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c13dbe0 con 0x7f2c2c15a770 2023-11-08T19:27:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 0x7f2c2c0c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 0x7f2c2c0c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54856/0 (socket says 172.21.15.119:54856) 2023-11-08T19:27:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2417436353 learned_addr learned my addr 172.21.15.119:0/2417436353 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c30dd4640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c2c15ab50 0x7f2c2c0c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c315d5640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c15a770 0x7f2c2c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c15a770 msgr2=0x7f2c2c0cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c15a770 0x7f2c2c0cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c2c15ab50 msgr2=0x7f2c2c0c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c2c15ab50 0x7f2c2c0c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.345+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2417436353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c13dec0 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.346+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 0x7f2c2c0c6990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c670 tx=0x7f2c24004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.346+0000 7f2c21ffb640 1 -- 172.21.15.119:0/2417436353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2402ccd0 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.346+0000 7f2c21ffb640 1 -- 172.21.15.119:0/2417436353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c24003990 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.346+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c0cdb00 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.346+0000 7f2c21ffb640 1 -- 172.21.15.119:0/2417436353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24003d40 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.346+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c1f5fc0 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.348+0000 7f2c21ffb640 1 -- 172.21.15.119:0/2417436353 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c2403b040 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.350+0000 7f2c21ffb640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2c1006aea0 0x7f2c1006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.350+0000 7f2c21ffb640 1 -- 172.21.15.119:0/2417436353 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2c240aad00 con 0x7f2c2c0cc6d0 2023-11-08T19:27:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.350+0000 7f2c315d5640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2c1006aea0 0x7f2c1006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.350+0000 7f2c363cc740 1 Processor -- start 2023-11-08T19:27:43.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 -- start start 2023-11-08T19:27:43.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c315d5640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2c1006aea0 0x7f2c1006d360 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c28001200 tx=0x7f2c28001110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cbf2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc22c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cbf3300 0x560a2cc23190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cb657e0 con 0x560a2cc1fec0 2023-11-08T19:27:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cb62ed0 con 0x560a2cc202a0 2023-11-08T19:27:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.351+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a2cb5e7c0 con 0x560a2cbf3300 2023-11-08T19:27:43.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cbf2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cbf2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54872/0 (socket says 172.21.15.119:54872) 2023-11-08T19:27:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 -- 172.21.15.119:0/529354038 learned_addr learned my addr 172.21.15.119:0/529354038 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c30dd4640 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc22c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cbf3300 0x560a2cc23190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 -- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cbf3300 msgr2=0x560a2cc23190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cbf3300 0x560a2cc23190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 -- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 msgr2=0x560a2cc22c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc22c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 -- 172.21.15.119:0/529354038 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a2cb31b10 con 0x560a2cc1fec0 2023-11-08T19:27:43.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c30dd4640 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc22c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:43.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.352+0000 7f2c315d5640 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cbf2dc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800f810 tx=0x7f2c28012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21264ffd30023b6f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c16ffd640 1 -- 172.21.15.119:0/529354038 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28016110 con 0x560a2cc1fec0 2023-11-08T19:27:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c16ffd640 1 -- 172.21.15.119:0/529354038 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28016270 con 0x560a2cc1fec0 2023-11-08T19:27:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c16ffd640 1 -- 172.21.15.119:0/529354038 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28016bd0 con 0x560a2cc1fec0 2023-11-08T19:27:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 -- 172.21.15.119:0/529354038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 msgr2=0x560a2cbf2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cbf2dc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800f810 tx=0x7f2c28012000 comp rx=0 tx=0).stop 2023-11-08T19:27:43.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 -- 172.21.15.119:0/529354038 shutdown_connections 2023-11-08T19:27:43.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cbf3300 0x560a2cc23190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc22c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 --2- 172.21.15.119:0/529354038 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cbf2dc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 -- 172.21.15.119:0/529354038 >> 172.21.15.119:0/529354038 conn(0x560a2cbf50d0 msgr2=0x560a2cc3dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 -- 172.21.15.119:0/529354038 shutdown_connections 2023-11-08T19:27:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.353+0000 7f2c363cc740 1 -- 172.21.15.119:0/529354038 wait complete. 2023-11-08T19:27:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.354+0000 7f2c363cc740 1 -- start start 2023-11-08T19:27:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.354+0000 7f2c363cc740 1 Processor -- start 2023-11-08T19:27:43.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc1fec0 0x560a2cc259d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 0x560a2cc1f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cafbc40 con 0x560a2cc3e0a0 2023-11-08T19:27:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cafb8e0 con 0x560a2cc1fec0 2023-11-08T19:27:43.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a2c8c1f20 con 0x560a2cc202a0 2023-11-08T19:27:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.355+0000 7f2c363cc740 10 client.0 _ll_register_callbacks cb 0x560a2cb1dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc516d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54888/0 (socket says 172.21.15.119:54888) 2023-11-08T19:27:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 -- 172.21.15.119:0/1990762920 learned_addr learned my addr 172.21.15.119:0/1990762920 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:43.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c315d5640 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc1fec0 0x560a2cc259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 -- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 msgr2=0x560a2cc1f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 0x560a2cc1f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 -- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc1fec0 msgr2=0x560a2cc259d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc1fec0 0x560a2cc259d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 -- 172.21.15.119:0/1990762920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c144b80 con 0x560a2cc3e0a0 2023-11-08T19:27:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc516d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007360 tx=0x7f2c24079f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb58577edc75ba4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:27:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c0bfff640 1 -- 172.21.15.119:0/1990762920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24081a80 con 0x560a2cc3e0a0 2023-11-08T19:27:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c0bfff640 1 -- 172.21.15.119:0/1990762920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2c240799e0 con 0x560a2cc3e0a0 2023-11-08T19:27:43.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.356+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a2cb32c80 con 0x560a2cc3e0a0 2023-11-08T19:27:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c0bfff640 1 -- 172.21.15.119:0/1990762920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c24079ba0 con 0x560a2cc3e0a0 2023-11-08T19:27:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 10 client.6300 fetch_fsmap learned FSMap version 35 2023-11-08T19:27:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x560a2cb5ce40 con 0x560a2cc3e0a0 2023-11-08T19:27:43.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c0bfff640 1 -- 172.21.15.119:0/1990762920 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560a2cb32c80 con 0x560a2cc3e0a0 2023-11-08T19:27:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c0bfff640 1 -- 172.21.15.119:0/1990762920 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c24048060 con 0x560a2cc3e0a0 2023-11-08T19:27:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 10 client.6300 fetch_fsmap finished waiting for FSMap version 35 2023-11-08T19:27:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 10 client.6300 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:27:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 20 client.6300 populate_metadata read hostname 'smithi119' 2023-11-08T19:27:43.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cc3c330 0x560a2cc39a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 4 client.6300 mds_command: new command op to 6147 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:27:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.357+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 --> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a2cb577a0 con 0x560a2cc3c330 2023-11-08T19:27:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.358+0000 7f2c315d5640 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cc3c330 0x560a2cc39a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.359+0000 7f2c0bfff640 10 client.6300 ms_handle_connect on v2:172.21.15.188:6834/913059196 2023-11-08T19:27:43.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.359+0000 7f2c315d5640 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cc3c330 0x560a2cc39a80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ed6c3d1d04b7f2d server_cookie=d9cba332a963b5a3 in_seq=0 out_seq=0 2023-11-08T19:27:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c0bfff640 1 -- 172.21.15.119:0/1990762920 <== mds.0 v2:172.21.15.188:6834/913059196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560a2cb577a0 con 0x560a2cc3c330 2023-11-08T19:27:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c0bfff640 10 client.6300 handle_command_reply: tid=0 2023-11-08T19:27:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 client.6300 shutdown 2023-11-08T19:27:43.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 msgr2=0x560a2cc516d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc516d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f2c24007360 tx=0x7f2c24079f40 comp rx=0 tx=0).stop 2023-11-08T19:27:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 shutdown_connections 2023-11-08T19:27:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cc3c330 0x560a2cc39a80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc516d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 0x560a2cc1f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 --2- 172.21.15.119:0/1990762920 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc1fec0 0x560a2cc259d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 >> 172.21.15.119:0/1990762920 conn(0x560a2cbf3300 msgr2=0x560a2cc41160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c0bfff640 0 client.6300 ms_handle_reset on v2:172.21.15.188:6834/913059196 2023-11-08T19:27:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 shutdown_connections 2023-11-08T19:27:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 1 -- 172.21.15.119:0/1990762920 wait complete. 2023-11-08T19:27:43.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.361+0000 7f2c363cc740 20 client.6300 trim_cache size 0 max 16384 2023-11-08T19:27:43.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.377+0000 7f2c363cc740 1 Processor -- start 2023-11-08T19:27:43.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.377+0000 7f2c363cc740 1 -- start start 2023-11-08T19:27:43.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cc58200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 0x560a2cc379c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc3e0a0 0x560a2cbf2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c363cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cc1fa10 con 0x560a2cc1fec0 2023-11-08T19:27:43.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cbee9a0 con 0x560a2cc3e0a0 2023-11-08T19:27:43.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a2cc4d750 con 0x560a2cc202a0 2023-11-08T19:27:43.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cc58200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cc58200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54896/0 (socket says 172.21.15.119:54896) 2023-11-08T19:27:43.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 -- 172.21.15.119:0/4403730 learned_addr learned my addr 172.21.15.119:0/4403730 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:43.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 -- 172.21.15.119:0/4403730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 msgr2=0x560a2cc379c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 0x560a2cc379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 -- 172.21.15.119:0/4403730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc3e0a0 msgr2=0x560a2cbf2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc3e0a0 0x560a2cbf2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.378+0000 7f2c315d5640 1 -- 172.21.15.119:0/4403730 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c2c13f520 con 0x560a2cc1fec0 2023-11-08T19:27:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c315d5640 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cc58200 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2c28016240 tx=0x7f2c28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c8d3882e4cbc81 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c177fe640 1 -- 172.21.15.119:0/4403730 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28016b50 con 0x560a2cc1fec0 2023-11-08T19:27:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c177fe640 1 -- 172.21.15.119:0/4403730 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28016cb0 con 0x560a2cc1fec0 2023-11-08T19:27:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c177fe640 1 -- 172.21.15.119:0/4403730 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2802e220 con 0x560a2cc1fec0 2023-11-08T19:27:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 -- 172.21.15.119:0/4403730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 msgr2=0x560a2cc58200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cc58200 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f2c28016240 tx=0x7f2c28013000 comp rx=0 tx=0).stop 2023-11-08T19:27:43.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 -- 172.21.15.119:0/4403730 shutdown_connections 2023-11-08T19:27:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc3e0a0 0x560a2cbf2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc202a0 0x560a2cc379c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 --2- 172.21.15.119:0/4403730 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc1fec0 0x560a2cc58200 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.379+0000 7f2c363cc740 1 -- 172.21.15.119:0/4403730 >> 172.21.15.119:0/4403730 conn(0x560a2cbf3300 msgr2=0x560a2cc1f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.380+0000 7f2c363cc740 1 -- 172.21.15.119:0/4403730 shutdown_connections 2023-11-08T19:27:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.380+0000 7f2c363cc740 1 -- 172.21.15.119:0/4403730 wait complete. 2023-11-08T19:27:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.380+0000 7f2c363cc740 1 -- start start 2023-11-08T19:27:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.380+0000 7f2c363cc740 1 Processor -- start 2023-11-08T19:27:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc1fec0 0x560a2cca80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc41160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc416a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cbf9010 con 0x560a2cc3e0a0 2023-11-08T19:27:43.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a2cc3b060 con 0x560a2cc202a0 2023-11-08T19:27:43.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x560a2cc3b590 con 0x560a2cc1fec0 2023-11-08T19:27:43.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c363cc740 10 client.0 _ll_register_callbacks cb 0x7f2c240790e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:27:43.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.381+0000 7f2c31dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc416a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc416a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54902/0 (socket says 172.21.15.119:54902) 2023-11-08T19:27:43.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2297602659 learned_addr learned my addr 172.21.15.119:0/2297602659 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:43.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c315d5640 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc1fec0 0x560a2cca80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc1fec0 msgr2=0x560a2cca80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc1fec0 0x560a2cca80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 msgr2=0x560a2cc41160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc41160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 -- 172.21.15.119:0/2297602659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560a2cb31b10 con 0x560a2cc3e0a0 2023-11-08T19:27:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c31dd6640 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc416a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c740 tx=0x7f2c24009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd7dace815f5697 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:27:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c0bfff640 1 -- 172.21.15.119:0/2297602659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c24081a80 con 0x560a2cc3e0a0 2023-11-08T19:27:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c0bfff640 1 -- 172.21.15.119:0/2297602659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2c2409add0 con 0x560a2cc3e0a0 2023-11-08T19:27:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.382+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c28016270 con 0x560a2cc3e0a0 2023-11-08T19:27:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c0bfff640 1 -- 172.21.15.119:0/2297602659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c24008000 con 0x560a2cc3e0a0 2023-11-08T19:27:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 10 client.6306 fetch_fsmap learned FSMap version 35 2023-11-08T19:27:43.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) v3 -- 0x560a2cafab90 con 0x560a2cc3e0a0 2023-11-08T19:27:43.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c0bfff640 1 -- 172.21.15.119:0/2297602659 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28016270 con 0x560a2cc3e0a0 2023-11-08T19:27:43.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c0bfff640 1 -- 172.21.15.119:0/2297602659 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 35) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2c24048070 con 0x560a2cc3e0a0 2023-11-08T19:27:43.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 10 client.6306 fetch_fsmap finished waiting for FSMap version 35 2023-11-08T19:27:43.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 10 client.6306 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:27:43.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 20 client.6306 populate_metadata read hostname 'smithi119' 2023-11-08T19:27:43.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cbf7f40 0x560a2cbf8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 4 client.6306 mds_command: new command op to 6147 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:27:43.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 --> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560a2cb574c0 con 0x560a2cbf7f40 2023-11-08T19:27:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.383+0000 7f2c315d5640 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cbf7f40 0x560a2cbf8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.384+0000 7f2c315d5640 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cbf7f40 0x560a2cbf8320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c28e4bfb0e06bf9c server_cookie=3cd6b157f2d00d24 in_seq=0 out_seq=0 2023-11-08T19:27:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.384+0000 7f2c0bfff640 10 client.6306 ms_handle_connect on v2:172.21.15.188:6834/913059196 2023-11-08T19:27:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c0bfff640 1 -- 172.21.15.119:0/2297602659 <== mds.0 v2:172.21.15.188:6834/913059196 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x560a2cb574c0 con 0x560a2cbf7f40 2023-11-08T19:27:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c0bfff640 10 client.6306 handle_command_reply: tid=0 2023-11-08T19:27:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 client.6306 shutdown 2023-11-08T19:27:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 msgr2=0x560a2cc416a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc416a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c740 tx=0x7f2c24009760 comp rx=0 tx=0).stop 2023-11-08T19:27:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 shutdown_connections 2023-11-08T19:27:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:6834/913059196,v1:172.21.15.188:6835/913059196] conn(0x560a2cbf7f40 0x560a2cbf8320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x560a2cc3e0a0 0x560a2cc416a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x560a2cc202a0 0x560a2cc41160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c0bfff640 0 client.6306 ms_handle_reset on v2:172.21.15.188:6834/913059196 2023-11-08T19:27:43.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 --2- 172.21.15.119:0/2297602659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x560a2cc1fec0 0x560a2cca80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 >> 172.21.15.119:0/2297602659 conn(0x560a2cbf3300 msgr2=0x560a2cc1f170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 shutdown_connections 2023-11-08T19:27:43.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 1 -- 172.21.15.119:0/2297602659 wait complete. 2023-11-08T19:27:43.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.385+0000 7f2c363cc740 20 client.6306 trim_cache size 0 max 16384 2023-11-08T19:27:43.401 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:27:43.402 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:27:43.402 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:27:43.402 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 6147, 2023-11-08T19:27:43.402 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:27:43.402 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:27:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:27:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.6924657930000002, 2023-11-08T19:27:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 35, 2023-11-08T19:27:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 67, 2023-11-08T19:27:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 65, 2023-11-08T19:27:43.403 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 12.539681923 2023-11-08T19:27:43.404 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:27:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2c1006aea0 msgr2=0x7f2c1006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2c1006aea0 0x7f2c1006d360 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f2c28001200 tx=0x7f2c28001110 comp rx=0 tx=0).stop 2023-11-08T19:27:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 msgr2=0x7f2c2c0c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 0x7f2c2c0c6990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f2c2402c670 tx=0x7f2c24004ab0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 shutdown_connections 2023-11-08T19:27:43.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2c1006aea0 0x7f2c1006d360 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2c2c0cc6d0 0x7f2c2c0c6990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2c2c15ab50 0x7f2c2c0c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 --2- 172.21.15.119:0/2417436353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2c2c15a770 0x7f2c2c0cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 >> 172.21.15.119:0/2417436353 conn(0x7f2c2c0cb610 msgr2=0x7f2c2c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 shutdown_connections 2023-11-08T19:27:43.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.386+0000 7f2c33839640 1 -- 172.21.15.119:0/2417436353 wait complete. 2023-11-08T19:27:43.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:27:43.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.609+0000 7ff68b18c640 1 Processor -- start 2023-11-08T19:27:43.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.609+0000 7ff68b18c640 1 -- start start 2023-11-08T19:27:43.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.610+0000 7ff68b18c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 0x7ff68415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.610+0000 7ff68b18c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840ca2e0 con 0x7ff6840cac30 2023-11-08T19:27:43.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.610+0000 7ff68b18c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840ca5c0 con 0x7ff6840ad9d0 2023-11-08T19:27:43.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.610+0000 7ff68b18c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff684071c70 con 0x7ff6840adf20 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff68a18a640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff6840cac30 0x7ff68415f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41842/0 (socket says 172.21.15.119:41842) 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff68a18a640 1 -- 172.21.15.119:0/3056535829 learned_addr learned my addr 172.21.15.119:0/3056535829 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689989640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 0x7ff68415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4127770576 0 0) 0x7ff6840ca2e0 con 0x7ff6840cac30 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff670003520 con 0x7ff6840cac30 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304133329 0 0) 0x7ff670003520 con 0x7ff6840cac30 2023-11-08T19:27:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 msgr2=0x7ff68415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 0x7ff68415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 >> v1:172.21.15.188:6789/0 conn(0x7ff6840ad9d0 legacy=0x7ff68406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.611+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68415eb80 con 0x7ff6840cac30 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7ff680001670 con 0x7ff6840cac30 2023-11-08T19:27:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 >> v1:172.21.15.119:6789/0 conn(0x7ff6840cac30 legacy=0x7ff68415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff670003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 0x7ff6700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff670004d00 0x7ff670005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684071c70 con 0x7ff6700039a0 2023-11-08T19:27:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67000ba30 con 0x7ff670004d00 2023-11-08T19:27:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67000bbf0 con 0x7ff6840adf20 2023-11-08T19:27:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff68a18a640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff670003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff68a18a640 1 -- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 msgr2=0x7ff6700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff689989640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 0x7ff6700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff68a18a640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6840adf20 0x7ff6700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff68a18a640 1 -- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff670004d00 msgr2=0x7ff670005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff68a18a640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff670004d00 0x7ff670005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.612+0000 7ff68a18a640 1 -- 172.21.15.119:0/3056535829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670003520 con 0x7ff6700039a0 2023-11-08T19:27:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.613+0000 7ff68a18a640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff670003dc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff68000c210 tx=0x7ff6700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8750ee198777243 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680005ad0 con 0x7ff6700039a0 2023-11-08T19:27:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680005c30 con 0x7ff6700039a0 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff689188640 1 -- 172.21.15.119:0/3056535829 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68002c890 con 0x7ff6700039a0 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68b18c640 1 -- 172.21.15.119:0/3056535829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 msgr2=0x7ff670003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68b18c640 1 --2- 172.21.15.119:0/3056535829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff670003dc0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff68000c210 tx=0x7ff6700057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68a98b640 1 -- 172.21.15.119:0/3056535829 reap_dead start 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68b18c640 1 -- 172.21.15.119:0/3056535829 shutdown_connections 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68b18c640 1 -- 172.21.15.119:0/3056535829 >> 172.21.15.119:0/3056535829 conn(0x7ff6840b0560 msgr2=0x7ff6840b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68b18c640 1 -- 172.21.15.119:0/3056535829 shutdown_connections 2023-11-08T19:27:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.614+0000 7ff68b18c640 1 -- 172.21.15.119:0/3056535829 wait complete. 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.616+0000 7ff68b18c640 1 Processor -- start 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.616+0000 7ff68b18c640 1 -- start start 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68b18c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff6841f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68b18c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff684159160 0x7ff6841595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68b18c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6841573f0 0x7ff684157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68b18c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684148980 con 0x7ff6700039a0 2023-11-08T19:27:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68b18c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684159b20 con 0x7ff6841573f0 2023-11-08T19:27:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68b18c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6841589c0 con 0x7ff684159160 2023-11-08T19:27:43.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68a18a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff6841f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68a18a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff6841f0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54910/0 (socket says 172.21.15.119:54910) 2023-11-08T19:27:43.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.617+0000 7ff68a18a640 1 -- 172.21.15.119:0/2922719542 learned_addr learned my addr 172.21.15.119:0/2922719542 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:43.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a18a640 1 -- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff684159160 msgr2=0x7ff6841595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a18a640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff684159160 0x7ff6841595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a18a640 1 -- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6841573f0 msgr2=0x7ff684157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a98b640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6841573f0 0x7ff684157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a18a640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6841573f0 0x7ff684157870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a18a640 1 -- 172.21.15.119:0/2922719542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684158120 con 0x7ff6700039a0 2023-11-08T19:27:43.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68a18a640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff6841f0050 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff68002c430 tx=0x7ff680001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680039070 con 0x7ff6700039a0 2023-11-08T19:27:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68002cca0 con 0x7ff6700039a0 2023-11-08T19:27:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68003c040 con 0x7ff6700039a0 2023-11-08T19:27:43.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff684158400 con 0x7ff6700039a0 2023-11-08T19:27:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.618+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6840c5c70 con 0x7ff6700039a0 2023-11-08T19:27:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.619+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6840c5e00 con 0x7ff6700039a0 2023-11-08T19:27:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.620+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff680032070 con 0x7ff6700039a0 2023-11-08T19:27:43.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.621+0000 7ff67affd640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff66806ae70 0x7ff66806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.621+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff6800aacd0 con 0x7ff6700039a0 2023-11-08T19:27:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.622+0000 7ff689989640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff66806ae70 0x7ff66806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:43.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.622+0000 7ff689989640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff66806ae70 0x7ff66806d330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6840b3650 tx=0x7ff674008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:43.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.625+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff680075a90 con 0x7ff6700039a0 2023-11-08T19:27:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.854+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff68404ece0 con 0x7ff6700039a0 2023-11-08T19:27:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.855+0000 7ff67affd640 1 -- 172.21.15.119:0/2922719542 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff6800790b0 con 0x7ff6700039a0 2023-11-08T19:27:43.856 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:27:43.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff66806ae70 msgr2=0x7ff66806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff66806ae70 0x7ff66806d330 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff6840b3650 tx=0x7ff674008040 comp rx=0 tx=0).stop 2023-11-08T19:27:43.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 msgr2=0x7ff6841f0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:43.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff6841f0050 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff68002c430 tx=0x7ff680001a00 comp rx=0 tx=0).stop 2023-11-08T19:27:43.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 shutdown_connections 2023-11-08T19:27:43.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff66806ae70 0x7ff66806d330 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff6841573f0 0x7ff684157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff684159160 0x7ff6841595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 --2- 172.21.15.119:0/2922719542 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff6700039a0 0x7ff6841f0050 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:43.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 >> 172.21.15.119:0/2922719542 conn(0x7ff6840b0560 msgr2=0x7ff68415dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:43.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 shutdown_connections 2023-11-08T19:27:43.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:43.857+0000 7ff68b18c640 1 -- 172.21.15.119:0/2922719542 wait complete. 2023-11-08T19:27:43.878 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:27:43.878 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:27:43.879 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:27:43.879 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:27:43.879 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:43.880 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:27:43.880 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:27:43.880 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:27:43.881 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:43.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:43.881 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:27:43.932 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:27:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:27:43.932 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:27:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:27:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.933 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:43.933 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:27:43.934 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:43.934 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:27:43.934 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:27:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:43.935 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:27:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:43.935 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:27:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42207sec preferred_lft 42207sec 2023-11-08T19:27:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:27:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.936 INFO:teuthology.orchestra.run.smithi119.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:27:43.936 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:27:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:27:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::e06c:64ff:fe55:3f8b/64 scope link 2023-11-08T19:27:43.937 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.937 INFO:teuthology.orchestra.run.smithi119.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:27:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c086:39ff:fe1e:9142/64 scope link 2023-11-08T19:27:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.938 INFO:teuthology.orchestra.run.smithi119.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:27:43.938 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:27:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::494:cbff:fed6:d560/64 scope link 2023-11-08T19:27:43.939 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:27:43.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:43.940 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:27:44.031 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:27:44.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.035 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:27:44.090 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:27:44.091 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:27:44.091 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:27:44.091 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:27:44.091 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:27:44.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.093 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:27:44.094 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:27:44.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:27:44.094 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:27:44.095 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:27:44.095 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:27:44.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.283 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:27:44.334 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:27:44.335 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:27:44.336 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:44.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.337 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:27:44.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:44.434 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:44.489 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:27:44.489 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:27:44.489 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:44.541 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T19:27:44.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.542 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:27:44.686 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:27:44.687 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:27:44.722 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:27:44.723 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:27:44.723 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:27:44.723 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:27:44.723 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:27:44.726 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-08T19:27:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:27:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:27:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:27:44.727 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:27:44.732 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:27:44.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.733 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:44.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:44.814 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:27:44.895 INFO:teuthology.orchestra.run.smithi119.stdout:client5245 2023-11-08T19:27:44.895 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:44.896 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:44.955 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_data_pool 2023-11-08T19:27:45.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.173+0000 7f4a28b6e640 1 Processor -- start 2023-11-08T19:27:45.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.173+0000 7f4a28b6e640 1 -- start start 2023-11-08T19:27:45.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.174+0000 7f4a28b6e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 0x7f4a2406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.174+0000 7f4a28b6e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a24071720 con 0x7f4a2415deb0 2023-11-08T19:27:45.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.174+0000 7f4a28b6e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240cf5e0 con 0x7f4a2415e9a0 2023-11-08T19:27:45.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.174+0000 7f4a28b6e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cf8c0 con 0x7f4a240cccf0 2023-11-08T19:27:45.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.175+0000 7f4a237fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4a2415deb0 0x7f4a2415e290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41844/0 (socket says 172.21.15.119:41844) 2023-11-08T19:27:45.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.175+0000 7f4a237fe640 1 -- 172.21.15.119:0/2389727552 learned_addr learned my addr 172.21.15.119:0/2389727552 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:45.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.175+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279940925 0 0) 0x7f4a24071720 con 0x7f4a2415deb0 2023-11-08T19:27:45.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.175+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a10003520 con 0x7f4a2415deb0 2023-11-08T19:27:45.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.175+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 0x7f4a2406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.175+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1806481905 0 0) 0x7f4a10003520 con 0x7f4a2415deb0 2023-11-08T19:27:45.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 msgr2=0x7f4a2406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 0x7f4a2406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 >> v1:172.21.15.188:6789/0 conn(0x7f4a2415e9a0 legacy=0x7f4a2406b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240710d0 con 0x7f4a2415deb0 2023-11-08T19:27:45.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f4a14001670 con 0x7f4a2415deb0 2023-11-08T19:27:45.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 >> v1:172.21.15.119:6789/0 conn(0x7f4a2415deb0 legacy=0x7f4a2415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a100039a0 0x7f4a10003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 0x7f4a100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10004d00 0x7f4a10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cf8c0 con 0x7f4a10004d00 2023-11-08T19:27:45.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1000bc00 con 0x7f4a100039a0 2023-11-08T19:27:45.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1000bdc0 con 0x7f4a240cccf0 2023-11-08T19:27:45.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10004d00 0x7f4a10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a23fff640 1 -- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 msgr2=0x7f4a100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a240cccf0 0x7f4a100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a23fff640 1 -- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a100039a0 msgr2=0x7f4a10003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a100039a0 0x7f4a10003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.176+0000 7f4a23fff640 1 -- 172.21.15.119:0/2389727552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a10003520 con 0x7f4a10004d00 2023-11-08T19:27:45.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10004d00 0x7f4a10005180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c007360 tx=0x7f4a100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8e255fc83beb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c02ccf0 con 0x7f4a10004d00 2023-11-08T19:27:45.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c02ce50 con 0x7f4a10004d00 2023-11-08T19:27:45.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a22ffd640 1 -- 172.21.15.119:0/2389727552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c034660 con 0x7f4a10004d00 2023-11-08T19:27:45.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2389727552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10004d00 msgr2=0x7f4a10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2389727552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a10004d00 0x7f4a10005180 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c007360 tx=0x7f4a100057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a23fff640 1 -- 172.21.15.119:0/2389727552 reap_dead start 2023-11-08T19:27:45.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.178+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2389727552 shutdown_connections 2023-11-08T19:27:45.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.179+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2389727552 >> 172.21.15.119:0/2389727552 conn(0x7f4a240ceac0 msgr2=0x7f4a240ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.179+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2389727552 shutdown_connections 2023-11-08T19:27:45.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.179+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2389727552 wait complete. 2023-11-08T19:27:45.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.180+0000 7f4a28b6e640 1 Processor -- start 2023-11-08T19:27:45.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.180+0000 7f4a28b6e640 1 -- start start 2023-11-08T19:27:45.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.181+0000 7f4a28b6e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a10004d00 0x7f4a240b1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.181+0000 7f4a28b6e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a24154140 0x7f4a240c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.181+0000 7f4a28b6e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 0x7f4a240ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.181+0000 7f4a28b6e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2415c6c0 con 0x7f4a240c6ae0 2023-11-08T19:27:45.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.181+0000 7f4a28b6e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2415ba80 con 0x7f4a10004d00 2023-11-08T19:27:45.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.181+0000 7f4a28b6e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a241545a0 con 0x7f4a24154140 2023-11-08T19:27:45.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 0x7f4a240ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 0x7f4a240ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54926/0 (socket says 172.21.15.119:54926) 2023-11-08T19:27:45.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 -- 172.21.15.119:0/2906997263 learned_addr learned my addr 172.21.15.119:0/2906997263 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:45.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a1affd640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a24154140 0x7f4a240c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 -- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a24154140 msgr2=0x7f4a240c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a237fe640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a10004d00 0x7f4a240b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a24154140 0x7f4a240c65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 -- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a10004d00 msgr2=0x7f4a240b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a10004d00 0x7f4a240b1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 -- 172.21.15.119:0/2906997263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a2415bd20 con 0x7f4a240c6ae0 2023-11-08T19:27:45.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.182+0000 7f4a23fff640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 0x7f4a240ac8d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c037040 tx=0x7f4a1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.183+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c03e600 con 0x7f4a240c6ae0 2023-11-08T19:27:45.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.183+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c03e760 con 0x7f4a240c6ae0 2023-11-08T19:27:45.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.183+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c03eb10 con 0x7f4a240c6ae0 2023-11-08T19:27:45.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.183+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a2415c000 con 0x7f4a240c6ae0 2023-11-08T19:27:45.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.183+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a240c86c0 con 0x7f4a240c6ae0 2023-11-08T19:27:45.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.184+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a2415c800 con 0x7f4a240c6ae0 2023-11-08T19:27:45.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.185+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a1c03ec70 con 0x7f4a240c6ae0 2023-11-08T19:27:45.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.186+0000 7f4a20ff9640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a0406ae70 0x7f4a0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.187+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4a1c0abfb0 con 0x7f4a240c6ae0 2023-11-08T19:27:45.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.187+0000 7f4a237fe640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a0406ae70 0x7f4a0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.187+0000 7f4a237fe640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a0406ae70 0x7f4a0406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4a140079f0 tx=0x7f4a14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.191+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a2415c800 con 0x7f4a240c6ae0 2023-11-08T19:27:45.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.385+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_data_pool"} v 0) v1 -- 0x7f4a2415c800 con 0x7f4a240c6ae0 2023-11-08T19:27:45.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.764+0000 7f4a20ff9640 1 -- 172.21.15.119:0/2906997263 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_data_pool"}]=0 pool 'test_data_pool' created v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4a2415c800 con 0x7f4a240c6ae0 2023-11-08T19:27:45.766 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'test_data_pool' created 2023-11-08T19:27:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.767+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a0406ae70 msgr2=0x7f4a0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.767+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a0406ae70 0x7f4a0406d330 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4a140079f0 tx=0x7f4a14009040 comp rx=0 tx=0).stop 2023-11-08T19:27:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.767+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 msgr2=0x7f4a240ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.767+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 0x7f4a240ac8d0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c037040 tx=0x7f4a1c004ab0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.768+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 shutdown_connections 2023-11-08T19:27:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.768+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a0406ae70 0x7f4a0406d330 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.768+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a240c6ae0 0x7f4a240ac8d0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.768+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a24154140 0x7f4a240c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.768+0000 7f4a28b6e640 1 --2- 172.21.15.119:0/2906997263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a10004d00 0x7f4a240b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.768+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 >> 172.21.15.119:0/2906997263 conn(0x7f4a240ceac0 msgr2=0x7f4a2406b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.769+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 shutdown_connections 2023-11-08T19:27:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.769+0000 7f4a28b6e640 1 -- 172.21.15.119:0/2906997263 wait complete. 2023-11-08T19:27:45.796 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create test_metadata_pool 2023-11-08T19:27:45.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.991+0000 7fe281e34640 1 Processor -- start 2023-11-08T19:27:45.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.991+0000 7fe281e34640 1 -- start start 2023-11-08T19:27:45.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.992+0000 7fe281e34640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe27c0d0320 0x7fe27c15d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.992+0000 7fe281e34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe27c0d10a0 0x7fe27c0d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:45.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.992+0000 7fe281e34640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe27c0d1500 con 0x7fe27c15d1a0 2023-11-08T19:27:45.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.992+0000 7fe281e34640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0ce770 con 0x7fe27c0d10a0 2023-11-08T19:27:45.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.992+0000 7fe281e34640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe27c0b0270 con 0x7fe27c0d0320 2023-11-08T19:27:45.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe27c15d1a0 0x7fe27c0afb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41846/0 (socket says 172.21.15.119:41846) 2023-11-08T19:27:45.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27b7fe640 1 -- 172.21.15.119:0/381293905 learned_addr learned my addr 172.21.15.119:0/381293905 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:45.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27bfff640 1 --2- 172.21.15.119:0/381293905 >> v2:172.21.15.119:3300/0 conn(0x7fe27c0d10a0 0x7fe27c0d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27affd640 1 --2- 172.21.15.119:0/381293905 >> v2:172.21.15.188:3300/0 conn(0x7fe27c0d0320 0x7fe27c15d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:45.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27a7fc640 1 -- 172.21.15.119:0/381293905 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 176644497 0 0) 0x7fe27c0d1500 con 0x7fe27c15d1a0 2023-11-08T19:27:45.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27a7fc640 1 -- 172.21.15.119:0/381293905 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe268003520 con 0x7fe27c15d1a0 2023-11-08T19:27:45.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27bfff640 1 -- 172.21.15.119:0/381293905 >> v2:172.21.15.188:3300/0 conn(0x7fe27c0d0320 msgr2=0x7fe27c15d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:45.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27bfff640 1 --2- 172.21.15.119:0/381293905 >> v2:172.21.15.188:3300/0 conn(0x7fe27c0d0320 0x7fe27c15d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:45.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27bfff640 1 -- 172.21.15.119:0/381293905 >> v1:172.21.15.119:6789/0 conn(0x7fe27c15d1a0 legacy=0x7fe27c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:45.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.993+0000 7fe27bfff640 1 -- 172.21.15.119:0/381293905 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe27c0d07c0 con 0x7fe27c0d10a0 2023-11-08T19:27:45.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.994+0000 7fe27bfff640 1 --2- 172.21.15.119:0/381293905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d10a0 0x7fe27c0d1780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe27c08d7a0 tx=0x7fe27402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55fc2a03e4658ff4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:45.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe27a7fc640 1 -- 172.21.15.119:0/381293905 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27402e070 con 0x7fe27c0d10a0 2023-11-08T19:27:45.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe27a7fc640 1 -- 172.21.15.119:0/381293905 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2740315a0 con 0x7fe27c0d10a0 2023-11-08T19:27:45.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe27a7fc640 1 -- 172.21.15.119:0/381293905 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274031910 con 0x7fe27c0d10a0 2023-11-08T19:27:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 -- 172.21.15.119:0/381293905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d10a0 msgr2=0x7fe27c0d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 --2- 172.21.15.119:0/381293905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d10a0 0x7fe27c0d1780 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe27c08d7a0 tx=0x7fe27402b840 comp rx=0 tx=0).stop 2023-11-08T19:27:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 -- 172.21.15.119:0/381293905 shutdown_connections 2023-11-08T19:27:46.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 --2- 172.21.15.119:0/381293905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d10a0 0x7fe27c0d1780 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 --2- 172.21.15.119:0/381293905 >> v2:172.21.15.188:3300/0 conn(0x7fe27c0d0320 0x7fe27c15d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 -- 172.21.15.119:0/381293905 >> 172.21.15.119:0/381293905 conn(0x7fe27c0d2f90 msgr2=0x7fe27c0d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.995+0000 7fe281e34640 1 -- 172.21.15.119:0/381293905 shutdown_connections 2023-11-08T19:27:46.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.996+0000 7fe281e34640 1 -- 172.21.15.119:0/381293905 wait complete. 2023-11-08T19:27:46.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.997+0000 7fe281e34640 1 Processor -- start 2023-11-08T19:27:46.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.997+0000 7fe281e34640 1 -- start start 2023-11-08T19:27:46.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe281e34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 0x7fe27c151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe281e34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27c0d10a0 0x7fe27c15f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe281e34640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe27c15d1a0 0x7fe27c15fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe281e34640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0a0450 con 0x7fe27c0d0320 2023-11-08T19:27:46.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe281e34640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c0c7a60 con 0x7fe27c15d1a0 2023-11-08T19:27:46.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe281e34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe27c09fb20 con 0x7fe27c0d10a0 2023-11-08T19:27:46.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.998+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 0x7fe27c151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 0x7fe27c151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54938/0 (socket says 172.21.15.119:54938) 2023-11-08T19:27:46.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 -- 172.21.15.119:0/3589972957 learned_addr learned my addr 172.21.15.119:0/3589972957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:46.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27affd640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27c0d10a0 0x7fe27c15f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 -- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27c0d10a0 msgr2=0x7fe27c15f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27c0d10a0 0x7fe27c15f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27bfff640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe27c15d1a0 0x7fe27c15fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 -- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe27c15d1a0 msgr2=0x7fe27c15fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe27c15d1a0 0x7fe27c15fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 -- 172.21.15.119:0/3589972957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe274009400 con 0x7fe27c0d0320 2023-11-08T19:27:46.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe27b7fe640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 0x7fe27c151250 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00caa0 tx=0x7fe26c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:45.999+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c00ac00 con 0x7fe27c0d0320 2023-11-08T19:27:46.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.000+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26c00ad60 con 0x7fe27c0d0320 2023-11-08T19:27:46.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.000+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26c012620 con 0x7fe27c0d0320 2023-11-08T19:27:46.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.000+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe27c09fe00 con 0x7fe27c0d0320 2023-11-08T19:27:46.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.000+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe27c151790 con 0x7fe27c0d0320 2023-11-08T19:27:46.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.001+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe27c0cea90 con 0x7fe27c0d0320 2023-11-08T19:27:46.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.002+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe26c007070 con 0x7fe27c0d0320 2023-11-08T19:27:46.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.003+0000 7fe273fff640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe25c06ae70 0x7fe25c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.003+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6793+0+0 (secure 0 0 0) 0x7fe26c089ec0 con 0x7fe27c0d0320 2023-11-08T19:27:46.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.003+0000 7fe27affd640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe25c06ae70 0x7fe25c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.004+0000 7fe27affd640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe25c06ae70 0x7fe25c06d330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.007+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe26c054b10 con 0x7fe27c0d0320 2023-11-08T19:27:46.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.196+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "test_metadata_pool"} v 0) v1 -- 0x7fe27c0722b0 con 0x7fe27c0d0320 2023-11-08T19:27:46.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.769+0000 7fe273fff640 1 -- 172.21.15.119:0/3589972957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "test_metadata_pool"}]=0 pool 'test_metadata_pool' created v69) v1 ==== 126+0+0 (secure 0 0 0) 0x7fe26c0170a0 con 0x7fe27c0d0320 2023-11-08T19:27:46.771 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'test_metadata_pool' created 2023-11-08T19:27:46.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe25c06ae70 msgr2=0x7fe25c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe25c06ae70 0x7fe25c06d330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe264006fd0 tx=0x7fe264008040 comp rx=0 tx=0).stop 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 msgr2=0x7fe27c151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 0x7fe27c151250 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00caa0 tx=0x7fe26c00cf60 comp rx=0 tx=0).stop 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 shutdown_connections 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe25c06ae70 0x7fe25c06d330 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe27c15d1a0 0x7fe27c15fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe27c0d10a0 0x7fe27c15f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 --2- 172.21.15.119:0/3589972957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe27c0d0320 0x7fe27c151250 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 >> 172.21.15.119:0/3589972957 conn(0x7fe27c0d2f90 msgr2=0x7fe27c0b03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 shutdown_connections 2023-11-08T19:27:46.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.774+0000 7fe281e34640 1 -- 172.21.15.119:0/3589972957 wait complete. 2023-11-08T19:27:46.809 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool mksnap snap4 2023-11-08T19:27:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.891+0000 7fec01e3bf40 1 Processor -- start 2023-11-08T19:27:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.891+0000 7fec01e3bf40 1 -- start start 2023-11-08T19:27:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7fec01e3bf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac36667700 0x55ac3664ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7fec01e3bf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 0x55ac366f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7fec01e3bf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac366f49a0 0x55ac366ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7fec01e3bf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac365cc340 con 0x55ac3666a280 2023-11-08T19:27:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7fec01e3bf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac36566a70 con 0x55ac36667700 2023-11-08T19:27:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7fec01e3bf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac36566710 con 0x55ac366f49a0 2023-11-08T19:27:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7febf7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 0x55ac366f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7febf7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 0x55ac366f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54960/0 (socket says 172.21.15.119:54960) 2023-11-08T19:27:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.892+0000 7febf7fff640 1 -- 172.21.15.119:0/1730392464 learned_addr learned my addr 172.21.15.119:0/1730392464 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febff107640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac36667700 0x55ac3664ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febf7fff640 1 -- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac366f49a0 msgr2=0x55ac366ede50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febff908640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac366f49a0 0x55ac366ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febf7fff640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac366f49a0 0x55ac366ede50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febf7fff640 1 -- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac36667700 msgr2=0x55ac3664ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febf7fff640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac36667700 0x55ac3664ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febf7fff640 1 -- 172.21.15.119:0/1730392464 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf0005de0 con 0x55ac3666a280 2023-11-08T19:27:46.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febff107640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac36667700 0x55ac3664ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:46.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febff908640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac366f49a0 0x55ac366ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:46.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febf7fff640 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 0x55ac366f3600 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7febe8001d60 tx=0x7febe800b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ac985baddc99cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febfe105640 1 -- 172.21.15.119:0/1730392464 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8014070 con 0x55ac3666a280 2023-11-08T19:27:46.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febfe105640 1 -- 172.21.15.119:0/1730392464 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe800f450 con 0x55ac3666a280 2023-11-08T19:27:46.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.893+0000 7febfe105640 1 -- 172.21.15.119:0/1730392464 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800f7e0 con 0x55ac3666a280 2023-11-08T19:27:46.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1730392464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 msgr2=0x55ac366f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 0x55ac366f3600 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7febe8001d60 tx=0x7febe800b600 comp rx=0 tx=0).stop 2023-11-08T19:27:46.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1730392464 shutdown_connections 2023-11-08T19:27:46.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac366f49a0 0x55ac366ede50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac3666a280 0x55ac366f3600 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1730392464 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac36667700 0x55ac3664ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1730392464 >> 172.21.15.119:0/1730392464 conn(0x55ac3661b570 msgr2=0x55ac36661780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:46.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1730392464 shutdown_connections 2023-11-08T19:27:46.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.894+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1730392464 wait complete. 2023-11-08T19:27:46.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.895+0000 7fec01e3bf40 1 Processor -- start 2023-11-08T19:27:46.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.895+0000 7fec01e3bf40 1 -- start start 2023-11-08T19:27:46.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7fec01e3bf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 0x55ac366eb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7fec01e3bf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac3666a280 0x55ac366f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7fec01e3bf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac366f49a0 0x55ac366f22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7fec01e3bf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac3655a520 con 0x55ac36667700 2023-11-08T19:27:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7fec01e3bf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac36661ba0 con 0x55ac366f49a0 2023-11-08T19:27:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7fec01e3bf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ac366eedd0 con 0x55ac3666a280 2023-11-08T19:27:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7febff107640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 0x55ac366eb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7febff107640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 0x55ac366eb9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54964/0 (socket says 172.21.15.119:54964) 2023-11-08T19:27:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.896+0000 7febff107640 1 -- 172.21.15.119:0/1516344488 learned_addr learned my addr 172.21.15.119:0/1516344488 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febf7fff640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac3666a280 0x55ac366f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febff107640 1 -- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac3666a280 msgr2=0x55ac366f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febff107640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac3666a280 0x55ac366f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febff107640 1 -- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac366f49a0 msgr2=0x55ac366f22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:46.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febff107640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac366f49a0 0x55ac366f22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:46.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febff107640 1 -- 172.21.15.119:0/1516344488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac365654a0 con 0x55ac36667700 2023-11-08T19:27:46.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febff107640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 0x55ac366eb9f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c00 tx=0x7febf0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:46.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0052850 con 0x55ac36667700 2023-11-08T19:27:46.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf00529b0 con 0x55ac36667700 2023-11-08T19:27:46.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ac36572680 con 0x55ac36667700 2023-11-08T19:27:46.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0052d40 con 0x55ac36667700 2023-11-08T19:27:46.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.897+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ac36565830 con 0x55ac36667700 2023-11-08T19:27:46.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.899+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febf0059050 con 0x55ac36667700 2023-11-08T19:27:46.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.900+0000 7febfd103640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7febe0069820 0x7febe006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:46.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.900+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 7150+0+0 (secure 0 0 0) 0x7febf00c9fc0 con 0x55ac36667700 2023-11-08T19:27:46.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.900+0000 7febf7fff640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7febe0069820 0x7febe006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:46.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.900+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- pool_op(create snap pool 12 tid 1 name snap4 v0) v4 -- 0x55ac365d62c0 con 0x55ac36667700 2023-11-08T19:27:46.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:46.901+0000 7febf7fff640 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7febe0069820 0x7febe006bce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7febe8001d00 tx=0x7febe8011040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.779+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v70) v1 ==== 43+0+0 (secure 0 0 0) 0x7febf0057080 con 0x55ac36667700 2023-11-08T19:27:47.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.779+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=70}) v3 -- 0x7febe0073fc0 con 0x55ac36667700 2023-11-08T19:27:47.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.781+0000 7febfd103640 1 -- 172.21.15.119:0/1516344488 <== mon.0 v2:172.21.15.119:3300/0 7 ==== osd_map(70..70 src has 1..70) v4 ==== 718+0+0 (secure 0 0 0) 0x7febf0099040 con 0x55ac36667700 2023-11-08T19:27:47.783 INFO:teuthology.orchestra.run.smithi119.stdout:created pool test_data_pool snap snap4 2023-11-08T19:27:47.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.782+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7febe0069820 msgr2=0x7febe006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.782+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7febe0069820 0x7febe006bce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7febe8001d00 tx=0x7febe8011040 comp rx=0 tx=0).stop 2023-11-08T19:27:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.782+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 msgr2=0x55ac366eb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.782+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 0x55ac366eb9f0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7febf0000c00 tx=0x7febf0002fb0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 shutdown_connections 2023-11-08T19:27:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7febe0069820 0x7febe006bce0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ac366f49a0 0x55ac366f22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ac3666a280 0x55ac366f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 --2- 172.21.15.119:0/1516344488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ac36667700 0x55ac366eb9f0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 >> 172.21.15.119:0/1516344488 conn(0x55ac3661b570 msgr2=0x55ac366fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 shutdown_connections 2023-11-08T19:27:47.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.783+0000 7fec01e3bf40 1 -- 172.21.15.119:0/1516344488 wait complete. 2023-11-08T19:27:47.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new testfs test_metadata_pool test_data_pool 2023-11-08T19:27:47.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.989+0000 7fa10f607640 1 Processor -- start 2023-11-08T19:27:47.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.989+0000 7fa10f607640 1 -- start start 2023-11-08T19:27:47.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.990+0000 7fa10f607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa108143900 0x7fa1080caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.990+0000 7fa10f607640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1080c5200 0x7fa10815d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.990+0000 7fa10f607640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1080c9d50 con 0x7fa108143520 2023-11-08T19:27:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10f607640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10815e5e0 con 0x7fa1080c5200 2023-11-08T19:27:47.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10f607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080c9fa0 con 0x7fa108143900 2023-11-08T19:27:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1080c5200 0x7fa10815d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa1080c5200 0x7fa10815d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54966/0 (socket says 172.21.15.119:54966) 2023-11-08T19:27:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 -- 172.21.15.119:0/3433662819 learned_addr learned my addr 172.21.15.119:0/3433662819 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:47.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 -- 172.21.15.119:0/3433662819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa108143900 msgr2=0x7fa1080caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 --2- 172.21.15.119:0/3433662819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa108143900 0x7fa1080caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 -- 172.21.15.119:0/3433662819 >> v1:172.21.15.119:6789/0 conn(0x7fa108143520 legacy=0x7fa1080c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.991+0000 7fa10dba4640 1 -- 172.21.15.119:0/3433662819 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1080ca280 con 0x7fa1080c5200 2023-11-08T19:27:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.992+0000 7fa10dba4640 1 --2- 172.21.15.119:0/3433662819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10815d870 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa100009e50 tx=0x7fa10002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a08c45223006ab6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa0fffff640 1 -- 172.21.15.119:0/3433662819 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa10002cb00 con 0x7fa1080c5200 2023-11-08T19:27:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa0fffff640 1 -- 172.21.15.119:0/3433662819 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa10002cc60 con 0x7fa1080c5200 2023-11-08T19:27:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa0fffff640 1 -- 172.21.15.119:0/3433662819 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000343f0 con 0x7fa1080c5200 2023-11-08T19:27:47.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 -- 172.21.15.119:0/3433662819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 msgr2=0x7fa10815d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 --2- 172.21.15.119:0/3433662819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10815d870 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa100009e50 tx=0x7fa10002bf60 comp rx=0 tx=0).stop 2023-11-08T19:27:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 -- 172.21.15.119:0/3433662819 shutdown_connections 2023-11-08T19:27:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 --2- 172.21.15.119:0/3433662819 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10815d870 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 --2- 172.21.15.119:0/3433662819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa108143900 0x7fa1080caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 -- 172.21.15.119:0/3433662819 >> 172.21.15.119:0/3433662819 conn(0x7fa1080b0950 msgr2=0x7fa1080b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 -- 172.21.15.119:0/3433662819 shutdown_connections 2023-11-08T19:27:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.993+0000 7fa10f607640 1 -- 172.21.15.119:0/3433662819 wait complete. 2023-11-08T19:27:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.995+0000 7fa10f607640 1 Processor -- start 2023-11-08T19:27:47.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.995+0000 7fa10f607640 1 -- start start 2023-11-08T19:27:48.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10f607640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10813f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10f607640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108143520 0x7fa108153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10f607640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1081f4880 0x7fa1081f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10f607640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1081f5270 con 0x7fa1080c5200 2023-11-08T19:27:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10f607640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080c6d50 con 0x7fa108143520 2023-11-08T19:27:48.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10f607640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1080b4490 con 0x7fa1081f4880 2023-11-08T19:27:48.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10d3a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10813f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10d3a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10813f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54972/0 (socket says 172.21.15.119:54972) 2023-11-08T19:27:48.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.996+0000 7fa10d3a3640 1 -- 172.21.15.119:0/531583479 learned_addr learned my addr 172.21.15.119:0/531583479 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:48.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10dba4640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1081f4880 0x7fa1081f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10d3a3640 1 -- 172.21.15.119:0/531583479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1081f4880 msgr2=0x7fa1081f4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10d3a3640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa1081f4880 0x7fa1081f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10d3a3640 1 -- 172.21.15.119:0/531583479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108143520 msgr2=0x7fa108153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10d3a3640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa108143520 0x7fa108153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10d3a3640 1 -- 172.21.15.119:0/531583479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009b30 con 0x7fa1080c5200 2023-11-08T19:27:48.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10d3a3640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa1080c5200 0x7fa10813f1d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa10400c450 tx=0x7fa10400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1040052d0 con 0x7fa1080c5200 2023-11-08T19:27:48.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa104005430 con 0x7fa1080c5200 2023-11-08T19:27:48.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa10f607640 1 -- 172.21.15.119:0/531583479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1080b3950 con 0x7fa1080c5200 2023-11-08T19:27:48.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.997+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104012870 con 0x7fa1080c5200 2023-11-08T19:27:48.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.998+0000 7fa10f607640 1 -- 172.21.15.119:0/531583479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10813f760 con 0x7fa1080c5200 2023-11-08T19:27:48.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:47.999+0000 7fa10f607640 1 -- 172.21.15.119:0/531583479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1080719c0 con 0x7fa1080c5200 2023-11-08T19:27:48.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.000+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1040129d0 con 0x7fa1080c5200 2023-11-08T19:27:48.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.001+0000 7fa0fdffb640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa0ec06ae70 0x7fa0ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.001+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7189+0+0 (secure 0 0 0) 0x7fa104089770 con 0x7fa1080c5200 2023-11-08T19:27:48.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.001+0000 7fa10cba2640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa0ec06ae70 0x7fa0ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.002+0000 7fa10cba2640 1 --2- 172.21.15.119:0/531583479 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa0ec06ae70 0x7fa0ec06d330 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1081f5910 tx=0x7fa0f8006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.006+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1040541b0 con 0x7fa1080c5200 2023-11-08T19:27:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.232+0000 7fa10f607640 1 -- 172.21.15.119:0/531583479 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"} v 0) v1 -- 0x7fa1080ca560 con 0x7fa1080c5200 2023-11-08T19:27:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.233+0000 7fa0fdffb640 1 -- 172.21.15.119:0/531583479 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "testfs", "metadata": "test_metadata_pool", "data": "test_data_pool"}]=-95 pool(12) already has mon-managed snaps; can't attach pool to fs v35) v1 ==== 198+0+0 (secure 0 0 0) 0x7fa1040577d0 con 0x7fa1080c5200 2023-11-08T19:27:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:Error ENOTSUP: pool(12) already has mon-managed snaps; can't attach pool to fs 2023-11-08T19:27:48.249 DEBUG:teuthology.orchestra.run:got remote process result: 95 2023-11-08T19:27:48.251 DEBUG:teuthology.orchestra.run.smithi119:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool test_data_pool rmsnap snap4 2023-11-08T19:27:48.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.347+0000 7f1a03a0bf40 1 Processor -- start 2023-11-08T19:27:48.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.347+0000 7f1a03a0bf40 1 -- start start 2023-11-08T19:27:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.348+0000 7f1a03a0bf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19ec002260 0x559df6d4ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.348+0000 7f1a03a0bf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58bd0 0x559df6caab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.348+0000 7f1a03a0bf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 0x559df6cab080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.348+0000 7f1a03a0bf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df6c2c340 con 0x559df6d58fb0 2023-11-08T19:27:48.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.348+0000 7f1a03a0bf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df6bc6a70 con 0x559df6d58bd0 2023-11-08T19:27:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.348+0000 7f1a03a0bf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559df6bc6710 con 0x7f19ec002260 2023-11-08T19:27:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 0x559df6cab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 0x559df6cab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54986/0 (socket says 172.21.15.119:54986) 2023-11-08T19:27:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 -- 172.21.15.119:0/46587350 learned_addr learned my addr 172.21.15.119:0/46587350 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:48.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a008b9640 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19ec002260 0x559df6d4ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 -- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19ec002260 msgr2=0x559df6d4ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19ec002260 0x559df6d4ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f19f80b8640 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58bd0 0x559df6caab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 -- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58bd0 msgr2=0x559df6caab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:48.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58bd0 0x559df6caab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 -- 172.21.15.119:0/46587350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559df6bd2680 con 0x559df6d58fb0 2023-11-08T19:27:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f1a010ba640 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 0x559df6cab080 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000fb0 tx=0x7f19fc001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2adcae7b5147599 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.349+0000 7f19fb7fe640 1 -- 172.21.15.119:0/46587350 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc014030 con 0x559df6d58fb0 2023-11-08T19:27:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f19fb7fe640 1 -- 172.21.15.119:0/46587350 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc00e440 con 0x559df6d58fb0 2023-11-08T19:27:48.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f19fb7fe640 1 -- 172.21.15.119:0/46587350 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc00e7f0 con 0x559df6d58fb0 2023-11-08T19:27:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/46587350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 msgr2=0x559df6cab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 0x559df6cab080 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f19fc000fb0 tx=0x7f19fc001e90 comp rx=0 tx=0).stop 2023-11-08T19:27:48.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/46587350 shutdown_connections 2023-11-08T19:27:48.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x559df6d58fb0 0x559df6cab080 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58bd0 0x559df6caab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/46587350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19ec002260 0x559df6d4ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/46587350 >> 172.21.15.119:0/46587350 conn(0x559df6c7ca30 msgr2=0x559df6c7c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/46587350 shutdown_connections 2023-11-08T19:27:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.350+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/46587350 wait complete. 2023-11-08T19:27:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.351+0000 7f1a03a0bf40 1 Processor -- start 2023-11-08T19:27:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.351+0000 7f1a03a0bf40 1 -- start start 2023-11-08T19:27:48.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a03a0bf40 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559df6d58bd0 0x559df6d42f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a03a0bf40 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58fb0 0x559df6cabae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a03a0bf40 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 0x559df6cac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a03a0bf40 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df6bba520 con 0x7f19ec002260 2023-11-08T19:27:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a03a0bf40 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df6cc3500 con 0x559df6d58fb0 2023-11-08T19:27:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a03a0bf40 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x559df6caa750 con 0x559df6d58bd0 2023-11-08T19:27:48.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a010ba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 0x559df6cac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a010ba640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 0x559df6cac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:54996/0 (socket says 172.21.15.119:54996) 2023-11-08T19:27:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a010ba640 1 -- 172.21.15.119:0/3879461170 learned_addr learned my addr 172.21.15.119:0/3879461170 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.352+0000 7f1a008b9640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559df6d58bd0 0x559df6d42f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f19f80b8640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58fb0 0x559df6cabae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a010ba640 1 -- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559df6d58bd0 msgr2=0x559df6d42f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a010ba640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559df6d58bd0 0x559df6d42f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a010ba640 1 -- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58fb0 msgr2=0x559df6cabae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a010ba640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58fb0 0x559df6cabae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a010ba640 1 -- 172.21.15.119:0/3879461170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559df6bec670 con 0x7f19ec002260 2023-11-08T19:27:48.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f19f80b8640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58fb0 0x559df6cabae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a010ba640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 0x559df6cac020 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x559df6d58920 tx=0x7f19fc001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc01b640 con 0x7f19ec002260 2023-11-08T19:27:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19fc01b7c0 con 0x7f19ec002260 2023-11-08T19:27:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559df6bc5830 con 0x7f19ec002260 2023-11-08T19:27:48.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19fc01bb70 con 0x7f19ec002260 2023-11-08T19:27:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.353+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559df6bc5290 con 0x7f19ec002260 2023-11-08T19:27:48.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.356+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f19fc022070 con 0x7f19ec002260 2023-11-08T19:27:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.356+0000 7f19fa7fc640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19e0069850 0x7f19e006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.357+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 7189+0+0 (secure 0 0 0) 0x7f19fc015070 con 0x7f19ec002260 2023-11-08T19:27:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.357+0000 7f1a008b9640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19e0069850 0x7f19e006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.357+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- pool_op(delete snap pool 12 tid 1 name snap4 v0) v4 -- 0x559df6c362c0 con 0x7f19ec002260 2023-11-08T19:27:48.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.357+0000 7f1a008b9640 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19e0069850 0x7f19e006bd10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19f0055000 tx=0x7f19f004c070 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.782+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 6 ==== pool_op_reply(tid 1 (0) Success v71) v1 ==== 43+0+0 (secure 0 0 0) 0x7f19fc0559e0 con 0x7f19ec002260 2023-11-08T19:27:48.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.782+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=71}) v3 -- 0x7f19e0074050 con 0x7f19ec002260 2023-11-08T19:27:48.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.782+0000 7f19fa7fc640 1 -- 172.21.15.119:0/3879461170 <== mon.0 v2:172.21.15.119:3300/0 7 ==== osd_map(71..71 src has 1..71) v4 ==== 970+0+0 (secure 0 0 0) 0x7f19fc059da0 con 0x7f19ec002260 2023-11-08T19:27:48.784 INFO:teuthology.orchestra.run.smithi119.stdout:removed pool test_data_pool snap snap4 2023-11-08T19:27:48.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.784+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19e0069850 msgr2=0x7f19e006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.784+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19e0069850 0x7f19e006bd10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f19f0055000 tx=0x7f19f004c070 comp rx=0 tx=0).stop 2023-11-08T19:27:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.784+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 msgr2=0x559df6cac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.784+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 0x559df6cac020 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x559df6d58920 tx=0x7f19fc001a00 comp rx=0 tx=0).stop 2023-11-08T19:27:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 shutdown_connections 2023-11-08T19:27:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19e0069850 0x7f19e006bd10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19ec002260 0x559df6cac020 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x559df6d58fb0 0x559df6cabae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 --2- 172.21.15.119:0/3879461170 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x559df6d58bd0 0x559df6d42f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 >> 172.21.15.119:0/3879461170 conn(0x559df6c7ca30 msgr2=0x559df6d50ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 shutdown_connections 2023-11-08T19:27:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:48.785+0000 7f1a03a0bf40 1 -- 172.21.15.119:0/3879461170 wait complete. 2023-11-08T19:27:48.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_data_pool 2023-11-08T19:27:49.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.001+0000 7f88f672c640 1 Processor -- start 2023-11-08T19:27:49.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.002+0000 7f88f672c640 1 -- start start 2023-11-08T19:27:49.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.003+0000 7f88f672c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f006a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.003+0000 7f88f672c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f88f00cc6d0 0x7f88f00ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.003+0000 7f88f672c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f0162ae0 con 0x7f88f006ba30 2023-11-08T19:27:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.003+0000 7f88f672c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f0165de0 con 0x7f88f00cc6d0 2023-11-08T19:27:49.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.003+0000 7f88f672c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f00cd050 con 0x7f88f0144690 2023-11-08T19:27:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f88f00cc6d0 0x7f88f00ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f88f00cc6d0 0x7f88f00ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41790/0 (socket says 172.21.15.119:41790) 2023-11-08T19:27:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 -- 172.21.15.119:0/3845476199 learned_addr learned my addr 172.21.15.119:0/3845476199 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:49.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88effff640 1 --2- 172.21.15.119:0/3845476199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 -- 172.21.15.119:0/3845476199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 msgr2=0x7f88f006a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 --2- 172.21.15.119:0/3845476199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f006a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 -- 172.21.15.119:0/3845476199 >> v1:172.21.15.188:6789/0 conn(0x7f88f006ba30 legacy=0x7f88f006bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.004+0000 7f88e7fff640 1 -- 172.21.15.119:0/3845476199 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f0165160 con 0x7f88f00cc6d0 2023-11-08T19:27:49.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.005+0000 7f88e7fff640 1 --2- 172.21.15.119:0/3845476199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f00cc6d0 0x7f88f00ccb10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009870 tx=0x7f88d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dbffd50d821132d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88ef7fe640 1 -- 172.21.15.119:0/3845476199 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8031640 con 0x7f88f00cc6d0 2023-11-08T19:27:49.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88ef7fe640 1 -- 172.21.15.119:0/3845476199 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d80317a0 con 0x7f88f00cc6d0 2023-11-08T19:27:49.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88ef7fe640 1 -- 172.21.15.119:0/3845476199 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8031b30 con 0x7f88f00cc6d0 2023-11-08T19:27:49.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 -- 172.21.15.119:0/3845476199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f00cc6d0 msgr2=0x7f88f00ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 --2- 172.21.15.119:0/3845476199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f00cc6d0 0x7f88f00ccb10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009870 tx=0x7f88d802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 -- 172.21.15.119:0/3845476199 shutdown_connections 2023-11-08T19:27:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 --2- 172.21.15.119:0/3845476199 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f00cc6d0 0x7f88f00ccb10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 --2- 172.21.15.119:0/3845476199 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 -- 172.21.15.119:0/3845476199 >> 172.21.15.119:0/3845476199 conn(0x7f88f00cb610 msgr2=0x7f88f00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 -- 172.21.15.119:0/3845476199 shutdown_connections 2023-11-08T19:27:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.006+0000 7f88f672c640 1 -- 172.21.15.119:0/3845476199 wait complete. 2023-11-08T19:27:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.008+0000 7f88f672c640 1 Processor -- start 2023-11-08T19:27:49.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.008+0000 7f88f672c640 1 -- start start 2023-11-08T19:27:49.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88f672c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f006ba30 0x7f88f00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88f672c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f00cc6d0 0x7f88f0143940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88f672c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f0163e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88f672c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0158430 con 0x7f88f006ba30 2023-11-08T19:27:49.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88f672c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f0143e80 con 0x7f88f00cc6d0 2023-11-08T19:27:49.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88f672c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f00c5d50 con 0x7f88f0144690 2023-11-08T19:27:49.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.009+0000 7f88effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f006ba30 0x7f88f00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f006ba30 0x7f88f00c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41796/0 (socket says 172.21.15.119:41796) 2023-11-08T19:27:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 -- 172.21.15.119:0/1324434272 learned_addr learned my addr 172.21.15.119:0/1324434272 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88e7fff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f00cc6d0 0x7f88f0143940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88f4cc9640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f0163e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 -- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 msgr2=0x7f88f0163e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f0163e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 -- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f00cc6d0 msgr2=0x7f88f0143940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f00cc6d0 0x7f88f0143940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 -- 172.21.15.119:0/1324434272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d80094d0 con 0x7f88f006ba30 2023-11-08T19:27:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88e7fff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f00cc6d0 0x7f88f0143940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88f4cc9640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88f0144690 0x7f88f0163e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88effff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f006ba30 0x7f88f00c6b10 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f88e000d5f0 tx=0x7f88e000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e00125f0 con 0x7f88f006ba30 2023-11-08T19:27:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.010+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e0012750 con 0x7f88f006ba30 2023-11-08T19:27:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.011+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0012b00 con 0x7f88f006ba30 2023-11-08T19:27:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.011+0000 7f88f672c640 1 -- 172.21.15.119:0/1324434272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f00c6030 con 0x7f88f006ba30 2023-11-08T19:27:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.011+0000 7f88f672c640 1 -- 172.21.15.119:0/1324434272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f00c65f0 con 0x7f88f006ba30 2023-11-08T19:27:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.012+0000 7f88f672c640 1 -- 172.21.15.119:0/1324434272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f0074430 con 0x7f88f006ba30 2023-11-08T19:27:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.013+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88e0012c60 con 0x7f88f006ba30 2023-11-08T19:27:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.013+0000 7f88ed7fa640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d006ae70 0x7f88d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.014+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f88e008a160 con 0x7f88f006ba30 2023-11-08T19:27:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.014+0000 7f88e7fff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d006ae70 0x7f88d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.014+0000 7f88e7fff640 1 --2- 172.21.15.119:0/1324434272 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d006ae70 0x7f88d006d330 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f88d8036000 tx=0x7f88d803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.018+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f88e0054b90 con 0x7f88f006ba30 2023-11-08T19:27:49.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.195+0000 7f88f672c640 1 -- 172.21.15.119:0/1324434272 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_data_pool"} v 0) v1 -- 0x7f88f006ae90 con 0x7f88f006ba30 2023-11-08T19:27:49.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.197+0000 7f88ed7fa640 1 -- 172.21.15.119:0/1324434272 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_data_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v71) v1 ==== 296+0+0 (secure 0 0 0) 0x7f88e00581b0 con 0x7f88f006ba30 2023-11-08T19:27:49.198 INFO:teuthology.orchestra.run.smithi119.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_data_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-08T19:27:49.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:49.213 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool delete test_metadata_pool 2023-11-08T19:27:49.396 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:49.405 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:27:49.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.410+0000 7f48f6916640 1 Processor -- start 2023-11-08T19:27:49.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.410+0000 7f48f6916640 1 -- start start 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.411+0000 7f48f6916640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f48f0143520 0x7f48f00caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.411+0000 7f48f6916640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f48f00c5200 0x7f48f015d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.411+0000 7f48f6916640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f00c9d50 con 0x7f48f0143900 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f6916640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f015e5e0 con 0x7f48f00c5200 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f6916640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f00c9fa0 con 0x7f48f0143520 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f48f00c5200 0x7f48f015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f48f00c5200 0x7f48f015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41800/0 (socket says 172.21.15.119:41800) 2023-11-08T19:27:49.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 -- 172.21.15.119:0/2378204999 learned_addr learned my addr 172.21.15.119:0/2378204999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 -- 172.21.15.119:0/2378204999 >> v2:172.21.15.188:3300/0 conn(0x7f48f0143520 msgr2=0x7f48f00caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 --2- 172.21.15.119:0/2378204999 >> v2:172.21.15.188:3300/0 conn(0x7f48f0143520 0x7f48f00caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 -- 172.21.15.119:0/2378204999 >> v1:172.21.15.188:6789/0 conn(0x7f48f0143900 legacy=0x7f48f00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:49.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.412+0000 7f48f4eb3640 1 -- 172.21.15.119:0/2378204999 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f00ca280 con 0x7f48f00c5200 2023-11-08T19:27:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.413+0000 7f48f4eb3640 1 --2- 172.21.15.119:0/2378204999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f00c5200 0x7f48f015d870 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009e50 tx=0x7f48e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a3c36b7858311a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48eeffd640 1 -- 172.21.15.119:0/2378204999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e802ca70 con 0x7f48f00c5200 2023-11-08T19:27:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48eeffd640 1 -- 172.21.15.119:0/2378204999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e802cbd0 con 0x7f48f00c5200 2023-11-08T19:27:49.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48eeffd640 1 -- 172.21.15.119:0/2378204999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e80343f0 con 0x7f48f00c5200 2023-11-08T19:27:49.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48f6916640 1 -- 172.21.15.119:0/2378204999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f00c5200 msgr2=0x7f48f015d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48f6916640 1 --2- 172.21.15.119:0/2378204999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f00c5200 0x7f48f015d870 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009e50 tx=0x7f48e802bed0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48f6916640 1 -- 172.21.15.119:0/2378204999 shutdown_connections 2023-11-08T19:27:49.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48f6916640 1 --2- 172.21.15.119:0/2378204999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f00c5200 0x7f48f015d870 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48f6916640 1 --2- 172.21.15.119:0/2378204999 >> v2:172.21.15.188:3300/0 conn(0x7f48f0143520 0x7f48f00caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.414+0000 7f48f6916640 1 -- 172.21.15.119:0/2378204999 >> 172.21.15.119:0/2378204999 conn(0x7f48f00b0950 msgr2=0x7f48f00b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:49.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.415+0000 7f48f6916640 1 -- 172.21.15.119:0/2378204999 shutdown_connections 2023-11-08T19:27:49.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.415+0000 7f48f6916640 1 -- 172.21.15.119:0/2378204999 wait complete. 2023-11-08T19:27:49.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.417+0000 7f48f6916640 1 Processor -- start 2023-11-08T19:27:49.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.417+0000 7f48f6916640 1 -- start start 2023-11-08T19:27:49.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f6916640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f00c5200 0x7f48f00c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f6916640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f0143520 0x7f48f00c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f6916640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01548b0 0x7f48f0154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f6916640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00c87e0 con 0x7f48f01548b0 2023-11-08T19:27:49.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f6916640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00c7b60 con 0x7f48f0143520 2023-11-08T19:27:49.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f6916640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f00c7e40 con 0x7f48f00c5200 2023-11-08T19:27:49.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01548b0 0x7f48f0154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01548b0 0x7f48f0154d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41816/0 (socket says 172.21.15.119:41816) 2023-11-08T19:27:49.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 -- 172.21.15.119:0/4159158642 learned_addr learned my addr 172.21.15.119:0/4159158642 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:49.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48ef7fe640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f0143520 0x7f48f00c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48effff640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f00c5200 0x7f48f00c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 -- 172.21.15.119:0/4159158642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f00c5200 msgr2=0x7f48f00c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f48f00c5200 0x7f48f00c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 -- 172.21.15.119:0/4159158642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f0143520 msgr2=0x7f48f00c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:49.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f48f0143520 0x7f48f00c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:49.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.418+0000 7f48f4eb3640 1 -- 172.21.15.119:0/4159158642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8009b30 con 0x7f48f01548b0 2023-11-08T19:27:49.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.419+0000 7f48f4eb3640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f48f01548b0 0x7f48f0154d40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f48e8036040 tx=0x7f48e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.419+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e803e450 con 0x7f48f01548b0 2023-11-08T19:27:49.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.419+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e803e5d0 con 0x7f48f01548b0 2023-11-08T19:27:49.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.419+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e803e980 con 0x7f48f01548b0 2023-11-08T19:27:49.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.419+0000 7f48f6916640 1 -- 172.21.15.119:0/4159158642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f00c8120 con 0x7f48f01548b0 2023-11-08T19:27:49.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.419+0000 7f48f6916640 1 -- 172.21.15.119:0/4159158642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f00c7860 con 0x7f48f01548b0 2023-11-08T19:27:49.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.420+0000 7f48f6916640 1 -- 172.21.15.119:0/4159158642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f00c8920 con 0x7f48f01548b0 2023-11-08T19:27:49.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.421+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48e803eae0 con 0x7f48f01548b0 2023-11-08T19:27:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.422+0000 7f48ecff9640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f48dc06ae70 0x7f48dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:49.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.422+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f48e80acb50 con 0x7f48f01548b0 2023-11-08T19:27:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.423+0000 7f48effff640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f48dc06ae70 0x7f48dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.423+0000 7f48effff640 1 --2- 172.21.15.119:0/4159158642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f48dc06ae70 0x7f48dc06d330 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f48e0009890 tx=0x7f48e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:49.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.427+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48e8077580 con 0x7f48f01548b0 2023-11-08T19:27:49.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.606+0000 7f48f6916640 1 -- 172.21.15.119:0/4159158642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "test_metadata_pool"} v 0) v1 -- 0x7f48f004ece0 con 0x7f48f01548b0 2023-11-08T19:27:49.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:49.607+0000 7f48ecff9640 1 -- 172.21.15.119:0/4159158642 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "test_metadata_pool"}]=-1 WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. v71) v1 ==== 304+0+0 (secure 0 0 0) 0x7f48e807aba0 con 0x7f48f01548b0 2023-11-08T19:27:49.608 INFO:teuthology.orchestra.run.smithi119.stderr:Error EPERM: WARNING: this will *PERMANENTLY DESTROY* all data stored in pool test_metadata_pool. If you are *ABSOLUTELY CERTAIN* that is what you want, pass the pool name *twice*, followed by --yes-i-really-really-mean-it. 2023-11-08T19:27:49.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:27:49.621 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:27:49.622 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:49.745 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:27:49.877 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:27:49.878 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:49.878 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:49.938 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:49.939 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:49.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:50.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.210+0000 7f1156acf640 1 Processor -- start 2023-11-08T19:27:50.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.210+0000 7f1156acf640 1 -- start start 2023-11-08T19:27:50.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f1156acf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11501447a0 0x7f115015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f1156acf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11500cad30 0x7f115006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f1156acf640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11500c3a50 con 0x7f11500cb170 2023-11-08T19:27:50.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f1156acf640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1150144b80 con 0x7f11501447a0 2023-11-08T19:27:50.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f1156acf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11500c9150 con 0x7f11500cad30 2023-11-08T19:27:50.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f115486b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11501447a0 0x7f115015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f115486b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f11501447a0 0x7f115015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41822/0 (socket says 172.21.15.119:41822) 2023-11-08T19:27:50.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.212+0000 7f115486b640 1 -- 172.21.15.119:0/3928814623 learned_addr learned my addr 172.21.15.119:0/3928814623 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:50.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.213+0000 7f115486b640 1 -- 172.21.15.119:0/3928814623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11500cad30 msgr2=0x7f115006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:50.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.213+0000 7f115486b640 1 --2- 172.21.15.119:0/3928814623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11500cad30 0x7f115006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.213+0000 7f115486b640 1 -- 172.21.15.119:0/3928814623 >> v1:172.21.15.188:6789/0 conn(0x7f11500cb170 legacy=0x7f115006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:50.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.213+0000 7f115486b640 1 -- 172.21.15.119:0/3928814623 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500c9410 con 0x7f11501447a0 2023-11-08T19:27:50.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.214+0000 7f115486b640 1 --2- 172.21.15.119:0/3928814623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11501447a0 0x7f115015f4c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1144009870 tx=0x7f114402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3770dbf2caa3eb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.214+0000 7f114f7fe640 1 -- 172.21.15.119:0/3928814623 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144031640 con 0x7f11501447a0 2023-11-08T19:27:50.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.214+0000 7f114f7fe640 1 -- 172.21.15.119:0/3928814623 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11440317a0 con 0x7f11501447a0 2023-11-08T19:27:50.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.214+0000 7f114f7fe640 1 -- 172.21.15.119:0/3928814623 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144031b30 con 0x7f11501447a0 2023-11-08T19:27:50.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 -- 172.21.15.119:0/3928814623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11501447a0 msgr2=0x7f115015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 --2- 172.21.15.119:0/3928814623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11501447a0 0x7f115015f4c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1144009870 tx=0x7f114402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 -- 172.21.15.119:0/3928814623 shutdown_connections 2023-11-08T19:27:50.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 --2- 172.21.15.119:0/3928814623 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11500cad30 0x7f115006f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 --2- 172.21.15.119:0/3928814623 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11501447a0 0x7f115015f4c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 -- 172.21.15.119:0/3928814623 >> 172.21.15.119:0/3928814623 conn(0x7f11500cb610 msgr2=0x7f11500cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:50.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 -- 172.21.15.119:0/3928814623 shutdown_connections 2023-11-08T19:27:50.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.215+0000 7f1156acf640 1 -- 172.21.15.119:0/3928814623 wait complete. 2023-11-08T19:27:50.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.217+0000 7f1156acf640 1 Processor -- start 2023-11-08T19:27:50.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.217+0000 7f1156acf640 1 -- start start 2023-11-08T19:27:50.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f1156acf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 0x7f115015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f1156acf640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11500cb170 0x7f11500b2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f1156acf640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11501447a0 0x7f11501421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f1156acf640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115015ae40 con 0x7f11500cad30 2023-11-08T19:27:50.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f1156acf640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11500ce6e0 con 0x7f11500cb170 2023-11-08T19:27:50.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f1156acf640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11500b29e0 con 0x7f11501447a0 2023-11-08T19:27:50.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f115486b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 0x7f115015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f115486b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 0x7f115015a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41836/0 (socket says 172.21.15.119:41836) 2023-11-08T19:27:50.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f115486b640 1 -- 172.21.15.119:0/3795109871 learned_addr learned my addr 172.21.15.119:0/3795109871 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:50.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f115486b640 1 -- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11501447a0 msgr2=0x7f11501421b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f115506c640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11501447a0 0x7f11501421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f114ffff640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11500cb170 0x7f11500b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.218+0000 7f115486b640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11501447a0 0x7f11501421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f115486b640 1 -- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11500cb170 msgr2=0x7f11500b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f115486b640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11500cb170 0x7f11500b2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f115486b640 1 -- 172.21.15.119:0/3795109871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11440094d0 con 0x7f11500cad30 2023-11-08T19:27:50.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f115486b640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 0x7f115015a900 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1144009870 tx=0x7f11440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114403ac60 con 0x7f11500cad30 2023-11-08T19:27:50.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114403adc0 con 0x7f11500cad30 2023-11-08T19:27:50.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1144044630 con 0x7f11500cad30 2023-11-08T19:27:50.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11500b43c0 con 0x7f11500cad30 2023-11-08T19:27:50.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.219+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11500b4980 con 0x7f11500cad30 2023-11-08T19:27:50.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.221+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1150074430 con 0x7f11500cad30 2023-11-08T19:27:50.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.221+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f114402f050 con 0x7f11500cad30 2023-11-08T19:27:50.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.222+0000 7f114d7fa640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f113406ae70 0x7f113406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.222+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f11440abbe0 con 0x7f11500cad30 2023-11-08T19:27:50.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.223+0000 7f114ffff640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f113406ae70 0x7f113406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.223+0000 7f114ffff640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f113406ae70 0x7f113406d330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11500c45c0 tx=0x7f113c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.227+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11440766a0 con 0x7f11500cad30 2023-11-08T19:27:50.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.418+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f115006caa0 con 0x7f11500cad30 2023-11-08T19:27:50.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.422+0000 7f114d7fa640 1 -- 172.21.15.119:0/3795109871 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+22199 (secure 0 0 0) 0x7f1144079cc0 con 0x7f11500cad30 2023-11-08T19:27:50.424 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:50.424 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":71,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:48.774452+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_removed_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f113406ae70 msgr2=0x7f113406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f113406ae70 0x7f113406d330 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11500c45c0 tx=0x7f113c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 msgr2=0x7f115015a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 0x7f115015a900 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1144009870 tx=0x7f11440046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 shutdown_connections 2023-11-08T19:27:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f113406ae70 0x7f113406d330 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11501447a0 0x7f11501421b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11500cb170 0x7f11500b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 --2- 172.21.15.119:0/3795109871 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11500cad30 0x7f115015a900 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.424+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 >> 172.21.15.119:0/3795109871 conn(0x7f11500cb610 msgr2=0x7f11500c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:50.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.425+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 shutdown_connections 2023-11-08T19:27:50.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.425+0000 7f1156acf640 1 -- 172.21.15.119:0/3795109871 wait complete. 2023-11-08T19:27:50.444 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:27:33.928136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:27:35.135101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:50.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:50.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.653+0000 7fcb01725640 1 Processor -- start 2023-11-08T19:27:50.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.653+0000 7fcb01725640 1 -- start start 2023-11-08T19:27:50.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcb01725640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcafc142ad0 0x7fcafc0b1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcb01725640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0b3150 0x7fcafc0af780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcb01725640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcafc15e350 con 0x7fcafc0afcc0 2023-11-08T19:27:50.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcb01725640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcafc0b0130 con 0x7fcafc142ad0 2023-11-08T19:27:50.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcb01725640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0cbf10 con 0x7fcafc0b3150 2023-11-08T19:27:50.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcafaffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcafc142ad0 0x7fcafc0b1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcafaffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcafc142ad0 0x7fcafc0b1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41842/0 (socket says 172.21.15.119:41842) 2023-11-08T19:27:50.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.655+0000 7fcafaffd640 1 -- 172.21.15.119:0/1429792374 learned_addr learned my addr 172.21.15.119:0/1429792374 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:50.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.656+0000 7fcafaffd640 1 -- 172.21.15.119:0/1429792374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0b3150 msgr2=0x7fcafc0af780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:50.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.656+0000 7fcafaffd640 1 --2- 172.21.15.119:0/1429792374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0b3150 0x7fcafc0af780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.656+0000 7fcafaffd640 1 -- 172.21.15.119:0/1429792374 >> v1:172.21.15.188:6789/0 conn(0x7fcafc0afcc0 legacy=0x7fcafc06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:50.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.656+0000 7fcafaffd640 1 -- 172.21.15.119:0/1429792374 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc15e7e0 con 0x7fcafc142ad0 2023-11-08T19:27:50.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.657+0000 7fcafaffd640 1 --2- 172.21.15.119:0/1429792374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc142ad0 0x7fcafc0b1a00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009870 tx=0x7fcaec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9501bd7af6ab1c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.657+0000 7fcaf9ffb640 1 -- 172.21.15.119:0/1429792374 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0317c0 con 0x7fcafc142ad0 2023-11-08T19:27:50.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.657+0000 7fcaf9ffb640 1 -- 172.21.15.119:0/1429792374 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec031920 con 0x7fcafc142ad0 2023-11-08T19:27:50.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.657+0000 7fcaf9ffb640 1 -- 172.21.15.119:0/1429792374 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec031cb0 con 0x7fcafc142ad0 2023-11-08T19:27:50.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 -- 172.21.15.119:0/1429792374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc142ad0 msgr2=0x7fcafc0b1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 --2- 172.21.15.119:0/1429792374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc142ad0 0x7fcafc0b1a00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fcaec009870 tx=0x7fcaec02b860 comp rx=0 tx=0).stop 2023-11-08T19:27:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 -- 172.21.15.119:0/1429792374 shutdown_connections 2023-11-08T19:27:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 --2- 172.21.15.119:0/1429792374 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc0b3150 0x7fcafc0af780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 --2- 172.21.15.119:0/1429792374 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc142ad0 0x7fcafc0b1a00 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 -- 172.21.15.119:0/1429792374 >> 172.21.15.119:0/1429792374 conn(0x7fcafc0b08e0 msgr2=0x7fcafc0b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:50.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 -- 172.21.15.119:0/1429792374 shutdown_connections 2023-11-08T19:27:50.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.658+0000 7fcb01725640 1 -- 172.21.15.119:0/1429792374 wait complete. 2023-11-08T19:27:50.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.660+0000 7fcb01725640 1 Processor -- start 2023-11-08T19:27:50.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.660+0000 7fcb01725640 1 -- start start 2023-11-08T19:27:50.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcb01725640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 0x7fcafc0ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcb01725640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc0b3150 0x7fcafc158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcb01725640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc142ad0 0x7fcafc13af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcb01725640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc15ad10 con 0x7fcafc0afcc0 2023-11-08T19:27:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcb01725640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0ae900 con 0x7fcafc0b3150 2023-11-08T19:27:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcb01725640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcafc0aebe0 con 0x7fcafc142ad0 2023-11-08T19:27:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcafaffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 0x7fcafc0ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcafaffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 0x7fcafc0ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41850/0 (socket says 172.21.15.119:41850) 2023-11-08T19:27:50.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.661+0000 7fcafaffd640 1 -- 172.21.15.119:0/1921223736 learned_addr learned my addr 172.21.15.119:0/1921223736 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafa7fc640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc0b3150 0x7fcafc158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafaffd640 1 -- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc142ad0 msgr2=0x7fcafc13af50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T19:27:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafaffd640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc142ad0 0x7fcafc13af50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafaffd640 1 -- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc0b3150 msgr2=0x7fcafc158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafaffd640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc0b3150 0x7fcafc158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafaffd640 1 -- 172.21.15.119:0/1921223736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec0094d0 con 0x7fcafc0afcc0 2023-11-08T19:27:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafa7fc640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc0b3150 0x7fcafc158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcafaffd640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 0x7fcafc0ae320 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c00 tx=0x7fcaec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0329a0 con 0x7fcafc0afcc0 2023-11-08T19:27:50.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaec032b20 con 0x7fcafc0afcc0 2023-11-08T19:27:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcafc0aeec0 con 0x7fcafc0afcc0 2023-11-08T19:27:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaec0453f0 con 0x7fcafc0afcc0 2023-11-08T19:27:50.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.662+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcafc159940 con 0x7fcafc0afcc0 2023-11-08T19:27:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.664+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcafc071b60 con 0x7fcafc0afcc0 2023-11-08T19:27:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.665+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcaec04e450 con 0x7fcafc0afcc0 2023-11-08T19:27:50.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.665+0000 7fcae37fe640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcae806aea0 0x7fcae806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:50.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.666+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7fcaec0abee0 con 0x7fcafc0afcc0 2023-11-08T19:27:50.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.666+0000 7fcafa7fc640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcae806aea0 0x7fcae806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:50.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.667+0000 7fcafa7fc640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcae806aea0 0x7fcae806d360 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0006fd0 tx=0x7fcaf0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:50.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.670+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcaec076b60 con 0x7fcafc0afcc0 2023-11-08T19:27:50.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.862+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcafc04ece0 con 0x7fcafc0afcc0 2023-11-08T19:27:50.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.863+0000 7fcae37fe640 1 -- 172.21.15.119:0/1921223736 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcaec07a180 con 0x7fcafc0afcc0 2023-11-08T19:27:50.864 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcae806aea0 msgr2=0x7fcae806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcae806aea0 0x7fcae806d360 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0006fd0 tx=0x7fcaf0008040 comp rx=0 tx=0).stop 2023-11-08T19:27:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 msgr2=0x7fcafc0ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:50.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 0x7fcafc0ae320 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fcaec000c00 tx=0x7fcaec0046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 shutdown_connections 2023-11-08T19:27:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcae806aea0 0x7fcae806d360 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcafc142ad0 0x7fcafc13af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcafc0b3150 0x7fcafc158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 --2- 172.21.15.119:0/1921223736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcafc0afcc0 0x7fcafc0ae320 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 >> 172.21.15.119:0/1921223736 conn(0x7fcafc0b08e0 msgr2=0x7fcafc0cc050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.865+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 shutdown_connections 2023-11-08T19:27:50.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:50.866+0000 7fcb01725640 1 -- 172.21.15.119:0/1921223736 wait complete. 2023-11-08T19:27:50.884 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:51.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.090+0000 7fbf65687640 1 Processor -- start 2023-11-08T19:27:51.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.090+0000 7fbf65687640 1 -- start start 2023-11-08T19:27:51.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.091+0000 7fbf65687640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf60144580 0x7fbf6015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.091+0000 7fbf65687640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.091+0000 7fbf65687640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6015ea60 con 0x7fbf6015f1d0 2023-11-08T19:27:51.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.091+0000 7fbf65687640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf600c9150 con 0x7fbf60144580 2023-11-08T19:27:51.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf65687640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600cb270 con 0x7fbf60144960 2023-11-08T19:27:51.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf5effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf60144580 0x7fbf6015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf5effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf60144580 0x7fbf6015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41856/0 (socket says 172.21.15.119:41856) 2023-11-08T19:27:51.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf5effd640 1 -- 172.21.15.119:0/2850032182 learned_addr learned my addr 172.21.15.119:0/2850032182 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:51.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf57fff640 1 --2- 172.21.15.119:0/2850032182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf5effd640 1 -- 172.21.15.119:0/2850032182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 msgr2=0x7fbf600cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf5effd640 1 --2- 172.21.15.119:0/2850032182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.092+0000 7fbf5effd640 1 -- 172.21.15.119:0/2850032182 >> v1:172.21.15.188:6789/0 conn(0x7fbf6015f1d0 legacy=0x7fbf600c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.093+0000 7fbf5effd640 1 -- 172.21.15.119:0/2850032182 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf600c9360 con 0x7fbf60144580 2023-11-08T19:27:51.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.094+0000 7fbf5effd640 1 --2- 172.21.15.119:0/2850032182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf6015bff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009870 tx=0x7fbf5002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6c5c609e996c6f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.094+0000 7fbf5e7fc640 1 -- 172.21.15.119:0/2850032182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50031640 con 0x7fbf60144580 2023-11-08T19:27:51.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.094+0000 7fbf5e7fc640 1 -- 172.21.15.119:0/2850032182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf500317a0 con 0x7fbf60144580 2023-11-08T19:27:51.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.094+0000 7fbf5e7fc640 1 -- 172.21.15.119:0/2850032182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50031b30 con 0x7fbf60144580 2023-11-08T19:27:51.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.094+0000 7fbf65687640 1 -- 172.21.15.119:0/2850032182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 msgr2=0x7fbf6015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.094+0000 7fbf65687640 1 --2- 172.21.15.119:0/2850032182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf6015bff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009870 tx=0x7fbf5002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.095+0000 7fbf65687640 1 -- 172.21.15.119:0/2850032182 shutdown_connections 2023-11-08T19:27:51.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.095+0000 7fbf65687640 1 --2- 172.21.15.119:0/2850032182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.095+0000 7fbf65687640 1 --2- 172.21.15.119:0/2850032182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf6015bff0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.095+0000 7fbf65687640 1 -- 172.21.15.119:0/2850032182 >> 172.21.15.119:0/2850032182 conn(0x7fbf600cb610 msgr2=0x7fbf600cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.095+0000 7fbf65687640 1 -- 172.21.15.119:0/2850032182 shutdown_connections 2023-11-08T19:27:51.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.095+0000 7fbf65687640 1 -- 172.21.15.119:0/2850032182 wait complete. 2023-11-08T19:27:51.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.096+0000 7fbf65687640 1 Processor -- start 2023-11-08T19:27:51.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.096+0000 7fbf65687640 1 -- start start 2023-11-08T19:27:51.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.097+0000 7fbf65687640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf600cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.097+0000 7fbf65687640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.097+0000 7fbf65687640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf6015f1d0 0x7fbf600b3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.097+0000 7fbf65687640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6006fe50 con 0x7fbf60144580 2023-11-08T19:27:51.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.097+0000 7fbf65687640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600acc80 con 0x7fbf6015f1d0 2023-11-08T19:27:51.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.097+0000 7fbf65687640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf600acf60 con 0x7fbf60144960 2023-11-08T19:27:51.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf600cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf600cd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41862/0 (socket says 172.21.15.119:41862) 2023-11-08T19:27:51.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 -- 172.21.15.119:0/1973229916 learned_addr learned my addr 172.21.15.119:0/1973229916 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:51.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf57fff640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 -- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 msgr2=0x7fbf600cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5f7fe640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf6015f1d0 0x7fbf600b3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 -- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf6015f1d0 msgr2=0x7fbf600b3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf6015f1d0 0x7fbf600b3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 -- 172.21.15.119:0/1973229916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf500094d0 con 0x7fbf60144580 2023-11-08T19:27:51.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.098+0000 7fbf5effd640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf600cd080 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.099+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5003ac60 con 0x7fbf60144580 2023-11-08T19:27:51.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.099+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5003ade0 con 0x7fbf60144580 2023-11-08T19:27:51.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.099+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50044630 con 0x7fbf60144580 2023-11-08T19:27:51.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.099+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf600ad240 con 0x7fbf60144580 2023-11-08T19:27:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.099+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf600ad800 con 0x7fbf60144580 2023-11-08T19:27:51.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.100+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6015f660 con 0x7fbf60144580 2023-11-08T19:27:51.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.101+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf5002f050 con 0x7fbf60144580 2023-11-08T19:27:51.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.102+0000 7fbf577fe640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.102+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7fbf500abd40 con 0x7fbf60144580 2023-11-08T19:27:51.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.102+0000 7fbf57fff640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.103+0000 7fbf57fff640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf4006ae70 0x7fbf4006d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf48006fd0 tx=0x7fbf48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.106+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf50076800 con 0x7fbf60144580 2023-11-08T19:27:51.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.298+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7fbf60074430 con 0x7fbf60144580 2023-11-08T19:27:51.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.299+0000 7fbf577fe640 1 -- 172.21.15.119:0/1973229916 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v71) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbf500350a0 con 0x7fbf60144580 2023-11-08T19:27:51.301 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:51.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf4006ae70 msgr2=0x7fbf4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf4006ae70 0x7fbf4006d330 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbf48006fd0 tx=0x7fbf48008040 comp rx=0 tx=0).stop 2023-11-08T19:27:51.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 msgr2=0x7fbf600cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf600cd080 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000c00 tx=0x7fbf500046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 shutdown_connections 2023-11-08T19:27:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf4006ae70 0x7fbf4006d330 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf6015f1d0 0x7fbf600b3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf60144960 0x7fbf600cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 --2- 172.21.15.119:0/1973229916 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf60144580 0x7fbf600cd080 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 >> 172.21.15.119:0/1973229916 conn(0x7fbf600cb610 msgr2=0x7fbf600c4510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 shutdown_connections 2023-11-08T19:27:51.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.302+0000 7fbf65687640 1 -- 172.21.15.119:0/1973229916 wait complete. 2023-11-08T19:27:51.322 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:51.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.534+0000 7f9781590640 1 Processor -- start 2023-11-08T19:27:51.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.534+0000 7f9781590640 1 -- start start 2023-11-08T19:27:51.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f9781590640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f977c143600 0x7f977c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f9781590640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f977c15d9f0 0x7f977c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f9781590640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977c15e440 con 0x7f977c06cb40 2023-11-08T19:27:51.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f9781590640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c0c4f50 con 0x7f977c15d9f0 2023-11-08T19:27:51.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f9781590640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977c0c4960 con 0x7f977c143600 2023-11-08T19:27:51.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f977a7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f977c15d9f0 0x7f977c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f977b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f977c06cb40 0x7f977c06cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46586/0 (socket says 172.21.15.119:46586) 2023-11-08T19:27:51.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f977b7fe640 1 -- 172.21.15.119:0/732502518 learned_addr learned my addr 172.21.15.119:0/732502518 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:51.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f977affd640 1 --2- 172.21.15.119:0/732502518 >> v2:172.21.15.188:3300/0 conn(0x7f977c143600 0x7f977c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f977a7fc640 1 -- 172.21.15.119:0/732502518 >> v2:172.21.15.188:3300/0 conn(0x7f977c143600 msgr2=0x7f977c0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.536+0000 7f977a7fc640 1 --2- 172.21.15.119:0/732502518 >> v2:172.21.15.188:3300/0 conn(0x7f977c143600 0x7f977c0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.537+0000 7f977a7fc640 1 -- 172.21.15.119:0/732502518 >> v1:172.21.15.119:6789/0 conn(0x7f977c06cb40 legacy=0x7f977c06cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.537+0000 7f977a7fc640 1 -- 172.21.15.119:0/732502518 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977c15e720 con 0x7f977c15d9f0 2023-11-08T19:27:51.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f977a7fc640 1 --2- 172.21.15.119:0/732502518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c15d9f0 0x7f977c15de60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c3f30 tx=0x7f977002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ac5439dceb76ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9779ffb640 1 -- 172.21.15.119:0/732502518 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977002e070 con 0x7f977c15d9f0 2023-11-08T19:27:51.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9779ffb640 1 -- 172.21.15.119:0/732502518 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97700315a0 con 0x7f977c15d9f0 2023-11-08T19:27:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9779ffb640 1 -- 172.21.15.119:0/732502518 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770031930 con 0x7f977c15d9f0 2023-11-08T19:27:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9781590640 1 -- 172.21.15.119:0/732502518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c15d9f0 msgr2=0x7f977c15de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9781590640 1 --2- 172.21.15.119:0/732502518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c15d9f0 0x7f977c15de60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c3f30 tx=0x7f977002b840 comp rx=0 tx=0).stop 2023-11-08T19:27:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9781590640 1 -- 172.21.15.119:0/732502518 shutdown_connections 2023-11-08T19:27:51.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9781590640 1 --2- 172.21.15.119:0/732502518 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c15d9f0 0x7f977c15de60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.538+0000 7f9781590640 1 --2- 172.21.15.119:0/732502518 >> v2:172.21.15.188:3300/0 conn(0x7f977c143600 0x7f977c0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.539+0000 7f9781590640 1 -- 172.21.15.119:0/732502518 >> 172.21.15.119:0/732502518 conn(0x7f977c0cb610 msgr2=0x7f977c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.539+0000 7f9781590640 1 -- 172.21.15.119:0/732502518 shutdown_connections 2023-11-08T19:27:51.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.539+0000 7f9781590640 1 -- 172.21.15.119:0/732502518 wait complete. 2023-11-08T19:27:51.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.540+0000 7f9781590640 1 Processor -- start 2023-11-08T19:27:51.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.540+0000 7f9781590640 1 -- start start 2023-11-08T19:27:51.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.541+0000 7f9781590640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f977c06cb40 0x7f977c149ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.541+0000 7f9781590640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 0x7f977c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.541+0000 7f9781590640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f977c15d9f0 0x7f977c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.541+0000 7f9781590640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c0a8840 con 0x7f977c143600 2023-11-08T19:27:51.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.541+0000 7f9781590640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977c14c090 con 0x7f977c06cb40 2023-11-08T19:27:51.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.541+0000 7f9781590640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977c1f23b0 con 0x7f977c15d9f0 2023-11-08T19:27:51.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 0x7f977c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 0x7f977c06ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41874/0 (socket says 172.21.15.119:41874) 2023-11-08T19:27:51.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 -- 172.21.15.119:0/1836629295 learned_addr learned my addr 172.21.15.119:0/1836629295 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:51.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977affd640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f977c06cb40 0x7f977c149ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977b7fe640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f977c15d9f0 0x7f977c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 -- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f977c15d9f0 msgr2=0x7f977c06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f977c15d9f0 0x7f977c06bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 -- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f977c06cb40 msgr2=0x7f977c149ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f977c06cb40 0x7f977c149ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 -- 172.21.15.119:0/1836629295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770009400 con 0x7f977c143600 2023-11-08T19:27:51.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977affd640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f977c06cb40 0x7f977c149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:51.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.542+0000 7f977a7fc640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 0x7f977c06ba30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9770009890 tx=0x7f97700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.543+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770032840 con 0x7f977c143600 2023-11-08T19:27:51.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.543+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97700329a0 con 0x7f977c143600 2023-11-08T19:27:51.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.543+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770032d50 con 0x7f977c143600 2023-11-08T19:27:51.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.543+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977c1f2690 con 0x7f977c143600 2023-11-08T19:27:51.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.543+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977c1f2c50 con 0x7f977c143600 2023-11-08T19:27:51.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.544+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977c074430 con 0x7f977c143600 2023-11-08T19:27:51.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.545+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f977003b3f0 con 0x7f977c143600 2023-11-08T19:27:51.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.545+0000 7f97637fe640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976806ae70 0x7f976806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.545+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f97700abd50 con 0x7f977c143600 2023-11-08T19:27:51.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.546+0000 7f977affd640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976806ae70 0x7f976806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.546+0000 7f977affd640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976806ae70 0x7f976806d330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c8750 tx=0x7f976c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.550+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9770076780 con 0x7f977c143600 2023-11-08T19:27:51.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.742+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f977c0c4230 con 0x7f977c143600 2023-11-08T19:27:51.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.743+0000 7f97637fe640 1 -- 172.21.15.119:0/1836629295 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9770079da0 con 0x7f977c143600 2023-11-08T19:27:51.745 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:51.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976806ae70 msgr2=0x7f976806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976806ae70 0x7f976806d330 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f977c0c8750 tx=0x7f976c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:51.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 msgr2=0x7f977c06ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 0x7f977c06ba30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9770009890 tx=0x7f97700046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 shutdown_connections 2023-11-08T19:27:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f976806ae70 0x7f976806d330 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f977c15d9f0 0x7f977c06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f977c143600 0x7f977c06ba30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 --2- 172.21.15.119:0/1836629295 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f977c06cb40 0x7f977c149ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 >> 172.21.15.119:0/1836629295 conn(0x7f977c0cb610 msgr2=0x7f977c0ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.746+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 shutdown_connections 2023-11-08T19:27:51.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.747+0000 7f9781590640 1 -- 172.21.15.119:0/1836629295 wait complete. 2023-11-08T19:27:51.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.983+0000 7ff0d824f640 1 Processor -- start 2023-11-08T19:27:51.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.983+0000 7ff0d824f640 1 -- start start 2023-11-08T19:27:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.984+0000 7ff0d824f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.984+0000 7ff0d824f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff0d015f1d0 0x7ff0d00c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d824f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d015ea60 con 0x7ff0d0144580 2023-11-08T19:27:51.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d824f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d00c9150 con 0x7ff0d015f1d0 2023-11-08T19:27:51.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d824f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00c9360 con 0x7ff0d0144960 2023-11-08T19:27:51.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d57ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d57ea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:55740/0 (socket says 172.21.15.119:55740) 2023-11-08T19:27:51.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d57ea640 1 -- 172.21.15.119:0/3784870101 learned_addr learned my addr 172.21.15.119:0/3784870101 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:51.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.985+0000 7ff0d67ec640 1 --2- 172.21.15.119:0/3784870101 >> v2:172.21.15.188:3300/0 conn(0x7ff0d015f1d0 0x7ff0d00c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.986+0000 7ff0d57ea640 1 -- 172.21.15.119:0/3784870101 >> v2:172.21.15.188:3300/0 conn(0x7ff0d015f1d0 msgr2=0x7ff0d00c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.986+0000 7ff0d57ea640 1 --2- 172.21.15.119:0/3784870101 >> v2:172.21.15.188:3300/0 conn(0x7ff0d015f1d0 0x7ff0d00c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.986+0000 7ff0d57ea640 1 -- 172.21.15.119:0/3784870101 >> v1:172.21.15.188:6789/0 conn(0x7ff0d0144580 legacy=0x7ff0d00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.986+0000 7ff0d57ea640 1 -- 172.21.15.119:0/3784870101 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d00c9640 con 0x7ff0d0144960 2023-11-08T19:27:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.987+0000 7ff0d57ea640 1 --2- 172.21.15.119:0/3784870101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015bff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00c9290 tx=0x7ff0c002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a672dfc0e5beda3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.988+0000 7ff0d4fe9640 1 -- 172.21.15.119:0/3784870101 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0031640 con 0x7ff0d0144960 2023-11-08T19:27:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.988+0000 7ff0d4fe9640 1 -- 172.21.15.119:0/3784870101 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c00317a0 con 0x7ff0d0144960 2023-11-08T19:27:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.988+0000 7ff0d4fe9640 1 -- 172.21.15.119:0/3784870101 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0031b30 con 0x7ff0d0144960 2023-11-08T19:27:51.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 -- 172.21.15.119:0/3784870101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 msgr2=0x7ff0d015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 --2- 172.21.15.119:0/3784870101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015bff0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00c9290 tx=0x7ff0c002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 -- 172.21.15.119:0/3784870101 shutdown_connections 2023-11-08T19:27:51.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 --2- 172.21.15.119:0/3784870101 >> v2:172.21.15.188:3300/0 conn(0x7ff0d015f1d0 0x7ff0d00c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 --2- 172.21.15.119:0/3784870101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015bff0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 -- 172.21.15.119:0/3784870101 >> 172.21.15.119:0/3784870101 conn(0x7ff0d00cb610 msgr2=0x7ff0d00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 -- 172.21.15.119:0/3784870101 shutdown_connections 2023-11-08T19:27:51.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.989+0000 7ff0d824f640 1 -- 172.21.15.119:0/3784870101 wait complete. 2023-11-08T19:27:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.991+0000 7ff0d824f640 1 Processor -- start 2023-11-08T19:27:51.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.991+0000 7ff0d824f640 1 -- start start 2023-11-08T19:27:51.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.992+0000 7ff0d824f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d0144580 0x7ff0d00c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.992+0000 7ff0d824f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.992+0000 7ff0d824f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 0x7ff0d01f1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.992+0000 7ff0d824f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d00c8ac0 con 0x7ff0d015f1d0 2023-11-08T19:27:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.992+0000 7ff0d824f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d015b140 con 0x7ff0d0144580 2023-11-08T19:27:51.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.992+0000 7ff0d824f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01f1680 con 0x7ff0d0144960 2023-11-08T19:27:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 0x7ff0d01f1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 0x7ff0d01f1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41888/0 (socket says 172.21.15.119:41888) 2023-11-08T19:27:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 -- 172.21.15.119:0/161890611 learned_addr learned my addr 172.21.15.119:0/161890611 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d5feb640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d0144580 0x7ff0d00c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d57ea640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:51.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 -- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 msgr2=0x7ff0d015ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 -- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d0144580 msgr2=0x7ff0d00c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:51.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d0144580 0x7ff0d00c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:51.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.993+0000 7ff0d67ec640 1 -- 172.21.15.119:0/161890611 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c00094d0 con 0x7ff0d015f1d0 2023-11-08T19:27:51.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.994+0000 7ff0d67ec640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 0x7ff0d01f1140 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800ec10 tx=0x7ff0c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:51.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.994+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c800cd20 con 0x7ff0d015f1d0 2023-11-08T19:27:51.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.994+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c800f040 con 0x7ff0d015f1d0 2023-11-08T19:27:51.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.994+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80146c0 con 0x7ff0d015f1d0 2023-11-08T19:27:51.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.994+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d00c87e0 con 0x7ff0d015f1d0 2023-11-08T19:27:51.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.994+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d00c6050 con 0x7ff0d015f1d0 2023-11-08T19:27:52.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.995+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d00c5bd0 con 0x7ff0d015f1d0 2023-11-08T19:27:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.996+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0c8014a30 con 0x7ff0d015f1d0 2023-11-08T19:27:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.998+0000 7ff0c67fc640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff0b406ae70 0x7ff0b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.998+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7ff0c808a8e0 con 0x7ff0d015f1d0 2023-11-08T19:27:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.998+0000 7ff0d5feb640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff0b406ae70 0x7ff0b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:51.999+0000 7ff0d5feb640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff0b406ae70 0x7ff0b406d330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff0d013f390 tx=0x7ff0cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.002+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0d00c5bd0 con 0x7ff0d015f1d0 2023-11-08T19:27:52.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.192+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0d00c5bd0 con 0x7ff0d015f1d0 2023-11-08T19:27:52.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.193+0000 7ff0c67fc640 1 -- 172.21.15.119:0/161890611 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0c8018020 con 0x7ff0d015f1d0 2023-11-08T19:27:52.194 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:27:52.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff0b406ae70 msgr2=0x7ff0b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff0b406ae70 0x7ff0b406d330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff0d013f390 tx=0x7ff0cc009040 comp rx=0 tx=0).stop 2023-11-08T19:27:52.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 msgr2=0x7ff0d01f1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 0x7ff0d01f1140 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff0c800ec10 tx=0x7ff0c800c490 comp rx=0 tx=0).stop 2023-11-08T19:27:52.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 shutdown_connections 2023-11-08T19:27:52.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff0b406ae70 0x7ff0b406d330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff0d015f1d0 0x7ff0d01f1140 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0d0144960 0x7ff0d015ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 --2- 172.21.15.119:0/161890611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff0d0144580 0x7ff0d00c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 >> 172.21.15.119:0/161890611 conn(0x7ff0d00cb610 msgr2=0x7ff0d0072e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 shutdown_connections 2023-11-08T19:27:52.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.196+0000 7ff0d824f640 1 -- 172.21.15.119:0/161890611 wait complete. 2023-11-08T19:27:52.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:52.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.421+0000 7fd7a6287640 1 Processor -- start 2023-11-08T19:27:52.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.422+0000 7fd7a6287640 1 -- start start 2023-11-08T19:27:52.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.423+0000 7fd7a6287640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7a0144960 0x7fd7a015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.423+0000 7fd7a6287640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a015ea60 con 0x7fd7a015f1d0 2023-11-08T19:27:52.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.423+0000 7fd7a6287640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a00c9360 con 0x7fd7a0144580 2023-11-08T19:27:52.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.423+0000 7fd7a6287640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a00c9150 con 0x7fd7a0144960 2023-11-08T19:27:52.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.423+0000 7fd7a4824640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd7a015f1d0 0x7fd7a00c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46598/0 (socket says 172.21.15.119:46598) 2023-11-08T19:27:52.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.423+0000 7fd7a4824640 1 -- 172.21.15.119:0/2717208443 learned_addr learned my addr 172.21.15.119:0/2717208443 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:52.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd7977fe640 1 --2- 172.21.15.119:0/2717208443 >> v2:172.21.15.188:3300/0 conn(0x7fd7a0144960 0x7fd7a015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034493243 0 0) 0x7fd7a015ea60 con 0x7fd7a015f1d0 2023-11-08T19:27:52.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd780003520 con 0x7fd7a015f1d0 2023-11-08T19:27:52.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3452387521 0 0) 0x7fd780003520 con 0x7fd7a015f1d0 2023-11-08T19:27:52.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 >> v2:172.21.15.188:3300/0 conn(0x7fd7a0144960 msgr2=0x7fd7a015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:52.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 --2- 172.21.15.119:0/2717208443 >> v2:172.21.15.188:3300/0 conn(0x7fd7a0144960 0x7fd7a015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 >> v1:172.21.15.188:6789/0 conn(0x7fd7a0144580 legacy=0x7fd7a00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00c9640 con 0x7fd7a015f1d0 2023-11-08T19:27:52.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd7980019f0 con 0x7fd7a015f1d0 2023-11-08T19:27:52.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 >> v1:172.21.15.119:6789/0 conn(0x7fd7a015f1d0 legacy=0x7fd7a00c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7800039c0 0x7fd780003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7800047c0 0x7fd780004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800051d0 0x7fd78000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00c9150 con 0x7fd7800039c0 2023-11-08T19:27:52.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78000c120 con 0x7fd7800047c0 2023-11-08T19:27:52.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.424+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78000c2e0 con 0x7fd7800051d0 2023-11-08T19:27:52.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd79ffff640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7800039c0 0x7fd780003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd79ffff640 1 -- 172.21.15.119:0/2717208443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800051d0 msgr2=0x7fd78000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:52.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd79ffff640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800051d0 0x7fd78000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd79ffff640 1 -- 172.21.15.119:0/2717208443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7800047c0 msgr2=0x7fd780004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:52.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd79ffff640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7800047c0 0x7fd780004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd79ffff640 1 -- 172.21.15.119:0/2717208443 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780003520 con 0x7fd7800039c0 2023-11-08T19:27:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.425+0000 7fd7a4824640 1 -- 172.21.15.119:0/2717208443 reap_dead start 2023-11-08T19:27:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.426+0000 7fd79ffff640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7800039c0 0x7fd780003de0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd790001720 tx=0x7fd7800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6033b74a45f8b3f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.426+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7900052d0 con 0x7fd7800039c0 2023-11-08T19:27:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.426+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd79002e050 con 0x7fd7800039c0 2023-11-08T19:27:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.426+0000 7fd79f7fe640 1 -- 172.21.15.119:0/2717208443 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd790003830 con 0x7fd7800039c0 2023-11-08T19:27:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 -- 172.21.15.119:0/2717208443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7800039c0 msgr2=0x7fd780003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7800039c0 0x7fd780003de0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fd790001720 tx=0x7fd7800057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 -- 172.21.15.119:0/2717208443 shutdown_connections 2023-11-08T19:27:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 --2- 172.21.15.119:0/2717208443 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7800039c0 0x7fd780003de0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 -- 172.21.15.119:0/2717208443 >> 172.21.15.119:0/2717208443 conn(0x7fd7a00cb610 msgr2=0x7fd7a00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 -- 172.21.15.119:0/2717208443 shutdown_connections 2023-11-08T19:27:52.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.427+0000 7fd7a6287640 1 -- 172.21.15.119:0/2717208443 wait complete. 2023-11-08T19:27:52.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.429+0000 7fd7a6287640 1 Processor -- start 2023-11-08T19:27:52.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.429+0000 7fd7a6287640 1 -- start start 2023-11-08T19:27:52.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.430+0000 7fd7a6287640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800039c0 0x7fd7a00aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.430+0000 7fd7a6287640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 0x7fd7a0143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7a6287640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a00c76c0 0x7fd7a006a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7a6287640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a0143540 con 0x7fd7a0130030 2023-11-08T19:27:52.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7a6287640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a01304b0 con 0x7fd7a00c76c0 2023-11-08T19:27:52.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7a6287640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00af540 con 0x7fd7800039c0 2023-11-08T19:27:52.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 0x7fd7a0143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 0x7fd7a0143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41904/0 (socket says 172.21.15.119:41904) 2023-11-08T19:27:52.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 -- 172.21.15.119:0/3561866026 learned_addr learned my addr 172.21.15.119:0/3561866026 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:52.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd79ffff640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800039c0 0x7fd7a00aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7a4824640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a00c76c0 0x7fd7a006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 -- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800039c0 msgr2=0x7fd7a00aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800039c0 0x7fd7a00aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 -- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a00c76c0 msgr2=0x7fd7a006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a00c76c0 0x7fd7a006a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.431+0000 7fd7977fe640 1 -- 172.21.15.119:0/3561866026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00b3200 con 0x7fd7a0130030 2023-11-08T19:27:52.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.432+0000 7fd7977fe640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 0x7fd7a0143000 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e7c0 tx=0x7fd78800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.432+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800eed0 con 0x7fd7a0130030 2023-11-08T19:27:52.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.432+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78800c940 con 0x7fd7a0130030 2023-11-08T19:27:52.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.432+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78800ccf0 con 0x7fd7a0130030 2023-11-08T19:27:52.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.432+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a00b34e0 con 0x7fd7a0130030 2023-11-08T19:27:52.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.432+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a00b2ef0 con 0x7fd7a0130030 2023-11-08T19:27:52.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.433+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a00c4f00 con 0x7fd7a0130030 2023-11-08T19:27:52.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.434+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd788013470 con 0x7fd7a0130030 2023-11-08T19:27:52.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.435+0000 7fd79d7fa640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406aea0 0x7fd78406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.435+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7fd788089ba0 con 0x7fd7a0130030 2023-11-08T19:27:52.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.436+0000 7fd79ffff640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406aea0 0x7fd78406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.436+0000 7fd79ffff640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406aea0 0x7fd78406d360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd790033000 tx=0x7fd790009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.440+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd788054700 con 0x7fd7a0130030 2023-11-08T19:27:52.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.630+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fd7a006cb10 con 0x7fd7a0130030 2023-11-08T19:27:52.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.631+0000 7fd79d7fa640 1 -- 172.21.15.119:0/3561866026 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v71) v1 ==== 103+0+11 (secure 0 0 0) 0x7fd788057d20 con 0x7fd7a0130030 2023-11-08T19:27:52.633 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:52.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406aea0 msgr2=0x7fd78406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406aea0 0x7fd78406d360 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd790033000 tx=0x7fd790009030 comp rx=0 tx=0).stop 2023-11-08T19:27:52.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 msgr2=0x7fd7a0143000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 0x7fd7a0143000 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd78800e7c0 tx=0x7fd78800ec80 comp rx=0 tx=0).stop 2023-11-08T19:27:52.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 shutdown_connections 2023-11-08T19:27:52.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd78406aea0 0x7fd78406d360 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7a00c76c0 0x7fd7a006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7a0130030 0x7fd7a0143000 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 --2- 172.21.15.119:0/3561866026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7800039c0 0x7fd7a00aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 >> 172.21.15.119:0/3561866026 conn(0x7fd7a00cb610 msgr2=0x7fd7a00c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 shutdown_connections 2023-11-08T19:27:52.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.634+0000 7fd7a6287640 1 -- 172.21.15.119:0/3561866026 wait complete. 2023-11-08T19:27:52.656 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:52.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.858+0000 7f7c662da640 1 Processor -- start 2023-11-08T19:27:52.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.858+0000 7f7c662da640 1 -- start start 2023-11-08T19:27:52.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c662da640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c6015ad60 0x7f7c6006ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c662da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6006a120 con 0x7f7c6006c340 2023-11-08T19:27:52.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c662da640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6006a6e0 con 0x7f7c6015a980 2023-11-08T19:27:52.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c662da640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6006b0b0 con 0x7f7c6015ad60 2023-11-08T19:27:52.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c64ad7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c6015ad60 0x7f7c6006ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c65ad9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7c6006c340 0x7f7c6006c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46610/0 (socket says 172.21.15.119:46610) 2023-11-08T19:27:52.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.860+0000 7f7c65ad9640 1 -- 172.21.15.119:0/3085553 learned_addr learned my addr 172.21.15.119:0/3085553 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:52.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.861+0000 7f7c64ad7640 1 -- 172.21.15.119:0/3085553 >> v1:172.21.15.188:6789/0 conn(0x7f7c6015a980 legacy=0x7f7c600709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.861+0000 7f7c64ad7640 1 -- 172.21.15.119:0/3085553 >> v1:172.21.15.119:6789/0 conn(0x7f7c6006c340 legacy=0x7f7c6006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.861+0000 7f7c64ad7640 1 -- 172.21.15.119:0/3085553 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c600cba40 con 0x7f7c6015ad60 2023-11-08T19:27:52.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.861+0000 7f7c57fff640 1 -- 172.21.15.119:0/3085553 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3368142319 0 0) 0x7f7c6006a120 con 0x7f7c6006c340 2023-11-08T19:27:52.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.862+0000 7f7c64ad7640 1 --2- 172.21.15.119:0/3085553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015ad60 0x7f7c6006ab70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7c500097a0 tx=0x7f7c5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce756f97460feae9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.862+0000 7f7c57fff640 1 -- 172.21.15.119:0/3085553 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5002e070 con 0x7f7c6015ad60 2023-11-08T19:27:52.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.862+0000 7f7c57fff640 1 -- 172.21.15.119:0/3085553 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500315a0 con 0x7f7c6015ad60 2023-11-08T19:27:52.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.862+0000 7f7c57fff640 1 -- 172.21.15.119:0/3085553 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50031930 con 0x7f7c6015ad60 2023-11-08T19:27:52.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 -- 172.21.15.119:0/3085553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015ad60 msgr2=0x7f7c6006ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 --2- 172.21.15.119:0/3085553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015ad60 0x7f7c6006ab70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7c500097a0 tx=0x7f7c5002b840 comp rx=0 tx=0).stop 2023-11-08T19:27:52.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 -- 172.21.15.119:0/3085553 shutdown_connections 2023-11-08T19:27:52.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 --2- 172.21.15.119:0/3085553 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015ad60 0x7f7c6006ab70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 -- 172.21.15.119:0/3085553 >> 172.21.15.119:0/3085553 conn(0x7f7c600cfab0 msgr2=0x7f7c600cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:52.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 -- 172.21.15.119:0/3085553 shutdown_connections 2023-11-08T19:27:52.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.863+0000 7f7c662da640 1 -- 172.21.15.119:0/3085553 wait complete. 2023-11-08T19:27:52.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.865+0000 7f7c662da640 1 Processor -- start 2023-11-08T19:27:52.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.865+0000 7f7c662da640 1 -- start start 2023-11-08T19:27:52.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c662da640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c6006c340 0x7f7c600c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c662da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 0x7f7c600b0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c662da640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c6015ad60 0x7f7c6015eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c662da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6015fec0 con 0x7f7c6015a980 2023-11-08T19:27:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c662da640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c601588a0 con 0x7f7c6006c340 2023-11-08T19:27:52.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c662da640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6015f3f0 con 0x7f7c6015ad60 2023-11-08T19:27:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c64ad7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 0x7f7c600b0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c64ad7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 0x7f7c600b0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41916/0 (socket says 172.21.15.119:41916) 2023-11-08T19:27:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.866+0000 7f7c64ad7640 1 -- 172.21.15.119:0/1053744981 learned_addr learned my addr 172.21.15.119:0/1053744981 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.867+0000 7f7c64ad7640 1 -- 172.21.15.119:0/1053744981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c6015ad60 msgr2=0x7f7c6015eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.867+0000 7f7c64ad7640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c6015ad60 0x7f7c6015eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.867+0000 7f7c64ad7640 1 -- 172.21.15.119:0/1053744981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c6006c340 msgr2=0x7f7c600c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.867+0000 7f7c64ad7640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c6006c340 0x7f7c600c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.867+0000 7f7c64ad7640 1 -- 172.21.15.119:0/1053744981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c50009400 con 0x7f7c6015a980 2023-11-08T19:27:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.867+0000 7f7c64ad7640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 0x7f7c600b0380 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7c500098c0 tx=0x7f7c500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.868+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032900 con 0x7f7c6015a980 2023-11-08T19:27:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.868+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50032a80 con 0x7f7c6015a980 2023-11-08T19:27:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.868+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50032e30 con 0x7f7c6015a980 2023-11-08T19:27:52.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.868+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c600adf20 con 0x7f7c6015a980 2023-11-08T19:27:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.868+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6015f900 con 0x7f7c6015a980 2023-11-08T19:27:52.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.869+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c600668a0 con 0x7f7c6015a980 2023-11-08T19:27:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.870+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c5003b3f0 con 0x7f7c6015a980 2023-11-08T19:27:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.871+0000 7f7c55ffb640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4406ae70 0x7f7c4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.871+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f7c500abcc0 con 0x7f7c6015a980 2023-11-08T19:27:52.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.871+0000 7f7c652d8640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4406ae70 0x7f7c4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:52.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.872+0000 7f7c652d8640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4406ae70 0x7f7c4406d330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000f60 tx=0x7f7c5c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:52.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:52.875+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c500b2080 con 0x7f7c6015a980 2023-11-08T19:27:53.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.065+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f7c6004ece0 con 0x7f7c6015a980 2023-11-08T19:27:53.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.066+0000 7f7c55ffb640 1 -- 172.21.15.119:0/1053744981 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v71) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7c50076780 con 0x7f7c6015a980 2023-11-08T19:27:53.068 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:53.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.068+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4406ae70 msgr2=0x7f7c4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.068+0000 7f7c662da640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4406ae70 0x7f7c4406d330 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c000f60 tx=0x7f7c5c0079b0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.068+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 msgr2=0x7f7c600b0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.068+0000 7f7c662da640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 0x7f7c600b0380 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f7c500098c0 tx=0x7f7c500046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 shutdown_connections 2023-11-08T19:27:53.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c4406ae70 0x7f7c4406d330 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c6015ad60 0x7f7c6015eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c6015a980 0x7f7c600b0380 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 --2- 172.21.15.119:0/1053744981 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c6006c340 0x7f7c600c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 >> 172.21.15.119:0/1053744981 conn(0x7f7c600cfab0 msgr2=0x7f7c6006ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 shutdown_connections 2023-11-08T19:27:53.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.069+0000 7f7c662da640 1 -- 172.21.15.119:0/1053744981 wait complete. 2023-11-08T19:27:53.088 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:27:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.295+0000 7f104bf18640 1 Processor -- start 2023-11-08T19:27:53.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.295+0000 7f104bf18640 1 -- start start 2023-11-08T19:27:53.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.296+0000 7f104bf18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f10441447a0 0x7f104415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f104bf18640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f104406a120 0x7f104406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f104bf18640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104406aa70 con 0x7f10440cad30 2023-11-08T19:27:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f104bf18640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1044070040 con 0x7f10441447a0 2023-11-08T19:27:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f104bf18640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440c3a50 con 0x7f104406a120 2023-11-08T19:27:53.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f10441447a0 0x7f104415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f10441447a0 0x7f104415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41920/0 (socket says 172.21.15.119:41920) 2023-11-08T19:27:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 -- 172.21.15.119:0/1649909930 learned_addr learned my addr 172.21.15.119:0/1649909930 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f104a4b5640 1 --2- 172.21.15.119:0/1649909930 >> v2:172.21.15.188:3300/0 conn(0x7f104406a120 0x7f104406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 -- 172.21.15.119:0/1649909930 >> v2:172.21.15.188:3300/0 conn(0x7f104406a120 msgr2=0x7f104406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:53.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 --2- 172.21.15.119:0/1649909930 >> v2:172.21.15.188:3300/0 conn(0x7f104406a120 0x7f104406a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 -- 172.21.15.119:0/1649909930 >> v1:172.21.15.188:6789/0 conn(0x7f10440cad30 legacy=0x7f104406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f1049cb4640 1 -- 172.21.15.119:0/1649909930 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1044144b80 con 0x7f10441447a0 2023-11-08T19:27:53.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.297+0000 7f104a4b5640 1 --2- 172.21.15.119:0/1649909930 >> v2:172.21.15.188:3300/0 conn(0x7f104406a120 0x7f104406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.298+0000 7f1049cb4640 1 --2- 172.21.15.119:0/1649909930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10441447a0 0x7f104415f4c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f10440cfbd0 tx=0x7f104002b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83203b12658d9b3f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f1048cb2640 1 -- 172.21.15.119:0/1649909930 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040031640 con 0x7f10441447a0 2023-11-08T19:27:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f1048cb2640 1 -- 172.21.15.119:0/1649909930 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10400317a0 con 0x7f10441447a0 2023-11-08T19:27:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f1048cb2640 1 -- 172.21.15.119:0/1649909930 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040031b30 con 0x7f10441447a0 2023-11-08T19:27:53.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f104bf18640 1 -- 172.21.15.119:0/1649909930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10441447a0 msgr2=0x7f104415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f104bf18640 1 --2- 172.21.15.119:0/1649909930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10441447a0 0x7f104415f4c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f10440cfbd0 tx=0x7f104002b8e0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f104bf18640 1 -- 172.21.15.119:0/1649909930 shutdown_connections 2023-11-08T19:27:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f104bf18640 1 --2- 172.21.15.119:0/1649909930 >> v2:172.21.15.188:3300/0 conn(0x7f104406a120 0x7f104406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f104bf18640 1 --2- 172.21.15.119:0/1649909930 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10441447a0 0x7f104415f4c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.299+0000 7f104bf18640 1 -- 172.21.15.119:0/1649909930 >> 172.21.15.119:0/1649909930 conn(0x7f10440cb610 msgr2=0x7f10440cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.300+0000 7f104bf18640 1 -- 172.21.15.119:0/1649909930 shutdown_connections 2023-11-08T19:27:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.300+0000 7f104bf18640 1 -- 172.21.15.119:0/1649909930 wait complete. 2023-11-08T19:27:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.301+0000 7f104bf18640 1 Processor -- start 2023-11-08T19:27:53.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.302+0000 7f104bf18640 1 -- start start 2023-11-08T19:27:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.302+0000 7f104bf18640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104406a120 0x7f1044139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.302+0000 7f104bf18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 0x7f104413c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.302+0000 7f104bf18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10441447a0 0x7f104413c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f104bf18640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440c6ec0 con 0x7f10440cad30 2023-11-08T19:27:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f104bf18640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104413d470 con 0x7f104406a120 2023-11-08T19:27:53.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f104bf18640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104413cbd0 con 0x7f10441447a0 2023-11-08T19:27:53.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 0x7f104413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 0x7f104413c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41922/0 (socket says 172.21.15.119:41922) 2023-11-08T19:27:53.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 -- 172.21.15.119:0/2016006497 learned_addr learned my addr 172.21.15.119:0/2016006497 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:53.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 -- 172.21.15.119:0/2016006497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10441447a0 msgr2=0x7f104413c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10441447a0 0x7f104413c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 -- 172.21.15.119:0/2016006497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104406a120 msgr2=0x7f1044139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104406a120 0x7f1044139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.303+0000 7f10494b3640 1 -- 172.21.15.119:0/2016006497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10400094d0 con 0x7f10440cad30 2023-11-08T19:27:53.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.304+0000 7f10494b3640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 0x7f104413c010 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f103400b3f0 tx=0x7f103400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.304+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034011830 con 0x7f10440cad30 2023-11-08T19:27:53.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.304+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1034011990 con 0x7f10440cad30 2023-11-08T19:27:53.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.304+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1034011d40 con 0x7f10440cad30 2023-11-08T19:27:53.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.304+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104413ceb0 con 0x7f10440cad30 2023-11-08T19:27:53.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.304+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1044139dc0 con 0x7f10440cad30 2023-11-08T19:27:53.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.305+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10440c6c20 con 0x7f10440cad30 2023-11-08T19:27:53.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.306+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f103401a400 con 0x7f10440cad30 2023-11-08T19:27:53.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.307+0000 7f103a7fc640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f102806ae70 0x7f102806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:53.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.307+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f1034089a00 con 0x7f10440cad30 2023-11-08T19:27:53.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.307+0000 7f1049cb4640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f102806ae70 0x7f102806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:53.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.308+0000 7f1049cb4640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f102806ae70 0x7f102806d330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1040036000 tx=0x7f1040030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:53.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.311+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10440c6c20 con 0x7f10440cad30 2023-11-08T19:27:53.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.514+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10440c7000 con 0x7f10440cad30 2023-11-08T19:27:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.514+0000 7f103a7fc640 1 -- 172.21.15.119:0/2016006497 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) v1 ==== 80+0+0 (secure 0 0 0) 0x7f10440c7000 con 0x7f10440cad30 2023-11-08T19:27:53.516 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:27:53.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.516+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f102806ae70 msgr2=0x7f102806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f102806ae70 0x7f102806d330 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1040036000 tx=0x7f1040030000 comp rx=0 tx=0).stop 2023-11-08T19:27:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 msgr2=0x7f104413c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 0x7f104413c010 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f103400b3f0 tx=0x7f103400b8b0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 shutdown_connections 2023-11-08T19:27:53.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f102806ae70 0x7f102806d330 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f10441447a0 0x7f104413c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f10440cad30 0x7f104413c010 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 --2- 172.21.15.119:0/2016006497 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104406a120 0x7f1044139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:53.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 >> 172.21.15.119:0/2016006497 conn(0x7f10440cb610 msgr2=0x7f104415cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:53.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 shutdown_connections 2023-11-08T19:27:53.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:53.517+0000 7f104bf18640 1 -- 172.21.15.119:0/2016006497 wait complete. 2023-11-08T19:27:53.536 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:27:53.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:53.537 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:27:53.640 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:27:53.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:27:53.641 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:27:53.689 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:27:53.690 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:53.743 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:27:53.744 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:27:53.744 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:53.801 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:27:53.802 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:27:53.802 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:27:53.858 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:27:53.859 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.081+0000 7fe2073f6640 1 Processor -- start 2023-11-08T19:27:54.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.082+0000 7fe2073f6640 1 -- start start 2023-11-08T19:27:54.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.083+0000 7fe2073f6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe2000adc20 0x7fe2000c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.083+0000 7fe2073f6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20006f8d0 con 0x7fe2000ccd50 2023-11-08T19:27:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.083+0000 7fe2073f6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20006b9c0 con 0x7fe2000ae000 2023-11-08T19:27:54.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.083+0000 7fe2073f6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2000cac30 con 0x7fe2000adc20 2023-11-08T19:27:54.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.084+0000 7fe206bf5640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe2000ccd50 0x7fe20006be50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46612/0 (socket says 172.21.15.119:46612) 2023-11-08T19:27:54.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.084+0000 7fe206bf5640 1 -- 172.21.15.119:0/3511447080 learned_addr learned my addr 172.21.15.119:0/3511447080 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:54.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.084+0000 7fe2063f4640 1 --2- 172.21.15.119:0/3511447080 >> v2:172.21.15.188:3300/0 conn(0x7fe2000adc20 0x7fe2000c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.084+0000 7fe2053f2640 1 -- 172.21.15.119:0/3511447080 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1196836884 0 0) 0x7fe20006f8d0 con 0x7fe2000ccd50 2023-11-08T19:27:54.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.084+0000 7fe2053f2640 1 -- 172.21.15.119:0/3511447080 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1ec003520 con 0x7fe2000ccd50 2023-11-08T19:27:54.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.085+0000 7fe2063f4640 1 -- 172.21.15.119:0/3511447080 >> v1:172.21.15.188:6789/0 conn(0x7fe2000ae000 legacy=0x7fe2000cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.085+0000 7fe2063f4640 1 -- 172.21.15.119:0/3511447080 >> v1:172.21.15.119:6789/0 conn(0x7fe2000ccd50 legacy=0x7fe20006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.085+0000 7fe2063f4640 1 -- 172.21.15.119:0/3511447080 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2000c9600 con 0x7fe2000adc20 2023-11-08T19:27:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.086+0000 7fe2063f4640 1 --2- 172.21.15.119:0/3511447080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000adc20 0x7fe2000c90c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0097a0 tx=0x7fe1fc02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb63e25ec7b9e076 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.087+0000 7fe2053f2640 1 -- 172.21.15.119:0/3511447080 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc031830 con 0x7fe2000adc20 2023-11-08T19:27:54.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.087+0000 7fe2053f2640 1 -- 172.21.15.119:0/3511447080 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc031990 con 0x7fe2000adc20 2023-11-08T19:27:54.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.087+0000 7fe2053f2640 1 -- 172.21.15.119:0/3511447080 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc031d00 con 0x7fe2000adc20 2023-11-08T19:27:54.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 -- 172.21.15.119:0/3511447080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000adc20 msgr2=0x7fe2000c90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 --2- 172.21.15.119:0/3511447080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000adc20 0x7fe2000c90c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc0097a0 tx=0x7fe1fc02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 -- 172.21.15.119:0/3511447080 shutdown_connections 2023-11-08T19:27:54.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 --2- 172.21.15.119:0/3511447080 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000adc20 0x7fe2000c90c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 -- 172.21.15.119:0/3511447080 >> 172.21.15.119:0/3511447080 conn(0x7fe2000b0560 msgr2=0x7fe2000b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 -- 172.21.15.119:0/3511447080 shutdown_connections 2023-11-08T19:27:54.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.088+0000 7fe2073f6640 1 -- 172.21.15.119:0/3511447080 wait complete. 2023-11-08T19:27:54.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.090+0000 7fe2073f6640 1 Processor -- start 2023-11-08T19:27:54.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.090+0000 7fe2073f6640 1 -- start start 2023-11-08T19:27:54.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2073f6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 0x7fe2000c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2073f6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2000ae000 0x7fe20015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2073f6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000ccd50 0x7fe2000af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2073f6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000b23c0 con 0x7fe2000adc20 2023-11-08T19:27:54.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2073f6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c2ae0 con 0x7fe2000ccd50 2023-11-08T19:27:54.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2073f6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2000c35b0 con 0x7fe2000ae000 2023-11-08T19:27:54.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2063f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 0x7fe2000c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2063f4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 0x7fe2000c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41924/0 (socket says 172.21.15.119:41924) 2023-11-08T19:27:54.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.091+0000 7fe2063f4640 1 -- 172.21.15.119:0/2548944455 learned_addr learned my addr 172.21.15.119:0/2548944455 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:54.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe2063f4640 1 -- 172.21.15.119:0/2548944455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2000ae000 msgr2=0x7fe20015ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe2063f4640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2000ae000 0x7fe20015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe2063f4640 1 -- 172.21.15.119:0/2548944455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000ccd50 msgr2=0x7fe2000af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe2063f4640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000ccd50 0x7fe2000af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe2063f4640 1 -- 172.21.15.119:0/2548944455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc009400 con 0x7fe2000adc20 2023-11-08T19:27:54.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe2063f4640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 0x7fe2000c25a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc034040 tx=0x7fe1fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.092+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc0317b0 con 0x7fe2000adc20 2023-11-08T19:27:54.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.093+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc03f070 con 0x7fe2000adc20 2023-11-08T19:27:54.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.093+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20015e310 con 0x7fe2000adc20 2023-11-08T19:27:54.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.093+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc003610 con 0x7fe2000adc20 2023-11-08T19:27:54.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.093+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2000c2ff0 con 0x7fe2000adc20 2023-11-08T19:27:54.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.094+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe20004ece0 con 0x7fe2000adc20 2023-11-08T19:27:54.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.095+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1fc041030 con 0x7fe2000adc20 2023-11-08T19:27:54.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.096+0000 7fe1f6ffd640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe1e406aea0 0x7fe1e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.096+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fe1fc0ab950 con 0x7fe2000adc20 2023-11-08T19:27:54.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.096+0000 7fe205bf3640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe1e406aea0 0x7fe1e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.097+0000 7fe205bf3640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe1e406aea0 0x7fe1e406d360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0000f60 tx=0x7fe1f00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.100+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1fc076430 con 0x7fe2000adc20 2023-11-08T19:27:54.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.328+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2000b1320 con 0x7fe2000adc20 2023-11-08T19:27:54.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.330+0000 7fe1f6ffd640 1 -- 172.21.15.119:0/2548944455 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fe1fc079a50 con 0x7fe2000adc20 2023-11-08T19:27:54.332 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:54.332 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:36.122432+0000","modified":"2023-11-08T19:27:39.195905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/913059196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":913059196},{"type":"v1","addr":"172.21.15.188:6835","nonce":913059196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:54.332 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2023-11-08T19:27:54.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe1e406aea0 msgr2=0x7fe1e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe1e406aea0 0x7fe1e406d360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0000f60 tx=0x7fe1f00079b0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 msgr2=0x7fe2000c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 0x7fe2000c25a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc034040 tx=0x7fe1fc0046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 shutdown_connections 2023-11-08T19:27:54.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe1e406aea0 0x7fe1e406d360 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe2000ccd50 0x7fe2000af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe2000ae000 0x7fe20015ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 --2- 172.21.15.119:0/2548944455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe2000adc20 0x7fe2000c25a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 >> 172.21.15.119:0/2548944455 conn(0x7fe2000b0560 msgr2=0x7fe200071210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 shutdown_connections 2023-11-08T19:27:54.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.333+0000 7fe2073f6640 1 -- 172.21.15.119:0/2548944455 wait complete. 2023-11-08T19:27:54.352 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:54.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.557+0000 7f0158490640 1 Processor -- start 2023-11-08T19:27:54.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.558+0000 7f0158490640 1 -- start start 2023-11-08T19:27:54.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f0158490640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01501447a0 0x7f015015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f0158490640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01500c3a50 con 0x7f01500cad30 2023-11-08T19:27:54.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f0158490640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0150144b80 con 0x7f015006a120 2023-11-08T19:27:54.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f0158490640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0150070040 con 0x7f01501447a0 2023-11-08T19:27:54.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f015622c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01501447a0 0x7f015015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f015622c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01501447a0 0x7f015015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41938/0 (socket says 172.21.15.119:41938) 2023-11-08T19:27:54.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.559+0000 7f015622c640 1 -- 172.21.15.119:0/1547020072 learned_addr learned my addr 172.21.15.119:0/1547020072 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:54.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.560+0000 7f015622c640 1 -- 172.21.15.119:0/1547020072 >> v1:172.21.15.188:6789/0 conn(0x7f015006a120 legacy=0x7f015006a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:54.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.560+0000 7f015622c640 1 -- 172.21.15.119:0/1547020072 >> v1:172.21.15.119:6789/0 conn(0x7f01500cad30 legacy=0x7f015006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.560+0000 7f015622c640 1 -- 172.21.15.119:0/1547020072 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500c9150 con 0x7f01501447a0 2023-11-08T19:27:54.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.561+0000 7f015622c640 1 --2- 172.21.15.119:0/1547020072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f015015f4c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f014c009870 tx=0x7f014c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8c6048f3da98b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f015522a640 1 -- 172.21.15.119:0/1547020072 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c030860 con 0x7f01501447a0 2023-11-08T19:27:54.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f015522a640 1 -- 172.21.15.119:0/1547020072 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014c0309c0 con 0x7f01501447a0 2023-11-08T19:27:54.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f015522a640 1 -- 172.21.15.119:0/1547020072 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014c030d50 con 0x7f01501447a0 2023-11-08T19:27:54.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 -- 172.21.15.119:0/1547020072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 msgr2=0x7f015015f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 --2- 172.21.15.119:0/1547020072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f015015f4c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f014c009870 tx=0x7f014c02b9a0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 -- 172.21.15.119:0/1547020072 shutdown_connections 2023-11-08T19:27:54.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 --2- 172.21.15.119:0/1547020072 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f015015f4c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 -- 172.21.15.119:0/1547020072 >> 172.21.15.119:0/1547020072 conn(0x7f01500cb610 msgr2=0x7f01500cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 -- 172.21.15.119:0/1547020072 shutdown_connections 2023-11-08T19:27:54.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.562+0000 7f0158490640 1 -- 172.21.15.119:0/1547020072 wait complete. 2023-11-08T19:27:54.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.564+0000 7f0158490640 1 Processor -- start 2023-11-08T19:27:54.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.564+0000 7f0158490640 1 -- start start 2023-11-08T19:27:54.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0158490640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015006a120 0x7f015015a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0158490640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01500cad30 0x7f0150131d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0158490640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f01501322b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0158490640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015012fce0 con 0x7f01501447a0 2023-11-08T19:27:54.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0158490640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015012ff30 con 0x7f01500cad30 2023-11-08T19:27:54.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0158490640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0150130210 con 0x7f015006a120 2023-11-08T19:27:54.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0156a2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f01501322b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0156a2d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f01501322b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41950/0 (socket says 172.21.15.119:41950) 2023-11-08T19:27:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.565+0000 7f0156a2d640 1 -- 172.21.15.119:0/1683652301 learned_addr learned my addr 172.21.15.119:0/1683652301 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0156a2d640 1 -- 172.21.15.119:0/1683652301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015006a120 msgr2=0x7f015015a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0156a2d640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015006a120 0x7f015015a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0156a2d640 1 -- 172.21.15.119:0/1683652301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01500cad30 msgr2=0x7f0150131d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0156a2d640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01500cad30 0x7f0150131d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0156a2d640 1 -- 172.21.15.119:0/1683652301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f014c0094d0 con 0x7f01501447a0 2023-11-08T19:27:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0156a2d640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f01501322b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f014800b970 tx=0x7f014800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f014800cb40 con 0x7f01501447a0 2023-11-08T19:27:54.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f014800cca0 con 0x7f01501447a0 2023-11-08T19:27:54.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148014680 con 0x7f01501447a0 2023-11-08T19:27:54.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.566+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01501304f0 con 0x7f01501447a0 2023-11-08T19:27:54.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.567+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0150143000 con 0x7f01501447a0 2023-11-08T19:27:54.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.568+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0150130a10 con 0x7f01501447a0 2023-11-08T19:27:54.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.568+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f014801d3b0 con 0x7f01501447a0 2023-11-08T19:27:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.569+0000 7f0146ffd640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f013406ae70 0x7f013406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.569+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f0148089a60 con 0x7f01501447a0 2023-11-08T19:27:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.570+0000 7f015622c640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f013406ae70 0x7f013406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.570+0000 7f015622c640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f013406ae70 0x7f013406d330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f014c036000 tx=0x7f014c03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:54.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.573+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0150130a10 con 0x7f01501447a0 2023-11-08T19:27:54.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.766+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0150130a10 con 0x7f01501447a0 2023-11-08T19:27:54.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.768+0000 7f0146ffd640 1 -- 172.21.15.119:0/1683652301 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+22155 (secure 0 0 0) 0x7f0150130a10 con 0x7f01501447a0 2023-11-08T19:27:54.770 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:54.771 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":72,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:53.645252+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:54.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f013406ae70 msgr2=0x7f013406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f013406ae70 0x7f013406d330 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f014c036000 tx=0x7f014c03b040 comp rx=0 tx=0).stop 2023-11-08T19:27:54.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 msgr2=0x7f01501322b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:54.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f01501322b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f014800b970 tx=0x7f014800be30 comp rx=0 tx=0).stop 2023-11-08T19:27:54.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 shutdown_connections 2023-11-08T19:27:54.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f013406ae70 0x7f013406d330 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01501447a0 0x7f01501322b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01500cad30 0x7f0150131d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 --2- 172.21.15.119:0/1683652301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f015006a120 0x7f015015a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:54.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 >> 172.21.15.119:0/1683652301 conn(0x7f01500cb610 msgr2=0x7f0150070180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:54.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 shutdown_connections 2023-11-08T19:27:54.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:54.771+0000 7f0158490640 1 -- 172.21.15.119:0/1683652301 wait complete. 2023-11-08T19:27:54.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:27:33.928136+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:27:35.135101+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:27:54.795 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:27:55.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.011+0000 7f1c15b5e640 1 Processor -- start 2023-11-08T19:27:55.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.012+0000 7f1c15b5e640 1 -- start start 2023-11-08T19:27:55.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.013+0000 7f1c15b5e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c1015deb0 0x7f1c100cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.013+0000 7f1c15b5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c1015e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.013+0000 7f1c15b5e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c1015a8c0 0x7f1c1015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.013+0000 7f1c15b5e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c10071720 con 0x7f1c1015deb0 2023-11-08T19:27:55.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.013+0000 7f1c15b5e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c100cf5e0 con 0x7f1c1015a8c0 2023-11-08T19:27:55.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.013+0000 7f1c15b5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100cf8c0 con 0x7f1c1015e290 2023-11-08T19:27:55.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c1015deb0 0x7f1c100cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c1015deb0 0x7f1c100cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41954/0 (socket says 172.21.15.119:41954) 2023-11-08T19:27:55.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 -- 172.21.15.119:0/3214213728 learned_addr learned my addr 172.21.15.119:0/3214213728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:55.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c1535d640 1 --2- 172.21.15.119:0/3214213728 >> v2:172.21.15.188:3300/0 conn(0x7f1c1015a8c0 0x7f1c1015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c0ffff640 1 --2- 172.21.15.119:0/3214213728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c1015e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 -- 172.21.15.119:0/3214213728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 msgr2=0x7f1c1015e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 --2- 172.21.15.119:0/3214213728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c1015e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 -- 172.21.15.119:0/3214213728 >> v2:172.21.15.188:3300/0 conn(0x7f1c1015a8c0 msgr2=0x7f1c1015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 --2- 172.21.15.119:0/3214213728 >> v2:172.21.15.188:3300/0 conn(0x7f1c1015a8c0 0x7f1c1015ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.014+0000 7f1c14b5c640 1 -- 172.21.15.119:0/3214213728 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100710d0 con 0x7f1c1015deb0 2023-11-08T19:27:55.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.015+0000 7f1c14b5c640 1 --2- 172.21.15.119:0/3214213728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c100cccf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1c1015ef50 tx=0x7f1c0402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64363dbcb978bce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.015+0000 7f1c0f7fe640 1 -- 172.21.15.119:0/3214213728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c040317e0 con 0x7f1c1015deb0 2023-11-08T19:27:55.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c0f7fe640 1 -- 172.21.15.119:0/3214213728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c04031940 con 0x7f1c1015deb0 2023-11-08T19:27:55.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c0f7fe640 1 -- 172.21.15.119:0/3214213728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c04031cd0 con 0x7f1c1015deb0 2023-11-08T19:27:55.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 -- 172.21.15.119:0/3214213728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 msgr2=0x7f1c100cccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/3214213728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c100cccf0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1c1015ef50 tx=0x7f1c0402b880 comp rx=0 tx=0).stop 2023-11-08T19:27:55.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 -- 172.21.15.119:0/3214213728 shutdown_connections 2023-11-08T19:27:55.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/3214213728 >> v2:172.21.15.188:3300/0 conn(0x7f1c1015a8c0 0x7f1c1015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/3214213728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c1015e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/3214213728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c100cccf0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 -- 172.21.15.119:0/3214213728 >> 172.21.15.119:0/3214213728 conn(0x7f1c100ceac0 msgr2=0x7f1c100ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 -- 172.21.15.119:0/3214213728 shutdown_connections 2023-11-08T19:27:55.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.016+0000 7f1c15b5e640 1 -- 172.21.15.119:0/3214213728 wait complete. 2023-11-08T19:27:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.018+0000 7f1c15b5e640 1 Processor -- start 2023-11-08T19:27:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.018+0000 7f1c15b5e640 1 -- start start 2023-11-08T19:27:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.019+0000 7f1c15b5e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c1015a8c0 0x7f1c100c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.019+0000 7f1c15b5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c1014ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.019+0000 7f1c15b5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c100c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.019+0000 7f1c15b5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100c8eb0 con 0x7f1c1015deb0 2023-11-08T19:27:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.019+0000 7f1c15b5e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100c88f0 con 0x7f1c1015a8c0 2023-11-08T19:27:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.019+0000 7f1c15b5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100c8bd0 con 0x7f1c1015e290 2023-11-08T19:27:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c1014ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c1014ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41968/0 (socket says 172.21.15.119:41968) 2023-11-08T19:27:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 -- 172.21.15.119:0/461835823 learned_addr learned my addr 172.21.15.119:0/461835823 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c14b5c640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c1015a8c0 0x7f1c100c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c1535d640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c100c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 -- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 msgr2=0x7f1c100c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c100c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 -- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c1015a8c0 msgr2=0x7f1c100c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c1015a8c0 0x7f1c100c8210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 -- 172.21.15.119:0/461835823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040094d0 con 0x7f1c1015deb0 2023-11-08T19:27:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.020+0000 7f1c0ffff640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c1014ed70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000d800 tx=0x7f1c0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.021+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000b6c0 con 0x7f1c1015deb0 2023-11-08T19:27:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.021+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0000b820 con 0x7f1c1015deb0 2023-11-08T19:27:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.021+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0000bbb0 con 0x7f1c1015deb0 2023-11-08T19:27:55.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.021+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c100b1640 con 0x7f1c1015deb0 2023-11-08T19:27:55.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.021+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1014f2b0 con 0x7f1c1015deb0 2023-11-08T19:27:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.022+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1004ece0 con 0x7f1c1015deb0 2023-11-08T19:27:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.023+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c0000bd10 con 0x7f1c1015deb0 2023-11-08T19:27:55.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.024+0000 7f1c0d7fa640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1bfc06ae70 0x7f1bfc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.024+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f1c0008a140 con 0x7f1c1015deb0 2023-11-08T19:27:55.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.024+0000 7f1c14b5c640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1bfc06ae70 0x7f1bfc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.025+0000 7f1c14b5c640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1bfc06ae70 0x7f1bfc06d330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c04036000 tx=0x7f1c0403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.029+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c00054c20 con 0x7f1c1015deb0 2023-11-08T19:27:55.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.218+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1c100cb1c0 con 0x7f1c1015deb0 2023-11-08T19:27:55.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.219+0000 7f1c0d7fa640 1 -- 172.21.15.119:0/461835823 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1c00058240 con 0x7f1c1015deb0 2023-11-08T19:27:55.224 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:27:55.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1bfc06ae70 msgr2=0x7f1bfc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1bfc06ae70 0x7f1bfc06d330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1c04036000 tx=0x7f1c0403d040 comp rx=0 tx=0).stop 2023-11-08T19:27:55.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 msgr2=0x7f1c1014ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c1014ed70 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1c0000d800 tx=0x7f1c0000dcc0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 shutdown_connections 2023-11-08T19:27:55.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1bfc06ae70 0x7f1bfc06d330 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c1015e290 0x7f1c100c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c1015deb0 0x7f1c1014ed70 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 --2- 172.21.15.119:0/461835823 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c1015a8c0 0x7f1c100c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.221+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 >> 172.21.15.119:0/461835823 conn(0x7f1c100ceac0 msgr2=0x7f1c10159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.222+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 shutdown_connections 2023-11-08T19:27:55.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.222+0000 7f1c15b5e640 1 -- 172.21.15.119:0/461835823 wait complete. 2023-11-08T19:27:55.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:27:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.449+0000 7f6fbac0d640 1 Processor -- start 2023-11-08T19:27:55.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.450+0000 7f6fbac0d640 1 -- start start 2023-11-08T19:27:55.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fbac0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160530 0x7f6fb40cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fbac0d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fb4160910 0x7f6fb40cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fbac0d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb40b27c0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fbac0d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb40b2aa0 con 0x7f6fb4160910 2023-11-08T19:27:55.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fbac0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40d1640 con 0x7f6fb4160530 2023-11-08T19:27:55.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fb940a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fb4160910 0x7f6fb40cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fb940a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6fb4160910 0x7f6fb40cdea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41974/0 (socket says 172.21.15.119:41974) 2023-11-08T19:27:55.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.451+0000 7f6fb940a640 1 -- 172.21.15.119:0/3031648368 learned_addr learned my addr 172.21.15.119:0/3031648368 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:55.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.452+0000 7f6fb9c0b640 1 --2- 172.21.15.119:0/3031648368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160530 0x7f6fb40cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.452+0000 7f6fb940a640 1 -- 172.21.15.119:0/3031648368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160530 msgr2=0x7f6fb40cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.452+0000 7f6fb940a640 1 --2- 172.21.15.119:0/3031648368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160530 0x7f6fb40cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.452+0000 7f6fb940a640 1 -- 172.21.15.119:0/3031648368 >> v1:172.21.15.188:6789/0 conn(0x7f6fb40d3ac0 legacy=0x7f6fb40cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.452+0000 7f6fb940a640 1 -- 172.21.15.119:0/3031648368 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb40d1920 con 0x7f6fb4160910 2023-11-08T19:27:55.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.453+0000 7f6fb940a640 1 --2- 172.21.15.119:0/3031648368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb4160910 0x7f6fb40cdea0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009870 tx=0x7f6fa802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c41d5405b4cb09 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.453+0000 7f6fb8c09640 1 -- 172.21.15.119:0/3031648368 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8031640 con 0x7f6fb4160910 2023-11-08T19:27:55.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.453+0000 7f6fb8c09640 1 -- 172.21.15.119:0/3031648368 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa80317a0 con 0x7f6fb4160910 2023-11-08T19:27:55.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.453+0000 7f6fb8c09640 1 -- 172.21.15.119:0/3031648368 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8031b30 con 0x7f6fb4160910 2023-11-08T19:27:55.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 -- 172.21.15.119:0/3031648368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb4160910 msgr2=0x7f6fb40cdea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/3031648368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb4160910 0x7f6fb40cdea0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8009870 tx=0x7f6fa802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 -- 172.21.15.119:0/3031648368 shutdown_connections 2023-11-08T19:27:55.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/3031648368 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb4160910 0x7f6fb40cdea0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/3031648368 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160530 0x7f6fb40cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 -- 172.21.15.119:0/3031648368 >> 172.21.15.119:0/3031648368 conn(0x7f6fb40d2ec0 msgr2=0x7f6fb40d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 -- 172.21.15.119:0/3031648368 shutdown_connections 2023-11-08T19:27:55.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.454+0000 7f6fbac0d640 1 -- 172.21.15.119:0/3031648368 wait complete. 2023-11-08T19:27:55.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.456+0000 7f6fbac0d640 1 Processor -- start 2023-11-08T19:27:55.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.456+0000 7f6fbac0d640 1 -- start start 2023-11-08T19:27:55.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fbac0d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 0x7f6fb40b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fbac0d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb4160530 0x7f6fb40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fbac0d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160910 0x7f6fb41e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fbac0d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb41e60f0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fbac0d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40b7210 con 0x7f6fb4160530 2023-11-08T19:27:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fbac0d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb40cce00 con 0x7f6fb4160910 2023-11-08T19:27:55.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fb9c0b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 0x7f6fb40b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fb9c0b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 0x7f6fb40b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41984/0 (socket says 172.21.15.119:41984) 2023-11-08T19:27:55.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.457+0000 7f6fb9c0b640 1 -- 172.21.15.119:0/971397967 learned_addr learned my addr 172.21.15.119:0/971397967 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:55.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb940a640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb4160530 0x7f6fb40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb9c0b640 1 -- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160910 msgr2=0x7f6fb41e6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb9c0b640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160910 0x7f6fb41e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb9c0b640 1 -- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb4160530 msgr2=0x7f6fb40cc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb9c0b640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb4160530 0x7f6fb40cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb9c0b640 1 -- 172.21.15.119:0/971397967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa80094d0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb9c0b640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 0x7f6fb40b0d90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009fc0 tx=0x7f6fa400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4014620 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4014780 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fb40b1390 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4014b30 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.458+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb41e5aa0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.460+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb404ece0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.460+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6fa4014c90 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.462+0000 7f6fb27fc640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fa006ae70 0x7f6fa006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.462+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f6fa408a6a0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.462+0000 7f6fb940a640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fa006ae70 0x7f6fa006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.463+0000 7f6fb940a640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fa006ae70 0x7f6fa006d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8036000 tx=0x7f6fa803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.466+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6fa40550f0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.654+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f6fb40cead0 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.654+0000 7f6fb27fc640 1 -- 172.21.15.119:0/971397967 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v72) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6fa4058710 con 0x7f6fb40d3ac0 2023-11-08T19:27:55.657 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:55.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fa006ae70 msgr2=0x7f6fa006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fa006ae70 0x7f6fa006d330 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8036000 tx=0x7f6fa803c040 comp rx=0 tx=0).stop 2023-11-08T19:27:55.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 msgr2=0x7f6fb40b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 0x7f6fb40b0d90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4009fc0 tx=0x7f6fa400e570 comp rx=0 tx=0).stop 2023-11-08T19:27:55.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 shutdown_connections 2023-11-08T19:27:55.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fa006ae70 0x7f6fa006d330 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6fb4160910 0x7f6fb41e6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6fb4160530 0x7f6fb40cc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 --2- 172.21.15.119:0/971397967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6fb40d3ac0 0x7f6fb40b0d90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 >> 172.21.15.119:0/971397967 conn(0x7f6fb40d2ec0 msgr2=0x7f6fb4072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 shutdown_connections 2023-11-08T19:27:55.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.657+0000 7f6fbac0d640 1 -- 172.21.15.119:0/971397967 wait complete. 2023-11-08T19:27:55.678 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:27:55.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.895+0000 7fdc8759e640 1 Processor -- start 2023-11-08T19:27:55.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.895+0000 7fdc8759e640 1 -- start start 2023-11-08T19:27:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc8759e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc8815b690 0x7fdc88071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc8759e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc880cf5e0 0x7fdc880cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc8759e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc880c9440 con 0x7fdc8815b2b0 2023-11-08T19:27:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc8759e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc880c9690 con 0x7fdc880cf5e0 2023-11-08T19:27:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc8759e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc880c9970 con 0x7fdc8815b690 2023-11-08T19:27:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc86d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc880cf5e0 0x7fdc880cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc86d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc880cf5e0 0x7fdc880cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41988/0 (socket says 172.21.15.119:41988) 2023-11-08T19:27:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.896+0000 7fdc86d9d640 1 -- 172.21.15.119:0/1783127017 learned_addr learned my addr 172.21.15.119:0/1783127017 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.897+0000 7fdc7dd9b640 1 --2- 172.21.15.119:0/1783127017 >> v2:172.21.15.188:3300/0 conn(0x7fdc8815b690 0x7fdc88071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.897+0000 7fdc86d9d640 1 -- 172.21.15.119:0/1783127017 >> v2:172.21.15.188:3300/0 conn(0x7fdc8815b690 msgr2=0x7fdc88071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.897+0000 7fdc86d9d640 1 --2- 172.21.15.119:0/1783127017 >> v2:172.21.15.188:3300/0 conn(0x7fdc8815b690 0x7fdc88071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.897+0000 7fdc86d9d640 1 -- 172.21.15.119:0/1783127017 >> v1:172.21.15.188:6789/0 conn(0x7fdc8815b2b0 legacy=0x7fdc880ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:55.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.897+0000 7fdc86d9d640 1 -- 172.21.15.119:0/1783127017 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc880c9c50 con 0x7fdc880cf5e0 2023-11-08T19:27:55.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.898+0000 7fdc86d9d640 1 --2- 172.21.15.119:0/1783127017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc880cf5e0 0x7fdc880cab60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdc880680a0 tx=0x7fdc8002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bee1fcfbd1cae5d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.898+0000 7fdc85d9b640 1 -- 172.21.15.119:0/1783127017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8002c9f0 con 0x7fdc880cf5e0 2023-11-08T19:27:55.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.898+0000 7fdc85d9b640 1 -- 172.21.15.119:0/1783127017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8002cb50 con 0x7fdc880cf5e0 2023-11-08T19:27:55.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.898+0000 7fdc85d9b640 1 -- 172.21.15.119:0/1783127017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc800343f0 con 0x7fdc880cf5e0 2023-11-08T19:27:55.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 -- 172.21.15.119:0/1783127017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc880cf5e0 msgr2=0x7fdc880cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1783127017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc880cf5e0 0x7fdc880cab60 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fdc880680a0 tx=0x7fdc8002be50 comp rx=0 tx=0).stop 2023-11-08T19:27:55.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 -- 172.21.15.119:0/1783127017 shutdown_connections 2023-11-08T19:27:55.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1783127017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc880cf5e0 0x7fdc880cab60 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1783127017 >> v2:172.21.15.188:3300/0 conn(0x7fdc8815b690 0x7fdc88071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 -- 172.21.15.119:0/1783127017 >> 172.21.15.119:0/1783127017 conn(0x7fdc880ceac0 msgr2=0x7fdc880ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:55.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 -- 172.21.15.119:0/1783127017 shutdown_connections 2023-11-08T19:27:55.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.899+0000 7fdc8759e640 1 -- 172.21.15.119:0/1783127017 wait complete. 2023-11-08T19:27:55.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.901+0000 7fdc8759e640 1 Processor -- start 2023-11-08T19:27:55.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.901+0000 7fdc8759e640 1 -- start start 2023-11-08T19:27:55.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc8759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc880cf5e0 0x7fdc881531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc8759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 0x7fdc880b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc8759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8815b690 0x7fdc880b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc8759e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88157210 con 0x7fdc8815b2b0 2023-11-08T19:27:55.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc8759e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc88153700 con 0x7fdc880cf5e0 2023-11-08T19:27:55.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc8759e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc880c82b0 con 0x7fdc8815b690 2023-11-08T19:27:55.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc7dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 0x7fdc880b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc7dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 0x7fdc880b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:41990/0 (socket says 172.21.15.119:41990) 2023-11-08T19:27:55.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.902+0000 7fdc7dd9b640 1 -- 172.21.15.119:0/1945010818 learned_addr learned my addr 172.21.15.119:0/1945010818 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:55.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc86d9d640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8815b690 0x7fdc880b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc8659c640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc880cf5e0 0x7fdc881531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7dd9b640 1 -- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8815b690 msgr2=0x7fdc880b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7dd9b640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8815b690 0x7fdc880b3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7dd9b640 1 -- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc880cf5e0 msgr2=0x7fdc881531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7dd9b640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc880cf5e0 0x7fdc881531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7dd9b640 1 -- 172.21.15.119:0/1945010818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc80009b30 con 0x7fdc8815b2b0 2023-11-08T19:27:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7dd9b640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 0x7fdc880b2dc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d5c0 tx=0x7fdc7000da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc700125f0 con 0x7fdc8815b2b0 2023-11-08T19:27:55.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc70012750 con 0x7fdc8815b2b0 2023-11-08T19:27:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc70012b00 con 0x7fdc8815b2b0 2023-11-08T19:27:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.903+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc880c8590 con 0x7fdc8815b2b0 2023-11-08T19:27:55.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.904+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc880c8b50 con 0x7fdc8815b2b0 2023-11-08T19:27:55.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.905+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc880cd9d0 con 0x7fdc8815b2b0 2023-11-08T19:27:55.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.906+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc70012c60 con 0x7fdc8815b2b0 2023-11-08T19:27:55.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.906+0000 7fdc7f7fe640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6806ae70 0x7fdc6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:55.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.907+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fdc7008a090 con 0x7fdc8815b2b0 2023-11-08T19:27:55.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.907+0000 7fdc8659c640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6806ae70 0x7fdc6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:55.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.908+0000 7fdc8659c640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6806ae70 0x7fdc6806d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc8815ea60 tx=0x7fdc78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:55.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:55.911+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc70054b70 con 0x7fdc8815b2b0 2023-11-08T19:27:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.094+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc8804ece0 con 0x7fdc8815b2b0 2023-11-08T19:27:56.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.095+0000 7fdc7f7fe640 1 -- 172.21.15.119:0/1945010818 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v72) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc70058190 con 0x7fdc8815b2b0 2023-11-08T19:27:56.098 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6806ae70 msgr2=0x7fdc6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6806ae70 0x7fdc6806d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fdc8815ea60 tx=0x7fdc78009040 comp rx=0 tx=0).stop 2023-11-08T19:27:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 msgr2=0x7fdc880b2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 0x7fdc880b2dc0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d5c0 tx=0x7fdc7000da80 comp rx=0 tx=0).stop 2023-11-08T19:27:56.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 shutdown_connections 2023-11-08T19:27:56.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc6806ae70 0x7fdc6806d330 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc8815b690 0x7fdc880b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc8815b2b0 0x7fdc880b2dc0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 --2- 172.21.15.119:0/1945010818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc880cf5e0 0x7fdc881531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 >> 172.21.15.119:0/1945010818 conn(0x7fdc880ceac0 msgr2=0x7fdc880cae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 shutdown_connections 2023-11-08T19:27:56.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.098+0000 7fdc8759e640 1 -- 172.21.15.119:0/1945010818 wait complete. 2023-11-08T19:27:56.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:27:56.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.322+0000 7fc0eef33640 1 Processor -- start 2023-11-08T19:27:56.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.322+0000 7fc0eef33640 1 -- start start 2023-11-08T19:27:56.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.324+0000 7fc0eef33640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0e815d000 0x7fc0e80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.324+0000 7fc0eef33640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.324+0000 7fc0eef33640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e806d220 con 0x7fc0e80ae250 2023-11-08T19:27:56.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.324+0000 7fc0eef33640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e80ccd50 con 0x7fc0e815d000 2023-11-08T19:27:56.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.324+0000 7fc0eef33640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e80aca40 con 0x7fc0e80c3a50 2023-11-08T19:27:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ecccf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0e815d000 0x7fc0e80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ed4d0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc0e80ae250 0x7fc0e8160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46630/0 (socket says 172.21.15.119:46630) 2023-11-08T19:27:56.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ecccf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc0e815d000 0x7fc0e80cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42004/0 (socket says 172.21.15.119:42004) 2023-11-08T19:27:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ed4d0640 1 -- 172.21.15.119:0/207726172 learned_addr learned my addr 172.21.15.119:0/207726172 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0dffff640 1 --2- 172.21.15.119:0/207726172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ecccf640 1 -- 172.21.15.119:0/207726172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 msgr2=0x7fc0e806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ecccf640 1 --2- 172.21.15.119:0/207726172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ecccf640 1 -- 172.21.15.119:0/207726172 >> v1:172.21.15.119:6789/0 conn(0x7fc0e80ae250 legacy=0x7fc0e8160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0ecccf640 1 -- 172.21.15.119:0/207726172 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e80acd20 con 0x7fc0e815d000 2023-11-08T19:27:56.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.325+0000 7fc0df7fe640 1 -- 172.21.15.119:0/207726172 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2004040921 0 0) 0x7fc0e806d220 con 0x7fc0e80ae250 2023-11-08T19:27:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.326+0000 7fc0ecccf640 1 --2- 172.21.15.119:0/207726172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e815d000 0x7fc0e80cc810 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d106e0f971201b85 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0df7fe640 1 -- 172.21.15.119:0/207726172 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e402cbd0 con 0x7fc0e815d000 2023-11-08T19:27:56.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0df7fe640 1 -- 172.21.15.119:0/207726172 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e402cd30 con 0x7fc0e815d000 2023-11-08T19:27:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0df7fe640 1 -- 172.21.15.119:0/207726172 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e402cea0 con 0x7fc0e815d000 2023-11-08T19:27:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 -- 172.21.15.119:0/207726172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e815d000 msgr2=0x7fc0e80cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 --2- 172.21.15.119:0/207726172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e815d000 0x7fc0e80cc810 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e402be20 comp rx=0 tx=0).stop 2023-11-08T19:27:56.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 -- 172.21.15.119:0/207726172 shutdown_connections 2023-11-08T19:27:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 --2- 172.21.15.119:0/207726172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 --2- 172.21.15.119:0/207726172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e815d000 0x7fc0e80cc810 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 -- 172.21.15.119:0/207726172 >> 172.21.15.119:0/207726172 conn(0x7fc0e80cb610 msgr2=0x7fc0e80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 -- 172.21.15.119:0/207726172 shutdown_connections 2023-11-08T19:27:56.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.327+0000 7fc0eef33640 1 -- 172.21.15.119:0/207726172 wait complete. 2023-11-08T19:27:56.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.329+0000 7fc0eef33640 1 Processor -- start 2023-11-08T19:27:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.329+0000 7fc0eef33640 1 -- start start 2023-11-08T19:27:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0eef33640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 0x7fc0e8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0eef33640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e815c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0eef33640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e815d000 0x7fc0e814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0eef33640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e815c860 con 0x7fc0e80ae250 2023-11-08T19:27:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0eef33640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e815deb0 con 0x7fc0e815d000 2023-11-08T19:27:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0eef33640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e815e190 con 0x7fc0e80c3a50 2023-11-08T19:27:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0ecccf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 0x7fc0e8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0ecccf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 0x7fc0e8151bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42012/0 (socket says 172.21.15.119:42012) 2023-11-08T19:27:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.330+0000 7fc0ecccf640 1 -- 172.21.15.119:0/691388782 learned_addr learned my addr 172.21.15.119:0/691388782 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0dffff640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e815c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ecccf640 1 -- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 msgr2=0x7fc0e815c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ecccf640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e815c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ecccf640 1 -- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e815d000 msgr2=0x7fc0e814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ed4d0640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e815d000 0x7fc0e814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ecccf640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e815d000 0x7fc0e814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ecccf640 1 -- 172.21.15.119:0/691388782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e4009a70 con 0x7fc0e80ae250 2023-11-08T19:27:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ed4d0640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e815d000 0x7fc0e814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:56.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.331+0000 7fc0ecccf640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 0x7fc0e8151bb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.332+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e402cea0 con 0x7fc0e80ae250 2023-11-08T19:27:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.332+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e4040650 con 0x7fc0e80ae250 2023-11-08T19:27:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.332+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e81510d0 con 0x7fc0e80ae250 2023-11-08T19:27:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.332+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4040a00 con 0x7fc0e80ae250 2023-11-08T19:27:56.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.332+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e8151690 con 0x7fc0e80ae250 2023-11-08T19:27:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.333+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e8151880 con 0x7fc0e80ae250 2023-11-08T19:27:56.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.334+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc0e4040b60 con 0x7fc0e80ae250 2023-11-08T19:27:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.335+0000 7fc0dd7fa640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0d406ae70 0x7fc0d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.335+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fc0e40ac880 con 0x7fc0e80ae250 2023-11-08T19:27:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.335+0000 7fc0dffff640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0d406ae70 0x7fc0d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.336+0000 7fc0dffff640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0d406ae70 0x7fc0d406d330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8153430 tx=0x7fc0d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.340+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc0e4077400 con 0x7fc0e80ae250 2023-11-08T19:27:56.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.516+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc0e8074430 con 0x7fc0e80ae250 2023-11-08T19:27:56.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.517+0000 7fc0dd7fa640 1 -- 172.21.15.119:0/691388782 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v72) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc0e407aa20 con 0x7fc0e80ae250 2023-11-08T19:27:56.520 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:27:56.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0d406ae70 msgr2=0x7fc0d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0d406ae70 0x7fc0d406d330 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8153430 tx=0x7fc0d8008040 comp rx=0 tx=0).stop 2023-11-08T19:27:56.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 msgr2=0x7fc0e8151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 0x7fc0e8151bb0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009d90 tx=0x7fc0e4004a40 comp rx=0 tx=0).stop 2023-11-08T19:27:56.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 shutdown_connections 2023-11-08T19:27:56.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0d406ae70 0x7fc0d406d330 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc0e815d000 0x7fc0e814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc0e80c3a50 0x7fc0e815c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 --2- 172.21.15.119:0/691388782 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc0e80ae250 0x7fc0e8151bb0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 >> 172.21.15.119:0/691388782 conn(0x7fc0e80cb610 msgr2=0x7fc0e80b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 shutdown_connections 2023-11-08T19:27:56.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.520+0000 7fc0eef33640 1 -- 172.21.15.119:0/691388782 wait complete. 2023-11-08T19:27:56.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:27:56.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.732+0000 7faca590c640 1 Processor -- start 2023-11-08T19:27:56.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.733+0000 7faca590c640 1 -- start start 2023-11-08T19:27:56.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca590c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faca015b2b0 0x7faca0071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca590c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca015b690 0x7faca00ca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca590c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca00c9440 con 0x7faca00cf5e0 2023-11-08T19:27:56.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca590c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca00c9690 con 0x7faca015b2b0 2023-11-08T19:27:56.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca590c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca00c9970 con 0x7faca015b690 2023-11-08T19:27:56.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca490a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faca015b2b0 0x7faca0071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca510b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faca00cf5e0 0x7faca00ca990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46636/0 (socket says 172.21.15.119:46636) 2023-11-08T19:27:56.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.734+0000 7faca510b640 1 -- 172.21.15.119:0/31020868 learned_addr learned my addr 172.21.15.119:0/31020868 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:56.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.735+0000 7fac9ffff640 1 --2- 172.21.15.119:0/31020868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca015b690 0x7faca00ca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.735+0000 7faca490a640 1 -- 172.21.15.119:0/31020868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca015b690 msgr2=0x7faca00ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:56.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.735+0000 7faca490a640 1 --2- 172.21.15.119:0/31020868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca015b690 0x7faca00ca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.735+0000 7faca490a640 1 -- 172.21.15.119:0/31020868 >> v1:172.21.15.119:6789/0 conn(0x7faca00cf5e0 legacy=0x7faca00ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.735+0000 7faca490a640 1 -- 172.21.15.119:0/31020868 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca00c9c50 con 0x7faca015b2b0 2023-11-08T19:27:56.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.735+0000 7fac9f7fe640 1 -- 172.21.15.119:0/31020868 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743133877 0 0) 0x7faca00c9440 con 0x7faca00cf5e0 2023-11-08T19:27:56.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.736+0000 7faca490a640 1 --2- 172.21.15.119:0/31020868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca0071660 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fac940097a0 tx=0x7fac9402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0beb0800b733713 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.736+0000 7fac9f7fe640 1 -- 172.21.15.119:0/31020868 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94031780 con 0x7faca015b2b0 2023-11-08T19:27:56.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.736+0000 7fac9f7fe640 1 -- 172.21.15.119:0/31020868 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac940318e0 con 0x7faca015b2b0 2023-11-08T19:27:56.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.736+0000 7fac9f7fe640 1 -- 172.21.15.119:0/31020868 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac94031c70 con 0x7faca015b2b0 2023-11-08T19:27:56.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 -- 172.21.15.119:0/31020868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 msgr2=0x7faca0071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 --2- 172.21.15.119:0/31020868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca0071660 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fac940097a0 tx=0x7fac9402b820 comp rx=0 tx=0).stop 2023-11-08T19:27:56.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 -- 172.21.15.119:0/31020868 shutdown_connections 2023-11-08T19:27:56.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 --2- 172.21.15.119:0/31020868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca015b690 0x7faca00ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 --2- 172.21.15.119:0/31020868 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca0071660 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 -- 172.21.15.119:0/31020868 >> 172.21.15.119:0/31020868 conn(0x7faca00ceac0 msgr2=0x7faca00ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 -- 172.21.15.119:0/31020868 shutdown_connections 2023-11-08T19:27:56.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.737+0000 7faca590c640 1 -- 172.21.15.119:0/31020868 wait complete. 2023-11-08T19:27:56.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.739+0000 7faca590c640 1 Processor -- start 2023-11-08T19:27:56.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.739+0000 7faca590c640 1 -- start start 2023-11-08T19:27:56.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7faca590c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca00cf5e0 0x7faca0148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7faca590c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7faca590c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca015b690 0x7faca01f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7faca590c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0148dc0 con 0x7faca015b2b0 2023-11-08T19:27:56.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7faca590c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca015bfe0 con 0x7faca015b690 2023-11-08T19:27:56.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7faca590c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca01491c0 con 0x7faca00cf5e0 2023-11-08T19:27:56.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7fac9ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42026/0 (socket says 172.21.15.119:42026) 2023-11-08T19:27:56.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.740+0000 7fac9ffff640 1 -- 172.21.15.119:0/390590741 learned_addr learned my addr 172.21.15.119:0/390590741 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:56.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7faca490a640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca00cf5e0 0x7faca0148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9ffff640 1 -- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca00cf5e0 msgr2=0x7faca0148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9ffff640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca00cf5e0 0x7faca0148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9ffff640 1 -- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca015b690 msgr2=0x7faca01f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9ffff640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca015b690 0x7faca01f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9ffff640 1 -- 172.21.15.119:0/390590741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac94009400 con 0x7faca015b2b0 2023-11-08T19:27:56.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9ffff640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca015baa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c7f0 tx=0x7fac8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c004220 con 0x7faca015b2b0 2023-11-08T19:27:56.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c004380 con 0x7faca015b2b0 2023-11-08T19:27:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c007840 con 0x7faca015b2b0 2023-11-08T19:27:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.741+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faca01494a0 con 0x7faca015b2b0 2023-11-08T19:27:56.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.742+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca0149a60 con 0x7faca015b2b0 2023-11-08T19:27:56.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.743+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca00b18d0 con 0x7faca015b2b0 2023-11-08T19:27:56.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.744+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac8c007bb0 con 0x7faca015b2b0 2023-11-08T19:27:56.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.745+0000 7fac9d7fa640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fac8406ae70 0x7fac8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:56.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.745+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7fac8c089c00 con 0x7faca015b2b0 2023-11-08T19:27:56.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.745+0000 7faca490a640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fac8406ae70 0x7fac8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:56.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.746+0000 7faca490a640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fac8406ae70 0x7fac8406d330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac94036000 tx=0x7fac9403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:56.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.749+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac8c054650 con 0x7faca015b2b0 2023-11-08T19:27:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.942+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7faca004ece0 con 0x7faca015b2b0 2023-11-08T19:27:56.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.943+0000 7fac9d7fa640 1 -- 172.21.15.119:0/390590741 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v72) v1 ==== 103+0+11 (secure 0 0 0) 0x7fac8c01e0a0 con 0x7faca015b2b0 2023-11-08T19:27:56.946 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:56.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.945+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fac8406ae70 msgr2=0x7fac8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.945+0000 7faca590c640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fac8406ae70 0x7fac8406d330 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fac94036000 tx=0x7fac9403d040 comp rx=0 tx=0).stop 2023-11-08T19:27:56.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.945+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 msgr2=0x7faca015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:56.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.945+0000 7faca590c640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca015baa0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fac8c00c7f0 tx=0x7fac8c00ccb0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 shutdown_connections 2023-11-08T19:27:56.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fac8406ae70 0x7fac8406d330 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faca015b690 0x7faca01f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faca015b2b0 0x7faca015baa0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 --2- 172.21.15.119:0/390590741 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faca00cf5e0 0x7faca0148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:56.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 >> 172.21.15.119:0/390590741 conn(0x7faca00ceac0 msgr2=0x7faca00cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:56.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 shutdown_connections 2023-11-08T19:27:56.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:56.946+0000 7faca590c640 1 -- 172.21.15.119:0/390590741 wait complete. 2023-11-08T19:27:56.967 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:27:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.174+0000 7f1a928d7640 1 Processor -- start 2023-11-08T19:27:57.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.174+0000 7f1a928d7640 1 -- start start 2023-11-08T19:27:57.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a928d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c06a830 0x7f1a8c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a928d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1a8c0c3a50 0x7f1a8c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a928d7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8c06d220 con 0x7f1a8c15d000 2023-11-08T19:27:57.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a928d7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8c0ccd50 con 0x7f1a8c0c3a50 2023-11-08T19:27:57.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a928d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0aca40 con 0x7f1a8c06a830 2023-11-08T19:27:57.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a8bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1a8c15d000 0x7f1a8c06a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:46646/0 (socket says 172.21.15.119:46646) 2023-11-08T19:27:57.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a8bfff640 1 -- 172.21.15.119:0/2489399271 learned_addr learned my addr 172.21.15.119:0/2489399271 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:57.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.176+0000 7f1a90e74640 1 --2- 172.21.15.119:0/2489399271 >> v2:172.21.15.119:3300/0 conn(0x7f1a8c0c3a50 0x7f1a8c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.177+0000 7f1a83fff640 1 --2- 172.21.15.119:0/2489399271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c06a830 0x7f1a8c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.177+0000 7f1a90e74640 1 -- 172.21.15.119:0/2489399271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c06a830 msgr2=0x7f1a8c0cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:57.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.177+0000 7f1a90e74640 1 --2- 172.21.15.119:0/2489399271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c06a830 0x7f1a8c0cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.177+0000 7f1a90e74640 1 -- 172.21.15.119:0/2489399271 >> v1:172.21.15.119:6789/0 conn(0x7f1a8c15d000 legacy=0x7f1a8c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.177+0000 7f1a90e74640 1 -- 172.21.15.119:0/2489399271 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c0acd20 con 0x7f1a8c0c3a50 2023-11-08T19:27:57.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.177+0000 7f1a8b7fe640 1 -- 172.21.15.119:0/2489399271 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1713926566 0 0) 0x7f1a8c06d220 con 0x7f1a8c15d000 2023-11-08T19:27:57.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.178+0000 7f1a90e74640 1 --2- 172.21.15.119:0/2489399271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c0c3a50 0x7f1a8c160e90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009d90 tx=0x7f1a8402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e7afaccbe11ae4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.178+0000 7f1a8b7fe640 1 -- 172.21.15.119:0/2489399271 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402cbc0 con 0x7f1a8c0c3a50 2023-11-08T19:27:57.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.178+0000 7f1a8b7fe640 1 -- 172.21.15.119:0/2489399271 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8402cd20 con 0x7f1a8c0c3a50 2023-11-08T19:27:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.178+0000 7f1a8b7fe640 1 -- 172.21.15.119:0/2489399271 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84034660 con 0x7f1a8c0c3a50 2023-11-08T19:27:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 -- 172.21.15.119:0/2489399271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c0c3a50 msgr2=0x7f1a8c160e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 --2- 172.21.15.119:0/2489399271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c0c3a50 0x7f1a8c160e90 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009d90 tx=0x7f1a8402be10 comp rx=0 tx=0).stop 2023-11-08T19:27:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 -- 172.21.15.119:0/2489399271 shutdown_connections 2023-11-08T19:27:57.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 --2- 172.21.15.119:0/2489399271 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c0c3a50 0x7f1a8c160e90 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 --2- 172.21.15.119:0/2489399271 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c06a830 0x7f1a8c0cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 -- 172.21.15.119:0/2489399271 >> 172.21.15.119:0/2489399271 conn(0x7f1a8c0cb610 msgr2=0x7f1a8c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 -- 172.21.15.119:0/2489399271 shutdown_connections 2023-11-08T19:27:57.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.179+0000 7f1a928d7640 1 -- 172.21.15.119:0/2489399271 wait complete. 2023-11-08T19:27:57.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.181+0000 7f1a928d7640 1 Processor -- start 2023-11-08T19:27:57.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.181+0000 7f1a928d7640 1 -- start start 2023-11-08T19:27:57.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a928d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 0x7f1a8c154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a928d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0c3a50 0x7f1a8c15d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a928d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c15d000 0x7f1a8c1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a928d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0b18b0 con 0x7f1a8c06a830 2023-11-08T19:27:57.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a928d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c154e70 con 0x7f1a8c0c3a50 2023-11-08T19:27:57.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a928d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c15dcc0 con 0x7f1a8c15d000 2023-11-08T19:27:57.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 0x7f1a8c154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 0x7f1a8c154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42042/0 (socket says 172.21.15.119:42042) 2023-11-08T19:27:57.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 -- 172.21.15.119:0/3655884552 learned_addr learned my addr 172.21.15.119:0/3655884552 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:57.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 -- 172.21.15.119:0/3655884552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c15d000 msgr2=0x7f1a8c1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c15d000 0x7f1a8c1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 -- 172.21.15.119:0/3655884552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0c3a50 msgr2=0x7f1a8c15d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0c3a50 0x7f1a8c15d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.182+0000 7f1a8bfff640 1 -- 172.21.15.119:0/3655884552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84009a70 con 0x7f1a8c06a830 2023-11-08T19:27:57.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.183+0000 7f1a8bfff640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 0x7f1a8c154930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c4d0 tx=0x7f1a7c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.183+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c0074e0 con 0x7f1a8c06a830 2023-11-08T19:27:57.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.183+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a7c007640 con 0x7f1a8c06a830 2023-11-08T19:27:57.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.183+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a7c004a20 con 0x7f1a8c06a830 2023-11-08T19:27:57.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.183+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c15dfa0 con 0x7f1a8c06a830 2023-11-08T19:27:57.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.183+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c15e560 con 0x7f1a8c06a830 2023-11-08T19:27:57.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.184+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c0b19f0 con 0x7f1a8c06a830 2023-11-08T19:27:57.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.185+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a7c004b80 con 0x7f1a8c06a830 2023-11-08T19:27:57.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.186+0000 7f1a897fa640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a6c06aea0 0x7f1a6c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.186+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f1a7c088d30 con 0x7f1a8c06a830 2023-11-08T19:27:57.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.186+0000 7f1a83fff640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a6c06aea0 0x7f1a6c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.187+0000 7f1a83fff640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a6c06aea0 0x7f1a6c06d360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a74006fd0 tx=0x7f1a74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.190+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a8c0b19f0 con 0x7f1a8c06a830 2023-11-08T19:27:57.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.378+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f1a8c0b19f0 con 0x7f1a8c06a830 2023-11-08T19:27:57.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.378+0000 7f1a897fa640 1 -- 172.21.15.119:0/3655884552 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v72) v1 ==== 107+0+11 (secure 0 0 0) 0x7f1a8c0b19f0 con 0x7f1a8c06a830 2023-11-08T19:27:57.381 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:27:57.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a6c06aea0 msgr2=0x7f1a6c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a6c06aea0 0x7f1a6c06d360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1a74006fd0 tx=0x7f1a74008040 comp rx=0 tx=0).stop 2023-11-08T19:27:57.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 msgr2=0x7f1a8c154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 0x7f1a8c154930 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f1a7c00c4d0 tx=0x7f1a7c00c990 comp rx=0 tx=0).stop 2023-11-08T19:27:57.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 shutdown_connections 2023-11-08T19:27:57.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1a6c06aea0 0x7f1a6c06d360 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a8c15d000 0x7f1a8c1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a8c0c3a50 0x7f1a8c15d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 --2- 172.21.15.119:0/3655884552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a8c06a830 0x7f1a8c154930 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.381+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 >> 172.21.15.119:0/3655884552 conn(0x7f1a8c0cb610 msgr2=0x7f1a8c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.382+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 shutdown_connections 2023-11-08T19:27:57.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.382+0000 7f1a928d7640 1 -- 172.21.15.119:0/3655884552 wait complete. 2023-11-08T19:27:57.401 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:57.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.602+0000 7f5578e75640 1 Processor -- start 2023-11-08T19:27:57.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.602+0000 7f5578e75640 1 -- start start 2023-11-08T19:27:57.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.603+0000 7f5578e75640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55741447a0 0x7f557415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f5578e75640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f557406a120 0x7f557406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f5578e75640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557406aa70 con 0x7f55740cad30 2023-11-08T19:27:57.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f5578e75640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5574070040 con 0x7f55741447a0 2023-11-08T19:27:57.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f5578e75640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55740c3a50 con 0x7f557406a120 2023-11-08T19:27:57.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f557259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55741447a0 0x7f557415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f557259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f55741447a0 0x7f557415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42046/0 (socket says 172.21.15.119:42046) 2023-11-08T19:27:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f557259c640 1 -- 172.21.15.119:0/3996481707 learned_addr learned my addr 172.21.15.119:0/3996481707 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:57.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.604+0000 7f5572d9d640 1 --2- 172.21.15.119:0/3996481707 >> v2:172.21.15.188:3300/0 conn(0x7f557406a120 0x7f557406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.605+0000 7f557259c640 1 -- 172.21.15.119:0/3996481707 >> v2:172.21.15.188:3300/0 conn(0x7f557406a120 msgr2=0x7f557406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:57.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.605+0000 7f557259c640 1 --2- 172.21.15.119:0/3996481707 >> v2:172.21.15.188:3300/0 conn(0x7f557406a120 0x7f557406a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.605+0000 7f557259c640 1 -- 172.21.15.119:0/3996481707 >> v1:172.21.15.188:6789/0 conn(0x7f55740cad30 legacy=0x7f557406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.605+0000 7f557259c640 1 -- 172.21.15.119:0/3996481707 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574144b80 con 0x7f55741447a0 2023-11-08T19:27:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.606+0000 7f557259c640 1 --2- 172.21.15.119:0/3996481707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55741447a0 0x7f557415f4c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f55740cfbd0 tx=0x7f556402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519d99a33d401fe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5571d9b640 1 -- 172.21.15.119:0/3996481707 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564031640 con 0x7f55741447a0 2023-11-08T19:27:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5571d9b640 1 -- 172.21.15.119:0/3996481707 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55640317a0 con 0x7f55741447a0 2023-11-08T19:27:57.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5571d9b640 1 -- 172.21.15.119:0/3996481707 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564031b30 con 0x7f55741447a0 2023-11-08T19:27:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 -- 172.21.15.119:0/3996481707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55741447a0 msgr2=0x7f557415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 --2- 172.21.15.119:0/3996481707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55741447a0 0x7f557415f4c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f55740cfbd0 tx=0x7f556402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 -- 172.21.15.119:0/3996481707 shutdown_connections 2023-11-08T19:27:57.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 --2- 172.21.15.119:0/3996481707 >> v2:172.21.15.188:3300/0 conn(0x7f557406a120 0x7f557406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 --2- 172.21.15.119:0/3996481707 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f55741447a0 0x7f557415f4c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 -- 172.21.15.119:0/3996481707 >> 172.21.15.119:0/3996481707 conn(0x7f55740cb610 msgr2=0x7f55740cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 -- 172.21.15.119:0/3996481707 shutdown_connections 2023-11-08T19:27:57.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.607+0000 7f5578e75640 1 -- 172.21.15.119:0/3996481707 wait complete. 2023-11-08T19:27:57.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.609+0000 7f5578e75640 1 Processor -- start 2023-11-08T19:27:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.609+0000 7f5578e75640 1 -- start start 2023-11-08T19:27:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f5578e75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 0x7f557415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f5578e75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55740cad30 0x7f55740ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f5578e75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55741447a0 0x7f55740ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f5578e75640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740aea90 con 0x7f557406a120 2023-11-08T19:27:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f5578e75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55740b43b0 con 0x7f55740cad30 2023-11-08T19:27:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f5578e75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55740c6d80 con 0x7f55741447a0 2023-11-08T19:27:57.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f557259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 0x7f557415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f557259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 0x7f557415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42054/0 (socket says 172.21.15.119:42054) 2023-11-08T19:27:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.610+0000 7f557259c640 1 -- 172.21.15.119:0/28891091 learned_addr learned my addr 172.21.15.119:0/28891091 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f5569d9b640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55740cad30 0x7f55740ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f5572d9d640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55741447a0 0x7f55740ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f557259c640 1 -- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55741447a0 msgr2=0x7f55740ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f557259c640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55741447a0 0x7f55740ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f557259c640 1 -- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55740cad30 msgr2=0x7f55740ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f557259c640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55740cad30 0x7f55740ad140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f557259c640 1 -- 172.21.15.119:0/28891091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640094d0 con 0x7f557406a120 2023-11-08T19:27:57.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f5569d9b640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55740cad30 0x7f55740ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:57.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f557259c640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 0x7f557415aa70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5564000c00 tx=0x7f55640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.611+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556403ac60 con 0x7f557406a120 2023-11-08T19:27:57.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.612+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556403adc0 con 0x7f557406a120 2023-11-08T19:27:57.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.612+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55740c7060 con 0x7f557406a120 2023-11-08T19:27:57.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.612+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564044610 con 0x7f557406a120 2023-11-08T19:27:57.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.612+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55740b3680 con 0x7f557406a120 2023-11-08T19:27:57.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.613+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55740c6be0 con 0x7f557406a120 2023-11-08T19:27:57.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.614+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f556402f050 con 0x7f557406a120 2023-11-08T19:27:57.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.614+0000 7f556b7fe640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f555406ae70 0x7f555406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:57.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.615+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f55640abb80 con 0x7f557406a120 2023-11-08T19:27:57.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.615+0000 7f5569d9b640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f555406ae70 0x7f555406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:57.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.615+0000 7f5569d9b640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f555406ae70 0x7f555406d330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f557406d130 tx=0x7f555c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:57.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.618+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55740c6be0 con 0x7f557406a120 2023-11-08T19:27:57.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.845+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55740c7580 con 0x7f557406a120 2023-11-08T19:27:57.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.849+0000 7f556b7fe640 1 -- 172.21.15.119:0/28891091 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f55740c7580 con 0x7f557406a120 2023-11-08T19:27:57.853 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:57.854 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:36.122432+0000","modified":"2023-11-08T19:27:39.195905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/913059196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":913059196},{"type":"v1","addr":"172.21.15.188:6835","nonce":913059196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f555406ae70 msgr2=0x7f555406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f555406ae70 0x7f555406d330 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f557406d130 tx=0x7f555c008040 comp rx=0 tx=0).stop 2023-11-08T19:27:57.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 msgr2=0x7f557415aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 0x7f557415aa70 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f5564000c00 tx=0x7f55640046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 shutdown_connections 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f555406ae70 0x7f555406d330 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f55741447a0 0x7f55740ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f55740cad30 0x7f55740ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 --2- 172.21.15.119:0/28891091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f557406a120 0x7f557415aa70 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:57.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 >> 172.21.15.119:0/28891091 conn(0x7f55740cb610 msgr2=0x7f557415b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 shutdown_connections 2023-11-08T19:27:57.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:57.852+0000 7f5578e75640 1 -- 172.21.15.119:0/28891091 wait complete. 2023-11-08T19:27:57.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:58.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.081+0000 7f7746b00640 1 Processor -- start 2023-11-08T19:27:58.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.081+0000 7f7746b00640 1 -- start start 2023-11-08T19:27:58.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f7746b00640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f774006a970 0x7f774006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f7746b00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f7746b00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f77400ccc10 0x7f77401653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f7746b00640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77401619b0 con 0x7f774006a970 2023-11-08T19:27:58.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f7746b00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77400cd0a0 con 0x7f77400ccc10 2023-11-08T19:27:58.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f7746b00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774006d4b0 con 0x7f77400c9150 2023-11-08T19:27:58.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f774006a970 0x7f774006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f774006a970 0x7f774006cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42066/0 (socket says 172.21.15.119:42066) 2023-11-08T19:27:58.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 -- 172.21.15.119:0/2744730882 learned_addr learned my addr 172.21.15.119:0/2744730882 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:58.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f773ffff640 1 --2- 172.21.15.119:0/2744730882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774509d640 1 --2- 172.21.15.119:0/2744730882 >> v2:172.21.15.188:3300/0 conn(0x7f77400ccc10 0x7f77401653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 -- 172.21.15.119:0/2744730882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 msgr2=0x7f77400cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 --2- 172.21.15.119:0/2744730882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 -- 172.21.15.119:0/2744730882 >> v2:172.21.15.188:3300/0 conn(0x7f77400ccc10 msgr2=0x7f77401653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 --2- 172.21.15.119:0/2744730882 >> v2:172.21.15.188:3300/0 conn(0x7f77400ccc10 0x7f77401653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.083+0000 7f774489c640 1 -- 172.21.15.119:0/2744730882 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7740147b80 con 0x7f774006a970 2023-11-08T19:27:58.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.085+0000 7f774489c640 1 --2- 172.21.15.119:0/2744730882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f774006a970 0x7f774006cf70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f774006b1c0 tx=0x7f773402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a28a37d5b69dddb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.085+0000 7f773f7fe640 1 -- 172.21.15.119:0/2744730882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77340317e0 con 0x7f774006a970 2023-11-08T19:27:58.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.085+0000 7f773f7fe640 1 -- 172.21.15.119:0/2744730882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7734031940 con 0x7f774006a970 2023-11-08T19:27:58.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.085+0000 7f773f7fe640 1 -- 172.21.15.119:0/2744730882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7734031cd0 con 0x7f774006a970 2023-11-08T19:27:58.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.085+0000 7f7746b00640 1 -- 172.21.15.119:0/2744730882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f774006a970 msgr2=0x7f774006cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.085+0000 7f7746b00640 1 --2- 172.21.15.119:0/2744730882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f774006a970 0x7f774006cf70 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f774006b1c0 tx=0x7f773402b880 comp rx=0 tx=0).stop 2023-11-08T19:27:58.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 -- 172.21.15.119:0/2744730882 shutdown_connections 2023-11-08T19:27:58.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 --2- 172.21.15.119:0/2744730882 >> v2:172.21.15.188:3300/0 conn(0x7f77400ccc10 0x7f77401653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 --2- 172.21.15.119:0/2744730882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 --2- 172.21.15.119:0/2744730882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f774006a970 0x7f774006cf70 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 -- 172.21.15.119:0/2744730882 >> 172.21.15.119:0/2744730882 conn(0x7f77400cb610 msgr2=0x7f77400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 -- 172.21.15.119:0/2744730882 shutdown_connections 2023-11-08T19:27:58.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.086+0000 7f7746b00640 1 -- 172.21.15.119:0/2744730882 wait complete. 2023-11-08T19:27:58.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.087+0000 7f7746b00640 1 Processor -- start 2023-11-08T19:27:58.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.087+0000 7f7746b00640 1 -- start start 2023-11-08T19:27:58.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.088+0000 7f7746b00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f774006a970 0x7f7740160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.088+0000 7f7746b00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.088+0000 7f7746b00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 0x7f77400c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.088+0000 7f7746b00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400b2fb0 con 0x7f77400ccc10 2023-11-08T19:27:58.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.088+0000 7f7746b00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7740161b00 con 0x7f774006a970 2023-11-08T19:27:58.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.088+0000 7f7746b00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774014a830 con 0x7f77400c9150 2023-11-08T19:27:58.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 0x7f77400c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 0x7f77400c6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42082/0 (socket says 172.21.15.119:42082) 2023-11-08T19:27:58.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 -- 172.21.15.119:0/3027622182 learned_addr learned my addr 172.21.15.119:0/3027622182 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:58.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774489c640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f774006a970 0x7f7740160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f773ffff640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 -- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 msgr2=0x7f77400c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 -- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f774006a970 msgr2=0x7f7740160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f774006a970 0x7f7740160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 -- 172.21.15.119:0/3027622182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77340094d0 con 0x7f77400ccc10 2023-11-08T19:27:58.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f773ffff640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:58.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774489c640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f774006a970 0x7f7740160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:58.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.089+0000 7f774509d640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 0x7f77400c6990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f773800cc30 tx=0x7f773800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.090+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773800ce40 con 0x7f77400ccc10 2023-11-08T19:27:58.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.090+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7738003990 con 0x7f77400ccc10 2023-11-08T19:27:58.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.090+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738003d20 con 0x7f77400ccc10 2023-11-08T19:27:58.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.090+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774014a550 con 0x7f77400ccc10 2023-11-08T19:27:58.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.090+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77400b2bd0 con 0x7f77400ccc10 2023-11-08T19:27:58.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.091+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77400b30f0 con 0x7f77400ccc10 2023-11-08T19:27:58.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.092+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7738003e80 con 0x7f77400ccc10 2023-11-08T19:27:58.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.093+0000 7f773d7fa640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f772406ae70 0x7f772406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.093+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f7738089fb0 con 0x7f77400ccc10 2023-11-08T19:27:58.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.093+0000 7f774489c640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f772406ae70 0x7f772406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.094+0000 7f774489c640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f772406ae70 0x7f772406d330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7734036000 tx=0x7f773403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.097+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7738054a00 con 0x7f77400ccc10 2023-11-08T19:27:58.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.277+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7740074430 con 0x7f77400ccc10 2023-11-08T19:27:58.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.281+0000 7f773d7fa640 1 -- 172.21.15.119:0/3027622182 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+22155 (secure 0 0 0) 0x7f773801d070 con 0x7f77400ccc10 2023-11-08T19:27:58.284 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:58.284 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":72,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:53.645252+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f772406ae70 msgr2=0x7f772406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f772406ae70 0x7f772406d330 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7734036000 tx=0x7f773403d040 comp rx=0 tx=0).stop 2023-11-08T19:27:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 msgr2=0x7f77400c6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 0x7f77400c6990 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f773800cc30 tx=0x7f773800a5c0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 shutdown_connections 2023-11-08T19:27:58.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f772406ae70 0x7f772406d330 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f77400ccc10 0x7f77400c6990 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f77400c9150 0x7f77400c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 --2- 172.21.15.119:0/3027622182 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f774006a970 0x7f7740160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.283+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 >> 172.21.15.119:0/3027622182 conn(0x7f77400cb610 msgr2=0x7f77400c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.284+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 shutdown_connections 2023-11-08T19:27:58.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.284+0000 7f7746b00640 1 -- 172.21.15.119:0/3027622182 wait complete. 2023-11-08T19:27:58.303 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:27:58.304 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:58.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.500+0000 7f5f8f59e640 1 Processor -- start 2023-11-08T19:27:58.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.500+0000 7f5f8f59e640 1 -- start start 2023-11-08T19:27:58.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.502+0000 7f5f8f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 0x7f5f900cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.502+0000 7f5f8f59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f900d1d50 0x7f5f900c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.502+0000 7f5f8f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f900c7100 con 0x7f5f9015d4c0 2023-11-08T19:27:58.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.502+0000 7f5f8f59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900d04f0 con 0x7f5f900d1d50 2023-11-08T19:27:58.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.502+0000 7f5f8f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900d07d0 con 0x7f5f9015d0e0 2023-11-08T19:27:58.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f900d1d50 0x7f5f900c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 0x7f5f900cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f87fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5f9015d4c0 0x7f5f900cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:40640/0 (socket says 172.21.15.119:40640) 2023-11-08T19:27:58.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5f900d1d50 0x7f5f900c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:53102/0 (socket says 172.21.15.119:53102) 2023-11-08T19:27:58.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 0x7f5f900cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:55924/0 (socket says 172.21.15.119:55924) 2023-11-08T19:27:58.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f87fff640 1 -- 172.21.15.119:0/3472095096 learned_addr learned my addr 172.21.15.119:0/3472095096 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:58.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8e59c640 1 -- 172.21.15.119:0/3472095096 >> v2:172.21.15.188:3300/0 conn(0x7f5f900d1d50 msgr2=0x7f5f900c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8e59c640 1 --2- 172.21.15.119:0/3472095096 >> v2:172.21.15.188:3300/0 conn(0x7f5f900d1d50 0x7f5f900c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.503+0000 7f5f8e59c640 1 -- 172.21.15.119:0/3472095096 >> v1:172.21.15.188:6789/0 conn(0x7f5f9015d4c0 legacy=0x7f5f900cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.504+0000 7f5f8e59c640 1 -- 172.21.15.119:0/3472095096 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900c6ab0 con 0x7f5f9015d0e0 2023-11-08T19:27:58.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.505+0000 7f5f8e59c640 1 --2- 172.21.15.119:0/3472095096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 0x7f5f900cda70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f800097a0 tx=0x7f5f8002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86ce0c6a681eaf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.506+0000 7f5f8dd9b640 1 -- 172.21.15.119:0/3472095096 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80030800 con 0x7f5f9015d0e0 2023-11-08T19:27:58.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.506+0000 7f5f8dd9b640 1 -- 172.21.15.119:0/3472095096 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f80030960 con 0x7f5f9015d0e0 2023-11-08T19:27:58.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.506+0000 7f5f8dd9b640 1 -- 172.21.15.119:0/3472095096 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80030cf0 con 0x7f5f9015d0e0 2023-11-08T19:27:58.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 -- 172.21.15.119:0/3472095096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 msgr2=0x7f5f900cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/3472095096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 0x7f5f900cda70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5f800097a0 tx=0x7f5f8002b940 comp rx=0 tx=0).stop 2023-11-08T19:27:58.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 -- 172.21.15.119:0/3472095096 shutdown_connections 2023-11-08T19:27:58.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/3472095096 >> v2:172.21.15.188:3300/0 conn(0x7f5f900d1d50 0x7f5f900c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/3472095096 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d0e0 0x7f5f900cda70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 -- 172.21.15.119:0/3472095096 >> 172.21.15.119:0/3472095096 conn(0x7f5f900d2f60 msgr2=0x7f5f900d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 -- 172.21.15.119:0/3472095096 shutdown_connections 2023-11-08T19:27:58.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.507+0000 7f5f8f59e640 1 -- 172.21.15.119:0/3472095096 wait complete. 2023-11-08T19:27:58.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.509+0000 7f5f8f59e640 1 Processor -- start 2023-11-08T19:27:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.509+0000 7f5f8f59e640 1 -- start start 2023-11-08T19:27:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 0x7f5f901590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9015d0e0 0x7f5f901601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d4c0 0x7f5f901f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f901606e0 con 0x7f5f900d1d50 2023-11-08T19:27:58.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9015d8d0 con 0x7f5f9015d0e0 2023-11-08T19:27:58.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f90159680 con 0x7f5f9015d4c0 2023-11-08T19:27:58.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 0x7f5f901590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 0x7f5f901590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:42094/0 (socket says 172.21.15.119:42094) 2023-11-08T19:27:58.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.510+0000 7f5f8e59c640 1 -- 172.21.15.119:0/892574895 learned_addr learned my addr 172.21.15.119:0/892574895 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:58.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8ed9d640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d4c0 0x7f5f901f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f87fff640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9015d0e0 0x7f5f901601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8e59c640 1 -- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d4c0 msgr2=0x7f5f901f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8e59c640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d4c0 0x7f5f901f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8e59c640 1 -- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9015d0e0 msgr2=0x7f5f901601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8e59c640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9015d0e0 0x7f5f901601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8e59c640 1 -- 172.21.15.119:0/892574895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80009400 con 0x7f5f900d1d50 2023-11-08T19:27:58.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f87fff640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9015d0e0 0x7f5f901601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:58.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8e59c640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 0x7f5f901590a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5f800098c0 tx=0x7f5f80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8003e040 con 0x7f5f900d1d50 2023-11-08T19:27:58.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f80039cd0 con 0x7f5f900d1d50 2023-11-08T19:27:58.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80044630 con 0x7f5f900d1d50 2023-11-08T19:27:58.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.511+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f90159960 con 0x7f5f900d1d50 2023-11-08T19:27:58.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.512+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9015de50 con 0x7f5f900d1d50 2023-11-08T19:27:58.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.513+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9015e090 con 0x7f5f900d1d50 2023-11-08T19:27:58.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.514+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f80043390 con 0x7f5f900d1d50 2023-11-08T19:27:58.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.515+0000 7f5f86ffd640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5f7006ae70 0x7f5f7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.515+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f5f800abec0 con 0x7f5f900d1d50 2023-11-08T19:27:58.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.515+0000 7f5f87fff640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5f7006ae70 0x7f5f7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.516+0000 7f5f87fff640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5f7006ae70 0x7f5f7006d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b44c0 tx=0x7f5f78009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.519+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f9015e090 con 0x7f5f900d1d50 2023-11-08T19:27:58.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.747+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f9015e090 con 0x7f5f900d1d50 2023-11-08T19:27:58.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.748+0000 7f5f86ffd640 1 -- 172.21.15.119:0/892574895 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f5f9015e090 con 0x7f5f900d1d50 2023-11-08T19:27:58.751 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:58.751 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:36.122432+0000","modified":"2023-11-08T19:27:39.195905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/913059196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":913059196},{"type":"v1","addr":"172.21.15.188:6835","nonce":913059196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:58.751 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2023-11-08T19:27:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5f7006ae70 msgr2=0x7f5f7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5f7006ae70 0x7f5f7006d330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b44c0 tx=0x7f5f78009040 comp rx=0 tx=0).stop 2023-11-08T19:27:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 msgr2=0x7f5f901590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 0x7f5f901590a0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f5f800098c0 tx=0x7f5f80004730 comp rx=0 tx=0).stop 2023-11-08T19:27:58.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 shutdown_connections 2023-11-08T19:27:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5f7006ae70 0x7f5f7006d330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5f9015d4c0 0x7f5f901f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5f9015d0e0 0x7f5f901601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 --2- 172.21.15.119:0/892574895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5f900d1d50 0x7f5f901590a0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 >> 172.21.15.119:0/892574895 conn(0x7f5f900d2f60 msgr2=0x7f5f900d15d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 shutdown_connections 2023-11-08T19:27:58.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.751+0000 7f5f8f59e640 1 -- 172.21.15.119:0/892574895 wait complete. 2023-11-08T19:27:58.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:27:58.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.985+0000 7f8c54026640 1 Processor -- start 2023-11-08T19:27:58.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.985+0000 7f8c54026640 1 -- start start 2023-11-08T19:27:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c54026640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c142f80 0x7f8c4c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c54026640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c4c15d9f0 0x7f8c4c0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c54026640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4c0c4960 con 0x7f8c4c06cb40 2023-11-08T19:27:58.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c54026640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4c143a40 con 0x7f8c4c15d9f0 2023-11-08T19:27:58.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c54026640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c15df60 con 0x7f8c4c142f80 2023-11-08T19:27:58.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c515c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c4c15d9f0 0x7f8c4c0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c515c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c4c15d9f0 0x7f8c4c0c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40322/0 (socket says 172.21.15.119:40322) 2023-11-08T19:27:58.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c515c1640 1 -- 172.21.15.119:0/1216573964 learned_addr learned my addr 172.21.15.119:0/1216573964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:58.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.987+0000 7f8c51dc2640 1 --2- 172.21.15.119:0/1216573964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c142f80 0x7f8c4c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:58.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.988+0000 7f8c515c1640 1 -- 172.21.15.119:0/1216573964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c142f80 msgr2=0x7f8c4c0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.988+0000 7f8c515c1640 1 --2- 172.21.15.119:0/1216573964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c142f80 0x7f8c4c0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.988+0000 7f8c515c1640 1 -- 172.21.15.119:0/1216573964 >> v1:172.21.15.188:6789/0 conn(0x7f8c4c06cb40 legacy=0x7f8c4c0c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:58.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.988+0000 7f8c515c1640 1 -- 172.21.15.119:0/1216573964 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4c15e240 con 0x7f8c4c15d9f0 2023-11-08T19:27:58.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.989+0000 7f8c515c1640 1 --2- 172.21.15.119:0/1216573964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c15d9f0 0x7f8c4c0c9cd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009870 tx=0x7f8c3c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831ad69ba195ed79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:58.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.989+0000 7f8c50dc0640 1 -- 172.21.15.119:0/1216573964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0317c0 con 0x7f8c4c15d9f0 2023-11-08T19:27:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.989+0000 7f8c50dc0640 1 -- 172.21.15.119:0/1216573964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c031920 con 0x7f8c4c15d9f0 2023-11-08T19:27:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.989+0000 7f8c50dc0640 1 -- 172.21.15.119:0/1216573964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c031cb0 con 0x7f8c4c15d9f0 2023-11-08T19:27:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 -- 172.21.15.119:0/1216573964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c15d9f0 msgr2=0x7f8c4c0c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 --2- 172.21.15.119:0/1216573964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c15d9f0 0x7f8c4c0c9cd0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c009870 tx=0x7f8c3c02b860 comp rx=0 tx=0).stop 2023-11-08T19:27:58.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 -- 172.21.15.119:0/1216573964 shutdown_connections 2023-11-08T19:27:58.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 --2- 172.21.15.119:0/1216573964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c15d9f0 0x7f8c4c0c9cd0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 --2- 172.21.15.119:0/1216573964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c142f80 0x7f8c4c0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:58.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 -- 172.21.15.119:0/1216573964 >> 172.21.15.119:0/1216573964 conn(0x7f8c4c0cb610 msgr2=0x7f8c4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:58.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 -- 172.21.15.119:0/1216573964 shutdown_connections 2023-11-08T19:27:58.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.990+0000 7f8c54026640 1 -- 172.21.15.119:0/1216573964 wait complete. 2023-11-08T19:27:58.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.992+0000 7f8c54026640 1 Processor -- start 2023-11-08T19:27:58.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.992+0000 7f8c54026640 1 -- start start 2023-11-08T19:27:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c54026640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c4c06cb40 0x7f8c4c15cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c54026640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 0x7f8c4c0ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c54026640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c15d9f0 0x7f8c4c0c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:58.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c54026640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c13abf0 con 0x7f8c4c142f80 2023-11-08T19:27:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c54026640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c1f2a30 con 0x7f8c4c06cb40 2023-11-08T19:27:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c54026640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4c1f2d10 con 0x7f8c4c15d9f0 2023-11-08T19:27:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 0x7f8c4c0ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 0x7f8c4c0ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40328/0 (socket says 172.21.15.119:40328) 2023-11-08T19:27:59.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 -- 172.21.15.119:0/3764692387 learned_addr learned my addr 172.21.15.119:0/3764692387 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:59.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c51dc2640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c4c06cb40 0x7f8c4c15cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 -- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c15d9f0 msgr2=0x7f8c4c0c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c15d9f0 0x7f8c4c0c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 -- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c4c06cb40 msgr2=0x7f8c4c15cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c4c06cb40 0x7f8c4c15cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c515c1640 1 -- 172.21.15.119:0/3764692387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c3c0094d0 con 0x7f8c4c142f80 2023-11-08T19:27:59.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.993+0000 7f8c51dc2640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c4c06cb40 0x7f8c4c15cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:27:59.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.994+0000 7f8c515c1640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 0x7f8c4c0ae7e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c000c00 tx=0x7f8c3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.994+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c032860 con 0x7f8c4c142f80 2023-11-08T19:27:59.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.994+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0329c0 con 0x7f8c4c142f80 2023-11-08T19:27:59.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.994+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c032d70 con 0x7f8c4c142f80 2023-11-08T19:27:59.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.994+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4c1f2ff0 con 0x7f8c4c142f80 2023-11-08T19:27:59.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.994+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c4c1f35b0 con 0x7f8c4c142f80 2023-11-08T19:27:59.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.995+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c4c0ac8c0 con 0x7f8c4c142f80 2023-11-08T19:27:59.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.996+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c3c04e450 con 0x7f8c4c142f80 2023-11-08T19:27:59.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.997+0000 7f8c427fc640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c3006ae70 0x7f8c3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.997+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f8c3c0abd40 con 0x7f8c4c142f80 2023-11-08T19:27:59.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.997+0000 7f8c51dc2640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c3006ae70 0x7f8c3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:58.998+0000 7f8c51dc2640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c3006ae70 0x7f8c3006d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006fd0 tx=0x7f8c48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.002+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c4c0ac8c0 con 0x7f8c4c142f80 2023-11-08T19:27:59.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.230+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c4c0ac8c0 con 0x7f8c4c142f80 2023-11-08T19:27:59.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.231+0000 7f8c427fc640 1 -- 172.21.15.119:0/3764692387 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f8c4c0ac8c0 con 0x7f8c4c142f80 2023-11-08T19:27:59.233 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:59.234 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":35,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:27:36.122432+0000","modified":"2023-11-08T19:27:39.195905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6147":{"gid":6147,"name":"d","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/913059196","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":913059196},{"type":"v1","addr":"172.21.15.188:6835","nonce":913059196}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T19:27:59.234 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 35 2023-11-08T19:27:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c3006ae70 msgr2=0x7f8c3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c3006ae70 0x7f8c3006d330 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8c48006fd0 tx=0x7f8c48008040 comp rx=0 tx=0).stop 2023-11-08T19:27:59.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 msgr2=0x7f8c4c0ae7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 0x7f8c4c0ae7e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f8c3c000c00 tx=0x7f8c3c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 shutdown_connections 2023-11-08T19:27:59.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c3006ae70 0x7f8c3006d330 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c4c15d9f0 0x7f8c4c0c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c4c142f80 0x7f8c4c0ae7e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 --2- 172.21.15.119:0/3764692387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c4c06cb40 0x7f8c4c15cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.233+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 >> 172.21.15.119:0/3764692387 conn(0x7f8c4c0cb610 msgr2=0x7f8c4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.234+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 shutdown_connections 2023-11-08T19:27:59.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.234+0000 7f8c54026640 1 -- 172.21.15.119:0/3764692387 wait complete. 2023-11-08T19:27:59.255 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:27:59.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.456+0000 7f3a80c4b640 1 Processor -- start 2023-11-08T19:27:59.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.456+0000 7f3a80c4b640 1 -- start start 2023-11-08T19:27:59.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.457+0000 7f3a80c4b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a7c15f4c0 0x7f3a7c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a80c4b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3a7c06a120 0x7f3a7c06a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a80c4b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c06aa70 con 0x7f3a7c1447a0 2023-11-08T19:27:59.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a80c4b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c15f930 con 0x7f3a7c15f4c0 2023-11-08T19:27:59.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a80c4b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c3a50 con 0x7f3a7c06a120 2023-11-08T19:27:59.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a7c15f4c0 0x7f3a7c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3a7c15f4c0 0x7f3a7c06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40338/0 (socket says 172.21.15.119:40338) 2023-11-08T19:27:59.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 -- 172.21.15.119:0/2755272272 learned_addr learned my addr 172.21.15.119:0/2755272272 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:59.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 -- 172.21.15.119:0/2755272272 >> v2:172.21.15.188:3300/0 conn(0x7f3a7c06a120 msgr2=0x7f3a7c06a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 --2- 172.21.15.119:0/2755272272 >> v2:172.21.15.188:3300/0 conn(0x7f3a7c06a120 0x7f3a7c06a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 -- 172.21.15.119:0/2755272272 >> v1:172.21.15.188:6789/0 conn(0x7f3a7c1447a0 legacy=0x7f3a7c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.458+0000 7f3a73fff640 1 -- 172.21.15.119:0/2755272272 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c144b80 con 0x7f3a7c15f4c0 2023-11-08T19:27:59.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.459+0000 7f3a73fff640 1 --2- 172.21.15.119:0/2755272272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c15f4c0 0x7f3a7c06f930 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c06df20 tx=0x7f3a6402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7801b1cd8fe984 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.460+0000 7f3a79d9b640 1 -- 172.21.15.119:0/2755272272 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64031860 con 0x7f3a7c15f4c0 2023-11-08T19:27:59.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.460+0000 7f3a79d9b640 1 -- 172.21.15.119:0/2755272272 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a640319c0 con 0x7f3a7c15f4c0 2023-11-08T19:27:59.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.460+0000 7f3a79d9b640 1 -- 172.21.15.119:0/2755272272 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a64031d50 con 0x7f3a7c15f4c0 2023-11-08T19:27:59.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.460+0000 7f3a80c4b640 1 -- 172.21.15.119:0/2755272272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c15f4c0 msgr2=0x7f3a7c06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.460+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/2755272272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c15f4c0 0x7f3a7c06f930 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c06df20 tx=0x7f3a6402b900 comp rx=0 tx=0).stop 2023-11-08T19:27:59.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.461+0000 7f3a80c4b640 1 -- 172.21.15.119:0/2755272272 shutdown_connections 2023-11-08T19:27:59.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.461+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/2755272272 >> v2:172.21.15.188:3300/0 conn(0x7f3a7c06a120 0x7f3a7c06a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.461+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/2755272272 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c15f4c0 0x7f3a7c06f930 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.461+0000 7f3a80c4b640 1 -- 172.21.15.119:0/2755272272 >> 172.21.15.119:0/2755272272 conn(0x7f3a7c0cb610 msgr2=0x7f3a7c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.461+0000 7f3a80c4b640 1 -- 172.21.15.119:0/2755272272 shutdown_connections 2023-11-08T19:27:59.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.461+0000 7f3a80c4b640 1 -- 172.21.15.119:0/2755272272 wait complete. 2023-11-08T19:27:59.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.462+0000 7f3a80c4b640 1 Processor -- start 2023-11-08T19:27:59.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.463+0000 7f3a80c4b640 1 -- start start 2023-11-08T19:27:59.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.463+0000 7f3a80c4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 0x7f3a7c15e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a80c4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c1447a0 0x7f3a7c15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a80c4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a7c15f4c0 0x7f3a7c0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a80c4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c15e560 con 0x7f3a7c06a120 2023-11-08T19:27:59.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a80c4b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c7170 con 0x7f3a7c1447a0 2023-11-08T19:27:59.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a80c4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c0c7450 con 0x7f3a7c15f4c0 2023-11-08T19:27:59.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 0x7f3a7c15e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 0x7f3a7c15e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40340/0 (socket says 172.21.15.119:40340) 2023-11-08T19:27:59.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 -- 172.21.15.119:0/1138947971 learned_addr learned my addr 172.21.15.119:0/1138947971 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:59.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 -- 172.21.15.119:0/1138947971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a7c15f4c0 msgr2=0x7f3a7c0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a7c15f4c0 0x7f3a7c0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 -- 172.21.15.119:0/1138947971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c1447a0 msgr2=0x7f3a7c15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c1447a0 0x7f3a7c15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 -- 172.21.15.119:0/1138947971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a640094d0 con 0x7f3a7c06a120 2023-11-08T19:27:59.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.464+0000 7f3a7a59c640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 0x7f3a7c15e020 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00c410 tx=0x7f3a6c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.465+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0035f0 con 0x7f3a7c06a120 2023-11-08T19:27:59.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.465+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c003750 con 0x7f3a7c06a120 2023-11-08T19:27:59.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.465+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c003b40 con 0x7f3a7c06a120 2023-11-08T19:27:59.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.465+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c0c7730 con 0x7f3a7c06a120 2023-11-08T19:27:59.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.465+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0c7cf0 con 0x7f3a7c06a120 2023-11-08T19:27:59.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.466+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c0c7ee0 con 0x7f3a7c06a120 2023-11-08T19:27:59.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.467+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3a6c003ca0 con 0x7f3a7c06a120 2023-11-08T19:27:59.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.468+0000 7f3a72ffd640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a5c06ae70 0x7f3a5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.468+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f3a6c088e20 con 0x7f3a7c06a120 2023-11-08T19:27:59.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.469+0000 7f3a73fff640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a5c06ae70 0x7f3a5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.469+0000 7f3a73fff640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a5c06ae70 0x7f3a5c06d330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a64036000 tx=0x7f3a64030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.473+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3a6c053900 con 0x7f3a7c06a120 2023-11-08T19:27:59.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.666+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a7c074430 con 0x7f3a7c06a120 2023-11-08T19:27:59.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.670+0000 7f3a72ffd640 1 -- 172.21.15.119:0/1138947971 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+22155 (secure 0 0 0) 0x7f3a6c056f20 con 0x7f3a7c06a120 2023-11-08T19:27:59.672 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:27:59.673 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":72,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:27:53.645252+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:27:33.928136+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T19:27:35.135101+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.26","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":1,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:27:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a5c06ae70 msgr2=0x7f3a5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a5c06ae70 0x7f3a5c06d330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3a64036000 tx=0x7f3a64030000 comp rx=0 tx=0).stop 2023-11-08T19:27:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 msgr2=0x7f3a7c15e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 0x7f3a7c15e020 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00c410 tx=0x7f3a6c00c8d0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 shutdown_connections 2023-11-08T19:27:59.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3a5c06ae70 0x7f3a5c06d330 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3a7c15f4c0 0x7f3a7c0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3a7c1447a0 0x7f3a7c15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 --2- 172.21.15.119:0/1138947971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3a7c06a120 0x7f3a7c15e020 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 >> 172.21.15.119:0/1138947971 conn(0x7f3a7c0cb610 msgr2=0x7f3a7c0cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.672+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 shutdown_connections 2023-11-08T19:27:59.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.673+0000 7f3a80c4b640 1 -- 172.21.15.119:0/1138947971 wait complete. 2023-11-08T19:27:59.695 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:27:59.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.904+0000 7f96b66d4640 1 Processor -- start 2023-11-08T19:27:59.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.905+0000 7f96b66d4640 1 -- start start 2023-11-08T19:27:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.906+0000 7f96b66d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 0x7f96b006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.906+0000 7f96b66d4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b015f900 con 0x7f96b006a120 2023-11-08T19:27:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.906+0000 7f96b66d4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b00c3a50 con 0x7f96b01447a0 2023-11-08T19:27:59.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.906+0000 7f96b66d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0144b80 con 0x7f96b015f4c0 2023-11-08T19:27:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.907+0000 7f96b4c71640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f96b006a120 0x7f96b00c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37672/0 (socket says 172.21.15.119:37672) 2023-11-08T19:27:59.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.907+0000 7f96b4c71640 1 -- 172.21.15.119:0/3468056962 learned_addr learned my addr 172.21.15.119:0/3468056962 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:27:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.907+0000 7f96a7fff640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 0x7f96b006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.907+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384170147 0 0) 0x7f96a8001490 con 0x7f96b006a120 2023-11-08T19:27:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.907+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969c003520 con 0x7f96b006a120 2023-11-08T19:27:59.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 256394235 0 0) 0x7f969c003520 con 0x7f96b006a120 2023-11-08T19:27:59.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 msgr2=0x7f96b006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 0x7f96b006f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 >> v1:172.21.15.188:6789/0 conn(0x7f96b01447a0 legacy=0x7f96b00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00c9150 con 0x7f96b006a120 2023-11-08T19:27:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f96a8001b80 con 0x7f96b006a120 2023-11-08T19:27:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 >> v1:172.21.15.119:6789/0 conn(0x7f96b006a120 legacy=0x7f96b00c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 0x7f969c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c003f20 0x7f969c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969c004d00 0x7f969c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0144b80 con 0x7f969c004d00 2023-11-08T19:27:59.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c00ba60 con 0x7f969c003f20 2023-11-08T19:27:59.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c00bc20 con 0x7f96b015f4c0 2023-11-08T19:27:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96b4c71640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969c004d00 0x7f969c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.908+0000 7f96affff640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 0x7f969c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.909+0000 7f96b4c71640 1 -- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 msgr2=0x7f969c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.909+0000 7f96a7fff640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c003f20 0x7f969c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.909+0000 7f96b4c71640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b015f4c0 0x7f969c0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.909+0000 7f96b4c71640 1 -- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c003f20 msgr2=0x7f969c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:27:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.909+0000 7f96b4c71640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c003f20 0x7f969c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.909+0000 7f96b4c71640 1 -- 172.21.15.119:0/3468056962 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c003520 con 0x7f969c004d00 2023-11-08T19:27:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96b4c71640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969c004d00 0x7f969c005180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f96a800a830 tx=0x7f969c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fded0654b0c52a2c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8034630 con 0x7f969c004d00 2023-11-08T19:27:59.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a802f070 con 0x7f969c004d00 2023-11-08T19:27:59.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96af7fe640 1 -- 172.21.15.119:0/3468056962 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8034a30 con 0x7f969c004d00 2023-11-08T19:27:59.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96b66d4640 1 -- 172.21.15.119:0/3468056962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969c004d00 msgr2=0x7f969c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96b66d4640 1 --2- 172.21.15.119:0/3468056962 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f969c004d00 0x7f969c005180 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f96a800a830 tx=0x7f969c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.910+0000 7f96b4c71640 1 -- 172.21.15.119:0/3468056962 reap_dead start 2023-11-08T19:27:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.911+0000 7f96b66d4640 1 -- 172.21.15.119:0/3468056962 shutdown_connections 2023-11-08T19:27:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.911+0000 7f96b66d4640 1 -- 172.21.15.119:0/3468056962 >> 172.21.15.119:0/3468056962 conn(0x7f96b00cb610 msgr2=0x7f96b00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:27:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.911+0000 7f96b66d4640 1 -- 172.21.15.119:0/3468056962 shutdown_connections 2023-11-08T19:27:59.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.911+0000 7f96b66d4640 1 -- 172.21.15.119:0/3468056962 wait complete. 2023-11-08T19:27:59.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.912+0000 7f96b66d4640 1 Processor -- start 2023-11-08T19:27:59.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.913+0000 7f96b66d4640 1 -- start start 2023-11-08T19:27:59.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96b66d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c004d00 0x7f96b00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96b66d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 0x7f96b00c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96b66d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b00c7f00 0x7f96b013d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96b66d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00b3f70 con 0x7f96b015a2a0 2023-11-08T19:27:59.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96b66d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00c8770 con 0x7f969c004d00 2023-11-08T19:27:59.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96b66d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00b4180 con 0x7f96b00c7f00 2023-11-08T19:27:59.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 0x7f96b00c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 0x7f96b00c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40346/0 (socket says 172.21.15.119:40346) 2023-11-08T19:27:59.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 -- 172.21.15.119:0/2352513029 learned_addr learned my addr 172.21.15.119:0/2352513029 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:27:59.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 -- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b00c7f00 msgr2=0x7f96b013d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b00c7f00 0x7f96b013d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 -- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c004d00 msgr2=0x7f96b00c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:27:59.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96affff640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c004d00 0x7f96b00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c004d00 0x7f96b00c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:27:59.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.914+0000 7f96a7fff640 1 -- 172.21.15.119:0/2352513029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00b4460 con 0x7f96b015a2a0 2023-11-08T19:27:59.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.915+0000 7f96a7fff640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 0x7f96b00c79c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9698010810 tx=0x7f9698010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.915+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9698009610 con 0x7f96b015a2a0 2023-11-08T19:27:59.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.915+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969800eb60 con 0x7f96b015a2a0 2023-11-08T19:27:59.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.915+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969801c400 con 0x7f96b015a2a0 2023-11-08T19:27:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.915+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b00c8400 con 0x7f96b015a2a0 2023-11-08T19:27:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.915+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b013ddb0 con 0x7f96b015a2a0 2023-11-08T19:27:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.916+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b00c8920 con 0x7f96b015a2a0 2023-11-08T19:27:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.917+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f969801c560 con 0x7f96b015a2a0 2023-11-08T19:27:59.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.918+0000 7f96ad7fa640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f969006aea0 0x7f969006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:27:59.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.918+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7178+0+0 (secure 0 0 0) 0x7f969808a180 con 0x7f96b015a2a0 2023-11-08T19:27:59.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.918+0000 7f96affff640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f969006aea0 0x7f969006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:27:59.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.919+0000 7f96affff640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f969006aea0 0x7f969006d360 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001590 tx=0x7f96a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:27:59.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:27:59.922+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96b00c8920 con 0x7f96b015a2a0 2023-11-08T19:28:00.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.157+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f96b00c8920 con 0x7f96b015a2a0 2023-11-08T19:28:00.256 INFO:tasks.ceph.mds.d.smithi188.stderr: -1903> 2023-11-08T19:27:42.181+0000 7f832698e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:00.256 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-11-08T19:28:00.158+0000 7f8fd7f7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:28:00.257 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:28:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.162+0000 7f96ad7fa640 1 -- 172.21.15.119:0/2352513029 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) v1 ==== 161+0+0 (secure 0 0 0) 0x7f96b00c8920 con 0x7f96b015a2a0 2023-11-08T19:28:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:28:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f969006aea0 msgr2=0x7f969006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f969006aea0 0x7f969006d360 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f96a0001590 tx=0x7f96a0009040 comp rx=0 tx=0).stop 2023-11-08T19:28:00.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 msgr2=0x7f96b00c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 0x7f96b00c79c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f9698010810 tx=0x7f9698010cd0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 shutdown_connections 2023-11-08T19:28:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f969006aea0 0x7f969006d360 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96b00c7f00 0x7f96b013d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f96b015a2a0 0x7f96b00c79c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 --2- 172.21.15.119:0/2352513029 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f969c004d00 0x7f96b00c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 >> 172.21.15.119:0/2352513029 conn(0x7f96b00cb610 msgr2=0x7f96b00cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 shutdown_connections 2023-11-08T19:28:00.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.165+0000 7f96b66d4640 1 -- 172.21.15.119:0/2352513029 wait complete. 2023-11-08T19:28:00.262 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:28:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.484+0000 7f707b778640 1 Processor -- start 2023-11-08T19:28:00.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.485+0000 7f707b778640 1 -- start start 2023-11-08T19:28:00.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.486+0000 7f707b778640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f707414af50 0x7f70740cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.486+0000 7f707b778640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70740c42a0 con 0x7f707414a1e0 2023-11-08T19:28:00.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.486+0000 7f707b778640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70740c4580 con 0x7f70740cb7c0 2023-11-08T19:28:00.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.486+0000 7f707b778640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70740c4860 con 0x7f707414af50 2023-11-08T19:28:00.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.486+0000 7f7078d13640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f707414af50 0x7f70740cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f7079514640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f707414a1e0 0x7f70740c39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37686/0 (socket says 172.21.15.119:37686) 2023-11-08T19:28:00.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f7079514640 1 -- 172.21.15.119:0/1690669069 learned_addr learned my addr 172.21.15.119:0/1690669069 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:00.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f7078d13640 1 --2- 172.21.15.119:0/1690669069 >> v2:172.21.15.119:3300/0 conn(0x7f707414af50 0x7f70740cb280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40348/0 (socket says 172.21.15.119:40348) 2023-11-08T19:28:00.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f7078d13640 1 -- 172.21.15.119:0/1690669069 >> v1:172.21.15.188:6789/0 conn(0x7f70740cb7c0 legacy=0x7f70740ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:00.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f7078d13640 1 -- 172.21.15.119:0/1690669069 >> v1:172.21.15.119:6789/0 conn(0x7f707414a1e0 legacy=0x7f70740c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f7078d13640 1 -- 172.21.15.119:0/1690669069 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70740cc730 con 0x7f707414af50 2023-11-08T19:28:00.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.487+0000 7f706bfff640 1 -- 172.21.15.119:0/1690669069 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113032575 0 0) 0x7f70740c42a0 con 0x7f707414a1e0 2023-11-08T19:28:00.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f7078d13640 1 --2- 172.21.15.119:0/1690669069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414af50 0x7f70740cb280 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f70640097a0 tx=0x7f706402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a882a48b0ff96b38 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f706bfff640 1 -- 172.21.15.119:0/1690669069 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064031830 con 0x7f707414af50 2023-11-08T19:28:00.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f706bfff640 1 -- 172.21.15.119:0/1690669069 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7064031990 con 0x7f707414af50 2023-11-08T19:28:00.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f706bfff640 1 -- 172.21.15.119:0/1690669069 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064031d20 con 0x7f707414af50 2023-11-08T19:28:00.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 -- 172.21.15.119:0/1690669069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414af50 msgr2=0x7f70740cb280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 --2- 172.21.15.119:0/1690669069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414af50 0x7f70740cb280 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f70640097a0 tx=0x7f706402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 -- 172.21.15.119:0/1690669069 shutdown_connections 2023-11-08T19:28:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 --2- 172.21.15.119:0/1690669069 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414af50 0x7f70740cb280 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 -- 172.21.15.119:0/1690669069 >> 172.21.15.119:0/1690669069 conn(0x7f70740b0950 msgr2=0x7f70740b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:00.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 -- 172.21.15.119:0/1690669069 shutdown_connections 2023-11-08T19:28:00.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.489+0000 7f707b778640 1 -- 172.21.15.119:0/1690669069 wait complete. 2023-11-08T19:28:00.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.491+0000 7f707b778640 1 Processor -- start 2023-11-08T19:28:00.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.491+0000 7f707b778640 1 -- start start 2023-11-08T19:28:00.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f707b778640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70740cb7c0 0x7f7074148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f707b778640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 0x7f7074144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f707b778640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707414af50 0x7f7074144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f707b778640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7074144c50 con 0x7f707414a1e0 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f707b778640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7074144f30 con 0x7f707414af50 2023-11-08T19:28:00.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f707b778640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7074145210 con 0x7f70740cb7c0 2023-11-08T19:28:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f7078d13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 0x7f7074144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f7078d13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 0x7f7074144100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40360/0 (socket says 172.21.15.119:40360) 2023-11-08T19:28:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.492+0000 7f7078d13640 1 -- 172.21.15.119:0/3975672008 learned_addr learned my addr 172.21.15.119:0/3975672008 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7078d13640 1 -- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70740cb7c0 msgr2=0x7f7074148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7078d13640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70740cb7c0 0x7f7074148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7078d13640 1 -- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707414af50 msgr2=0x7f7074144640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7079d15640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707414af50 0x7f7074144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7078d13640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707414af50 0x7f7074144640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7078d13640 1 -- 172.21.15.119:0/3975672008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7064009400 con 0x7f707414a1e0 2023-11-08T19:28:00.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7079d15640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707414af50 0x7f7074144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7078d13640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 0x7f7074144100 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f70640097a0 tx=0x7f70640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.493+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706403aa60 con 0x7f707414a1e0 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.494+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706403abc0 con 0x7f707414a1e0 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.494+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70741454f0 con 0x7f707414a1e0 2023-11-08T19:28:00.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.494+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70640443f0 con 0x7f707414a1e0 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.494+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7074163670 con 0x7f707414a1e0 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.495+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707404ece0 con 0x7f707414a1e0 2023-11-08T19:28:00.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.496+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f706404d450 con 0x7f707414a1e0 2023-11-08T19:28:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.497+0000 7f7069ffb640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705806ae70 0x7f705806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.497+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7236+0+0 (secure 0 0 0) 0x7f70640abf00 con 0x7f707414a1e0 2023-11-08T19:28:00.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.497+0000 7f7079514640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705806ae70 0x7f705806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:00.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.498+0000 7f7079514640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705806ae70 0x7f705806d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7070007fd0 tx=0x7f7070007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:00.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.501+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7064076910 con 0x7f707414a1e0 2023-11-08T19:28:00.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:00.743+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70740c5720 con 0x7f707414a1e0 2023-11-08T19:28:01.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.175+0000 7f7069ffb640 1 -- 172.21.15.119:0/3975672008 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7064079f30 con 0x7f707414a1e0 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705806ae70 msgr2=0x7f705806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705806ae70 0x7f705806d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7070007fd0 tx=0x7f7070007920 comp rx=0 tx=0).stop 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 msgr2=0x7f7074144100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 0x7f7074144100 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f70640097a0 tx=0x7f70640046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 shutdown_connections 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705806ae70 0x7f705806d330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f707414af50 0x7f7074144640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f707414a1e0 0x7f7074144100 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 --2- 172.21.15.119:0/3975672008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70740cb7c0 0x7f7074148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 >> 172.21.15.119:0/3975672008 conn(0x7f70740b0950 msgr2=0x7f70740ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:01.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 shutdown_connections 2023-11-08T19:28:01.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.179+0000 7f707b778640 1 -- 172.21.15.119:0/3975672008 wait complete. 2023-11-08T19:28:01.203 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:28:01.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.423+0000 7fe0cb303640 1 Processor -- start 2023-11-08T19:28:01.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.423+0000 7fe0cb303640 1 -- start start 2023-11-08T19:28:01.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cb303640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe0c406aa70 0x7fe0c40b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cb303640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe0c40cb410 0x7fe0c406b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cb303640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c415b380 con 0x7fe0c4158870 2023-11-08T19:28:01.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cb303640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c415c3e0 con 0x7fe0c40cb410 2023-11-08T19:28:01.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cb303640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0c40b3dd0 con 0x7fe0c406aa70 2023-11-08T19:28:01.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cab02640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe0c40cb410 0x7fe0c406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cab02640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe0c40cb410 0x7fe0c406b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40368/0 (socket says 172.21.15.119:40368) 2023-11-08T19:28:01.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.425+0000 7fe0cab02640 1 -- 172.21.15.119:0/287385344 learned_addr learned my addr 172.21.15.119:0/287385344 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:01.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.426+0000 7fe0c9b00640 1 --2- 172.21.15.119:0/287385344 >> v2:172.21.15.188:3300/0 conn(0x7fe0c406aa70 0x7fe0c40b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.426+0000 7fe0cab02640 1 -- 172.21.15.119:0/287385344 >> v2:172.21.15.188:3300/0 conn(0x7fe0c406aa70 msgr2=0x7fe0c40b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:01.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.426+0000 7fe0cab02640 1 --2- 172.21.15.119:0/287385344 >> v2:172.21.15.188:3300/0 conn(0x7fe0c406aa70 0x7fe0c40b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.426+0000 7fe0cab02640 1 -- 172.21.15.119:0/287385344 >> v1:172.21.15.188:6789/0 conn(0x7fe0c4158870 legacy=0x7fe0c406a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:01.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.426+0000 7fe0cab02640 1 -- 172.21.15.119:0/287385344 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c415b620 con 0x7fe0c40cb410 2023-11-08T19:28:01.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.427+0000 7fe0cab02640 1 --2- 172.21.15.119:0/287385344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c406b920 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4071b80 tx=0x7fe0bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a6ee0f70142ac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.427+0000 7fe0c92ff640 1 -- 172.21.15.119:0/287385344 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc02ca80 con 0x7fe0c40cb410 2023-11-08T19:28:01.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0c92ff640 1 -- 172.21.15.119:0/287385344 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc02cbe0 con 0x7fe0c40cb410 2023-11-08T19:28:01.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0c92ff640 1 -- 172.21.15.119:0/287385344 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0343f0 con 0x7fe0c40cb410 2023-11-08T19:28:01.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 -- 172.21.15.119:0/287385344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 msgr2=0x7fe0c406b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 --2- 172.21.15.119:0/287385344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c406b920 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4071b80 tx=0x7fe0bc02bee0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 -- 172.21.15.119:0/287385344 shutdown_connections 2023-11-08T19:28:01.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 --2- 172.21.15.119:0/287385344 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c406b920 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 --2- 172.21.15.119:0/287385344 >> v2:172.21.15.188:3300/0 conn(0x7fe0c406aa70 0x7fe0c40b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 -- 172.21.15.119:0/287385344 >> 172.21.15.119:0/287385344 conn(0x7fe0c40b0870 msgr2=0x7fe0c40b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:01.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 -- 172.21.15.119:0/287385344 shutdown_connections 2023-11-08T19:28:01.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.428+0000 7fe0cb303640 1 -- 172.21.15.119:0/287385344 wait complete. 2023-11-08T19:28:01.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.430+0000 7fe0cb303640 1 Processor -- start 2023-11-08T19:28:01.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.430+0000 7fe0cb303640 1 -- start start 2023-11-08T19:28:01.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.431+0000 7fe0cb303640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c406aa70 0x7fe0c4153600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.431+0000 7fe0cb303640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c4159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.431+0000 7fe0cb303640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c4158870 0x7fe0c415a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.431+0000 7fe0cb303640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c406fee0 con 0x7fe0c40cb410 2023-11-08T19:28:01.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.431+0000 7fe0cb303640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c4153b40 con 0x7fe0c4158870 2023-11-08T19:28:01.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.431+0000 7fe0cb303640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c415a6b0 con 0x7fe0c406aa70 2023-11-08T19:28:01.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c4159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c4159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40382/0 (socket says 172.21.15.119:40382) 2023-11-08T19:28:01.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 -- 172.21.15.119:0/1485296628 learned_addr learned my addr 172.21.15.119:0/1485296628 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:01.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0cab02640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c4158870 0x7fe0c415a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 -- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c406aa70 msgr2=0x7fe0c4153600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c406aa70 0x7fe0c4153600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 -- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c4158870 msgr2=0x7fe0c415a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:01.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c4158870 0x7fe0c415a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:01.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 -- 172.21.15.119:0/1485296628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc009b30 con 0x7fe0c40cb410 2023-11-08T19:28:01.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.432+0000 7fe0c9b00640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c4159b40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400d800 tx=0x7fe0b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.433+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400b6c0 con 0x7fe0c40cb410 2023-11-08T19:28:01.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.433+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b400b820 con 0x7fe0c40cb410 2023-11-08T19:28:01.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.433+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c415a990 con 0x7fe0c40cb410 2023-11-08T19:28:01.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.433+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b400bbb0 con 0x7fe0c40cb410 2023-11-08T19:28:01.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.433+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c406fb70 con 0x7fe0c40cb410 2023-11-08T19:28:01.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.434+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c404ece0 con 0x7fe0c40cb410 2023-11-08T19:28:01.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.435+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0b400bd10 con 0x7fe0c40cb410 2023-11-08T19:28:01.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.436+0000 7fe0baffd640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe0a806ae70 0x7fe0a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:01.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.436+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 7236+0+0 (secure 0 0 0) 0x7fe0b408a180 con 0x7fe0c40cb410 2023-11-08T19:28:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.436+0000 7fe0ca301640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe0a806ae70 0x7fe0a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.437+0000 7fe0ca301640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe0a806ae70 0x7fe0a806d330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0006fd0 tx=0x7fe0c0006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:01.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.440+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe0b4054c20 con 0x7fe0c40cb410 2023-11-08T19:28:01.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:01.631+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe0c406d4d0 con 0x7fe0c40cb410 2023-11-08T19:28:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.179+0000 7fe0baffd640 1 -- 172.21.15.119:0/1485296628 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe0b4058240 con 0x7fe0c40cb410 2023-11-08T19:28:02.181 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:28:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe0a806ae70 msgr2=0x7fe0a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe0a806ae70 0x7fe0a806d330 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0006fd0 tx=0x7fe0c0006bc0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 msgr2=0x7fe0c4159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c4159b40 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe0b400d800 tx=0x7fe0b400dcc0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 shutdown_connections 2023-11-08T19:28:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe0a806ae70 0x7fe0a806d330 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0c4158870 0x7fe0c415a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0c40cb410 0x7fe0c4159b40 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 --2- 172.21.15.119:0/1485296628 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0c406aa70 0x7fe0c4153600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 >> 172.21.15.119:0/1485296628 conn(0x7fe0c40b0870 msgr2=0x7fe0c40b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:02.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 shutdown_connections 2023-11-08T19:28:02.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.183+0000 7fe0cb303640 1 -- 172.21.15.119:0/1485296628 wait complete. 2023-11-08T19:28:02.208 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:28:02.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.415+0000 7fd18b3cc640 1 Processor -- start 2023-11-08T19:28:02.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.416+0000 7fd18b3cc640 1 -- start start 2023-11-08T19:28:02.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.417+0000 7fd18b3cc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.417+0000 7fd18b3cc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1840cb990 0x7fd1840ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.417+0000 7fd18b3cc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1840c42a0 con 0x7fd18414af50 2023-11-08T19:28:02.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.417+0000 7fd18b3cc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1840c4580 con 0x7fd1840cb990 2023-11-08T19:28:02.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.417+0000 7fd18b3cc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1840c4860 con 0x7fd18414a1e0 2023-11-08T19:28:02.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1840cb990 0x7fd1840ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1840cb990 0x7fd1840ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40384/0 (socket says 172.21.15.119:40384) 2023-11-08T19:28:02.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 -- 172.21.15.119:0/1377604822 learned_addr learned my addr 172.21.15.119:0/1377604822 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:02.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd188967640 1 --1- 172.21.15.119:0/1377604822 >> v1:172.21.15.119:6789/0 conn(0x7fd18414af50 0x7fd1840cb280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37700/0 (socket says 172.21.15.119:37700) 2023-11-08T19:28:02.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189168640 1 --2- 172.21.15.119:0/1377604822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 -- 172.21.15.119:0/1377604822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 msgr2=0x7fd1840c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:02.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 --2- 172.21.15.119:0/1377604822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 -- 172.21.15.119:0/1377604822 >> v1:172.21.15.119:6789/0 conn(0x7fd18414af50 legacy=0x7fd1840cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.418+0000 7fd189969640 1 -- 172.21.15.119:0/1377604822 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1840cc730 con 0x7fd1840cb990 2023-11-08T19:28:02.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.419+0000 7fd189969640 1 --2- 172.21.15.119:0/1377604822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1840cb990 0x7fd1840ccf30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6057492a83e5784 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:02.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.419+0000 7fd183fff640 1 -- 172.21.15.119:0/1377604822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c030840 con 0x7fd1840cb990 2023-11-08T19:28:02.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.419+0000 7fd183fff640 1 -- 172.21.15.119:0/1377604822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c0309a0 con 0x7fd1840cb990 2023-11-08T19:28:02.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.419+0000 7fd183fff640 1 -- 172.21.15.119:0/1377604822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c030d30 con 0x7fd1840cb990 2023-11-08T19:28:02.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 -- 172.21.15.119:0/1377604822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1840cb990 msgr2=0x7fd1840ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/1377604822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1840cb990 0x7fd1840ccf30 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c02b980 comp rx=0 tx=0).stop 2023-11-08T19:28:02.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 -- 172.21.15.119:0/1377604822 shutdown_connections 2023-11-08T19:28:02.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/1377604822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1840cb990 0x7fd1840ccf30 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/1377604822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 -- 172.21.15.119:0/1377604822 >> 172.21.15.119:0/1377604822 conn(0x7fd1840b0950 msgr2=0x7fd1840b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:02.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 -- 172.21.15.119:0/1377604822 shutdown_connections 2023-11-08T19:28:02.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.420+0000 7fd18b3cc640 1 -- 172.21.15.119:0/1377604822 wait complete. 2023-11-08T19:28:02.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.422+0000 7fd18b3cc640 1 Processor -- start 2023-11-08T19:28:02.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.422+0000 7fd18b3cc640 1 -- start start 2023-11-08T19:28:02.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd18b3cc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1840cb990 0x7fd1841f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd18b3cc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd18b3cc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 0x7fd1841486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd18b3cc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184161350 con 0x7fd18414af50 2023-11-08T19:28:02.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd18b3cc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd184161e90 con 0x7fd1840cb990 2023-11-08T19:28:02.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd18b3cc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd184161560 con 0x7fd18414a1e0 2023-11-08T19:28:02.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd189969640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 0x7fd1841486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd189969640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 0x7fd1841486c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40400/0 (socket says 172.21.15.119:40400) 2023-11-08T19:28:02.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.423+0000 7fd189969640 1 -- 172.21.15.119:0/3498543093 learned_addr learned my addr 172.21.15.119:0/3498543093 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:02.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189168640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1840cb990 0x7fd1841f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd188967640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189969640 1 -- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 msgr2=0x7fd1840c6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189969640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189969640 1 -- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1840cb990 msgr2=0x7fd1841f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:02.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189969640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1840cb990 0x7fd1841f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:02.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189969640 1 -- 172.21.15.119:0/3498543093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0094d0 con 0x7fd18414af50 2023-11-08T19:28:02.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189168640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1840cb990 0x7fd1841f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:02.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd189969640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 0x7fd1841486c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:02.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c039c90 con 0x7fd18414af50 2023-11-08T19:28:02.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17c039e10 con 0x7fd18414af50 2023-11-08T19:28:02.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17c043640 con 0x7fd18414af50 2023-11-08T19:28:02.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.424+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd184161840 con 0x7fd18414af50 2023-11-08T19:28:02.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.425+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd184147a90 con 0x7fd18414af50 2023-11-08T19:28:02.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.426+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd184162020 con 0x7fd18414af50 2023-11-08T19:28:02.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.427+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd17c0437a0 con 0x7fd18414af50 2023-11-08T19:28:02.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.428+0000 7fd181ffb640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd17006ae70 0x7fd17006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:02.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.428+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6798+0+0 (secure 0 0 0) 0x7fd17c03e070 con 0x7fd18414af50 2023-11-08T19:28:02.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.428+0000 7fd189168640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd17006ae70 0x7fd17006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:02.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.428+0000 7fd189168640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd17006ae70 0x7fd17006d330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd174007330 tx=0x7fd17400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:02.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.432+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd17c0777b0 con 0x7fd18414af50 2023-11-08T19:28:02.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:02.626+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd18404ece0 con 0x7fd18414af50 2023-11-08T19:28:03.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.183+0000 7fd181ffb640 1 -- 172.21.15.119:0/3498543093 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd17c0340d0 con 0x7fd18414af50 2023-11-08T19:28:03.185 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-11-08T19:28:03.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd17006ae70 msgr2=0x7fd17006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd17006ae70 0x7fd17006d330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fd174007330 tx=0x7fd17400b040 comp rx=0 tx=0).stop 2023-11-08T19:28:03.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 msgr2=0x7fd1841486c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 0x7fd1841486c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fd17c009870 tx=0x7fd17c004730 comp rx=0 tx=0).stop 2023-11-08T19:28:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 shutdown_connections 2023-11-08T19:28:03.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd17006ae70 0x7fd17006d330 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd18414af50 0x7fd1841486c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd18414a1e0 0x7fd1840c6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 --2- 172.21.15.119:0/3498543093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1840cb990 0x7fd1841f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.188+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 >> 172.21.15.119:0/3498543093 conn(0x7fd1840b0950 msgr2=0x7fd184164f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.190+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 shutdown_connections 2023-11-08T19:28:03.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.190+0000 7fd18b3cc640 1 -- 172.21.15.119:0/3498543093 wait complete. 2023-11-08T19:28:03.220 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:28:03.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.412+0000 7f5dec426640 1 Processor -- start 2023-11-08T19:28:03.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.412+0000 7f5dec426640 1 -- start start 2023-11-08T19:28:03.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.413+0000 7f5dec426640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5de4143520 0x7f5de40caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.413+0000 7f5dec426640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de415d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.413+0000 7f5dec426640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5de40c9d50 con 0x7f5de4143900 2023-11-08T19:28:03.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.413+0000 7f5dec426640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5de415e5e0 con 0x7f5de4143520 2023-11-08T19:28:03.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.413+0000 7f5dec426640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40c9fa0 con 0x7f5de40c5200 2023-11-08T19:28:03.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.414+0000 7f5dea1c2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5de4143520 0x7f5de40caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.414+0000 7f5dea1c2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5de4143520 0x7f5de40caa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:33342/0 (socket says 172.21.15.119:33342) 2023-11-08T19:28:03.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.414+0000 7f5dea1c2640 1 -- 172.21.15.119:0/2195314709 learned_addr learned my addr 172.21.15.119:0/2195314709 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:03.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.414+0000 7f5dea9c3640 1 --2- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de415d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.415+0000 7f5dea1c2640 1 -- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 msgr2=0x7f5de415d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:03.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.415+0000 7f5dea1c2640 1 --2- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de415d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.415+0000 7f5dea1c2640 1 -- 172.21.15.119:0/2195314709 >> v1:172.21.15.188:6789/0 conn(0x7f5de4143900 legacy=0x7f5de40c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.415+0000 7f5dea1c2640 1 -- 172.21.15.119:0/2195314709 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5de40ca280 con 0x7f5de4143520 2023-11-08T19:28:03.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.416+0000 7f5dea1c2640 1 --2- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de40caa60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009e80 tx=0x7f5dd802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e101335eab85d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.417+0000 7f5de91c0640 1 -- 172.21.15.119:0/2195314709 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd802caf0 con 0x7f5de4143520 2023-11-08T19:28:03.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.417+0000 7f5de91c0640 1 -- 172.21.15.119:0/2195314709 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd802cc50 con 0x7f5de4143520 2023-11-08T19:28:03.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.417+0000 7f5de91c0640 1 -- 172.21.15.119:0/2195314709 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd80343f0 con 0x7f5de4143520 2023-11-08T19:28:03.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 -- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 msgr2=0x7f5de40caa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 --2- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de40caa60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009e80 tx=0x7f5dd802bf50 comp rx=0 tx=0).stop 2023-11-08T19:28:03.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 -- 172.21.15.119:0/2195314709 shutdown_connections 2023-11-08T19:28:03.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 --2- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de415d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 --2- 172.21.15.119:0/2195314709 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de40caa60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 -- 172.21.15.119:0/2195314709 >> 172.21.15.119:0/2195314709 conn(0x7f5de40b0950 msgr2=0x7f5de40b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:03.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 -- 172.21.15.119:0/2195314709 shutdown_connections 2023-11-08T19:28:03.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.418+0000 7f5dec426640 1 -- 172.21.15.119:0/2195314709 wait complete. 2023-11-08T19:28:03.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.420+0000 7f5dec426640 1 Processor -- start 2023-11-08T19:28:03.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.420+0000 7f5dec426640 1 -- start start 2023-11-08T19:28:03.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dec426640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de41393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dec426640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de4139920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dec426640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 0x7f5de406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dec426640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de40b1dc0 con 0x7f5de413da90 2023-11-08T19:28:03.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dec426640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4139e60 con 0x7f5de4143520 2023-11-08T19:28:03.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dec426640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5de4153b10 con 0x7f5de40c5200 2023-11-08T19:28:03.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dea9c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 0x7f5de406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dea9c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 0x7f5de406ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40402/0 (socket says 172.21.15.119:40402) 2023-11-08T19:28:03.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dea9c3640 1 -- 172.21.15.119:0/3177864938 learned_addr learned my addr 172.21.15.119:0/3177864938 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:03.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5dea1c2640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de41393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.422+0000 7f5de99c1640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de4139920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dea9c3640 1 -- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 msgr2=0x7f5de41393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dea9c3640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de41393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dea9c3640 1 -- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 msgr2=0x7f5de4139920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dea9c3640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de4139920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dea9c3640 1 -- 172.21.15.119:0/3177864938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd8009b30 con 0x7f5de413da90 2023-11-08T19:28:03.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dea9c3640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 0x7f5de406ca20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00e850 tx=0x7f5ddc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc009630 con 0x7f5de413da90 2023-11-08T19:28:03.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ddc00c940 con 0x7f5de413da90 2023-11-08T19:28:03.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc00ccf0 con 0x7f5de413da90 2023-11-08T19:28:03.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5de4153df0 con 0x7f5de413da90 2023-11-08T19:28:03.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.423+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5de40b3be0 con 0x7f5de413da90 2023-11-08T19:28:03.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.425+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5de40b1f00 con 0x7f5de413da90 2023-11-08T19:28:03.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.426+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ddc013470 con 0x7f5de413da90 2023-11-08T19:28:03.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.426+0000 7f5de2ffd640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5dc806ae70 0x7f5dc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.427+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f5ddc089ae0 con 0x7f5de413da90 2023-11-08T19:28:03.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.427+0000 7f5dea1c2640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5dc806ae70 0x7f5dc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.428+0000 7f5dea1c2640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5dc806ae70 0x7f5dc806d330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8035000 tx=0x7f5dd8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.431+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5de40b1f00 con 0x7f5de413da90 2023-11-08T19:28:03.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.629+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5de40b1f00 con 0x7f5de413da90 2023-11-08T19:28:03.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.637+0000 7f5de2ffd640 1 -- 172.21.15.119:0/3177864938 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5de40b1f00 con 0x7f5de413da90 2023-11-08T19:28:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.639+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5dc806ae70 msgr2=0x7f5dc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.639+0000 7f5dec426640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5dc806ae70 0x7f5dc806d330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8035000 tx=0x7f5dd8030000 comp rx=0 tx=0).stop 2023-11-08T19:28:03.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.639+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 msgr2=0x7f5de406ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.639+0000 7f5dec426640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 0x7f5de406ca20 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc00e850 tx=0x7f5ddc00ed10 comp rx=0 tx=0).stop 2023-11-08T19:28:03.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 shutdown_connections 2023-11-08T19:28:03.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5dc806ae70 0x7f5dc806d330 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5de413da90 0x7f5de406ca20 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5de4143520 0x7f5de4139920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 --2- 172.21.15.119:0/3177864938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5de40c5200 0x7f5de41393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 >> 172.21.15.119:0/3177864938 conn(0x7f5de40b0950 msgr2=0x7f5de40c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:03.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 shutdown_connections 2023-11-08T19:28:03.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.640+0000 7f5dec426640 1 -- 172.21.15.119:0/3177864938 wait complete. 2023-11-08T19:28:03.660 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation' 2023-11-08T19:28:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.869+0000 7f00d9f18640 1 Processor -- start 2023-11-08T19:28:03.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.869+0000 7f00d9f18640 1 -- start start 2023-11-08T19:28:03.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d9f18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00d4142f80 0x7f00d40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d9f18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415d9f0 0x7f00d40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d9f18640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d40c4960 con 0x7f00d406cb40 2023-11-08T19:28:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d9f18640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d4143a40 con 0x7f00d4142f80 2023-11-08T19:28:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d9f18640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415df60 con 0x7f00d415d9f0 2023-11-08T19:28:03.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00d4142f80 0x7f00d40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00d4142f80 0x7f00d40cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40416/0 (socket says 172.21.15.119:40416) 2023-11-08T19:28:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d37fe640 1 -- 172.21.15.119:0/4074574248 learned_addr learned my addr 172.21.15.119:0/4074574248 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:03.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d3fff640 1 --1- 172.21.15.119:0/4074574248 >> v1:172.21.15.119:6789/0 conn(0x7f00d406cb40 0x7f00d40c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37708/0 (socket says 172.21.15.119:37708) 2023-11-08T19:28:03.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.871+0000 7f00d2ffd640 1 --2- 172.21.15.119:0/4074574248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415d9f0 0x7f00d40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.872+0000 7f00d37fe640 1 -- 172.21.15.119:0/4074574248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415d9f0 msgr2=0x7f00d40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:03.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.872+0000 7f00d37fe640 1 --2- 172.21.15.119:0/4074574248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415d9f0 0x7f00d40c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.872+0000 7f00d37fe640 1 -- 172.21.15.119:0/4074574248 >> v1:172.21.15.119:6789/0 conn(0x7f00d406cb40 legacy=0x7f00d40c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.872+0000 7f00d27fc640 1 -- 172.21.15.119:0/4074574248 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359958792 0 0) 0x7f00d40c4960 con 0x7f00d406cb40 2023-11-08T19:28:03.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.872+0000 7f00d37fe640 1 -- 172.21.15.119:0/4074574248 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d415e240 con 0x7f00d4142f80 2023-11-08T19:28:03.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.873+0000 7f00d37fe640 1 --2- 172.21.15.119:0/4074574248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d40cc6d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009e50 tx=0x7f00c402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15da3eaaff62b00 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.873+0000 7f00d27fc640 1 -- 172.21.15.119:0/4074574248 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c402ca70 con 0x7f00d4142f80 2023-11-08T19:28:03.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.873+0000 7f00d27fc640 1 -- 172.21.15.119:0/4074574248 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c402cbd0 con 0x7f00d4142f80 2023-11-08T19:28:03.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.873+0000 7f00d27fc640 1 -- 172.21.15.119:0/4074574248 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c40343f0 con 0x7f00d4142f80 2023-11-08T19:28:03.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 -- 172.21.15.119:0/4074574248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 msgr2=0x7f00d40cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 --2- 172.21.15.119:0/4074574248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d40cc6d0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009e50 tx=0x7f00c402bed0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 -- 172.21.15.119:0/4074574248 shutdown_connections 2023-11-08T19:28:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 --2- 172.21.15.119:0/4074574248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d415d9f0 0x7f00d40c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 --2- 172.21.15.119:0/4074574248 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d40cc6d0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 -- 172.21.15.119:0/4074574248 >> 172.21.15.119:0/4074574248 conn(0x7f00d40cb610 msgr2=0x7f00d40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:03.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 -- 172.21.15.119:0/4074574248 shutdown_connections 2023-11-08T19:28:03.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.874+0000 7f00d9f18640 1 -- 172.21.15.119:0/4074574248 wait complete. 2023-11-08T19:28:03.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.876+0000 7f00d9f18640 1 Processor -- start 2023-11-08T19:28:03.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.876+0000 7f00d9f18640 1 -- start start 2023-11-08T19:28:03.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d9f18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d406cb40 0x7f00d41f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d9f18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d9f18640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d415d9f0 0x7f00d40b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d9f18640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d40c6c20 con 0x7f00d4142f80 2023-11-08T19:28:03.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d9f18640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d41f23d0 con 0x7f00d415d9f0 2023-11-08T19:28:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d9f18640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d415ccb0 con 0x7f00d406cb40 2023-11-08T19:28:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40420/0 (socket says 172.21.15.119:40420) 2023-11-08T19:28:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 -- 172.21.15.119:0/3007253120 learned_addr learned my addr 172.21.15.119:0/3007253120 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:03.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d37fe640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d406cb40 0x7f00d41f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 -- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d406cb40 msgr2=0x7f00d41f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d3fff640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d415d9f0 0x7f00d40b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d406cb40 0x7f00d41f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 -- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d415d9f0 msgr2=0x7f00d40b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d415d9f0 0x7f00d40b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.877+0000 7f00d2ffd640 1 -- 172.21.15.119:0/3007253120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c4009b30 con 0x7f00d4142f80 2023-11-08T19:28:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.878+0000 7f00d2ffd640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d415c280 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d620 tx=0x7f00bc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.878+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc0125f0 con 0x7f00d4142f80 2023-11-08T19:28:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.878+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc012750 con 0x7f00d4142f80 2023-11-08T19:28:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.878+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc012b00 con 0x7f00d4142f80 2023-11-08T19:28:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.878+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d40c6e70 con 0x7f00d4142f80 2023-11-08T19:28:03.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.878+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d40c7430 con 0x7f00d4142f80 2023-11-08T19:28:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.879+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d4074430 con 0x7f00d4142f80 2023-11-08T19:28:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.880+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00bc012c60 con 0x7f00d4142f80 2023-11-08T19:28:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.881+0000 7f00cbfff640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00b406ae70 0x7f00b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:03.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.881+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f00bc089d50 con 0x7f00d4142f80 2023-11-08T19:28:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.881+0000 7f00d37fe640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00b406ae70 0x7f00b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.882+0000 7f00d37fe640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00b406ae70 0x7f00b406d330 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f00c4035000 tx=0x7f00c4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:03.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:03.885+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00bc054af0 con 0x7f00d4142f80 2023-11-08T19:28:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.142+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]} v 0) v1 -- 0x7f00d40c4230 con 0x7f00d4142f80 2023-11-08T19:28:04.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.199+0000 7f00cbfff640 1 -- 172.21.15.119:0/3007253120 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestMonSnapsAndFsPools.test_using_pool_with_snap_fails_fs_creation"]}]=0 v202) v1 ==== 173+0+0 (secure 0 0 0) 0x7f00bc058110 con 0x7f00d4142f80 2023-11-08T19:28:04.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00b406ae70 msgr2=0x7f00b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00b406ae70 0x7f00b406d330 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f00c4035000 tx=0x7f00c4030000 comp rx=0 tx=0).stop 2023-11-08T19:28:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 msgr2=0x7f00d415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d415c280 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d620 tx=0x7f00bc00dae0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 shutdown_connections 2023-11-08T19:28:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00b406ae70 0x7f00b406d330 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00d415d9f0 0x7f00d40b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00d4142f80 0x7f00d415c280 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 --2- 172.21.15.119:0/3007253120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00d406cb40 0x7f00d41f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.202+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 >> 172.21.15.119:0/3007253120 conn(0x7f00d40cb610 msgr2=0x7f00d40ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.203+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 shutdown_connections 2023-11-08T19:28:04.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.203+0000 7f00d9f18640 1 -- 172.21.15.119:0/3007253120 wait complete. 2023-11-08T19:28:04.221 INFO:tasks.cephfs_test_runner:Test that using a pool with snaps for fs creation fails ... ok 2023-11-08T19:28:04.222 INFO:tasks.cephfs_test_runner:Starting test: test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) Check whether 'allow_new_snaps' setting works 2023-11-08T19:28:04.225 INFO:tasks.cephfs_test_runner:test_allow_new_snaps_config (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:28:04.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:04.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.438+0000 7fbe852e1640 1 Processor -- start 2023-11-08T19:28:04.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.439+0000 7fbe852e1640 1 -- start start 2023-11-08T19:28:04.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe852e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe800cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe852e1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe800c4960 con 0x7fbe8015d9f0 2023-11-08T19:28:04.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe852e1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8015df60 con 0x7fbe8006cb40 2023-11-08T19:28:04.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe852e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80143a40 con 0x7fbe80142f80 2023-11-08T19:28:04.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe767fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbe8015d9f0 0x7fbe800c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37718/0 (socket says 172.21.15.119:37718) 2023-11-08T19:28:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe767fc640 1 -- 172.21.15.119:0/586662793 learned_addr learned my addr 172.21.15.119:0/586662793 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.441+0000 7fbe7effd640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe800cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3272717213 0 0) 0x7fbe800c4960 con 0x7fbe8015d9f0 2023-11-08T19:28:04.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe6c003520 con 0x7fbe8015d9f0 2023-11-08T19:28:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3597254674 0 0) 0x7fbe6c003520 con 0x7fbe8015d9f0 2023-11-08T19:28:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 msgr2=0x7fbe800cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe800cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 >> v1:172.21.15.188:6789/0 conn(0x7fbe8006cb40 legacy=0x7fbe800c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8015e240 con 0x7fbe8015d9f0 2023-11-08T19:28:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fbe68001670 con 0x7fbe8015d9f0 2023-11-08T19:28:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 >> v1:172.21.15.119:6789/0 conn(0x7fbe8015d9f0 legacy=0x7fbe800c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0039a0 0x7fbe6c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c004d00 0x7fbe6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80143a40 con 0x7fbe6c004d00 2023-11-08T19:28:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c00bab0 con 0x7fbe6c0039a0 2023-11-08T19:28:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.442+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6c00de80 con 0x7fbe80142f80 2023-11-08T19:28:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c004d00 0x7fbe6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7effd640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0039a0 0x7fbe6c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe767fc640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 msgr2=0x7fbe6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0039a0 msgr2=0x7fbe6c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:04.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c0039a0 0x7fbe6c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/586662793 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe6c003520 con 0x7fbe6c004d00 2023-11-08T19:28:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.443+0000 7fbe767fc640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe80142f80 0x7fbe6c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:04.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.444+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c004d00 0x7fbe6c005180 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbe78001e50 tx=0x7fbe6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20df0b3823378b34 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.444+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78003b70 con 0x7fbe6c004d00 2023-11-08T19:28:04.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.444+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78003cd0 con 0x7fbe6c004d00 2023-11-08T19:28:04.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.444+0000 7fbe7e7fc640 1 -- 172.21.15.119:0/586662793 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78005310 con 0x7fbe6c004d00 2023-11-08T19:28:04.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe852e1640 1 -- 172.21.15.119:0/586662793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c004d00 msgr2=0x7fbe6c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe852e1640 1 --2- 172.21.15.119:0/586662793 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe6c004d00 0x7fbe6c005180 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fbe78001e50 tx=0x7fbe6c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/586662793 reap_dead start 2023-11-08T19:28:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe852e1640 1 -- 172.21.15.119:0/586662793 shutdown_connections 2023-11-08T19:28:04.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe852e1640 1 -- 172.21.15.119:0/586662793 >> 172.21.15.119:0/586662793 conn(0x7fbe800cb610 msgr2=0x7fbe800cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe852e1640 1 -- 172.21.15.119:0/586662793 shutdown_connections 2023-11-08T19:28:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.445+0000 7fbe852e1640 1 -- 172.21.15.119:0/586662793 wait complete. 2023-11-08T19:28:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.447+0000 7fbe852e1640 1 Processor -- start 2023-11-08T19:28:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.447+0000 7fbe852e1640 1 -- start start 2023-11-08T19:28:04.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe852e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c004d00 0x7fbe80159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe852e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe8015bd90 0x7fbe8006ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe852e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 0x7fbe801f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe852e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8006c3f0 con 0x7fbe8006bf70 2023-11-08T19:28:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe852e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe800c8840 con 0x7fbe6c004d00 2023-11-08T19:28:04.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe852e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe80159c70 con 0x7fbe8015bd90 2023-11-08T19:28:04.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 0x7fbe801f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe7f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 0x7fbe801f6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40438/0 (socket says 172.21.15.119:40438) 2023-11-08T19:28:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/4132091795 learned_addr learned my addr 172.21.15.119:0/4132091795 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.448+0000 7fbe7effd640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c004d00 0x7fbe80159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe767fc640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe8015bd90 0x7fbe8006ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe8015bd90 msgr2=0x7fbe8006ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe8015bd90 0x7fbe8006ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c004d00 msgr2=0x7fbe80159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c004d00 0x7fbe80159730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe7f7fe640 1 -- 172.21.15.119:0/4132091795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe800c7c30 con 0x7fbe8006bf70 2023-11-08T19:28:04.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe767fc640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe8015bd90 0x7fbe8006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:04.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe7f7fe640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 0x7fbe801f6450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbe78001f70 tx=0x7fbe780014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7803e070 con 0x7fbe8006bf70 2023-11-08T19:28:04.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe78003e40 con 0x7fbe8006bf70 2023-11-08T19:28:04.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe78006d90 con 0x7fbe8006bf70 2023-11-08T19:28:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe800c7f10 con 0x7fbe8006bf70 2023-11-08T19:28:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.449+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe800c84d0 con 0x7fbe8006bf70 2023-11-08T19:28:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.451+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe800c8660 con 0x7fbe8006bf70 2023-11-08T19:28:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.451+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe78007030 con 0x7fbe8006bf70 2023-11-08T19:28:04.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.452+0000 7fbe77fff640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe6006ae70 0x7fbe6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.452+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7fbe780aabe0 con 0x7fbe8006bf70 2023-11-08T19:28:04.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.453+0000 7fbe7effd640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe6006ae70 0x7fbe6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.453+0000 7fbe7effd640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe6006ae70 0x7fbe6006d330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009a10 tx=0x7fbe70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.457+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe78075980 con 0x7fbe8006bf70 2023-11-08T19:28:04.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.645+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe80074430 con 0x7fbe8006bf70 2023-11-08T19:28:04.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.647+0000 7fbe77fff640 1 -- 172.21.15.119:0/4132091795 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+18432 (secure 0 0 0) 0x7fbe78048020 con 0x7fbe8006bf70 2023-11-08T19:28:04.649 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:04.649 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":75,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:03.179982+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.188:6835/913059196":"2023-11-09T19:28:00.159709+0000","172.21.15.188:6834/913059196":"2023-11-09T19:28:00.159709+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:04.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe6006ae70 msgr2=0x7fbe6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe6006ae70 0x7fbe6006d330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009a10 tx=0x7fbe70008040 comp rx=0 tx=0).stop 2023-11-08T19:28:04.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 msgr2=0x7fbe801f6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 0x7fbe801f6450 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fbe78001f70 tx=0x7fbe780014a0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 shutdown_connections 2023-11-08T19:28:04.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe6006ae70 0x7fbe6006d330 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe8006bf70 0x7fbe801f6450 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe8015bd90 0x7fbe8006ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 --2- 172.21.15.119:0/4132091795 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe6c004d00 0x7fbe80159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 >> 172.21.15.119:0/4132091795 conn(0x7fbe800cb610 msgr2=0x7fbe80070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 shutdown_connections 2023-11-08T19:28:04.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.650+0000 7fbe852e1640 1 -- 172.21.15.119:0/4132091795 wait complete. 2023-11-08T19:28:04.671 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:28:04.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:28:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.888+0000 7f0d99990640 1 Processor -- start 2023-11-08T19:28:04.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.889+0000 7f0d99990640 1 -- start start 2023-11-08T19:28:04.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d99990640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0d9415d4a0 0x7f0d940d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d99990640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d94161870 0x7f0d940cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d99990640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d9415df50 con 0x7f0d940cfa00 2023-11-08T19:28:04.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d99990640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d940d03d0 con 0x7f0d94161870 2023-11-08T19:28:04.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d99990640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d940d1e60 con 0x7f0d9415d4a0 2023-11-08T19:28:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d937fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0d94161870 0x7f0d940cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d927fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0d940cfa00 0x7f0d94161160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37724/0 (socket says 172.21.15.119:37724) 2023-11-08T19:28:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.890+0000 7f0d927fc640 1 -- 172.21.15.119:0/2466225517 learned_addr learned my addr 172.21.15.119:0/2466225517 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:04.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.891+0000 7f0d92ffd640 1 --2- 172.21.15.119:0/2466225517 >> v2:172.21.15.188:3300/0 conn(0x7f0d9415d4a0 0x7f0d940d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.891+0000 7f0d937fe640 1 -- 172.21.15.119:0/2466225517 >> v2:172.21.15.188:3300/0 conn(0x7f0d9415d4a0 msgr2=0x7f0d940d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:04.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.891+0000 7f0d937fe640 1 --2- 172.21.15.119:0/2466225517 >> v2:172.21.15.188:3300/0 conn(0x7f0d9415d4a0 0x7f0d940d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.891+0000 7f0d937fe640 1 -- 172.21.15.119:0/2466225517 >> v1:172.21.15.119:6789/0 conn(0x7f0d940cfa00 legacy=0x7f0d94161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.891+0000 7f0d91ffb640 1 -- 172.21.15.119:0/2466225517 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586787176 0 0) 0x7f0d9415df50 con 0x7f0d940cfa00 2023-11-08T19:28:04.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.891+0000 7f0d937fe640 1 -- 172.21.15.119:0/2466225517 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d940d2830 con 0x7f0d94161870 2023-11-08T19:28:04.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.892+0000 7f0d937fe640 1 --2- 172.21.15.119:0/2466225517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d94161870 0x7f0d940cfe90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0d94072830 tx=0x7f0d8c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae94af288cb1509 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.892+0000 7f0d91ffb640 1 -- 172.21.15.119:0/2466225517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c02e070 con 0x7f0d94161870 2023-11-08T19:28:04.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.892+0000 7f0d91ffb640 1 -- 172.21.15.119:0/2466225517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c0315a0 con 0x7f0d94161870 2023-11-08T19:28:04.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.892+0000 7f0d91ffb640 1 -- 172.21.15.119:0/2466225517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c031930 con 0x7f0d94161870 2023-11-08T19:28:04.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 -- 172.21.15.119:0/2466225517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d94161870 msgr2=0x7f0d940cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 --2- 172.21.15.119:0/2466225517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d94161870 0x7f0d940cfe90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0d94072830 tx=0x7f0d8c02b840 comp rx=0 tx=0).stop 2023-11-08T19:28:04.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 -- 172.21.15.119:0/2466225517 shutdown_connections 2023-11-08T19:28:04.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 --2- 172.21.15.119:0/2466225517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d94161870 0x7f0d940cfe90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 --2- 172.21.15.119:0/2466225517 >> v2:172.21.15.188:3300/0 conn(0x7f0d9415d4a0 0x7f0d940d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 -- 172.21.15.119:0/2466225517 >> 172.21.15.119:0/2466225517 conn(0x7f0d9406a9f0 msgr2=0x7f0d9406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:04.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 -- 172.21.15.119:0/2466225517 shutdown_connections 2023-11-08T19:28:04.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.893+0000 7f0d99990640 1 -- 172.21.15.119:0/2466225517 wait complete. 2023-11-08T19:28:04.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.895+0000 7f0d99990640 1 Processor -- start 2023-11-08T19:28:04.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.895+0000 7f0d99990640 1 -- start start 2023-11-08T19:28:04.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d99990640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 0x7f0d94147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d99990640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9415d4a0 0x7f0d9414ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d99990640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d94161870 0x7f0d940c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d99990640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9414b090 con 0x7f0d940cfa00 2023-11-08T19:28:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d99990640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d94145e50 con 0x7f0d9415d4a0 2023-11-08T19:28:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d99990640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d940c8320 con 0x7f0d94161870 2023-11-08T19:28:04.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d92ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 0x7f0d94147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d92ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 0x7f0d94147e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40446/0 (socket says 172.21.15.119:40446) 2023-11-08T19:28:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.896+0000 7f0d92ffd640 1 -- 172.21.15.119:0/3865304972 learned_addr learned my addr 172.21.15.119:0/3865304972 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d927fc640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9415d4a0 0x7f0d9414ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d92ffd640 1 -- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d94161870 msgr2=0x7f0d940c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d92ffd640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d94161870 0x7f0d940c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d92ffd640 1 -- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9415d4a0 msgr2=0x7f0d9414ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:04.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d92ffd640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9415d4a0 0x7f0d9414ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:04.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d92ffd640 1 -- 172.21.15.119:0/3865304972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c009400 con 0x7f0d940cfa00 2023-11-08T19:28:04.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d92ffd640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 0x7f0d94147e10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0d84007ef0 tx=0x7f0d8400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84014620 con 0x7f0d940cfa00 2023-11-08T19:28:04.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d84014780 con 0x7f0d940cfa00 2023-11-08T19:28:04.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84014b30 con 0x7f0d940cfa00 2023-11-08T19:28:04.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d94145240 con 0x7f0d940cfa00 2023-11-08T19:28:04.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.897+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d94145800 con 0x7f0d940cfa00 2023-11-08T19:28:04.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.898+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d94074430 con 0x7f0d940cfa00 2023-11-08T19:28:04.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.899+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d84014c90 con 0x7f0d940cfa00 2023-11-08T19:28:04.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.900+0000 7f0d7bfff640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0d8006aea0 0x7f0d8006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:04.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.900+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f0d8408a200 con 0x7f0d940cfa00 2023-11-08T19:28:04.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.900+0000 7f0d927fc640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0d8006aea0 0x7f0d8006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:04.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.901+0000 7f0d927fc640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0d8006aea0 0x7f0d8006d360 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d88007fd0 tx=0x7f0d88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:04.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:04.905+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d84055040 con 0x7f0d940cfa00 2023-11-08T19:28:05.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.093+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d940d10f0 con 0x7f0d940cfa00 2023-11-08T19:28:05.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.094+0000 7f0d7bfff640 1 -- 172.21.15.119:0/3865304972 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d84058660 con 0x7f0d940cfa00 2023-11-08T19:28:05.095 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:28:05.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0d8006aea0 msgr2=0x7f0d8006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0d8006aea0 0x7f0d8006d360 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0d88007fd0 tx=0x7f0d88009040 comp rx=0 tx=0).stop 2023-11-08T19:28:05.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 msgr2=0x7f0d94147e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 0x7f0d94147e10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f0d84007ef0 tx=0x7f0d8400e4b0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 shutdown_connections 2023-11-08T19:28:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0d8006aea0 0x7f0d8006d360 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0d94161870 0x7f0d940c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0d9415d4a0 0x7f0d9414ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 --2- 172.21.15.119:0/3865304972 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0d940cfa00 0x7f0d94147e10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.096+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 >> 172.21.15.119:0/3865304972 conn(0x7f0d9406a9f0 msgr2=0x7f0d9415c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.097+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 shutdown_connections 2023-11-08T19:28:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.097+0000 7f0d99990640 1 -- 172.21.15.119:0/3865304972 wait complete. 2023-11-08T19:28:05.117 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:28:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.330+0000 7f5c8a6c9640 1 Processor -- start 2023-11-08T19:28:05.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.330+0000 7f5c8a6c9640 1 -- start start 2023-11-08T19:28:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.331+0000 7f5c8a6c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c84144580 0x7f5c8415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.331+0000 7f5c8a6c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c8415f1d0 0x7f5c840c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c8a6c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8415ea60 con 0x7f5c84144960 2023-11-08T19:28:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c8a6c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c840c9150 con 0x7f5c8415f1d0 2023-11-08T19:28:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c8a6c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840c9360 con 0x7f5c84144580 2023-11-08T19:28:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c8415f1d0 0x7f5c840c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c8415f1d0 0x7f5c840c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40460/0 (socket says 172.21.15.119:40460) 2023-11-08T19:28:05.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 -- 172.21.15.119:0/426764043 learned_addr learned my addr 172.21.15.119:0/426764043 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:05.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 -- 172.21.15.119:0/426764043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c84144580 msgr2=0x7f5c8415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:05.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 --2- 172.21.15.119:0/426764043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c84144580 0x7f5c8415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 -- 172.21.15.119:0/426764043 >> v1:172.21.15.188:6789/0 conn(0x7f5c84144960 legacy=0x7f5c840cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.332+0000 7f5c88c66640 1 -- 172.21.15.119:0/426764043 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840c9640 con 0x7f5c8415f1d0 2023-11-08T19:28:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.333+0000 7f5c88c66640 1 --2- 172.21.15.119:0/426764043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c8415f1d0 0x7f5c840c3a50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5c840c9290 tx=0x7f5c7c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684009e4e794be60 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c837fe640 1 -- 172.21.15.119:0/426764043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c02c9c0 con 0x7f5c8415f1d0 2023-11-08T19:28:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c837fe640 1 -- 172.21.15.119:0/426764043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c02cb20 con 0x7f5c8415f1d0 2023-11-08T19:28:05.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c837fe640 1 -- 172.21.15.119:0/426764043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0343f0 con 0x7f5c8415f1d0 2023-11-08T19:28:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/426764043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c8415f1d0 msgr2=0x7f5c840c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/426764043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c8415f1d0 0x7f5c840c3a50 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f5c840c9290 tx=0x7f5c7c02be20 comp rx=0 tx=0).stop 2023-11-08T19:28:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/426764043 shutdown_connections 2023-11-08T19:28:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/426764043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c8415f1d0 0x7f5c840c3a50 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/426764043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c84144580 0x7f5c8415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.334+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/426764043 >> 172.21.15.119:0/426764043 conn(0x7f5c840cb610 msgr2=0x7f5c840cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.335+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/426764043 shutdown_connections 2023-11-08T19:28:05.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.335+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/426764043 wait complete. 2023-11-08T19:28:05.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.336+0000 7f5c8a6c9640 1 Processor -- start 2023-11-08T19:28:05.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.336+0000 7f5c8a6c9640 1 -- start start 2023-11-08T19:28:05.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c8a6c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c84144580 0x7f5c8413d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c8a6c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 0x7f5c840c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c8a6c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8415f1d0 0x7f5c841f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c8a6c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840c8a70 con 0x7f5c84144960 2023-11-08T19:28:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c8a6c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840ce120 con 0x7f5c84144580 2023-11-08T19:28:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c8a6c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c840cd880 con 0x7f5c8415f1d0 2023-11-08T19:28:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.337+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 0x7f5c840c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 0x7f5c840c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40472/0 (socket says 172.21.15.119:40472) 2023-11-08T19:28:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 -- 172.21.15.119:0/3604847836 learned_addr learned my addr 172.21.15.119:0/3604847836 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 -- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8415f1d0 msgr2=0x7f5c841f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c88c66640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8415f1d0 0x7f5c841f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c83fff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c84144580 0x7f5c8413d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8415f1d0 0x7f5c841f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 -- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c84144580 msgr2=0x7f5c8413d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c84144580 0x7f5c8413d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 -- 172.21.15.119:0/3604847836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c009b30 con 0x7f5c84144960 2023-11-08T19:28:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c83fff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c84144580 0x7f5c8413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c7bfff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 0x7f5c840c8220 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c440 tx=0x7f5c6c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c005480 con 0x7f5c84144960 2023-11-08T19:28:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.338+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6c0035f0 con 0x7f5c84144960 2023-11-08T19:28:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.339+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0039c0 con 0x7f5c84144960 2023-11-08T19:28:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.339+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c840cdb60 con 0x7f5c84144960 2023-11-08T19:28:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.339+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c840c8760 con 0x7f5c84144960 2023-11-08T19:28:05.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.340+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8413d990 con 0x7f5c84144960 2023-11-08T19:28:05.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.341+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c6c003b20 con 0x7f5c84144960 2023-11-08T19:28:05.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.342+0000 7f5c817fa640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c6406ae70 0x7f5c6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.342+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f5c6c088a90 con 0x7f5c84144960 2023-11-08T19:28:05.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.342+0000 7f5c83fff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c6406ae70 0x7f5c6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.343+0000 7f5c83fff640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c6406ae70 0x7f5c6406d330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006fd0 tx=0x7f5c74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.346+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c6c0538c0 con 0x7f5c84144960 2023-11-08T19:28:05.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.535+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f5c8406cb10 con 0x7f5c84144960 2023-11-08T19:28:05.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.536+0000 7f5c817fa640 1 -- 172.21.15.119:0/3604847836 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v75) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5c6c017020 con 0x7f5c84144960 2023-11-08T19:28:05.537 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:05.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c6406ae70 msgr2=0x7f5c6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c6406ae70 0x7f5c6406d330 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c74006fd0 tx=0x7f5c74008040 comp rx=0 tx=0).stop 2023-11-08T19:28:05.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 msgr2=0x7f5c840c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 0x7f5c840c8220 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c440 tx=0x7f5c6c00c900 comp rx=0 tx=0).stop 2023-11-08T19:28:05.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 shutdown_connections 2023-11-08T19:28:05.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c6406ae70 0x7f5c6406d330 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c8415f1d0 0x7f5c841f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c84144960 0x7f5c840c8220 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 --2- 172.21.15.119:0/3604847836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c84144580 0x7f5c8413d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.538+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 >> 172.21.15.119:0/3604847836 conn(0x7f5c840cb610 msgr2=0x7f5c8415da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.539+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 shutdown_connections 2023-11-08T19:28:05.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.539+0000 7f5c8a6c9640 1 -- 172.21.15.119:0/3604847836 wait complete. 2023-11-08T19:28:05.558 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.769+0000 7ff731955640 1 Processor -- start 2023-11-08T19:28:05.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.769+0000 7ff731955640 1 -- start start 2023-11-08T19:28:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.771+0000 7ff731955640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff72c15bff0 0x7ff72c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.771+0000 7ff731955640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c15ea60 con 0x7ff72c144960 2023-11-08T19:28:05.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.771+0000 7ff731955640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72c15c460 con 0x7ff72c144580 2023-11-08T19:28:05.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.771+0000 7ff731955640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72c0c9150 con 0x7ff72c15bff0 2023-11-08T19:28:05.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff72c15bff0 0x7ff72c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff72c15bff0 0x7ff72c0c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40480/0 (socket says 172.21.15.119:40480) 2023-11-08T19:28:05.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72b7fe640 1 -- 172.21.15.119:0/1127977648 learned_addr learned my addr 172.21.15.119:0/1127977648 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72b7fe640 1 -- 172.21.15.119:0/1127977648 >> v1:172.21.15.188:6789/0 conn(0x7ff72c144580 legacy=0x7ff72c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72b7fe640 1 -- 172.21.15.119:0/1127977648 >> v1:172.21.15.119:6789/0 conn(0x7ff72c144960 legacy=0x7ff72c15f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72b7fe640 1 -- 172.21.15.119:0/1127977648 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72c15f8e0 con 0x7ff72c15bff0 2023-11-08T19:28:05.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.772+0000 7ff72a7fc640 1 -- 172.21.15.119:0/1127977648 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268772914 0 0) 0x7ff72c15ea60 con 0x7ff72c144960 2023-11-08T19:28:05.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.773+0000 7ff72b7fe640 1 --2- 172.21.15.119:0/1127977648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c15bff0 0x7ff72c0c3a50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff724009870 tx=0x7ff72402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9868446b6c94ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff72a7fc640 1 -- 172.21.15.119:0/1127977648 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724031870 con 0x7ff72c15bff0 2023-11-08T19:28:05.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff72a7fc640 1 -- 172.21.15.119:0/1127977648 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7240319d0 con 0x7ff72c15bff0 2023-11-08T19:28:05.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff72a7fc640 1 -- 172.21.15.119:0/1127977648 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724031d60 con 0x7ff72c15bff0 2023-11-08T19:28:05.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 -- 172.21.15.119:0/1127977648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c15bff0 msgr2=0x7ff72c0c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 --2- 172.21.15.119:0/1127977648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c15bff0 0x7ff72c0c3a50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff724009870 tx=0x7ff72402b910 comp rx=0 tx=0).stop 2023-11-08T19:28:05.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 -- 172.21.15.119:0/1127977648 shutdown_connections 2023-11-08T19:28:05.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 --2- 172.21.15.119:0/1127977648 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c15bff0 0x7ff72c0c3a50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 -- 172.21.15.119:0/1127977648 >> 172.21.15.119:0/1127977648 conn(0x7ff72c0cb610 msgr2=0x7ff72c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 -- 172.21.15.119:0/1127977648 shutdown_connections 2023-11-08T19:28:05.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.774+0000 7ff731955640 1 -- 172.21.15.119:0/1127977648 wait complete. 2023-11-08T19:28:05.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.776+0000 7ff731955640 1 Processor -- start 2023-11-08T19:28:05.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.776+0000 7ff731955640 1 -- start start 2023-11-08T19:28:05.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff731955640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 0x7ff72c0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff731955640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72c144960 0x7ff72c15aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff731955640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72c15bff0 0x7ff72c0b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff731955640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c1f0d30 con 0x7ff72c144580 2023-11-08T19:28:05.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff731955640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c7a00 con 0x7ff72c144960 2023-11-08T19:28:05.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff731955640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72c0c7ce0 con 0x7ff72c15bff0 2023-11-08T19:28:05.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff72affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 0x7ff72c0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff72affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 0x7ff72c0c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40484/0 (socket says 172.21.15.119:40484) 2023-11-08T19:28:05.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.777+0000 7ff72affd640 1 -- 172.21.15.119:0/1791381971 learned_addr learned my addr 172.21.15.119:0/1791381971 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:05.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff7227fc640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72c144960 0x7ff72c15aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72b7fe640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72c15bff0 0x7ff72c0b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72affd640 1 -- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72c15bff0 msgr2=0x7ff72c0b1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72affd640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72c15bff0 0x7ff72c0b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72affd640 1 -- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72c144960 msgr2=0x7ff72c15aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72affd640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72c144960 0x7ff72c15aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72affd640 1 -- 172.21.15.119:0/1791381971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240094d0 con 0x7ff72c144580 2023-11-08T19:28:05.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff7227fc640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72c144960 0x7ff72c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:05.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff72affd640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 0x7ff72c0c7420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00c7c0 tx=0x7ff71c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c00aa50 con 0x7ff72c144580 2023-11-08T19:28:05.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c00abb0 con 0x7ff72c144580 2023-11-08T19:28:05.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c012420 con 0x7ff72c144580 2023-11-08T19:28:05.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.778+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72c0c7fc0 con 0x7ff72c144580 2023-11-08T19:28:05.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.779+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff72c0b2380 con 0x7ff72c144580 2023-11-08T19:28:05.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.780+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72c06cb10 con 0x7ff72c144580 2023-11-08T19:28:05.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.781+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff71c012580 con 0x7ff72c144580 2023-11-08T19:28:05.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.782+0000 7ff728ff9640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff70c06ae70 0x7ff70c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:05.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.782+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7ff71c089830 con 0x7ff72c144580 2023-11-08T19:28:05.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.782+0000 7ff7227fc640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff70c06ae70 0x7ff70c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:05.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.783+0000 7ff7227fc640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff70c06ae70 0x7ff70c06d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0b47d0 tx=0x7ff714009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:05.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.786+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff71c054660 con 0x7ff72c144580 2023-11-08T19:28:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.973+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7ff72c0718c0 con 0x7ff72c144580 2023-11-08T19:28:05.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.974+0000 7ff728ff9640 1 -- 172.21.15.119:0/1791381971 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v75) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff71c057c80 con 0x7ff72c144580 2023-11-08T19:28:05.976 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.976+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff70c06ae70 msgr2=0x7ff70c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.976+0000 7ff731955640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff70c06ae70 0x7ff70c06d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0b47d0 tx=0x7ff714009040 comp rx=0 tx=0).stop 2023-11-08T19:28:05.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.976+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 msgr2=0x7ff72c0c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.976+0000 7ff731955640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 0x7ff72c0c7420 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7ff71c00c7c0 tx=0x7ff71c00cc80 comp rx=0 tx=0).stop 2023-11-08T19:28:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 shutdown_connections 2023-11-08T19:28:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff70c06ae70 0x7ff70c06d330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff72c15bff0 0x7ff72c0b1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff72c144960 0x7ff72c15aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 --2- 172.21.15.119:0/1791381971 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff72c144580 0x7ff72c0c7420 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 >> 172.21.15.119:0/1791381971 conn(0x7ff72c0cb610 msgr2=0x7ff72c0cad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 shutdown_connections 2023-11-08T19:28:05.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:05.977+0000 7ff731955640 1 -- 172.21.15.119:0/1791381971 wait complete. 2023-11-08T19:28:05.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:06.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.209+0000 7f5443755640 1 Processor -- start 2023-11-08T19:28:06.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.209+0000 7f5443755640 1 -- start start 2023-11-08T19:28:06.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f5443755640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f543c06d250 0x7f543c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f5443755640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c15e440 con 0x7f543c143600 2023-11-08T19:28:06.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f5443755640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f543c15e720 con 0x7f543c15d9f0 2023-11-08T19:28:06.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f5443755640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f543c0c4f50 con 0x7f543c06d250 2023-11-08T19:28:06.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f54414f1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f543c143600 0x7f543c0c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37744/0 (socket says 172.21.15.119:37744) 2023-11-08T19:28:06.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f54414f1640 1 -- 172.21.15.119:0/2301840130 learned_addr learned my addr 172.21.15.119:0/2301840130 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:06.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.212+0000 7f5441cf2640 1 --2- 172.21.15.119:0/2301840130 >> v2:172.21.15.188:3300/0 conn(0x7f543c06d250 0x7f543c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.213+0000 7f5441cf2640 1 -- 172.21.15.119:0/2301840130 >> v1:172.21.15.188:6789/0 conn(0x7f543c15d9f0 legacy=0x7f543c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.213+0000 7f5441cf2640 1 -- 172.21.15.119:0/2301840130 >> v1:172.21.15.119:6789/0 conn(0x7f543c143600 legacy=0x7f543c0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.213+0000 7f5441cf2640 1 -- 172.21.15.119:0/2301840130 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f543c0c4b70 con 0x7f543c06d250 2023-11-08T19:28:06.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.214+0000 7f5441cf2640 1 --2- 172.21.15.119:0/2301840130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c06d250 0x7f543c15de60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5434009d90 tx=0x7f543402e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f793cc17322cd562 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.217+0000 7f5433fff640 1 -- 172.21.15.119:0/2301840130 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543402ee70 con 0x7f543c06d250 2023-11-08T19:28:06.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.217+0000 7f5433fff640 1 -- 172.21.15.119:0/2301840130 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434036440 con 0x7f543c06d250 2023-11-08T19:28:06.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.217+0000 7f5433fff640 1 -- 172.21.15.119:0/2301840130 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54340367d0 con 0x7f543c06d250 2023-11-08T19:28:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 -- 172.21.15.119:0/2301840130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c06d250 msgr2=0x7f543c15de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 --2- 172.21.15.119:0/2301840130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c06d250 0x7f543c15de60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5434009d90 tx=0x7f543402e0c0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 -- 172.21.15.119:0/2301840130 shutdown_connections 2023-11-08T19:28:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 --2- 172.21.15.119:0/2301840130 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c06d250 0x7f543c15de60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 -- 172.21.15.119:0/2301840130 >> 172.21.15.119:0/2301840130 conn(0x7f543c0cb610 msgr2=0x7f543c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:06.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 -- 172.21.15.119:0/2301840130 shutdown_connections 2023-11-08T19:28:06.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.218+0000 7f5443755640 1 -- 172.21.15.119:0/2301840130 wait complete. 2023-11-08T19:28:06.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.220+0000 7f5443755640 1 Processor -- start 2023-11-08T19:28:06.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.220+0000 7f5443755640 1 -- start start 2023-11-08T19:28:06.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5443755640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f543c06d250 0x7f543c0b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5443755640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 0x7f543c13f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5443755640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c15d9f0 0x7f543c14dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5443755640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c13fb90 con 0x7f543c143600 2023-11-08T19:28:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5443755640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c0b43a0 con 0x7f543c15d9f0 2023-11-08T19:28:06.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5443755640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f543c14e190 con 0x7f543c06d250 2023-11-08T19:28:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5440cf0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 0x7f543c13f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5440cf0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 0x7f543c13f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40494/0 (socket says 172.21.15.119:40494) 2023-11-08T19:28:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.221+0000 7f5440cf0640 1 -- 172.21.15.119:0/4252923892 learned_addr learned my addr 172.21.15.119:0/4252923892 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:06.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f54414f1640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f543c06d250 0x7f543c0b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5441cf2640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c15d9f0 0x7f543c14dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5440cf0640 1 -- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f543c06d250 msgr2=0x7f543c0b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5440cf0640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f543c06d250 0x7f543c0b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5440cf0640 1 -- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c15d9f0 msgr2=0x7f543c14dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5440cf0640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c15d9f0 0x7f543c14dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5440cf0640 1 -- 172.21.15.119:0/4252923892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5434009a70 con 0x7f543c143600 2023-11-08T19:28:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5441cf2640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c15d9f0 0x7f543c14dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5440cf0640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 0x7f543c13f650 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f542c00c430 tx=0x7f542c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c0052d0 con 0x7f543c143600 2023-11-08T19:28:06.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f542c005430 con 0x7f543c143600 2023-11-08T19:28:06.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c0b3790 con 0x7f543c143600 2023-11-08T19:28:06.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f542c003830 con 0x7f543c143600 2023-11-08T19:28:06.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.222+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c0b3d50 con 0x7f543c143600 2023-11-08T19:28:06.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.224+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c0c83d0 con 0x7f543c143600 2023-11-08T19:28:06.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.225+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f542c003990 con 0x7f543c143600 2023-11-08T19:28:06.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.226+0000 7f5431ffb640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f542006ae70 0x7f542006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.226+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f542c088e70 con 0x7f543c143600 2023-11-08T19:28:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.226+0000 7f54414f1640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f542006ae70 0x7f542006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.227+0000 7f54414f1640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f542006ae70 0x7f542006d330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5438005880 tx=0x7f5438009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.230+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f542c053c10 con 0x7f543c143600 2023-11-08T19:28:06.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.421+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f543c0c4230 con 0x7f543c143600 2023-11-08T19:28:06.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.422+0000 7f5431ffb640 1 -- 172.21.15.119:0/4252923892 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v75) v1 ==== 107+0+11 (secure 0 0 0) 0x7f542c0170b0 con 0x7f543c143600 2023-11-08T19:28:06.424 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:06.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f542006ae70 msgr2=0x7f542006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f542006ae70 0x7f542006d330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5438005880 tx=0x7f5438009040 comp rx=0 tx=0).stop 2023-11-08T19:28:06.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 msgr2=0x7f543c13f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 0x7f543c13f650 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f542c00c430 tx=0x7f542c00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 shutdown_connections 2023-11-08T19:28:06.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f542006ae70 0x7f542006d330 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f543c15d9f0 0x7f543c14dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f543c143600 0x7f543c13f650 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 --2- 172.21.15.119:0/4252923892 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f543c06d250 0x7f543c0b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 >> 172.21.15.119:0/4252923892 conn(0x7f543c0cb610 msgr2=0x7f543c141450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:06.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 shutdown_connections 2023-11-08T19:28:06.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.425+0000 7f5443755640 1 -- 172.21.15.119:0/4252923892 wait complete. 2023-11-08T19:28:06.444 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2023-11-08T19:28:06.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.652+0000 7f5af1eea640 1 Processor -- start 2023-11-08T19:28:06.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.652+0000 7f5af1eea640 1 -- start start 2023-11-08T19:28:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af1eea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af1eea640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5aec0710a0 0x7f5aec0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af1eea640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aec06cff0 con 0x7f5aec15b0d0 2023-11-08T19:28:06.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af1eea640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aec06d2d0 con 0x7f5aec0710a0 2023-11-08T19:28:06.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af1eea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec06bd20 con 0x7f5aec0ad7d0 2023-11-08T19:28:06.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af16e9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5aec0710a0 0x7f5aec0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af16e9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5aec0710a0 0x7f5aec0c98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40510/0 (socket says 172.21.15.119:40510) 2023-11-08T19:28:06.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5af16e9640 1 -- 172.21.15.119:0/2061750070 learned_addr learned my addr 172.21.15.119:0/2061750070 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:06.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.654+0000 7f5aebfff640 1 --2- 172.21.15.119:0/2061750070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.655+0000 7f5af16e9640 1 -- 172.21.15.119:0/2061750070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 msgr2=0x7f5aec070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.655+0000 7f5af16e9640 1 --2- 172.21.15.119:0/2061750070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.655+0000 7f5af16e9640 1 -- 172.21.15.119:0/2061750070 >> v1:172.21.15.119:6789/0 conn(0x7f5aec15b0d0 legacy=0x7f5aec15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.655+0000 7f5af16e9640 1 -- 172.21.15.119:0/2061750070 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aec06d760 con 0x7f5aec0710a0 2023-11-08T19:28:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.656+0000 7f5af16e9640 1 --2- 172.21.15.119:0/2061750070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec0710a0 0x7f5aec0c98c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009e50 tx=0x7f5ae402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52be93f370f5fd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.656+0000 7f5aeb7fe640 1 -- 172.21.15.119:0/2061750070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae402ca80 con 0x7f5aec0710a0 2023-11-08T19:28:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.656+0000 7f5aeb7fe640 1 -- 172.21.15.119:0/2061750070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae402cbe0 con 0x7f5aec0710a0 2023-11-08T19:28:06.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.656+0000 7f5aeb7fe640 1 -- 172.21.15.119:0/2061750070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae40343f0 con 0x7f5aec0710a0 2023-11-08T19:28:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 -- 172.21.15.119:0/2061750070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec0710a0 msgr2=0x7f5aec0c98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 --2- 172.21.15.119:0/2061750070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec0710a0 0x7f5aec0c98c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009e50 tx=0x7f5ae402bee0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 -- 172.21.15.119:0/2061750070 shutdown_connections 2023-11-08T19:28:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 --2- 172.21.15.119:0/2061750070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec0710a0 0x7f5aec0c98c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 --2- 172.21.15.119:0/2061750070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 -- 172.21.15.119:0/2061750070 >> 172.21.15.119:0/2061750070 conn(0x7f5aec0b0560 msgr2=0x7f5aec0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:06.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 -- 172.21.15.119:0/2061750070 shutdown_connections 2023-11-08T19:28:06.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.657+0000 7f5af1eea640 1 -- 172.21.15.119:0/2061750070 wait complete. 2023-11-08T19:28:06.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.659+0000 7f5af1eea640 1 Processor -- start 2023-11-08T19:28:06.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.659+0000 7f5af1eea640 1 -- start start 2023-11-08T19:28:06.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af1eea640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aec0710a0 0x7f5aec1e03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af1eea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec0aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af1eea640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 0x7f5aec0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af1eea640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec1f2e30 con 0x7f5aec15b0d0 2023-11-08T19:28:06.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af1eea640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0af4e0 con 0x7f5aec0710a0 2023-11-08T19:28:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af1eea640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec1f30f0 con 0x7f5aec0ad7d0 2023-11-08T19:28:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af16e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 0x7f5aec0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af16e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 0x7f5aec0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40518/0 (socket says 172.21.15.119:40518) 2023-11-08T19:28:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af16e9640 1 -- 172.21.15.119:0/846860999 learned_addr learned my addr 172.21.15.119:0/846860999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:06.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5aebfff640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec0aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af16e9640 1 -- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 msgr2=0x7f5aec0aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af16e9640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec0aefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af0ee8640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aec0710a0 0x7f5aec1e03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.660+0000 7f5af16e9640 1 -- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aec0710a0 msgr2=0x7f5aec1e03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:06.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5af16e9640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aec0710a0 0x7f5aec1e03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5af16e9640 1 -- 172.21.15.119:0/846860999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae4009b30 con 0x7f5aec15b0d0 2023-11-08T19:28:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5af0ee8640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aec0710a0 0x7f5aec1e03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5af16e9640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 0x7f5aec0ac8d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009e50 tx=0x7f5ae4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae4009510 con 0x7f5aec15b0d0 2023-11-08T19:28:06.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae403e450 con 0x7f5aec15b0d0 2023-11-08T19:28:06.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec1f33d0 con 0x7f5aec15b0d0 2023-11-08T19:28:06.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae403e800 con 0x7f5aec15b0d0 2023-11-08T19:28:06.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.661+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec1f3990 con 0x7f5aec15b0d0 2023-11-08T19:28:06.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.662+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec1f3b50 con 0x7f5aec15b0d0 2023-11-08T19:28:06.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.663+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ae402cd50 con 0x7f5aec15b0d0 2023-11-08T19:28:06.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.664+0000 7f5ae97fa640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ad006aea0 0x7f5ad006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:06.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.664+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f5ae40ac350 con 0x7f5aec15b0d0 2023-11-08T19:28:06.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.665+0000 7f5af0ee8640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ad006aea0 0x7f5ad006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:06.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.665+0000 7f5af0ee8640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ad006aea0 0x7f5ad006d360 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0ad4e0 tx=0x7f5ae0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:06.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.669+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ae4077180 con 0x7f5aec15b0d0 2023-11-08T19:28:06.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:06.912+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) v1 -- 0x7f5aec0adc30 con 0x7f5aec15b0d0 2023-11-08T19:28:07.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.222+0000 7f5ae97fa640 1 -- 172.21.15.119:0/846860999 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v205) v1 ==== 151+0+0 (secure 0 0 0) 0x7f5aec0adc30 con 0x7f5aec15b0d0 2023-11-08T19:28:07.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ad006aea0 msgr2=0x7f5ad006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ad006aea0 0x7f5ad006d360 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0ad4e0 tx=0x7f5ae0009040 comp rx=0 tx=0).stop 2023-11-08T19:28:07.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 msgr2=0x7f5aec0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 0x7f5aec0ac8d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009e50 tx=0x7f5ae4004a40 comp rx=0 tx=0).stop 2023-11-08T19:28:07.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 shutdown_connections 2023-11-08T19:28:07.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5ad006aea0 0x7f5ad006d360 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5aec15b0d0 0x7f5aec0ac8d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5aec0ad7d0 0x7f5aec0aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 --2- 172.21.15.119:0/846860999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5aec0710a0 0x7f5aec1e03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.225+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 >> 172.21.15.119:0/846860999 conn(0x7f5aec0b0560 msgr2=0x7f5aec0c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.226+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 shutdown_connections 2023-11-08T19:28:07.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.226+0000 7f5af1eea640 1 -- 172.21.15.119:0/846860999 wait complete. 2023-11-08T19:28:07.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:28:07.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.465+0000 7f43a1fd0640 1 Processor -- start 2023-11-08T19:28:07.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.465+0000 7f43a1fd0640 1 -- start start 2023-11-08T19:28:07.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.466+0000 7f43a1fd0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f439c15deb0 0x7f439c0cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.466+0000 7f43a1fd0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f439c15e290 0x7f439c15e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.466+0000 7f43a1fd0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c071720 con 0x7f439c15a8c0 2023-11-08T19:28:07.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f43a1fd0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0cf5e0 con 0x7f439c15e290 2023-11-08T19:28:07.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f43a1fd0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0cf8c0 con 0x7f439c15deb0 2023-11-08T19:28:07.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f439c15e290 0x7f439c15e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f439c15e290 0x7f439c15e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40524/0 (socket says 172.21.15.119:40524) 2023-11-08T19:28:07.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 -- 172.21.15.119:0/3804930462 learned_addr learned my addr 172.21.15.119:0/3804930462 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:07.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f43a17cf640 1 --1- 172.21.15.119:0/3804930462 >> v1:172.21.15.119:6789/0 conn(0x7f439c15a8c0 0x7f439c15acf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:37770/0 (socket says 172.21.15.119:37770) 2023-11-08T19:28:07.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f43a0fce640 1 --2- 172.21.15.119:0/3804930462 >> v2:172.21.15.188:3300/0 conn(0x7f439c15deb0 0x7f439c0cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 -- 172.21.15.119:0/3804930462 >> v2:172.21.15.188:3300/0 conn(0x7f439c15deb0 msgr2=0x7f439c0cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 --2- 172.21.15.119:0/3804930462 >> v2:172.21.15.188:3300/0 conn(0x7f439c15deb0 0x7f439c0cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f43937fe640 1 -- 172.21.15.119:0/3804930462 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933838693 0 0) 0x7f439c071720 con 0x7f439c15a8c0 2023-11-08T19:28:07.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 -- 172.21.15.119:0/3804930462 >> v1:172.21.15.119:6789/0 conn(0x7f439c15a8c0 legacy=0x7f439c15acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.467+0000 7f4393fff640 1 -- 172.21.15.119:0/3804930462 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0710d0 con 0x7f439c15e290 2023-11-08T19:28:07.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.469+0000 7f4393fff640 1 --2- 172.21.15.119:0/3804930462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15e290 0x7f439c15e700 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f439c0cc600 tx=0x7f438c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11c2cd735171d1c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.469+0000 7f43937fe640 1 -- 172.21.15.119:0/3804930462 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c031870 con 0x7f439c15e290 2023-11-08T19:28:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.469+0000 7f43937fe640 1 -- 172.21.15.119:0/3804930462 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c0319d0 con 0x7f439c15e290 2023-11-08T19:28:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.469+0000 7f43937fe640 1 -- 172.21.15.119:0/3804930462 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c031d60 con 0x7f439c15e290 2023-11-08T19:28:07.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.469+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3804930462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15e290 msgr2=0x7f439c15e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.469+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3804930462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15e290 0x7f439c15e700 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f439c0cc600 tx=0x7f438c02b910 comp rx=0 tx=0).stop 2023-11-08T19:28:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.470+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3804930462 shutdown_connections 2023-11-08T19:28:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.470+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3804930462 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15e290 0x7f439c15e700 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.470+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3804930462 >> v2:172.21.15.188:3300/0 conn(0x7f439c15deb0 0x7f439c0cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.470+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3804930462 >> 172.21.15.119:0/3804930462 conn(0x7f439c0ceac0 msgr2=0x7f439c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.470+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3804930462 shutdown_connections 2023-11-08T19:28:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.470+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3804930462 wait complete. 2023-11-08T19:28:07.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.472+0000 7f43a1fd0640 1 Processor -- start 2023-11-08T19:28:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.472+0000 7f43a1fd0640 1 -- start start 2023-11-08T19:28:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a1fd0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c15a8c0 0x7f439c158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a1fd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 0x7f439c0c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a1fd0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c15e290 0x7f439c1f31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a1fd0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c1f29f0 con 0x7f439c15deb0 2023-11-08T19:28:07.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a1fd0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c158ca0 con 0x7f439c15a8c0 2023-11-08T19:28:07.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a1fd0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c0cb930 con 0x7f439c15e290 2023-11-08T19:28:07.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 0x7f439c0c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 0x7f439c0c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40538/0 (socket says 172.21.15.119:40538) 2023-11-08T19:28:07.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 -- 172.21.15.119:0/3552109262 learned_addr learned my addr 172.21.15.119:0/3552109262 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:07.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f43a0fce640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c15a8c0 0x7f439c158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 -- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c15e290 msgr2=0x7f439c1f31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c15e290 0x7f439c1f31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 -- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c15a8c0 msgr2=0x7f439c158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c15a8c0 0x7f439c158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.473+0000 7f4393fff640 1 -- 172.21.15.119:0/3552109262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0094d0 con 0x7f439c15deb0 2023-11-08T19:28:07.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.474+0000 7f4393fff640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 0x7f439c0c84b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f438c000c00 tx=0x7f438c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.474+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c004230 con 0x7f439c15deb0 2023-11-08T19:28:07.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.474+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c004390 con 0x7f439c15deb0 2023-11-08T19:28:07.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.474+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c0cbc10 con 0x7f439c15deb0 2023-11-08T19:28:07.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.474+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c03ba60 con 0x7f439c15deb0 2023-11-08T19:28:07.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.474+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c1f2cb0 con 0x7f439c15deb0 2023-11-08T19:28:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.475+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c04ece0 con 0x7f439c15deb0 2023-11-08T19:28:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.476+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f438c043070 con 0x7f439c15deb0 2023-11-08T19:28:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.477+0000 7f43917fa640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f438806ae70 0x7f438806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:07.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.477+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f438c02d040 con 0x7f439c15deb0 2023-11-08T19:28:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.477+0000 7f43a0fce640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f438806ae70 0x7f438806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.478+0000 7f43a0fce640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f438806ae70 0x7f438806d330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4398006fd0 tx=0x7f4398008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:07.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.481+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f438c076870 con 0x7f439c15deb0 2023-11-08T19:28:07.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.670+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f439c0b4a40 con 0x7f439c15deb0 2023-11-08T19:28:07.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.676+0000 7f43917fa640 1 -- 172.21.15.119:0/3552109262 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v17)=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f438c0350c0 con 0x7f439c15deb0 2023-11-08T19:28:07.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f438806ae70 msgr2=0x7f438806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f438806ae70 0x7f438806d330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4398006fd0 tx=0x7f4398008040 comp rx=0 tx=0).stop 2023-11-08T19:28:07.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 msgr2=0x7f439c0c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:07.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 0x7f439c0c84b0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f438c000c00 tx=0x7f438c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 shutdown_connections 2023-11-08T19:28:07.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f438806ae70 0x7f438806d330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f439c15e290 0x7f439c1f31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f439c15deb0 0x7f439c0c84b0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 --2- 172.21.15.119:0/3552109262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f439c15a8c0 0x7f439c158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:07.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.678+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 >> 172.21.15.119:0/3552109262 conn(0x7f439c0ceac0 msgr2=0x7f439c072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:07.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.679+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 shutdown_connections 2023-11-08T19:28:07.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:07.679+0000 7f43a1fd0640 1 -- 172.21.15.119:0/3552109262 wait complete. 2023-11-08T19:28:07.700 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:28:07.701 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:28:07.816 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:28:07.939 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:28:07.939 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:07.940 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:08.000 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:28:08.001 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:28:08.001 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:08.064 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:28:08.065 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:08.065 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:08.127 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:28:08.128 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:28:08.128 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:08.188 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:28:08.188 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:28:08.189 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:08.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.411+0000 7f1183355640 1 Processor -- start 2023-11-08T19:28:08.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.411+0000 7f1183355640 1 -- start start 2023-11-08T19:28:08.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f1183355640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f117c1447a0 0x7f117c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f1183355640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f117c0cad30 0x7f117c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f1183355640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c06a120 0x7f117c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f1183355640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c06aa90 con 0x7f117c1447a0 2023-11-08T19:28:08.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f1183355640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c0c3a50 con 0x7f117c0cad30 2023-11-08T19:28:08.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f1183355640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c144b80 con 0x7f117c06a120 2023-11-08T19:28:08.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f117c1447a0 0x7f117c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f117c1447a0 0x7f117c15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40542/0 (socket says 172.21.15.119:40542) 2023-11-08T19:28:08.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 -- 172.21.15.119:0/2156681410 learned_addr learned my addr 172.21.15.119:0/2156681410 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 -- 172.21.15.119:0/2156681410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c06a120 msgr2=0x7f117c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 --2- 172.21.15.119:0/2156681410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c06a120 0x7f117c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 -- 172.21.15.119:0/2156681410 >> v2:172.21.15.188:3300/0 conn(0x7f117c0cad30 msgr2=0x7f117c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 --2- 172.21.15.119:0/2156681410 >> v2:172.21.15.188:3300/0 conn(0x7f117c0cad30 0x7f117c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.413+0000 7f11810f1640 1 -- 172.21.15.119:0/2156681410 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0c9150 con 0x7f117c1447a0 2023-11-08T19:28:08.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.414+0000 7f11810f1640 1 --2- 172.21.15.119:0/2156681410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c1447a0 0x7f117c15f4c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f117c06df20 tx=0x7f117802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeceaf2bcb65edd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.415+0000 7f1173fff640 1 -- 172.21.15.119:0/2156681410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178031880 con 0x7f117c1447a0 2023-11-08T19:28:08.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.415+0000 7f1173fff640 1 -- 172.21.15.119:0/2156681410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11780319e0 con 0x7f117c1447a0 2023-11-08T19:28:08.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.415+0000 7f1173fff640 1 -- 172.21.15.119:0/2156681410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1178031d70 con 0x7f117c1447a0 2023-11-08T19:28:08.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.415+0000 7f1183355640 1 -- 172.21.15.119:0/2156681410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c1447a0 msgr2=0x7f117c15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.415+0000 7f1183355640 1 --2- 172.21.15.119:0/2156681410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c1447a0 0x7f117c15f4c0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f117c06df20 tx=0x7f117802b920 comp rx=0 tx=0).stop 2023-11-08T19:28:08.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 -- 172.21.15.119:0/2156681410 shutdown_connections 2023-11-08T19:28:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 --2- 172.21.15.119:0/2156681410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c06a120 0x7f117c06a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 --2- 172.21.15.119:0/2156681410 >> v2:172.21.15.188:3300/0 conn(0x7f117c0cad30 0x7f117c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 --2- 172.21.15.119:0/2156681410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c1447a0 0x7f117c15f4c0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 -- 172.21.15.119:0/2156681410 >> 172.21.15.119:0/2156681410 conn(0x7f117c0cb610 msgr2=0x7f117c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 -- 172.21.15.119:0/2156681410 shutdown_connections 2023-11-08T19:28:08.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.416+0000 7f1183355640 1 -- 172.21.15.119:0/2156681410 wait complete. 2023-11-08T19:28:08.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.418+0000 7f1183355640 1 Processor -- start 2023-11-08T19:28:08.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.418+0000 7f1183355640 1 -- start start 2023-11-08T19:28:08.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.419+0000 7f1183355640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f117c06a120 0x7f117c13fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.419+0000 7f1183355640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 0x7f117c141cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.419+0000 7f1183355640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c1447a0 0x7f117c0b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.419+0000 7f1183355640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c6630 con 0x7f117c0cad30 2023-11-08T19:28:08.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.419+0000 7f1183355640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b2fe0 con 0x7f117c06a120 2023-11-08T19:28:08.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.419+0000 7f1183355640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0c5d00 con 0x7f117c1447a0 2023-11-08T19:28:08.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 0x7f117c141cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 0x7f117c141cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40544/0 (socket says 172.21.15.119:40544) 2023-11-08T19:28:08.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 -- 172.21.15.119:0/3849092365 learned_addr learned my addr 172.21.15.119:0/3849092365 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11810f1640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f117c06a120 0x7f117c13fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 -- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c1447a0 msgr2=0x7f117c0b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c1447a0 0x7f117c0b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 -- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f117c06a120 msgr2=0x7f117c13fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f117c06a120 0x7f117c13fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 -- 172.21.15.119:0/3849092365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11780094d0 con 0x7f117c0cad30 2023-11-08T19:28:08.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.420+0000 7f11808f0640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 0x7f117c141cb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b410 tx=0x7f116c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.421+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011830 con 0x7f117c0cad30 2023-11-08T19:28:08.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.421+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c0119b0 con 0x7f117c0cad30 2023-11-08T19:28:08.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.421+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c0c5fe0 con 0x7f117c0cad30 2023-11-08T19:28:08.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.421+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c011d60 con 0x7f117c0cad30 2023-11-08T19:28:08.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.421+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c140300 con 0x7f117c0cad30 2023-11-08T19:28:08.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.422+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c0c67a0 con 0x7f117c0cad30 2023-11-08T19:28:08.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.423+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f116c01a400 con 0x7f117c0cad30 2023-11-08T19:28:08.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.424+0000 7f1171ffb640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f116006ae70 0x7f116006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.424+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f116c089990 con 0x7f117c0cad30 2023-11-08T19:28:08.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.424+0000 7f11810f1640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f116006ae70 0x7f116006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.425+0000 7f11810f1640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f116006ae70 0x7f116006d330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1178036000 tx=0x7f117803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.428+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f116c054860 con 0x7f117c0cad30 2023-11-08T19:28:08.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.655+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f117c074430 con 0x7f117c0cad30 2023-11-08T19:28:08.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.658+0000 7f1171ffb640 1 -- 172.21.15.119:0/3849092365 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f116c015020 con 0x7f117c0cad30 2023-11-08T19:28:08.659 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:08.660 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":37,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4895,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4058539247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4058539247},{"type":"v1","addr":"172.21.15.188:6835","nonce":4058539247}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6135,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6835/2235632518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2235632518},{"type":"v1","addr":"172.21.15.119:6835","nonce":2235632518}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6138,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1496375796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6836","nonce":1496375796},{"type":"v1","addr":"172.21.15.119:6837","nonce":1496375796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":6144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/2726208358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":2726208358},{"type":"v1","addr":"172.21.15.188:6833","nonce":2726208358}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[]} 2023-11-08T19:28:08.660 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 37 2023-11-08T19:28:08.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f116006ae70 msgr2=0x7f116006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f116006ae70 0x7f116006d330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1178036000 tx=0x7f117803c040 comp rx=0 tx=0).stop 2023-11-08T19:28:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 msgr2=0x7f117c141cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 0x7f117c141cb0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f116c00b410 tx=0x7f116c00b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 shutdown_connections 2023-11-08T19:28:08.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f116006ae70 0x7f116006d330 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f117c1447a0 0x7f117c0b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f117c0cad30 0x7f117c141cb0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 --2- 172.21.15.119:0/3849092365 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f117c06a120 0x7f117c13fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 >> 172.21.15.119:0/3849092365 conn(0x7f117c0cb610 msgr2=0x7f117c1424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 shutdown_connections 2023-11-08T19:28:08.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.661+0000 7f1183355640 1 -- 172.21.15.119:0/3849092365 wait complete. 2023-11-08T19:28:08.681 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:28:08.682 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:28:08.682 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:28:08.682 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:08.683 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:28:08.683 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:28:08.683 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:28:08.683 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:08.684 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:28:08.684 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:28:08.684 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:28:08.684 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:08.685 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:28:08.685 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:28:08.685 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:28:08.686 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:08.706 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:28:08.707 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:28:08.708 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:28:08.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:28:08.710 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:28:08.710 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:28:08.711 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:28:08.712 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:28:08.755 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:28:08.755 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:08.761 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:28:08.761 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:28:08.762 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:08.762 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:08.766 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:28:08.767 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:08.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:28:08.907 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:28:08.910 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:28:08.926 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:28:08.929 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:28:08.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.951+0000 7f3120426640 1 Processor -- start 2023-11-08T19:28:08.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.951+0000 7f3120426640 1 -- start start 2023-11-08T19:28:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f3120426640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f31180b3aa0 0x7f31180cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f3120426640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31181444a0 con 0x7f311806a360 2023-11-08T19:28:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f3120426640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31180b1a00 con 0x7f31181440c0 2023-11-08T19:28:08.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f3120426640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31180b22a0 con 0x7f31180b3aa0 2023-11-08T19:28:08.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311e9c3640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f311806a360 0x7f31180b2730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54502/0 (socket says 172.21.15.119:54502) 2023-11-08T19:28:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311e9c3640 1 -- 172.21.15.119:0/4170072710 learned_addr learned my addr 172.21.15.119:0/4170072710 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311d9c1640 1 --2- 172.21.15.119:0/4170072710 >> v2:172.21.15.119:3300/0 conn(0x7f31180b3aa0 0x7f31180cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311d9c1640 1 -- 172.21.15.119:0/4170072710 >> v1:172.21.15.188:6789/0 conn(0x7f31181440c0 legacy=0x7f311815b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311d1c0640 1 -- 172.21.15.119:0/4170072710 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715495807 0 0) 0x7f3110001490 con 0x7f311806a360 2023-11-08T19:28:08.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311d9c1640 1 -- 172.21.15.119:0/4170072710 >> v1:172.21.15.119:6789/0 conn(0x7f311806a360 legacy=0x7f31180b2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.953+0000 7f311d9c1640 1 -- 172.21.15.119:0/4170072710 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31180b3f00 con 0x7f31180b3aa0 2023-11-08T19:28:08.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.954+0000 7f311d9c1640 1 --2- 172.21.15.119:0/4170072710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f31180cdb30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f31080097a0 tx=0x7f310802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3177c42615fedb06 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f311d1c0640 1 -- 172.21.15.119:0/4170072710 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310802e070 con 0x7f31180b3aa0 2023-11-08T19:28:08.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f311d1c0640 1 -- 172.21.15.119:0/4170072710 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080315a0 con 0x7f31180b3aa0 2023-11-08T19:28:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f311d1c0640 1 -- 172.21.15.119:0/4170072710 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108031930 con 0x7f31180b3aa0 2023-11-08T19:28:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 -- 172.21.15.119:0/4170072710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 msgr2=0x7f31180cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 --2- 172.21.15.119:0/4170072710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f31180cdb30 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f31080097a0 tx=0x7f310802b840 comp rx=0 tx=0).stop 2023-11-08T19:28:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 -- 172.21.15.119:0/4170072710 shutdown_connections 2023-11-08T19:28:08.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 --2- 172.21.15.119:0/4170072710 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f31180cdb30 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 -- 172.21.15.119:0/4170072710 >> 172.21.15.119:0/4170072710 conn(0x7f31180b08e0 msgr2=0x7f31180b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 -- 172.21.15.119:0/4170072710 shutdown_connections 2023-11-08T19:28:08.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.955+0000 7f3120426640 1 -- 172.21.15.119:0/4170072710 wait complete. 2023-11-08T19:28:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.957+0000 7f3120426640 1 Processor -- start 2023-11-08T19:28:08.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.957+0000 7f3120426640 1 -- start start 2023-11-08T19:28:08.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f3120426640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311806a360 0x7f311813c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f3120426640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f311813ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f3120426640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31181440c0 0x7f311813bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f3120426640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3118139ff0 con 0x7f31180b3aa0 2023-11-08T19:28:08.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f3120426640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31181572a0 con 0x7f311806a360 2023-11-08T19:28:08.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f3120426640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31181f13e0 con 0x7f31181440c0 2023-11-08T19:28:08.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f311d9c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f311813ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f311d9c1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f311813ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46218/0 (socket says 172.21.15.119:46218) 2023-11-08T19:28:08.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f311d9c1640 1 -- 172.21.15.119:0/750436864 learned_addr learned my addr 172.21.15.119:0/750436864 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:08.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f311e1c2640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311806a360 0x7f311813c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.958+0000 7f311e9c3640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31181440c0 0x7f311813bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f311d9c1640 1 -- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31181440c0 msgr2=0x7f311813bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f311d9c1640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31181440c0 0x7f311813bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f311d9c1640 1 -- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311806a360 msgr2=0x7f311813c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:08.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f311d9c1640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311806a360 0x7f311813c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:08.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f311d9c1640 1 -- 172.21.15.119:0/750436864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3108009400 con 0x7f31180b3aa0 2023-11-08T19:28:08.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f311d9c1640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f311813ba10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f31080097a0 tx=0x7f31080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108032840 con 0x7f31180b3aa0 2023-11-08T19:28:08.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31080329a0 con 0x7f31180b3aa0 2023-11-08T19:28:08.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108032d50 con 0x7f31180b3aa0 2023-11-08T19:28:08.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.959+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31181f16c0 con 0x7f31180b3aa0 2023-11-08T19:28:08.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.960+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31181f1c80 con 0x7f31180b3aa0 2023-11-08T19:28:08.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.961+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31180cf9a0 con 0x7f31180b3aa0 2023-11-08T19:28:08.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.962+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f310803b3f0 con 0x7f31180b3aa0 2023-11-08T19:28:08.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.963+0000 7f310effd640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f30fc06ae70 0x7f30fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:08.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.963+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6326+0+0 (secure 0 0 0) 0x7f31080ab920 con 0x7f31180b3aa0 2023-11-08T19:28:08.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.963+0000 7f311e1c2640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f30fc06ae70 0x7f30fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:08.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.964+0000 7f311e1c2640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f30fc06ae70 0x7f30fc06d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f311815db50 tx=0x7f3114009060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:08.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:08.967+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3108076770 con 0x7f31180b3aa0 2023-11-08T19:28:09.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.171+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f31180671e0 con 0x7f31180b3aa0 2023-11-08T19:28:09.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.678+0000 7f310effd640 1 -- 172.21.15.119:0/750436864 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3108079d90 con 0x7f31180b3aa0 2023-11-08T19:28:09.680 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-11-08T19:28:09.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f30fc06ae70 msgr2=0x7f30fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f30fc06ae70 0x7f30fc06d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f311815db50 tx=0x7f3114009060 comp rx=0 tx=0).stop 2023-11-08T19:28:09.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 msgr2=0x7f311813ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f311813ba10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f31080097a0 tx=0x7f31080046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 shutdown_connections 2023-11-08T19:28:09.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f30fc06ae70 0x7f30fc06d330 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f31181440c0 0x7f311813bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f31180b3aa0 0x7f311813ba10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 --2- 172.21.15.119:0/750436864 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f311806a360 0x7f311813c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 >> 172.21.15.119:0/750436864 conn(0x7f31180b08e0 msgr2=0x7f3118141a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:09.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 shutdown_connections 2023-11-08T19:28:09.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.683+0000 7f3120426640 1 -- 172.21.15.119:0/750436864 wait complete. 2023-11-08T19:28:09.709 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:28:09.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.905+0000 7f707ec4f640 1 Processor -- start 2023-11-08T19:28:09.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.905+0000 7f707ec4f640 1 -- start start 2023-11-08T19:28:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.906+0000 7f707ec4f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f70781447a0 0x7f707815f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.906+0000 7f707ec4f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70780c3a50 con 0x7f70780cad30 2023-11-08T19:28:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f707ec4f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7078144b80 con 0x7f707806a120 2023-11-08T19:28:09.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f707ec4f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7078070040 con 0x7f70781447a0 2023-11-08T19:28:09.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f7077fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f70780cad30 0x7f707806f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54506/0 (socket says 172.21.15.119:54506) 2023-11-08T19:28:09.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f7077fff640 1 -- 172.21.15.119:0/1069294755 learned_addr learned my addr 172.21.15.119:0/1069294755 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f707c9eb640 1 --2- 172.21.15.119:0/1069294755 >> v2:172.21.15.188:3300/0 conn(0x7f70781447a0 0x7f707815f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986020618 0 0) 0x7f70780c3a50 con 0x7f70780cad30 2023-11-08T19:28:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.907+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7068003520 con 0x7f70780cad30 2023-11-08T19:28:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3318663144 0 0) 0x7f7068003520 con 0x7f70780cad30 2023-11-08T19:28:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 >> v2:172.21.15.188:3300/0 conn(0x7f70781447a0 msgr2=0x7f707815f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 --2- 172.21.15.119:0/1069294755 >> v2:172.21.15.188:3300/0 conn(0x7f70781447a0 0x7f707815f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 >> v1:172.21.15.188:6789/0 conn(0x7f707806a120 legacy=0x7f707806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70780c9150 con 0x7f70780cad30 2023-11-08T19:28:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f7064001690 con 0x7f70780cad30 2023-11-08T19:28:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 >> v1:172.21.15.119:6789/0 conn(0x7f70780cad30 legacy=0x7f707806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7068003970 0x7f7068003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680047c0 0x7f7068004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f706800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078070040 con 0x7f7068005180 2023-11-08T19:28:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706800c020 con 0x7f70680047c0 2023-11-08T19:28:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706800c1e0 con 0x7f7068003970 2023-11-08T19:28:09.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f707d1ec640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f706800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.908+0000 7f707c9eb640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7068003970 0x7f7068003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.909+0000 7f7077fff640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680047c0 0x7f7068004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.909+0000 7f707d1ec640 1 -- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7068003970 msgr2=0x7f7068003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:09.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.909+0000 7f707d1ec640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7068003970 0x7f7068003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.909+0000 7f707d1ec640 1 -- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680047c0 msgr2=0x7f7068004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.909+0000 7f707d1ec640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70680047c0 0x7f7068004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.909+0000 7f707d1ec640 1 -- 172.21.15.119:0/1069294755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068003520 con 0x7f7068005180 2023-11-08T19:28:09.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.910+0000 7f707d1ec640 1 -- 172.21.15.119:0/1069294755 reap_dead start 2023-11-08T19:28:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.910+0000 7f707d1ec640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f706800b980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7070001e50 tx=0x7f70680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a867fe4b6532de42 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.910+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070003b70 con 0x7f7068005180 2023-11-08T19:28:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.910+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7070003d10 con 0x7f7068005180 2023-11-08T19:28:09.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.910+0000 7f70777fe640 1 -- 172.21.15.119:0/1069294755 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7070005310 con 0x7f7068005180 2023-11-08T19:28:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 -- 172.21.15.119:0/1069294755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 msgr2=0x7f706800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f706800b980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7070001e50 tx=0x7f70680057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 -- 172.21.15.119:0/1069294755 shutdown_connections 2023-11-08T19:28:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 --2- 172.21.15.119:0/1069294755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f706800b980 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 -- 172.21.15.119:0/1069294755 >> 172.21.15.119:0/1069294755 conn(0x7f70780cb610 msgr2=0x7f70780cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:09.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 -- 172.21.15.119:0/1069294755 shutdown_connections 2023-11-08T19:28:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.911+0000 7f707ec4f640 1 -- 172.21.15.119:0/1069294755 wait complete. 2023-11-08T19:28:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.913+0000 7f707ec4f640 1 Processor -- start 2023-11-08T19:28:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.913+0000 7f707ec4f640 1 -- start start 2023-11-08T19:28:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707ec4f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f7078141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707ec4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f707813fea0 0x7f70780c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707ec4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7078140320 0x7f70780c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707ec4f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7078141090 con 0x7f7068005180 2023-11-08T19:28:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707ec4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70781f6e70 con 0x7f7078140320 2023-11-08T19:28:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707ec4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7078140760 con 0x7f707813fea0 2023-11-08T19:28:09.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707c9eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f7078141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707c9eb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f7078141520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46230/0 (socket says 172.21.15.119:46230) 2023-11-08T19:28:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.914+0000 7f707c9eb640 1 -- 172.21.15.119:0/867731353 learned_addr learned my addr 172.21.15.119:0/867731353 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707d1ec640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7078140320 0x7f70780c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707c9eb640 1 -- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f707813fea0 msgr2=0x7f70780c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f7077fff640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f707813fea0 0x7f70780c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707c9eb640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f707813fea0 0x7f70780c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707c9eb640 1 -- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7078140320 msgr2=0x7f70780c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:09.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707c9eb640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7078140320 0x7f70780c7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:09.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707c9eb640 1 -- 172.21.15.119:0/867731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7078140a40 con 0x7f7068005180 2023-11-08T19:28:09.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707c9eb640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f7078141520 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f706c011540 tx=0x7f706c011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:09.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c015930 con 0x7f7068005180 2023-11-08T19:28:09.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706c015a90 con 0x7f7068005180 2023-11-08T19:28:09.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706c015e60 con 0x7f7068005180 2023-11-08T19:28:09.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.915+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7078140d20 con 0x7f7068005180 2023-11-08T19:28:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.916+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70780c5bd0 con 0x7f7068005180 2023-11-08T19:28:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.917+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f707806caa0 con 0x7f7068005180 2023-11-08T19:28:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.918+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f706c00f460 con 0x7f7068005180 2023-11-08T19:28:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.918+0000 7f70757fa640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705c06ae70 0x7f705c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:09.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.919+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f706c089fe0 con 0x7f7068005180 2023-11-08T19:28:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.919+0000 7f7077fff640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705c06ae70 0x7f705c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:09.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.919+0000 7f7077fff640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705c06ae70 0x7f705c06d330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7064001220 tx=0x7f7064001150 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:09.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:09.923+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f706c054ed0 con 0x7f7068005180 2023-11-08T19:28:10.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.170+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f70780c5330 con 0x7f7068005180 2023-11-08T19:28:10.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.171+0000 7f70757fa640 1 -- 172.21.15.119:0/867731353 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f706c0584f0 con 0x7f7068005180 2023-11-08T19:28:10.173 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:10.174 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:28:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-11-08T19:28:10.174 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-11-08T19:28:10.175 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg==", 2023-11-08T19:28:10.176 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.176 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.176 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:10.177 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:10.177 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.178 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-11-08T19:28:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ==", 2023-11-08T19:28:10.179 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:10.180 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.181 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-11-08T19:28:10.182 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw==", 2023-11-08T19:28:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.183 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.184 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-11-08T19:28:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ==", 2023-11-08T19:28:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.185 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.186 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-11-08T19:28:10.187 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA==", 2023-11-08T19:28:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.188 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-11-08T19:28:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A==", 2023-11-08T19:28:10.189 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.190 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-11-08T19:28:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA==", 2023-11-08T19:28:10.191 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.192 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-11-08T19:28:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ==", 2023-11-08T19:28:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.193 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.194 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-11-08T19:28:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ==", 2023-11-08T19:28:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.195 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.196 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-11-08T19:28:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw==", 2023-11-08T19:28:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.197 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.198 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-11-08T19:28:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g==", 2023-11-08T19:28:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.199 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.200 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-11-08T19:28:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ==", 2023-11-08T19:28:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.201 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-11-08T19:28:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw==", 2023-11-08T19:28:10.202 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-11-08T19:28:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:28:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.203 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-11-08T19:28:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ==", 2023-11-08T19:28:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.204 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:28:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:28:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:28:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.205 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-11-08T19:28:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q==", 2023-11-08T19:28:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.206 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:28:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:28:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:28:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.207 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-11-08T19:28:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ==", 2023-11-08T19:28:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:10.208 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:28:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:28:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:28:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.209 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-11-08T19:28:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBH4Etl5+oDFBAAMV8QbfI7L35tUuiNz9bvdw==", 2023-11-08T19:28:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:28:10.210 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.211 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-11-08T19:28:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA==", 2023-11-08T19:28:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:28:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:28:10.212 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-11-08T19:28:10.213 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ==", 2023-11-08T19:28:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:28:10.214 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:28:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:10.215 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-11-08T19:28:10.215 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:28:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.173+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705c06ae70 msgr2=0x7f705c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.173+0000 7f707ec4f640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705c06ae70 0x7f705c06d330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f7064001220 tx=0x7f7064001150 comp rx=0 tx=0).stop 2023-11-08T19:28:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.173+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 msgr2=0x7f7078141520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.173+0000 7f707ec4f640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f7078141520 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f706c011540 tx=0x7f706c011a00 comp rx=0 tx=0).stop 2023-11-08T19:28:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 shutdown_connections 2023-11-08T19:28:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f705c06ae70 0x7f705c06d330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7078140320 0x7f70780c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f707813fea0 0x7f70780c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 --2- 172.21.15.119:0/867731353 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7068005180 0x7f7078141520 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 >> 172.21.15.119:0/867731353 conn(0x7f70780cb610 msgr2=0x7f707815d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:10.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 shutdown_connections 2023-11-08T19:28:10.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.174+0000 7f707ec4f640 1 -- 172.21.15.119:0/867731353 wait complete. 2023-11-08T19:28:10.220 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:10.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.424+0000 7f848ef76640 1 Processor -- start 2023-11-08T19:28:10.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.424+0000 7f848ef76640 1 -- start start 2023-11-08T19:28:10.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.426+0000 7f848ef76640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f84880cc810 0x7f84880c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.426+0000 7f848ef76640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84880ae410 con 0x7f848806a390 2023-11-08T19:28:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.426+0000 7f848ef76640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848806ba30 con 0x7f848815d180 2023-11-08T19:28:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.426+0000 7f848ef76640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848806bcf0 con 0x7f84880cc810 2023-11-08T19:28:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.426+0000 7f848cd12640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f848806a390 0x7f848815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54520/0 (socket says 172.21.15.119:54520) 2023-11-08T19:28:10.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.426+0000 7f848cd12640 1 -- 172.21.15.119:0/2563444603 learned_addr learned my addr 172.21.15.119:0/2563444603 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f8487fff640 1 --2- 172.21.15.119:0/2563444603 >> v2:172.21.15.188:3300/0 conn(0x7f84880cc810 0x7f84880c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015123025 0 0) 0x7f84880ae410 con 0x7f848806a390 2023-11-08T19:28:10.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8478003520 con 0x7f848806a390 2023-11-08T19:28:10.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2670524162 0 0) 0x7f8478003520 con 0x7f848806a390 2023-11-08T19:28:10.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 >> v2:172.21.15.188:3300/0 conn(0x7f84880cc810 msgr2=0x7f84880c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:10.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 --2- 172.21.15.119:0/2563444603 >> v2:172.21.15.188:3300/0 conn(0x7f84880cc810 0x7f84880c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 >> v1:172.21.15.188:6789/0 conn(0x7f848815d180 legacy=0x7f8488160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848806bfd0 con 0x7f848806a390 2023-11-08T19:28:10.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f847c001670 con 0x7f848806a390 2023-11-08T19:28:10.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 >> v1:172.21.15.119:6789/0 conn(0x7f848806a390 legacy=0x7f848815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84780039c0 0x7f8478003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.427+0000 7f84877fe640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848806bcf0 con 0x7f8478005180 2023-11-08T19:28:10.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847800c080 con 0x7f84780047c0 2023-11-08T19:28:10.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847800c240 con 0x7f84780039c0 2023-11-08T19:28:10.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848d513640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f8487fff640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848cd12640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84780039c0 0x7f8478003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848d513640 1 -- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84780039c0 msgr2=0x7f8478003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:10.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848d513640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84780039c0 0x7f8478003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848d513640 1 -- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84780047c0 msgr2=0x7f8478004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:10.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848d513640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84780047c0 0x7f8478004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.428+0000 7f848d513640 1 -- 172.21.15.119:0/2563444603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478003520 con 0x7f8478005180 2023-11-08T19:28:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.429+0000 7f848d513640 1 -- 172.21.15.119:0/2563444603 reap_dead start 2023-11-08T19:28:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.429+0000 7f848d513640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8478005180 0x7f847800b980 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8480001e50 tx=0x7f84780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f389072084457cd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480003b70 con 0x7f8478005180 2023-11-08T19:28:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480003cf0 con 0x7f8478005180 2023-11-08T19:28:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f84877fe640 1 -- 172.21.15.119:0/2563444603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480005310 con 0x7f8478005180 2023-11-08T19:28:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 -- 172.21.15.119:0/2563444603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8478005180 msgr2=0x7f847800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8478005180 0x7f847800b980 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8480001e50 tx=0x7f84780057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 -- 172.21.15.119:0/2563444603 shutdown_connections 2023-11-08T19:28:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 --2- 172.21.15.119:0/2563444603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8478005180 0x7f847800b980 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 -- 172.21.15.119:0/2563444603 >> 172.21.15.119:0/2563444603 conn(0x7f84880cb610 msgr2=0x7f84880cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 -- 172.21.15.119:0/2563444603 shutdown_connections 2023-11-08T19:28:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.430+0000 7f848ef76640 1 -- 172.21.15.119:0/2563444603 wait complete. 2023-11-08T19:28:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.432+0000 7f848ef76640 1 Processor -- start 2023-11-08T19:28:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.432+0000 7f848ef76640 1 -- start start 2023-11-08T19:28:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848ef76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8478005180 0x7f84881581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848ef76640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84880c8490 0x7f84880b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848ef76640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 0x7f848815f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848ef76640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848815fba0 con 0x7f848815f1e0 2023-11-08T19:28:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848ef76640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c8000 con 0x7f84880c8490 2023-11-08T19:28:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848ef76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c73f0 con 0x7f8478005180 2023-11-08T19:28:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848d513640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 0x7f848815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848d513640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 0x7f848815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46256/0 (socket says 172.21.15.119:46256) 2023-11-08T19:28:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.433+0000 7f848d513640 1 -- 172.21.15.119:0/336108586 learned_addr learned my addr 172.21.15.119:0/336108586 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f8487fff640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84880c8490 0x7f84880b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848cd12640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8478005180 0x7f84881581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848d513640 1 -- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8478005180 msgr2=0x7f84881581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848d513640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8478005180 0x7f84881581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848d513640 1 -- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84880c8490 msgr2=0x7f84880b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848d513640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84880c8490 0x7f84880b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848d513640 1 -- 172.21.15.119:0/336108586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880c76d0 con 0x7f848815f1e0 2023-11-08T19:28:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848d513640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 0x7f848815f660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8480001f70 tx=0x7f84800014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480006d50 con 0x7f848815f1e0 2023-11-08T19:28:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480003b80 con 0x7f848815f1e0 2023-11-08T19:28:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84880c79b0 con 0x7f848815f1e0 2023-11-08T19:28:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.434+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480039680 con 0x7f848815f1e0 2023-11-08T19:28:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.435+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8488158ab0 con 0x7f848815f1e0 2023-11-08T19:28:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.436+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848815c150 con 0x7f848815f1e0 2023-11-08T19:28:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.437+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84800397e0 con 0x7f848815f1e0 2023-11-08T19:28:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.437+0000 7f84857fa640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846c06aea0 0x7f846c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.437+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f84800ab6a0 con 0x7f848815f1e0 2023-11-08T19:28:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.438+0000 7f848cd12640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846c06aea0 0x7f846c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.438+0000 7f848cd12640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846c06aea0 0x7f846c06d360 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f847c000c00 tx=0x7f847c001470 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.442+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8480076420 con 0x7f848815f1e0 2023-11-08T19:28:10.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.635+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8488074430 con 0x7f848815f1e0 2023-11-08T19:28:10.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.636+0000 7f84857fa640 1 -- 172.21.15.119:0/336108586 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+18303 (secure 0 0 0) 0x7f8480048090 con 0x7f848815f1e0 2023-11-08T19:28:10.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:10.639 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":79,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:09.691628+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:10.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846c06aea0 msgr2=0x7f846c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846c06aea0 0x7f846c06d360 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f847c000c00 tx=0x7f847c001470 comp rx=0 tx=0).stop 2023-11-08T19:28:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 msgr2=0x7f848815f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 0x7f848815f660 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f8480001f70 tx=0x7f84800014a0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 shutdown_connections 2023-11-08T19:28:10.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846c06aea0 0x7f846c06d360 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848815f1e0 0x7f848815f660 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f84880c8490 0x7f84880b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 --2- 172.21.15.119:0/336108586 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8478005180 0x7f84881581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 >> 172.21.15.119:0/336108586 conn(0x7f84880cb610 msgr2=0x7f848815ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:10.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 shutdown_connections 2023-11-08T19:28:10.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.639+0000 7f848ef76640 1 -- 172.21.15.119:0/336108586 wait complete. 2023-11-08T19:28:10.661 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:28:10.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:28:10.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.876+0000 7ffa5b2c2640 1 Processor -- start 2023-11-08T19:28:10.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.876+0000 7ffa5b2c2640 1 -- start start 2023-11-08T19:28:10.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5b2c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa540cf410 0x7ffa540cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5b2c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa5415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5b2c2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa540cf8a0 con 0x7ffa5406a120 2023-11-08T19:28:10.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5b2c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa540cfad0 con 0x7ffa540cf410 2023-11-08T19:28:10.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5b2c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5406e790 con 0x7ffa5415d520 2023-11-08T19:28:10.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5885d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa540cf410 0x7ffa540cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5885d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffa540cf410 0x7ffa540cd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46260/0 (socket says 172.21.15.119:46260) 2023-11-08T19:28:10.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.878+0000 7ffa5885d640 1 -- 172.21.15.119:0/250815899 learned_addr learned my addr 172.21.15.119:0/250815899 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:10.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.879+0000 7ffa5985f640 1 --2- 172.21.15.119:0/250815899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa5415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.879+0000 7ffa5885d640 1 -- 172.21.15.119:0/250815899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 msgr2=0x7ffa5415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:10.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.879+0000 7ffa5885d640 1 --2- 172.21.15.119:0/250815899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa5415d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.879+0000 7ffa5885d640 1 -- 172.21.15.119:0/250815899 >> v1:172.21.15.188:6789/0 conn(0x7ffa5406a120 legacy=0x7ffa5406a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:10.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.879+0000 7ffa5885d640 1 -- 172.21.15.119:0/250815899 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5406d9d0 con 0x7ffa540cf410 2023-11-08T19:28:10.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.880+0000 7ffa5885d640 1 --2- 172.21.15.119:0/250815899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa540cf410 0x7ffa540cd960 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009870 tx=0x7ffa4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a8c606581f99ae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:10.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.880+0000 7ffa4bfff640 1 -- 172.21.15.119:0/250815899 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44031640 con 0x7ffa540cf410 2023-11-08T19:28:10.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.880+0000 7ffa4bfff640 1 -- 172.21.15.119:0/250815899 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa440317a0 con 0x7ffa540cf410 2023-11-08T19:28:10.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.880+0000 7ffa4bfff640 1 -- 172.21.15.119:0/250815899 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44031b30 con 0x7ffa540cf410 2023-11-08T19:28:10.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/250815899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa540cf410 msgr2=0x7ffa540cd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/250815899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa540cf410 0x7ffa540cd960 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009870 tx=0x7ffa4402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/250815899 shutdown_connections 2023-11-08T19:28:10.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/250815899 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa540cf410 0x7ffa540cd960 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/250815899 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa5415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/250815899 >> 172.21.15.119:0/250815899 conn(0x7ffa540cfd20 msgr2=0x7ffa540d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:10.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/250815899 shutdown_connections 2023-11-08T19:28:10.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.881+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/250815899 wait complete. 2023-11-08T19:28:10.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.883+0000 7ffa5b2c2640 1 Processor -- start 2023-11-08T19:28:10.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.883+0000 7ffa5b2c2640 1 -- start start 2023-11-08T19:28:10.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5b2c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 0x7ffa540c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5b2c2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa540cf410 0x7ffa5414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5b2c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa54070d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5b2c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540b4e00 con 0x7ffa5406a120 2023-11-08T19:28:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5b2c2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540b50e0 con 0x7ffa540cf410 2023-11-08T19:28:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5b2c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540b53c0 con 0x7ffa5415d520 2023-11-08T19:28:10.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 0x7ffa540c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 0x7ffa540c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46276/0 (socket says 172.21.15.119:46276) 2023-11-08T19:28:10.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 -- 172.21.15.119:0/3049269147 learned_addr learned my addr 172.21.15.119:0/3049269147 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:10.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 -- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 msgr2=0x7ffa54070d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa54070d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 -- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa540cf410 msgr2=0x7ffa5414bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:10.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5885d640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa540cf410 0x7ffa5414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa540cf410 0x7ffa5414bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:10.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.884+0000 7ffa5905e640 1 -- 172.21.15.119:0/3049269147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa440094d0 con 0x7ffa5406a120 2023-11-08T19:28:10.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.885+0000 7ffa5905e640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 0x7ffa540c6110 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c410 tx=0x7ffa5000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.885+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa500035f0 con 0x7ffa5406a120 2023-11-08T19:28:10.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.885+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa50003750 con 0x7ffa5406a120 2023-11-08T19:28:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.885+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50003b40 con 0x7ffa5406a120 2023-11-08T19:28:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.885+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa540b56a0 con 0x7ffa5406a120 2023-11-08T19:28:10.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.885+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa540c6650 con 0x7ffa5406a120 2023-11-08T19:28:10.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.886+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa540c3780 con 0x7ffa5406a120 2023-11-08T19:28:10.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.887+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa50003ca0 con 0x7ffa5406a120 2023-11-08T19:28:10.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.888+0000 7ffa49ffb640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa3806aea0 0x7ffa3806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:10.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.888+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7ffa50088a90 con 0x7ffa5406a120 2023-11-08T19:28:10.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.888+0000 7ffa5885d640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa3806aea0 0x7ffa3806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.889+0000 7ffa5885d640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa3806aea0 0x7ffa3806d360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa44036000 tx=0x7ffa4403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:10.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:10.892+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa540c3780 con 0x7ffa5406a120 2023-11-08T19:28:11.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.068+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa540c3780 con 0x7ffa5406a120 2023-11-08T19:28:11.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.069+0000 7ffa49ffb640 1 -- 172.21.15.119:0/3049269147 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa50017020 con 0x7ffa5406a120 2023-11-08T19:28:11.070 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:28:11.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa3806aea0 msgr2=0x7ffa3806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa3806aea0 0x7ffa3806d360 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ffa44036000 tx=0x7ffa4403c040 comp rx=0 tx=0).stop 2023-11-08T19:28:11.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 msgr2=0x7ffa540c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 0x7ffa540c6110 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c410 tx=0x7ffa5000c8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 shutdown_connections 2023-11-08T19:28:11.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa3806aea0 0x7ffa3806d360 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.071+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffa5415d520 0x7ffa54070d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.072+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffa540cf410 0x7ffa5414bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.072+0000 7ffa5b2c2640 1 --2- 172.21.15.119:0/3049269147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffa5406a120 0x7ffa540c6110 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.072+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 >> 172.21.15.119:0/3049269147 conn(0x7ffa540cfd20 msgr2=0x7ffa5406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:11.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.072+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 shutdown_connections 2023-11-08T19:28:11.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.072+0000 7ffa5b2c2640 1 -- 172.21.15.119:0/3049269147 wait complete. 2023-11-08T19:28:11.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:28:11.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.292+0000 7ffab180e640 1 Processor -- start 2023-11-08T19:28:11.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.292+0000 7ffab180e640 1 -- start start 2023-11-08T19:28:11.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffab180e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffaac15d9f0 0x7ffaac15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffab180e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ffaac06cb40 0x7ffaac06cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffab180e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac06d4a0 con 0x7ffaac143600 2023-11-08T19:28:11.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffab180e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0c4f50 con 0x7ffaac06cb40 2023-11-08T19:28:11.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffab180e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac0c4960 con 0x7ffaac15d9f0 2023-11-08T19:28:11.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffaaaffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ffaac143600 0x7ffaac0c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54536/0 (socket says 172.21.15.119:54536) 2023-11-08T19:28:11.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffaaaffd640 1 -- 172.21.15.119:0/2344767993 learned_addr learned my addr 172.21.15.119:0/2344767993 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:11.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.294+0000 7ffaab7fe640 1 --2- 172.21.15.119:0/2344767993 >> v2:172.21.15.119:3300/0 conn(0x7ffaac06cb40 0x7ffaac06cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaaa7fc640 1 -- 172.21.15.119:0/2344767993 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648487352 0 0) 0x7ffaac06d4a0 con 0x7ffaac143600 2023-11-08T19:28:11.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaaa7fc640 1 -- 172.21.15.119:0/2344767993 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa98003520 con 0x7ffaac143600 2023-11-08T19:28:11.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaa27fc640 1 --2- 172.21.15.119:0/2344767993 >> v2:172.21.15.188:3300/0 conn(0x7ffaac15d9f0 0x7ffaac15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaab7fe640 1 -- 172.21.15.119:0/2344767993 >> v2:172.21.15.188:3300/0 conn(0x7ffaac15d9f0 msgr2=0x7ffaac15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:11.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaab7fe640 1 --2- 172.21.15.119:0/2344767993 >> v2:172.21.15.188:3300/0 conn(0x7ffaac15d9f0 0x7ffaac15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaab7fe640 1 -- 172.21.15.119:0/2344767993 >> v1:172.21.15.119:6789/0 conn(0x7ffaac143600 legacy=0x7ffaac0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.295+0000 7ffaab7fe640 1 -- 172.21.15.119:0/2344767993 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac15e440 con 0x7ffaac06cb40 2023-11-08T19:28:11.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.296+0000 7ffaab7fe640 1 --2- 172.21.15.119:0/2344767993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac06cb40 0x7ffaac06cf60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0c91e0 tx=0x7ffaa402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a117f073d115420 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:11.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.296+0000 7ffaaa7fc640 1 -- 172.21.15.119:0/2344767993 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa402cbe0 con 0x7ffaac06cb40 2023-11-08T19:28:11.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.296+0000 7ffaaa7fc640 1 -- 172.21.15.119:0/2344767993 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa402cd40 con 0x7ffaac06cb40 2023-11-08T19:28:11.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.296+0000 7ffaaa7fc640 1 -- 172.21.15.119:0/2344767993 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4034660 con 0x7ffaac06cb40 2023-11-08T19:28:11.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.296+0000 7ffab180e640 1 -- 172.21.15.119:0/2344767993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac06cb40 msgr2=0x7ffaac06cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.296+0000 7ffab180e640 1 --2- 172.21.15.119:0/2344767993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac06cb40 0x7ffaac06cf60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0c91e0 tx=0x7ffaa402be30 comp rx=0 tx=0).stop 2023-11-08T19:28:11.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.297+0000 7ffab180e640 1 -- 172.21.15.119:0/2344767993 shutdown_connections 2023-11-08T19:28:11.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.297+0000 7ffab180e640 1 --2- 172.21.15.119:0/2344767993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac06cb40 0x7ffaac06cf60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.297+0000 7ffab180e640 1 --2- 172.21.15.119:0/2344767993 >> v2:172.21.15.188:3300/0 conn(0x7ffaac15d9f0 0x7ffaac15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.297+0000 7ffab180e640 1 -- 172.21.15.119:0/2344767993 >> 172.21.15.119:0/2344767993 conn(0x7ffaac0cb610 msgr2=0x7ffaac0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:11.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.297+0000 7ffab180e640 1 -- 172.21.15.119:0/2344767993 shutdown_connections 2023-11-08T19:28:11.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.297+0000 7ffab180e640 1 -- 172.21.15.119:0/2344767993 wait complete. 2023-11-08T19:28:11.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.298+0000 7ffab180e640 1 Processor -- start 2023-11-08T19:28:11.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.298+0000 7ffab180e640 1 -- start start 2023-11-08T19:28:11.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.299+0000 7ffab180e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaac06cb40 0x7ffaac15aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.299+0000 7ffab180e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 0x7ffaac1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.299+0000 7ffab180e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaac15d9f0 0x7ffaac13b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.299+0000 7ffab180e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac163e20 con 0x7ffaac143600 2023-11-08T19:28:11.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.299+0000 7ffab180e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac15c180 con 0x7ffaac15d9f0 2023-11-08T19:28:11.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.299+0000 7ffab180e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac15b570 con 0x7ffaac06cb40 2023-11-08T19:28:11.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 0x7ffaac1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 0x7ffaac1638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46286/0 (socket says 172.21.15.119:46286) 2023-11-08T19:28:11.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 -- 172.21.15.119:0/1147033637 learned_addr learned my addr 172.21.15.119:0/1147033637 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:11.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 -- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaac06cb40 msgr2=0x7ffaac15aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaac06cb40 0x7ffaac15aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaab7fe640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaac15d9f0 0x7ffaac13b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 -- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaac15d9f0 msgr2=0x7ffaac13b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaac15d9f0 0x7ffaac13b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 -- 172.21.15.119:0/1147033637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4009a70 con 0x7ffaac143600 2023-11-08T19:28:11.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa27fc640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 0x7ffaac1638e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d5f0 tx=0x7ffa9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:11.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa940125f0 con 0x7ffaac143600 2023-11-08T19:28:11.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa94012750 con 0x7ffaac143600 2023-11-08T19:28:11.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.300+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94012b00 con 0x7ffaac143600 2023-11-08T19:28:11.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.301+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac15b850 con 0x7ffaac143600 2023-11-08T19:28:11.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.301+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac15be10 con 0x7ffaac143600 2023-11-08T19:28:11.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.302+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac06cf20 con 0x7ffaac143600 2023-11-08T19:28:11.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.303+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa94012c60 con 0x7ffaac143600 2023-11-08T19:28:11.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.303+0000 7ffaa8ff9640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa8c06ae70 0x7ffa8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.304+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7ffa94089c80 con 0x7ffaac143600 2023-11-08T19:28:11.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.304+0000 7ffaaaffd640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa8c06ae70 0x7ffa8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.304+0000 7ffaaaffd640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa8c06ae70 0x7ffa8c06d330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffaac13f520 tx=0x7ffa9c007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:11.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.308+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa94054bb0 con 0x7ffaac143600 2023-11-08T19:28:11.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.481+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7ffaac15c2c0 con 0x7ffaac143600 2023-11-08T19:28:11.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.482+0000 7ffaa8ff9640 1 -- 172.21.15.119:0/1147033637 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v79) v1 ==== 103+0+11 (secure 0 0 0) 0x7ffaac15c2c0 con 0x7ffaac143600 2023-11-08T19:28:11.483 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.484+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa8c06ae70 msgr2=0x7ffa8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.484+0000 7ffab180e640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa8c06ae70 0x7ffa8c06d330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ffaac13f520 tx=0x7ffa9c007930 comp rx=0 tx=0).stop 2023-11-08T19:28:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.484+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 msgr2=0x7ffaac1638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.484+0000 7ffab180e640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 0x7ffaac1638e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d5f0 tx=0x7ffa9400dab0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 shutdown_connections 2023-11-08T19:28:11.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ffa8c06ae70 0x7ffa8c06d330 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffaac15d9f0 0x7ffaac13b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ffaac143600 0x7ffaac1638e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 --2- 172.21.15.119:0/1147033637 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffaac06cb40 0x7ffaac15aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 >> 172.21.15.119:0/1147033637 conn(0x7ffaac0cb610 msgr2=0x7ffaac0ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:11.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 shutdown_connections 2023-11-08T19:28:11.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.485+0000 7ffab180e640 1 -- 172.21.15.119:0/1147033637 wait complete. 2023-11-08T19:28:11.505 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.711+0000 7f11ad6d4640 1 Processor -- start 2023-11-08T19:28:11.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.711+0000 7f11ad6d4640 1 -- start start 2023-11-08T19:28:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.712+0000 7f11ad6d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f11a806a830 0x7f11a80cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11ad6d4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a806d220 con 0x7f11a815d000 2023-11-08T19:28:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11ad6d4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a80aca40 con 0x7f11a80c3a50 2023-11-08T19:28:11.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11ad6d4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a80ccd50 con 0x7f11a806a830 2023-11-08T19:28:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a6ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f11a815d000 0x7f11a806a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54550/0 (socket says 172.21.15.119:54550) 2023-11-08T19:28:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a6ffd640 1 -- 172.21.15.119:0/2488231703 learned_addr learned my addr 172.21.15.119:0/2488231703 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2025303672 0 0) 0x7f11a806d220 con 0x7f11a815d000 2023-11-08T19:28:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1194003520 con 0x7f11a815d000 2023-11-08T19:28:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2488231703 >> v2:172.21.15.188:3300/0 conn(0x7f11a806a830 0x7f11a80cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061333249 0 0) 0x7f1194003520 con 0x7f11a815d000 2023-11-08T19:28:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 >> v2:172.21.15.188:3300/0 conn(0x7f11a806a830 msgr2=0x7f11a80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a5ffb640 1 --2- 172.21.15.119:0/2488231703 >> v2:172.21.15.188:3300/0 conn(0x7f11a806a830 0x7f11a80cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.713+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 >> v1:172.21.15.188:6789/0 conn(0x7f11a80c3a50 legacy=0x7f11a8160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80acd20 con 0x7f11a815d000 2023-11-08T19:28:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f1198001690 con 0x7f11a815d000 2023-11-08T19:28:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 >> v1:172.21.15.119:6789/0 conn(0x7f11a815d000 legacy=0x7f11a806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11940039c0 0x7f1194003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11940047c0 0x7f1194004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1194005250 0x7f119400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80ccd50 con 0x7f11940047c0 2023-11-08T19:28:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119400c1a0 con 0x7f1194005250 2023-11-08T19:28:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f119400c360 con 0x7f11940039c0 2023-11-08T19:28:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11940047c0 0x7f1194004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a6ffd640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11940039c0 0x7f1194003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a67fc640 1 -- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11940039c0 msgr2=0x7f1194003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11940039c0 0x7f1194003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a67fc640 1 -- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1194005250 msgr2=0x7f119400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a77fe640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1194005250 0x7f119400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1194005250 0x7f119400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a67fc640 1 -- 172.21.15.119:0/2488231703 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1194003520 con 0x7f11940047c0 2023-11-08T19:28:11.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a77fe640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1194005250 0x7f119400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:11.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.714+0000 7f11a77fe640 1 -- 172.21.15.119:0/2488231703 reap_dead start 2023-11-08T19:28:11.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.715+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11940047c0 0x7f1194004c40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1190007360 tx=0x7f11940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1e0133994ab913 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:11.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119002ccb0 con 0x7f11940047c0 2023-11-08T19:28:11.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119002ce10 con 0x7f11940047c0 2023-11-08T19:28:11.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11a5ffb640 1 -- 172.21.15.119:0/2488231703 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190034660 con 0x7f11940047c0 2023-11-08T19:28:11.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2488231703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11940047c0 msgr2=0x7f1194004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11940047c0 0x7f1194004c40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1190007360 tx=0x7f11940057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2488231703 shutdown_connections 2023-11-08T19:28:11.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2488231703 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11940047c0 0x7f1194004c40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2488231703 >> 172.21.15.119:0/2488231703 conn(0x7f11a80cb610 msgr2=0x7f11a80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:11.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2488231703 shutdown_connections 2023-11-08T19:28:11.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.716+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2488231703 wait complete. 2023-11-08T19:28:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.718+0000 7f11ad6d4640 1 Processor -- start 2023-11-08T19:28:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.718+0000 7f11ad6d4640 1 -- start start 2023-11-08T19:28:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11ad6d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11940047c0 0x7f11a81f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11ad6d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 0x7f11a8070980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11ad6d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a8070ec0 0x7f11a81f60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11ad6d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a8071340 con 0x7f11a81f3570 2023-11-08T19:28:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11ad6d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80ae010 con 0x7f11940047c0 2023-11-08T19:28:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11ad6d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a80aee70 con 0x7f11a8070ec0 2023-11-08T19:28:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11a67fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 0x7f11a8070980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11a67fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 0x7f11a8070980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46308/0 (socket says 172.21.15.119:46308) 2023-11-08T19:28:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.719+0000 7f11a67fc640 1 -- 172.21.15.119:0/2282209718 learned_addr learned my addr 172.21.15.119:0/2282209718 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a6ffd640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11940047c0 0x7f11a81f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a67fc640 1 -- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a8070ec0 msgr2=0x7f11a81f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a8070ec0 0x7f11a81f60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a67fc640 1 -- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11940047c0 msgr2=0x7f11a81f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11940047c0 0x7f11a81f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a67fc640 1 -- 172.21.15.119:0/2282209718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80ae260 con 0x7f11a81f3570 2023-11-08T19:28:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a6ffd640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11940047c0 0x7f11a81f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11a67fc640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 0x7f11a8070980 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f119002c650 tx=0x7f1190004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190044070 con 0x7f11a81f3570 2023-11-08T19:28:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190003990 con 0x7f11a81f3570 2023-11-08T19:28:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a80ae540 con 0x7f11a81f3570 2023-11-08T19:28:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190003d40 con 0x7f11a81f3570 2023-11-08T19:28:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.720+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a80aeb00 con 0x7f11a81f3570 2023-11-08T19:28:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.721+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a80712e0 con 0x7f11a81f3570 2023-11-08T19:28:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.722+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1190003ea0 con 0x7f11a81f3570 2023-11-08T19:28:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.723+0000 7f119f7fe640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f118806aea0 0x7f118806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.723+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f11900ab7f0 con 0x7f11a81f3570 2023-11-08T19:28:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.724+0000 7f11a6ffd640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f118806aea0 0x7f118806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.724+0000 7f11a6ffd640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f118806aea0 0x7f118806d360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1198001eb0 tx=0x7f1198009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.727+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11a80712e0 con 0x7f11a81f3570 2023-11-08T19:28:11.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.907+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f11a80aec90 con 0x7f11a81f3570 2023-11-08T19:28:11.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.907+0000 7f119f7fe640 1 -- 172.21.15.119:0/2282209718 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v79) v1 ==== 103+0+11 (secure 0 0 0) 0x7f119007a0a0 con 0x7f11a81f3570 2023-11-08T19:28:11.909 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:11.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.909+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f118806aea0 msgr2=0x7f118806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.909+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f118806aea0 0x7f118806d360 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f1198001eb0 tx=0x7f1198009040 comp rx=0 tx=0).stop 2023-11-08T19:28:11.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 msgr2=0x7f11a8070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:11.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 0x7f11a8070980 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f119002c650 tx=0x7f1190004ab0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 shutdown_connections 2023-11-08T19:28:11.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f118806aea0 0x7f118806d360 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a8070ec0 0x7f11a81f60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f11a81f3570 0x7f11a8070980 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 --2- 172.21.15.119:0/2282209718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11940047c0 0x7f11a81f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:11.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 >> 172.21.15.119:0/2282209718 conn(0x7f11a80cb610 msgr2=0x7f11a815aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:11.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 shutdown_connections 2023-11-08T19:28:11.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:11.910+0000 7f11ad6d4640 1 -- 172.21.15.119:0/2282209718 wait complete. 2023-11-08T19:28:11.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:12.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.141+0000 7fcb6d9ee640 1 Processor -- start 2023-11-08T19:28:12.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.141+0000 7fcb6d9ee640 1 -- start start 2023-11-08T19:28:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.143+0000 7fcb6d9ee640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcb68144690 0x7fcb6806a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.143+0000 7fcb6d9ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680ccb40 0x7fcb6806c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.143+0000 7fcb6d9ee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb68162ae0 con 0x7fcb680cc6d0 2023-11-08T19:28:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.143+0000 7fcb6d9ee640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb68165de0 con 0x7fcb68144690 2023-11-08T19:28:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.143+0000 7fcb6d9ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680ccfd0 con 0x7fcb680ccb40 2023-11-08T19:28:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.143+0000 7fcb66ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcb68144690 0x7fcb6806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.144+0000 7fcb66ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcb68144690 0x7fcb6806a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46310/0 (socket says 172.21.15.119:46310) 2023-11-08T19:28:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.144+0000 7fcb66ffd640 1 -- 172.21.15.119:0/185782436 learned_addr learned my addr 172.21.15.119:0/185782436 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.144+0000 7fcb66ffd640 1 -- 172.21.15.119:0/185782436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680ccb40 msgr2=0x7fcb6806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.144+0000 7fcb66ffd640 1 --2- 172.21.15.119:0/185782436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680ccb40 0x7fcb6806c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.144+0000 7fcb66ffd640 1 -- 172.21.15.119:0/185782436 >> v1:172.21.15.119:6789/0 conn(0x7fcb680cc6d0 legacy=0x7fcb6806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.144+0000 7fcb66ffd640 1 -- 172.21.15.119:0/185782436 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb68165160 con 0x7fcb68144690 2023-11-08T19:28:12.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.145+0000 7fcb66ffd640 1 --2- 172.21.15.119:0/185782436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb6806a550 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009870 tx=0x7fcb5802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20a61cea70fa498 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.145+0000 7fcb65ffb640 1 -- 172.21.15.119:0/185782436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58030840 con 0x7fcb68144690 2023-11-08T19:28:12.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.145+0000 7fcb65ffb640 1 -- 172.21.15.119:0/185782436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb580309a0 con 0x7fcb68144690 2023-11-08T19:28:12.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.145+0000 7fcb65ffb640 1 -- 172.21.15.119:0/185782436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58030d30 con 0x7fcb68144690 2023-11-08T19:28:12.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/185782436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 msgr2=0x7fcb6806a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/185782436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb6806a550 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fcb58009870 tx=0x7fcb5802b980 comp rx=0 tx=0).stop 2023-11-08T19:28:12.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/185782436 shutdown_connections 2023-11-08T19:28:12.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/185782436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680ccb40 0x7fcb6806c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/185782436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb6806a550 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/185782436 >> 172.21.15.119:0/185782436 conn(0x7fcb680cb610 msgr2=0x7fcb680cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/185782436 shutdown_connections 2023-11-08T19:28:12.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.146+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/185782436 wait complete. 2023-11-08T19:28:12.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.148+0000 7fcb6d9ee640 1 Processor -- start 2023-11-08T19:28:12.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.148+0000 7fcb6d9ee640 1 -- start start 2023-11-08T19:28:12.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb6d9ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680cc6d0 0x7fcb681438e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb6d9ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ccb40 0x7fcb680c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb6d9ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb680b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb6d9ee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680c8ae0 con 0x7fcb68144690 2023-11-08T19:28:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb6d9ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680b3ab0 con 0x7fcb680ccb40 2023-11-08T19:28:12.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb6d9ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680b3d90 con 0x7fcb680cc6d0 2023-11-08T19:28:12.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb680b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb677fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb680b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46318/0 (socket says 172.21.15.119:46318) 2023-11-08T19:28:12.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.149+0000 7fcb677fe640 1 -- 172.21.15.119:0/2099403181 learned_addr learned my addr 172.21.15.119:0/2099403181 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:12.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb667fc640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ccb40 0x7fcb680c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb66ffd640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680cc6d0 0x7fcb681438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb677fe640 1 -- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680cc6d0 msgr2=0x7fcb681438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb677fe640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680cc6d0 0x7fcb681438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb677fe640 1 -- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ccb40 msgr2=0x7fcb680c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb677fe640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ccb40 0x7fcb680c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb677fe640 1 -- 172.21.15.119:0/2099403181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb580094d0 con 0x7fcb68144690 2023-11-08T19:28:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb667fc640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ccb40 0x7fcb680c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb66ffd640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680cc6d0 0x7fcb681438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb677fe640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb680b3240 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b9e0 tx=0x7fcb6000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6000caa0 con 0x7fcb68144690 2023-11-08T19:28:12.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb6000cc20 con 0x7fcb68144690 2023-11-08T19:28:12.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb680b4070 con 0x7fcb68144690 2023-11-08T19:28:12.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb60014440 con 0x7fcb68144690 2023-11-08T19:28:12.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.150+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb680b4630 con 0x7fcb68144690 2023-11-08T19:28:12.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.152+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6806ae90 con 0x7fcb68144690 2023-11-08T19:28:12.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.153+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb6001d3b0 con 0x7fcb68144690 2023-11-08T19:28:12.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.154+0000 7fcb5ffff640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb4806ae70 0x7fcb4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.154+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7fcb60089790 con 0x7fcb68144690 2023-11-08T19:28:12.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.154+0000 7fcb66ffd640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb4806ae70 0x7fcb4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.155+0000 7fcb66ffd640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb4806ae70 0x7fcb4806d330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcb58036000 tx=0x7fcb5803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.158+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb600545b0 con 0x7fcb68144690 2023-11-08T19:28:12.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.347+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fcb6806cd30 con 0x7fcb68144690 2023-11-08T19:28:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.348+0000 7fcb5ffff640 1 -- 172.21.15.119:0/2099403181 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v79) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcb60019020 con 0x7fcb68144690 2023-11-08T19:28:12.350 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:12.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb4806ae70 msgr2=0x7fcb4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb4806ae70 0x7fcb4806d330 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcb58036000 tx=0x7fcb5803b040 comp rx=0 tx=0).stop 2023-11-08T19:28:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 msgr2=0x7fcb680b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb680b3240 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fcb6000b9e0 tx=0x7fcb6000bea0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 shutdown_connections 2023-11-08T19:28:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcb4806ae70 0x7fcb4806d330 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcb68144690 0x7fcb680b3240 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcb680ccb40 0x7fcb680c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 --2- 172.21.15.119:0/2099403181 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcb680cc6d0 0x7fcb681438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 >> 172.21.15.119:0/2099403181 conn(0x7fcb680cb610 msgr2=0x7fcb680cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 shutdown_connections 2023-11-08T19:28:12.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.351+0000 7fcb6d9ee640 1 -- 172.21.15.119:0/2099403181 wait complete. 2023-11-08T19:28:12.372 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:28:12.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.583+0000 7f2dff834640 1 Processor -- start 2023-11-08T19:28:12.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.583+0000 7f2dff834640 1 -- start start 2023-11-08T19:28:12.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dff834640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dff834640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2df806a120 0x7f2df806a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dff834640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df815f900 con 0x7f2df81447a0 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dff834640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df806aab0 con 0x7f2df806a120 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dff834640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80c3a50 con 0x7f2df815f4c0 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dfd5d0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2df81447a0 0x7f2df80cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54566/0 (socket says 172.21.15.119:54566) 2023-11-08T19:28:12.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dfd5d0640 1 -- 172.21.15.119:0/1801970876 learned_addr learned my addr 172.21.15.119:0/1801970876 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:12.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dfddd1640 1 --2- 172.21.15.119:0/1801970876 >> v2:172.21.15.119:3300/0 conn(0x7f2df806a120 0x7f2df806a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.585+0000 7f2dfcdcf640 1 --2- 172.21.15.119:0/1801970876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.586+0000 7f2deffff640 1 -- 172.21.15.119:0/1801970876 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 296650548 0 0) 0x7f2df4001110 con 0x7f2df81447a0 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.586+0000 7f2deffff640 1 -- 172.21.15.119:0/1801970876 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de4003520 con 0x7f2df81447a0 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.586+0000 7f2dfddd1640 1 -- 172.21.15.119:0/1801970876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 msgr2=0x7f2df806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.586+0000 7f2dfddd1640 1 --2- 172.21.15.119:0/1801970876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.586+0000 7f2dfddd1640 1 -- 172.21.15.119:0/1801970876 >> v1:172.21.15.119:6789/0 conn(0x7f2df81447a0 legacy=0x7f2df80cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.586+0000 7f2dfddd1640 1 -- 172.21.15.119:0/1801970876 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8144b80 con 0x7f2df806a120 2023-11-08T19:28:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.587+0000 7f2dfddd1640 1 --2- 172.21.15.119:0/1801970876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df806a120 0x7f2df806a570 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009d90 tx=0x7f2df002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5994113e29a15a0d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.587+0000 7f2deffff640 1 -- 172.21.15.119:0/1801970876 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df002cc50 con 0x7f2df806a120 2023-11-08T19:28:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.587+0000 7f2deffff640 1 -- 172.21.15.119:0/1801970876 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df002cdb0 con 0x7f2df806a120 2023-11-08T19:28:12.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.587+0000 7f2deffff640 1 -- 172.21.15.119:0/1801970876 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df0034660 con 0x7f2df806a120 2023-11-08T19:28:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 -- 172.21.15.119:0/1801970876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df806a120 msgr2=0x7f2df806a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 --2- 172.21.15.119:0/1801970876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df806a120 0x7f2df806a570 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009d90 tx=0x7f2df002bea0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 -- 172.21.15.119:0/1801970876 shutdown_connections 2023-11-08T19:28:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 --2- 172.21.15.119:0/1801970876 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df806a120 0x7f2df806a570 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 --2- 172.21.15.119:0/1801970876 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 -- 172.21.15.119:0/1801970876 >> 172.21.15.119:0/1801970876 conn(0x7f2df80cb610 msgr2=0x7f2df80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 -- 172.21.15.119:0/1801970876 shutdown_connections 2023-11-08T19:28:12.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.588+0000 7f2dff834640 1 -- 172.21.15.119:0/1801970876 wait complete. 2023-11-08T19:28:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.590+0000 7f2dff834640 1 Processor -- start 2023-11-08T19:28:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.590+0000 7f2dff834640 1 -- start start 2023-11-08T19:28:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dff834640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df806a120 0x7f2df815aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dff834640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 0x7f2df815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dff834640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df80b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dff834640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df815af40 con 0x7f2df81447a0 2023-11-08T19:28:12.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dff834640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df813db10 con 0x7f2df806a120 2023-11-08T19:28:12.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dff834640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df815cc10 con 0x7f2df815f4c0 2023-11-08T19:28:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dfcdcf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 0x7f2df815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dfcdcf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 0x7f2df815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46340/0 (socket says 172.21.15.119:46340) 2023-11-08T19:28:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dfcdcf640 1 -- 172.21.15.119:0/3270997882 learned_addr learned my addr 172.21.15.119:0/3270997882 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.591+0000 7f2dfd5d0640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df806a120 0x7f2df815aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfddd1640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df80b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfcdcf640 1 -- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 msgr2=0x7f2df80b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfcdcf640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df80b3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfcdcf640 1 -- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df806a120 msgr2=0x7f2df815aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfcdcf640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df806a120 0x7f2df815aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfcdcf640 1 -- 172.21.15.119:0/3270997882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df0009a70 con 0x7f2df81447a0 2023-11-08T19:28:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfd5d0640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df806a120 0x7f2df815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dfcdcf640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 0x7f2df815b4d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2de800d800 tx=0x7f2de800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800b6c0 con 0x7f2df81447a0 2023-11-08T19:28:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de800b820 con 0x7f2df81447a0 2023-11-08T19:28:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800bbb0 con 0x7f2df81447a0 2023-11-08T19:28:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df813ddb0 con 0x7f2df81447a0 2023-11-08T19:28:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.592+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df815cef0 con 0x7f2df81447a0 2023-11-08T19:28:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.593+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df80c5960 con 0x7f2df81447a0 2023-11-08T19:28:12.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.595+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2de800bd10 con 0x7f2df81447a0 2023-11-08T19:28:12.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.595+0000 7f2dedffb640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ddc06ae70 0x7f2ddc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.595+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f2de8089d80 con 0x7f2df81447a0 2023-11-08T19:28:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.596+0000 7f2dfd5d0640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ddc06ae70 0x7f2ddc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.596+0000 7f2dfd5d0640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ddc06ae70 0x7f2ddc06d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2df4001370 tx=0x7f2df4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.600+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2de8054c20 con 0x7f2df81447a0 2023-11-08T19:28:12.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.793+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2df806caa0 con 0x7f2df81447a0 2023-11-08T19:28:12.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.794+0000 7f2dedffb640 1 -- 172.21.15.119:0/3270997882 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) v1 ==== 84+0+287 (secure 0 0 0) 0x7f2de8058240 con 0x7f2df81447a0 2023-11-08T19:28:12.795 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:12.796 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-11-08T19:28:12.796 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:12.796 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-11-08T19:28:12.796 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 7, 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "snap-test-pool" 2023-11-08T19:28:12.797 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 12, 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "test_data_pool" 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:12.798 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 13, 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "test_metadata_pool" 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:12.799 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-11-08T19:28:12.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ddc06ae70 msgr2=0x7f2ddc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ddc06ae70 0x7f2ddc06d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2df4001370 tx=0x7f2df4009040 comp rx=0 tx=0).stop 2023-11-08T19:28:12.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 msgr2=0x7f2df815b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:12.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 0x7f2df815b4d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2de800d800 tx=0x7f2de800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 shutdown_connections 2023-11-08T19:28:12.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ddc06ae70 0x7f2ddc06d330 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2df815f4c0 0x7f2df80b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2df81447a0 0x7f2df815b4d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 --2- 172.21.15.119:0/3270997882 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2df806a120 0x7f2df815aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:12.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 >> 172.21.15.119:0/3270997882 conn(0x7f2df80cb610 msgr2=0x7f2df8143970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:12.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 shutdown_connections 2023-11-08T19:28:12.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:12.797+0000 7f2dff834640 1 -- 172.21.15.119:0/3270997882 wait complete. 2023-11-08T19:28:12.817 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:28:12.818 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:28:13.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.032+0000 7f507fc00640 1 Processor -- start 2023-11-08T19:28:13.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.032+0000 7f507fc00640 1 -- start start 2023-11-08T19:28:13.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.034+0000 7f507fc00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5078144580 0x7f507815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.034+0000 7f507fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5078144960 0x7f50780cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.034+0000 7f507fc00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507815ea60 con 0x7f507815f1d0 2023-11-08T19:28:13.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.034+0000 7f507fc00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50780c9150 con 0x7f5078144580 2023-11-08T19:28:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.034+0000 7f507fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50780cb270 con 0x7f5078144960 2023-11-08T19:28:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.035+0000 7f507d99c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5078144580 0x7f507815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.035+0000 7f507d99c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5078144580 0x7f507815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:56662/0 (socket says 172.21.15.119:56662) 2023-11-08T19:28:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.035+0000 7f507e19d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f507815f1d0 0x7f50780c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:34696/0 (socket says 172.21.15.119:34696) 2023-11-08T19:28:13.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.035+0000 7f507d99c640 1 -- 172.21.15.119:0/1633845413 learned_addr learned my addr 172.21.15.119:0/1633845413 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:13.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.035+0000 7f507d19b640 1 --2- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5078144960 0x7f50780cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.036+0000 7f507d99c640 1 -- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5078144960 msgr2=0x7f50780cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:13.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.036+0000 7f507d99c640 1 --2- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5078144960 0x7f50780cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.036+0000 7f507d99c640 1 -- 172.21.15.119:0/1633845413 >> v1:172.21.15.188:6789/0 conn(0x7f507815f1d0 legacy=0x7f50780c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.036+0000 7f507d99c640 1 -- 172.21.15.119:0/1633845413 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780c9360 con 0x7f5078144580 2023-11-08T19:28:13.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.037+0000 7f507d99c640 1 --2- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507815bff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5074009870 tx=0x7f507402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43a7b196de5aa4ff server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.038+0000 7f507c99a640 1 -- 172.21.15.119:0/1633845413 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074031640 con 0x7f5078144580 2023-11-08T19:28:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.038+0000 7f507c99a640 1 -- 172.21.15.119:0/1633845413 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50740317a0 con 0x7f5078144580 2023-11-08T19:28:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.038+0000 7f507c99a640 1 -- 172.21.15.119:0/1633845413 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074031b30 con 0x7f5078144580 2023-11-08T19:28:13.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.038+0000 7f507fc00640 1 -- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 msgr2=0x7f507815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.038+0000 7f507fc00640 1 --2- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507815bff0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f5074009870 tx=0x7f507402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.039+0000 7f507fc00640 1 -- 172.21.15.119:0/1633845413 shutdown_connections 2023-11-08T19:28:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.039+0000 7f507fc00640 1 --2- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5078144960 0x7f50780cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.039+0000 7f507fc00640 1 --2- 172.21.15.119:0/1633845413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507815bff0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.039+0000 7f507fc00640 1 -- 172.21.15.119:0/1633845413 >> 172.21.15.119:0/1633845413 conn(0x7f50780cb610 msgr2=0x7f50780cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:13.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.039+0000 7f507fc00640 1 -- 172.21.15.119:0/1633845413 shutdown_connections 2023-11-08T19:28:13.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.039+0000 7f507fc00640 1 -- 172.21.15.119:0/1633845413 wait complete. 2023-11-08T19:28:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.041+0000 7f507fc00640 1 Processor -- start 2023-11-08T19:28:13.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.041+0000 7f507fc00640 1 -- start start 2023-11-08T19:28:13.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.042+0000 7f507fc00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507813c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.042+0000 7f507fc00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 0x7f507815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.042+0000 7f507fc00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f507815f1d0 0x7f507813db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.042+0000 7f507fc00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507813ea00 con 0x7f5078144960 2023-11-08T19:28:13.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.042+0000 7f507fc00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507813ece0 con 0x7f5078144580 2023-11-08T19:28:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.042+0000 7f507fc00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507813efc0 con 0x7f507815f1d0 2023-11-08T19:28:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 0x7f507815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 0x7f507815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46352/0 (socket says 172.21.15.119:46352) 2023-11-08T19:28:13.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 -- 172.21.15.119:0/2947769837 learned_addr learned my addr 172.21.15.119:0/2947769837 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507e19d640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f507815f1d0 0x7f507813db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d99c640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507813c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 -- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f507815f1d0 msgr2=0x7f507813db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f507815f1d0 0x7f507813db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 -- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 msgr2=0x7f507813c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507813c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.043+0000 7f507d19b640 1 -- 172.21.15.119:0/2947769837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740094d0 con 0x7f5078144960 2023-11-08T19:28:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.044+0000 7f507d19b640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 0x7f507815a1e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f506800e9b0 tx=0x7f506800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.044+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5068009670 con 0x7f5078144960 2023-11-08T19:28:13.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.044+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506800c9a0 con 0x7f5078144960 2023-11-08T19:28:13.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.044+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506800cd50 con 0x7f5078144960 2023-11-08T19:28:13.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.044+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507813f2a0 con 0x7f5078144960 2023-11-08T19:28:13.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.044+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507813c5c0 con 0x7f5078144960 2023-11-08T19:28:13.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.045+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5078074430 con 0x7f5078144960 2023-11-08T19:28:13.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.046+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5068013470 con 0x7f5078144960 2023-11-08T19:28:13.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.047+0000 7f506e7fc640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f505c06ae70 0x7f505c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.047+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6240+0+0 (secure 0 0 0) 0x7f5068089ad0 con 0x7f5078144960 2023-11-08T19:28:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.047+0000 7f507d99c640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f505c06ae70 0x7f505c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.048+0000 7f507d99c640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f505c06ae70 0x7f505c06d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5074036000 tx=0x7f507403c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.051+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5068054970 con 0x7f5078144960 2023-11-08T19:28:13.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.237+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f507806cb10 con 0x7f5078144960 2023-11-08T19:28:13.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.699+0000 7f506e7fc640 1 -- 172.21.15.119:0/2947769837 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) v1 ==== 138+0+0 (secure 0 0 0) 0x7f50680170b0 con 0x7f5078144960 2023-11-08T19:28:13.701 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-11-08T19:28:13.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.703+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f505c06ae70 msgr2=0x7f505c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.703+0000 7f507fc00640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f505c06ae70 0x7f505c06d330 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5074036000 tx=0x7f507403c040 comp rx=0 tx=0).stop 2023-11-08T19:28:13.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.703+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 msgr2=0x7f507815a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.703+0000 7f507fc00640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 0x7f507815a1e0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f506800e9b0 tx=0x7f506800ee70 comp rx=0 tx=0).stop 2023-11-08T19:28:13.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 shutdown_connections 2023-11-08T19:28:13.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f505c06ae70 0x7f505c06d330 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f507815f1d0 0x7f507813db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5078144960 0x7f507815a1e0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 --2- 172.21.15.119:0/2947769837 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5078144580 0x7f507813c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 >> 172.21.15.119:0/2947769837 conn(0x7f50780cb610 msgr2=0x7f50780cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:13.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 shutdown_connections 2023-11-08T19:28:13.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.704+0000 7f507fc00640 1 -- 172.21.15.119:0/2947769837 wait complete. 2023-11-08T19:28:13.731 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:28:13.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.927+0000 7f7bd4125640 1 Processor -- start 2023-11-08T19:28:13.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.927+0000 7f7bd4125640 1 -- start start 2023-11-08T19:28:13.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.928+0000 7f7bd4125640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7bcc144550 0x7f7bcc0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd4125640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0cbd20 con 0x7f7bcc15bce0 2023-11-08T19:28:13.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd4125640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bcc0c43b0 con 0x7f7bcc0cb8c0 2023-11-08T19:28:13.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd4125640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bcc15c150 con 0x7f7bcc144550 2023-11-08T19:28:13.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd26c2640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7bcc15bce0 0x7f7bcc0c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54572/0 (socket says 172.21.15.119:54572) 2023-11-08T19:28:13.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd1ec1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7bcc144550 0x7f7bcc0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd26c2640 1 -- 172.21.15.119:0/3676798447 learned_addr learned my addr 172.21.15.119:0/3676798447 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:13.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.929+0000 7f7bd0ebf640 1 -- 172.21.15.119:0/3676798447 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099865795 0 0) 0x7f7bcc0cbd20 con 0x7f7bcc15bce0 2023-11-08T19:28:13.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.930+0000 7f7bd0ebf640 1 -- 172.21.15.119:0/3676798447 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb8003520 con 0x7f7bcc15bce0 2023-11-08T19:28:13.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.930+0000 7f7bd1ec1640 1 -- 172.21.15.119:0/3676798447 >> v1:172.21.15.188:6789/0 conn(0x7f7bcc0cb8c0 legacy=0x7f7bcc15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:13.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.930+0000 7f7bd1ec1640 1 -- 172.21.15.119:0/3676798447 >> v1:172.21.15.119:6789/0 conn(0x7f7bcc15bce0 legacy=0x7f7bcc0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.930+0000 7f7bd1ec1640 1 -- 172.21.15.119:0/3676798447 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc15c340 con 0x7f7bcc144550 2023-11-08T19:28:13.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.931+0000 7f7bd1ec1640 1 --2- 172.21.15.119:0/3676798447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0cb380 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80097a0 tx=0x7f7bc802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f182226367ff14 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.931+0000 7f7bd0ebf640 1 -- 172.21.15.119:0/3676798447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8030610 con 0x7f7bcc144550 2023-11-08T19:28:13.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.931+0000 7f7bd0ebf640 1 -- 172.21.15.119:0/3676798447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8030770 con 0x7f7bcc144550 2023-11-08T19:28:13.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.931+0000 7f7bd0ebf640 1 -- 172.21.15.119:0/3676798447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8030ae0 con 0x7f7bcc144550 2023-11-08T19:28:13.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 -- 172.21.15.119:0/3676798447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 msgr2=0x7f7bcc0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 --2- 172.21.15.119:0/3676798447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0cb380 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f7bc80097a0 tx=0x7f7bc802b960 comp rx=0 tx=0).stop 2023-11-08T19:28:13.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 -- 172.21.15.119:0/3676798447 shutdown_connections 2023-11-08T19:28:13.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 --2- 172.21.15.119:0/3676798447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0cb380 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 -- 172.21.15.119:0/3676798447 >> 172.21.15.119:0/3676798447 conn(0x7f7bcc0b0950 msgr2=0x7f7bcc0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:13.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 -- 172.21.15.119:0/3676798447 shutdown_connections 2023-11-08T19:28:13.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.932+0000 7f7bd4125640 1 -- 172.21.15.119:0/3676798447 wait complete. 2023-11-08T19:28:13.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.934+0000 7f7bd4125640 1 Processor -- start 2023-11-08T19:28:13.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.934+0000 7f7bd4125640 1 -- start start 2023-11-08T19:28:13.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.935+0000 7f7bd4125640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bcc0cb8c0 0x7f7bcc1395c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.935+0000 7f7bd4125640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.935+0000 7f7bd4125640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc15bce0 0x7f7bcc0b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.935+0000 7f7bd4125640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc13d9c0 con 0x7f7bcc144550 2023-11-08T19:28:13.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.935+0000 7f7bd4125640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc13f400 con 0x7f7bcc15bce0 2023-11-08T19:28:13.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.935+0000 7f7bd4125640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc13d0f0 con 0x7f7bcc0cb8c0 2023-11-08T19:28:13.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46364/0 (socket says 172.21.15.119:46364) 2023-11-08T19:28:13.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 -- 172.21.15.119:0/1841580982 learned_addr learned my addr 172.21.15.119:0/1841580982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd1ec1640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bcc0cb8c0 0x7f7bcc1395c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd26c2640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc15bce0 0x7f7bcc0b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 -- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bcc0cb8c0 msgr2=0x7f7bcc1395c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bcc0cb8c0 0x7f7bcc1395c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 -- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc15bce0 msgr2=0x7f7bcc0b2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:13.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc15bce0 0x7f7bcc0b2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:13.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 -- 172.21.15.119:0/1841580982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc8009400 con 0x7f7bcc144550 2023-11-08T19:28:13.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd1ec1640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bcc0cb8c0 0x7f7bcc1395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:13.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd26c2640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc15bce0 0x7f7bcc0b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:13.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.936+0000 7f7bd16c0640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0b24e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00c400 tx=0x7f7bbc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.937+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0052d0 con 0x7f7bcc144550 2023-11-08T19:28:13.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.937+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc005450 con 0x7f7bcc144550 2023-11-08T19:28:13.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.937+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc007260 con 0x7f7bcc144550 2023-11-08T19:28:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.937+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc13ed80 con 0x7f7bcc144550 2023-11-08T19:28:13.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.937+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc13ce10 con 0x7f7bcc144550 2023-11-08T19:28:13.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.938+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc04ece0 con 0x7f7bcc144550 2023-11-08T19:28:13.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.939+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7bbc0073c0 con 0x7f7bcc144550 2023-11-08T19:28:13.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.940+0000 7f7bc27fc640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7bb006ae70 0x7f7bb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:13.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.940+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6610+0+0 (secure 0 0 0) 0x7f7bbc088de0 con 0x7f7bcc144550 2023-11-08T19:28:13.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.940+0000 7f7bd1ec1640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7bb006ae70 0x7f7bb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:13.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.941+0000 7f7bd1ec1640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7bb006ae70 0x7f7bb006d330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8036000 tx=0x7f7bc803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:13.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:13.945+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7bbc053af0 con 0x7f7bcc144550 2023-11-08T19:28:14.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.135+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7bcc0b0fe0 con 0x7f7bcc144550 2023-11-08T19:28:14.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.715+0000 7f7bc27fc640 1 -- 172.21.15.119:0/1841580982 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7bbc057110 con 0x7f7bcc144550 2023-11-08T19:28:14.717 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-11-08T19:28:14.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.719+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7bb006ae70 msgr2=0x7f7bb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.719+0000 7f7bd4125640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7bb006ae70 0x7f7bb006d330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7bc8036000 tx=0x7f7bc803b040 comp rx=0 tx=0).stop 2023-11-08T19:28:14.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.719+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 msgr2=0x7f7bcc0b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.719+0000 7f7bd4125640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0b24e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00c400 tx=0x7f7bbc00c8c0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.720+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 shutdown_connections 2023-11-08T19:28:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.720+0000 7f7bd4125640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7bb006ae70 0x7f7bb006d330 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.720+0000 7f7bd4125640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7bcc15bce0 0x7f7bcc0b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.720+0000 7f7bd4125640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7bcc144550 0x7f7bcc0b24e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.720+0000 7f7bd4125640 1 --2- 172.21.15.119:0/1841580982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7bcc0cb8c0 0x7f7bcc1395c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.720+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 >> 172.21.15.119:0/1841580982 conn(0x7f7bcc0b0950 msgr2=0x7f7bcc06b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.721+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 shutdown_connections 2023-11-08T19:28:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.722+0000 7f7bd4125640 1 -- 172.21.15.119:0/1841580982 wait complete. 2023-11-08T19:28:14.758 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:28:14.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.943+0000 7f744d933640 1 Processor -- start 2023-11-08T19:28:14.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.944+0000 7f744d933640 1 -- start start 2023-11-08T19:28:14.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.945+0000 7f744d933640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448144960 0x7f744815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.945+0000 7f744d933640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f744815f1d0 0x7f74480c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.945+0000 7f744d933640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744815ea60 con 0x7f7448144580 2023-11-08T19:28:14.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.945+0000 7f744d933640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74480c9150 con 0x7f744815f1d0 2023-11-08T19:28:14.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.945+0000 7f744d933640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c9360 con 0x7f7448144960 2023-11-08T19:28:14.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.946+0000 7f7446ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7448144580 0x7f74480cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54586/0 (socket says 172.21.15.119:54586) 2023-11-08T19:28:14.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.946+0000 7f7446ffd640 1 -- 172.21.15.119:0/1367049792 learned_addr learned my addr 172.21.15.119:0/1367049792 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:14.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.946+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450149607 0 0) 0x7f744815ea60 con 0x7f7448144580 2023-11-08T19:28:14.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.946+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7434003520 con 0x7f7448144580 2023-11-08T19:28:14.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.946+0000 7f743e7fc640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448144960 0x7f744815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.946+0000 7f74477fe640 1 --2- 172.21.15.119:0/1367049792 >> v2:172.21.15.188:3300/0 conn(0x7f744815f1d0 0x7f74480c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070326177 0 0) 0x7f7434003520 con 0x7f7448144580 2023-11-08T19:28:14.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448144960 msgr2=0x7f744815bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448144960 0x7f744815bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 >> v2:172.21.15.188:3300/0 conn(0x7f744815f1d0 msgr2=0x7f74480c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 --2- 172.21.15.119:0/1367049792 >> v2:172.21.15.188:3300/0 conn(0x7f744815f1d0 0x7f74480c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74480c9640 con 0x7f7448144580 2023-11-08T19:28:14.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f7438001670 con 0x7f7448144580 2023-11-08T19:28:14.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 >> v1:172.21.15.119:6789/0 conn(0x7f7448144580 legacy=0x7f74480cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f7434003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74340047c0 0x7f7434004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434005180 0x7f743400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c9150 con 0x7f7434003970 2023-11-08T19:28:14.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c9360 con 0x7f74340047c0 2023-11-08T19:28:14.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743400c080 con 0x7f7434005180 2023-11-08T19:28:14.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f74477fe640 1 --2- 172.21.15.119:0/1367049792 >> v2:172.21.15.188:3300/0 conn(0x7f744815f1d0 0x7f74480c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f743e7fc640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448144960 0x7f744815bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:14.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f7446ffd640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f7434003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.947+0000 7f7446ffd640 1 -- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434005180 msgr2=0x7f743400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.948+0000 7f7446ffd640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7434005180 0x7f743400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.948+0000 7f7446ffd640 1 -- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74340047c0 msgr2=0x7f7434004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:14.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.948+0000 7f7446ffd640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74340047c0 0x7f7434004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.948+0000 7f7446ffd640 1 -- 172.21.15.119:0/1367049792 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434003520 con 0x7f7434003970 2023-11-08T19:28:14.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f7446ffd640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f7434003db0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f743800c210 tx=0x7f74340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef295d6d4cc7d19c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438005ad0 con 0x7f7434003970 2023-11-08T19:28:14.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7438005c30 con 0x7f7434003970 2023-11-08T19:28:14.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f74467fc640 1 -- 172.21.15.119:0/1367049792 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743802c8d0 con 0x7f7434003970 2023-11-08T19:28:14.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f744d933640 1 -- 172.21.15.119:0/1367049792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 msgr2=0x7f7434003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f744d933640 1 --2- 172.21.15.119:0/1367049792 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f7434003db0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f743800c210 tx=0x7f74340057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.949+0000 7f74477fe640 1 -- 172.21.15.119:0/1367049792 reap_dead start 2023-11-08T19:28:14.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.950+0000 7f744d933640 1 -- 172.21.15.119:0/1367049792 shutdown_connections 2023-11-08T19:28:14.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.950+0000 7f744d933640 1 -- 172.21.15.119:0/1367049792 >> 172.21.15.119:0/1367049792 conn(0x7f74480cb610 msgr2=0x7f74480cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:14.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.950+0000 7f744d933640 1 -- 172.21.15.119:0/1367049792 shutdown_connections 2023-11-08T19:28:14.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.950+0000 7f744d933640 1 -- 172.21.15.119:0/1367049792 wait complete. 2023-11-08T19:28:14.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.951+0000 7f744d933640 1 Processor -- start 2023-11-08T19:28:14.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 -- start start 2023-11-08T19:28:14.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f744813b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74480c71e0 0x7f744815a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448070980 0x7f7448070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448071340 con 0x7f7434003970 2023-11-08T19:28:14.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c8a20 con 0x7f74480c71e0 2023-11-08T19:28:14.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.952+0000 7f744d933640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74480c84b0 con 0x7f7448070980 2023-11-08T19:28:14.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f744813b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f744813b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46374/0 (socket says 172.21.15.119:46374) 2023-11-08T19:28:14.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 -- 172.21.15.119:0/2687738783 learned_addr learned my addr 172.21.15.119:0/2687738783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:14.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 -- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448070980 msgr2=0x7f7448070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448070980 0x7f7448070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 -- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74480c71e0 msgr2=0x7f744815a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:14.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f743e7fc640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74480c71e0 0x7f744815a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74480c71e0 0x7f744815a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:14.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 -- 172.21.15.119:0/2687738783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744815adc0 con 0x7f7434003970 2023-11-08T19:28:14.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.953+0000 7f7446ffd640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f744813b7d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f743800c210 tx=0x7f744806cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.954+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7438039070 con 0x7f7434003970 2023-11-08T19:28:14.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.954+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743802cce0 con 0x7f7434003970 2023-11-08T19:28:14.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.954+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743803c040 con 0x7f7434003970 2023-11-08T19:28:14.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.954+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74480c7660 con 0x7f7434003970 2023-11-08T19:28:14.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.954+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74480c7c10 con 0x7f7434003970 2023-11-08T19:28:14.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.955+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744806cb10 con 0x7f7434003970 2023-11-08T19:28:14.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.956+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7438032070 con 0x7f7434003970 2023-11-08T19:28:14.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.958+0000 7f7444ff9640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f742806aea0 0x7f742806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:14.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.958+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6992+0+0 (secure 0 0 0) 0x7f74380aaee0 con 0x7f7434003970 2023-11-08T19:28:14.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.958+0000 7f743e7fc640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f742806aea0 0x7f742806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:14.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.958+0000 7f743e7fc640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f742806aea0 0x7f742806d360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7430006a90 tx=0x7f7430008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:14.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:14.963+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7438079530 con 0x7f7434003970 2023-11-08T19:28:15.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.189+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f74481424b0 con 0x7f7434003970 2023-11-08T19:28:15.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.720+0000 7f7444ff9640 1 -- 172.21.15.119:0/2687738783 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v40) v1 ==== 174+0+0 (secure 0 0 0) 0x7f74380790c0 con 0x7f7434003970 2023-11-08T19:28:15.722 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 14 and data pool 15 2023-11-08T19:28:15.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f742806aea0 msgr2=0x7f742806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f742806aea0 0x7f742806d360 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7430006a90 tx=0x7f7430008040 comp rx=0 tx=0).stop 2023-11-08T19:28:15.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 msgr2=0x7f744813b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f744813b7d0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f743800c210 tx=0x7f744806cd30 comp rx=0 tx=0).stop 2023-11-08T19:28:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 shutdown_connections 2023-11-08T19:28:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f742806aea0 0x7f742806d360 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7448070980 0x7f7448070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74480c71e0 0x7f744815a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 --2- 172.21.15.119:0/2687738783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7434003970 0x7f744813b7d0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 >> 172.21.15.119:0/2687738783 conn(0x7f74480cb610 msgr2=0x7f744806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:15.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.724+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 shutdown_connections 2023-11-08T19:28:15.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.725+0000 7f744d933640 1 -- 172.21.15.119:0/2687738783 wait complete. 2023-11-08T19:28:15.768 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:15.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.949+0000 7f450a87c640 1 Processor -- start 2023-11-08T19:28:15.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.950+0000 7f450a87c640 1 -- start start 2023-11-08T19:28:15.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.951+0000 7f450a87c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f45040cc810 0x7f45040c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.951+0000 7f450a87c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f450415d180 0x7f450406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.951+0000 7f450a87c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040ae410 con 0x7f450406a390 2023-11-08T19:28:15.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.951+0000 7f450a87c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450406ba30 con 0x7f450415d180 2023-11-08T19:28:15.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.951+0000 7f450a87c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450406bcf0 con 0x7f45040cc810 2023-11-08T19:28:15.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.951+0000 7f4508e19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f450415d180 0x7f450406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f4508e19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f450415d180 0x7f450406cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46378/0 (socket says 172.21.15.119:46378) 2023-11-08T19:28:15.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f4508e19640 1 -- 172.21.15.119:0/1933709832 learned_addr learned my addr 172.21.15.119:0/1933709832 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:15.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f44fbfff640 1 --2- 172.21.15.119:0/1933709832 >> v2:172.21.15.188:3300/0 conn(0x7f45040cc810 0x7f45040c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f4508e19640 1 -- 172.21.15.119:0/1933709832 >> v2:172.21.15.188:3300/0 conn(0x7f45040cc810 msgr2=0x7f45040c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:15.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f4508e19640 1 --2- 172.21.15.119:0/1933709832 >> v2:172.21.15.188:3300/0 conn(0x7f45040cc810 0x7f45040c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f4508e19640 1 -- 172.21.15.119:0/1933709832 >> v1:172.21.15.188:6789/0 conn(0x7f450406a390 legacy=0x7f450415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:15.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.952+0000 7f4508e19640 1 -- 172.21.15.119:0/1933709832 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450406bfd0 con 0x7f450415d180 2023-11-08T19:28:15.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.953+0000 7f4508e19640 1 --2- 172.21.15.119:0/1933709832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450415d180 0x7f450406cf70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f45040b1850 tx=0x7f44fc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acad2f8109617c9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:15.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.953+0000 7f45037fe640 1 -- 172.21.15.119:0/1933709832 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc02ca80 con 0x7f450415d180 2023-11-08T19:28:15.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f45037fe640 1 -- 172.21.15.119:0/1933709832 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc02cbe0 con 0x7f450415d180 2023-11-08T19:28:15.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f45037fe640 1 -- 172.21.15.119:0/1933709832 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0343f0 con 0x7f450415d180 2023-11-08T19:28:15.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 -- 172.21.15.119:0/1933709832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450415d180 msgr2=0x7f450406cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 --2- 172.21.15.119:0/1933709832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450415d180 0x7f450406cf70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f45040b1850 tx=0x7f44fc02bee0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 -- 172.21.15.119:0/1933709832 shutdown_connections 2023-11-08T19:28:15.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 --2- 172.21.15.119:0/1933709832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450415d180 0x7f450406cf70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 --2- 172.21.15.119:0/1933709832 >> v2:172.21.15.188:3300/0 conn(0x7f45040cc810 0x7f45040c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 -- 172.21.15.119:0/1933709832 >> 172.21.15.119:0/1933709832 conn(0x7f45040cb610 msgr2=0x7f45040cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:15.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 -- 172.21.15.119:0/1933709832 shutdown_connections 2023-11-08T19:28:15.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.954+0000 7f450a87c640 1 -- 172.21.15.119:0/1933709832 wait complete. 2023-11-08T19:28:15.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.956+0000 7f450a87c640 1 Processor -- start 2023-11-08T19:28:15.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.956+0000 7f450a87c640 1 -- start start 2023-11-08T19:28:15.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.957+0000 7f450a87c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 0x7f450414d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.957+0000 7f450a87c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45040cc810 0x7f45040af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.957+0000 7f450a87c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f450415d180 0x7f450415f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.957+0000 7f450a87c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450414d790 con 0x7f450406a390 2023-11-08T19:28:15.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.957+0000 7f450a87c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040b0180 con 0x7f450415d180 2023-11-08T19:28:15.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.957+0000 7f450a87c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040b0460 con 0x7f45040cc810 2023-11-08T19:28:15.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 0x7f450414d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 0x7f450414d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46392/0 (socket says 172.21.15.119:46392) 2023-11-08T19:28:15.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 -- 172.21.15.119:0/705244442 learned_addr learned my addr 172.21.15.119:0/705244442 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:15.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 -- 172.21.15.119:0/705244442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45040cc810 msgr2=0x7f45040af7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45040cc810 0x7f45040af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 -- 172.21.15.119:0/705244442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f450415d180 msgr2=0x7f450415f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:15.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f450415d180 0x7f450415f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:15.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 -- 172.21.15.119:0/705244442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc009b30 con 0x7f450406a390 2023-11-08T19:28:15.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.958+0000 7f4503fff640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 0x7f450414d250 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f44f400c410 tx=0x7f44f400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:15.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.959+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f40035f0 con 0x7f450406a390 2023-11-08T19:28:15.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.959+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f4003750 con 0x7f450406a390 2023-11-08T19:28:15.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.959+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45040b0740 con 0x7f450406a390 2023-11-08T19:28:15.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.959+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f4003b40 con 0x7f450406a390 2023-11-08T19:28:15.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.959+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45040b0d90 con 0x7f450406a390 2023-11-08T19:28:15.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.960+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450415a0d0 con 0x7f450406a390 2023-11-08T19:28:15.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.961+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44f4003ca0 con 0x7f450406a390 2023-11-08T19:28:15.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.962+0000 7f45017fa640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44e406ae70 0x7f44e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:15.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.962+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f44f4088f10 con 0x7f450406a390 2023-11-08T19:28:15.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.962+0000 7f44fbfff640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44e406ae70 0x7f44e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:15.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.963+0000 7f44fbfff640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44e406ae70 0x7f44e406d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44ec006fd0 tx=0x7f44ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:15.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:15.967+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44f40539b0 con 0x7f450406a390 2023-11-08T19:28:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.150+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4504074430 con 0x7f450406a390 2023-11-08T19:28:16.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.152+0000 7f45017fa640 1 -- 172.21.15.119:0/705244442 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21966 (secure 0 0 0) 0x7f44f4017090 con 0x7f450406a390 2023-11-08T19:28:16.154 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:16.154 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:15.190926+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:16.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44e406ae70 msgr2=0x7f44e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44e406ae70 0x7f44e406d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f44ec006fd0 tx=0x7f44ec008040 comp rx=0 tx=0).stop 2023-11-08T19:28:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 msgr2=0x7f450414d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 0x7f450414d250 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f44f400c410 tx=0x7f44f400c8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 shutdown_connections 2023-11-08T19:28:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f44e406ae70 0x7f44e406d330 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f450415d180 0x7f450415f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f45040cc810 0x7f45040af7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 --2- 172.21.15.119:0/705244442 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f450406a390 0x7f450414d250 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 >> 172.21.15.119:0/705244442 conn(0x7f45040cb610 msgr2=0x7f45040ad150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 shutdown_connections 2023-11-08T19:28:16.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.154+0000 7f450a87c640 1 -- 172.21.15.119:0/705244442 wait complete. 2023-11-08T19:28:16.172 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.379+0000 7f9d4d6a3640 1 Processor -- start 2023-11-08T19:28:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.379+0000 7f9d4d6a3640 1 -- start start 2023-11-08T19:28:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d4d6a3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d480adf20 0x7f9d4815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d4d6a3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d480ca2e0 con 0x7f9d480cac30 2023-11-08T19:28:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d4d6a3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d480ca5c0 con 0x7f9d480ad9d0 2023-11-08T19:28:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d4d6a3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d48071c70 con 0x7f9d480adf20 2023-11-08T19:28:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d3ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9d480adf20 0x7f9d4815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d47fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9d480cac30 0x7f9d4815f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54590/0 (socket says 172.21.15.119:54590) 2023-11-08T19:28:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.381+0000 7f9d47fff640 1 -- 172.21.15.119:0/1782376049 learned_addr learned my addr 172.21.15.119:0/1782376049 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:16.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.382+0000 7f9d3ffff640 1 -- 172.21.15.119:0/1782376049 >> v1:172.21.15.188:6789/0 conn(0x7f9d480ad9d0 legacy=0x7f9d4806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.382+0000 7f9d3ffff640 1 -- 172.21.15.119:0/1782376049 >> v1:172.21.15.119:6789/0 conn(0x7f9d480cac30 legacy=0x7f9d4815f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.382+0000 7f9d3ffff640 1 -- 172.21.15.119:0/1782376049 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4815eb80 con 0x7f9d480adf20 2023-11-08T19:28:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.383+0000 7f9d3ffff640 1 --2- 172.21.15.119:0/1782376049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480adf20 0x7f9d4815a910 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d300097a0 tx=0x7f9d3002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c59e4753222982 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.383+0000 7f9d477fe640 1 -- 172.21.15.119:0/1782376049 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30031830 con 0x7f9d480adf20 2023-11-08T19:28:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.383+0000 7f9d477fe640 1 -- 172.21.15.119:0/1782376049 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30031990 con 0x7f9d480adf20 2023-11-08T19:28:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.383+0000 7f9d477fe640 1 -- 172.21.15.119:0/1782376049 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30031d20 con 0x7f9d480adf20 2023-11-08T19:28:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/1782376049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480adf20 msgr2=0x7f9d4815a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/1782376049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480adf20 0x7f9d4815a910 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9d300097a0 tx=0x7f9d3002b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/1782376049 shutdown_connections 2023-11-08T19:28:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/1782376049 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480adf20 0x7f9d4815a910 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/1782376049 >> 172.21.15.119:0/1782376049 conn(0x7f9d480b0560 msgr2=0x7f9d480b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/1782376049 shutdown_connections 2023-11-08T19:28:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.384+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/1782376049 wait complete. 2023-11-08T19:28:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.386+0000 7f9d4d6a3640 1 Processor -- start 2023-11-08T19:28:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.386+0000 7f9d4d6a3640 1 -- start start 2023-11-08T19:28:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4d6a3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d480ad9d0 0x7f9d48158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4d6a3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d480adf20 0x7f9d48159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4d6a3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 0x7f9d48158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4d6a3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d48158870 con 0x7f9d480cac30 2023-11-08T19:28:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4d6a3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480ccea0 con 0x7f9d480ad9d0 2023-11-08T19:28:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4d6a3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d480cd160 con 0x7f9d480adf20 2023-11-08T19:28:16.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 0x7f9d48158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 0x7f9d48158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46424/0 (socket says 172.21.15.119:46424) 2023-11-08T19:28:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 -- 172.21.15.119:0/2670875661 learned_addr learned my addr 172.21.15.119:0/2670875661 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 -- 172.21.15.119:0/2670875661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d480adf20 msgr2=0x7f9d48159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d480adf20 0x7f9d48159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 -- 172.21.15.119:0/2670875661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d480ad9d0 msgr2=0x7f9d48158a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d480ad9d0 0x7f9d48158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.387+0000 7f9d4cea2640 1 -- 172.21.15.119:0/2670875661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d30009400 con 0x7f9d480cac30 2023-11-08T19:28:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.388+0000 7f9d4cea2640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 0x7f9d48158330 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000c6d0 tx=0x7f9d4000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.388+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000a840 con 0x7f9d480cac30 2023-11-08T19:28:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.388+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4000a9a0 con 0x7f9d480cac30 2023-11-08T19:28:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.388+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000ad90 con 0x7f9d480cac30 2023-11-08T19:28:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.388+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d480cd440 con 0x7f9d480cac30 2023-11-08T19:28:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.388+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d480c75d0 con 0x7f9d480cac30 2023-11-08T19:28:16.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.389+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d480b3e50 con 0x7f9d480cac30 2023-11-08T19:28:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.390+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d40012630 con 0x7f9d480cac30 2023-11-08T19:28:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.391+0000 7f9d457fa640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d2806ae70 0x7f9d2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.391+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f9d4008a260 con 0x7f9d480cac30 2023-11-08T19:28:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.391+0000 7f9d47fff640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d2806ae70 0x7f9d2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.392+0000 7f9d47fff640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d2806ae70 0x7f9d2806d330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b2720 tx=0x7f9d38007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.396+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d40054d90 con 0x7f9d480cac30 2023-11-08T19:28:16.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.586+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d480cd630 con 0x7f9d480cac30 2023-11-08T19:28:16.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.589+0000 7f9d457fa640 1 -- 172.21.15.119:0/2670875661 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+21966 (secure 0 0 0) 0x7f9d40016090 con 0x7f9d480cac30 2023-11-08T19:28:16.591 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:16.591 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:15.190926+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:16.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.591+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d2806ae70 msgr2=0x7f9d2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.591+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d2806ae70 0x7f9d2806d330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9d480b2720 tx=0x7f9d38007a40 comp rx=0 tx=0).stop 2023-11-08T19:28:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 msgr2=0x7f9d48158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 0x7f9d48158330 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000c6d0 tx=0x7f9d4000cb90 comp rx=0 tx=0).stop 2023-11-08T19:28:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 shutdown_connections 2023-11-08T19:28:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d2806ae70 0x7f9d2806d330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d480cac30 0x7f9d48158330 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d480adf20 0x7f9d48159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 --2- 172.21.15.119:0/2670875661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d480ad9d0 0x7f9d48158a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 >> 172.21.15.119:0/2670875661 conn(0x7f9d480b0560 msgr2=0x7f9d480b2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 shutdown_connections 2023-11-08T19:28:16.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.592+0000 7f9d4d6a3640 1 -- 172.21.15.119:0/2670875661 wait complete. 2023-11-08T19:28:16.613 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:28:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.827+0000 7f15c07bb640 1 Processor -- start 2023-11-08T19:28:16.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.828+0000 7f15c07bb640 1 -- start start 2023-11-08T19:28:16.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.829+0000 7f15c07bb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 0x7f15b806a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.829+0000 7f15c07bb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b806dd90 con 0x7f15b806a660 2023-11-08T19:28:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.829+0000 7f15c07bb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b806cff0 con 0x7f15b815b040 2023-11-08T19:28:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.829+0000 7f15c07bb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b806aad0 con 0x7f15b80ca2e0 2023-11-08T19:28:16.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.829+0000 7f15bed58640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f15b806a660 0x7f15b80b2b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:54592/0 (socket says 172.21.15.119:54592) 2023-11-08T19:28:16.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bed58640 1 -- 172.21.15.119:0/2838351172 learned_addr learned my addr 172.21.15.119:0/2838351172 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:16.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bdd56640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 0x7f15b806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120133212 0 0) 0x7f15b806dd90 con 0x7f15b806a660 2023-11-08T19:28:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15a4003520 con 0x7f15b806a660 2023-11-08T19:28:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3939083420 0 0) 0x7f15a4003520 con 0x7f15b806a660 2023-11-08T19:28:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 msgr2=0x7f15b806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 0x7f15b806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 >> v1:172.21.15.188:6789/0 conn(0x7f15b815b040 legacy=0x7f15b8071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b8071cb0 con 0x7f15b806a660 2023-11-08T19:28:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f15b0001670 con 0x7f15b806a660 2023-11-08T19:28:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 >> v1:172.21.15.119:6789/0 conn(0x7f15b806a660 legacy=0x7f15b80b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a40039a0 0x7f15a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 0x7f15a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b806aad0 con 0x7f15a40047c0 2023-11-08T19:28:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a400bac0 con 0x7f15a40039a0 2023-11-08T19:28:16.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.830+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15a400bc80 con 0x7f15b80ca2e0 2023-11-08T19:28:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bdd56640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15be557640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a40039a0 0x7f15a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bed58640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 0x7f15a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bdd56640 1 -- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 msgr2=0x7f15a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bdd56640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b80ca2e0 0x7f15a4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bdd56640 1 -- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a40039a0 msgr2=0x7f15a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bdd56640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a40039a0 0x7f15a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15bdd56640 1 -- 172.21.15.119:0/2838351172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a4003520 con 0x7f15a40047c0 2023-11-08T19:28:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.831+0000 7f15be557640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15a40039a0 0x7f15a4003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:16.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.832+0000 7f15bdd56640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15a4004c40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009f90 tx=0x7f15a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81e80e034c7a0fb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.832+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b402cba0 con 0x7f15a40047c0 2023-11-08T19:28:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.832+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15b400f040 con 0x7f15a40047c0 2023-11-08T19:28:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.832+0000 7f15bd555640 1 -- 172.21.15.119:0/2838351172 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15b4034440 con 0x7f15a40047c0 2023-11-08T19:28:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15c07bb640 1 -- 172.21.15.119:0/2838351172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 msgr2=0x7f15a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15c07bb640 1 --2- 172.21.15.119:0/2838351172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15a4004c40 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009f90 tx=0x7f15a40057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15bed58640 1 -- 172.21.15.119:0/2838351172 reap_dead start 2023-11-08T19:28:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15c07bb640 1 -- 172.21.15.119:0/2838351172 shutdown_connections 2023-11-08T19:28:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15c07bb640 1 -- 172.21.15.119:0/2838351172 >> 172.21.15.119:0/2838351172 conn(0x7f15b806b230 msgr2=0x7f15b806b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15c07bb640 1 -- 172.21.15.119:0/2838351172 shutdown_connections 2023-11-08T19:28:16.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.833+0000 7f15c07bb640 1 -- 172.21.15.119:0/2838351172 wait complete. 2023-11-08T19:28:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.835+0000 7f15c07bb640 1 Processor -- start 2023-11-08T19:28:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.835+0000 7f15c07bb640 1 -- start start 2023-11-08T19:28:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15c07bb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15b815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15c07bb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b814fce0 0x7f15b81536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15c07bb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b815b770 0x7f15b815bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15c07bb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b815c130 con 0x7f15a40047c0 2023-11-08T19:28:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15c07bb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80c7b80 con 0x7f15b814fce0 2023-11-08T19:28:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15c07bb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80c7830 con 0x7f15b815b770 2023-11-08T19:28:16.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15be557640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15b815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15be557640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15b815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46442/0 (socket says 172.21.15.119:46442) 2023-11-08T19:28:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.836+0000 7f15be557640 1 -- 172.21.15.119:0/4257734429 learned_addr learned my addr 172.21.15.119:0/4257734429 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15be557640 1 -- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b815b770 msgr2=0x7f15b815bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15be557640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b815b770 0x7f15b815bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15bdd56640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b814fce0 0x7f15b81536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15be557640 1 -- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b814fce0 msgr2=0x7f15b81536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:16.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15be557640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b814fce0 0x7f15b81536c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:16.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15be557640 1 -- 172.21.15.119:0/4257734429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b815dd90 con 0x7f15a40047c0 2023-11-08T19:28:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15be557640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15b815d790 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00d7d0 tx=0x7f15ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac017070 con 0x7f15a40047c0 2023-11-08T19:28:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ac005070 con 0x7f15a40047c0 2023-11-08T19:28:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ac005440 con 0x7f15a40047c0 2023-11-08T19:28:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b815e070 con 0x7f15a40047c0 2023-11-08T19:28:16.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.837+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b8153c00 con 0x7f15a40047c0 2023-11-08T19:28:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.839+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b80ca0c0 con 0x7f15a40047c0 2023-11-08T19:28:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.839+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15ac007030 con 0x7f15a40047c0 2023-11-08T19:28:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.840+0000 7f15aaffd640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f159c06ae70 0x7f159c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.840+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f15ac0893c0 con 0x7f15a40047c0 2023-11-08T19:28:16.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.841+0000 7f15bdd56640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f159c06ae70 0x7f159c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:16.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.841+0000 7f15bdd56640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f159c06ae70 0x7f159c06d330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f15b4036000 tx=0x7f15b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:16.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:16.845+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15ac053f00 con 0x7f15a40047c0 2023-11-08T19:28:17.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.070+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f15b806cae0 con 0x7f15a40047c0 2023-11-08T19:28:17.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.753+0000 7f15aaffd640 1 -- 172.21.15.119:0/4257734429 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v43) v1 ==== 118+0+0 (secure 0 0 0) 0x7f15ac057520 con 0x7f15a40047c0 2023-11-08T19:28:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f159c06ae70 msgr2=0x7f159c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f159c06ae70 0x7f159c06d330 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f15b4036000 tx=0x7f15b402e000 comp rx=0 tx=0).stop 2023-11-08T19:28:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 msgr2=0x7f15b815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15b815d790 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00d7d0 tx=0x7f15ac00dc90 comp rx=0 tx=0).stop 2023-11-08T19:28:17.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 shutdown_connections 2023-11-08T19:28:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f159c06ae70 0x7f159c06d330 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f15b815b770 0x7f15b815bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f15b814fce0 0x7f15b81536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 --2- 172.21.15.119:0/4257734429 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f15a40047c0 0x7f15b815d790 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 >> 172.21.15.119:0/4257734429 conn(0x7f15b806b230 msgr2=0x7f15b80b4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:17.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 shutdown_connections 2023-11-08T19:28:17.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.757+0000 7f15c07bb640 1 -- 172.21.15.119:0/4257734429 wait complete. 2023-11-08T19:28:17.778 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:28:17.779 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:28:17.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.990+0000 7f252afa5640 1 Processor -- start 2023-11-08T19:28:17.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.990+0000 7f252afa5640 1 -- start start 2023-11-08T19:28:17.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f252afa5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f252415d9f0 0x7f252415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f252afa5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f252406cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:17.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f252afa5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252406d4f0 con 0x7f2524143600 2023-11-08T19:28:17.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f252afa5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25240c4f50 con 0x7f252415d9f0 2023-11-08T19:28:17.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f252afa5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25240c4960 con 0x7f252406cb40 2023-11-08T19:28:17.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f251bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f252415d9f0 0x7f252415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f251bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f252415d9f0 0x7f252415de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46446/0 (socket says 172.21.15.119:46446) 2023-11-08T19:28:17.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.992+0000 7f251bfff640 1 -- 172.21.15.119:0/2314088135 learned_addr learned my addr 172.21.15.119:0/2314088135 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:17.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.993+0000 7f2529542640 1 --2- 172.21.15.119:0/2314088135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f252406cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:17.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.993+0000 7f251bfff640 1 -- 172.21.15.119:0/2314088135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 msgr2=0x7f252406cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:17.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.993+0000 7f251bfff640 1 --2- 172.21.15.119:0/2314088135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f252406cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:17.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.993+0000 7f251bfff640 1 -- 172.21.15.119:0/2314088135 >> v1:172.21.15.188:6789/0 conn(0x7f2524143600 legacy=0x7f25240c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:17.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.993+0000 7f251bfff640 1 -- 172.21.15.119:0/2314088135 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252415e440 con 0x7f252415d9f0 2023-11-08T19:28:17.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.994+0000 7f251bfff640 1 --2- 172.21.15.119:0/2314088135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252415d9f0 0x7f252415de60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f25240c91e0 tx=0x7f251402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcdf3e751581a184 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:17.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.994+0000 7f251b7fe640 1 -- 172.21.15.119:0/2314088135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251402e070 con 0x7f252415d9f0 2023-11-08T19:28:17.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.994+0000 7f251b7fe640 1 -- 172.21.15.119:0/2314088135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25140315a0 con 0x7f252415d9f0 2023-11-08T19:28:17.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.994+0000 7f251b7fe640 1 -- 172.21.15.119:0/2314088135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514031930 con 0x7f252415d9f0 2023-11-08T19:28:17.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.994+0000 7f252afa5640 1 -- 172.21.15.119:0/2314088135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252415d9f0 msgr2=0x7f252415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:17.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.994+0000 7f252afa5640 1 --2- 172.21.15.119:0/2314088135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252415d9f0 0x7f252415de60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f25240c91e0 tx=0x7f251402b840 comp rx=0 tx=0).stop 2023-11-08T19:28:17.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.995+0000 7f252afa5640 1 -- 172.21.15.119:0/2314088135 shutdown_connections 2023-11-08T19:28:18.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.995+0000 7f252afa5640 1 --2- 172.21.15.119:0/2314088135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f252415d9f0 0x7f252415de60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.995+0000 7f252afa5640 1 --2- 172.21.15.119:0/2314088135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f252406cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.995+0000 7f252afa5640 1 -- 172.21.15.119:0/2314088135 >> 172.21.15.119:0/2314088135 conn(0x7f25240cb610 msgr2=0x7f25240cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:18.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.995+0000 7f252afa5640 1 -- 172.21.15.119:0/2314088135 shutdown_connections 2023-11-08T19:28:18.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.995+0000 7f252afa5640 1 -- 172.21.15.119:0/2314088135 wait complete. 2023-11-08T19:28:18.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.996+0000 7f252afa5640 1 Processor -- start 2023-11-08T19:28:18.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.996+0000 7f252afa5640 1 -- start start 2023-11-08T19:28:18.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.997+0000 7f252afa5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f25241f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:18.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.997+0000 7f252afa5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 0x7f25241349f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:18.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.997+0000 7f252afa5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252415d9f0 0x7f252415c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:18.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.997+0000 7f252afa5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524134730 con 0x7f2524143600 2023-11-08T19:28:18.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.997+0000 7f252afa5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2524136a50 con 0x7f252415d9f0 2023-11-08T19:28:18.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.997+0000 7f252afa5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25241f39e0 con 0x7f252406cb40 2023-11-08T19:28:18.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 0x7f25241349f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:18.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 0x7f25241349f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46462/0 (socket says 172.21.15.119:46462) 2023-11-08T19:28:18.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 -- 172.21.15.119:0/3188060873 learned_addr learned my addr 172.21.15.119:0/3188060873 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:18.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 -- 172.21.15.119:0/3188060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 msgr2=0x7f25241f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:18.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f25241f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 -- 172.21.15.119:0/3188060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252415d9f0 msgr2=0x7f252415c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:18.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252415d9f0 0x7f252415c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 -- 172.21.15.119:0/3188060873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140094d0 con 0x7f2524143600 2023-11-08T19:28:18.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f251bfff640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 0x7f25241349f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f251402b880 tx=0x7f25140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:18.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.998+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140328c0 con 0x7f2524143600 2023-11-08T19:28:18.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.999+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25241f6540 con 0x7f2524143600 2023-11-08T19:28:18.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.999+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2514032a40 con 0x7f2524143600 2023-11-08T19:28:18.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.999+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2514032df0 con 0x7f2524143600 2023-11-08T19:28:18.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:17.999+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25241f6b00 con 0x7f2524143600 2023-11-08T19:28:18.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.000+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2524074430 con 0x7f2524143600 2023-11-08T19:28:18.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.001+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f251403b3f0 con 0x7f2524143600 2023-11-08T19:28:18.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.001+0000 7f25197fa640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f251006ae70 0x7f251006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:18.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.002+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f25140abd30 con 0x7f2524143600 2023-11-08T19:28:18.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.002+0000 7f2528d41640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f251006ae70 0x7f251006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:18.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.002+0000 7f2528d41640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f251006ae70 0x7f251006d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2520003cc0 tx=0x7f25200079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:18.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.006+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2514076870 con 0x7f2524143600 2023-11-08T19:28:18.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.232+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f25240c4230 con 0x7f2524143600 2023-11-08T19:28:18.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.773+0000 7f25197fa640 1 -- 172.21.15.119:0/3188060873 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v44) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2514079e90 con 0x7f2524143600 2023-11-08T19:28:18.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.777+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f251006ae70 msgr2=0x7f251006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:18.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.777+0000 7f252afa5640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f251006ae70 0x7f251006d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f2520003cc0 tx=0x7f25200079b0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.777+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 msgr2=0x7f25241349f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:18.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.777+0000 7f252afa5640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 0x7f25241349f0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f251402b880 tx=0x7f25140046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 shutdown_connections 2023-11-08T19:28:18.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f251006ae70 0x7f251006d330 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f252415d9f0 0x7f252415c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2524143600 0x7f25241349f0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 --2- 172.21.15.119:0/3188060873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f252406cb40 0x7f25241f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:18.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 >> 172.21.15.119:0/3188060873 conn(0x7f25240cb610 msgr2=0x7f25240c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:18.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 shutdown_connections 2023-11-08T19:28:18.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:18.778+0000 7f252afa5640 1 -- 172.21.15.119:0/3188060873 wait complete. 2023-11-08T19:28:18.802 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:28:18.802 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:19.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.019+0000 7f0e8ecd7640 1 Processor -- start 2023-11-08T19:28:19.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.019+0000 7f0e8ecd7640 1 -- start start 2023-11-08T19:28:19.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.021+0000 7f0e8ecd7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0e8806a390 0x7f0e880c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.021+0000 7f0e8ecd7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e880ae410 con 0x7f0e880cc810 2023-11-08T19:28:19.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.021+0000 7f0e8ecd7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e8806ba30 con 0x7f0e8815d180 2023-11-08T19:28:19.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.021+0000 7f0e8ecd7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e8806bcf0 con 0x7f0e8806a390 2023-11-08T19:28:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.021+0000 7f0e87fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0e880cc810 0x7f0e8815ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50080/0 (socket says 172.21.15.119:50080) 2023-11-08T19:28:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.021+0000 7f0e87fff640 1 -- 172.21.15.119:0/2241249086 learned_addr learned my addr 172.21.15.119:0/2241249086 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:19.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.022+0000 7f0e8ca73640 1 --2- 172.21.15.119:0/2241249086 >> v2:172.21.15.188:3300/0 conn(0x7f0e8806a390 0x7f0e880c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.022+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471061008 0 0) 0x7f0e880ae410 con 0x7f0e880cc810 2023-11-08T19:28:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.022+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e70003520 con 0x7f0e880cc810 2023-11-08T19:28:19.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.022+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2699552840 0 0) 0x7f0e70003520 con 0x7f0e880cc810 2023-11-08T19:28:19.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.022+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 >> v2:172.21.15.188:3300/0 conn(0x7f0e8806a390 msgr2=0x7f0e880c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.022+0000 7f0e877fe640 1 --2- 172.21.15.119:0/2241249086 >> v2:172.21.15.188:3300/0 conn(0x7f0e8806a390 0x7f0e880c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 >> v1:172.21.15.188:6789/0 conn(0x7f0e8815d180 legacy=0x7f0e88160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e8806bfd0 con 0x7f0e880cc810 2023-11-08T19:28:19.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f0e7c001670 con 0x7f0e880cc810 2023-11-08T19:28:19.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 >> v1:172.21.15.119:6789/0 conn(0x7f0e880cc810 legacy=0x7f0e8815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e70003970 0x7f0e70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e700047c0 0x7f0e70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e70005180 0x7f0e7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8806bcf0 con 0x7f0e700047c0 2023-11-08T19:28:19.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7000c020 con 0x7f0e70005180 2023-11-08T19:28:19.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7000c1e0 con 0x7f0e70003970 2023-11-08T19:28:19.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e87fff640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e700047c0 0x7f0e70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e8ca73640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e70003970 0x7f0e70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e8d274640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e70005180 0x7f0e7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e87fff640 1 -- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e70003970 msgr2=0x7f0e70003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e87fff640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e70003970 0x7f0e70003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e87fff640 1 -- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e70005180 msgr2=0x7f0e7000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e87fff640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e70005180 0x7f0e7000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.023+0000 7f0e87fff640 1 -- 172.21.15.119:0/2241249086 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e70003520 con 0x7f0e700047c0 2023-11-08T19:28:19.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.024+0000 7f0e8d274640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e70005180 0x7f0e7000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:19.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.024+0000 7f0e8d274640 1 -- 172.21.15.119:0/2241249086 reap_dead start 2023-11-08T19:28:19.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e87fff640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e700047c0 0x7f0e70004c40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c210 tx=0x7f0e700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc043f6905e8067 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c005ad0 con 0x7f0e700047c0 2023-11-08T19:28:19.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7c005c30 con 0x7f0e700047c0 2023-11-08T19:28:19.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e877fe640 1 -- 172.21.15.119:0/2241249086 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c02cad0 con 0x7f0e700047c0 2023-11-08T19:28:19.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/2241249086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e700047c0 msgr2=0x7f0e70004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e700047c0 0x7f0e70004c40 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00c210 tx=0x7f0e700057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/2241249086 shutdown_connections 2023-11-08T19:28:19.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/2241249086 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e700047c0 0x7f0e70004c40 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.025+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/2241249086 >> 172.21.15.119:0/2241249086 conn(0x7f0e880cb610 msgr2=0x7f0e880cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.026+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/2241249086 shutdown_connections 2023-11-08T19:28:19.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.026+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/2241249086 wait complete. 2023-11-08T19:28:19.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.027+0000 7f0e8ecd7640 1 Processor -- start 2023-11-08T19:28:19.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.027+0000 7f0e8ecd7640 1 -- start start 2023-11-08T19:28:19.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.028+0000 7f0e8ecd7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e700047c0 0x7f0e881f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.028+0000 7f0e8ecd7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e880af5f0 0x7f0e880afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.028+0000 7f0e8ecd7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 0x7f0e880b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.028+0000 7f0e8ecd7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880affb0 con 0x7f0e880b32c0 2023-11-08T19:28:19.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.028+0000 7f0e8ecd7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880ace20 con 0x7f0e880af5f0 2023-11-08T19:28:19.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.028+0000 7f0e8ecd7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e880b4140 con 0x7f0e700047c0 2023-11-08T19:28:19.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 0x7f0e880b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 0x7f0e880b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35562/0 (socket says 172.21.15.119:35562) 2023-11-08T19:28:19.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 -- 172.21.15.119:0/3378199869 learned_addr learned my addr 172.21.15.119:0/3378199869 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:19.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8ca73640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e700047c0 0x7f0e881f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e87fff640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e880af5f0 0x7f0e880afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 -- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e700047c0 msgr2=0x7f0e881f1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e700047c0 0x7f0e881f1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 -- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e880af5f0 msgr2=0x7f0e880afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e880af5f0 0x7f0e880afa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 -- 172.21.15.119:0/3378199869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e880ad0c0 con 0x7f0e880b32c0 2023-11-08T19:28:19.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8ca73640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e700047c0 0x7f0e881f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:19.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.029+0000 7f0e8d274640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 0x7f0e880b3740 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0e880ce180 tx=0x7f0e8000d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.030+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80003d80 con 0x7f0e880b32c0 2023-11-08T19:28:19.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.030+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e80005070 con 0x7f0e880b32c0 2023-11-08T19:28:19.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.030+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e880ad3a0 con 0x7f0e880b32c0 2023-11-08T19:28:19.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.030+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e80005420 con 0x7f0e880b32c0 2023-11-08T19:28:19.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.030+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e881f2310 con 0x7f0e880b32c0 2023-11-08T19:28:19.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.031+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e88074430 con 0x7f0e880b32c0 2023-11-08T19:28:19.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.032+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e80005580 con 0x7f0e880b32c0 2023-11-08T19:28:19.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.033+0000 7f0e857fa640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e7406ae70 0x7f0e7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.033+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f0e800893e0 con 0x7f0e880b32c0 2023-11-08T19:28:19.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.033+0000 7f0e8ca73640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e7406ae70 0x7f0e7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.034+0000 7f0e8ca73640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e7406ae70 0x7f0e7406d330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009a80 tx=0x7f0e78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.037+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e80053fb0 con 0x7f0e880b32c0 2023-11-08T19:28:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.263+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e8806cd50 con 0x7f0e880b32c0 2023-11-08T19:28:19.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.264+0000 7f0e857fa640 1 -- 172.21.15.119:0/3378199869 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f0e80005830 con 0x7f0e880b32c0 2023-11-08T19:28:19.266 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:19.266 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:28:15.191469+0000","modified":"2023-11-08T19:28:18.233986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/4016780496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4016780496},{"type":"v1","addr":"172.21.15.119:6837","nonce":4016780496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:28:19.267 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 44 2023-11-08T19:28:19.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e7406ae70 msgr2=0x7f0e7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e7406ae70 0x7f0e7406d330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009a80 tx=0x7f0e78008040 comp rx=0 tx=0).stop 2023-11-08T19:28:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 msgr2=0x7f0e880b3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 0x7f0e880b3740 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f0e880ce180 tx=0x7f0e8000d9d0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 shutdown_connections 2023-11-08T19:28:19.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e7406ae70 0x7f0e7406d330 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e880b32c0 0x7f0e880b3740 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e880af5f0 0x7f0e880afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 --2- 172.21.15.119:0/3378199869 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e700047c0 0x7f0e881f1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 >> 172.21.15.119:0/3378199869 conn(0x7f0e880cb610 msgr2=0x7f0e8806fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 shutdown_connections 2023-11-08T19:28:19.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.267+0000 7f0e8ecd7640 1 -- 172.21.15.119:0/3378199869 wait complete. 2023-11-08T19:28:19.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.501+0000 7f19678e5640 1 Processor -- start 2023-11-08T19:28:19.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.502+0000 7f19678e5640 1 -- start start 2023-11-08T19:28:19.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.503+0000 7f19678e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 0x7f19600cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.503+0000 7f19678e5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19600c4960 con 0x7f196006cb40 2023-11-08T19:28:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.503+0000 7f19678e5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f196015df60 con 0x7f196015d9f0 2023-11-08T19:28:19.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.503+0000 7f19678e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1960143a40 con 0x7f1960142f80 2023-11-08T19:28:19.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.504+0000 7f1965e82640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f196006cb40 0x7f19600c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50084/0 (socket says 172.21.15.119:50084) 2023-11-08T19:28:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.504+0000 7f1965e82640 1 -- 172.21.15.119:0/4072926487 learned_addr learned my addr 172.21.15.119:0/4072926487 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.504+0000 7f1965681640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 0x7f19600cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.504+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22573674 0 0) 0x7f19600c4960 con 0x7f196006cb40 2023-11-08T19:28:19.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.504+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194c003520 con 0x7f196006cb40 2023-11-08T19:28:19.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3942438998 0 0) 0x7f194c003520 con 0x7f196006cb40 2023-11-08T19:28:19.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 msgr2=0x7f19600cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 0x7f19600cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 >> v1:172.21.15.188:6789/0 conn(0x7f196015d9f0 legacy=0x7f19600c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196015e240 con 0x7f196006cb40 2023-11-08T19:28:19.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f19580019f0 con 0x7f196006cb40 2023-11-08T19:28:19.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 >> v1:172.21.15.119:6789/0 conn(0x7f196006cb40 legacy=0x7f19600c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c0039a0 0x7f194c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 0x7f194c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c004d00 0x7f194c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1960143a40 con 0x7f194c004d00 2023-11-08T19:28:19.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c00bab0 con 0x7f194c0039a0 2023-11-08T19:28:19.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c00bc70 con 0x7f1960142f80 2023-11-08T19:28:19.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965e82640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c004d00 0x7f194c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1964e80640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 0x7f194c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965681640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c0039a0 0x7f194c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965e82640 1 -- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 msgr2=0x7f194c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965e82640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1960142f80 0x7f194c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965e82640 1 -- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c0039a0 msgr2=0x7f194c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965e82640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f194c0039a0 0x7f194c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.505+0000 7f1965e82640 1 -- 172.21.15.119:0/4072926487 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c003520 con 0x7f194c004d00 2023-11-08T19:28:19.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f1965e82640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c004d00 0x7f194c005180 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f195800a830 tx=0x7f194c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0f5764162925fe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580072d0 con 0x7f194c004d00 2023-11-08T19:28:19.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195802f070 con 0x7f194c004d00 2023-11-08T19:28:19.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f1957fff640 1 -- 172.21.15.119:0/4072926487 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19580076b0 con 0x7f194c004d00 2023-11-08T19:28:19.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f19678e5640 1 -- 172.21.15.119:0/4072926487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c004d00 msgr2=0x7f194c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f19678e5640 1 --2- 172.21.15.119:0/4072926487 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f194c004d00 0x7f194c005180 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f195800a830 tx=0x7f194c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f1965e82640 1 -- 172.21.15.119:0/4072926487 reap_dead start 2023-11-08T19:28:19.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f19678e5640 1 -- 172.21.15.119:0/4072926487 shutdown_connections 2023-11-08T19:28:19.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.507+0000 7f19678e5640 1 -- 172.21.15.119:0/4072926487 >> 172.21.15.119:0/4072926487 conn(0x7f19600cb610 msgr2=0x7f19600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.508+0000 7f19678e5640 1 -- 172.21.15.119:0/4072926487 shutdown_connections 2023-11-08T19:28:19.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.508+0000 7f19678e5640 1 -- 172.21.15.119:0/4072926487 wait complete. 2023-11-08T19:28:19.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.509+0000 7f19678e5640 1 Processor -- start 2023-11-08T19:28:19.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.509+0000 7f19678e5640 1 -- start start 2023-11-08T19:28:19.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.510+0000 7f19678e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c004d00 0x7f1960158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.510+0000 7f19678e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 0x7f196013b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.510+0000 7f19678e5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19600c5bd0 0x7f19600c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.510+0000 7f19678e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600c65a0 con 0x7f19601f2bd0 2023-11-08T19:28:19.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.510+0000 7f19678e5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19601f3bf0 con 0x7f19600c5bd0 2023-11-08T19:28:19.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.510+0000 7f19678e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19601f3070 con 0x7f194c004d00 2023-11-08T19:28:19.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 0x7f196013b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 0x7f196013b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35576/0 (socket says 172.21.15.119:35576) 2023-11-08T19:28:19.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 -- 172.21.15.119:0/996214493 learned_addr learned my addr 172.21.15.119:0/996214493 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:19.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1965681640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c004d00 0x7f1960158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 -- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c004d00 msgr2=0x7f1960158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c004d00 0x7f1960158db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1965e82640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19600c5bd0 0x7f19600c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 -- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19600c5bd0 msgr2=0x7f19600c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19600c5bd0 0x7f19600c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 -- 172.21.15.119:0/996214493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19601f3350 con 0x7f19601f2bd0 2023-11-08T19:28:19.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.511+0000 7f1964e80640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 0x7f196013b9e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f195000e580 tx=0x7f195000ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.512+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19500052e0 con 0x7f19601f2bd0 2023-11-08T19:28:19.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.512+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1950005440 con 0x7f19601f2bd0 2023-11-08T19:28:19.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.512+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19601f3630 con 0x7f19601f2bd0 2023-11-08T19:28:19.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.512+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1950003810 con 0x7f19601f2bd0 2023-11-08T19:28:19.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.512+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19601592f0 con 0x7f19601f2bd0 2023-11-08T19:28:19.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.513+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1960074430 con 0x7f19601f2bd0 2023-11-08T19:28:19.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.514+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1950003970 con 0x7f19601f2bd0 2023-11-08T19:28:19.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.515+0000 7f1955ffb640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f194406aea0 0x7f194406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.515+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f1950089990 con 0x7f19601f2bd0 2023-11-08T19:28:19.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.515+0000 7f1965681640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f194406aea0 0x7f194406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.516+0000 7f1965681640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f194406aea0 0x7f194406d360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f195c009910 tx=0x7f195c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.519+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1950054440 con 0x7f19601f2bd0 2023-11-08T19:28:19.533 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:19.539 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:19.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.714+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19600c4230 con 0x7f19601f2bd0 2023-11-08T19:28:19.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.716+0000 7f1955ffb640 1 -- 172.21.15.119:0/996214493 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+21953 (secure 0 0 0) 0x7f1950057a60 con 0x7f19601f2bd0 2023-11-08T19:28:19.718 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:19.718 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":83,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:16.741197+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:19.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f194406aea0 msgr2=0x7f194406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f194406aea0 0x7f194406d360 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f195c009910 tx=0x7f195c008040 comp rx=0 tx=0).stop 2023-11-08T19:28:19.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 msgr2=0x7f196013b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 0x7f196013b9e0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f195000e580 tx=0x7f195000ea40 comp rx=0 tx=0).stop 2023-11-08T19:28:19.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 shutdown_connections 2023-11-08T19:28:19.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f194406aea0 0x7f194406d360 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19600c5bd0 0x7f19600c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f19601f2bd0 0x7f196013b9e0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 --2- 172.21.15.119:0/996214493 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f194c004d00 0x7f1960158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 >> 172.21.15.119:0/996214493 conn(0x7f19600cb610 msgr2=0x7f19600cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 shutdown_connections 2023-11-08T19:28:19.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.719+0000 7f19678e5640 1 -- 172.21.15.119:0/996214493 wait complete. 2023-11-08T19:28:19.740 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:28:19.741 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.955+0000 7f0e540a2640 1 Processor -- start 2023-11-08T19:28:19.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.955+0000 7f0e540a2640 1 -- start start 2023-11-08T19:28:19.958 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e540a2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0e4c06d250 0x7f0e4c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e540a2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c0c4960 con 0x7f0e4c142f80 2023-11-08T19:28:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e540a2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e4c15df60 con 0x7f0e4c15d9f0 2023-11-08T19:28:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e540a2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e4c143a40 con 0x7f0e4c06d250 2023-11-08T19:28:19.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e51e3e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0e4c142f80 0x7f0e4c0c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50088/0 (socket says 172.21.15.119:50088) 2023-11-08T19:28:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e51e3e640 1 -- 172.21.15.119:0/1183385913 learned_addr learned my addr 172.21.15.119:0/1183385913 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:19.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.957+0000 7f0e5263f640 1 --2- 172.21.15.119:0/1183385913 >> v2:172.21.15.119:3300/0 conn(0x7f0e4c06d250 0x7f0e4c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.958+0000 7f0e5263f640 1 -- 172.21.15.119:0/1183385913 >> v1:172.21.15.188:6789/0 conn(0x7f0e4c15d9f0 legacy=0x7f0e4c06cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:19.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.958+0000 7f0e5263f640 1 -- 172.21.15.119:0/1183385913 >> v1:172.21.15.119:6789/0 conn(0x7f0e4c142f80 legacy=0x7f0e4c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.958+0000 7f0e50e3c640 1 -- 172.21.15.119:0/1183385913 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2359503943 0 0) 0x7f0e4c0c4960 con 0x7f0e4c142f80 2023-11-08T19:28:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.958+0000 7f0e5263f640 1 -- 172.21.15.119:0/1183385913 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e4c15e240 con 0x7f0e4c06d250 2023-11-08T19:28:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.959+0000 7f0e5263f640 1 --2- 172.21.15.119:0/1183385913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c0cc6d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009d90 tx=0x7f0e4402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2096088e34861e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.959+0000 7f0e50e3c640 1 -- 172.21.15.119:0/1183385913 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4402ee70 con 0x7f0e4c06d250 2023-11-08T19:28:19.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.959+0000 7f0e50e3c640 1 -- 172.21.15.119:0/1183385913 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e44036440 con 0x7f0e4c06d250 2023-11-08T19:28:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.959+0000 7f0e50e3c640 1 -- 172.21.15.119:0/1183385913 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e440367d0 con 0x7f0e4c06d250 2023-11-08T19:28:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 -- 172.21.15.119:0/1183385913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 msgr2=0x7f0e4c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 --2- 172.21.15.119:0/1183385913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c0cc6d0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009d90 tx=0x7f0e4402e0c0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 -- 172.21.15.119:0/1183385913 shutdown_connections 2023-11-08T19:28:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 --2- 172.21.15.119:0/1183385913 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c0cc6d0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 -- 172.21.15.119:0/1183385913 >> 172.21.15.119:0/1183385913 conn(0x7f0e4c0cb610 msgr2=0x7f0e4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 -- 172.21.15.119:0/1183385913 shutdown_connections 2023-11-08T19:28:19.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.960+0000 7f0e540a2640 1 -- 172.21.15.119:0/1183385913 wait complete. 2023-11-08T19:28:19.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.962+0000 7f0e540a2640 1 Processor -- start 2023-11-08T19:28:19.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.962+0000 7f0e540a2640 1 -- start start 2023-11-08T19:28:19.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e540a2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c1351f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e540a2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c142f80 0x7f0e4c0adce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e540a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c15d9f0 0x7f0e4c15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e540a2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0ae630 con 0x7f0e4c06d250 2023-11-08T19:28:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e540a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0ae910 con 0x7f0e4c15d9f0 2023-11-08T19:28:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e540a2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e4c0aebf0 con 0x7f0e4c142f80 2023-11-08T19:28:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e51e3e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c1351f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e51e3e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c1351f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35582/0 (socket says 172.21.15.119:35582) 2023-11-08T19:28:19.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.963+0000 7f0e51e3e640 1 -- 172.21.15.119:0/3145814885 learned_addr learned my addr 172.21.15.119:0/3145814885 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e51e3e640 1 -- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c142f80 msgr2=0x7f0e4c0adce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e51e3e640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c142f80 0x7f0e4c0adce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e51e3e640 1 -- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c15d9f0 msgr2=0x7f0e4c15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e5263f640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c15d9f0 0x7f0e4c15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e51e3e640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c15d9f0 0x7f0e4c15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:19.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e51e3e640 1 -- 172.21.15.119:0/3145814885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e44009a70 con 0x7f0e4c06d250 2023-11-08T19:28:19.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e51e3e640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c1351f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800c430 tx=0x7f0e4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e480074e0 con 0x7f0e4c06d250 2023-11-08T19:28:19.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e48007660 con 0x7f0e4c06d250 2023-11-08T19:28:19.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48004a20 con 0x7f0e4c06d250 2023-11-08T19:28:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.964+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4c0aeed0 con 0x7f0e4c06d250 2023-11-08T19:28:19.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.965+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4c0af6d0 con 0x7f0e4c06d250 2023-11-08T19:28:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.966+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e4c074430 con 0x7f0e4c06d250 2023-11-08T19:28:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.967+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e48004b80 con 0x7f0e4c06d250 2023-11-08T19:28:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.968+0000 7f0e427fc640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e3006aea0 0x7f0e3006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:19.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.968+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f0e480899f0 con 0x7f0e4c06d250 2023-11-08T19:28:19.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.968+0000 7f0e5163d640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e3006aea0 0x7f0e3006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:19.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.969+0000 7f0e5163d640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e3006aea0 0x7f0e3006d360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c135730 tx=0x7f0e3c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:19.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:19.972+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e48054530 con 0x7f0e4c06d250 2023-11-08T19:28:20.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.154+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e4c0aea80 con 0x7f0e3006aea0 2023-11-08T19:28:20.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.195+0000 7f0e427fc640 1 -- 172.21.15.119:0/3145814885 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529102 (secure 0 0 0) 0x7f0e4c0aea80 con 0x7f0e3006aea0 2023-11-08T19:28:20.201 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:20.207 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:28:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e3006aea0 msgr2=0x7f0e3006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e3006aea0 0x7f0e3006d360 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c135730 tx=0x7f0e3c0079b0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 msgr2=0x7f0e4c1351f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c1351f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800c430 tx=0x7f0e4800c8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 shutdown_connections 2023-11-08T19:28:20.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0e3006aea0 0x7f0e3006d360 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0e4c15d9f0 0x7f0e4c15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0e4c142f80 0x7f0e4c0adce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 --2- 172.21.15.119:0/3145814885 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0e4c06d250 0x7f0e4c1351f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 >> 172.21.15.119:0/3145814885 conn(0x7f0e4c0cb610 msgr2=0x7f0e4c0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 shutdown_connections 2023-11-08T19:28:20.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.205+0000 7f0e540a2640 1 -- 172.21.15.119:0/3145814885 wait complete. 2023-11-08T19:28:20.224 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2023-11-08T19:28:18.726768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":281,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254360,"kb_used_data":9056,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720360,"statfs":{"total":773094113280,"available":772833648640,"internally_reserved":0,"allocated":9273344,"data_stored":5085158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023741"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759604+0000","last_change":"2023-11-08T19:28:15.752428+0000","last_active":"2023-11-08T19:28:15.759604+0000","last_peered":"2023-11-08T19:28:15.759604+0000","last_clean":"2023-11-08T19:28:15.759604+0000","last_became_active":"2023-11-08T19:28:15.752015+0000","last_became_peered":"2023-11-08T19:28:15.752015+0000","last_unstale":"2023-11-08T19:28:15.759604+0000","last_undegraded":"2023-11-08T19:28:15.759604+0000","last_fullsized":"2023-11-08T19:28:15.759604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:35.620894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759367+0000","last_change":"2023-11-08T19:28:15.756465+0000","last_active":"2023-11-08T19:28:15.759367+0000","last_peered":"2023-11-08T19:28:15.759367+0000","last_clean":"2023-11-08T19:28:15.759367+0000","last_became_active":"2023-11-08T19:28:15.756200+0000","last_became_peered":"2023-11-08T19:28:15.756200+0000","last_unstale":"2023-11-08T19:28:15.759367+0000","last_undegraded":"2023-11-08T19:28:15.759367+0000","last_fullsized":"2023-11-08T19:28:15.759367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:17:33.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.760396+0000","last_change":"2023-11-08T19:28:15.756571+0000","last_active":"2023-11-08T19:28:15.760396+0000","last_peered":"2023-11-08T19:28:15.760396+0000","last_clean":"2023-11-08T19:28:15.760396+0000","last_became_active":"2023-11-08T19:28:15.756317+0000","last_became_peered":"2023-11-08T19:28:15.756317+0000","last_unstale":"2023-11-08T19:28:15.760396+0000","last_undegraded":"2023-11-08T19:28:15.760396+0000","last_fullsized":"2023-11-08T19:28:15.760396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:34.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.757765+0000","last_change":"2023-11-08T19:28:15.755798+0000","last_active":"2023-11-08T19:28:16.757765+0000","last_peered":"2023-11-08T19:28:16.757765+0000","last_clean":"2023-11-08T19:28:16.757765+0000","last_became_active":"2023-11-08T19:28:15.755482+0000","last_became_peered":"2023-11-08T19:28:15.755482+0000","last_unstale":"2023-11-08T19:28:16.757765+0000","last_undegraded":"2023-11-08T19:28:16.757765+0000","last_fullsized":"2023-11-08T19:28:16.757765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:17:33.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.769626+0000","last_change":"2023-11-08T19:28:15.762747+0000","last_active":"2023-11-08T19:28:15.769626+0000","last_peered":"2023-11-08T19:28:15.769626+0000","last_clean":"2023-11-08T19:28:15.769626+0000","last_became_active":"2023-11-08T19:28:15.762454+0000","last_became_peered":"2023-11-08T19:28:15.762454+0000","last_unstale":"2023-11-08T19:28:15.769626+0000","last_undegraded":"2023-11-08T19:28:15.769626+0000","last_fullsized":"2023-11-08T19:28:15.769626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:57.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773314+0000","last_change":"2023-11-08T19:28:15.769873+0000","last_active":"2023-11-08T19:28:15.773314+0000","last_peered":"2023-11-08T19:28:15.773314+0000","last_clean":"2023-11-08T19:28:15.773314+0000","last_became_active":"2023-11-08T19:28:15.769551+0000","last_became_peered":"2023-11-08T19:28:15.769551+0000","last_unstale":"2023-11-08T19:28:15.773314+0000","last_undegraded":"2023-11-08T19:28:15.773314+0000","last_fullsized":"2023-11-08T19:28:15.773314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:14.523591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770951+0000","last_change":"2023-11-08T19:28:15.768035+0000","last_active":"2023-11-08T19:28:15.770951+0000","last_peered":"2023-11-08T19:28:15.770951+0000","last_clean":"2023-11-08T19:28:15.770951+0000","last_became_active":"2023-11-08T19:28:15.767769+0000","last_became_peered":"2023-11-08T19:28:15.767769+0000","last_unstale":"2023-11-08T19:28:15.770951+0000","last_undegraded":"2023-11-08T19:28:15.770951+0000","last_fullsized":"2023-11-08T19:28:15.770951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:44.189210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765385+0000","last_change":"2023-11-08T19:28:15.762884+0000","last_active":"2023-11-08T19:28:15.765385+0000","last_peered":"2023-11-08T19:28:15.765385+0000","last_clean":"2023-11-08T19:28:15.765385+0000","last_became_active":"2023-11-08T19:28:15.762624+0000","last_became_peered":"2023-11-08T19:28:15.762624+0000","last_unstale":"2023-11-08T19:28:15.765385+0000","last_undegraded":"2023-11-08T19:28:15.765385+0000","last_fullsized":"2023-11-08T19:28:15.765385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:48.522052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767584+0000","last_change":"2023-11-08T19:28:15.760605+0000","last_active":"2023-11-08T19:28:15.767584+0000","last_peered":"2023-11-08T19:28:15.767584+0000","last_clean":"2023-11-08T19:28:15.767584+0000","last_became_active":"2023-11-08T19:28:15.760301+0000","last_became_peered":"2023-11-08T19:28:15.760301+0000","last_unstale":"2023-11-08T19:28:15.767584+0000","last_undegraded":"2023-11-08T19:28:15.767584+0000","last_fullsized":"2023-11-08T19:28:15.767584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:46.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765706+0000","last_change":"2023-11-08T19:28:15.755416+0000","last_active":"2023-11-08T19:28:15.765706+0000","last_peered":"2023-11-08T19:28:15.765706+0000","last_clean":"2023-11-08T19:28:15.765706+0000","last_became_active":"2023-11-08T19:28:15.755140+0000","last_became_peered":"2023-11-08T19:28:15.755140+0000","last_unstale":"2023-11-08T19:28:15.765706+0000","last_undegraded":"2023-11-08T19:28:15.765706+0000","last_fullsized":"2023-11-08T19:28:15.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:44.189210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.777009+0000","last_change":"2023-11-08T19:28:15.772381+0000","last_active":"2023-11-08T19:28:15.777009+0000","last_peered":"2023-11-08T19:28:15.777009+0000","last_clean":"2023-11-08T19:28:15.777009+0000","last_became_active":"2023-11-08T19:28:15.772079+0000","last_became_peered":"2023-11-08T19:28:15.772079+0000","last_unstale":"2023-11-08T19:28:15.777009+0000","last_undegraded":"2023-11-08T19:28:15.777009+0000","last_fullsized":"2023-11-08T19:28:15.777009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:40.670042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773334+0000","last_change":"2023-11-08T19:28:15.767699+0000","last_active":"2023-11-08T19:28:15.773334+0000","last_peered":"2023-11-08T19:28:15.773334+0000","last_clean":"2023-11-08T19:28:15.773334+0000","last_became_active":"2023-11-08T19:28:15.766388+0000","last_became_peered":"2023-11-08T19:28:15.766388+0000","last_unstale":"2023-11-08T19:28:15.773334+0000","last_undegraded":"2023-11-08T19:28:15.773334+0000","last_fullsized":"2023-11-08T19:28:15.773334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:44.189210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755167+0000","last_change":"2023-11-08T19:28:15.764595+0000","last_active":"2023-11-08T19:28:16.755167+0000","last_peered":"2023-11-08T19:28:16.755167+0000","last_clean":"2023-11-08T19:28:16.755167+0000","last_became_active":"2023-11-08T19:28:15.764329+0000","last_became_peered":"2023-11-08T19:28:15.764329+0000","last_unstale":"2023-11-08T19:28:16.755167+0000","last_undegraded":"2023-11-08T19:28:16.755167+0000","last_fullsized":"2023-11-08T19:28:16.755167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:31.491334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.756789+0000","last_change":"2023-11-08T19:28:15.752823+0000","last_active":"2023-11-08T19:28:15.756789+0000","last_peered":"2023-11-08T19:28:15.756789+0000","last_clean":"2023-11-08T19:28:15.756789+0000","last_became_active":"2023-11-08T19:28:15.752520+0000","last_became_peered":"2023-11-08T19:28:15.752520+0000","last_unstale":"2023-11-08T19:28:15.756789+0000","last_undegraded":"2023-11-08T19:28:15.756789+0000","last_fullsized":"2023-11-08T19:28:15.756789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:26.931128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756694+0000","last_change":"2023-11-08T19:28:15.760259+0000","last_active":"2023-11-08T19:28:16.756694+0000","last_peered":"2023-11-08T19:28:16.756694+0000","last_clean":"2023-11-08T19:28:16.756694+0000","last_became_active":"2023-11-08T19:28:15.759998+0000","last_became_peered":"2023-11-08T19:28:15.759998+0000","last_unstale":"2023-11-08T19:28:16.756694+0000","last_undegraded":"2023-11-08T19:28:16.756694+0000","last_fullsized":"2023-11-08T19:28:16.756694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:00.119152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773323+0000","last_change":"2023-11-08T19:28:15.769582+0000","last_active":"2023-11-08T19:28:15.773323+0000","last_peered":"2023-11-08T19:28:15.773323+0000","last_clean":"2023-11-08T19:28:15.773323+0000","last_became_active":"2023-11-08T19:28:15.769271+0000","last_became_peered":"2023-11-08T19:28:15.769271+0000","last_unstale":"2023-11-08T19:28:15.773323+0000","last_undegraded":"2023-11-08T19:28:15.773323+0000","last_fullsized":"2023-11-08T19:28:15.773323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:46.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768377+0000","last_change":"2023-11-08T19:28:15.764301+0000","last_active":"2023-11-08T19:28:15.768377+0000","last_peered":"2023-11-08T19:28:15.768377+0000","last_clean":"2023-11-08T19:28:15.768377+0000","last_became_active":"2023-11-08T19:28:15.763768+0000","last_became_peered":"2023-11-08T19:28:15.763768+0000","last_unstale":"2023-11-08T19:28:15.768377+0000","last_undegraded":"2023-11-08T19:28:15.768377+0000","last_fullsized":"2023-11-08T19:28:15.768377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:10:58.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755905+0000","last_change":"2023-11-08T19:28:15.760215+0000","last_active":"2023-11-08T19:28:16.755905+0000","last_peered":"2023-11-08T19:28:16.755905+0000","last_clean":"2023-11-08T19:28:16.755905+0000","last_became_active":"2023-11-08T19:28:15.759631+0000","last_became_peered":"2023-11-08T19:28:15.759631+0000","last_unstale":"2023-11-08T19:28:16.755905+0000","last_undegraded":"2023-11-08T19:28:16.755905+0000","last_fullsized":"2023-11-08T19:28:16.755905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:00.979814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770708+0000","last_change":"2023-11-08T19:28:15.768655+0000","last_active":"2023-11-08T19:28:15.770708+0000","last_peered":"2023-11-08T19:28:15.770708+0000","last_clean":"2023-11-08T19:28:15.770708+0000","last_became_active":"2023-11-08T19:28:15.764796+0000","last_became_peered":"2023-11-08T19:28:15.764796+0000","last_unstale":"2023-11-08T19:28:15.770708+0000","last_undegraded":"2023-11-08T19:28:15.770708+0000","last_fullsized":"2023-11-08T19:28:15.770708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:31:17.396479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768141+0000","last_change":"2023-11-08T19:28:15.760217+0000","last_active":"2023-11-08T19:28:15.768141+0000","last_peered":"2023-11-08T19:28:15.768141+0000","last_clean":"2023-11-08T19:28:15.768141+0000","last_became_active":"2023-11-08T19:28:15.759934+0000","last_became_peered":"2023-11-08T19:28:15.759934+0000","last_unstale":"2023-11-08T19:28:15.768141+0000","last_undegraded":"2023-11-08T19:28:15.768141+0000","last_fullsized":"2023-11-08T19:28:15.768141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:19.077175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.769151+0000","last_change":"2023-11-08T19:28:15.767669+0000","last_active":"2023-11-08T19:28:15.769151+0000","last_peered":"2023-11-08T19:28:15.769151+0000","last_clean":"2023-11-08T19:28:15.769151+0000","last_became_active":"2023-11-08T19:28:15.767405+0000","last_became_peered":"2023-11-08T19:28:15.767405+0000","last_unstale":"2023-11-08T19:28:15.769151+0000","last_undegraded":"2023-11-08T19:28:15.769151+0000","last_fullsized":"2023-11-08T19:28:15.769151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:50.914147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757077+0000","last_change":"2023-11-08T19:28:15.750930+0000","last_active":"2023-11-08T19:28:15.757077+0000","last_peered":"2023-11-08T19:28:15.757077+0000","last_clean":"2023-11-08T19:28:15.757077+0000","last_became_active":"2023-11-08T19:28:15.750623+0000","last_became_peered":"2023-11-08T19:28:15.750623+0000","last_unstale":"2023-11-08T19:28:15.757077+0000","last_undegraded":"2023-11-08T19:28:15.757077+0000","last_fullsized":"2023-11-08T19:28:15.757077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:02.182611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759299+0000","last_change":"2023-11-08T19:28:15.756466+0000","last_active":"2023-11-08T19:28:15.759299+0000","last_peered":"2023-11-08T19:28:15.759299+0000","last_clean":"2023-11-08T19:28:15.759299+0000","last_became_active":"2023-11-08T19:28:15.753727+0000","last_became_peered":"2023-11-08T19:28:15.753727+0000","last_unstale":"2023-11-08T19:28:15.759299+0000","last_undegraded":"2023-11-08T19:28:15.759299+0000","last_fullsized":"2023-11-08T19:28:15.759299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:16:48.799049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767065+0000","last_change":"2023-11-08T19:28:15.761676+0000","last_active":"2023-11-08T19:28:15.767065+0000","last_peered":"2023-11-08T19:28:15.767065+0000","last_clean":"2023-11-08T19:28:15.767065+0000","last_became_active":"2023-11-08T19:28:15.761378+0000","last_became_peered":"2023-11-08T19:28:15.761378+0000","last_unstale":"2023-11-08T19:28:15.767065+0000","last_undegraded":"2023-11-08T19:28:15.767065+0000","last_fullsized":"2023-11-08T19:28:15.767065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:57.342920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765976+0000","last_change":"2023-11-08T19:28:15.757254+0000","last_active":"2023-11-08T19:28:15.765976+0000","last_peered":"2023-11-08T19:28:15.765976+0000","last_clean":"2023-11-08T19:28:15.765976+0000","last_became_active":"2023-11-08T19:28:15.754720+0000","last_became_peered":"2023-11-08T19:28:15.754720+0000","last_unstale":"2023-11-08T19:28:15.765976+0000","last_undegraded":"2023-11-08T19:28:15.765976+0000","last_fullsized":"2023-11-08T19:28:15.765976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:54.686302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765017+0000","last_change":"2023-11-08T19:28:15.761643+0000","last_active":"2023-11-08T19:28:15.765017+0000","last_peered":"2023-11-08T19:28:15.765017+0000","last_clean":"2023-11-08T19:28:15.765017+0000","last_became_active":"2023-11-08T19:28:15.761367+0000","last_became_peered":"2023-11-08T19:28:15.761367+0000","last_unstale":"2023-11-08T19:28:15.765017+0000","last_undegraded":"2023-11-08T19:28:15.765017+0000","last_fullsized":"2023-11-08T19:28:15.765017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:26.949146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773520+0000","last_change":"2023-11-08T19:28:15.770161+0000","last_active":"2023-11-08T19:28:15.773520+0000","last_peered":"2023-11-08T19:28:15.773520+0000","last_clean":"2023-11-08T19:28:15.773520+0000","last_became_active":"2023-11-08T19:28:15.769868+0000","last_became_peered":"2023-11-08T19:28:15.769868+0000","last_unstale":"2023-11-08T19:28:15.773520+0000","last_undegraded":"2023-11-08T19:28:15.773520+0000","last_fullsized":"2023-11-08T19:28:15.773520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:48:34.435941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.769968+0000","last_change":"2023-11-08T19:28:15.765121+0000","last_active":"2023-11-08T19:28:15.769968+0000","last_peered":"2023-11-08T19:28:15.769968+0000","last_clean":"2023-11-08T19:28:15.769968+0000","last_became_active":"2023-11-08T19:28:15.764847+0000","last_became_peered":"2023-11-08T19:28:15.764847+0000","last_unstale":"2023-11-08T19:28:15.769968+0000","last_undegraded":"2023-11-08T19:28:15.769968+0000","last_fullsized":"2023-11-08T19:28:15.769968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:46.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757518+0000","last_change":"2023-11-08T19:28:15.752467+0000","last_active":"2023-11-08T19:28:15.757518+0000","last_peered":"2023-11-08T19:28:15.757518+0000","last_clean":"2023-11-08T19:28:15.757518+0000","last_became_active":"2023-11-08T19:28:15.752213+0000","last_became_peered":"2023-11-08T19:28:15.752213+0000","last_unstale":"2023-11-08T19:28:15.757518+0000","last_undegraded":"2023-11-08T19:28:15.757518+0000","last_fullsized":"2023-11-08T19:28:15.757518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:09.019260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767421+0000","last_change":"2023-11-08T19:28:15.762793+0000","last_active":"2023-11-08T19:28:15.767421+0000","last_peered":"2023-11-08T19:28:15.767421+0000","last_clean":"2023-11-08T19:28:15.767421+0000","last_became_active":"2023-11-08T19:28:15.760637+0000","last_became_peered":"2023-11-08T19:28:15.760637+0000","last_unstale":"2023-11-08T19:28:15.767421+0000","last_undegraded":"2023-11-08T19:28:15.767421+0000","last_fullsized":"2023-11-08T19:28:15.767421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:33.778386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756997+0000","last_change":"2023-11-08T19:28:15.769059+0000","last_active":"2023-11-08T19:28:16.756997+0000","last_peered":"2023-11-08T19:28:16.756997+0000","last_clean":"2023-11-08T19:28:16.756997+0000","last_became_active":"2023-11-08T19:28:15.768440+0000","last_became_peered":"2023-11-08T19:28:15.768440+0000","last_unstale":"2023-11-08T19:28:16.756997+0000","last_undegraded":"2023-11-08T19:28:16.756997+0000","last_fullsized":"2023-11-08T19:28:16.756997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:26.949146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.758114+0000","last_change":"2023-11-08T19:28:15.749493+0000","last_active":"2023-11-08T19:28:15.758114+0000","last_peered":"2023-11-08T19:28:15.758114+0000","last_clean":"2023-11-08T19:28:15.758114+0000","last_became_active":"2023-11-08T19:28:15.749210+0000","last_became_peered":"2023-11-08T19:28:15.749210+0000","last_unstale":"2023-11-08T19:28:15.758114+0000","last_undegraded":"2023-11-08T19:28:15.758114+0000","last_fullsized":"2023-11-08T19:28:15.758114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:06.290558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773500+0000","last_change":"2023-11-08T19:28:15.769546+0000","last_active":"2023-11-08T19:28:15.773500+0000","last_peered":"2023-11-08T19:28:15.773500+0000","last_clean":"2023-11-08T19:28:15.773500+0000","last_became_active":"2023-11-08T19:28:15.769197+0000","last_became_peered":"2023-11-08T19:28:15.769197+0000","last_unstale":"2023-11-08T19:28:15.773500+0000","last_undegraded":"2023-11-08T19:28:15.773500+0000","last_fullsized":"2023-11-08T19:28:15.773500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:13:07.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.777205+0000","last_change":"2023-11-08T19:28:14.746051+0000","last_active":"2023-11-08T19:28:15.777205+0000","last_peered":"2023-11-08T19:28:15.777205+0000","last_clean":"2023-11-08T19:28:15.777205+0000","last_became_active":"2023-11-08T19:28:14.745719+0000","last_became_peered":"2023-11-08T19:28:14.745719+0000","last_unstale":"2023-11-08T19:28:15.777205+0000","last_undegraded":"2023-11-08T19:28:15.777205+0000","last_fullsized":"2023-11-08T19:28:15.777205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:21.923551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.728881+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:14.728881+0000","last_peered":"2023-11-08T19:28:14.728881+0000","last_clean":"2023-11-08T19:28:14.728881+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:14.728881+0000","last_undegraded":"2023-11-08T19:28:14.728881+0000","last_fullsized":"2023-11-08T19:28:14.728881+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.746541+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:14.746541+0000","last_peered":"2023-11-08T19:28:14.746541+0000","last_clean":"2023-11-08T19:28:14.746541+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:14.746541+0000","last_undegraded":"2023-11-08T19:28:14.746541+0000","last_fullsized":"2023-11-08T19:28:14.746541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722971+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:14.722971+0000","last_peered":"2023-11-08T19:28:14.722971+0000","last_clean":"2023-11-08T19:28:14.722971+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:14.722971+0000","last_undegraded":"2023-11-08T19:28:14.722971+0000","last_fullsized":"2023-11-08T19:28:14.722971+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757920+0000","last_change":"2023-11-08T19:28:15.750436+0000","last_active":"2023-11-08T19:28:15.757920+0000","last_peered":"2023-11-08T19:28:15.757920+0000","last_clean":"2023-11-08T19:28:15.757920+0000","last_became_active":"2023-11-08T19:28:15.750068+0000","last_became_peered":"2023-11-08T19:28:15.750068+0000","last_unstale":"2023-11-08T19:28:15.757920+0000","last_undegraded":"2023-11-08T19:28:15.757920+0000","last_fullsized":"2023-11-08T19:28:15.757920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:13:07.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"82'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773880+0000","last_change":"2023-11-08T19:28:14.756005+0000","last_active":"2023-11-08T19:28:15.773880+0000","last_peered":"2023-11-08T19:28:15.773880+0000","last_clean":"2023-11-08T19:28:15.773880+0000","last_became_active":"2023-11-08T19:28:14.755503+0000","last_became_peered":"2023-11-08T19:28:14.755503+0000","last_unstale":"2023-11-08T19:28:15.773880+0000","last_undegraded":"2023-11-08T19:28:15.773880+0000","last_fullsized":"2023-11-08T19:28:15.773880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:40.572679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.747671+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:15.747671+0000","last_peered":"2023-11-08T19:28:15.747671+0000","last_clean":"2023-11-08T19:28:15.747671+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:15.747671+0000","last_undegraded":"2023-11-08T19:28:15.747671+0000","last_fullsized":"2023-11-08T19:28:15.747671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747038+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:14.747038+0000","last_peered":"2023-11-08T19:28:14.747038+0000","last_clean":"2023-11-08T19:28:14.747038+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:14.747038+0000","last_undegraded":"2023-11-08T19:28:14.747038+0000","last_fullsized":"2023-11-08T19:28:14.747038+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.729228+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:14.729228+0000","last_peered":"2023-11-08T19:28:14.729228+0000","last_clean":"2023-11-08T19:28:14.729228+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:14.729228+0000","last_undegraded":"2023-11-08T19:28:14.729228+0000","last_fullsized":"2023-11-08T19:28:14.729228+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767561+0000","last_change":"2023-11-08T19:28:15.758399+0000","last_active":"2023-11-08T19:28:15.767561+0000","last_peered":"2023-11-08T19:28:15.767561+0000","last_clean":"2023-11-08T19:28:15.767561+0000","last_became_active":"2023-11-08T19:28:15.757230+0000","last_became_peered":"2023-11-08T19:28:15.757230+0000","last_unstale":"2023-11-08T19:28:15.767561+0000","last_undegraded":"2023-11-08T19:28:15.767561+0000","last_fullsized":"2023-11-08T19:28:15.767561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:40.670042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755986+0000","last_change":"2023-11-08T19:28:14.753569+0000","last_active":"2023-11-08T19:28:16.755986+0000","last_peered":"2023-11-08T19:28:16.755986+0000","last_clean":"2023-11-08T19:28:16.755986+0000","last_became_active":"2023-11-08T19:28:14.753273+0000","last_became_peered":"2023-11-08T19:28:14.753273+0000","last_unstale":"2023-11-08T19:28:16.755986+0000","last_undegraded":"2023-11-08T19:28:16.755986+0000","last_fullsized":"2023-11-08T19:28:16.755986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:25.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742009+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:14.742009+0000","last_peered":"2023-11-08T19:28:14.742009+0000","last_clean":"2023-11-08T19:28:14.742009+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:14.742009+0000","last_undegraded":"2023-11-08T19:28:14.742009+0000","last_fullsized":"2023-11-08T19:28:14.742009+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.728775+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:14.728775+0000","last_peered":"2023-11-08T19:28:14.728775+0000","last_clean":"2023-11-08T19:28:14.728775+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:14.728775+0000","last_undegraded":"2023-11-08T19:28:14.728775+0000","last_fullsized":"2023-11-08T19:28:14.728775+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747354+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:14.747354+0000","last_peered":"2023-11-08T19:28:14.747354+0000","last_clean":"2023-11-08T19:28:14.747354+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:14.747354+0000","last_undegraded":"2023-11-08T19:28:14.747354+0000","last_fullsized":"2023-11-08T19:28:14.747354+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765036+0000","last_change":"2023-11-08T19:28:15.762240+0000","last_active":"2023-11-08T19:28:15.765036+0000","last_peered":"2023-11-08T19:28:15.765036+0000","last_clean":"2023-11-08T19:28:15.765036+0000","last_became_active":"2023-11-08T19:28:15.761942+0000","last_became_peered":"2023-11-08T19:28:15.761942+0000","last_unstale":"2023-11-08T19:28:15.765036+0000","last_undegraded":"2023-11-08T19:28:15.765036+0000","last_fullsized":"2023-11-08T19:28:15.765036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:00.979814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.737507+0000","last_change":"2023-11-08T19:28:14.747536+0000","last_active":"2023-11-08T19:28:15.737507+0000","last_peered":"2023-11-08T19:28:15.737507+0000","last_clean":"2023-11-08T19:28:15.737507+0000","last_became_active":"2023-11-08T19:28:14.747255+0000","last_became_peered":"2023-11-08T19:28:14.747255+0000","last_unstale":"2023-11-08T19:28:15.737507+0000","last_undegraded":"2023-11-08T19:28:15.737507+0000","last_fullsized":"2023-11-08T19:28:15.737507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:11:05.256770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742497+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:14.742497+0000","last_peered":"2023-11-08T19:28:14.742497+0000","last_clean":"2023-11-08T19:28:14.742497+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:14.742497+0000","last_undegraded":"2023-11-08T19:28:14.742497+0000","last_fullsized":"2023-11-08T19:28:14.742497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745585+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:14.745585+0000","last_peered":"2023-11-08T19:28:14.745585+0000","last_clean":"2023-11-08T19:28:14.745585+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:14.745585+0000","last_undegraded":"2023-11-08T19:28:14.745585+0000","last_fullsized":"2023-11-08T19:28:14.745585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722752+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:14.722752+0000","last_peered":"2023-11-08T19:28:14.722752+0000","last_clean":"2023-11-08T19:28:14.722752+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:14.722752+0000","last_undegraded":"2023-11-08T19:28:14.722752+0000","last_fullsized":"2023-11-08T19:28:14.722752+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759089+0000","last_change":"2023-11-08T19:28:15.755259+0000","last_active":"2023-11-08T19:28:15.759089+0000","last_peered":"2023-11-08T19:28:15.759089+0000","last_clean":"2023-11-08T19:28:15.759089+0000","last_became_active":"2023-11-08T19:28:15.754988+0000","last_became_peered":"2023-11-08T19:28:15.754988+0000","last_unstale":"2023-11-08T19:28:15.759089+0000","last_undegraded":"2023-11-08T19:28:15.759089+0000","last_fullsized":"2023-11-08T19:28:15.759089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:14.726099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765417+0000","last_change":"2023-11-08T19:28:14.751203+0000","last_active":"2023-11-08T19:28:15.765417+0000","last_peered":"2023-11-08T19:28:15.765417+0000","last_clean":"2023-11-08T19:28:15.765417+0000","last_became_active":"2023-11-08T19:28:14.750951+0000","last_became_peered":"2023-11-08T19:28:14.750951+0000","last_unstale":"2023-11-08T19:28:15.765417+0000","last_undegraded":"2023-11-08T19:28:15.765417+0000","last_fullsized":"2023-11-08T19:28:15.765417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:07.976116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747409+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:14.747409+0000","last_peered":"2023-11-08T19:28:14.747409+0000","last_clean":"2023-11-08T19:28:14.747409+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:14.747409+0000","last_undegraded":"2023-11-08T19:28:14.747409+0000","last_fullsized":"2023-11-08T19:28:14.747409+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.749131+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:15.749131+0000","last_peered":"2023-11-08T19:28:15.749131+0000","last_clean":"2023-11-08T19:28:15.749131+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:15.749131+0000","last_undegraded":"2023-11-08T19:28:15.749131+0000","last_fullsized":"2023-11-08T19:28:15.749131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745669+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:14.745669+0000","last_peered":"2023-11-08T19:28:14.745669+0000","last_clean":"2023-11-08T19:28:14.745669+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:14.745669+0000","last_undegraded":"2023-11-08T19:28:14.745669+0000","last_fullsized":"2023-11-08T19:28:14.745669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766978+0000","last_change":"2023-11-08T19:28:15.756538+0000","last_active":"2023-11-08T19:28:15.766978+0000","last_peered":"2023-11-08T19:28:15.766978+0000","last_clean":"2023-11-08T19:28:15.766978+0000","last_became_active":"2023-11-08T19:28:15.756252+0000","last_became_peered":"2023-11-08T19:28:15.756252+0000","last_unstale":"2023-11-08T19:28:15.766978+0000","last_undegraded":"2023-11-08T19:28:15.766978+0000","last_fullsized":"2023-11-08T19:28:15.766978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:13:07.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.746644+0000","last_change":"2023-11-08T19:28:14.747095+0000","last_active":"2023-11-08T19:28:15.746644+0000","last_peered":"2023-11-08T19:28:15.746644+0000","last_clean":"2023-11-08T19:28:15.746644+0000","last_became_active":"2023-11-08T19:28:14.744802+0000","last_became_peered":"2023-11-08T19:28:14.744802+0000","last_unstale":"2023-11-08T19:28:15.746644+0000","last_undegraded":"2023-11-08T19:28:15.746644+0000","last_fullsized":"2023-11-08T19:28:15.746644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:53.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742055+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:14.742055+0000","last_peered":"2023-11-08T19:28:14.742055+0000","last_clean":"2023-11-08T19:28:14.742055+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:14.742055+0000","last_undegraded":"2023-11-08T19:28:14.742055+0000","last_fullsized":"2023-11-08T19:28:14.742055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722692+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:14.722692+0000","last_peered":"2023-11-08T19:28:14.722692+0000","last_clean":"2023-11-08T19:28:14.722692+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:14.722692+0000","last_undegraded":"2023-11-08T19:28:14.722692+0000","last_fullsized":"2023-11-08T19:28:14.722692+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.741233+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:14.741233+0000","last_peered":"2023-11-08T19:28:14.741233+0000","last_clean":"2023-11-08T19:28:14.741233+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:14.741233+0000","last_undegraded":"2023-11-08T19:28:14.741233+0000","last_fullsized":"2023-11-08T19:28:14.741233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759591+0000","last_change":"2023-11-08T19:28:15.754071+0000","last_active":"2023-11-08T19:28:15.759591+0000","last_peered":"2023-11-08T19:28:15.759591+0000","last_clean":"2023-11-08T19:28:15.759591+0000","last_became_active":"2023-11-08T19:28:15.753616+0000","last_became_peered":"2023-11-08T19:28:15.753616+0000","last_unstale":"2023-11-08T19:28:15.759591+0000","last_undegraded":"2023-11-08T19:28:15.759591+0000","last_fullsized":"2023-11-08T19:28:15.759591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:00.872144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.738550+0000","last_change":"2023-11-08T19:28:14.750055+0000","last_active":"2023-11-08T19:28:15.738550+0000","last_peered":"2023-11-08T19:28:15.738550+0000","last_clean":"2023-11-08T19:28:15.738550+0000","last_became_active":"2023-11-08T19:28:14.747228+0000","last_became_peered":"2023-11-08T19:28:14.747228+0000","last_unstale":"2023-11-08T19:28:15.738550+0000","last_undegraded":"2023-11-08T19:28:15.738550+0000","last_fullsized":"2023-11-08T19:28:15.738550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:09.925303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.720932+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:14.720932+0000","last_peered":"2023-11-08T19:28:14.720932+0000","last_clean":"2023-11-08T19:28:14.720932+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:14.720932+0000","last_undegraded":"2023-11-08T19:28:14.720932+0000","last_fullsized":"2023-11-08T19:28:14.720932+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.741154+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:14.741154+0000","last_peered":"2023-11-08T19:28:14.741154+0000","last_clean":"2023-11-08T19:28:14.741154+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:14.741154+0000","last_undegraded":"2023-11-08T19:28:14.741154+0000","last_fullsized":"2023-11-08T19:28:14.741154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742299+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:14.742299+0000","last_peered":"2023-11-08T19:28:14.742299+0000","last_clean":"2023-11-08T19:28:14.742299+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:14.742299+0000","last_undegraded":"2023-11-08T19:28:14.742299+0000","last_fullsized":"2023-11-08T19:28:14.742299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756259+0000","last_change":"2023-11-08T19:28:15.765799+0000","last_active":"2023-11-08T19:28:16.756259+0000","last_peered":"2023-11-08T19:28:16.756259+0000","last_clean":"2023-11-08T19:28:16.756259+0000","last_became_active":"2023-11-08T19:28:15.765542+0000","last_became_peered":"2023-11-08T19:28:15.765542+0000","last_unstale":"2023-11-08T19:28:16.756259+0000","last_undegraded":"2023-11-08T19:28:16.756259+0000","last_fullsized":"2023-11-08T19:28:16.756259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:40.606689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766203+0000","last_change":"2023-11-08T19:28:14.752387+0000","last_active":"2023-11-08T19:28:15.766203+0000","last_peered":"2023-11-08T19:28:15.766203+0000","last_clean":"2023-11-08T19:28:15.766203+0000","last_became_active":"2023-11-08T19:28:14.752105+0000","last_became_peered":"2023-11-08T19:28:14.752105+0000","last_unstale":"2023-11-08T19:28:15.766203+0000","last_undegraded":"2023-11-08T19:28:15.766203+0000","last_fullsized":"2023-11-08T19:28:15.766203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:06.283767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.721910+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:14.721910+0000","last_peered":"2023-11-08T19:28:14.721910+0000","last_clean":"2023-11-08T19:28:14.721910+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:14.721910+0000","last_undegraded":"2023-11-08T19:28:14.721910+0000","last_fullsized":"2023-11-08T19:28:14.721910+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.727728+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:14.727728+0000","last_peered":"2023-11-08T19:28:14.727728+0000","last_clean":"2023-11-08T19:28:14.727728+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:14.727728+0000","last_undegraded":"2023-11-08T19:28:14.727728+0000","last_fullsized":"2023-11-08T19:28:14.727728+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733666+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:14.733666+0000","last_peered":"2023-11-08T19:28:14.733666+0000","last_clean":"2023-11-08T19:28:14.733666+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:14.733666+0000","last_undegraded":"2023-11-08T19:28:14.733666+0000","last_fullsized":"2023-11-08T19:28:14.733666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.772873+0000","last_change":"2023-11-08T19:28:15.767990+0000","last_active":"2023-11-08T19:28:15.772873+0000","last_peered":"2023-11-08T19:28:15.772873+0000","last_clean":"2023-11-08T19:28:15.772873+0000","last_became_active":"2023-11-08T19:28:15.767733+0000","last_became_peered":"2023-11-08T19:28:15.767733+0000","last_unstale":"2023-11-08T19:28:15.772873+0000","last_undegraded":"2023-11-08T19:28:15.772873+0000","last_fullsized":"2023-11-08T19:28:15.772873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:57.342920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768120+0000","last_change":"2023-11-08T19:28:14.748780+0000","last_active":"2023-11-08T19:28:15.768120+0000","last_peered":"2023-11-08T19:28:15.768120+0000","last_clean":"2023-11-08T19:28:15.768120+0000","last_became_active":"2023-11-08T19:28:14.748358+0000","last_became_peered":"2023-11-08T19:28:14.748358+0000","last_unstale":"2023-11-08T19:28:15.768120+0000","last_undegraded":"2023-11-08T19:28:15.768120+0000","last_fullsized":"2023-11-08T19:28:15.768120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:25.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747276+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:14.747276+0000","last_peered":"2023-11-08T19:28:14.747276+0000","last_clean":"2023-11-08T19:28:14.747276+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:14.747276+0000","last_undegraded":"2023-11-08T19:28:14.747276+0000","last_fullsized":"2023-11-08T19:28:14.747276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745131+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:14.745131+0000","last_peered":"2023-11-08T19:28:14.745131+0000","last_clean":"2023-11-08T19:28:14.745131+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:14.745131+0000","last_undegraded":"2023-11-08T19:28:14.745131+0000","last_fullsized":"2023-11-08T19:28:14.745131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.749397+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:14.749397+0000","last_peered":"2023-11-08T19:28:14.749397+0000","last_clean":"2023-11-08T19:28:14.749397+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:14.749397+0000","last_undegraded":"2023-11-08T19:28:14.749397+0000","last_fullsized":"2023-11-08T19:28:14.749397+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.763731+0000","last_change":"2023-11-08T19:28:15.762004+0000","last_active":"2023-11-08T19:28:15.763731+0000","last_peered":"2023-11-08T19:28:15.763731+0000","last_clean":"2023-11-08T19:28:15.763731+0000","last_became_active":"2023-11-08T19:28:15.761560+0000","last_became_peered":"2023-11-08T19:28:15.761560+0000","last_unstale":"2023-11-08T19:28:15.763731+0000","last_undegraded":"2023-11-08T19:28:15.763731+0000","last_fullsized":"2023-11-08T19:28:15.763731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:36.912107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.758475+0000","last_change":"2023-11-08T19:28:14.749771+0000","last_active":"2023-11-08T19:28:15.758475+0000","last_peered":"2023-11-08T19:28:15.758475+0000","last_clean":"2023-11-08T19:28:15.758475+0000","last_became_active":"2023-11-08T19:28:14.749496+0000","last_became_peered":"2023-11-08T19:28:14.749496+0000","last_unstale":"2023-11-08T19:28:15.758475+0000","last_undegraded":"2023-11-08T19:28:15.758475+0000","last_fullsized":"2023-11-08T19:28:15.758475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:45:08.642083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744777+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:14.744777+0000","last_peered":"2023-11-08T19:28:14.744777+0000","last_clean":"2023-11-08T19:28:14.744777+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:14.744777+0000","last_undegraded":"2023-11-08T19:28:14.744777+0000","last_fullsized":"2023-11-08T19:28:14.744777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.746927+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:14.746927+0000","last_peered":"2023-11-08T19:28:14.746927+0000","last_clean":"2023-11-08T19:28:14.746927+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:14.746927+0000","last_undegraded":"2023-11-08T19:28:14.746927+0000","last_fullsized":"2023-11-08T19:28:14.746927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.750095+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:15.750095+0000","last_peered":"2023-11-08T19:28:15.750095+0000","last_clean":"2023-11-08T19:28:15.750095+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:15.750095+0000","last_undegraded":"2023-11-08T19:28:15.750095+0000","last_fullsized":"2023-11-08T19:28:15.750095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755396+0000","last_change":"2023-11-08T19:28:15.771829+0000","last_active":"2023-11-08T19:28:16.755396+0000","last_peered":"2023-11-08T19:28:16.755396+0000","last_clean":"2023-11-08T19:28:16.755396+0000","last_became_active":"2023-11-08T19:28:15.770178+0000","last_became_peered":"2023-11-08T19:28:15.770178+0000","last_unstale":"2023-11-08T19:28:16.755396+0000","last_undegraded":"2023-11-08T19:28:16.755396+0000","last_fullsized":"2023-11-08T19:28:16.755396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:09.019260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773989+0000","last_change":"2023-11-08T19:28:14.759338+0000","last_active":"2023-11-08T19:28:15.773989+0000","last_peered":"2023-11-08T19:28:15.773989+0000","last_clean":"2023-11-08T19:28:15.773989+0000","last_became_active":"2023-11-08T19:28:14.758982+0000","last_became_peered":"2023-11-08T19:28:14.758982+0000","last_unstale":"2023-11-08T19:28:15.773989+0000","last_undegraded":"2023-11-08T19:28:15.773989+0000","last_fullsized":"2023-11-08T19:28:15.773989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:53.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.748790+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:14.748790+0000","last_peered":"2023-11-08T19:28:14.748790+0000","last_clean":"2023-11-08T19:28:14.748790+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:14.748790+0000","last_undegraded":"2023-11-08T19:28:14.748790+0000","last_fullsized":"2023-11-08T19:28:14.748790+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742643+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:14.742643+0000","last_peered":"2023-11-08T19:28:14.742643+0000","last_clean":"2023-11-08T19:28:14.742643+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:14.742643+0000","last_undegraded":"2023-11-08T19:28:14.742643+0000","last_fullsized":"2023-11-08T19:28:14.742643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750961+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:14.750961+0000","last_peered":"2023-11-08T19:28:14.750961+0000","last_clean":"2023-11-08T19:28:14.750961+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:14.750961+0000","last_undegraded":"2023-11-08T19:28:14.750961+0000","last_fullsized":"2023-11-08T19:28:14.750961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768644+0000","last_change":"2023-11-08T19:28:15.766459+0000","last_active":"2023-11-08T19:28:15.768644+0000","last_peered":"2023-11-08T19:28:15.768644+0000","last_clean":"2023-11-08T19:28:15.768644+0000","last_became_active":"2023-11-08T19:28:15.766140+0000","last_became_peered":"2023-11-08T19:28:15.766140+0000","last_unstale":"2023-11-08T19:28:15.768644+0000","last_undegraded":"2023-11-08T19:28:15.768644+0000","last_fullsized":"2023-11-08T19:28:15.768644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:53.530158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"82'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768452+0000","last_change":"2023-11-08T19:28:14.755044+0000","last_active":"2023-11-08T19:28:15.768452+0000","last_peered":"2023-11-08T19:28:15.768452+0000","last_clean":"2023-11-08T19:28:15.768452+0000","last_became_active":"2023-11-08T19:28:14.754750+0000","last_became_peered":"2023-11-08T19:28:14.754750+0000","last_unstale":"2023-11-08T19:28:15.768452+0000","last_undegraded":"2023-11-08T19:28:15.768452+0000","last_fullsized":"2023-11-08T19:28:15.768452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:13.380399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745572+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:14.745572+0000","last_peered":"2023-11-08T19:28:14.745572+0000","last_clean":"2023-11-08T19:28:14.745572+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:14.745572+0000","last_undegraded":"2023-11-08T19:28:14.745572+0000","last_fullsized":"2023-11-08T19:28:14.745572+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743381+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:14.743381+0000","last_peered":"2023-11-08T19:28:14.743381+0000","last_clean":"2023-11-08T19:28:14.743381+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:14.743381+0000","last_undegraded":"2023-11-08T19:28:14.743381+0000","last_fullsized":"2023-11-08T19:28:14.743381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745381+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:14.745381+0000","last_peered":"2023-11-08T19:28:14.745381+0000","last_clean":"2023-11-08T19:28:14.745381+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:14.745381+0000","last_undegraded":"2023-11-08T19:28:14.745381+0000","last_fullsized":"2023-11-08T19:28:14.745381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766998+0000","last_change":"2023-11-08T19:28:15.753808+0000","last_active":"2023-11-08T19:28:15.766998+0000","last_peered":"2023-11-08T19:28:15.766998+0000","last_clean":"2023-11-08T19:28:15.766998+0000","last_became_active":"2023-11-08T19:28:15.753523+0000","last_became_peered":"2023-11-08T19:28:15.753523+0000","last_unstale":"2023-11-08T19:28:15.766998+0000","last_undegraded":"2023-11-08T19:28:15.766998+0000","last_fullsized":"2023-11-08T19:28:15.766998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:57.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.731969+0000","last_change":"2023-11-08T19:28:14.742054+0000","last_active":"2023-11-08T19:28:15.731969+0000","last_peered":"2023-11-08T19:28:15.731969+0000","last_clean":"2023-11-08T19:28:15.731969+0000","last_became_active":"2023-11-08T19:28:14.741766+0000","last_became_peered":"2023-11-08T19:28:14.741766+0000","last_unstale":"2023-11-08T19:28:15.731969+0000","last_undegraded":"2023-11-08T19:28:15.731969+0000","last_fullsized":"2023-11-08T19:28:15.731969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:23.666540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733774+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:14.733774+0000","last_peered":"2023-11-08T19:28:14.733774+0000","last_clean":"2023-11-08T19:28:14.733774+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:14.733774+0000","last_undegraded":"2023-11-08T19:28:14.733774+0000","last_fullsized":"2023-11-08T19:28:14.733774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733221+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:14.733221+0000","last_peered":"2023-11-08T19:28:14.733221+0000","last_clean":"2023-11-08T19:28:14.733221+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:14.733221+0000","last_undegraded":"2023-11-08T19:28:14.733221+0000","last_fullsized":"2023-11-08T19:28:14.733221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743039+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:14.743039+0000","last_peered":"2023-11-08T19:28:14.743039+0000","last_clean":"2023-11-08T19:28:14.743039+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:14.743039+0000","last_undegraded":"2023-11-08T19:28:14.743039+0000","last_fullsized":"2023-11-08T19:28:14.743039+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755568+0000","last_change":"2023-11-08T19:28:15.760589+0000","last_active":"2023-11-08T19:28:16.755568+0000","last_peered":"2023-11-08T19:28:16.755568+0000","last_clean":"2023-11-08T19:28:16.755568+0000","last_became_active":"2023-11-08T19:28:15.760326+0000","last_became_peered":"2023-11-08T19:28:15.760326+0000","last_unstale":"2023-11-08T19:28:16.755568+0000","last_undegraded":"2023-11-08T19:28:16.755568+0000","last_fullsized":"2023-11-08T19:28:16.755568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:17.476942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.734825+0000","last_change":"2023-11-08T19:28:14.754260+0000","last_active":"2023-11-08T19:28:15.734825+0000","last_peered":"2023-11-08T19:28:15.734825+0000","last_clean":"2023-11-08T19:28:15.734825+0000","last_became_active":"2023-11-08T19:28:14.753948+0000","last_became_peered":"2023-11-08T19:28:14.753948+0000","last_unstale":"2023-11-08T19:28:15.734825+0000","last_undegraded":"2023-11-08T19:28:15.734825+0000","last_fullsized":"2023-11-08T19:28:15.734825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:13.513400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.721124+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:14.721124+0000","last_peered":"2023-11-08T19:28:14.721124+0000","last_clean":"2023-11-08T19:28:14.721124+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:14.721124+0000","last_undegraded":"2023-11-08T19:28:14.721124+0000","last_fullsized":"2023-11-08T19:28:14.721124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733969+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:14.733969+0000","last_peered":"2023-11-08T19:28:14.733969+0000","last_clean":"2023-11-08T19:28:14.733969+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:14.733969+0000","last_undegraded":"2023-11-08T19:28:14.733969+0000","last_fullsized":"2023-11-08T19:28:14.733969+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748192+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:15.748192+0000","last_peered":"2023-11-08T19:28:15.748192+0000","last_clean":"2023-11-08T19:28:15.748192+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:15.748192+0000","last_undegraded":"2023-11-08T19:28:15.748192+0000","last_fullsized":"2023-11-08T19:28:15.748192+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773048+0000","last_change":"2023-11-08T19:28:15.768033+0000","last_active":"2023-11-08T19:28:15.773048+0000","last_peered":"2023-11-08T19:28:15.773048+0000","last_clean":"2023-11-08T19:28:15.773048+0000","last_became_active":"2023-11-08T19:28:15.767708+0000","last_became_peered":"2023-11-08T19:28:15.767708+0000","last_unstale":"2023-11-08T19:28:15.773048+0000","last_undegraded":"2023-11-08T19:28:15.773048+0000","last_fullsized":"2023-11-08T19:28:15.773048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:51.558891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.760916+0000","last_change":"2023-11-08T19:28:14.752035+0000","last_active":"2023-11-08T19:28:15.760916+0000","last_peered":"2023-11-08T19:28:15.760916+0000","last_clean":"2023-11-08T19:28:15.760916+0000","last_became_active":"2023-11-08T19:28:14.751710+0000","last_became_peered":"2023-11-08T19:28:14.751710+0000","last_unstale":"2023-11-08T19:28:15.760916+0000","last_undegraded":"2023-11-08T19:28:15.760916+0000","last_fullsized":"2023-11-08T19:28:15.760916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:59:53.400186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748566+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:15.748566+0000","last_peered":"2023-11-08T19:28:15.748566+0000","last_clean":"2023-11-08T19:28:15.748566+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:15.748566+0000","last_undegraded":"2023-11-08T19:28:15.748566+0000","last_fullsized":"2023-11-08T19:28:15.748566+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722022+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:14.722022+0000","last_peered":"2023-11-08T19:28:14.722022+0000","last_clean":"2023-11-08T19:28:14.722022+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:14.722022+0000","last_undegraded":"2023-11-08T19:28:14.722022+0000","last_fullsized":"2023-11-08T19:28:14.722022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742366+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:14.742366+0000","last_peered":"2023-11-08T19:28:14.742366+0000","last_clean":"2023-11-08T19:28:14.742366+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:14.742366+0000","last_undegraded":"2023-11-08T19:28:14.742366+0000","last_fullsized":"2023-11-08T19:28:14.742366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766092+0000","last_change":"2023-11-08T19:28:15.754355+0000","last_active":"2023-11-08T19:28:15.766092+0000","last_peered":"2023-11-08T19:28:15.766092+0000","last_clean":"2023-11-08T19:28:15.766092+0000","last_became_active":"2023-11-08T19:28:15.754041+0000","last_became_peered":"2023-11-08T19:28:15.754041+0000","last_unstale":"2023-11-08T19:28:15.766092+0000","last_undegraded":"2023-11-08T19:28:15.766092+0000","last_fullsized":"2023-11-08T19:28:15.766092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:57.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.761036+0000","last_change":"2023-11-08T19:28:14.750245+0000","last_active":"2023-11-08T19:28:15.761036+0000","last_peered":"2023-11-08T19:28:15.761036+0000","last_clean":"2023-11-08T19:28:15.761036+0000","last_became_active":"2023-11-08T19:28:14.749951+0000","last_became_peered":"2023-11-08T19:28:14.749951+0000","last_unstale":"2023-11-08T19:28:15.761036+0000","last_undegraded":"2023-11-08T19:28:15.761036+0000","last_fullsized":"2023-11-08T19:28:15.761036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:56.096338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744053+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:14.744053+0000","last_peered":"2023-11-08T19:28:14.744053+0000","last_clean":"2023-11-08T19:28:14.744053+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:14.744053+0000","last_undegraded":"2023-11-08T19:28:14.744053+0000","last_fullsized":"2023-11-08T19:28:14.744053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742130+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:14.742130+0000","last_peered":"2023-11-08T19:28:14.742130+0000","last_clean":"2023-11-08T19:28:14.742130+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:14.742130+0000","last_undegraded":"2023-11-08T19:28:14.742130+0000","last_fullsized":"2023-11-08T19:28:14.742130+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.749495+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:14.749495+0000","last_peered":"2023-11-08T19:28:14.749495+0000","last_clean":"2023-11-08T19:28:14.749495+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:14.749495+0000","last_undegraded":"2023-11-08T19:28:14.749495+0000","last_fullsized":"2023-11-08T19:28:14.749495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770074+0000","last_change":"2023-11-08T19:28:15.765356+0000","last_active":"2023-11-08T19:28:15.770074+0000","last_peered":"2023-11-08T19:28:15.770074+0000","last_clean":"2023-11-08T19:28:15.770074+0000","last_became_active":"2023-11-08T19:28:15.765089+0000","last_became_peered":"2023-11-08T19:28:15.765089+0000","last_unstale":"2023-11-08T19:28:15.770074+0000","last_undegraded":"2023-11-08T19:28:15.770074+0000","last_fullsized":"2023-11-08T19:28:15.770074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:40.670042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.733911+0000","last_change":"2023-11-08T19:28:14.751940+0000","last_active":"2023-11-08T19:28:15.733911+0000","last_peered":"2023-11-08T19:28:15.733911+0000","last_clean":"2023-11-08T19:28:15.733911+0000","last_became_active":"2023-11-08T19:28:14.751598+0000","last_became_peered":"2023-11-08T19:28:14.751598+0000","last_unstale":"2023-11-08T19:28:15.733911+0000","last_undegraded":"2023-11-08T19:28:15.733911+0000","last_fullsized":"2023-11-08T19:28:15.733911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:40.572679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750767+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:14.750767+0000","last_peered":"2023-11-08T19:28:14.750767+0000","last_clean":"2023-11-08T19:28:14.750767+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:14.750767+0000","last_undegraded":"2023-11-08T19:28:14.750767+0000","last_fullsized":"2023-11-08T19:28:14.750767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722585+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:14.722585+0000","last_peered":"2023-11-08T19:28:14.722585+0000","last_clean":"2023-11-08T19:28:14.722585+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:14.722585+0000","last_undegraded":"2023-11-08T19:28:14.722585+0000","last_fullsized":"2023-11-08T19:28:14.722585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.746240+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:14.746240+0000","last_peered":"2023-11-08T19:28:14.746240+0000","last_clean":"2023-11-08T19:28:14.746240+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:14.746240+0000","last_undegraded":"2023-11-08T19:28:14.746240+0000","last_fullsized":"2023-11-08T19:28:14.746240+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.762346+0000","last_change":"2023-11-08T19:28:15.752353+0000","last_active":"2023-11-08T19:28:15.762346+0000","last_peered":"2023-11-08T19:28:15.762346+0000","last_clean":"2023-11-08T19:28:15.762346+0000","last_became_active":"2023-11-08T19:28:15.751576+0000","last_became_peered":"2023-11-08T19:28:15.751576+0000","last_unstale":"2023-11-08T19:28:15.762346+0000","last_undegraded":"2023-11-08T19:28:15.762346+0000","last_fullsized":"2023-11-08T19:28:15.762346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:06.852441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.727933+0000","last_change":"2023-11-08T19:28:14.755494+0000","last_active":"2023-11-08T19:28:15.727933+0000","last_peered":"2023-11-08T19:28:15.727933+0000","last_clean":"2023-11-08T19:28:15.727933+0000","last_became_active":"2023-11-08T19:28:14.754954+0000","last_became_peered":"2023-11-08T19:28:14.754954+0000","last_unstale":"2023-11-08T19:28:15.727933+0000","last_undegraded":"2023-11-08T19:28:15.727933+0000","last_fullsized":"2023-11-08T19:28:15.727933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:37:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745305+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:14.745305+0000","last_peered":"2023-11-08T19:28:14.745305+0000","last_clean":"2023-11-08T19:28:14.745305+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:14.745305+0000","last_undegraded":"2023-11-08T19:28:14.745305+0000","last_fullsized":"2023-11-08T19:28:14.745305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.723302+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:14.723302+0000","last_peered":"2023-11-08T19:28:14.723302+0000","last_clean":"2023-11-08T19:28:14.723302+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:14.723302+0000","last_undegraded":"2023-11-08T19:28:14.723302+0000","last_fullsized":"2023-11-08T19:28:14.723302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745850+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:14.745850+0000","last_peered":"2023-11-08T19:28:14.745850+0000","last_clean":"2023-11-08T19:28:14.745850+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:14.745850+0000","last_undegraded":"2023-11-08T19:28:14.745850+0000","last_fullsized":"2023-11-08T19:28:14.745850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759582+0000","last_change":"2023-11-08T19:28:15.752263+0000","last_active":"2023-11-08T19:28:15.759582+0000","last_peered":"2023-11-08T19:28:15.759582+0000","last_clean":"2023-11-08T19:28:15.759582+0000","last_became_active":"2023-11-08T19:28:15.751722+0000","last_became_peered":"2023-11-08T19:28:15.751722+0000","last_unstale":"2023-11-08T19:28:15.759582+0000","last_undegraded":"2023-11-08T19:28:15.759582+0000","last_fullsized":"2023-11-08T19:28:15.759582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:57:25.668314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.731824+0000","last_change":"2023-11-08T19:28:14.752706+0000","last_active":"2023-11-08T19:28:15.731824+0000","last_peered":"2023-11-08T19:28:15.731824+0000","last_clean":"2023-11-08T19:28:15.731824+0000","last_became_active":"2023-11-08T19:28:14.751744+0000","last_became_peered":"2023-11-08T19:28:14.751744+0000","last_unstale":"2023-11-08T19:28:15.731824+0000","last_undegraded":"2023-11-08T19:28:15.731824+0000","last_fullsized":"2023-11-08T19:28:15.731824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:37:59.108961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742241+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:14.742241+0000","last_peered":"2023-11-08T19:28:14.742241+0000","last_clean":"2023-11-08T19:28:14.742241+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:14.742241+0000","last_undegraded":"2023-11-08T19:28:14.742241+0000","last_fullsized":"2023-11-08T19:28:14.742241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745669+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:14.745669+0000","last_peered":"2023-11-08T19:28:14.745669+0000","last_clean":"2023-11-08T19:28:14.745669+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:14.745669+0000","last_undegraded":"2023-11-08T19:28:14.745669+0000","last_fullsized":"2023-11-08T19:28:14.745669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743150+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:14.743150+0000","last_peered":"2023-11-08T19:28:14.743150+0000","last_clean":"2023-11-08T19:28:14.743150+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:14.743150+0000","last_undegraded":"2023-11-08T19:28:14.743150+0000","last_fullsized":"2023-11-08T19:28:14.743150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759244+0000","last_change":"2023-11-08T19:28:15.754884+0000","last_active":"2023-11-08T19:28:15.759244+0000","last_peered":"2023-11-08T19:28:15.759244+0000","last_clean":"2023-11-08T19:28:15.759244+0000","last_became_active":"2023-11-08T19:28:15.754594+0000","last_became_peered":"2023-11-08T19:28:15.754594+0000","last_unstale":"2023-11-08T19:28:15.759244+0000","last_undegraded":"2023-11-08T19:28:15.759244+0000","last_fullsized":"2023-11-08T19:28:15.759244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:40.606689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.772149+0000","last_change":"2023-11-08T19:28:14.749740+0000","last_active":"2023-11-08T19:28:15.772149+0000","last_peered":"2023-11-08T19:28:15.772149+0000","last_clean":"2023-11-08T19:28:15.772149+0000","last_became_active":"2023-11-08T19:28:14.749459+0000","last_became_peered":"2023-11-08T19:28:14.749459+0000","last_unstale":"2023-11-08T19:28:15.772149+0000","last_undegraded":"2023-11-08T19:28:15.772149+0000","last_fullsized":"2023-11-08T19:28:15.772149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:45.277928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745774+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:14.745774+0000","last_peered":"2023-11-08T19:28:14.745774+0000","last_clean":"2023-11-08T19:28:14.745774+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:14.745774+0000","last_undegraded":"2023-11-08T19:28:14.745774+0000","last_fullsized":"2023-11-08T19:28:14.745774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745464+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:14.745464+0000","last_peered":"2023-11-08T19:28:14.745464+0000","last_clean":"2023-11-08T19:28:14.745464+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:14.745464+0000","last_undegraded":"2023-11-08T19:28:14.745464+0000","last_fullsized":"2023-11-08T19:28:14.745464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744408+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:14.744408+0000","last_peered":"2023-11-08T19:28:14.744408+0000","last_clean":"2023-11-08T19:28:14.744408+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:14.744408+0000","last_undegraded":"2023-11-08T19:28:14.744408+0000","last_fullsized":"2023-11-08T19:28:14.744408+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770419+0000","last_change":"2023-11-08T19:28:15.764787+0000","last_active":"2023-11-08T19:28:15.770419+0000","last_peered":"2023-11-08T19:28:15.770419+0000","last_clean":"2023-11-08T19:28:15.770419+0000","last_became_active":"2023-11-08T19:28:15.764476+0000","last_became_peered":"2023-11-08T19:28:15.764476+0000","last_unstale":"2023-11-08T19:28:15.770419+0000","last_undegraded":"2023-11-08T19:28:15.770419+0000","last_fullsized":"2023-11-08T19:28:15.770419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:31.801434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.736717+0000","last_change":"2023-11-08T19:28:14.754289+0000","last_active":"2023-11-08T19:28:15.736717+0000","last_peered":"2023-11-08T19:28:15.736717+0000","last_clean":"2023-11-08T19:28:15.736717+0000","last_became_active":"2023-11-08T19:28:14.754024+0000","last_became_peered":"2023-11-08T19:28:14.754024+0000","last_unstale":"2023-11-08T19:28:15.736717+0000","last_undegraded":"2023-11-08T19:28:15.736717+0000","last_fullsized":"2023-11-08T19:28:15.736717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:57.783594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742060+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:14.742060+0000","last_peered":"2023-11-08T19:28:14.742060+0000","last_clean":"2023-11-08T19:28:14.742060+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:14.742060+0000","last_undegraded":"2023-11-08T19:28:14.742060+0000","last_fullsized":"2023-11-08T19:28:14.742060+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733432+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:14.733432+0000","last_peered":"2023-11-08T19:28:14.733432+0000","last_clean":"2023-11-08T19:28:14.733432+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:14.733432+0000","last_undegraded":"2023-11-08T19:28:14.733432+0000","last_fullsized":"2023-11-08T19:28:14.733432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742021+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:14.742021+0000","last_peered":"2023-11-08T19:28:14.742021+0000","last_clean":"2023-11-08T19:28:14.742021+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:14.742021+0000","last_undegraded":"2023-11-08T19:28:14.742021+0000","last_fullsized":"2023-11-08T19:28:14.742021+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759122+0000","last_change":"2023-11-08T19:28:15.754694+0000","last_active":"2023-11-08T19:28:15.759122+0000","last_peered":"2023-11-08T19:28:15.759122+0000","last_clean":"2023-11-08T19:28:15.759122+0000","last_became_active":"2023-11-08T19:28:15.753441+0000","last_became_peered":"2023-11-08T19:28:15.753441+0000","last_unstale":"2023-11-08T19:28:15.759122+0000","last_undegraded":"2023-11-08T19:28:15.759122+0000","last_fullsized":"2023-11-08T19:28:15.759122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:48.543261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756792+0000","last_change":"2023-11-08T19:28:14.741270+0000","last_active":"2023-11-08T19:28:16.756792+0000","last_peered":"2023-11-08T19:28:16.756792+0000","last_clean":"2023-11-08T19:28:16.756792+0000","last_became_active":"2023-11-08T19:28:14.740995+0000","last_became_peered":"2023-11-08T19:28:14.740995+0000","last_unstale":"2023-11-08T19:28:16.756792+0000","last_undegraded":"2023-11-08T19:28:16.756792+0000","last_fullsized":"2023-11-08T19:28:16.756792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:12.090286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.729040+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:14.729040+0000","last_peered":"2023-11-08T19:28:14.729040+0000","last_clean":"2023-11-08T19:28:14.729040+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:14.729040+0000","last_undegraded":"2023-11-08T19:28:14.729040+0000","last_fullsized":"2023-11-08T19:28:14.729040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742219+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:14.742219+0000","last_peered":"2023-11-08T19:28:14.742219+0000","last_clean":"2023-11-08T19:28:14.742219+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:14.742219+0000","last_undegraded":"2023-11-08T19:28:14.742219+0000","last_fullsized":"2023-11-08T19:28:14.742219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.740234+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:14.740234+0000","last_peered":"2023-11-08T19:28:14.740234+0000","last_clean":"2023-11-08T19:28:14.740234+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:14.740234+0000","last_undegraded":"2023-11-08T19:28:14.740234+0000","last_fullsized":"2023-11-08T19:28:14.740234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.771392+0000","last_change":"2023-11-08T19:28:15.761636+0000","last_active":"2023-11-08T19:28:15.771392+0000","last_peered":"2023-11-08T19:28:15.771392+0000","last_clean":"2023-11-08T19:28:15.771392+0000","last_became_active":"2023-11-08T19:28:15.759087+0000","last_became_peered":"2023-11-08T19:28:15.759087+0000","last_unstale":"2023-11-08T19:28:15.771392+0000","last_undegraded":"2023-11-08T19:28:15.771392+0000","last_fullsized":"2023-11-08T19:28:15.771392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:31.801434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.732308+0000","last_change":"2023-11-08T19:28:14.754762+0000","last_active":"2023-11-08T19:28:15.732308+0000","last_peered":"2023-11-08T19:28:15.732308+0000","last_clean":"2023-11-08T19:28:15.732308+0000","last_became_active":"2023-11-08T19:28:14.754429+0000","last_became_peered":"2023-11-08T19:28:14.754429+0000","last_unstale":"2023-11-08T19:28:15.732308+0000","last_undegraded":"2023-11-08T19:28:15.732308+0000","last_fullsized":"2023-11-08T19:28:15.732308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:18.066983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745303+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:14.745303+0000","last_peered":"2023-11-08T19:28:14.745303+0000","last_clean":"2023-11-08T19:28:14.745303+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:14.745303+0000","last_undegraded":"2023-11-08T19:28:14.745303+0000","last_fullsized":"2023-11-08T19:28:14.745303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733180+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:14.733180+0000","last_peered":"2023-11-08T19:28:14.733180+0000","last_clean":"2023-11-08T19:28:14.733180+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:14.733180+0000","last_undegraded":"2023-11-08T19:28:14.733180+0000","last_fullsized":"2023-11-08T19:28:14.733180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722810+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:14.722810+0000","last_peered":"2023-11-08T19:28:14.722810+0000","last_clean":"2023-11-08T19:28:14.722810+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:14.722810+0000","last_undegraded":"2023-11-08T19:28:14.722810+0000","last_fullsized":"2023-11-08T19:28:14.722810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.762531+0000","last_change":"2023-11-08T19:28:15.754080+0000","last_active":"2023-11-08T19:28:15.762531+0000","last_peered":"2023-11-08T19:28:15.762531+0000","last_clean":"2023-11-08T19:28:15.762531+0000","last_became_active":"2023-11-08T19:28:15.753740+0000","last_became_peered":"2023-11-08T19:28:15.753740+0000","last_unstale":"2023-11-08T19:28:15.762531+0000","last_undegraded":"2023-11-08T19:28:15.762531+0000","last_fullsized":"2023-11-08T19:28:15.762531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:47.085580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.729018+0000","last_change":"2023-11-08T19:28:14.746433+0000","last_active":"2023-11-08T19:28:15.729018+0000","last_peered":"2023-11-08T19:28:15.729018+0000","last_clean":"2023-11-08T19:28:15.729018+0000","last_became_active":"2023-11-08T19:28:14.746149+0000","last_became_peered":"2023-11-08T19:28:14.746149+0000","last_unstale":"2023-11-08T19:28:15.729018+0000","last_undegraded":"2023-11-08T19:28:15.729018+0000","last_fullsized":"2023-11-08T19:28:15.729018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:30.481143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744155+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:14.744155+0000","last_peered":"2023-11-08T19:28:14.744155+0000","last_clean":"2023-11-08T19:28:14.744155+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:14.744155+0000","last_undegraded":"2023-11-08T19:28:14.744155+0000","last_fullsized":"2023-11-08T19:28:14.744155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745464+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:14.745464+0000","last_peered":"2023-11-08T19:28:14.745464+0000","last_clean":"2023-11-08T19:28:14.745464+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:14.745464+0000","last_undegraded":"2023-11-08T19:28:14.745464+0000","last_fullsized":"2023-11-08T19:28:14.745464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.749259+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:15.749259+0000","last_peered":"2023-11-08T19:28:15.749259+0000","last_clean":"2023-11-08T19:28:15.749259+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:15.749259+0000","last_undegraded":"2023-11-08T19:28:15.749259+0000","last_fullsized":"2023-11-08T19:28:15.749259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.777033+0000","last_change":"2023-11-08T19:28:15.772651+0000","last_active":"2023-11-08T19:28:15.777033+0000","last_peered":"2023-11-08T19:28:15.777033+0000","last_clean":"2023-11-08T19:28:15.777033+0000","last_became_active":"2023-11-08T19:28:15.772183+0000","last_became_peered":"2023-11-08T19:28:15.772183+0000","last_unstale":"2023-11-08T19:28:15.777033+0000","last_undegraded":"2023-11-08T19:28:15.777033+0000","last_fullsized":"2023-11-08T19:28:15.777033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:38:48.264507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.728756+0000","last_change":"2023-11-08T19:28:14.748551+0000","last_active":"2023-11-08T19:28:15.728756+0000","last_peered":"2023-11-08T19:28:15.728756+0000","last_clean":"2023-11-08T19:28:15.728756+0000","last_became_active":"2023-11-08T19:28:14.748175+0000","last_became_peered":"2023-11-08T19:28:14.748175+0000","last_unstale":"2023-11-08T19:28:15.728756+0000","last_undegraded":"2023-11-08T19:28:15.728756+0000","last_fullsized":"2023-11-08T19:28:15.728756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:12.090286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747198+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:14.747198+0000","last_peered":"2023-11-08T19:28:14.747198+0000","last_clean":"2023-11-08T19:28:14.747198+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:14.747198+0000","last_undegraded":"2023-11-08T19:28:14.747198+0000","last_fullsized":"2023-11-08T19:28:14.747198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743714+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:14.743714+0000","last_peered":"2023-11-08T19:28:14.743714+0000","last_clean":"2023-11-08T19:28:14.743714+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:14.743714+0000","last_undegraded":"2023-11-08T19:28:14.743714+0000","last_fullsized":"2023-11-08T19:28:14.743714+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750528+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:14.750528+0000","last_peered":"2023-11-08T19:28:14.750528+0000","last_clean":"2023-11-08T19:28:14.750528+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:14.750528+0000","last_undegraded":"2023-11-08T19:28:14.750528+0000","last_fullsized":"2023-11-08T19:28:14.750528+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766047+0000","last_change":"2023-11-08T19:28:15.756945+0000","last_active":"2023-11-08T19:28:15.766047+0000","last_peered":"2023-11-08T19:28:15.766047+0000","last_clean":"2023-11-08T19:28:15.766047+0000","last_became_active":"2023-11-08T19:28:15.756618+0000","last_became_peered":"2023-11-08T19:28:15.756618+0000","last_unstale":"2023-11-08T19:28:15.766047+0000","last_undegraded":"2023-11-08T19:28:15.766047+0000","last_fullsized":"2023-11-08T19:28:15.766047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:31.801434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.771516+0000","last_change":"2023-11-08T19:28:14.748630+0000","last_active":"2023-11-08T19:28:15.771516+0000","last_peered":"2023-11-08T19:28:15.771516+0000","last_clean":"2023-11-08T19:28:15.771516+0000","last_became_active":"2023-11-08T19:28:14.748275+0000","last_became_peered":"2023-11-08T19:28:14.748275+0000","last_unstale":"2023-11-08T19:28:15.771516+0000","last_undegraded":"2023-11-08T19:28:15.771516+0000","last_fullsized":"2023-11-08T19:28:15.771516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:21.923551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750935+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:14.750935+0000","last_peered":"2023-11-08T19:28:14.750935+0000","last_clean":"2023-11-08T19:28:14.750935+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:14.750935+0000","last_undegraded":"2023-11-08T19:28:14.750935+0000","last_fullsized":"2023-11-08T19:28:14.750935+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747123+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:14.747123+0000","last_peered":"2023-11-08T19:28:14.747123+0000","last_clean":"2023-11-08T19:28:14.747123+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:14.747123+0000","last_undegraded":"2023-11-08T19:28:14.747123+0000","last_fullsized":"2023-11-08T19:28:14.747123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742716+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:14.742716+0000","last_peered":"2023-11-08T19:28:14.742716+0000","last_clean":"2023-11-08T19:28:14.742716+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:14.742716+0000","last_undegraded":"2023-11-08T19:28:14.742716+0000","last_fullsized":"2023-11-08T19:28:14.742716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766383+0000","last_change":"2023-11-08T19:28:15.757403+0000","last_active":"2023-11-08T19:28:15.766383+0000","last_peered":"2023-11-08T19:28:15.766383+0000","last_clean":"2023-11-08T19:28:15.766383+0000","last_became_active":"2023-11-08T19:28:15.757090+0000","last_became_peered":"2023-11-08T19:28:15.757090+0000","last_unstale":"2023-11-08T19:28:15.766383+0000","last_undegraded":"2023-11-08T19:28:15.766383+0000","last_fullsized":"2023-11-08T19:28:15.766383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:57.342920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.763762+0000","last_change":"2023-11-08T19:28:14.750109+0000","last_active":"2023-11-08T19:28:15.763762+0000","last_peered":"2023-11-08T19:28:15.763762+0000","last_clean":"2023-11-08T19:28:15.763762+0000","last_became_active":"2023-11-08T19:28:14.749769+0000","last_became_peered":"2023-11-08T19:28:14.749769+0000","last_unstale":"2023-11-08T19:28:15.763762+0000","last_undegraded":"2023-11-08T19:28:15.763762+0000","last_fullsized":"2023-11-08T19:28:15.763762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:40.572679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742124+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:14.742124+0000","last_peered":"2023-11-08T19:28:14.742124+0000","last_clean":"2023-11-08T19:28:14.742124+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:14.742124+0000","last_undegraded":"2023-11-08T19:28:14.742124+0000","last_fullsized":"2023-11-08T19:28:14.742124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742574+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:14.742574+0000","last_peered":"2023-11-08T19:28:14.742574+0000","last_clean":"2023-11-08T19:28:14.742574+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:14.742574+0000","last_undegraded":"2023-11-08T19:28:14.742574+0000","last_fullsized":"2023-11-08T19:28:14.742574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747455+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:14.747455+0000","last_peered":"2023-11-08T19:28:14.747455+0000","last_clean":"2023-11-08T19:28:14.747455+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:14.747455+0000","last_undegraded":"2023-11-08T19:28:14.747455+0000","last_fullsized":"2023-11-08T19:28:14.747455+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770398+0000","last_change":"2023-11-08T19:28:15.762902+0000","last_active":"2023-11-08T19:28:15.770398+0000","last_peered":"2023-11-08T19:28:15.770398+0000","last_clean":"2023-11-08T19:28:15.770398+0000","last_became_active":"2023-11-08T19:28:15.762563+0000","last_became_peered":"2023-11-08T19:28:15.762563+0000","last_unstale":"2023-11-08T19:28:15.770398+0000","last_undegraded":"2023-11-08T19:28:15.770398+0000","last_fullsized":"2023-11-08T19:28:15.770398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:38:48.264507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.726799+0000","last_change":"2023-11-08T19:28:14.747542+0000","last_active":"2023-11-08T19:28:15.726799+0000","last_peered":"2023-11-08T19:28:15.726799+0000","last_clean":"2023-11-08T19:28:15.726799+0000","last_became_active":"2023-11-08T19:28:14.747121+0000","last_became_peered":"2023-11-08T19:28:14.747121+0000","last_unstale":"2023-11-08T19:28:15.726799+0000","last_undegraded":"2023-11-08T19:28:15.726799+0000","last_fullsized":"2023-11-08T19:28:15.726799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:49:22.533166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.726792+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:14.726792+0000","last_peered":"2023-11-08T19:28:14.726792+0000","last_clean":"2023-11-08T19:28:14.726792+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:14.726792+0000","last_undegraded":"2023-11-08T19:28:14.726792+0000","last_fullsized":"2023-11-08T19:28:14.726792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.739090+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:14.739090+0000","last_peered":"2023-11-08T19:28:14.739090+0000","last_clean":"2023-11-08T19:28:14.739090+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:14.739090+0000","last_undegraded":"2023-11-08T19:28:14.739090+0000","last_fullsized":"2023-11-08T19:28:14.739090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.734680+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:14.734680+0000","last_peered":"2023-11-08T19:28:14.734680+0000","last_clean":"2023-11-08T19:28:14.734680+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:14.734680+0000","last_undegraded":"2023-11-08T19:28:14.734680+0000","last_fullsized":"2023-11-08T19:28:14.734680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.760106+0000","last_change":"2023-11-08T19:28:15.755779+0000","last_active":"2023-11-08T19:28:15.760106+0000","last_peered":"2023-11-08T19:28:15.760106+0000","last_clean":"2023-11-08T19:28:15.760106+0000","last_became_active":"2023-11-08T19:28:15.755532+0000","last_became_peered":"2023-11-08T19:28:15.755532+0000","last_unstale":"2023-11-08T19:28:15.760106+0000","last_undegraded":"2023-11-08T19:28:15.760106+0000","last_fullsized":"2023-11-08T19:28:15.760106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:03.335683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.737993+0000","last_change":"2023-11-08T19:28:14.748127+0000","last_active":"2023-11-08T19:28:15.737993+0000","last_peered":"2023-11-08T19:28:15.737993+0000","last_clean":"2023-11-08T19:28:15.737993+0000","last_became_active":"2023-11-08T19:28:14.747832+0000","last_became_peered":"2023-11-08T19:28:14.747832+0000","last_unstale":"2023-11-08T19:28:15.737993+0000","last_undegraded":"2023-11-08T19:28:15.737993+0000","last_fullsized":"2023-11-08T19:28:15.737993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:18.066983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.739183+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:14.739183+0000","last_peered":"2023-11-08T19:28:14.739183+0000","last_clean":"2023-11-08T19:28:14.739183+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:14.739183+0000","last_undegraded":"2023-11-08T19:28:14.739183+0000","last_fullsized":"2023-11-08T19:28:14.739183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748834+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:15.748834+0000","last_peered":"2023-11-08T19:28:15.748834+0000","last_clean":"2023-11-08T19:28:15.748834+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:15.748834+0000","last_undegraded":"2023-11-08T19:28:15.748834+0000","last_fullsized":"2023-11-08T19:28:15.748834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.726615+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:14.726615+0000","last_peered":"2023-11-08T19:28:14.726615+0000","last_clean":"2023-11-08T19:28:14.726615+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:14.726615+0000","last_undegraded":"2023-11-08T19:28:14.726615+0000","last_fullsized":"2023-11-08T19:28:14.726615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.753706+0000","last_change":"2023-11-08T19:28:15.750541+0000","last_active":"2023-11-08T19:28:15.753706+0000","last_peered":"2023-11-08T19:28:15.753706+0000","last_clean":"2023-11-08T19:28:15.753706+0000","last_became_active":"2023-11-08T19:28:15.750259+0000","last_became_peered":"2023-11-08T19:28:15.750259+0000","last_unstale":"2023-11-08T19:28:15.753706+0000","last_undegraded":"2023-11-08T19:28:15.753706+0000","last_fullsized":"2023-11-08T19:28:15.753706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:03:24.026858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.739321+0000","last_change":"2023-11-08T19:28:14.744027+0000","last_active":"2023-11-08T19:28:15.739321+0000","last_peered":"2023-11-08T19:28:15.739321+0000","last_clean":"2023-11-08T19:28:15.739321+0000","last_became_active":"2023-11-08T19:28:14.743733+0000","last_became_peered":"2023-11-08T19:28:14.743733+0000","last_unstale":"2023-11-08T19:28:15.739321+0000","last_undegraded":"2023-11-08T19:28:15.739321+0000","last_fullsized":"2023-11-08T19:28:15.739321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:43.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.727346+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:14.727346+0000","last_peered":"2023-11-08T19:28:14.727346+0000","last_clean":"2023-11-08T19:28:14.727346+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:14.727346+0000","last_undegraded":"2023-11-08T19:28:14.727346+0000","last_fullsized":"2023-11-08T19:28:14.727346+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748099+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:15.748099+0000","last_peered":"2023-11-08T19:28:15.748099+0000","last_clean":"2023-11-08T19:28:15.748099+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:15.748099+0000","last_undegraded":"2023-11-08T19:28:15.748099+0000","last_fullsized":"2023-11-08T19:28:15.748099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.750693+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:15.750693+0000","last_peered":"2023-11-08T19:28:15.750693+0000","last_clean":"2023-11-08T19:28:15.750693+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:15.750693+0000","last_undegraded":"2023-11-08T19:28:15.750693+0000","last_fullsized":"2023-11-08T19:28:15.750693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767442+0000","last_change":"2023-11-08T19:28:15.763123+0000","last_active":"2023-11-08T19:28:15.767442+0000","last_peered":"2023-11-08T19:28:15.767442+0000","last_clean":"2023-11-08T19:28:15.767442+0000","last_became_active":"2023-11-08T19:28:15.762804+0000","last_became_peered":"2023-11-08T19:28:15.762804+0000","last_unstale":"2023-11-08T19:28:15.767442+0000","last_undegraded":"2023-11-08T19:28:15.767442+0000","last_fullsized":"2023-11-08T19:28:15.767442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:03:00.483579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757584+0000","last_change":"2023-11-08T19:28:14.749481+0000","last_active":"2023-11-08T19:28:15.757584+0000","last_peered":"2023-11-08T19:28:15.757584+0000","last_clean":"2023-11-08T19:28:15.757584+0000","last_became_active":"2023-11-08T19:28:14.749228+0000","last_became_peered":"2023-11-08T19:28:14.749228+0000","last_unstale":"2023-11-08T19:28:15.757584+0000","last_undegraded":"2023-11-08T19:28:15.757584+0000","last_fullsized":"2023-11-08T19:28:15.757584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:16.466751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.740089+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:14.740089+0000","last_peered":"2023-11-08T19:28:14.740089+0000","last_clean":"2023-11-08T19:28:14.740089+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:14.740089+0000","last_undegraded":"2023-11-08T19:28:14.740089+0000","last_fullsized":"2023-11-08T19:28:14.740089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.741938+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:14.741938+0000","last_peered":"2023-11-08T19:28:14.741938+0000","last_clean":"2023-11-08T19:28:14.741938+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:14.741938+0000","last_undegraded":"2023-11-08T19:28:14.741938+0000","last_fullsized":"2023-11-08T19:28:14.741938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.749396+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:15.749396+0000","last_peered":"2023-11-08T19:28:15.749396+0000","last_clean":"2023-11-08T19:28:15.749396+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:15.749396+0000","last_undegraded":"2023-11-08T19:28:15.749396+0000","last_fullsized":"2023-11-08T19:28:15.749396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.764472+0000","last_change":"2023-11-08T19:28:15.757390+0000","last_active":"2023-11-08T19:28:15.764472+0000","last_peered":"2023-11-08T19:28:15.764472+0000","last_clean":"2023-11-08T19:28:15.764472+0000","last_became_active":"2023-11-08T19:28:15.757144+0000","last_became_peered":"2023-11-08T19:28:15.757144+0000","last_unstale":"2023-11-08T19:28:15.764472+0000","last_undegraded":"2023-11-08T19:28:15.764472+0000","last_fullsized":"2023-11-08T19:28:15.764472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:16:13.791856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773371+0000","last_change":"2023-11-08T19:28:14.759109+0000","last_active":"2023-11-08T19:28:15.773371+0000","last_peered":"2023-11-08T19:28:15.773371+0000","last_clean":"2023-11-08T19:28:15.773371+0000","last_became_active":"2023-11-08T19:28:14.758784+0000","last_became_peered":"2023-11-08T19:28:14.758784+0000","last_unstale":"2023-11-08T19:28:15.773371+0000","last_undegraded":"2023-11-08T19:28:15.773371+0000","last_fullsized":"2023-11-08T19:28:15.773371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:21.923551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722132+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:14.722132+0000","last_peered":"2023-11-08T19:28:14.722132+0000","last_clean":"2023-11-08T19:28:14.722132+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:14.722132+0000","last_undegraded":"2023-11-08T19:28:14.722132+0000","last_fullsized":"2023-11-08T19:28:14.722132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.749316+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:14.749316+0000","last_peered":"2023-11-08T19:28:14.749316+0000","last_clean":"2023-11-08T19:28:14.749316+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:14.749316+0000","last_undegraded":"2023-11-08T19:28:14.749316+0000","last_fullsized":"2023-11-08T19:28:14.749316+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743838+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:14.743838+0000","last_peered":"2023-11-08T19:28:14.743838+0000","last_clean":"2023-11-08T19:28:14.743838+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:14.743838+0000","last_undegraded":"2023-11-08T19:28:14.743838+0000","last_fullsized":"2023-11-08T19:28:14.743838+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744369+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:14.744369+0000","last_peered":"2023-11-08T19:28:14.744369+0000","last_clean":"2023-11-08T19:28:14.744369+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:14.744369+0000","last_undegraded":"2023-11-08T19:28:14.744369+0000","last_fullsized":"2023-11-08T19:28:14.744369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1078185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.52,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.52},{"interface":"front","average":{"1min":1.141,"5min":1.25,"15min":1.3120000000000001},"min":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1360000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.96799999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.196,"15min":1.2569999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.518,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.125,"15min":1.1359999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.242}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.405,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.129},{"interface":"front","average":{"1min":1.056,"5min":1.28,"15min":1.3620000000000001},"min":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.389,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.452}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3129999999999999,"15min":1.367},"min":{"1min":0.79000000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.591,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.506,"15min":1.5700000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2200000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.399}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.798,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.635},{"interface":"front","average":{"1min":1.321,"5min":1.427,"15min":1.4970000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.774,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.726}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.73299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7829999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.571}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.546,"15min":1.579},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.9770000000000001,"5min":2.427,"15min":2.427},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.5680000000000001,"15min":1.631},"min":{"1min":0.997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.8939999999999999,"5min":2.468,"15min":2.468},"last":2.117}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":488357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96999999999999997,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6279999999999999,"5min":1.752,"15min":1.752},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.083,"5min":1.1140000000000001,"15min":1.1060000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.581,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.095,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.47,"5min":1.679,"15min":1.679},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.2789999999999999,"15min":1.337},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6559999999999999,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":1.4570000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.238,"15min":1.292},"min":{"1min":0.68799999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4359999999999999,"5min":4.609,"15min":4.609},"last":1.375},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.861,"15min":1.861},"last":1.3340000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4350000000000001,"15min":1.4870000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.756,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.278},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1819999999999999,"15min":1.181},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.482,"5min":1.784,"15min":1.784},"last":1.4410000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.371,"15min":1.377},"min":{"1min":0.79600000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.732,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.125},{"interface":"front","average":{"1min":1.585,"5min":1.518,"15min":1.5029999999999999},"min":{"1min":0.89500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9510000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.556}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7829999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.347,"15min":1.335},"min":{"1min":0.83099999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8660000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.4179999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5669999999999999,"15min":1.55},"min":{"1min":0.85899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8759999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.454,"15min":1.454},"min":{"1min":0.81499999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8779999999999999,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.2330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":487775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.093,"15min":1.0740000000000001},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6990000000000001,"5min":1.76,"15min":1.76},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.234},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.107}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.141,"15min":1.1699999999999999},"min":{"1min":0.753,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5349999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.877},{"interface":"front","average":{"1min":1.181,"5min":1.343,"15min":1.4410000000000001},"min":{"1min":0.76200000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.617,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1830000000000001,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.639,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3939999999999999,"15min":1.4990000000000001},"min":{"1min":0.94299999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.806,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":0.81399999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.288},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.804,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3109999999999999,"15min":1.3049999999999999},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2150000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.85399999999999998,"5min":0.755,"15min":0.755},"max":{"1min":1.635,"5min":2.347,"15min":2.347},"last":1.046},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5149999999999999,"15min":1.5529999999999999},"min":{"1min":1.0589999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.097,"5min":3.153,"15min":3.153},"last":1.2589999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4930000000000001,"15min":1.528},"min":{"1min":0.89100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.613},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.5009999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.899,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.341}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.6399999999999999,"15min":1.7},"min":{"1min":1.03,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0449999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.581,"15min":1.5900000000000001},"min":{"1min":1.1439999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.831,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.5069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1024000,"data_stored":489441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0549999999999999,"15min":1.0469999999999999},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.496,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.635},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2549999999999999,"15min":1.2090000000000001},"min":{"1min":0.85699999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.696,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.4730000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.069,"15min":1.091},"min":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1279999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.425},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.89100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.7450000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.3879999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4610000000000001,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.784},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.423},"min":{"1min":0.82099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.524}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.696,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":1.04},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3600000000000001,"15min":1.387},"min":{"1min":0.83599999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.629,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.6839999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4399999999999999,"15min":1.5129999999999999},"min":{"1min":0.96699999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.667,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.4970000000000001,"15min":1.5489999999999999},"min":{"1min":0.95899999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8560000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.161}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5229999999999999,"15min":1.5620000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.782,"5min":4.335,"15min":4.335},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.5880000000000001,"15min":1.6259999999999999},"min":{"1min":1.3140000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8080000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.7250000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.6040000000000001,"15min":1.631},"min":{"1min":1.1799999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.77,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.331},{"interface":"front","average":{"1min":1.603,"5min":1.6859999999999999,"15min":1.732},"min":{"1min":1.127,"5min":1.095,"15min":1.095},"max":{"1min":1.901,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.845}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":489339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.98599999999999999,"15min":1.0720000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":3.972,"15min":3.972},"last":1.028},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.032,"15min":1.095},"min":{"1min":0.372,"5min":0.311,"15min":0.311},"max":{"1min":1.899,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.399}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.103,"15min":1.1850000000000001},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2609999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1080000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.8520000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.91900000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2709999999999999,"15min":1.325},"min":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6739999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2669999999999999},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.754,"5min":2.73,"15min":2.73},"last":0.755}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3740000000000001,"15min":1.448},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.663,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.393,"15min":1.452},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":3.052,"15min":3.052},"last":1.2549999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4339999999999999,"15min":1.498},"min":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0430000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.335},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.4670000000000001,"15min":1.5309999999999999},"min":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.948,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.878,"15min":0.878},"max":{"1min":3.21,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.5269999999999999,"15min":1.585},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0880000000000001,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.456}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6140000000000001,"15min":1.71},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.948,"5min":4.407,"15min":4.407},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.528,"15min":1.581},"min":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0059999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.3180000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1003520,"data_stored":486720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.86299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.879,"15min":0.92700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.171,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.198,"5min":1.393,"15min":1.393},"last":1.109},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":1.016},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4690000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.022,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0600000000000001,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6060000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.22}]},{"osd":4,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.407},"min":{"1min":0.81499999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2969999999999999,"15min":1.365},"min":{"1min":0.81999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7,"5min":2.278,"15min":2.278},"last":1.369}]},{"osd":5,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.425},"min":{"1min":0.76600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1600000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.601},{"interface":"front","average":{"1min":1.264,"5min":1.373,"15min":1.4610000000000001},"min":{"1min":0.78500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.954,"5min":2.536,"15min":2.536},"last":1.331}]},{"osd":6,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.347,"15min":1.389},"min":{"1min":0.93100000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9119999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.036,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.4139999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4930000000000001,"15min":1.577},"min":{"1min":0.85799999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.861,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.46},{"interface":"front","average":{"1min":1.403,"5min":1.492,"15min":1.556},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.0030000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.635}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1003520,"data_stored":487705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.745},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.304,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.745},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.123,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79900000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.863}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.65000000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.0700000000000001,"15min":1.0860000000000001},"min":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2690000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.647}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.1539999999999999},"min":{"1min":0.78900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.556,"5min":2.165,"15min":2.165},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.252},"min":{"1min":0.77400000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.454,"5min":1.885,"15min":1.885},"last":1.25}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2549999999999999},"min":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.528},{"interface":"front","average":{"1min":1.179,"5min":1.274,"15min":1.321},"min":{"1min":0.78900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.579,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.0669999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3580000000000001,"15min":1.3660000000000001},"min":{"1min":0.84999999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1509999999999998,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.92400000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.724,"5min":2.282,"15min":2.282},"last":1.4239999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3200000000000001,"15min":1.3300000000000001},"min":{"1min":1.109,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.909,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.746},{"interface":"front","average":{"1min":1.371,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4830000000000001,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4490000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1597440,"data_stored":1077636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.751},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.88900000000000001,"15min":0.93300000000000005},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1319999999999999,"5min":1.7,"15min":1.7},"last":1.0600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83799999999999997,"15min":0.86899999999999999},"min":{"1min":0.502,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.508,"15min":1.508},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.97399999999999998},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.48,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.98299999999999998,"15min":1.006},"min":{"1min":0.60599999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.335,"5min":1.748,"15min":1.748},"last":1.133},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0720000000000001,"15min":1.1100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":2.496,"15min":2.496},"last":0.90900000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2350000000000001},"min":{"1min":0.88100000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.794,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.3560000000000001,"15min":1.4370000000000001},"min":{"1min":0.875,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5580000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.284}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3160000000000001,"15min":1.3700000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8620000000000001,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.198},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3859999999999999,"15min":1.4430000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.77,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.337}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.431,"15min":1.4990000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.871,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.369,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.878,"15min":0.878},"max":{"1min":1.7450000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.379,"15min":1.4019999999999999},"min":{"1min":1.028,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9299999999999999,"5min":2.173,"15min":2.173},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4279999999999999,"15min":1.4850000000000001},"min":{"1min":0.96199999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.774,"5min":2.306,"15min":2.306},"last":1.381}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:20.236 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:20.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.466+0000 7f80d0333640 1 Processor -- start 2023-11-08T19:28:20.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.466+0000 7f80d0333640 1 -- start start 2023-11-08T19:28:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.467+0000 7f80d0333640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80c8142f80 0x7f80c80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80d0333640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80d0333640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c80c4960 con 0x7f80c815d9f0 2023-11-08T19:28:20.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80d0333640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c8143a40 con 0x7f80c8142f80 2023-11-08T19:28:20.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80d0333640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80ca3e0 con 0x7f80c806cb40 2023-11-08T19:28:20.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80c8142f80 0x7f80c80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80c8142f80 0x7f80c80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35588/0 (socket says 172.21.15.119:35588) 2023-11-08T19:28:20.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 -- 172.21.15.119:0/2334919433 learned_addr learned my addr 172.21.15.119:0/2334919433 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:20.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce8d0640 1 --2- 172.21.15.119:0/2334919433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 -- 172.21.15.119:0/2334919433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 msgr2=0x7f80c806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 --2- 172.21.15.119:0/2334919433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c806cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 -- 172.21.15.119:0/2334919433 >> v1:172.21.15.188:6789/0 conn(0x7f80c815d9f0 legacy=0x7f80c80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:20.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.468+0000 7f80ce0cf640 1 -- 172.21.15.119:0/2334919433 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c815df60 con 0x7f80c8142f80 2023-11-08T19:28:20.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.469+0000 7f80ce0cf640 1 --2- 172.21.15.119:0/2334919433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c80cc6d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009870 tx=0x7f80c402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e73067a29b59b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80cd0cd640 1 -- 172.21.15.119:0/2334919433 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4031640 con 0x7f80c8142f80 2023-11-08T19:28:20.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80cd0cd640 1 -- 172.21.15.119:0/2334919433 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c40317a0 con 0x7f80c8142f80 2023-11-08T19:28:20.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80cd0cd640 1 -- 172.21.15.119:0/2334919433 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c4031b30 con 0x7f80c8142f80 2023-11-08T19:28:20.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80d0333640 1 -- 172.21.15.119:0/2334919433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 msgr2=0x7f80c80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80d0333640 1 --2- 172.21.15.119:0/2334919433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c80cc6d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f80c4009870 tx=0x7f80c402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80d0333640 1 -- 172.21.15.119:0/2334919433 shutdown_connections 2023-11-08T19:28:20.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80d0333640 1 --2- 172.21.15.119:0/2334919433 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.470+0000 7f80d0333640 1 --2- 172.21.15.119:0/2334919433 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c80cc6d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.471+0000 7f80d0333640 1 -- 172.21.15.119:0/2334919433 >> 172.21.15.119:0/2334919433 conn(0x7f80c80cb610 msgr2=0x7f80c80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.471+0000 7f80d0333640 1 -- 172.21.15.119:0/2334919433 shutdown_connections 2023-11-08T19:28:20.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.471+0000 7f80d0333640 1 -- 172.21.15.119:0/2334919433 wait complete. 2023-11-08T19:28:20.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.472+0000 7f80d0333640 1 Processor -- start 2023-11-08T19:28:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.472+0000 7f80d0333640 1 -- start start 2023-11-08T19:28:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.473+0000 7f80d0333640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c8159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.473+0000 7f80d0333640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c813f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.473+0000 7f80d0333640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80c815d9f0 0x7f80c813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.473+0000 7f80d0333640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80c81d0 con 0x7f80c8142f80 2023-11-08T19:28:20.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.473+0000 7f80d0333640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80c84b0 con 0x7f80c815d9f0 2023-11-08T19:28:20.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.473+0000 7f80d0333640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c80c8790 con 0x7f80c806cb40 2023-11-08T19:28:20.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c813f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c813f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35594/0 (socket says 172.21.15.119:35594) 2023-11-08T19:28:20.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 -- 172.21.15.119:0/4206074205 learned_addr learned my addr 172.21.15.119:0/4206074205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:20.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80ce8d0640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80c815d9f0 0x7f80c813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 -- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 msgr2=0x7f80c8159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c8159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 -- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80c815d9f0 msgr2=0x7f80c813da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80c815d9f0 0x7f80c813da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 -- 172.21.15.119:0/4206074205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c40094d0 con 0x7f80c8142f80 2023-11-08T19:28:20.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.474+0000 7f80cd8ce640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c813f9d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b3f0 tx=0x7f80b800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.475+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011830 con 0x7f80c8142f80 2023-11-08T19:28:20.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.475+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8011990 con 0x7f80c8142f80 2023-11-08T19:28:20.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.475+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011d40 con 0x7f80c8142f80 2023-11-08T19:28:20.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.475+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80c80c8a70 con 0x7f80c8142f80 2023-11-08T19:28:20.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.475+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c813ff10 con 0x7f80c8142f80 2023-11-08T19:28:20.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.476+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c806f930 con 0x7f80c8142f80 2023-11-08T19:28:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.477+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80b801a400 con 0x7f80c8142f80 2023-11-08T19:28:20.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.477+0000 7f80beffd640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80ac06ae70 0x7f80ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.478+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f80b80899d0 con 0x7f80c8142f80 2023-11-08T19:28:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.478+0000 7f80ce0cf640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80ac06ae70 0x7f80ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.478+0000 7f80ce0cf640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80ac06ae70 0x7f80ac06d330 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80c4036000 tx=0x7f80c4030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:20.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.482+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80b8054510 con 0x7f80c8142f80 2023-11-08T19:28:20.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.668+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f80c80c8620 con 0x7f80ac06ae70 2023-11-08T19:28:20.685 INFO:tasks.ceph.mds.c.smithi119.stderr:2023-11-08T19:28:20.683+0000 7f803bb8e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:20.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.707+0000 7f80beffd640 1 -- 172.21.15.119:0/4206074205 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529102 (secure 0 0 0) 0x7f80c80c8620 con 0x7f80ac06ae70 2023-11-08T19:28:20.715 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:20.719 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:28:20.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.713+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80ac06ae70 msgr2=0x7f80ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.713+0000 7f80d0333640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80ac06ae70 0x7f80ac06d330 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f80c4036000 tx=0x7f80c4030000 comp rx=0 tx=0).stop 2023-11-08T19:28:20.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 msgr2=0x7f80c813f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c813f9d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f80b800b3f0 tx=0x7f80b800b8b0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 shutdown_connections 2023-11-08T19:28:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80ac06ae70 0x7f80ac06d330 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80c815d9f0 0x7f80c813da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80c8142f80 0x7f80c813f9d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 --2- 172.21.15.119:0/4206074205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80c806cb40 0x7f80c8159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 >> 172.21.15.119:0/4206074205 conn(0x7f80c80cb610 msgr2=0x7f80c80cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 shutdown_connections 2023-11-08T19:28:20.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:20.714+0000 7f80d0333640 1 -- 172.21.15.119:0/4206074205 wait complete. 2023-11-08T19:28:20.735 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2023-11-08T19:28:18.726768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":281,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":254360,"kb_used_data":9056,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754720360,"statfs":{"total":773094113280,"available":772833648640,"internally_reserved":0,"allocated":9273344,"data_stored":5085158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":47,"apply_latency_ms":47,"commit_latency_ns":47000000,"apply_latency_ns":47000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023741"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759604+0000","last_change":"2023-11-08T19:28:15.752428+0000","last_active":"2023-11-08T19:28:15.759604+0000","last_peered":"2023-11-08T19:28:15.759604+0000","last_clean":"2023-11-08T19:28:15.759604+0000","last_became_active":"2023-11-08T19:28:15.752015+0000","last_became_peered":"2023-11-08T19:28:15.752015+0000","last_unstale":"2023-11-08T19:28:15.759604+0000","last_undegraded":"2023-11-08T19:28:15.759604+0000","last_fullsized":"2023-11-08T19:28:15.759604+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:35.620894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759367+0000","last_change":"2023-11-08T19:28:15.756465+0000","last_active":"2023-11-08T19:28:15.759367+0000","last_peered":"2023-11-08T19:28:15.759367+0000","last_clean":"2023-11-08T19:28:15.759367+0000","last_became_active":"2023-11-08T19:28:15.756200+0000","last_became_peered":"2023-11-08T19:28:15.756200+0000","last_unstale":"2023-11-08T19:28:15.759367+0000","last_undegraded":"2023-11-08T19:28:15.759367+0000","last_fullsized":"2023-11-08T19:28:15.759367+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:17:33.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.760396+0000","last_change":"2023-11-08T19:28:15.756571+0000","last_active":"2023-11-08T19:28:15.760396+0000","last_peered":"2023-11-08T19:28:15.760396+0000","last_clean":"2023-11-08T19:28:15.760396+0000","last_became_active":"2023-11-08T19:28:15.756317+0000","last_became_peered":"2023-11-08T19:28:15.756317+0000","last_unstale":"2023-11-08T19:28:15.760396+0000","last_undegraded":"2023-11-08T19:28:15.760396+0000","last_fullsized":"2023-11-08T19:28:15.760396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:34.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.757765+0000","last_change":"2023-11-08T19:28:15.755798+0000","last_active":"2023-11-08T19:28:16.757765+0000","last_peered":"2023-11-08T19:28:16.757765+0000","last_clean":"2023-11-08T19:28:16.757765+0000","last_became_active":"2023-11-08T19:28:15.755482+0000","last_became_peered":"2023-11-08T19:28:15.755482+0000","last_unstale":"2023-11-08T19:28:16.757765+0000","last_undegraded":"2023-11-08T19:28:16.757765+0000","last_fullsized":"2023-11-08T19:28:16.757765+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:17:33.726264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.769626+0000","last_change":"2023-11-08T19:28:15.762747+0000","last_active":"2023-11-08T19:28:15.769626+0000","last_peered":"2023-11-08T19:28:15.769626+0000","last_clean":"2023-11-08T19:28:15.769626+0000","last_became_active":"2023-11-08T19:28:15.762454+0000","last_became_peered":"2023-11-08T19:28:15.762454+0000","last_unstale":"2023-11-08T19:28:15.769626+0000","last_undegraded":"2023-11-08T19:28:15.769626+0000","last_fullsized":"2023-11-08T19:28:15.769626+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:57.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773314+0000","last_change":"2023-11-08T19:28:15.769873+0000","last_active":"2023-11-08T19:28:15.773314+0000","last_peered":"2023-11-08T19:28:15.773314+0000","last_clean":"2023-11-08T19:28:15.773314+0000","last_became_active":"2023-11-08T19:28:15.769551+0000","last_became_peered":"2023-11-08T19:28:15.769551+0000","last_unstale":"2023-11-08T19:28:15.773314+0000","last_undegraded":"2023-11-08T19:28:15.773314+0000","last_fullsized":"2023-11-08T19:28:15.773314+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:14.523591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770951+0000","last_change":"2023-11-08T19:28:15.768035+0000","last_active":"2023-11-08T19:28:15.770951+0000","last_peered":"2023-11-08T19:28:15.770951+0000","last_clean":"2023-11-08T19:28:15.770951+0000","last_became_active":"2023-11-08T19:28:15.767769+0000","last_became_peered":"2023-11-08T19:28:15.767769+0000","last_unstale":"2023-11-08T19:28:15.770951+0000","last_undegraded":"2023-11-08T19:28:15.770951+0000","last_fullsized":"2023-11-08T19:28:15.770951+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:44.189210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765385+0000","last_change":"2023-11-08T19:28:15.762884+0000","last_active":"2023-11-08T19:28:15.765385+0000","last_peered":"2023-11-08T19:28:15.765385+0000","last_clean":"2023-11-08T19:28:15.765385+0000","last_became_active":"2023-11-08T19:28:15.762624+0000","last_became_peered":"2023-11-08T19:28:15.762624+0000","last_unstale":"2023-11-08T19:28:15.765385+0000","last_undegraded":"2023-11-08T19:28:15.765385+0000","last_fullsized":"2023-11-08T19:28:15.765385+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:06:48.522052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767584+0000","last_change":"2023-11-08T19:28:15.760605+0000","last_active":"2023-11-08T19:28:15.767584+0000","last_peered":"2023-11-08T19:28:15.767584+0000","last_clean":"2023-11-08T19:28:15.767584+0000","last_became_active":"2023-11-08T19:28:15.760301+0000","last_became_peered":"2023-11-08T19:28:15.760301+0000","last_unstale":"2023-11-08T19:28:15.767584+0000","last_undegraded":"2023-11-08T19:28:15.767584+0000","last_fullsized":"2023-11-08T19:28:15.767584+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:46.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765706+0000","last_change":"2023-11-08T19:28:15.755416+0000","last_active":"2023-11-08T19:28:15.765706+0000","last_peered":"2023-11-08T19:28:15.765706+0000","last_clean":"2023-11-08T19:28:15.765706+0000","last_became_active":"2023-11-08T19:28:15.755140+0000","last_became_peered":"2023-11-08T19:28:15.755140+0000","last_unstale":"2023-11-08T19:28:15.765706+0000","last_undegraded":"2023-11-08T19:28:15.765706+0000","last_fullsized":"2023-11-08T19:28:15.765706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:44.189210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.777009+0000","last_change":"2023-11-08T19:28:15.772381+0000","last_active":"2023-11-08T19:28:15.777009+0000","last_peered":"2023-11-08T19:28:15.777009+0000","last_clean":"2023-11-08T19:28:15.777009+0000","last_became_active":"2023-11-08T19:28:15.772079+0000","last_became_peered":"2023-11-08T19:28:15.772079+0000","last_unstale":"2023-11-08T19:28:15.777009+0000","last_undegraded":"2023-11-08T19:28:15.777009+0000","last_fullsized":"2023-11-08T19:28:15.777009+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:40.670042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773334+0000","last_change":"2023-11-08T19:28:15.767699+0000","last_active":"2023-11-08T19:28:15.773334+0000","last_peered":"2023-11-08T19:28:15.773334+0000","last_clean":"2023-11-08T19:28:15.773334+0000","last_became_active":"2023-11-08T19:28:15.766388+0000","last_became_peered":"2023-11-08T19:28:15.766388+0000","last_unstale":"2023-11-08T19:28:15.773334+0000","last_undegraded":"2023-11-08T19:28:15.773334+0000","last_fullsized":"2023-11-08T19:28:15.773334+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:44.189210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755167+0000","last_change":"2023-11-08T19:28:15.764595+0000","last_active":"2023-11-08T19:28:16.755167+0000","last_peered":"2023-11-08T19:28:16.755167+0000","last_clean":"2023-11-08T19:28:16.755167+0000","last_became_active":"2023-11-08T19:28:15.764329+0000","last_became_peered":"2023-11-08T19:28:15.764329+0000","last_unstale":"2023-11-08T19:28:16.755167+0000","last_undegraded":"2023-11-08T19:28:16.755167+0000","last_fullsized":"2023-11-08T19:28:16.755167+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:31.491334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.756789+0000","last_change":"2023-11-08T19:28:15.752823+0000","last_active":"2023-11-08T19:28:15.756789+0000","last_peered":"2023-11-08T19:28:15.756789+0000","last_clean":"2023-11-08T19:28:15.756789+0000","last_became_active":"2023-11-08T19:28:15.752520+0000","last_became_peered":"2023-11-08T19:28:15.752520+0000","last_unstale":"2023-11-08T19:28:15.756789+0000","last_undegraded":"2023-11-08T19:28:15.756789+0000","last_fullsized":"2023-11-08T19:28:15.756789+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:22:26.931128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756694+0000","last_change":"2023-11-08T19:28:15.760259+0000","last_active":"2023-11-08T19:28:16.756694+0000","last_peered":"2023-11-08T19:28:16.756694+0000","last_clean":"2023-11-08T19:28:16.756694+0000","last_became_active":"2023-11-08T19:28:15.759998+0000","last_became_peered":"2023-11-08T19:28:15.759998+0000","last_unstale":"2023-11-08T19:28:16.756694+0000","last_undegraded":"2023-11-08T19:28:16.756694+0000","last_fullsized":"2023-11-08T19:28:16.756694+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:00.119152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773323+0000","last_change":"2023-11-08T19:28:15.769582+0000","last_active":"2023-11-08T19:28:15.773323+0000","last_peered":"2023-11-08T19:28:15.773323+0000","last_clean":"2023-11-08T19:28:15.773323+0000","last_became_active":"2023-11-08T19:28:15.769271+0000","last_became_peered":"2023-11-08T19:28:15.769271+0000","last_unstale":"2023-11-08T19:28:15.773323+0000","last_undegraded":"2023-11-08T19:28:15.773323+0000","last_fullsized":"2023-11-08T19:28:15.773323+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:46.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768377+0000","last_change":"2023-11-08T19:28:15.764301+0000","last_active":"2023-11-08T19:28:15.768377+0000","last_peered":"2023-11-08T19:28:15.768377+0000","last_clean":"2023-11-08T19:28:15.768377+0000","last_became_active":"2023-11-08T19:28:15.763768+0000","last_became_peered":"2023-11-08T19:28:15.763768+0000","last_unstale":"2023-11-08T19:28:15.768377+0000","last_undegraded":"2023-11-08T19:28:15.768377+0000","last_fullsized":"2023-11-08T19:28:15.768377+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:10:58.354765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755905+0000","last_change":"2023-11-08T19:28:15.760215+0000","last_active":"2023-11-08T19:28:16.755905+0000","last_peered":"2023-11-08T19:28:16.755905+0000","last_clean":"2023-11-08T19:28:16.755905+0000","last_became_active":"2023-11-08T19:28:15.759631+0000","last_became_peered":"2023-11-08T19:28:15.759631+0000","last_unstale":"2023-11-08T19:28:16.755905+0000","last_undegraded":"2023-11-08T19:28:16.755905+0000","last_fullsized":"2023-11-08T19:28:16.755905+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:00.979814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770708+0000","last_change":"2023-11-08T19:28:15.768655+0000","last_active":"2023-11-08T19:28:15.770708+0000","last_peered":"2023-11-08T19:28:15.770708+0000","last_clean":"2023-11-08T19:28:15.770708+0000","last_became_active":"2023-11-08T19:28:15.764796+0000","last_became_peered":"2023-11-08T19:28:15.764796+0000","last_unstale":"2023-11-08T19:28:15.770708+0000","last_undegraded":"2023-11-08T19:28:15.770708+0000","last_fullsized":"2023-11-08T19:28:15.770708+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:31:17.396479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768141+0000","last_change":"2023-11-08T19:28:15.760217+0000","last_active":"2023-11-08T19:28:15.768141+0000","last_peered":"2023-11-08T19:28:15.768141+0000","last_clean":"2023-11-08T19:28:15.768141+0000","last_became_active":"2023-11-08T19:28:15.759934+0000","last_became_peered":"2023-11-08T19:28:15.759934+0000","last_unstale":"2023-11-08T19:28:15.768141+0000","last_undegraded":"2023-11-08T19:28:15.768141+0000","last_fullsized":"2023-11-08T19:28:15.768141+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:19.077175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.769151+0000","last_change":"2023-11-08T19:28:15.767669+0000","last_active":"2023-11-08T19:28:15.769151+0000","last_peered":"2023-11-08T19:28:15.769151+0000","last_clean":"2023-11-08T19:28:15.769151+0000","last_became_active":"2023-11-08T19:28:15.767405+0000","last_became_peered":"2023-11-08T19:28:15.767405+0000","last_unstale":"2023-11-08T19:28:15.769151+0000","last_undegraded":"2023-11-08T19:28:15.769151+0000","last_fullsized":"2023-11-08T19:28:15.769151+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:58:50.914147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757077+0000","last_change":"2023-11-08T19:28:15.750930+0000","last_active":"2023-11-08T19:28:15.757077+0000","last_peered":"2023-11-08T19:28:15.757077+0000","last_clean":"2023-11-08T19:28:15.757077+0000","last_became_active":"2023-11-08T19:28:15.750623+0000","last_became_peered":"2023-11-08T19:28:15.750623+0000","last_unstale":"2023-11-08T19:28:15.757077+0000","last_undegraded":"2023-11-08T19:28:15.757077+0000","last_fullsized":"2023-11-08T19:28:15.757077+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:02.182611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759299+0000","last_change":"2023-11-08T19:28:15.756466+0000","last_active":"2023-11-08T19:28:15.759299+0000","last_peered":"2023-11-08T19:28:15.759299+0000","last_clean":"2023-11-08T19:28:15.759299+0000","last_became_active":"2023-11-08T19:28:15.753727+0000","last_became_peered":"2023-11-08T19:28:15.753727+0000","last_unstale":"2023-11-08T19:28:15.759299+0000","last_undegraded":"2023-11-08T19:28:15.759299+0000","last_fullsized":"2023-11-08T19:28:15.759299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:16:48.799049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767065+0000","last_change":"2023-11-08T19:28:15.761676+0000","last_active":"2023-11-08T19:28:15.767065+0000","last_peered":"2023-11-08T19:28:15.767065+0000","last_clean":"2023-11-08T19:28:15.767065+0000","last_became_active":"2023-11-08T19:28:15.761378+0000","last_became_peered":"2023-11-08T19:28:15.761378+0000","last_unstale":"2023-11-08T19:28:15.767065+0000","last_undegraded":"2023-11-08T19:28:15.767065+0000","last_fullsized":"2023-11-08T19:28:15.767065+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:57.342920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765976+0000","last_change":"2023-11-08T19:28:15.757254+0000","last_active":"2023-11-08T19:28:15.765976+0000","last_peered":"2023-11-08T19:28:15.765976+0000","last_clean":"2023-11-08T19:28:15.765976+0000","last_became_active":"2023-11-08T19:28:15.754720+0000","last_became_peered":"2023-11-08T19:28:15.754720+0000","last_unstale":"2023-11-08T19:28:15.765976+0000","last_undegraded":"2023-11-08T19:28:15.765976+0000","last_fullsized":"2023-11-08T19:28:15.765976+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:54.686302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765017+0000","last_change":"2023-11-08T19:28:15.761643+0000","last_active":"2023-11-08T19:28:15.765017+0000","last_peered":"2023-11-08T19:28:15.765017+0000","last_clean":"2023-11-08T19:28:15.765017+0000","last_became_active":"2023-11-08T19:28:15.761367+0000","last_became_peered":"2023-11-08T19:28:15.761367+0000","last_unstale":"2023-11-08T19:28:15.765017+0000","last_undegraded":"2023-11-08T19:28:15.765017+0000","last_fullsized":"2023-11-08T19:28:15.765017+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:26.949146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773520+0000","last_change":"2023-11-08T19:28:15.770161+0000","last_active":"2023-11-08T19:28:15.773520+0000","last_peered":"2023-11-08T19:28:15.773520+0000","last_clean":"2023-11-08T19:28:15.773520+0000","last_became_active":"2023-11-08T19:28:15.769868+0000","last_became_peered":"2023-11-08T19:28:15.769868+0000","last_unstale":"2023-11-08T19:28:15.773520+0000","last_undegraded":"2023-11-08T19:28:15.773520+0000","last_fullsized":"2023-11-08T19:28:15.773520+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:48:34.435941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.769968+0000","last_change":"2023-11-08T19:28:15.765121+0000","last_active":"2023-11-08T19:28:15.769968+0000","last_peered":"2023-11-08T19:28:15.769968+0000","last_clean":"2023-11-08T19:28:15.769968+0000","last_became_active":"2023-11-08T19:28:15.764847+0000","last_became_peered":"2023-11-08T19:28:15.764847+0000","last_unstale":"2023-11-08T19:28:15.769968+0000","last_undegraded":"2023-11-08T19:28:15.769968+0000","last_fullsized":"2023-11-08T19:28:15.769968+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:46.288119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757518+0000","last_change":"2023-11-08T19:28:15.752467+0000","last_active":"2023-11-08T19:28:15.757518+0000","last_peered":"2023-11-08T19:28:15.757518+0000","last_clean":"2023-11-08T19:28:15.757518+0000","last_became_active":"2023-11-08T19:28:15.752213+0000","last_became_peered":"2023-11-08T19:28:15.752213+0000","last_unstale":"2023-11-08T19:28:15.757518+0000","last_undegraded":"2023-11-08T19:28:15.757518+0000","last_fullsized":"2023-11-08T19:28:15.757518+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:09.019260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767421+0000","last_change":"2023-11-08T19:28:15.762793+0000","last_active":"2023-11-08T19:28:15.767421+0000","last_peered":"2023-11-08T19:28:15.767421+0000","last_clean":"2023-11-08T19:28:15.767421+0000","last_became_active":"2023-11-08T19:28:15.760637+0000","last_became_peered":"2023-11-08T19:28:15.760637+0000","last_unstale":"2023-11-08T19:28:15.767421+0000","last_undegraded":"2023-11-08T19:28:15.767421+0000","last_fullsized":"2023-11-08T19:28:15.767421+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:33.778386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756997+0000","last_change":"2023-11-08T19:28:15.769059+0000","last_active":"2023-11-08T19:28:16.756997+0000","last_peered":"2023-11-08T19:28:16.756997+0000","last_clean":"2023-11-08T19:28:16.756997+0000","last_became_active":"2023-11-08T19:28:15.768440+0000","last_became_peered":"2023-11-08T19:28:15.768440+0000","last_unstale":"2023-11-08T19:28:16.756997+0000","last_undegraded":"2023-11-08T19:28:16.756997+0000","last_fullsized":"2023-11-08T19:28:16.756997+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:38:26.949146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.758114+0000","last_change":"2023-11-08T19:28:15.749493+0000","last_active":"2023-11-08T19:28:15.758114+0000","last_peered":"2023-11-08T19:28:15.758114+0000","last_clean":"2023-11-08T19:28:15.758114+0000","last_became_active":"2023-11-08T19:28:15.749210+0000","last_became_peered":"2023-11-08T19:28:15.749210+0000","last_unstale":"2023-11-08T19:28:15.758114+0000","last_undegraded":"2023-11-08T19:28:15.758114+0000","last_fullsized":"2023-11-08T19:28:15.758114+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:06.290558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773500+0000","last_change":"2023-11-08T19:28:15.769546+0000","last_active":"2023-11-08T19:28:15.773500+0000","last_peered":"2023-11-08T19:28:15.773500+0000","last_clean":"2023-11-08T19:28:15.773500+0000","last_became_active":"2023-11-08T19:28:15.769197+0000","last_became_peered":"2023-11-08T19:28:15.769197+0000","last_unstale":"2023-11-08T19:28:15.773500+0000","last_undegraded":"2023-11-08T19:28:15.773500+0000","last_fullsized":"2023-11-08T19:28:15.773500+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:13:07.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"82'6","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.777205+0000","last_change":"2023-11-08T19:28:14.746051+0000","last_active":"2023-11-08T19:28:15.777205+0000","last_peered":"2023-11-08T19:28:15.777205+0000","last_clean":"2023-11-08T19:28:15.777205+0000","last_became_active":"2023-11-08T19:28:14.745719+0000","last_became_peered":"2023-11-08T19:28:14.745719+0000","last_unstale":"2023-11-08T19:28:15.777205+0000","last_undegraded":"2023-11-08T19:28:15.777205+0000","last_fullsized":"2023-11-08T19:28:15.777205+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:21.923551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.728881+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:14.728881+0000","last_peered":"2023-11-08T19:28:14.728881+0000","last_clean":"2023-11-08T19:28:14.728881+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:14.728881+0000","last_undegraded":"2023-11-08T19:28:14.728881+0000","last_fullsized":"2023-11-08T19:28:14.728881+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.746541+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:14.746541+0000","last_peered":"2023-11-08T19:28:14.746541+0000","last_clean":"2023-11-08T19:28:14.746541+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:14.746541+0000","last_undegraded":"2023-11-08T19:28:14.746541+0000","last_fullsized":"2023-11-08T19:28:14.746541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722971+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:14.722971+0000","last_peered":"2023-11-08T19:28:14.722971+0000","last_clean":"2023-11-08T19:28:14.722971+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:14.722971+0000","last_undegraded":"2023-11-08T19:28:14.722971+0000","last_fullsized":"2023-11-08T19:28:14.722971+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757920+0000","last_change":"2023-11-08T19:28:15.750436+0000","last_active":"2023-11-08T19:28:15.757920+0000","last_peered":"2023-11-08T19:28:15.757920+0000","last_clean":"2023-11-08T19:28:15.757920+0000","last_became_active":"2023-11-08T19:28:15.750068+0000","last_became_peered":"2023-11-08T19:28:15.750068+0000","last_unstale":"2023-11-08T19:28:15.757920+0000","last_undegraded":"2023-11-08T19:28:15.757920+0000","last_fullsized":"2023-11-08T19:28:15.757920+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:13:07.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"82'3","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773880+0000","last_change":"2023-11-08T19:28:14.756005+0000","last_active":"2023-11-08T19:28:15.773880+0000","last_peered":"2023-11-08T19:28:15.773880+0000","last_clean":"2023-11-08T19:28:15.773880+0000","last_became_active":"2023-11-08T19:28:14.755503+0000","last_became_peered":"2023-11-08T19:28:14.755503+0000","last_unstale":"2023-11-08T19:28:15.773880+0000","last_undegraded":"2023-11-08T19:28:15.773880+0000","last_fullsized":"2023-11-08T19:28:15.773880+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:40.572679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.747671+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:15.747671+0000","last_peered":"2023-11-08T19:28:15.747671+0000","last_clean":"2023-11-08T19:28:15.747671+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:15.747671+0000","last_undegraded":"2023-11-08T19:28:15.747671+0000","last_fullsized":"2023-11-08T19:28:15.747671+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747038+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:14.747038+0000","last_peered":"2023-11-08T19:28:14.747038+0000","last_clean":"2023-11-08T19:28:14.747038+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:14.747038+0000","last_undegraded":"2023-11-08T19:28:14.747038+0000","last_fullsized":"2023-11-08T19:28:14.747038+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.729228+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:14.729228+0000","last_peered":"2023-11-08T19:28:14.729228+0000","last_clean":"2023-11-08T19:28:14.729228+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:14.729228+0000","last_undegraded":"2023-11-08T19:28:14.729228+0000","last_fullsized":"2023-11-08T19:28:14.729228+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767561+0000","last_change":"2023-11-08T19:28:15.758399+0000","last_active":"2023-11-08T19:28:15.767561+0000","last_peered":"2023-11-08T19:28:15.767561+0000","last_clean":"2023-11-08T19:28:15.767561+0000","last_became_active":"2023-11-08T19:28:15.757230+0000","last_became_peered":"2023-11-08T19:28:15.757230+0000","last_unstale":"2023-11-08T19:28:15.767561+0000","last_undegraded":"2023-11-08T19:28:15.767561+0000","last_fullsized":"2023-11-08T19:28:15.767561+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:40.670042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755986+0000","last_change":"2023-11-08T19:28:14.753569+0000","last_active":"2023-11-08T19:28:16.755986+0000","last_peered":"2023-11-08T19:28:16.755986+0000","last_clean":"2023-11-08T19:28:16.755986+0000","last_became_active":"2023-11-08T19:28:14.753273+0000","last_became_peered":"2023-11-08T19:28:14.753273+0000","last_unstale":"2023-11-08T19:28:16.755986+0000","last_undegraded":"2023-11-08T19:28:16.755986+0000","last_fullsized":"2023-11-08T19:28:16.755986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:25.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742009+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:14.742009+0000","last_peered":"2023-11-08T19:28:14.742009+0000","last_clean":"2023-11-08T19:28:14.742009+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:14.742009+0000","last_undegraded":"2023-11-08T19:28:14.742009+0000","last_fullsized":"2023-11-08T19:28:14.742009+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.728775+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:14.728775+0000","last_peered":"2023-11-08T19:28:14.728775+0000","last_clean":"2023-11-08T19:28:14.728775+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:14.728775+0000","last_undegraded":"2023-11-08T19:28:14.728775+0000","last_fullsized":"2023-11-08T19:28:14.728775+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747354+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:14.747354+0000","last_peered":"2023-11-08T19:28:14.747354+0000","last_clean":"2023-11-08T19:28:14.747354+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:14.747354+0000","last_undegraded":"2023-11-08T19:28:14.747354+0000","last_fullsized":"2023-11-08T19:28:14.747354+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765036+0000","last_change":"2023-11-08T19:28:15.762240+0000","last_active":"2023-11-08T19:28:15.765036+0000","last_peered":"2023-11-08T19:28:15.765036+0000","last_clean":"2023-11-08T19:28:15.765036+0000","last_became_active":"2023-11-08T19:28:15.761942+0000","last_became_peered":"2023-11-08T19:28:15.761942+0000","last_unstale":"2023-11-08T19:28:15.765036+0000","last_undegraded":"2023-11-08T19:28:15.765036+0000","last_fullsized":"2023-11-08T19:28:15.765036+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:00.979814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.737507+0000","last_change":"2023-11-08T19:28:14.747536+0000","last_active":"2023-11-08T19:28:15.737507+0000","last_peered":"2023-11-08T19:28:15.737507+0000","last_clean":"2023-11-08T19:28:15.737507+0000","last_became_active":"2023-11-08T19:28:14.747255+0000","last_became_peered":"2023-11-08T19:28:14.747255+0000","last_unstale":"2023-11-08T19:28:15.737507+0000","last_undegraded":"2023-11-08T19:28:15.737507+0000","last_fullsized":"2023-11-08T19:28:15.737507+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:11:05.256770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742497+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:14.742497+0000","last_peered":"2023-11-08T19:28:14.742497+0000","last_clean":"2023-11-08T19:28:14.742497+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:14.742497+0000","last_undegraded":"2023-11-08T19:28:14.742497+0000","last_fullsized":"2023-11-08T19:28:14.742497+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745585+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:14.745585+0000","last_peered":"2023-11-08T19:28:14.745585+0000","last_clean":"2023-11-08T19:28:14.745585+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:14.745585+0000","last_undegraded":"2023-11-08T19:28:14.745585+0000","last_fullsized":"2023-11-08T19:28:14.745585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722752+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:14.722752+0000","last_peered":"2023-11-08T19:28:14.722752+0000","last_clean":"2023-11-08T19:28:14.722752+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:14.722752+0000","last_undegraded":"2023-11-08T19:28:14.722752+0000","last_fullsized":"2023-11-08T19:28:14.722752+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759089+0000","last_change":"2023-11-08T19:28:15.755259+0000","last_active":"2023-11-08T19:28:15.759089+0000","last_peered":"2023-11-08T19:28:15.759089+0000","last_clean":"2023-11-08T19:28:15.759089+0000","last_became_active":"2023-11-08T19:28:15.754988+0000","last_became_peered":"2023-11-08T19:28:15.754988+0000","last_unstale":"2023-11-08T19:28:15.759089+0000","last_undegraded":"2023-11-08T19:28:15.759089+0000","last_fullsized":"2023-11-08T19:28:15.759089+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:14.726099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"82'8","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.765417+0000","last_change":"2023-11-08T19:28:14.751203+0000","last_active":"2023-11-08T19:28:15.765417+0000","last_peered":"2023-11-08T19:28:15.765417+0000","last_clean":"2023-11-08T19:28:15.765417+0000","last_became_active":"2023-11-08T19:28:14.750951+0000","last_became_peered":"2023-11-08T19:28:14.750951+0000","last_unstale":"2023-11-08T19:28:15.765417+0000","last_undegraded":"2023-11-08T19:28:15.765417+0000","last_fullsized":"2023-11-08T19:28:15.765417+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:38:07.976116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747409+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:14.747409+0000","last_peered":"2023-11-08T19:28:14.747409+0000","last_clean":"2023-11-08T19:28:14.747409+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:14.747409+0000","last_undegraded":"2023-11-08T19:28:14.747409+0000","last_fullsized":"2023-11-08T19:28:14.747409+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.749131+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:15.749131+0000","last_peered":"2023-11-08T19:28:15.749131+0000","last_clean":"2023-11-08T19:28:15.749131+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:15.749131+0000","last_undegraded":"2023-11-08T19:28:15.749131+0000","last_fullsized":"2023-11-08T19:28:15.749131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745669+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:14.745669+0000","last_peered":"2023-11-08T19:28:14.745669+0000","last_clean":"2023-11-08T19:28:14.745669+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:14.745669+0000","last_undegraded":"2023-11-08T19:28:14.745669+0000","last_fullsized":"2023-11-08T19:28:14.745669+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766978+0000","last_change":"2023-11-08T19:28:15.756538+0000","last_active":"2023-11-08T19:28:15.766978+0000","last_peered":"2023-11-08T19:28:15.766978+0000","last_clean":"2023-11-08T19:28:15.766978+0000","last_became_active":"2023-11-08T19:28:15.756252+0000","last_became_peered":"2023-11-08T19:28:15.756252+0000","last_unstale":"2023-11-08T19:28:15.766978+0000","last_undegraded":"2023-11-08T19:28:15.766978+0000","last_fullsized":"2023-11-08T19:28:15.766978+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:13:07.820305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.746644+0000","last_change":"2023-11-08T19:28:14.747095+0000","last_active":"2023-11-08T19:28:15.746644+0000","last_peered":"2023-11-08T19:28:15.746644+0000","last_clean":"2023-11-08T19:28:15.746644+0000","last_became_active":"2023-11-08T19:28:14.744802+0000","last_became_peered":"2023-11-08T19:28:14.744802+0000","last_unstale":"2023-11-08T19:28:15.746644+0000","last_undegraded":"2023-11-08T19:28:15.746644+0000","last_fullsized":"2023-11-08T19:28:15.746644+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:53.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742055+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:14.742055+0000","last_peered":"2023-11-08T19:28:14.742055+0000","last_clean":"2023-11-08T19:28:14.742055+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:14.742055+0000","last_undegraded":"2023-11-08T19:28:14.742055+0000","last_fullsized":"2023-11-08T19:28:14.742055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722692+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:14.722692+0000","last_peered":"2023-11-08T19:28:14.722692+0000","last_clean":"2023-11-08T19:28:14.722692+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:14.722692+0000","last_undegraded":"2023-11-08T19:28:14.722692+0000","last_fullsized":"2023-11-08T19:28:14.722692+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.741233+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:14.741233+0000","last_peered":"2023-11-08T19:28:14.741233+0000","last_clean":"2023-11-08T19:28:14.741233+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:14.741233+0000","last_undegraded":"2023-11-08T19:28:14.741233+0000","last_fullsized":"2023-11-08T19:28:14.741233+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759591+0000","last_change":"2023-11-08T19:28:15.754071+0000","last_active":"2023-11-08T19:28:15.759591+0000","last_peered":"2023-11-08T19:28:15.759591+0000","last_clean":"2023-11-08T19:28:15.759591+0000","last_became_active":"2023-11-08T19:28:15.753616+0000","last_became_peered":"2023-11-08T19:28:15.753616+0000","last_unstale":"2023-11-08T19:28:15.759591+0000","last_undegraded":"2023-11-08T19:28:15.759591+0000","last_fullsized":"2023-11-08T19:28:15.759591+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:00.872144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.738550+0000","last_change":"2023-11-08T19:28:14.750055+0000","last_active":"2023-11-08T19:28:15.738550+0000","last_peered":"2023-11-08T19:28:15.738550+0000","last_clean":"2023-11-08T19:28:15.738550+0000","last_became_active":"2023-11-08T19:28:14.747228+0000","last_became_peered":"2023-11-08T19:28:14.747228+0000","last_unstale":"2023-11-08T19:28:15.738550+0000","last_undegraded":"2023-11-08T19:28:15.738550+0000","last_fullsized":"2023-11-08T19:28:15.738550+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:09.925303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.720932+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:14.720932+0000","last_peered":"2023-11-08T19:28:14.720932+0000","last_clean":"2023-11-08T19:28:14.720932+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:14.720932+0000","last_undegraded":"2023-11-08T19:28:14.720932+0000","last_fullsized":"2023-11-08T19:28:14.720932+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.741154+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:14.741154+0000","last_peered":"2023-11-08T19:28:14.741154+0000","last_clean":"2023-11-08T19:28:14.741154+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:14.741154+0000","last_undegraded":"2023-11-08T19:28:14.741154+0000","last_fullsized":"2023-11-08T19:28:14.741154+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742299+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:14.742299+0000","last_peered":"2023-11-08T19:28:14.742299+0000","last_clean":"2023-11-08T19:28:14.742299+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:14.742299+0000","last_undegraded":"2023-11-08T19:28:14.742299+0000","last_fullsized":"2023-11-08T19:28:14.742299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756259+0000","last_change":"2023-11-08T19:28:15.765799+0000","last_active":"2023-11-08T19:28:16.756259+0000","last_peered":"2023-11-08T19:28:16.756259+0000","last_clean":"2023-11-08T19:28:16.756259+0000","last_became_active":"2023-11-08T19:28:15.765542+0000","last_became_peered":"2023-11-08T19:28:15.765542+0000","last_unstale":"2023-11-08T19:28:16.756259+0000","last_undegraded":"2023-11-08T19:28:16.756259+0000","last_fullsized":"2023-11-08T19:28:16.756259+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:40.606689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766203+0000","last_change":"2023-11-08T19:28:14.752387+0000","last_active":"2023-11-08T19:28:15.766203+0000","last_peered":"2023-11-08T19:28:15.766203+0000","last_clean":"2023-11-08T19:28:15.766203+0000","last_became_active":"2023-11-08T19:28:14.752105+0000","last_became_peered":"2023-11-08T19:28:14.752105+0000","last_unstale":"2023-11-08T19:28:15.766203+0000","last_undegraded":"2023-11-08T19:28:15.766203+0000","last_fullsized":"2023-11-08T19:28:15.766203+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:06.283767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.721910+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:14.721910+0000","last_peered":"2023-11-08T19:28:14.721910+0000","last_clean":"2023-11-08T19:28:14.721910+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:14.721910+0000","last_undegraded":"2023-11-08T19:28:14.721910+0000","last_fullsized":"2023-11-08T19:28:14.721910+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.727728+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:14.727728+0000","last_peered":"2023-11-08T19:28:14.727728+0000","last_clean":"2023-11-08T19:28:14.727728+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:14.727728+0000","last_undegraded":"2023-11-08T19:28:14.727728+0000","last_fullsized":"2023-11-08T19:28:14.727728+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733666+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:14.733666+0000","last_peered":"2023-11-08T19:28:14.733666+0000","last_clean":"2023-11-08T19:28:14.733666+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:14.733666+0000","last_undegraded":"2023-11-08T19:28:14.733666+0000","last_fullsized":"2023-11-08T19:28:14.733666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.772873+0000","last_change":"2023-11-08T19:28:15.767990+0000","last_active":"2023-11-08T19:28:15.772873+0000","last_peered":"2023-11-08T19:28:15.772873+0000","last_clean":"2023-11-08T19:28:15.772873+0000","last_became_active":"2023-11-08T19:28:15.767733+0000","last_became_peered":"2023-11-08T19:28:15.767733+0000","last_unstale":"2023-11-08T19:28:15.772873+0000","last_undegraded":"2023-11-08T19:28:15.772873+0000","last_fullsized":"2023-11-08T19:28:15.772873+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:57.342920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768120+0000","last_change":"2023-11-08T19:28:14.748780+0000","last_active":"2023-11-08T19:28:15.768120+0000","last_peered":"2023-11-08T19:28:15.768120+0000","last_clean":"2023-11-08T19:28:15.768120+0000","last_became_active":"2023-11-08T19:28:14.748358+0000","last_became_peered":"2023-11-08T19:28:14.748358+0000","last_unstale":"2023-11-08T19:28:15.768120+0000","last_undegraded":"2023-11-08T19:28:15.768120+0000","last_fullsized":"2023-11-08T19:28:15.768120+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:25.568976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747276+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:14.747276+0000","last_peered":"2023-11-08T19:28:14.747276+0000","last_clean":"2023-11-08T19:28:14.747276+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:14.747276+0000","last_undegraded":"2023-11-08T19:28:14.747276+0000","last_fullsized":"2023-11-08T19:28:14.747276+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745131+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:14.745131+0000","last_peered":"2023-11-08T19:28:14.745131+0000","last_clean":"2023-11-08T19:28:14.745131+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:14.745131+0000","last_undegraded":"2023-11-08T19:28:14.745131+0000","last_fullsized":"2023-11-08T19:28:14.745131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.749397+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:14.749397+0000","last_peered":"2023-11-08T19:28:14.749397+0000","last_clean":"2023-11-08T19:28:14.749397+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:14.749397+0000","last_undegraded":"2023-11-08T19:28:14.749397+0000","last_fullsized":"2023-11-08T19:28:14.749397+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.763731+0000","last_change":"2023-11-08T19:28:15.762004+0000","last_active":"2023-11-08T19:28:15.763731+0000","last_peered":"2023-11-08T19:28:15.763731+0000","last_clean":"2023-11-08T19:28:15.763731+0000","last_became_active":"2023-11-08T19:28:15.761560+0000","last_became_peered":"2023-11-08T19:28:15.761560+0000","last_unstale":"2023-11-08T19:28:15.763731+0000","last_undegraded":"2023-11-08T19:28:15.763731+0000","last_fullsized":"2023-11-08T19:28:15.763731+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:36.912107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.758475+0000","last_change":"2023-11-08T19:28:14.749771+0000","last_active":"2023-11-08T19:28:15.758475+0000","last_peered":"2023-11-08T19:28:15.758475+0000","last_clean":"2023-11-08T19:28:15.758475+0000","last_became_active":"2023-11-08T19:28:14.749496+0000","last_became_peered":"2023-11-08T19:28:14.749496+0000","last_unstale":"2023-11-08T19:28:15.758475+0000","last_undegraded":"2023-11-08T19:28:15.758475+0000","last_fullsized":"2023-11-08T19:28:15.758475+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:45:08.642083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744777+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:14.744777+0000","last_peered":"2023-11-08T19:28:14.744777+0000","last_clean":"2023-11-08T19:28:14.744777+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:14.744777+0000","last_undegraded":"2023-11-08T19:28:14.744777+0000","last_fullsized":"2023-11-08T19:28:14.744777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.746927+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:14.746927+0000","last_peered":"2023-11-08T19:28:14.746927+0000","last_clean":"2023-11-08T19:28:14.746927+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:14.746927+0000","last_undegraded":"2023-11-08T19:28:14.746927+0000","last_fullsized":"2023-11-08T19:28:14.746927+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.750095+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:15.750095+0000","last_peered":"2023-11-08T19:28:15.750095+0000","last_clean":"2023-11-08T19:28:15.750095+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:15.750095+0000","last_undegraded":"2023-11-08T19:28:15.750095+0000","last_fullsized":"2023-11-08T19:28:15.750095+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755396+0000","last_change":"2023-11-08T19:28:15.771829+0000","last_active":"2023-11-08T19:28:16.755396+0000","last_peered":"2023-11-08T19:28:16.755396+0000","last_clean":"2023-11-08T19:28:16.755396+0000","last_became_active":"2023-11-08T19:28:15.770178+0000","last_became_peered":"2023-11-08T19:28:15.770178+0000","last_unstale":"2023-11-08T19:28:16.755396+0000","last_undegraded":"2023-11-08T19:28:16.755396+0000","last_fullsized":"2023-11-08T19:28:16.755396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:09.019260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773989+0000","last_change":"2023-11-08T19:28:14.759338+0000","last_active":"2023-11-08T19:28:15.773989+0000","last_peered":"2023-11-08T19:28:15.773989+0000","last_clean":"2023-11-08T19:28:15.773989+0000","last_became_active":"2023-11-08T19:28:14.758982+0000","last_became_peered":"2023-11-08T19:28:14.758982+0000","last_unstale":"2023-11-08T19:28:15.773989+0000","last_undegraded":"2023-11-08T19:28:15.773989+0000","last_fullsized":"2023-11-08T19:28:15.773989+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:53.676110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.748790+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:14.748790+0000","last_peered":"2023-11-08T19:28:14.748790+0000","last_clean":"2023-11-08T19:28:14.748790+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:14.748790+0000","last_undegraded":"2023-11-08T19:28:14.748790+0000","last_fullsized":"2023-11-08T19:28:14.748790+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742643+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:14.742643+0000","last_peered":"2023-11-08T19:28:14.742643+0000","last_clean":"2023-11-08T19:28:14.742643+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:14.742643+0000","last_undegraded":"2023-11-08T19:28:14.742643+0000","last_fullsized":"2023-11-08T19:28:14.742643+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750961+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:14.750961+0000","last_peered":"2023-11-08T19:28:14.750961+0000","last_clean":"2023-11-08T19:28:14.750961+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:14.750961+0000","last_undegraded":"2023-11-08T19:28:14.750961+0000","last_fullsized":"2023-11-08T19:28:14.750961+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768644+0000","last_change":"2023-11-08T19:28:15.766459+0000","last_active":"2023-11-08T19:28:15.768644+0000","last_peered":"2023-11-08T19:28:15.768644+0000","last_clean":"2023-11-08T19:28:15.768644+0000","last_became_active":"2023-11-08T19:28:15.766140+0000","last_became_peered":"2023-11-08T19:28:15.766140+0000","last_unstale":"2023-11-08T19:28:15.768644+0000","last_undegraded":"2023-11-08T19:28:15.768644+0000","last_fullsized":"2023-11-08T19:28:15.768644+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:05:53.530158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"82'4","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.768452+0000","last_change":"2023-11-08T19:28:14.755044+0000","last_active":"2023-11-08T19:28:15.768452+0000","last_peered":"2023-11-08T19:28:15.768452+0000","last_clean":"2023-11-08T19:28:15.768452+0000","last_became_active":"2023-11-08T19:28:14.754750+0000","last_became_peered":"2023-11-08T19:28:14.754750+0000","last_unstale":"2023-11-08T19:28:15.768452+0000","last_undegraded":"2023-11-08T19:28:15.768452+0000","last_fullsized":"2023-11-08T19:28:15.768452+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:13.380399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745572+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:14.745572+0000","last_peered":"2023-11-08T19:28:14.745572+0000","last_clean":"2023-11-08T19:28:14.745572+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:14.745572+0000","last_undegraded":"2023-11-08T19:28:14.745572+0000","last_fullsized":"2023-11-08T19:28:14.745572+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743381+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:14.743381+0000","last_peered":"2023-11-08T19:28:14.743381+0000","last_clean":"2023-11-08T19:28:14.743381+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:14.743381+0000","last_undegraded":"2023-11-08T19:28:14.743381+0000","last_fullsized":"2023-11-08T19:28:14.743381+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745381+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:14.745381+0000","last_peered":"2023-11-08T19:28:14.745381+0000","last_clean":"2023-11-08T19:28:14.745381+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:14.745381+0000","last_undegraded":"2023-11-08T19:28:14.745381+0000","last_fullsized":"2023-11-08T19:28:14.745381+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766998+0000","last_change":"2023-11-08T19:28:15.753808+0000","last_active":"2023-11-08T19:28:15.766998+0000","last_peered":"2023-11-08T19:28:15.766998+0000","last_clean":"2023-11-08T19:28:15.766998+0000","last_became_active":"2023-11-08T19:28:15.753523+0000","last_became_peered":"2023-11-08T19:28:15.753523+0000","last_unstale":"2023-11-08T19:28:15.766998+0000","last_undegraded":"2023-11-08T19:28:15.766998+0000","last_fullsized":"2023-11-08T19:28:15.766998+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:57.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.731969+0000","last_change":"2023-11-08T19:28:14.742054+0000","last_active":"2023-11-08T19:28:15.731969+0000","last_peered":"2023-11-08T19:28:15.731969+0000","last_clean":"2023-11-08T19:28:15.731969+0000","last_became_active":"2023-11-08T19:28:14.741766+0000","last_became_peered":"2023-11-08T19:28:14.741766+0000","last_unstale":"2023-11-08T19:28:15.731969+0000","last_undegraded":"2023-11-08T19:28:15.731969+0000","last_fullsized":"2023-11-08T19:28:15.731969+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:23.666540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733774+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:14.733774+0000","last_peered":"2023-11-08T19:28:14.733774+0000","last_clean":"2023-11-08T19:28:14.733774+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:14.733774+0000","last_undegraded":"2023-11-08T19:28:14.733774+0000","last_fullsized":"2023-11-08T19:28:14.733774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733221+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:14.733221+0000","last_peered":"2023-11-08T19:28:14.733221+0000","last_clean":"2023-11-08T19:28:14.733221+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:14.733221+0000","last_undegraded":"2023-11-08T19:28:14.733221+0000","last_fullsized":"2023-11-08T19:28:14.733221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743039+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:14.743039+0000","last_peered":"2023-11-08T19:28:14.743039+0000","last_clean":"2023-11-08T19:28:14.743039+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:14.743039+0000","last_undegraded":"2023-11-08T19:28:14.743039+0000","last_fullsized":"2023-11-08T19:28:14.743039+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":11,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.755568+0000","last_change":"2023-11-08T19:28:15.760589+0000","last_active":"2023-11-08T19:28:16.755568+0000","last_peered":"2023-11-08T19:28:16.755568+0000","last_clean":"2023-11-08T19:28:16.755568+0000","last_became_active":"2023-11-08T19:28:15.760326+0000","last_became_peered":"2023-11-08T19:28:15.760326+0000","last_unstale":"2023-11-08T19:28:16.755568+0000","last_undegraded":"2023-11-08T19:28:16.755568+0000","last_fullsized":"2023-11-08T19:28:16.755568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:17.476942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.734825+0000","last_change":"2023-11-08T19:28:14.754260+0000","last_active":"2023-11-08T19:28:15.734825+0000","last_peered":"2023-11-08T19:28:15.734825+0000","last_clean":"2023-11-08T19:28:15.734825+0000","last_became_active":"2023-11-08T19:28:14.753948+0000","last_became_peered":"2023-11-08T19:28:14.753948+0000","last_unstale":"2023-11-08T19:28:15.734825+0000","last_undegraded":"2023-11-08T19:28:15.734825+0000","last_fullsized":"2023-11-08T19:28:15.734825+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:13.513400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.721124+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:14.721124+0000","last_peered":"2023-11-08T19:28:14.721124+0000","last_clean":"2023-11-08T19:28:14.721124+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:14.721124+0000","last_undegraded":"2023-11-08T19:28:14.721124+0000","last_fullsized":"2023-11-08T19:28:14.721124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733969+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:14.733969+0000","last_peered":"2023-11-08T19:28:14.733969+0000","last_clean":"2023-11-08T19:28:14.733969+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:14.733969+0000","last_undegraded":"2023-11-08T19:28:14.733969+0000","last_fullsized":"2023-11-08T19:28:14.733969+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748192+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:15.748192+0000","last_peered":"2023-11-08T19:28:15.748192+0000","last_clean":"2023-11-08T19:28:15.748192+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:15.748192+0000","last_undegraded":"2023-11-08T19:28:15.748192+0000","last_fullsized":"2023-11-08T19:28:15.748192+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773048+0000","last_change":"2023-11-08T19:28:15.768033+0000","last_active":"2023-11-08T19:28:15.773048+0000","last_peered":"2023-11-08T19:28:15.773048+0000","last_clean":"2023-11-08T19:28:15.773048+0000","last_became_active":"2023-11-08T19:28:15.767708+0000","last_became_peered":"2023-11-08T19:28:15.767708+0000","last_unstale":"2023-11-08T19:28:15.773048+0000","last_undegraded":"2023-11-08T19:28:15.773048+0000","last_fullsized":"2023-11-08T19:28:15.773048+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:51.558891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.760916+0000","last_change":"2023-11-08T19:28:14.752035+0000","last_active":"2023-11-08T19:28:15.760916+0000","last_peered":"2023-11-08T19:28:15.760916+0000","last_clean":"2023-11-08T19:28:15.760916+0000","last_became_active":"2023-11-08T19:28:14.751710+0000","last_became_peered":"2023-11-08T19:28:14.751710+0000","last_unstale":"2023-11-08T19:28:15.760916+0000","last_undegraded":"2023-11-08T19:28:15.760916+0000","last_fullsized":"2023-11-08T19:28:15.760916+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:59:53.400186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748566+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:15.748566+0000","last_peered":"2023-11-08T19:28:15.748566+0000","last_clean":"2023-11-08T19:28:15.748566+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:15.748566+0000","last_undegraded":"2023-11-08T19:28:15.748566+0000","last_fullsized":"2023-11-08T19:28:15.748566+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722022+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:14.722022+0000","last_peered":"2023-11-08T19:28:14.722022+0000","last_clean":"2023-11-08T19:28:14.722022+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:14.722022+0000","last_undegraded":"2023-11-08T19:28:14.722022+0000","last_fullsized":"2023-11-08T19:28:14.722022+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742366+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:14.742366+0000","last_peered":"2023-11-08T19:28:14.742366+0000","last_clean":"2023-11-08T19:28:14.742366+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:14.742366+0000","last_undegraded":"2023-11-08T19:28:14.742366+0000","last_fullsized":"2023-11-08T19:28:14.742366+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766092+0000","last_change":"2023-11-08T19:28:15.754355+0000","last_active":"2023-11-08T19:28:15.766092+0000","last_peered":"2023-11-08T19:28:15.766092+0000","last_clean":"2023-11-08T19:28:15.766092+0000","last_became_active":"2023-11-08T19:28:15.754041+0000","last_became_peered":"2023-11-08T19:28:15.754041+0000","last_unstale":"2023-11-08T19:28:15.766092+0000","last_undegraded":"2023-11-08T19:28:15.766092+0000","last_fullsized":"2023-11-08T19:28:15.766092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:51:57.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.761036+0000","last_change":"2023-11-08T19:28:14.750245+0000","last_active":"2023-11-08T19:28:15.761036+0000","last_peered":"2023-11-08T19:28:15.761036+0000","last_clean":"2023-11-08T19:28:15.761036+0000","last_became_active":"2023-11-08T19:28:14.749951+0000","last_became_peered":"2023-11-08T19:28:14.749951+0000","last_unstale":"2023-11-08T19:28:15.761036+0000","last_undegraded":"2023-11-08T19:28:15.761036+0000","last_fullsized":"2023-11-08T19:28:15.761036+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:56.096338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744053+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:14.744053+0000","last_peered":"2023-11-08T19:28:14.744053+0000","last_clean":"2023-11-08T19:28:14.744053+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:14.744053+0000","last_undegraded":"2023-11-08T19:28:14.744053+0000","last_fullsized":"2023-11-08T19:28:14.744053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742130+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:14.742130+0000","last_peered":"2023-11-08T19:28:14.742130+0000","last_clean":"2023-11-08T19:28:14.742130+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:14.742130+0000","last_undegraded":"2023-11-08T19:28:14.742130+0000","last_fullsized":"2023-11-08T19:28:14.742130+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.749495+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:14.749495+0000","last_peered":"2023-11-08T19:28:14.749495+0000","last_clean":"2023-11-08T19:28:14.749495+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:14.749495+0000","last_undegraded":"2023-11-08T19:28:14.749495+0000","last_fullsized":"2023-11-08T19:28:14.749495+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770074+0000","last_change":"2023-11-08T19:28:15.765356+0000","last_active":"2023-11-08T19:28:15.770074+0000","last_peered":"2023-11-08T19:28:15.770074+0000","last_clean":"2023-11-08T19:28:15.770074+0000","last_became_active":"2023-11-08T19:28:15.765089+0000","last_became_peered":"2023-11-08T19:28:15.765089+0000","last_unstale":"2023-11-08T19:28:15.770074+0000","last_undegraded":"2023-11-08T19:28:15.770074+0000","last_fullsized":"2023-11-08T19:28:15.770074+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:40.670042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.733911+0000","last_change":"2023-11-08T19:28:14.751940+0000","last_active":"2023-11-08T19:28:15.733911+0000","last_peered":"2023-11-08T19:28:15.733911+0000","last_clean":"2023-11-08T19:28:15.733911+0000","last_became_active":"2023-11-08T19:28:14.751598+0000","last_became_peered":"2023-11-08T19:28:14.751598+0000","last_unstale":"2023-11-08T19:28:15.733911+0000","last_undegraded":"2023-11-08T19:28:15.733911+0000","last_fullsized":"2023-11-08T19:28:15.733911+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:40.572679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750767+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:14.750767+0000","last_peered":"2023-11-08T19:28:14.750767+0000","last_clean":"2023-11-08T19:28:14.750767+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:14.750767+0000","last_undegraded":"2023-11-08T19:28:14.750767+0000","last_fullsized":"2023-11-08T19:28:14.750767+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722585+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:14.722585+0000","last_peered":"2023-11-08T19:28:14.722585+0000","last_clean":"2023-11-08T19:28:14.722585+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:14.722585+0000","last_undegraded":"2023-11-08T19:28:14.722585+0000","last_fullsized":"2023-11-08T19:28:14.722585+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.746240+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:14.746240+0000","last_peered":"2023-11-08T19:28:14.746240+0000","last_clean":"2023-11-08T19:28:14.746240+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:14.746240+0000","last_undegraded":"2023-11-08T19:28:14.746240+0000","last_fullsized":"2023-11-08T19:28:14.746240+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.762346+0000","last_change":"2023-11-08T19:28:15.752353+0000","last_active":"2023-11-08T19:28:15.762346+0000","last_peered":"2023-11-08T19:28:15.762346+0000","last_clean":"2023-11-08T19:28:15.762346+0000","last_became_active":"2023-11-08T19:28:15.751576+0000","last_became_peered":"2023-11-08T19:28:15.751576+0000","last_unstale":"2023-11-08T19:28:15.762346+0000","last_undegraded":"2023-11-08T19:28:15.762346+0000","last_fullsized":"2023-11-08T19:28:15.762346+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:06.852441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.727933+0000","last_change":"2023-11-08T19:28:14.755494+0000","last_active":"2023-11-08T19:28:15.727933+0000","last_peered":"2023-11-08T19:28:15.727933+0000","last_clean":"2023-11-08T19:28:15.727933+0000","last_became_active":"2023-11-08T19:28:14.754954+0000","last_became_peered":"2023-11-08T19:28:14.754954+0000","last_unstale":"2023-11-08T19:28:15.727933+0000","last_undegraded":"2023-11-08T19:28:15.727933+0000","last_fullsized":"2023-11-08T19:28:15.727933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:37:58.433609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745305+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:14.745305+0000","last_peered":"2023-11-08T19:28:14.745305+0000","last_clean":"2023-11-08T19:28:14.745305+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:14.745305+0000","last_undegraded":"2023-11-08T19:28:14.745305+0000","last_fullsized":"2023-11-08T19:28:14.745305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":26,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.723302+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:14.723302+0000","last_peered":"2023-11-08T19:28:14.723302+0000","last_clean":"2023-11-08T19:28:14.723302+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:14.723302+0000","last_undegraded":"2023-11-08T19:28:14.723302+0000","last_fullsized":"2023-11-08T19:28:14.723302+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745850+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:14.745850+0000","last_peered":"2023-11-08T19:28:14.745850+0000","last_clean":"2023-11-08T19:28:14.745850+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:14.745850+0000","last_undegraded":"2023-11-08T19:28:14.745850+0000","last_fullsized":"2023-11-08T19:28:14.745850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759582+0000","last_change":"2023-11-08T19:28:15.752263+0000","last_active":"2023-11-08T19:28:15.759582+0000","last_peered":"2023-11-08T19:28:15.759582+0000","last_clean":"2023-11-08T19:28:15.759582+0000","last_became_active":"2023-11-08T19:28:15.751722+0000","last_became_peered":"2023-11-08T19:28:15.751722+0000","last_unstale":"2023-11-08T19:28:15.759582+0000","last_undegraded":"2023-11-08T19:28:15.759582+0000","last_fullsized":"2023-11-08T19:28:15.759582+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:57:25.668314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.731824+0000","last_change":"2023-11-08T19:28:14.752706+0000","last_active":"2023-11-08T19:28:15.731824+0000","last_peered":"2023-11-08T19:28:15.731824+0000","last_clean":"2023-11-08T19:28:15.731824+0000","last_became_active":"2023-11-08T19:28:14.751744+0000","last_became_peered":"2023-11-08T19:28:14.751744+0000","last_unstale":"2023-11-08T19:28:15.731824+0000","last_undegraded":"2023-11-08T19:28:15.731824+0000","last_fullsized":"2023-11-08T19:28:15.731824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:37:59.108961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742241+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:14.742241+0000","last_peered":"2023-11-08T19:28:14.742241+0000","last_clean":"2023-11-08T19:28:14.742241+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:14.742241+0000","last_undegraded":"2023-11-08T19:28:14.742241+0000","last_fullsized":"2023-11-08T19:28:14.742241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745669+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:14.745669+0000","last_peered":"2023-11-08T19:28:14.745669+0000","last_clean":"2023-11-08T19:28:14.745669+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:14.745669+0000","last_undegraded":"2023-11-08T19:28:14.745669+0000","last_fullsized":"2023-11-08T19:28:14.745669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743150+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:14.743150+0000","last_peered":"2023-11-08T19:28:14.743150+0000","last_clean":"2023-11-08T19:28:14.743150+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:14.743150+0000","last_undegraded":"2023-11-08T19:28:14.743150+0000","last_fullsized":"2023-11-08T19:28:14.743150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759244+0000","last_change":"2023-11-08T19:28:15.754884+0000","last_active":"2023-11-08T19:28:15.759244+0000","last_peered":"2023-11-08T19:28:15.759244+0000","last_clean":"2023-11-08T19:28:15.759244+0000","last_became_active":"2023-11-08T19:28:15.754594+0000","last_became_peered":"2023-11-08T19:28:15.754594+0000","last_unstale":"2023-11-08T19:28:15.759244+0000","last_undegraded":"2023-11-08T19:28:15.759244+0000","last_fullsized":"2023-11-08T19:28:15.759244+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:40.606689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.772149+0000","last_change":"2023-11-08T19:28:14.749740+0000","last_active":"2023-11-08T19:28:15.772149+0000","last_peered":"2023-11-08T19:28:15.772149+0000","last_clean":"2023-11-08T19:28:15.772149+0000","last_became_active":"2023-11-08T19:28:14.749459+0000","last_became_peered":"2023-11-08T19:28:14.749459+0000","last_unstale":"2023-11-08T19:28:15.772149+0000","last_undegraded":"2023-11-08T19:28:15.772149+0000","last_fullsized":"2023-11-08T19:28:15.772149+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:45.277928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745774+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:14.745774+0000","last_peered":"2023-11-08T19:28:14.745774+0000","last_clean":"2023-11-08T19:28:14.745774+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:14.745774+0000","last_undegraded":"2023-11-08T19:28:14.745774+0000","last_fullsized":"2023-11-08T19:28:14.745774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745464+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:14.745464+0000","last_peered":"2023-11-08T19:28:14.745464+0000","last_clean":"2023-11-08T19:28:14.745464+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:14.745464+0000","last_undegraded":"2023-11-08T19:28:14.745464+0000","last_fullsized":"2023-11-08T19:28:14.745464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744408+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:14.744408+0000","last_peered":"2023-11-08T19:28:14.744408+0000","last_clean":"2023-11-08T19:28:14.744408+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:14.744408+0000","last_undegraded":"2023-11-08T19:28:14.744408+0000","last_fullsized":"2023-11-08T19:28:14.744408+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770419+0000","last_change":"2023-11-08T19:28:15.764787+0000","last_active":"2023-11-08T19:28:15.770419+0000","last_peered":"2023-11-08T19:28:15.770419+0000","last_clean":"2023-11-08T19:28:15.770419+0000","last_became_active":"2023-11-08T19:28:15.764476+0000","last_became_peered":"2023-11-08T19:28:15.764476+0000","last_unstale":"2023-11-08T19:28:15.770419+0000","last_undegraded":"2023-11-08T19:28:15.770419+0000","last_fullsized":"2023-11-08T19:28:15.770419+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:31.801434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.736717+0000","last_change":"2023-11-08T19:28:14.754289+0000","last_active":"2023-11-08T19:28:15.736717+0000","last_peered":"2023-11-08T19:28:15.736717+0000","last_clean":"2023-11-08T19:28:15.736717+0000","last_became_active":"2023-11-08T19:28:14.754024+0000","last_became_peered":"2023-11-08T19:28:14.754024+0000","last_unstale":"2023-11-08T19:28:15.736717+0000","last_undegraded":"2023-11-08T19:28:15.736717+0000","last_fullsized":"2023-11-08T19:28:15.736717+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:57.783594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742060+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:14.742060+0000","last_peered":"2023-11-08T19:28:14.742060+0000","last_clean":"2023-11-08T19:28:14.742060+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:14.742060+0000","last_undegraded":"2023-11-08T19:28:14.742060+0000","last_fullsized":"2023-11-08T19:28:14.742060+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733432+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:14.733432+0000","last_peered":"2023-11-08T19:28:14.733432+0000","last_clean":"2023-11-08T19:28:14.733432+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:14.733432+0000","last_undegraded":"2023-11-08T19:28:14.733432+0000","last_fullsized":"2023-11-08T19:28:14.733432+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742021+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:14.742021+0000","last_peered":"2023-11-08T19:28:14.742021+0000","last_clean":"2023-11-08T19:28:14.742021+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:14.742021+0000","last_undegraded":"2023-11-08T19:28:14.742021+0000","last_fullsized":"2023-11-08T19:28:14.742021+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.759122+0000","last_change":"2023-11-08T19:28:15.754694+0000","last_active":"2023-11-08T19:28:15.759122+0000","last_peered":"2023-11-08T19:28:15.759122+0000","last_clean":"2023-11-08T19:28:15.759122+0000","last_became_active":"2023-11-08T19:28:15.753441+0000","last_became_peered":"2023-11-08T19:28:15.753441+0000","last_unstale":"2023-11-08T19:28:15.759122+0000","last_undegraded":"2023-11-08T19:28:15.759122+0000","last_fullsized":"2023-11-08T19:28:15.759122+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:48.543261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2023-11-08T19:28:16.756792+0000","last_change":"2023-11-08T19:28:14.741270+0000","last_active":"2023-11-08T19:28:16.756792+0000","last_peered":"2023-11-08T19:28:16.756792+0000","last_clean":"2023-11-08T19:28:16.756792+0000","last_became_active":"2023-11-08T19:28:14.740995+0000","last_became_peered":"2023-11-08T19:28:14.740995+0000","last_unstale":"2023-11-08T19:28:16.756792+0000","last_undegraded":"2023-11-08T19:28:16.756792+0000","last_fullsized":"2023-11-08T19:28:16.756792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:12.090286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.729040+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:14.729040+0000","last_peered":"2023-11-08T19:28:14.729040+0000","last_clean":"2023-11-08T19:28:14.729040+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:14.729040+0000","last_undegraded":"2023-11-08T19:28:14.729040+0000","last_fullsized":"2023-11-08T19:28:14.729040+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742219+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:14.742219+0000","last_peered":"2023-11-08T19:28:14.742219+0000","last_clean":"2023-11-08T19:28:14.742219+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:14.742219+0000","last_undegraded":"2023-11-08T19:28:14.742219+0000","last_fullsized":"2023-11-08T19:28:14.742219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":50,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.740234+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:14.740234+0000","last_peered":"2023-11-08T19:28:14.740234+0000","last_clean":"2023-11-08T19:28:14.740234+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:14.740234+0000","last_undegraded":"2023-11-08T19:28:14.740234+0000","last_fullsized":"2023-11-08T19:28:14.740234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.771392+0000","last_change":"2023-11-08T19:28:15.761636+0000","last_active":"2023-11-08T19:28:15.771392+0000","last_peered":"2023-11-08T19:28:15.771392+0000","last_clean":"2023-11-08T19:28:15.771392+0000","last_became_active":"2023-11-08T19:28:15.759087+0000","last_became_peered":"2023-11-08T19:28:15.759087+0000","last_unstale":"2023-11-08T19:28:15.771392+0000","last_undegraded":"2023-11-08T19:28:15.771392+0000","last_fullsized":"2023-11-08T19:28:15.771392+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:31.801434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.732308+0000","last_change":"2023-11-08T19:28:14.754762+0000","last_active":"2023-11-08T19:28:15.732308+0000","last_peered":"2023-11-08T19:28:15.732308+0000","last_clean":"2023-11-08T19:28:15.732308+0000","last_became_active":"2023-11-08T19:28:14.754429+0000","last_became_peered":"2023-11-08T19:28:14.754429+0000","last_unstale":"2023-11-08T19:28:15.732308+0000","last_undegraded":"2023-11-08T19:28:15.732308+0000","last_fullsized":"2023-11-08T19:28:15.732308+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:18.066983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745303+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:14.745303+0000","last_peered":"2023-11-08T19:28:14.745303+0000","last_clean":"2023-11-08T19:28:14.745303+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:14.745303+0000","last_undegraded":"2023-11-08T19:28:14.745303+0000","last_fullsized":"2023-11-08T19:28:14.745303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.733180+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:14.733180+0000","last_peered":"2023-11-08T19:28:14.733180+0000","last_clean":"2023-11-08T19:28:14.733180+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:14.733180+0000","last_undegraded":"2023-11-08T19:28:14.733180+0000","last_fullsized":"2023-11-08T19:28:14.733180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722810+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:14.722810+0000","last_peered":"2023-11-08T19:28:14.722810+0000","last_clean":"2023-11-08T19:28:14.722810+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:14.722810+0000","last_undegraded":"2023-11-08T19:28:14.722810+0000","last_fullsized":"2023-11-08T19:28:14.722810+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.762531+0000","last_change":"2023-11-08T19:28:15.754080+0000","last_active":"2023-11-08T19:28:15.762531+0000","last_peered":"2023-11-08T19:28:15.762531+0000","last_clean":"2023-11-08T19:28:15.762531+0000","last_became_active":"2023-11-08T19:28:15.753740+0000","last_became_peered":"2023-11-08T19:28:15.753740+0000","last_unstale":"2023-11-08T19:28:15.762531+0000","last_undegraded":"2023-11-08T19:28:15.762531+0000","last_fullsized":"2023-11-08T19:28:15.762531+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:47.085580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.729018+0000","last_change":"2023-11-08T19:28:14.746433+0000","last_active":"2023-11-08T19:28:15.729018+0000","last_peered":"2023-11-08T19:28:15.729018+0000","last_clean":"2023-11-08T19:28:15.729018+0000","last_became_active":"2023-11-08T19:28:14.746149+0000","last_became_peered":"2023-11-08T19:28:14.746149+0000","last_unstale":"2023-11-08T19:28:15.729018+0000","last_undegraded":"2023-11-08T19:28:15.729018+0000","last_fullsized":"2023-11-08T19:28:15.729018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:40:30.481143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744155+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:14.744155+0000","last_peered":"2023-11-08T19:28:14.744155+0000","last_clean":"2023-11-08T19:28:14.744155+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:14.744155+0000","last_undegraded":"2023-11-08T19:28:14.744155+0000","last_fullsized":"2023-11-08T19:28:14.744155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.745464+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:14.745464+0000","last_peered":"2023-11-08T19:28:14.745464+0000","last_clean":"2023-11-08T19:28:14.745464+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:14.745464+0000","last_undegraded":"2023-11-08T19:28:14.745464+0000","last_fullsized":"2023-11-08T19:28:14.745464+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.749259+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:15.749259+0000","last_peered":"2023-11-08T19:28:15.749259+0000","last_clean":"2023-11-08T19:28:15.749259+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:15.749259+0000","last_undegraded":"2023-11-08T19:28:15.749259+0000","last_fullsized":"2023-11-08T19:28:15.749259+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.777033+0000","last_change":"2023-11-08T19:28:15.772651+0000","last_active":"2023-11-08T19:28:15.777033+0000","last_peered":"2023-11-08T19:28:15.777033+0000","last_clean":"2023-11-08T19:28:15.777033+0000","last_became_active":"2023-11-08T19:28:15.772183+0000","last_became_peered":"2023-11-08T19:28:15.772183+0000","last_unstale":"2023-11-08T19:28:15.777033+0000","last_undegraded":"2023-11-08T19:28:15.777033+0000","last_fullsized":"2023-11-08T19:28:15.777033+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:38:48.264507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.728756+0000","last_change":"2023-11-08T19:28:14.748551+0000","last_active":"2023-11-08T19:28:15.728756+0000","last_peered":"2023-11-08T19:28:15.728756+0000","last_clean":"2023-11-08T19:28:15.728756+0000","last_became_active":"2023-11-08T19:28:14.748175+0000","last_became_peered":"2023-11-08T19:28:14.748175+0000","last_unstale":"2023-11-08T19:28:15.728756+0000","last_undegraded":"2023-11-08T19:28:15.728756+0000","last_fullsized":"2023-11-08T19:28:15.728756+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:12.090286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747198+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:14.747198+0000","last_peered":"2023-11-08T19:28:14.747198+0000","last_clean":"2023-11-08T19:28:14.747198+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:14.747198+0000","last_undegraded":"2023-11-08T19:28:14.747198+0000","last_fullsized":"2023-11-08T19:28:14.747198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743714+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:14.743714+0000","last_peered":"2023-11-08T19:28:14.743714+0000","last_clean":"2023-11-08T19:28:14.743714+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:14.743714+0000","last_undegraded":"2023-11-08T19:28:14.743714+0000","last_fullsized":"2023-11-08T19:28:14.743714+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750528+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:14.750528+0000","last_peered":"2023-11-08T19:28:14.750528+0000","last_clean":"2023-11-08T19:28:14.750528+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:14.750528+0000","last_undegraded":"2023-11-08T19:28:14.750528+0000","last_fullsized":"2023-11-08T19:28:14.750528+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766047+0000","last_change":"2023-11-08T19:28:15.756945+0000","last_active":"2023-11-08T19:28:15.766047+0000","last_peered":"2023-11-08T19:28:15.766047+0000","last_clean":"2023-11-08T19:28:15.766047+0000","last_became_active":"2023-11-08T19:28:15.756618+0000","last_became_peered":"2023-11-08T19:28:15.756618+0000","last_unstale":"2023-11-08T19:28:15.766047+0000","last_undegraded":"2023-11-08T19:28:15.766047+0000","last_fullsized":"2023-11-08T19:28:15.766047+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:31.801434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.771516+0000","last_change":"2023-11-08T19:28:14.748630+0000","last_active":"2023-11-08T19:28:15.771516+0000","last_peered":"2023-11-08T19:28:15.771516+0000","last_clean":"2023-11-08T19:28:15.771516+0000","last_became_active":"2023-11-08T19:28:14.748275+0000","last_became_peered":"2023-11-08T19:28:14.748275+0000","last_unstale":"2023-11-08T19:28:15.771516+0000","last_undegraded":"2023-11-08T19:28:15.771516+0000","last_fullsized":"2023-11-08T19:28:15.771516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:21.923551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.750935+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:14.750935+0000","last_peered":"2023-11-08T19:28:14.750935+0000","last_clean":"2023-11-08T19:28:14.750935+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:14.750935+0000","last_undegraded":"2023-11-08T19:28:14.750935+0000","last_fullsized":"2023-11-08T19:28:14.750935+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747123+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:14.747123+0000","last_peered":"2023-11-08T19:28:14.747123+0000","last_clean":"2023-11-08T19:28:14.747123+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:14.747123+0000","last_undegraded":"2023-11-08T19:28:14.747123+0000","last_fullsized":"2023-11-08T19:28:14.747123+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742716+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:14.742716+0000","last_peered":"2023-11-08T19:28:14.742716+0000","last_clean":"2023-11-08T19:28:14.742716+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:14.742716+0000","last_undegraded":"2023-11-08T19:28:14.742716+0000","last_fullsized":"2023-11-08T19:28:14.742716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.766383+0000","last_change":"2023-11-08T19:28:15.757403+0000","last_active":"2023-11-08T19:28:15.766383+0000","last_peered":"2023-11-08T19:28:15.766383+0000","last_clean":"2023-11-08T19:28:15.766383+0000","last_became_active":"2023-11-08T19:28:15.757090+0000","last_became_peered":"2023-11-08T19:28:15.757090+0000","last_unstale":"2023-11-08T19:28:15.766383+0000","last_undegraded":"2023-11-08T19:28:15.766383+0000","last_fullsized":"2023-11-08T19:28:15.766383+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:57.342920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.763762+0000","last_change":"2023-11-08T19:28:14.750109+0000","last_active":"2023-11-08T19:28:15.763762+0000","last_peered":"2023-11-08T19:28:15.763762+0000","last_clean":"2023-11-08T19:28:15.763762+0000","last_became_active":"2023-11-08T19:28:14.749769+0000","last_became_peered":"2023-11-08T19:28:14.749769+0000","last_unstale":"2023-11-08T19:28:15.763762+0000","last_undegraded":"2023-11-08T19:28:15.763762+0000","last_fullsized":"2023-11-08T19:28:15.763762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:54:40.572679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742124+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:14.742124+0000","last_peered":"2023-11-08T19:28:14.742124+0000","last_clean":"2023-11-08T19:28:14.742124+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:14.742124+0000","last_undegraded":"2023-11-08T19:28:14.742124+0000","last_fullsized":"2023-11-08T19:28:14.742124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.742574+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:14.742574+0000","last_peered":"2023-11-08T19:28:14.742574+0000","last_clean":"2023-11-08T19:28:14.742574+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:14.742574+0000","last_undegraded":"2023-11-08T19:28:14.742574+0000","last_fullsized":"2023-11-08T19:28:14.742574+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.747455+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:14.747455+0000","last_peered":"2023-11-08T19:28:14.747455+0000","last_clean":"2023-11-08T19:28:14.747455+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:14.747455+0000","last_undegraded":"2023-11-08T19:28:14.747455+0000","last_fullsized":"2023-11-08T19:28:14.747455+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.770398+0000","last_change":"2023-11-08T19:28:15.762902+0000","last_active":"2023-11-08T19:28:15.770398+0000","last_peered":"2023-11-08T19:28:15.770398+0000","last_clean":"2023-11-08T19:28:15.770398+0000","last_became_active":"2023-11-08T19:28:15.762563+0000","last_became_peered":"2023-11-08T19:28:15.762563+0000","last_unstale":"2023-11-08T19:28:15.770398+0000","last_undegraded":"2023-11-08T19:28:15.770398+0000","last_fullsized":"2023-11-08T19:28:15.770398+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:38:48.264507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.726799+0000","last_change":"2023-11-08T19:28:14.747542+0000","last_active":"2023-11-08T19:28:15.726799+0000","last_peered":"2023-11-08T19:28:15.726799+0000","last_clean":"2023-11-08T19:28:15.726799+0000","last_became_active":"2023-11-08T19:28:14.747121+0000","last_became_peered":"2023-11-08T19:28:14.747121+0000","last_unstale":"2023-11-08T19:28:15.726799+0000","last_undegraded":"2023-11-08T19:28:15.726799+0000","last_fullsized":"2023-11-08T19:28:15.726799+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:49:22.533166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.726792+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:14.726792+0000","last_peered":"2023-11-08T19:28:14.726792+0000","last_clean":"2023-11-08T19:28:14.726792+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:14.726792+0000","last_undegraded":"2023-11-08T19:28:14.726792+0000","last_fullsized":"2023-11-08T19:28:14.726792+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.739090+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:14.739090+0000","last_peered":"2023-11-08T19:28:14.739090+0000","last_clean":"2023-11-08T19:28:14.739090+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:14.739090+0000","last_undegraded":"2023-11-08T19:28:14.739090+0000","last_fullsized":"2023-11-08T19:28:14.739090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.734680+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:14.734680+0000","last_peered":"2023-11-08T19:28:14.734680+0000","last_clean":"2023-11-08T19:28:14.734680+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:14.734680+0000","last_undegraded":"2023-11-08T19:28:14.734680+0000","last_fullsized":"2023-11-08T19:28:14.734680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.760106+0000","last_change":"2023-11-08T19:28:15.755779+0000","last_active":"2023-11-08T19:28:15.760106+0000","last_peered":"2023-11-08T19:28:15.760106+0000","last_clean":"2023-11-08T19:28:15.760106+0000","last_became_active":"2023-11-08T19:28:15.755532+0000","last_became_peered":"2023-11-08T19:28:15.755532+0000","last_unstale":"2023-11-08T19:28:15.760106+0000","last_undegraded":"2023-11-08T19:28:15.760106+0000","last_fullsized":"2023-11-08T19:28:15.760106+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:03.335683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.737993+0000","last_change":"2023-11-08T19:28:14.748127+0000","last_active":"2023-11-08T19:28:15.737993+0000","last_peered":"2023-11-08T19:28:15.737993+0000","last_clean":"2023-11-08T19:28:15.737993+0000","last_became_active":"2023-11-08T19:28:14.747832+0000","last_became_peered":"2023-11-08T19:28:14.747832+0000","last_unstale":"2023-11-08T19:28:15.737993+0000","last_undegraded":"2023-11-08T19:28:15.737993+0000","last_fullsized":"2023-11-08T19:28:15.737993+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:18.066983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.739183+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:14.739183+0000","last_peered":"2023-11-08T19:28:14.739183+0000","last_clean":"2023-11-08T19:28:14.739183+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:14.739183+0000","last_undegraded":"2023-11-08T19:28:14.739183+0000","last_fullsized":"2023-11-08T19:28:14.739183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748834+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:15.748834+0000","last_peered":"2023-11-08T19:28:15.748834+0000","last_clean":"2023-11-08T19:28:15.748834+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:15.748834+0000","last_undegraded":"2023-11-08T19:28:15.748834+0000","last_fullsized":"2023-11-08T19:28:15.748834+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":53,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.726615+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:14.726615+0000","last_peered":"2023-11-08T19:28:14.726615+0000","last_clean":"2023-11-08T19:28:14.726615+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:14.726615+0000","last_undegraded":"2023-11-08T19:28:14.726615+0000","last_fullsized":"2023-11-08T19:28:14.726615+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.753706+0000","last_change":"2023-11-08T19:28:15.750541+0000","last_active":"2023-11-08T19:28:15.753706+0000","last_peered":"2023-11-08T19:28:15.753706+0000","last_clean":"2023-11-08T19:28:15.753706+0000","last_became_active":"2023-11-08T19:28:15.750259+0000","last_became_peered":"2023-11-08T19:28:15.750259+0000","last_unstale":"2023-11-08T19:28:15.753706+0000","last_undegraded":"2023-11-08T19:28:15.753706+0000","last_fullsized":"2023-11-08T19:28:15.753706+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:03:24.026858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.739321+0000","last_change":"2023-11-08T19:28:14.744027+0000","last_active":"2023-11-08T19:28:15.739321+0000","last_peered":"2023-11-08T19:28:15.739321+0000","last_clean":"2023-11-08T19:28:15.739321+0000","last_became_active":"2023-11-08T19:28:14.743733+0000","last_became_peered":"2023-11-08T19:28:14.743733+0000","last_unstale":"2023-11-08T19:28:15.739321+0000","last_undegraded":"2023-11-08T19:28:15.739321+0000","last_fullsized":"2023-11-08T19:28:15.739321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:58:43.179019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.727346+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:14.727346+0000","last_peered":"2023-11-08T19:28:14.727346+0000","last_clean":"2023-11-08T19:28:14.727346+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:14.727346+0000","last_undegraded":"2023-11-08T19:28:14.727346+0000","last_fullsized":"2023-11-08T19:28:14.727346+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.748099+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:15.748099+0000","last_peered":"2023-11-08T19:28:15.748099+0000","last_clean":"2023-11-08T19:28:15.748099+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:15.748099+0000","last_undegraded":"2023-11-08T19:28:15.748099+0000","last_fullsized":"2023-11-08T19:28:15.748099+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.750693+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:15.750693+0000","last_peered":"2023-11-08T19:28:15.750693+0000","last_clean":"2023-11-08T19:28:15.750693+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:15.750693+0000","last_undegraded":"2023-11-08T19:28:15.750693+0000","last_fullsized":"2023-11-08T19:28:15.750693+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.767442+0000","last_change":"2023-11-08T19:28:15.763123+0000","last_active":"2023-11-08T19:28:15.767442+0000","last_peered":"2023-11-08T19:28:15.767442+0000","last_clean":"2023-11-08T19:28:15.767442+0000","last_became_active":"2023-11-08T19:28:15.762804+0000","last_became_peered":"2023-11-08T19:28:15.762804+0000","last_unstale":"2023-11-08T19:28:15.767442+0000","last_undegraded":"2023-11-08T19:28:15.767442+0000","last_fullsized":"2023-11-08T19:28:15.767442+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:03:00.483579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"82'2","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.757584+0000","last_change":"2023-11-08T19:28:14.749481+0000","last_active":"2023-11-08T19:28:15.757584+0000","last_peered":"2023-11-08T19:28:15.757584+0000","last_clean":"2023-11-08T19:28:15.757584+0000","last_became_active":"2023-11-08T19:28:14.749228+0000","last_became_peered":"2023-11-08T19:28:14.749228+0000","last_unstale":"2023-11-08T19:28:15.757584+0000","last_undegraded":"2023-11-08T19:28:15.757584+0000","last_fullsized":"2023-11-08T19:28:15.757584+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:16.466751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.740089+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:14.740089+0000","last_peered":"2023-11-08T19:28:14.740089+0000","last_clean":"2023-11-08T19:28:14.740089+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:14.740089+0000","last_undegraded":"2023-11-08T19:28:14.740089+0000","last_fullsized":"2023-11-08T19:28:14.740089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.741938+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:14.741938+0000","last_peered":"2023-11-08T19:28:14.741938+0000","last_clean":"2023-11-08T19:28:14.741938+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:14.741938+0000","last_undegraded":"2023-11-08T19:28:14.741938+0000","last_fullsized":"2023-11-08T19:28:14.741938+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":50,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.749396+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:15.749396+0000","last_peered":"2023-11-08T19:28:15.749396+0000","last_clean":"2023-11-08T19:28:15.749396+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:15.749396+0000","last_undegraded":"2023-11-08T19:28:15.749396+0000","last_fullsized":"2023-11-08T19:28:15.749396+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.764472+0000","last_change":"2023-11-08T19:28:15.757390+0000","last_active":"2023-11-08T19:28:15.764472+0000","last_peered":"2023-11-08T19:28:15.764472+0000","last_clean":"2023-11-08T19:28:15.764472+0000","last_became_active":"2023-11-08T19:28:15.757144+0000","last_became_peered":"2023-11-08T19:28:15.757144+0000","last_unstale":"2023-11-08T19:28:15.764472+0000","last_undegraded":"2023-11-08T19:28:15.764472+0000","last_fullsized":"2023-11-08T19:28:15.764472+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:14.700710+0000","last_clean_scrub_stamp":"2023-11-08T19:28:14.700710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:16:13.791856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"82'1","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2023-11-08T19:28:15.773371+0000","last_change":"2023-11-08T19:28:14.759109+0000","last_active":"2023-11-08T19:28:15.773371+0000","last_peered":"2023-11-08T19:28:15.773371+0000","last_clean":"2023-11-08T19:28:15.773371+0000","last_became_active":"2023-11-08T19:28:14.758784+0000","last_became_peered":"2023-11-08T19:28:14.758784+0000","last_unstale":"2023-11-08T19:28:15.773371+0000","last_undegraded":"2023-11-08T19:28:15.773371+0000","last_fullsized":"2023-11-08T19:28:15.773371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:13.690519+0000","last_clean_scrub_stamp":"2023-11-08T19:28:13.690519+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:21.923551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.722132+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:14.722132+0000","last_peered":"2023-11-08T19:28:14.722132+0000","last_clean":"2023-11-08T19:28:14.722132+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:14.722132+0000","last_undegraded":"2023-11-08T19:28:14.722132+0000","last_fullsized":"2023-11-08T19:28:14.722132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.749316+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:14.749316+0000","last_peered":"2023-11-08T19:28:14.749316+0000","last_clean":"2023-11-08T19:28:14.749316+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:14.749316+0000","last_undegraded":"2023-11-08T19:28:14.749316+0000","last_fullsized":"2023-11-08T19:28:14.749316+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":49,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.743838+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:14.743838+0000","last_peered":"2023-11-08T19:28:14.743838+0000","last_clean":"2023-11-08T19:28:14.743838+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:14.743838+0000","last_undegraded":"2023-11-08T19:28:14.743838+0000","last_fullsized":"2023-11-08T19:28:14.743838+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T19:28:14.744369+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:14.744369+0000","last_peered":"2023-11-08T19:28:14.744369+0000","last_clean":"2023-11-08T19:28:14.744369+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:14.744369+0000","last_undegraded":"2023-11-08T19:28:14.744369+0000","last_fullsized":"2023-11-08T19:28:14.744369+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1605632,"data_stored":1078185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.52,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":1.52},{"interface":"front","average":{"1min":1.141,"5min":1.25,"15min":1.3120000000000001},"min":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1360000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.96799999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.196,"15min":1.2569999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.518,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.125,"15min":1.1359999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.242}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.405,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.129},{"interface":"front","average":{"1min":1.056,"5min":1.28,"15min":1.3620000000000001},"min":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.389,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.452}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3129999999999999,"15min":1.367},"min":{"1min":0.79000000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.591,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.506,"15min":1.5700000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2200000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.399}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.798,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.635},{"interface":"front","average":{"1min":1.321,"5min":1.427,"15min":1.4970000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.774,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.726}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.73299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7829999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.571}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.546,"15min":1.579},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.9770000000000001,"5min":2.427,"15min":2.427},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.5680000000000001,"15min":1.631},"min":{"1min":0.997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.8939999999999999,"5min":2.468,"15min":2.468},"last":2.117}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1011712,"data_stored":488357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96999999999999997,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6279999999999999,"5min":1.752,"15min":1.752},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.083,"5min":1.1140000000000001,"15min":1.1060000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.581,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.095,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.47,"5min":1.679,"15min":1.679},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.2789999999999999,"15min":1.337},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6559999999999999,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":1.4570000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.238,"15min":1.292},"min":{"1min":0.68799999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4359999999999999,"5min":4.609,"15min":4.609},"last":1.375},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.861,"15min":1.861},"last":1.3340000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4350000000000001,"15min":1.4870000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.756,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.278},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1819999999999999,"15min":1.181},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.482,"5min":1.784,"15min":1.784},"last":1.4410000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.371,"15min":1.377},"min":{"1min":0.79600000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.732,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.125},{"interface":"front","average":{"1min":1.585,"5min":1.518,"15min":1.5029999999999999},"min":{"1min":0.89500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9510000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.556}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7829999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.347,"15min":1.335},"min":{"1min":0.83099999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8660000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.4179999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5669999999999999,"15min":1.55},"min":{"1min":0.85899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8759999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.512,"5min":1.454,"15min":1.454},"min":{"1min":0.81499999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8779999999999999,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.2330000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":487775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.093,"15min":1.0740000000000001},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6990000000000001,"5min":1.76,"15min":1.76},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.234},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.107}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.141,"15min":1.1699999999999999},"min":{"1min":0.753,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5349999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.877},{"interface":"front","average":{"1min":1.181,"5min":1.343,"15min":1.4410000000000001},"min":{"1min":0.76200000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.617,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":0.94499999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1830000000000001,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.639,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3939999999999999,"15min":1.4990000000000001},"min":{"1min":0.94299999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.806,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":0.81399999999999995}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.288},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.804,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3109999999999999,"15min":1.3049999999999999},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.2150000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.85399999999999998,"5min":0.755,"15min":0.755},"max":{"1min":1.635,"5min":2.347,"15min":2.347},"last":1.046},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5149999999999999,"15min":1.5529999999999999},"min":{"1min":1.0589999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.097,"5min":3.153,"15min":3.153},"last":1.2589999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4930000000000001,"15min":1.528},"min":{"1min":0.89100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.613},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.5009999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.899,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.341}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.6399999999999999,"15min":1.7},"min":{"1min":1.03,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0449999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.581,"15min":1.5900000000000001},"min":{"1min":1.1439999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.831,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.5069999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1024000,"data_stored":489441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0549999999999999,"15min":1.0469999999999999},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.496,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.635},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2549999999999999,"15min":1.2090000000000001},"min":{"1min":0.85699999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.696,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.4730000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.069,"15min":1.091},"min":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1279999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.425},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.89100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.7450000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.3879999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4610000000000001,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.784},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.423},"min":{"1min":0.82099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.524}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.696,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":1.04},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3600000000000001,"15min":1.387},"min":{"1min":0.83599999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.629,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.6839999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4399999999999999,"15min":1.5129999999999999},"min":{"1min":0.96699999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.667,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.4970000000000001,"15min":1.5489999999999999},"min":{"1min":0.95899999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8560000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.161}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5229999999999999,"15min":1.5620000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.782,"5min":4.335,"15min":4.335},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.52,"5min":1.5880000000000001,"15min":1.6259999999999999},"min":{"1min":1.3140000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8080000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.7250000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.6040000000000001,"15min":1.631},"min":{"1min":1.1799999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.77,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.331},{"interface":"front","average":{"1min":1.603,"5min":1.6859999999999999,"15min":1.732},"min":{"1min":1.127,"5min":1.095,"15min":1.095},"max":{"1min":1.901,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.845}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32180,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339660,"statfs":{"total":96636764160,"available":96603811840,"internally_reserved":0,"allocated":1019904,"data_stored":489339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.98599999999999999,"15min":1.0720000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":3.972,"15min":3.972},"last":1.028},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.032,"15min":1.095},"min":{"1min":0.372,"5min":0.311,"15min":0.311},"max":{"1min":1.899,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.399}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.103,"15min":1.1850000000000001},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2609999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1080000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.8520000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.91900000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2709999999999999,"15min":1.325},"min":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6739999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2669999999999999},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.754,"5min":2.73,"15min":2.73},"last":0.755}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3740000000000001,"15min":1.448},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.663,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.393,"15min":1.452},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":3.052,"15min":3.052},"last":1.2549999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4339999999999999,"15min":1.498},"min":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0430000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.335},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.4670000000000001,"15min":1.5309999999999999},"min":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.948,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.878,"15min":0.878},"max":{"1min":3.21,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.5269999999999999,"15min":1.585},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0880000000000001,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.456}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6140000000000001,"15min":1.71},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.948,"5min":4.407,"15min":4.407},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.528,"15min":1.581},"min":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0059999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.3180000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1003520,"data_stored":486720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.86299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.879,"15min":0.92700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.171,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.89700000000000002}]},{"osd":1,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.198,"5min":1.393,"15min":1.393},"last":1.109},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":1.016},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4690000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.79800000000000004}]},{"osd":3,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.022,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0600000000000001,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6060000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.22}]},{"osd":4,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.407},"min":{"1min":0.81499999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2969999999999999,"15min":1.365},"min":{"1min":0.81999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7,"5min":2.278,"15min":2.278},"last":1.369}]},{"osd":5,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.425},"min":{"1min":0.76600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1600000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.601},{"interface":"front","average":{"1min":1.264,"5min":1.373,"15min":1.4610000000000001},"min":{"1min":0.78500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.954,"5min":2.536,"15min":2.536},"last":1.331}]},{"osd":6,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.347,"15min":1.389},"min":{"1min":0.93100000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9119999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.036,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.4139999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4930000000000001,"15min":1.577},"min":{"1min":0.85799999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.861,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.46},{"interface":"front","average":{"1min":1.403,"5min":1.492,"15min":1.556},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.0030000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.635}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":1003520,"data_stored":487705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.745},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.304,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.745},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.123,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79900000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.863}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.65000000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.9570000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.0700000000000001,"15min":1.0860000000000001},"min":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2690000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.647}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.1539999999999999},"min":{"1min":0.78900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.556,"5min":2.165,"15min":2.165},"last":1.1819999999999999},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.252},"min":{"1min":0.77400000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.454,"5min":1.885,"15min":1.885},"last":1.25}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2549999999999999},"min":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.528},{"interface":"front","average":{"1min":1.179,"5min":1.274,"15min":1.321},"min":{"1min":0.78900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.579,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.0669999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3580000000000001,"15min":1.3660000000000001},"min":{"1min":0.84999999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1509999999999998,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.92400000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.724,"5min":2.282,"15min":2.282},"last":1.4239999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3200000000000001,"15min":1.3300000000000001},"min":{"1min":1.109,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.909,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.746},{"interface":"front","average":{"1min":1.371,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4830000000000001,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4490000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1597440,"data_stored":1077636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.75600000000000001,"15min":0.77500000000000002},"min":{"1min":0.503,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.751},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.88900000000000001,"15min":0.93300000000000005},"min":{"1min":0.55800000000000005,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.1319999999999999,"5min":1.7,"15min":1.7},"last":1.0600000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.83799999999999997,"15min":0.86899999999999999},"min":{"1min":0.502,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1479999999999999,"5min":1.508,"15min":1.508},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.872,"5min":0.92900000000000005,"15min":0.97399999999999998},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.48,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.98299999999999998,"15min":1.006},"min":{"1min":0.60599999999999998,"5min":0.44,"15min":0.44},"max":{"1min":1.335,"5min":1.748,"15min":1.748},"last":1.133},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0720000000000001,"15min":1.1100000000000001},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.694,"5min":2.496,"15min":2.496},"last":0.90900000000000003}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2410000000000001,"15min":1.2350000000000001},"min":{"1min":0.88100000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.794,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.3560000000000001,"15min":1.4370000000000001},"min":{"1min":0.875,"5min":0.75800000000000001,"15min":0.75800000000000001},"max":{"1min":1.5580000000000001,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.284}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3160000000000001,"15min":1.3700000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.8620000000000001,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.198},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3859999999999999,"15min":1.4430000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.77,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.337}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.431,"15min":1.4990000000000001},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.871,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.369,"15min":1.4099999999999999},"min":{"1min":0.90100000000000002,"5min":0.878,"15min":0.878},"max":{"1min":1.7450000000000001,"5min":1.9890000000000001,"15min":1.9890000000000001},"last":1.4199999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.379,"15min":1.4019999999999999},"min":{"1min":1.028,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":1.9299999999999999,"5min":2.173,"15min":2.173},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.4279999999999999,"15min":1.4850000000000001},"min":{"1min":0.96199999999999997,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.774,"5min":2.306,"15min":2.306},"last":1.381}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:20.746 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:28:20.747 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:28:21.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.005+0000 7f508fc75640 1 Processor -- start 2023-11-08T19:28:21.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.005+0000 7f508fc75640 1 -- start start 2023-11-08T19:28:21.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.007+0000 7f508fc75640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5088144690 0x7f508806a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.007+0000 7f508fc75640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5088162ae0 con 0x7f50880cc6d0 2023-11-08T19:28:21.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.007+0000 7f508fc75640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508806c850 con 0x7f50880ccb40 2023-11-08T19:28:21.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.007+0000 7f508fc75640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5088165de0 con 0x7f5088144690 2023-11-08T19:28:21.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508d210640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f50880cc6d0 0x7f508806ba30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50102/0 (socket says 172.21.15.119:50102) 2023-11-08T19:28:21.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508d210640 1 -- 172.21.15.119:0/2247388568 learned_addr learned my addr 172.21.15.119:0/2247388568 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:21.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508da11640 1 --2- 172.21.15.119:0/2247388568 >> v2:172.21.15.188:3300/0 conn(0x7f5088144690 0x7f508806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320670491 0 0) 0x7f5088162ae0 con 0x7f50880cc6d0 2023-11-08T19:28:21.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5074003520 con 0x7f50880cc6d0 2023-11-08T19:28:21.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 767788660 0 0) 0x7f5074003520 con 0x7f50880cc6d0 2023-11-08T19:28:21.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 >> v2:172.21.15.188:3300/0 conn(0x7f5088144690 msgr2=0x7f508806a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:21.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508ca0f640 1 --2- 172.21.15.119:0/2247388568 >> v2:172.21.15.188:3300/0 conn(0x7f5088144690 0x7f508806a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.008+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 >> v1:172.21.15.188:6789/0 conn(0x7f50880ccb40 legacy=0x7f508806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880ccfd0 con 0x7f50880cc6d0 2023-11-08T19:28:21.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f5078001670 con 0x7f50880cc6d0 2023-11-08T19:28:21.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 >> v1:172.21.15.119:6789/0 conn(0x7f50880cc6d0 legacy=0x7f508806ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074003970 0x7f5074003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50740047c0 0x7f5074004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074005180 0x7f507400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5088165de0 con 0x7f5074005180 2023-11-08T19:28:21.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507400c020 con 0x7f5074003970 2023-11-08T19:28:21.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507400c1e0 con 0x7f50740047c0 2023-11-08T19:28:21.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508e212640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074005180 0x7f507400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.009+0000 7f508da11640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074003970 0x7f5074003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.010+0000 7f508e212640 1 -- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50740047c0 msgr2=0x7f5074004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:21.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.010+0000 7f508e212640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50740047c0 0x7f5074004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.010+0000 7f508e212640 1 -- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074003970 msgr2=0x7f5074003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.010+0000 7f508e212640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074003970 0x7f5074003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.010+0000 7f508e212640 1 -- 172.21.15.119:0/2247388568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5074003520 con 0x7f5074005180 2023-11-08T19:28:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.010+0000 7f508e212640 1 -- 172.21.15.119:0/2247388568 reap_dead start 2023-11-08T19:28:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.011+0000 7f508e212640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074005180 0x7f507400b980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5080001e50 tx=0x7f50740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1f60ebedac1ffa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.011+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080003b70 con 0x7f5074005180 2023-11-08T19:28:21.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.011+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5080003d10 con 0x7f5074005180 2023-11-08T19:28:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.011+0000 7f508ca0f640 1 -- 172.21.15.119:0/2247388568 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5080005310 con 0x7f5074005180 2023-11-08T19:28:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.011+0000 7f508fc75640 1 -- 172.21.15.119:0/2247388568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074005180 msgr2=0x7f507400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.011+0000 7f508fc75640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074005180 0x7f507400b980 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f5080001e50 tx=0x7f50740057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.012+0000 7f508fc75640 1 -- 172.21.15.119:0/2247388568 shutdown_connections 2023-11-08T19:28:21.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.012+0000 7f508fc75640 1 --2- 172.21.15.119:0/2247388568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5074005180 0x7f507400b980 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.012+0000 7f508fc75640 1 -- 172.21.15.119:0/2247388568 >> 172.21.15.119:0/2247388568 conn(0x7f50880cb610 msgr2=0x7f50880cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.012+0000 7f508fc75640 1 -- 172.21.15.119:0/2247388568 shutdown_connections 2023-11-08T19:28:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.012+0000 7f508fc75640 1 -- 172.21.15.119:0/2247388568 wait complete. 2023-11-08T19:28:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.013+0000 7f508fc75640 1 Processor -- start 2023-11-08T19:28:21.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.014+0000 7f508fc75640 1 -- start start 2023-11-08T19:28:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.014+0000 7f508fc75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074005180 0x7f5088150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.014+0000 7f508fc75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 0x7f5088143c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.014+0000 7f508fc75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50880c4470 0x7f50880c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.014+0000 7f508fc75640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880c4e30 con 0x7f50880c7960 2023-11-08T19:28:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.014+0000 7f508fc75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508814e360 con 0x7f5074005180 2023-11-08T19:28:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508fc75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880c8190 con 0x7f50880c4470 2023-11-08T19:28:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 0x7f5088143c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 0x7f5088143c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35626/0 (socket says 172.21.15.119:35626) 2023-11-08T19:28:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 -- 172.21.15.119:0/950475908 learned_addr learned my addr 172.21.15.119:0/950475908 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:21.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508e212640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50880c4470 0x7f50880c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 -- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50880c4470 msgr2=0x7f50880c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50880c4470 0x7f50880c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 -- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074005180 msgr2=0x7f5088150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074005180 0x7f5088150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 -- 172.21.15.119:0/950475908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880c8470 con 0x7f50880c7960 2023-11-08T19:28:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.015+0000 7f508d210640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 0x7f5088143c60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f507800c6c0 tx=0x7f507800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.016+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780072d0 con 0x7f50880c7960 2023-11-08T19:28:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.016+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078007430 con 0x7f50880c7960 2023-11-08T19:28:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.016+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50780047e0 con 0x7f50880c7960 2023-11-08T19:28:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.016+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50880c8750 con 0x7f50880c7960 2023-11-08T19:28:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.016+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50880c7de0 con 0x7f50880c7960 2023-11-08T19:28:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.017+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508806ae90 con 0x7f50880c7960 2023-11-08T19:28:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.018+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5078004940 con 0x7f50880c7960 2023-11-08T19:28:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.019+0000 7f507e7fc640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f506c06ae70 0x7f506c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.019+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f5078089160 con 0x7f50880c7960 2023-11-08T19:28:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.020+0000 7f508da11640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f506c06ae70 0x7f506c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.020+0000 7f508da11640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f506c06ae70 0x7f506c06d330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5084009890 tx=0x7f5084008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:21.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.024+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5078053b70 con 0x7f50880c7960 2023-11-08T19:28:21.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.286+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f508806cd50 con 0x7f50880c7960 2023-11-08T19:28:21.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.290+0000 7f507e7fc640 1 -- 172.21.15.119:0/950475908 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5078018020 con 0x7f50880c7960 2023-11-08T19:28:21.292 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-11-08T19:28:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:28:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2023-11-08T19:28:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2023-11-08T19:28:21.292 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:28:21.293 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-11-08T19:28:21.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f506c06ae70 msgr2=0x7f506c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f506c06ae70 0x7f506c06d330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5084009890 tx=0x7f5084008040 comp rx=0 tx=0).stop 2023-11-08T19:28:21.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 msgr2=0x7f5088143c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 0x7f5088143c60 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f507800c6c0 tx=0x7f507800cb80 comp rx=0 tx=0).stop 2023-11-08T19:28:21.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 shutdown_connections 2023-11-08T19:28:21.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f506c06ae70 0x7f506c06d330 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50880c4470 0x7f50880c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50880c7960 0x7f5088143c60 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 --2- 172.21.15.119:0/950475908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5074005180 0x7f5088150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 >> 172.21.15.119:0/950475908 conn(0x7f50880cb610 msgr2=0x7f50880c99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 shutdown_connections 2023-11-08T19:28:21.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.293+0000 7f508fc75640 1 -- 172.21.15.119:0/950475908 wait complete. 2023-11-08T19:28:21.312 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:21.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.523+0000 7fecfa5e2640 1 Processor -- start 2023-11-08T19:28:21.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.524+0000 7fecfa5e2640 1 -- start start 2023-11-08T19:28:21.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.525+0000 7fecfa5e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 0x7fecf40cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.525+0000 7fecfa5e2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40c7100 con 0x7fecf40d1d50 2023-11-08T19:28:21.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.525+0000 7fecfa5e2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecf40d04f0 con 0x7fecf415d4c0 2023-11-08T19:28:21.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.525+0000 7fecfa5e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40d07d0 con 0x7fecf415d0e0 2023-11-08T19:28:21.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.525+0000 7fecf9de1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fecf40d1d50 0x7fecf40c7590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50112/0 (socket says 172.21.15.119:50112) 2023-11-08T19:28:21.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.525+0000 7fecf9de1640 1 -- 172.21.15.119:0/1840717468 learned_addr learned my addr 172.21.15.119:0/1840717468 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:21.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecf95e0640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 0x7fecf40cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4064983236 0 0) 0x7fecf40c7100 con 0x7fecf40d1d50 2023-11-08T19:28:21.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece0003520 con 0x7fecf40d1d50 2023-11-08T19:28:21.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4082111107 0 0) 0x7fece0003520 con 0x7fecf40d1d50 2023-11-08T19:28:21.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 msgr2=0x7fecf40cda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 0x7fecf40cda70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 >> v1:172.21.15.188:6789/0 conn(0x7fecf415d4c0 legacy=0x7fecf40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40c6ab0 con 0x7fecf40d1d50 2023-11-08T19:28:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fecec0019f0 con 0x7fecf40d1d50 2023-11-08T19:28:21.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 >> v1:172.21.15.119:6789/0 conn(0x7fecf40d1d50 legacy=0x7fecf40c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 0x7fece00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fece0003f20 0x7fece00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fece0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40d07d0 con 0x7fece0004d00 2023-11-08T19:28:21.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece000baf0 con 0x7fece0003f20 2023-11-08T19:28:21.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece000bcb0 con 0x7fecf415d0e0 2023-11-08T19:28:21.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.526+0000 7fecf9de1640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fece0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf8ddf640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fece0003f20 0x7fece00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf95e0640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 0x7fece00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf9de1640 1 -- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 msgr2=0x7fece00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:21.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf9de1640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf415d0e0 0x7fece00039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf9de1640 1 -- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fece0003f20 msgr2=0x7fece00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:21.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf9de1640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fece0003f20 0x7fece00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf9de1640 1 -- 172.21.15.119:0/1840717468 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0003520 con 0x7fece0004d00 2023-11-08T19:28:21.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.527+0000 7fecf8ddf640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fece0003f20 0x7fece00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:21.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.528+0000 7fecf9de1640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fece0005180 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fecf408c340 tx=0x7fece00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5fdf904f2218d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:21.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.528+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0072d0 con 0x7fece0004d00 2023-11-08T19:28:21.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.528+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecec02f070 con 0x7fece0004d00 2023-11-08T19:28:21.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.528+0000 7fecebfff640 1 -- 172.21.15.119:0/1840717468 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec0076f0 con 0x7fece0004d00 2023-11-08T19:28:21.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecfa5e2640 1 -- 172.21.15.119:0/1840717468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 msgr2=0x7fece0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/1840717468 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fece0005180 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fecf408c340 tx=0x7fece00057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecf9de1640 1 -- 172.21.15.119:0/1840717468 reap_dead start 2023-11-08T19:28:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecfa5e2640 1 -- 172.21.15.119:0/1840717468 shutdown_connections 2023-11-08T19:28:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecfa5e2640 1 -- 172.21.15.119:0/1840717468 >> 172.21.15.119:0/1840717468 conn(0x7fecf40d2f60 msgr2=0x7fecf40d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecfa5e2640 1 -- 172.21.15.119:0/1840717468 shutdown_connections 2023-11-08T19:28:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.529+0000 7fecfa5e2640 1 -- 172.21.15.119:0/1840717468 wait complete. 2023-11-08T19:28:21.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.531+0000 7fecfa5e2640 1 Processor -- start 2023-11-08T19:28:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.531+0000 7fecfa5e2640 1 -- start start 2023-11-08T19:28:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecfa5e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fecf415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecfa5e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecf41e7560 0x7fecf40b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecfa5e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf41e5f40 0x7fecf41e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecfa5e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf4155cf0 con 0x7fece0004d00 2023-11-08T19:28:21.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecfa5e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecf40b8bb0 con 0x7fecf41e7560 2023-11-08T19:28:21.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecfa5e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecf41e79e0 con 0x7fecf41e5f40 2023-11-08T19:28:21.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fecf415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fecf415fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35642/0 (socket says 172.21.15.119:35642) 2023-11-08T19:28:21.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 -- 172.21.15.119:0/2601353888 learned_addr learned my addr 172.21.15.119:0/2601353888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:21.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 -- 172.21.15.119:0/2601353888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf41e5f40 msgr2=0x7fecf41e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf41e5f40 0x7fecf41e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 -- 172.21.15.119:0/2601353888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecf41e7560 msgr2=0x7fecf40b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecf41e7560 0x7fecf40b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.532+0000 7fecf95e0640 1 -- 172.21.15.119:0/2601353888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40b6f40 con 0x7fece0004d00 2023-11-08T19:28:21.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.533+0000 7fecf95e0640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fecf415fd70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fecf0010860 tx=0x7fecf0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:21.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.533+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0009610 con 0x7fece0004d00 2023-11-08T19:28:21.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.533+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf000ea80 con 0x7fece0004d00 2023-11-08T19:28:21.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.533+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf000ee10 con 0x7fece0004d00 2023-11-08T19:28:21.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.533+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecf415efb0 con 0x7fece0004d00 2023-11-08T19:28:21.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.533+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf415f570 con 0x7fece0004d00 2023-11-08T19:28:21.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.534+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecf404ece0 con 0x7fece0004d00 2023-11-08T19:28:21.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.535+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fecf001c400 con 0x7fece0004d00 2023-11-08T19:28:21.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.536+0000 7fece9ffb640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fecd806aea0 0x7fecd806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:21.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.536+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fecf008a010 con 0x7fece0004d00 2023-11-08T19:28:21.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.536+0000 7fecf8ddf640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fecd806aea0 0x7fecd806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:21.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.537+0000 7fecf8ddf640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fecd806aea0 0x7fecd806d360 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fecf415d9c0 tx=0x7fece4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:21.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.540+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fecf0054b50 con 0x7fece0004d00 2023-11-08T19:28:21.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.768+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecf408d200 con 0x7fece0004d00 2023-11-08T19:28:21.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.769+0000 7fece9ffb640 1 -- 172.21.15.119:0/2601353888 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fecf0058170 con 0x7fece0004d00 2023-11-08T19:28:21.771 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:21.771 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:28:15.191469+0000","modified":"2023-11-08T19:28:18.233986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/4016780496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4016780496},{"type":"v1","addr":"172.21.15.119:6837","nonce":4016780496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:28:21.772 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 44 2023-11-08T19:28:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fecd806aea0 msgr2=0x7fecd806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fecd806aea0 0x7fecd806d360 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fecf415d9c0 tx=0x7fece4009040 comp rx=0 tx=0).stop 2023-11-08T19:28:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 msgr2=0x7fecf415fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fecf415fd70 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fecf0010860 tx=0x7fecf0010d20 comp rx=0 tx=0).stop 2023-11-08T19:28:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 shutdown_connections 2023-11-08T19:28:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fecd806aea0 0x7fecd806d360 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fecf41e5f40 0x7fecf41e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fecf41e7560 0x7fecf40b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 --2- 172.21.15.119:0/2601353888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fece0004d00 0x7fecf415fd70 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.772+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 >> 172.21.15.119:0/2601353888 conn(0x7fecf40d2f60 msgr2=0x7fecf40c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.773+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 shutdown_connections 2023-11-08T19:28:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:21.773+0000 7fecfa5e2640 1 -- 172.21.15.119:0/2601353888 wait complete. 2023-11-08T19:28:22.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:28:15.191469+0000', 'modified': '2023-11-08T19:28:18.233986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6624}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6624': {'gid': 6624, 'name': 'c', 'rank': 0, 'incarnation': 40, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.119:6837/4016780496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 4016780496}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 4016780496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:28:22.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:28:22.198 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T19:28:22.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.420+0000 7efe51355640 1 Processor -- start 2023-11-08T19:28:22.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.420+0000 7efe51355640 1 -- start start 2023-11-08T19:28:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.421+0000 7efe51355640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 0x7efe4c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe51355640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c15f900 con 0x7efe4c06a120 2023-11-08T19:28:22.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe51355640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe4c0c3a50 con 0x7efe4c1447a0 2023-11-08T19:28:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe51355640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c144b80 con 0x7efe4c15f4c0 2023-11-08T19:28:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe4b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7efe4c06a120 0x7efe4c0c4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50114/0 (socket says 172.21.15.119:50114) 2023-11-08T19:28:22.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe4b7fe640 1 -- 172.21.15.119:0/3536664802 learned_addr learned my addr 172.21.15.119:0/3536664802 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe427fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 0x7efe4c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.422+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531892161 0 0) 0x7efe44001490 con 0x7efe4c06a120 2023-11-08T19:28:22.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe2c003520 con 0x7efe4c06a120 2023-11-08T19:28:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2833251566 0 0) 0x7efe2c003520 con 0x7efe4c06a120 2023-11-08T19:28:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 msgr2=0x7efe4c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 0x7efe4c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 >> v1:172.21.15.188:6789/0 conn(0x7efe4c1447a0 legacy=0x7efe4c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0c9150 con 0x7efe4c06a120 2023-11-08T19:28:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7efe44001b80 con 0x7efe4c06a120 2023-11-08T19:28:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 >> v1:172.21.15.119:6789/0 conn(0x7efe4c06a120 legacy=0x7efe4c0c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 0x7efe2c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe2c003f70 0x7efe2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004d00 0x7efe2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c144b80 con 0x7efe2c003f70 2023-11-08T19:28:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00bac0 con 0x7efe2c004d00 2023-11-08T19:28:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c00bc80 con 0x7efe4c15f4c0 2023-11-08T19:28:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe427fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe2c003f70 0x7efe2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.423+0000 7efe4affd640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 0x7efe2c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.424+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004d00 0x7efe2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.424+0000 7efe427fc640 1 -- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 msgr2=0x7efe2c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.424+0000 7efe427fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c15f4c0 0x7efe2c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.424+0000 7efe427fc640 1 -- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004d00 msgr2=0x7efe2c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.424+0000 7efe427fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c004d00 0x7efe2c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.424+0000 7efe427fc640 1 -- 172.21.15.119:0/3536664802 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c003520 con 0x7efe2c003f70 2023-11-08T19:28:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.425+0000 7efe427fc640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe2c003f70 0x7efe2c0047c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efe34009ac0 tx=0x7efe2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bccd05c97d5991e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.425+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3402c600 con 0x7efe2c003f70 2023-11-08T19:28:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.425+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3402c760 con 0x7efe2c003f70 2023-11-08T19:28:22.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.425+0000 7efe4a7fc640 1 -- 172.21.15.119:0/3536664802 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3402caf0 con 0x7efe2c003f70 2023-11-08T19:28:22.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe51355640 1 -- 172.21.15.119:0/3536664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe2c003f70 msgr2=0x7efe2c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe51355640 1 --2- 172.21.15.119:0/3536664802 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe2c003f70 0x7efe2c0047c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7efe34009ac0 tx=0x7efe2c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe4b7fe640 1 -- 172.21.15.119:0/3536664802 reap_dead start 2023-11-08T19:28:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe51355640 1 -- 172.21.15.119:0/3536664802 shutdown_connections 2023-11-08T19:28:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe51355640 1 -- 172.21.15.119:0/3536664802 >> 172.21.15.119:0/3536664802 conn(0x7efe4c0cb610 msgr2=0x7efe4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe51355640 1 -- 172.21.15.119:0/3536664802 shutdown_connections 2023-11-08T19:28:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.426+0000 7efe51355640 1 -- 172.21.15.119:0/3536664802 wait complete. 2023-11-08T19:28:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.427+0000 7efe51355640 1 Processor -- start 2023-11-08T19:28:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.428+0000 7efe51355640 1 -- start start 2023-11-08T19:28:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.428+0000 7efe51355640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003f70 0x7efe4c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.428+0000 7efe51355640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 0x7efe4c134410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe51355640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c142aa0 0x7efe4c142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe51355640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0c7080 con 0x7efe4c0b41d0 2023-11-08T19:28:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe51355640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c143490 con 0x7efe2c003f70 2023-11-08T19:28:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe51355640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4c0c74a0 con 0x7efe4c142aa0 2023-11-08T19:28:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 0x7efe4c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 0x7efe4c134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35664/0 (socket says 172.21.15.119:35664) 2023-11-08T19:28:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 -- 172.21.15.119:0/1657601375 learned_addr learned my addr 172.21.15.119:0/1657601375 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c142aa0 0x7efe4c142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 -- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c142aa0 msgr2=0x7efe4c142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe4affd640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003f70 0x7efe4c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c142aa0 0x7efe4c142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 -- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003f70 msgr2=0x7efe4c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003f70 0x7efe4c0c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe427fc640 1 -- 172.21.15.119:0/1657601375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0c6400 con 0x7efe4c0b41d0 2023-11-08T19:28:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.429+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c142aa0 0x7efe4c142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:22.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.430+0000 7efe427fc640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 0x7efe4c134410 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7efe34009ac0 tx=0x7efe34004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.430+0000 7efe43fff640 1 -- 172.21.15.119:0/1657601375 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34003810 con 0x7efe4c0b41d0 2023-11-08T19:28:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.430+0000 7efe43fff640 1 -- 172.21.15.119:0/1657601375 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe34003970 con 0x7efe4c0b41d0 2023-11-08T19:28:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.430+0000 7efe43fff640 1 -- 172.21.15.119:0/1657601375 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34003d00 con 0x7efe4c0b41d0 2023-11-08T19:28:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.430+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe4c0c66e0 con 0x7efe4c0b41d0 2023-11-08T19:28:22.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.430+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0c6ca0 con 0x7efe4c0b41d0 2023-11-08T19:28:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.432+0000 7efe43fff640 1 -- 172.21.15.119:0/1657601375 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe34040450 con 0x7efe4c0b41d0 2023-11-08T19:28:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.433+0000 7efe43fff640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.433+0000 7efe43fff640 1 -- 172.21.15.119:0/1657601375 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7efe340abab0 con 0x7efe4c0b41d0 2023-11-08T19:28:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.434+0000 7efe4affd640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.434+0000 7efe53fc0740 1 Processor -- start 2023-11-08T19:28:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.434+0000 7efe53fc0740 1 -- start start 2023-11-08T19:28:22.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.434+0000 7efe4affd640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efe3c007fd0 tx=0x7efe3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123922dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239502a0 0x556123952c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 0x556123953190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe53fc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561238957e0 con 0x556123923300 2023-11-08T19:28:22.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556123892ed0 con 0x55612394fec0 2023-11-08T19:28:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55612388e7c0 con 0x5561239502a0 2023-11-08T19:28:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 0x556123953190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 0x556123953190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35666/0 (socket says 172.21.15.119:35666) 2023-11-08T19:28:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.435+0000 7efe4b7fe640 1 -- 172.21.15.119:0/3826971028 learned_addr learned my addr 172.21.15.119:0/3826971028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4affd640 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123922dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe427fc640 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239502a0 0x556123952c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4b7fe640 1 -- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239502a0 msgr2=0x556123952c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239502a0 0x556123952c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4b7fe640 1 -- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 msgr2=0x556123922dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123922dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4b7fe640 1 -- 172.21.15.119:0/3826971028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556123861b10 con 0x556123923300 2023-11-08T19:28:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 0x556123953190 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0ce900 tx=0x7efe4400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c775187a93631ffe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe40ff9640 1 -- 172.21.15.119:0/3826971028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44004d60 con 0x556123923300 2023-11-08T19:28:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe40ff9640 1 -- 172.21.15.119:0/3826971028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4400d040 con 0x556123923300 2023-11-08T19:28:22.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.436+0000 7efe40ff9640 1 -- 172.21.15.119:0/3826971028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe440072b0 con 0x556123923300 2023-11-08T19:28:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 -- 172.21.15.119:0/3826971028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 msgr2=0x556123953190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 0x556123953190 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0ce900 tx=0x7efe4400e590 comp rx=0 tx=0).stop 2023-11-08T19:28:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 -- 172.21.15.119:0/3826971028 shutdown_connections 2023-11-08T19:28:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x556123923300 0x556123953190 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239502a0 0x556123952c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3826971028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123922dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 -- 172.21.15.119:0/3826971028 >> 172.21.15.119:0/3826971028 conn(0x5561239250d0 msgr2=0x55612396dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 -- 172.21.15.119:0/3826971028 shutdown_connections 2023-11-08T19:28:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.437+0000 7efe53fc0740 1 -- 172.21.15.119:0/3826971028 wait complete. 2023-11-08T19:28:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.438+0000 7efe53fc0740 1 -- start start 2023-11-08T19:28:22.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.438+0000 7efe53fc0740 1 Processor -- start 2023-11-08T19:28:22.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55612394fec0 0x556123944180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5561239502a0 0x556123955e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 0x5561239672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55612382bc40 con 0x5561239563a0 2023-11-08T19:28:22.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55612382b8e0 con 0x5561239502a0 2023-11-08T19:28:22.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561235f1f20 con 0x55612394fec0 2023-11-08T19:28:22.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe53fc0740 10 client.0 _ll_register_callbacks cb 0x55612384dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:22.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 0x5561239672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe4b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 0x5561239672a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35672/0 (socket says 172.21.15.119:35672) 2023-11-08T19:28:22.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.439+0000 7efe4b7fe640 1 -- 172.21.15.119:0/1332677334 learned_addr learned my addr 172.21.15.119:0/1332677334 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4affd640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55612394fec0 0x556123944180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe427fc640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5561239502a0 0x556123955e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4b7fe640 1 -- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55612394fec0 msgr2=0x556123944180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55612394fec0 0x556123944180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4b7fe640 1 -- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5561239502a0 msgr2=0x556123955e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5561239502a0 0x556123955e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4b7fe640 1 -- 172.21.15.119:0/1332677334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0c9150 con 0x5561239563a0 2023-11-08T19:28:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 0x5561239672a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efe44007c40 tx=0x7efe44001e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4336cdd9f92f5a8f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:28:22.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe39ffb640 1 -- 172.21.15.119:0/1332677334 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44007830 con 0x5561239563a0 2023-11-08T19:28:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe39ffb640 1 -- 172.21.15.119:0/1332677334 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7efe4400ee80 con 0x5561239563a0 2023-11-08T19:28:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.440+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556123862c80 con 0x5561239563a0 2023-11-08T19:28:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe39ffb640 1 -- 172.21.15.119:0/1332677334 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe44001ac0 con 0x5561239563a0 2023-11-08T19:28:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 10 client.6792 fetch_fsmap learned FSMap version 44 2023-11-08T19:28:22.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55612388ce40 con 0x5561239563a0 2023-11-08T19:28:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe39ffb640 1 -- 172.21.15.119:0/1332677334 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556123862c80 con 0x5561239563a0 2023-11-08T19:28:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe39ffb640 1 -- 172.21.15.119:0/1332677334 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe44013460 con 0x5561239563a0 2023-11-08T19:28:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 10 client.6792 fetch_fsmap finished waiting for FSMap version 44 2023-11-08T19:28:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 10 client.6792 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:28:22.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 20 client.6792 populate_metadata read hostname 'smithi119' 2023-11-08T19:28:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x556123954c90 0x556123955070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 4 client.6792 mds_command: new command op to 6624 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:28:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 --> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561238877a0 con 0x556123954c90 2023-11-08T19:28:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.441+0000 7efe4affd640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x556123954c90 0x556123955070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.442+0000 7efe4affd640 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x556123954c90 0x556123955070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30387522d445872 server_cookie=d73d60b5a03e423c in_seq=0 out_seq=0 2023-11-08T19:28:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.442+0000 7efe39ffb640 10 client.6792 ms_handle_connect on v2:172.21.15.119:6835/4016780496 2023-11-08T19:28:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe39ffb640 1 -- 172.21.15.119:0/1332677334 <== mds.0 v2:172.21.15.119:6835/4016780496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561238877a0 con 0x556123954c90 2023-11-08T19:28:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe39ffb640 10 client.6792 handle_command_reply: tid=0 2023-11-08T19:28:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 client.6792 shutdown 2023-11-08T19:28:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 msgr2=0x5561239672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 0x5561239672a0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7efe44007c40 tx=0x7efe44001e40 comp rx=0 tx=0).stop 2023-11-08T19:28:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 shutdown_connections 2023-11-08T19:28:22.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x556123954c90 0x556123955070 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239563a0 0x5561239672a0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x5561239502a0 0x556123955e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe39ffb640 0 client.6792 ms_handle_reset on v2:172.21.15.119:6835/4016780496 2023-11-08T19:28:22.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 --2- 172.21.15.119:0/1332677334 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55612394fec0 0x556123944180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 >> 172.21.15.119:0/1332677334 conn(0x556123923300 msgr2=0x556123926c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 shutdown_connections 2023-11-08T19:28:22.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.443+0000 7efe53fc0740 1 -- 172.21.15.119:0/1332677334 wait complete. 2023-11-08T19:28:22.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.444+0000 7efe53fc0740 20 client.6792 trim_cache size 0 max 16384 2023-11-08T19:28:22.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.460+0000 7efe53fc0740 1 Processor -- start 2023-11-08T19:28:22.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.460+0000 7efe53fc0740 1 -- start start 2023-11-08T19:28:22.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123990430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123955070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123966c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe53fc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561239291a0 con 0x5561239502a0 2023-11-08T19:28:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x556123843da0 con 0x55612394fec0 2023-11-08T19:28:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55612396ed30 con 0x5561239563a0 2023-11-08T19:28:22.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe427fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123955070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe427fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123955070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35680/0 (socket says 172.21.15.119:35680) 2023-11-08T19:28:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.461+0000 7efe427fc640 1 -- 172.21.15.119:0/3256054923 learned_addr learned my addr 172.21.15.119:0/3256054923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe4affd640 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123990430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe4b7fe640 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123966c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe427fc640 1 -- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 msgr2=0x556123966c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe427fc640 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123966c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe427fc640 1 -- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 msgr2=0x556123990430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe427fc640 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123990430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe427fc640 1 -- 172.21.15.119:0/3256054923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556123970d60 con 0x5561239502a0 2023-11-08T19:28:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe427fc640 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123955070 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7efe340a9a60 tx=0x7efe34009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749b62053814e5f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe417fa640 1 -- 172.21.15.119:0/3256054923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34080040 con 0x5561239502a0 2023-11-08T19:28:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe417fa640 1 -- 172.21.15.119:0/3256054923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3402de20 con 0x5561239502a0 2023-11-08T19:28:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.462+0000 7efe417fa640 1 -- 172.21.15.119:0/3256054923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3407a9a0 con 0x5561239502a0 2023-11-08T19:28:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 -- 172.21.15.119:0/3256054923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 msgr2=0x556123955070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123955070 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7efe340a9a60 tx=0x7efe34009190 comp rx=0 tx=0).stop 2023-11-08T19:28:22.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 -- 172.21.15.119:0/3256054923 shutdown_connections 2023-11-08T19:28:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123966c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123955070 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3256054923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123990430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 -- 172.21.15.119:0/3256054923 >> 172.21.15.119:0/3256054923 conn(0x556123923300 msgr2=0x556123956c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 -- 172.21.15.119:0/3256054923 shutdown_connections 2023-11-08T19:28:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.463+0000 7efe53fc0740 1 -- 172.21.15.119:0/3256054923 wait complete. 2023-11-08T19:28:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.464+0000 7efe53fc0740 1 -- start start 2023-11-08T19:28:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.464+0000 7efe53fc0740 1 Processor -- start 2023-11-08T19:28:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123952750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123952c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123927b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x556123970740 con 0x5561239502a0 2023-11-08T19:28:22.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55612394ec60 con 0x55612394fec0 2023-11-08T19:28:22.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55612396dc80 con 0x5561239563a0 2023-11-08T19:28:22.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe53fc0740 10 client.0 _ll_register_callbacks cb 0x7efe44016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:28:22.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe427fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123952c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe427fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123952c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35694/0 (socket says 172.21.15.119:35694) 2023-11-08T19:28:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.465+0000 7efe427fc640 1 -- 172.21.15.119:0/3960087139 learned_addr learned my addr 172.21.15.119:0/3960087139 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe427fc640 1 -- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 msgr2=0x556123927b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe427fc640 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123927b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe427fc640 1 -- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 msgr2=0x556123952750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe4affd640 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123952750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe427fc640 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123952750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe427fc640 1 -- 172.21.15.119:0/3960087139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe4c0c6ca0 con 0x5561239502a0 2023-11-08T19:28:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe427fc640 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123952c90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efe340767a0 tx=0x7efe34000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72627c89763419d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:28:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe39ffb640 1 -- 172.21.15.119:0/3960087139 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34080040 con 0x5561239502a0 2023-11-08T19:28:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe39ffb640 1 -- 172.21.15.119:0/3960087139 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7efe3407e3f0 con 0x5561239502a0 2023-11-08T19:28:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe4400d040 con 0x5561239502a0 2023-11-08T19:28:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe39ffb640 1 -- 172.21.15.119:0/3960087139 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe340825a0 con 0x5561239502a0 2023-11-08T19:28:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe53fc0740 10 client.6798 fetch_fsmap learned FSMap version 44 2023-11-08T19:28:22.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.466+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=2+}) v3 -- 0x55612382ab90 con 0x5561239502a0 2023-11-08T19:28:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe39ffb640 1 -- 172.21.15.119:0/3960087139 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4400d040 con 0x5561239502a0 2023-11-08T19:28:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe39ffb640 1 -- 172.21.15.119:0/3960087139 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 44) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efe3407a500 con 0x5561239502a0 2023-11-08T19:28:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe53fc0740 10 client.6798 fetch_fsmap finished waiting for FSMap version 44 2023-11-08T19:28:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe53fc0740 10 client.6798 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T19:28:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe53fc0740 20 client.6798 populate_metadata read hostname 'smithi119' 2023-11-08T19:28:22.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x5561239280a0 0x556123928480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe53fc0740 4 client.6798 mds_command: new command op to 6624 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:28:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 --> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5561238874c0 con 0x5561239280a0 2023-11-08T19:28:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe4affd640 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x5561239280a0 0x556123928480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe4affd640 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x5561239280a0 0x556123928480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db299b82a49e6e2a server_cookie=e7f8b104507bf73c in_seq=0 out_seq=0 2023-11-08T19:28:22.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.467+0000 7efe39ffb640 10 client.6798 ms_handle_connect on v2:172.21.15.119:6835/4016780496 2023-11-08T19:28:22.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe39ffb640 1 -- 172.21.15.119:0/3960087139 <== mds.0 v2:172.21.15.119:6835/4016780496 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5561238874c0 con 0x5561239280a0 2023-11-08T19:28:22.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe39ffb640 10 client.6798 handle_command_reply: tid=0 2023-11-08T19:28:22.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 client.6798 shutdown 2023-11-08T19:28:22.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 msgr2=0x556123952c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123952c90 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7efe340767a0 tx=0x7efe34000bc0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 shutdown_connections 2023-11-08T19:28:22.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:6835/4016780496,v1:172.21.15.119:6837/4016780496] conn(0x5561239280a0 0x556123928480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x5561239563a0 0x556123927b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x5561239502a0 0x556123952c90 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 --2- 172.21.15.119:0/3960087139 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55612394fec0 0x556123952750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe39ffb640 0 client.6798 ms_handle_reset on v2:172.21.15.119:6835/4016780496 2023-11-08T19:28:22.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 >> 172.21.15.119:0/3960087139 conn(0x556123923300 msgr2=0x556123964bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 shutdown_connections 2023-11-08T19:28:22.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 1 -- 172.21.15.119:0/3960087139 wait complete. 2023-11-08T19:28:22.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.468+0000 7efe53fc0740 20 client.6798 trim_cache size 0 max 16384 2023-11-08T19:28:22.489 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:28:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:28:22.489 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:28:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 6624, 2023-11-08T19:28:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:28:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:28:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:28:22.490 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 6.7458922069999998, 2023-11-08T19:28:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 44, 2023-11-08T19:28:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 83, 2023-11-08T19:28:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 83, 2023-11-08T19:28:22.491 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 13.554875280999999 2023-11-08T19:28:22.491 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:28:22.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 msgr2=0x7efe3006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7efe3c007fd0 tx=0x7efe3c009040 comp rx=0 tx=0).stop 2023-11-08T19:28:22.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 msgr2=0x7efe4c134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 0x7efe4c134410 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7efe34009ac0 tx=0x7efe34004730 comp rx=0 tx=0).stop 2023-11-08T19:28:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 shutdown_connections 2023-11-08T19:28:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe4c142aa0 0x7efe4c142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe4c0b41d0 0x7efe4c134410 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 --2- 172.21.15.119:0/1657601375 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe2c003f70 0x7efe4c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 >> 172.21.15.119:0/1657601375 conn(0x7efe4c0cb610 msgr2=0x7efe4c15bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 shutdown_connections 2023-11-08T19:28:22.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.469+0000 7efe51355640 1 -- 172.21.15.119:0/1657601375 wait complete. 2023-11-08T19:28:22.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:28:22.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.693+0000 7fe6cffff640 1 Processor -- start 2023-11-08T19:28:22.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.693+0000 7fe6cffff640 1 -- start start 2023-11-08T19:28:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6cffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6d015b0d0 0x7fe6d00ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d015b900 0x7fe6d0070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6cffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6d00710a0 0x7fe6d015da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6cffff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d006cff0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6cffff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6d006d2d0 con 0x7fe6d00710a0 2023-11-08T19:28:22.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6cffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d006bd20 con 0x7fe6d015b900 2023-11-08T19:28:22.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6ceffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6d015b0d0 0x7fe6d00ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.695+0000 7fe6ceffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe6d015b0d0 0x7fe6d00ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35702/0 (socket says 172.21.15.119:35702) 2023-11-08T19:28:22.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6ceffd640 1 -- 172.21.15.119:0/3794920526 learned_addr learned my addr 172.21.15.119:0/3794920526 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6c7fff640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d015b900 0x7fe6d0070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6cf7fe640 1 --2- 172.21.15.119:0/3794920526 >> v2:172.21.15.188:3300/0 conn(0x7fe6d00710a0 0x7fe6d015da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6ceffd640 1 -- 172.21.15.119:0/3794920526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d015b900 msgr2=0x7fe6d0070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6ceffd640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d015b900 0x7fe6d0070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6ceffd640 1 -- 172.21.15.119:0/3794920526 >> v2:172.21.15.188:3300/0 conn(0x7fe6d00710a0 msgr2=0x7fe6d015da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:22.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6ceffd640 1 --2- 172.21.15.119:0/3794920526 >> v2:172.21.15.188:3300/0 conn(0x7fe6d00710a0 0x7fe6d015da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6ceffd640 1 -- 172.21.15.119:0/3794920526 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d006d760 con 0x7fe6d015b0d0 2023-11-08T19:28:22.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.696+0000 7fe6c7fff640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d015b900 0x7fe6d0070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:22.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.697+0000 7fe6ceffd640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00ad7d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0066cb0 tx=0x7fe6c002e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b99d85a1447c58 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6ce7fc640 1 -- 172.21.15.119:0/3794920526 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c002edf0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6ce7fc640 1 -- 172.21.15.119:0/3794920526 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6c0036440 con 0x7fe6d015b0d0 2023-11-08T19:28:22.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6ce7fc640 1 -- 172.21.15.119:0/3794920526 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6c00367d0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 -- 172.21.15.119:0/3794920526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 msgr2=0x7fe6d00ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00ad7d0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0066cb0 tx=0x7fe6c002e040 comp rx=0 tx=0).stop 2023-11-08T19:28:22.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 -- 172.21.15.119:0/3794920526 shutdown_connections 2023-11-08T19:28:22.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3794920526 >> v2:172.21.15.188:3300/0 conn(0x7fe6d00710a0 0x7fe6d015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.703 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d015b900 0x7fe6d0070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3794920526 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00ad7d0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.698+0000 7fe6cffff640 1 -- 172.21.15.119:0/3794920526 >> 172.21.15.119:0/3794920526 conn(0x7fe6d00b0560 msgr2=0x7fe6d00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.699+0000 7fe6cffff640 1 -- 172.21.15.119:0/3794920526 shutdown_connections 2023-11-08T19:28:22.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.699+0000 7fe6cffff640 1 -- 172.21.15.119:0/3794920526 wait complete. 2023-11-08T19:28:22.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.700+0000 7fe6cffff640 1 Processor -- start 2023-11-08T19:28:22.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.701+0000 7fe6cffff640 1 -- start start 2023-11-08T19:28:22.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.701+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d00710a0 0x7fe6d01f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.701+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.701+0000 7fe6cffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d015b900 0x7fe6d00ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6cffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00c77c0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6cffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00b4000 con 0x7fe6d015b900 2023-11-08T19:28:22.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6cffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d00b2c70 con 0x7fe6d00710a0 2023-11-08T19:28:22.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00c25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35706/0 (socket says 172.21.15.119:35706) 2023-11-08T19:28:22.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 -- 172.21.15.119:0/3094773959 learned_addr learned my addr 172.21.15.119:0/3094773959 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:22.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6ceffd640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d00710a0 0x7fe6d01f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6cf7fe640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d015b900 0x7fe6d00ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 -- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d00710a0 msgr2=0x7fe6d01f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d00710a0 0x7fe6d01f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 -- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d015b900 msgr2=0x7fe6d00ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d015b900 0x7fe6d00ae5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.702+0000 7fe6c7fff640 1 -- 172.21.15.119:0/3094773959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c000b770 con 0x7fe6d015b0d0 2023-11-08T19:28:22.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.703+0000 7fe6c7fff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00c25a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e7e0 tx=0x7fe6b800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.703+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004030 con 0x7fe6d015b0d0 2023-11-08T19:28:22.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.703+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b8004190 con 0x7fe6d015b0d0 2023-11-08T19:28:22.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.703+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6d00c7100 con 0x7fe6d015b0d0 2023-11-08T19:28:22.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.703+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b8004520 con 0x7fe6d015b0d0 2023-11-08T19:28:22.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.703+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6d00b2f50 con 0x7fe6d015b0d0 2023-11-08T19:28:22.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.704+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6d00c7900 con 0x7fe6d015b0d0 2023-11-08T19:28:22.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.705+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6b800c6d0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.706+0000 7fe6ccff9640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6b006ae70 0x7fe6b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:22.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.706+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fe6b8089890 con 0x7fe6d015b0d0 2023-11-08T19:28:22.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.706+0000 7fe6ceffd640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6b006ae70 0x7fe6b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:22.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.707+0000 7fe6ceffd640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6b006ae70 0x7fe6b006d330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0038000 tx=0x7fe6c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:22.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.710+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6b80543b0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.950+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe6d004ece0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.951+0000 7fe6ccff9640 1 -- 172.21.15.119:0/3094773959 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v44) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe6b80579d0 con 0x7fe6d015b0d0 2023-11-08T19:28:22.952 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:28:22.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6b006ae70 msgr2=0x7fe6b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6b006ae70 0x7fe6b006d330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0038000 tx=0x7fe6c0008040 comp rx=0 tx=0).stop 2023-11-08T19:28:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 msgr2=0x7fe6d00c25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00c25a0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fe6b800e7e0 tx=0x7fe6b800eca0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 shutdown_connections 2023-11-08T19:28:22.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6b006ae70 0x7fe6b006d330 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6d015b900 0x7fe6d00ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe6d015b0d0 0x7fe6d00c25a0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 --2- 172.21.15.119:0/3094773959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6d00710a0 0x7fe6d01f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:22.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 >> 172.21.15.119:0/3094773959 conn(0x7fe6d00b0560 msgr2=0x7fe6d015a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:22.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 shutdown_connections 2023-11-08T19:28:22.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:22.953+0000 7fe6cffff640 1 -- 172.21.15.119:0/3094773959 wait complete. 2023-11-08T19:28:22.973 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:28:22.974 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:28:22.974 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:28:22.974 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:28:22.975 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:22.975 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:28:22.975 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:28:22.976 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:28:22.976 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:28:22.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:22.977 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:28:23.029 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:28:23.029 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:28:23.030 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:28:23.030 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.030 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:28:23.030 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.031 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:28:23.031 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:23.031 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:28:23.031 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:28:23.031 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:23.032 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:28:23.032 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:28:23.032 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:23.032 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:28:23.033 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:23.033 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:28:23.033 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42167sec preferred_lft 42167sec 2023-11-08T19:28:23.033 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:28:23.033 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.034 INFO:teuthology.orchestra.run.smithi119.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:28:23.034 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:28:23.034 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:28:23.034 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.034 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::e06c:64ff:fe55:3f8b/64 scope link 2023-11-08T19:28:23.035 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.035 INFO:teuthology.orchestra.run.smithi119.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:28:23.035 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:28:23.035 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c086:39ff:fe1e:9142/64 scope link 2023-11-08T19:28:23.036 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.036 INFO:teuthology.orchestra.run.smithi119.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:28:23.036 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:28:23.036 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::494:cbff:fed6:d560/64 scope link 2023-11-08T19:28:23.036 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:28:23.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.037 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:28:23.126 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:28:23.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.131 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:28:23.188 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:28:23.188 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:28:23.188 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:28:23.188 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:28:23.189 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:28:23.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.189 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:28:23.190 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:28:23.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:28:23.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:28:23.191 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:28:23.191 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:28:23.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.373 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:28:23.424 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:28:23.424 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:28:23.425 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:28:23.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:28:23.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:28:23.517 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:23.573 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:28:23.573 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:28:23.574 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:23.625 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T19:28:23.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:28:23.761 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:28:23.762 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:28:23.798 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:28:23.799 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:28:23.799 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:28:23.799 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:28:23.799 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:28:23.801 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-08T19:28:23.801 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:28:23.802 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:28:23.802 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:28:23.802 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:28:23.807 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:28:23.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.807 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:23.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:23.889 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:28:23.947 INFO:teuthology.orchestra.run.smithi119.stdout:client4973 2023-11-08T19:28:23.947 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:23.948 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:23.999 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2023-11-08T19:28:24.222 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_max_snaps_per_dir":"100"} 2023-11-08T19:28:24.222 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2023-11-08T19:28:24.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:24.223 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps) 2023-11-08T19:28:24.283 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps false --yes-i-really-mean-it 2023-11-08T19:28:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.501+0000 7f9414ad7640 1 Processor -- start 2023-11-08T19:28:24.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.502+0000 7f9414ad7640 1 -- start start 2023-11-08T19:28:24.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f9414ad7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f94100cfb80 0x7f94100cff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f9414ad7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100d04a0 0x7f94101602f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f9414ad7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100d16e0 con 0x7f94100b3de0 2023-11-08T19:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f9414ad7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94100d0990 con 0x7f94100cfb80 2023-11-08T19:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f9414ad7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100d1980 con 0x7f94100d04a0 2023-11-08T19:28:24.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f940ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f94100b3de0 0x7f94100cda50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50120/0 (socket says 172.21.15.119:50120) 2023-11-08T19:28:24.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f940ffff640 1 -- 172.21.15.119:0/372944132 learned_addr learned my addr 172.21.15.119:0/372944132 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:24.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f940f7fe640 1 --2- 172.21.15.119:0/372944132 >> v2:172.21.15.119:3300/0 conn(0x7f94100cfb80 0x7f94100cff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:24.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.503+0000 7f940effd640 1 --2- 172.21.15.119:0/372944132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100d04a0 0x7f94101602f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.504+0000 7f940f7fe640 1 -- 172.21.15.119:0/372944132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100d04a0 msgr2=0x7f94101602f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.504+0000 7f940f7fe640 1 --2- 172.21.15.119:0/372944132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100d04a0 0x7f94101602f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.504+0000 7f940f7fe640 1 -- 172.21.15.119:0/372944132 >> v1:172.21.15.119:6789/0 conn(0x7f94100b3de0 legacy=0x7f94100cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.504+0000 7f940e7fc640 1 -- 172.21.15.119:0/372944132 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461547904 0 0) 0x7f94100d16e0 con 0x7f94100b3de0 2023-11-08T19:28:24.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.504+0000 7f940f7fe640 1 -- 172.21.15.119:0/372944132 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100d1c60 con 0x7f94100cfb80 2023-11-08T19:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.505+0000 7f940f7fe640 1 --2- 172.21.15.119:0/372944132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100cfb80 0x7f94100cff60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9400009d90 tx=0x7f940002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d69d685495ec353 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.505+0000 7f940e7fc640 1 -- 172.21.15.119:0/372944132 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940002cc50 con 0x7f94100cfb80 2023-11-08T19:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.505+0000 7f940e7fc640 1 -- 172.21.15.119:0/372944132 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940002cdb0 con 0x7f94100cfb80 2023-11-08T19:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.505+0000 7f940e7fc640 1 -- 172.21.15.119:0/372944132 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400034660 con 0x7f94100cfb80 2023-11-08T19:28:24.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 -- 172.21.15.119:0/372944132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100cfb80 msgr2=0x7f94100cff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 --2- 172.21.15.119:0/372944132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100cfb80 0x7f94100cff60 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f9400009d90 tx=0x7f940002bea0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 -- 172.21.15.119:0/372944132 shutdown_connections 2023-11-08T19:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 --2- 172.21.15.119:0/372944132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100d04a0 0x7f94101602f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 --2- 172.21.15.119:0/372944132 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100cfb80 0x7f94100cff60 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 -- 172.21.15.119:0/372944132 >> 172.21.15.119:0/372944132 conn(0x7f94100d2f60 msgr2=0x7f94100d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:24.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 -- 172.21.15.119:0/372944132 shutdown_connections 2023-11-08T19:28:24.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.506+0000 7f9414ad7640 1 -- 172.21.15.119:0/372944132 wait complete. 2023-11-08T19:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.508+0000 7f9414ad7640 1 Processor -- start 2023-11-08T19:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.508+0000 7f9414ad7640 1 -- start start 2023-11-08T19:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f9414ad7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 0x7f941014c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:24.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f9414ad7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100cfb80 0x7f9410159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f9414ad7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94100d04a0 0x7f941015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f9414ad7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100c9630 con 0x7f94100b3de0 2023-11-08T19:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f9414ad7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100ca980 con 0x7f94100d04a0 2023-11-08T19:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f9414ad7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941014c940 con 0x7f94100cfb80 2023-11-08T19:28:24.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 0x7f941014c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 0x7f941014c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35726/0 (socket says 172.21.15.119:35726) 2023-11-08T19:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 -- 172.21.15.119:0/2716570427 learned_addr learned my addr 172.21.15.119:0/2716570427 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940effd640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100cfb80 0x7f9410159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940ffff640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94100d04a0 0x7f941015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 -- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100cfb80 msgr2=0x7f9410159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:24.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100cfb80 0x7f9410159e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 -- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94100d04a0 msgr2=0x7f941015b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:24.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94100d04a0 0x7f941015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.509+0000 7f940f7fe640 1 -- 172.21.15.119:0/2716570427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400009a70 con 0x7f94100b3de0 2023-11-08T19:28:24.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.510+0000 7f940f7fe640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 0x7f941014c400 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9400037040 tx=0x7f9400004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.510+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400040650 con 0x7f94100b3de0 2023-11-08T19:28:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.510+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94000407b0 con 0x7f94100b3de0 2023-11-08T19:28:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.510+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100c9070 con 0x7f94100b3de0 2023-11-08T19:28:24.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.510+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9400040ba0 con 0x7f94100b3de0 2023-11-08T19:28:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.510+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100ca5a0 con 0x7f94100b3de0 2023-11-08T19:28:24.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.511+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100ca400 con 0x7f94100b3de0 2023-11-08T19:28:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.512+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9400040d00 con 0x7f94100b3de0 2023-11-08T19:28:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.513+0000 7f9407fff640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93f006ae70 0x7f93f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.513+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f94000acdc0 con 0x7f94100b3de0 2023-11-08T19:28:24.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.514+0000 7f940effd640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93f006ae70 0x7f93f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:24.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.514+0000 7f940effd640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93f006ae70 0x7f93f006d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006fd0 tx=0x7f93f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:24.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.517+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9400077a00 con 0x7f94100b3de0 2023-11-08T19:28:24.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.748+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f94100ca400 con 0x7f94100b3de0 2023-11-08T19:28:24.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.801+0000 7f9407fff640 1 -- 172.21.15.119:0/2716570427 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "false", "yes_i_really_mean_it": true}]=0 disabled new snapshots v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7f94100ca400 con 0x7f94100b3de0 2023-11-08T19:28:24.803 INFO:teuthology.orchestra.run.smithi119.stderr:disabled new snapshots 2023-11-08T19:28:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93f006ae70 msgr2=0x7f93f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93f006ae70 0x7f93f006d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f93f8006fd0 tx=0x7f93f8008040 comp rx=0 tx=0).stop 2023-11-08T19:28:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 msgr2=0x7f941014c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:24.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 0x7f941014c400 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9400037040 tx=0x7f9400004a40 comp rx=0 tx=0).stop 2023-11-08T19:28:24.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 shutdown_connections 2023-11-08T19:28:24.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93f006ae70 0x7f93f006d330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94100d04a0 0x7f941015b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f94100cfb80 0x7f9410159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 --2- 172.21.15.119:0/2716570427 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f94100b3de0 0x7f941014c400 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:24.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 >> 172.21.15.119:0/2716570427 conn(0x7f94100d2f60 msgr2=0x7f94100b7880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:24.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 shutdown_connections 2023-11-08T19:28:24.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:24.804+0000 7f9414ad7640 1 -- 172.21.15.119:0/2716570427 wait complete. 2023-11-08T19:28:24.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:24.827 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2023-11-08T19:28:24.884 INFO:teuthology.orchestra.run.smithi119.stderr:mkdir: cannot create directory ‘test-allow-snaps/.snap/snap00’: Operation not permitted 2023-11-08T19:28:24.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:28:24.885 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-08T19:28:25.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.098+0000 7faa78f47640 1 Processor -- start 2023-11-08T19:28:25.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.098+0000 7faa78f47640 1 -- start start 2023-11-08T19:28:25.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.100+0000 7faa78f47640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa7406aa70 0x7faa740b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.100+0000 7faa78f47640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa740cb410 0x7faa7406b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.100+0000 7faa78f47640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7415b380 con 0x7faa74158870 2023-11-08T19:28:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.100+0000 7faa78f47640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa7415c3e0 con 0x7faa740cb410 2023-11-08T19:28:25.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.100+0000 7faa78f47640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa740b3dd0 con 0x7faa7406aa70 2023-11-08T19:28:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.100+0000 7faa73fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa740cb410 0x7faa7406b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa73fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa740cb410 0x7faa7406b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35740/0 (socket says 172.21.15.119:35740) 2023-11-08T19:28:25.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa73fff640 1 -- 172.21.15.119:0/418347838 learned_addr learned my addr 172.21.15.119:0/418347838 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:25.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa6bfff640 1 --2- 172.21.15.119:0/418347838 >> v2:172.21.15.188:3300/0 conn(0x7faa7406aa70 0x7faa740b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa73fff640 1 -- 172.21.15.119:0/418347838 >> v2:172.21.15.188:3300/0 conn(0x7faa7406aa70 msgr2=0x7faa740b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa73fff640 1 --2- 172.21.15.119:0/418347838 >> v2:172.21.15.188:3300/0 conn(0x7faa7406aa70 0x7faa740b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa73fff640 1 -- 172.21.15.119:0/418347838 >> v1:172.21.15.188:6789/0 conn(0x7faa74158870 legacy=0x7faa7406a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:25.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.101+0000 7faa73fff640 1 -- 172.21.15.119:0/418347838 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7415b620 con 0x7faa740cb410 2023-11-08T19:28:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.102+0000 7faa73fff640 1 --2- 172.21.15.119:0/418347838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa740cb410 0x7faa7406b920 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7faa74071b80 tx=0x7faa6c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29b1f9d1a0491920 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.102+0000 7faa72ffd640 1 -- 172.21.15.119:0/418347838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c02ca80 con 0x7faa740cb410 2023-11-08T19:28:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.102+0000 7faa72ffd640 1 -- 172.21.15.119:0/418347838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c02cbe0 con 0x7faa740cb410 2023-11-08T19:28:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.102+0000 7faa72ffd640 1 -- 172.21.15.119:0/418347838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c0343f0 con 0x7faa740cb410 2023-11-08T19:28:25.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 -- 172.21.15.119:0/418347838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa740cb410 msgr2=0x7faa7406b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 --2- 172.21.15.119:0/418347838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa740cb410 0x7faa7406b920 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7faa74071b80 tx=0x7faa6c02bee0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 -- 172.21.15.119:0/418347838 shutdown_connections 2023-11-08T19:28:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 --2- 172.21.15.119:0/418347838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa740cb410 0x7faa7406b920 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 --2- 172.21.15.119:0/418347838 >> v2:172.21.15.188:3300/0 conn(0x7faa7406aa70 0x7faa740b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 -- 172.21.15.119:0/418347838 >> 172.21.15.119:0/418347838 conn(0x7faa740b0870 msgr2=0x7faa740b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 -- 172.21.15.119:0/418347838 shutdown_connections 2023-11-08T19:28:25.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.103+0000 7faa78f47640 1 -- 172.21.15.119:0/418347838 wait complete. 2023-11-08T19:28:25.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.106+0000 7faa78f47640 1 Processor -- start 2023-11-08T19:28:25.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.106+0000 7faa78f47640 1 -- start start 2023-11-08T19:28:25.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa78f47640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7406aa70 0x7faa741535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa78f47640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa740cb410 0x7faa7415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa78f47640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 0x7faa74158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa78f47640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa74071960 con 0x7faa74158870 2023-11-08T19:28:25.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa78f47640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa740710c0 con 0x7faa7406aa70 2023-11-08T19:28:25.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa78f47640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740713a0 con 0x7faa740cb410 2023-11-08T19:28:25.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa73fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 0x7faa74158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.107+0000 7faa73fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 0x7faa74158c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35746/0 (socket says 172.21.15.119:35746) 2023-11-08T19:28:25.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 -- 172.21.15.119:0/3457291957 learned_addr learned my addr 172.21.15.119:0/3457291957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:25.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa6bfff640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa740cb410 0x7faa7415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa737fe640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7406aa70 0x7faa741535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 -- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa740cb410 msgr2=0x7faa7415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa740cb410 0x7faa7415a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 -- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7406aa70 msgr2=0x7faa741535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7406aa70 0x7faa741535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 -- 172.21.15.119:0/3457291957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c009b30 con 0x7faa74158870 2023-11-08T19:28:25.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa73fff640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 0x7faa74158c50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7faa6c036040 tx=0x7faa6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:25.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c009510 con 0x7faa74158870 2023-11-08T19:28:25.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c03e450 con 0x7faa74158870 2023-11-08T19:28:25.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.108+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c03e7e0 con 0x7faa74158870 2023-11-08T19:28:25.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.109+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa74071680 con 0x7faa74158870 2023-11-08T19:28:25.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.109+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7415ad30 con 0x7faa74158870 2023-11-08T19:28:25.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.110+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa741f5f90 con 0x7faa74158870 2023-11-08T19:28:25.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.111+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa6c02cd50 con 0x7faa74158870 2023-11-08T19:28:25.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.112+0000 7faa70ff9640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa5406ae70 0x7faa5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:25.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.112+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7faa6c0ac720 con 0x7faa74158870 2023-11-08T19:28:25.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.112+0000 7faa737fe640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa5406ae70 0x7faa5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:25.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.113+0000 7faa737fe640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa5406ae70 0x7faa5406d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faa6400a750 tx=0x7faa64008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:25.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.116+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa741f5f90 con 0x7faa74158870 2023-11-08T19:28:25.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.342+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7faa741f5f90 con 0x7faa74158870 2023-11-08T19:28:25.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.813+0000 7faa70ff9640 1 -- 172.21.15.119:0/3457291957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v46) v1 ==== 167+0+0 (secure 0 0 0) 0x7faa741f5f90 con 0x7faa74158870 2023-11-08T19:28:25.815 INFO:teuthology.orchestra.run.smithi119.stderr:enabled new snapshots 2023-11-08T19:28:25.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa5406ae70 msgr2=0x7faa5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa5406ae70 0x7faa5406d330 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faa6400a750 tx=0x7faa64008040 comp rx=0 tx=0).stop 2023-11-08T19:28:25.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 msgr2=0x7faa74158c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:25.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 0x7faa74158c50 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7faa6c036040 tx=0x7faa6c004a40 comp rx=0 tx=0).stop 2023-11-08T19:28:25.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 shutdown_connections 2023-11-08T19:28:25.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa5406ae70 0x7faa5406d330 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa74158870 0x7faa74158c50 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa740cb410 0x7faa7415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 --2- 172.21.15.119:0/3457291957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa7406aa70 0x7faa741535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:25.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 >> 172.21.15.119:0/3457291957 conn(0x7faa740b0870 msgr2=0x7faa740b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:25.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 shutdown_connections 2023-11-08T19:28:25.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:25.817+0000 7faa78f47640 1 -- 172.21.15.119:0/3457291957 wait complete. 2023-11-08T19:28:25.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:25.838 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir test-allow-snaps/.snap/snap00) 2023-11-08T19:28:25.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:25.921 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps/.snap/snap00) 2023-11-08T19:28:25.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:25.991 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir test-allow-snaps) 2023-11-08T19:28:26.048 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:28:26.048 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:28:26.181 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:28:26.306 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:28:26.307 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:26.307 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:26.370 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:26.371 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:26.426 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:26.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.686+0000 7f251935e640 1 Processor -- start 2023-11-08T19:28:26.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.686+0000 7f251935e640 1 -- start start 2023-11-08T19:28:26.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.687+0000 7f251935e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 0x7f251415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.687+0000 7f251935e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f25140ad9d0 0x7f251415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.687+0000 7f251935e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25140ca2e0 con 0x7f25140cac30 2023-11-08T19:28:26.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.687+0000 7f251935e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2514071c70 con 0x7f25140ad9d0 2023-11-08T19:28:26.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.687+0000 7f251935e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251415b390 con 0x7f25140adf20 2023-11-08T19:28:26.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f2513fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f25140cac30 0x7f251415f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50132/0 (socket says 172.21.15.119:50132) 2023-11-08T19:28:26.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f2513fff640 1 -- 172.21.15.119:0/2907175207 learned_addr learned my addr 172.21.15.119:0/2907175207 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:26.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f250bfff640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 0x7f251415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f2518b5d640 1 --2- 172.21.15.119:0/2907175207 >> v2:172.21.15.188:3300/0 conn(0x7f25140ad9d0 0x7f251415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619239157 0 0) 0x7f25140ca2e0 con 0x7f25140cac30 2023-11-08T19:28:26.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2500003520 con 0x7f25140cac30 2023-11-08T19:28:26.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.688+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1034971581 0 0) 0x7f2500003520 con 0x7f25140cac30 2023-11-08T19:28:26.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 msgr2=0x7f251415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:26.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 0x7f251415a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 >> v2:172.21.15.188:3300/0 conn(0x7f25140ad9d0 msgr2=0x7f251415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 --2- 172.21.15.119:0/2907175207 >> v2:172.21.15.188:3300/0 conn(0x7f25140ad9d0 0x7f251415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140ca5c0 con 0x7f25140cac30 2023-11-08T19:28:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2504001670 con 0x7f25140cac30 2023-11-08T19:28:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 >> v1:172.21.15.119:6789/0 conn(0x7f25140cac30 legacy=0x7f251415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.697 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500003970 0x7f2500003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 0x7f25000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500004d00 0x7f2500005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2514071c70 con 0x7f2500003970 2023-11-08T19:28:26.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250000b980 con 0x7f2500004d00 2023-11-08T19:28:26.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250000bb40 con 0x7f25140adf20 2023-11-08T19:28:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2518b5d640 1 --2- 172.21.15.119:0/2907175207 >> v2:172.21.15.188:3300/0 conn(0x7f25140ad9d0 0x7f251415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2513fff640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500003970 0x7f2500003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2513fff640 1 -- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 msgr2=0x7f25000047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2513fff640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25140adf20 0x7f25000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2513fff640 1 -- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500004d00 msgr2=0x7f2500005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:26.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2513fff640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500004d00 0x7f2500005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.689+0000 7f2513fff640 1 -- 172.21.15.119:0/2907175207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2500003520 con 0x7f2500003970 2023-11-08T19:28:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.690+0000 7f2513fff640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500003970 0x7f2500003d90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f250400c160 tx=0x7f25000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3dd6e4e0d77365b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2504005ad0 con 0x7f2500003970 2023-11-08T19:28:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2504005c30 con 0x7f2500003970 2023-11-08T19:28:26.700 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f25137fe640 1 -- 172.21.15.119:0/2907175207 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250402c810 con 0x7f2500003970 2023-11-08T19:28:26.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f251935e640 1 -- 172.21.15.119:0/2907175207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500003970 msgr2=0x7f2500003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f251935e640 1 --2- 172.21.15.119:0/2907175207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2500003970 0x7f2500003d90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f250400c160 tx=0x7f25000057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f2518b5d640 1 -- 172.21.15.119:0/2907175207 reap_dead start 2023-11-08T19:28:26.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f251935e640 1 -- 172.21.15.119:0/2907175207 shutdown_connections 2023-11-08T19:28:26.701 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f251935e640 1 -- 172.21.15.119:0/2907175207 >> 172.21.15.119:0/2907175207 conn(0x7f25140b0560 msgr2=0x7f25140b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f251935e640 1 -- 172.21.15.119:0/2907175207 shutdown_connections 2023-11-08T19:28:26.702 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.691+0000 7f251935e640 1 -- 172.21.15.119:0/2907175207 wait complete. 2023-11-08T19:28:26.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.693+0000 7f251935e640 1 Processor -- start 2023-11-08T19:28:26.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.693+0000 7f251935e640 1 -- start start 2023-11-08T19:28:26.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.694+0000 7f251935e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500003970 0x7f25141f3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.694+0000 7f251935e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 0x7f25141e71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.694+0000 7f251935e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25141e7700 0x7f25140ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.694+0000 7f251935e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141e6280 con 0x7f25141f5f40 2023-11-08T19:28:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.694+0000 7f251935e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25141f3970 con 0x7f2500003970 2023-11-08T19:28:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.694+0000 7f251935e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25141f6e70 con 0x7f25141e7700 2023-11-08T19:28:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 0x7f25141e71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 0x7f25141e71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35764/0 (socket says 172.21.15.119:35764) 2023-11-08T19:28:26.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 -- 172.21.15.119:0/969453093 learned_addr learned my addr 172.21.15.119:0/969453093 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f2513fff640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500003970 0x7f25141f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f2518b5d640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25141e7700 0x7f25140ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 -- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25141e7700 msgr2=0x7f25140ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25141e7700 0x7f25140ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 -- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500003970 msgr2=0x7f25141f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500003970 0x7f25141f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 -- 172.21.15.119:0/969453093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25141f68b0 con 0x7f25141f5f40 2023-11-08T19:28:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.695+0000 7f250bfff640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 0x7f25141e71c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00e820 tx=0x7f24fc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.696+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00eed0 con 0x7f25141f5f40 2023-11-08T19:28:26.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.696+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc00c940 con 0x7f25141f5f40 2023-11-08T19:28:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.696+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00ccd0 con 0x7f25141f5f40 2023-11-08T19:28:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.696+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25141f6b90 con 0x7f25141f5f40 2023-11-08T19:28:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.696+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25141e6ca0 con 0x7f25141f5f40 2023-11-08T19:28:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.697+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251404ece0 con 0x7f25141f5f40 2023-11-08T19:28:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.698+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24fc013470 con 0x7f25141f5f40 2023-11-08T19:28:26.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.699+0000 7f25117fa640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24f406ae70 0x7f24f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:26.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.699+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f24fc089d20 con 0x7f25141f5f40 2023-11-08T19:28:26.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.699+0000 7f2513fff640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24f406ae70 0x7f24f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:26.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.700+0000 7f2513fff640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24f406ae70 0x7f24f406d330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2504001050 tx=0x7f2504007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:26.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.703+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24fc0547c0 con 0x7f25141f5f40 2023-11-08T19:28:26.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.905+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25140b1110 con 0x7f25141f5f40 2023-11-08T19:28:26.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.909+0000 7f25117fa640 1 -- 172.21.15.119:0/969453093 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+21953 (secure 0 0 0) 0x7f24fc057de0 con 0x7f25141f5f40 2023-11-08T19:28:26.911 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:26.912 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":83,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:16.741197+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24f406ae70 msgr2=0x7f24f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24f406ae70 0x7f24f406d330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2504001050 tx=0x7f2504007000 comp rx=0 tx=0).stop 2023-11-08T19:28:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 msgr2=0x7f25141e71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 0x7f25141e71c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f24fc00e820 tx=0x7f24fc00ece0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 shutdown_connections 2023-11-08T19:28:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24f406ae70 0x7f24f406d330 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25141e7700 0x7f25140ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f25141f5f40 0x7f25141e71c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 --2- 172.21.15.119:0/969453093 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2500003970 0x7f25141f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:26.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 >> 172.21.15.119:0/969453093 conn(0x7f25140b0560 msgr2=0x7f25140ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:26.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.912+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 shutdown_connections 2023-11-08T19:28:26.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:26.913+0000 7f251935e640 1 -- 172.21.15.119:0/969453093 wait complete. 2023-11-08T19:28:26.933 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:28:13.287561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:28:14.184954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:28:26.934 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:28:27.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.150+0000 7f01e715e640 1 Processor -- start 2023-11-08T19:28:27.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.150+0000 7f01e715e640 1 -- start start 2023-11-08T19:28:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e715e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01e0143600 0x7f01e00c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e715e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f01e015d9f0 0x7f01e015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e715e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e006cb40 0x7f01e006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e715e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00c4f50 con 0x7f01e0143600 2023-11-08T19:28:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e715e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00c4960 con 0x7f01e015d9f0 2023-11-08T19:28:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e715e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e006d4f0 con 0x7f01e006cb40 2023-11-08T19:28:27.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01e0143600 0x7f01e00c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f01e0143600 0x7f01e00c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35776/0 (socket says 172.21.15.119:35776) 2023-11-08T19:28:27.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 -- 172.21.15.119:0/3904737684 learned_addr learned my addr 172.21.15.119:0/3904737684 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:27.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e56fb640 1 --2- 172.21.15.119:0/3904737684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e006cb40 0x7f01e006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01d7fff640 1 --2- 172.21.15.119:0/3904737684 >> v2:172.21.15.188:3300/0 conn(0x7f01e015d9f0 0x7f01e015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 -- 172.21.15.119:0/3904737684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e006cb40 msgr2=0x7f01e006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 --2- 172.21.15.119:0/3904737684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e006cb40 0x7f01e006cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 -- 172.21.15.119:0/3904737684 >> v2:172.21.15.188:3300/0 conn(0x7f01e015d9f0 msgr2=0x7f01e015de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:27.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.152+0000 7f01e4efa640 1 --2- 172.21.15.119:0/3904737684 >> v2:172.21.15.188:3300/0 conn(0x7f01e015d9f0 0x7f01e015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.153+0000 7f01e4efa640 1 -- 172.21.15.119:0/3904737684 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e015e440 con 0x7f01e0143600 2023-11-08T19:28:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.154+0000 7f01e4efa640 1 --2- 172.21.15.119:0/3904737684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e00c5370 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f01e00c91e0 tx=0x7f01dc02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e50e58c68d96f9f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.154+0000 7f01d77fe640 1 -- 172.21.15.119:0/3904737684 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0317d0 con 0x7f01e0143600 2023-11-08T19:28:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.154+0000 7f01d77fe640 1 -- 172.21.15.119:0/3904737684 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc031930 con 0x7f01e0143600 2023-11-08T19:28:27.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.154+0000 7f01d77fe640 1 -- 172.21.15.119:0/3904737684 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc031cc0 con 0x7f01e0143600 2023-11-08T19:28:27.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.154+0000 7f01e715e640 1 -- 172.21.15.119:0/3904737684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 msgr2=0x7f01e00c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 --2- 172.21.15.119:0/3904737684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e00c5370 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f01e00c91e0 tx=0x7f01dc02b870 comp rx=0 tx=0).stop 2023-11-08T19:28:27.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 -- 172.21.15.119:0/3904737684 shutdown_connections 2023-11-08T19:28:27.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 --2- 172.21.15.119:0/3904737684 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e006cb40 0x7f01e006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 --2- 172.21.15.119:0/3904737684 >> v2:172.21.15.188:3300/0 conn(0x7f01e015d9f0 0x7f01e015de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 --2- 172.21.15.119:0/3904737684 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e00c5370 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 -- 172.21.15.119:0/3904737684 >> 172.21.15.119:0/3904737684 conn(0x7f01e00cb610 msgr2=0x7f01e00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 -- 172.21.15.119:0/3904737684 shutdown_connections 2023-11-08T19:28:27.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.155+0000 7f01e715e640 1 -- 172.21.15.119:0/3904737684 wait complete. 2023-11-08T19:28:27.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.157+0000 7f01e715e640 1 Processor -- start 2023-11-08T19:28:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.157+0000 7f01e715e640 1 -- start start 2023-11-08T19:28:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e715e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e006cb40 0x7f01e00aae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e715e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e01f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e715e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e015d9f0 0x7f01e015cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e715e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00ac090 con 0x7f01e0143600 2023-11-08T19:28:27.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e715e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00ab410 con 0x7f01e006cb40 2023-11-08T19:28:27.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e715e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00ab6f0 con 0x7f01e015d9f0 2023-11-08T19:28:27.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01d7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e01f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01d7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e01f3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35784/0 (socket says 172.21.15.119:35784) 2023-11-08T19:28:27.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01d7fff640 1 -- 172.21.15.119:0/525125520 learned_addr learned my addr 172.21.15.119:0/525125520 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:27.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e4efa640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e006cb40 0x7f01e00aae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01e56fb640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e015d9f0 0x7f01e015cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01d7fff640 1 -- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e015d9f0 msgr2=0x7f01e015cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01d7fff640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e015d9f0 0x7f01e015cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.158+0000 7f01d7fff640 1 -- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e006cb40 msgr2=0x7f01e00aae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01d7fff640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e006cb40 0x7f01e00aae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01d7fff640 1 -- 172.21.15.119:0/525125520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc0094d0 con 0x7f01e0143600 2023-11-08T19:28:27.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01d7fff640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e01f3e50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c7f0 tx=0x7f01d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:27.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0004220 con 0x7f01e0143600 2023-11-08T19:28:27.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0004380 con 0x7f01e0143600 2023-11-08T19:28:27.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0007840 con 0x7f01e0143600 2023-11-08T19:28:27.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e00ab9d0 con 0x7f01e0143600 2023-11-08T19:28:27.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.159+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e00ac4b0 con 0x7f01e0143600 2023-11-08T19:28:27.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.160+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e0074430 con 0x7f01e0143600 2023-11-08T19:28:27.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.161+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01d0007bb0 con 0x7f01e0143600 2023-11-08T19:28:27.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.163+0000 7f01d57fa640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f01cc06ae70 0x7f01cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.163+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f01d0089af0 con 0x7f01e0143600 2023-11-08T19:28:27.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.163+0000 7f01e4efa640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f01cc06ae70 0x7f01cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.164+0000 7f01e4efa640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f01cc06ae70 0x7f01cc06d330 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f01dc036000 tx=0x7f01dc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:27.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.167+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01d0054620 con 0x7f01e0143600 2023-11-08T19:28:27.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.358+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f01e00c4230 con 0x7f01e0143600 2023-11-08T19:28:27.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.359+0000 7f01d57fa640 1 -- 172.21.15.119:0/525125520 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f01d001e0a0 con 0x7f01e0143600 2023-11-08T19:28:27.360 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:28:27.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f01cc06ae70 msgr2=0x7f01cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f01cc06ae70 0x7f01cc06d330 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f01dc036000 tx=0x7f01dc03d040 comp rx=0 tx=0).stop 2023-11-08T19:28:27.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 msgr2=0x7f01e01f3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e01f3e50 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c7f0 tx=0x7f01d000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 shutdown_connections 2023-11-08T19:28:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f01cc06ae70 0x7f01cc06d330 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.361+0000 7f01e715e640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f01e015d9f0 0x7f01e015cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.362+0000 7f01e715e640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f01e0143600 0x7f01e01f3e50 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.362+0000 7f01e715e640 1 --2- 172.21.15.119:0/525125520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f01e006cb40 0x7f01e00aae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.362+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 >> 172.21.15.119:0/525125520 conn(0x7f01e00cb610 msgr2=0x7f01e00caaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.362+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 shutdown_connections 2023-11-08T19:28:27.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.362+0000 7f01e715e640 1 -- 172.21.15.119:0/525125520 wait complete. 2023-11-08T19:28:27.381 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:28:27.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.597+0000 7ff89e683640 1 Processor -- start 2023-11-08T19:28:27.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.597+0000 7ff89e683640 1 -- start start 2023-11-08T19:28:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.598+0000 7ff89e683640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff898161a10 0x7ff8980c9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89e683640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8980cb3a0 con 0x7ff89806bcf0 2023-11-08T19:28:27.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89e683640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8980cd400 con 0x7ff898165e60 2023-11-08T19:28:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89e683640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff898162e00 con 0x7ff898161a10 2023-11-08T19:28:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89d681640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff898161a10 0x7ff8980c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89de82640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff89806bcf0 0x7ff8980cd890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50140/0 (socket says 172.21.15.119:50140) 2023-11-08T19:28:27.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89de82640 1 -- 172.21.15.119:0/1903219804 learned_addr learned my addr 172.21.15.119:0/1903219804 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89d681640 1 -- 172.21.15.119:0/1903219804 >> v1:172.21.15.188:6789/0 conn(0x7ff898165e60 legacy=0x7ff89806b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:27.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89d681640 1 -- 172.21.15.119:0/1903219804 >> v1:172.21.15.119:6789/0 conn(0x7ff89806bcf0 legacy=0x7ff8980cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.599+0000 7ff89d681640 1 -- 172.21.15.119:0/1903219804 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8981663b0 con 0x7ff898161a10 2023-11-08T19:28:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.600+0000 7ff89d681640 1 --2- 172.21.15.119:0/1903219804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c9440 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff8940097a0 tx=0x7ff89402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326c4f0affc23133 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.601+0000 7ff88ffff640 1 -- 172.21.15.119:0/1903219804 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894031830 con 0x7ff898161a10 2023-11-08T19:28:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.601+0000 7ff88ffff640 1 -- 172.21.15.119:0/1903219804 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff894031990 con 0x7ff898161a10 2023-11-08T19:28:27.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.601+0000 7ff88ffff640 1 -- 172.21.15.119:0/1903219804 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894031d20 con 0x7ff898161a10 2023-11-08T19:28:27.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.601+0000 7ff89e683640 1 -- 172.21.15.119:0/1903219804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 msgr2=0x7ff8980c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.601+0000 7ff89e683640 1 --2- 172.21.15.119:0/1903219804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c9440 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff8940097a0 tx=0x7ff89402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.602+0000 7ff89e683640 1 -- 172.21.15.119:0/1903219804 shutdown_connections 2023-11-08T19:28:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.602+0000 7ff89e683640 1 --2- 172.21.15.119:0/1903219804 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c9440 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.602+0000 7ff89e683640 1 -- 172.21.15.119:0/1903219804 >> 172.21.15.119:0/1903219804 conn(0x7ff8980ceac0 msgr2=0x7ff8980ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:27.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.602+0000 7ff89e683640 1 -- 172.21.15.119:0/1903219804 shutdown_connections 2023-11-08T19:28:27.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.602+0000 7ff89e683640 1 -- 172.21.15.119:0/1903219804 wait complete. 2023-11-08T19:28:27.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.603+0000 7ff89e683640 1 Processor -- start 2023-11-08T19:28:27.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.604+0000 7ff89e683640 1 -- start start 2023-11-08T19:28:27.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.604+0000 7ff89e683640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89806bcf0 0x7ff8980b4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.604+0000 7ff89e683640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.604+0000 7ff89e683640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff898165e60 0x7ff8980c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.604+0000 7ff89e683640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980c36c0 con 0x7ff898161a10 2023-11-08T19:28:27.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89e683640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff898150c50 con 0x7ff89806bcf0 2023-11-08T19:28:27.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89e683640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8980c60d0 con 0x7ff898165e60 2023-11-08T19:28:27.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35804/0 (socket says 172.21.15.119:35804) 2023-11-08T19:28:27.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 -- 172.21.15.119:0/3598658729 learned_addr learned my addr 172.21.15.119:0/3598658729 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:27.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 -- 172.21.15.119:0/3598658729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff898165e60 msgr2=0x7ff8980c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff898165e60 0x7ff8980c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 -- 172.21.15.119:0/3598658729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89806bcf0 msgr2=0x7ff8980b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89806bcf0 0x7ff8980b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 -- 172.21.15.119:0/3598658729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff894009400 con 0x7ff898161a10 2023-11-08T19:28:27.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.605+0000 7ff89ce80640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c5b90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b3f0 tx=0x7ff88800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.606+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888011830 con 0x7ff898161a10 2023-11-08T19:28:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.606+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff888011990 con 0x7ff898161a10 2023-11-08T19:28:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.606+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8980b1b80 con 0x7ff898161a10 2023-11-08T19:28:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.606+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff888011d40 con 0x7ff898161a10 2023-11-08T19:28:27.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.606+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8980c32e0 con 0x7ff898161a10 2023-11-08T19:28:27.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.607+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8980c3800 con 0x7ff898161a10 2023-11-08T19:28:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.608+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff88801a400 con 0x7ff898161a10 2023-11-08T19:28:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.609+0000 7ff88dffb640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff87c06ae70 0x7ff87c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.610+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7ff8880899f0 con 0x7ff898161a10 2023-11-08T19:28:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.610+0000 7ff89d681640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff87c06ae70 0x7ff87c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:27.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.611+0000 7ff89d681640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff87c06ae70 0x7ff87c06d330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff894036000 tx=0x7ff894030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:27.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.614+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8980c3800 con 0x7ff898161a10 2023-11-08T19:28:27.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.806+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7ff8980c3800 con 0x7ff898161a10 2023-11-08T19:28:27.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.807+0000 7ff88dffb640 1 -- 172.21.15.119:0/3598658729 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff8980c3800 con 0x7ff898161a10 2023-11-08T19:28:27.809 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:27.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff87c06ae70 msgr2=0x7ff87c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff87c06ae70 0x7ff87c06d330 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff894036000 tx=0x7ff894030000 comp rx=0 tx=0).stop 2023-11-08T19:28:27.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 msgr2=0x7ff8980c5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:27.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c5b90 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff88800b3f0 tx=0x7ff88800b8b0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 shutdown_connections 2023-11-08T19:28:27.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff87c06ae70 0x7ff87c06d330 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff898165e60 0x7ff8980c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff898161a10 0x7ff8980c5b90 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 --2- 172.21.15.119:0/3598658729 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff89806bcf0 0x7ff8980b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:27.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 >> 172.21.15.119:0/3598658729 conn(0x7ff8980ceac0 msgr2=0x7ff8980ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:27.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.810+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 shutdown_connections 2023-11-08T19:28:27.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:27.811+0000 7ff89e683640 1 -- 172.21.15.119:0/3598658729 wait complete. 2023-11-08T19:28:27.830 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:28.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.039+0000 7f038b810640 1 Processor -- start 2023-11-08T19:28:28.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.039+0000 7f038b810640 1 -- start start 2023-11-08T19:28:28.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f038b810640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f038415d000 0x7f03840cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f038b810640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 0x7f038406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f038b810640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038406d220 con 0x7f03840ae250 2023-11-08T19:28:28.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f038b810640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03840ccd50 con 0x7f038415d000 2023-11-08T19:28:28.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f038b810640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03840aca40 con 0x7f03840c3a50 2023-11-08T19:28:28.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f0389dad640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f03840ae250 0x7f0384160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:50152/0 (socket says 172.21.15.119:50152) 2023-11-08T19:28:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f0389dad640 1 -- 172.21.15.119:0/633204532 learned_addr learned my addr 172.21.15.119:0/633204532 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f0388dab640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 0x7f038406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.041+0000 7f03895ac640 1 --2- 172.21.15.119:0/633204532 >> v2:172.21.15.188:3300/0 conn(0x7f038415d000 0x7f03840cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353714124 0 0) 0x7f038406d220 con 0x7f03840ae250 2023-11-08T19:28:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0370003520 con 0x7f03840ae250 2023-11-08T19:28:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340279895 0 0) 0x7f0370003520 con 0x7f03840ae250 2023-11-08T19:28:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 msgr2=0x7f038406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 0x7f038406a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 >> v2:172.21.15.188:3300/0 conn(0x7f038415d000 msgr2=0x7f03840cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 --2- 172.21.15.119:0/633204532 >> v2:172.21.15.188:3300/0 conn(0x7f038415d000 0x7f03840cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03840acd20 con 0x7f03840ae250 2023-11-08T19:28:28.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f037c0019f0 con 0x7f03840ae250 2023-11-08T19:28:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 >> v1:172.21.15.119:6789/0 conn(0x7f03840ae250 legacy=0x7f0384160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 0x7f0370003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370003ef0 0x7f03700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0370004d00 0x7f0370005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840ccd50 con 0x7f0370004d00 2023-11-08T19:28:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03840aca40 con 0x7f0370003ef0 2023-11-08T19:28:28.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.042+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037000b9d0 con 0x7f03840c3a50 2023-11-08T19:28:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0389dad640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0370004d00 0x7f0370005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f03895ac640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 0x7f0370003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0388dab640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370003ef0 0x7f03700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0389dad640 1 -- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 msgr2=0x7f0370003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0389dad640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f03840c3a50 0x7f0370003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0389dad640 1 -- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370003ef0 msgr2=0x7f03700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0389dad640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370003ef0 0x7f03700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.043+0000 7f0389dad640 1 -- 172.21.15.119:0/633204532 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0370003520 con 0x7f0370004d00 2023-11-08T19:28:28.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.044+0000 7f0389dad640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0370004d00 0x7f0370005180 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f037c00a890 tx=0x7f03700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782bb13d18553624 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.044+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c034630 con 0x7f0370004d00 2023-11-08T19:28:28.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.044+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c02f070 con 0x7f0370004d00 2023-11-08T19:28:28.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.044+0000 7f037bfff640 1 -- 172.21.15.119:0/633204532 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c034a30 con 0x7f0370004d00 2023-11-08T19:28:28.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f038b810640 1 -- 172.21.15.119:0/633204532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0370004d00 msgr2=0x7f0370005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f038b810640 1 --2- 172.21.15.119:0/633204532 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0370004d00 0x7f0370005180 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f037c00a890 tx=0x7f03700057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f0389dad640 1 -- 172.21.15.119:0/633204532 reap_dead start 2023-11-08T19:28:28.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f038b810640 1 -- 172.21.15.119:0/633204532 shutdown_connections 2023-11-08T19:28:28.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f038b810640 1 -- 172.21.15.119:0/633204532 >> 172.21.15.119:0/633204532 conn(0x7f03840cb610 msgr2=0x7f03840cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f038b810640 1 -- 172.21.15.119:0/633204532 shutdown_connections 2023-11-08T19:28:28.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.045+0000 7f038b810640 1 -- 172.21.15.119:0/633204532 wait complete. 2023-11-08T19:28:28.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.047+0000 7f038b810640 1 Processor -- start 2023-11-08T19:28:28.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.047+0000 7f038b810640 1 -- start start 2023-11-08T19:28:28.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f038b810640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370004d00 0x7f038415d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f038b810640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f038415bb60 0x7f0384150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f038b810640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 0x7f03840cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f038b810640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0384154b80 con 0x7f0384159f60 2023-11-08T19:28:28.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f038b810640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f038415d920 con 0x7f0370004d00 2023-11-08T19:28:28.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f038b810640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03841513f0 con 0x7f038415bb60 2023-11-08T19:28:28.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f0389dad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 0x7f03840cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f0389dad640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 0x7f03840cdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35822/0 (socket says 172.21.15.119:35822) 2023-11-08T19:28:28.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.048+0000 7f0389dad640 1 -- 172.21.15.119:0/1324497760 learned_addr learned my addr 172.21.15.119:0/1324497760 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:28.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f03895ac640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370004d00 0x7f038415d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0388dab640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f038415bb60 0x7f0384150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0389dad640 1 -- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f038415bb60 msgr2=0x7f0384150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0389dad640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f038415bb60 0x7f0384150eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0389dad640 1 -- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370004d00 msgr2=0x7f038415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0389dad640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370004d00 0x7f038415d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0389dad640 1 -- 172.21.15.119:0/1324497760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038415c120 con 0x7f0384159f60 2023-11-08T19:28:28.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0389dad640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 0x7f03840cdee0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f037c00a890 tx=0x7f037c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c001fd0 con 0x7f0384159f60 2023-11-08T19:28:28.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c034640 con 0x7f0384159f60 2023-11-08T19:28:28.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c0349b0 con 0x7f0384159f60 2023-11-08T19:28:28.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f038415c400 con 0x7f0384159f60 2023-11-08T19:28:28.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.049+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0384154dd0 con 0x7f0384159f60 2023-11-08T19:28:28.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.050+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f038406cd50 con 0x7f0384159f60 2023-11-08T19:28:28.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.051+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f037c031030 con 0x7f0384159f60 2023-11-08T19:28:28.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.052+0000 7f0379ffb640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f036806ae70 0x7f036806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.052+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f037c0abd10 con 0x7f0384159f60 2023-11-08T19:28:28.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.052+0000 7f03895ac640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f036806ae70 0x7f036806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.053+0000 7f03895ac640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f036806ae70 0x7f036806d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03800099d0 tx=0x7f0380008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.056+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f037c07a240 con 0x7f0384159f60 2023-11-08T19:28:28.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.245+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f038406cd50 con 0x7f0384159f60 2023-11-08T19:28:28.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.246+0000 7f0379ffb640 1 -- 172.21.15.119:0/1324497760 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f037c079dd0 con 0x7f0384159f60 2023-11-08T19:28:28.247 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:28.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f036806ae70 msgr2=0x7f036806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f036806ae70 0x7f036806d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f03800099d0 tx=0x7f0380008040 comp rx=0 tx=0).stop 2023-11-08T19:28:28.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 msgr2=0x7f03840cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 0x7f03840cdee0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f037c00a890 tx=0x7f037c001d80 comp rx=0 tx=0).stop 2023-11-08T19:28:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 shutdown_connections 2023-11-08T19:28:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f036806ae70 0x7f036806d330 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0384159f60 0x7f03840cdee0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f038415bb60 0x7f0384150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 --2- 172.21.15.119:0/1324497760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0370004d00 0x7f038415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.248+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 >> 172.21.15.119:0/1324497760 conn(0x7f03840cb610 msgr2=0x7f0384158c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.249+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 shutdown_connections 2023-11-08T19:28:28.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.249+0000 7f038b810640 1 -- 172.21.15.119:0/1324497760 wait complete. 2023-11-08T19:28:28.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.481+0000 7f7386ec2640 1 Processor -- start 2023-11-08T19:28:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.481+0000 7f7386ec2640 1 -- start start 2023-11-08T19:28:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.482+0000 7f7386ec2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f73800cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.482+0000 7f7386ec2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73800c3a50 0x7f738006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.482+0000 7f7386ec2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73800ae250 0x7f738006a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.482+0000 7f7386ec2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73800ccd50 con 0x7f73800c3a50 2023-11-08T19:28:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.482+0000 7f7386ec2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738006d220 con 0x7f73800ae250 2023-11-08T19:28:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.482+0000 7f7386ec2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73800aca40 con 0x7f738015d000 2023-11-08T19:28:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73800c3a50 0x7f738006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73800c3a50 0x7f738006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35830/0 (socket says 172.21.15.119:35830) 2023-11-08T19:28:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 -- 172.21.15.119:0/1933228895 learned_addr learned my addr 172.21.15.119:0/1933228895 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f7384c5e640 1 --2- 172.21.15.119:0/1933228895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f73800cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 -- 172.21.15.119:0/1933228895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 msgr2=0x7f73800cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 --2- 172.21.15.119:0/1933228895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f73800cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f738545f640 1 --2- 172.21.15.119:0/1933228895 >> v2:172.21.15.188:3300/0 conn(0x7f73800ae250 0x7f738006a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 -- 172.21.15.119:0/1933228895 >> v2:172.21.15.188:3300/0 conn(0x7f73800ae250 msgr2=0x7f738006a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 --2- 172.21.15.119:0/1933228895 >> v2:172.21.15.188:3300/0 conn(0x7f73800ae250 0x7f738006a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.483+0000 7f737ffff640 1 -- 172.21.15.119:0/1933228895 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73800acd20 con 0x7f73800c3a50 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.484+0000 7f737ffff640 1 --2- 172.21.15.119:0/1933228895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800c3a50 0x7f738006a120 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f738006dcd0 tx=0x7f737002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265dbfc4103b654f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f737f7fe640 1 -- 172.21.15.119:0/1933228895 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73700317d0 con 0x7f73800c3a50 2023-11-08T19:28:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f737f7fe640 1 -- 172.21.15.119:0/1933228895 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7370031930 con 0x7f73800c3a50 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f737f7fe640 1 -- 172.21.15.119:0/1933228895 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7370031cc0 con 0x7f73800c3a50 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 -- 172.21.15.119:0/1933228895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800c3a50 msgr2=0x7f738006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 --2- 172.21.15.119:0/1933228895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800c3a50 0x7f738006a120 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f738006dcd0 tx=0x7f737002b870 comp rx=0 tx=0).stop 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 -- 172.21.15.119:0/1933228895 shutdown_connections 2023-11-08T19:28:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 --2- 172.21.15.119:0/1933228895 >> v2:172.21.15.188:3300/0 conn(0x7f73800ae250 0x7f738006a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 --2- 172.21.15.119:0/1933228895 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800c3a50 0x7f738006a120 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 --2- 172.21.15.119:0/1933228895 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f73800cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 -- 172.21.15.119:0/1933228895 >> 172.21.15.119:0/1933228895 conn(0x7f73800cb610 msgr2=0x7f73800cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 -- 172.21.15.119:0/1933228895 shutdown_connections 2023-11-08T19:28:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.485+0000 7f7386ec2640 1 -- 172.21.15.119:0/1933228895 wait complete. 2023-11-08T19:28:28.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.487+0000 7f7386ec2640 1 Processor -- start 2023-11-08T19:28:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.487+0000 7f7386ec2640 1 -- start start 2023-11-08T19:28:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7386ec2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 0x7f73801f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7386ec2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73800c3a50 0x7f738015c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7386ec2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f7380153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7386ec2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73801f2ff0 con 0x7f73800ae250 2023-11-08T19:28:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7386ec2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738015c8f0 con 0x7f73800c3a50 2023-11-08T19:28:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7386ec2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7380153ed0 con 0x7f738015d000 2023-11-08T19:28:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7384c5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 0x7f73801f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7384c5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 0x7f73801f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35834/0 (socket says 172.21.15.119:35834) 2023-11-08T19:28:28.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.488+0000 7f7384c5e640 1 -- 172.21.15.119:0/3852555191 learned_addr learned my addr 172.21.15.119:0/3852555191 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:28.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f737ffff640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73800c3a50 0x7f738015c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f738545f640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f7380153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7384c5e640 1 -- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 msgr2=0x7f7380153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7384c5e640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f7380153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7384c5e640 1 -- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73800c3a50 msgr2=0x7f738015c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7384c5e640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73800c3a50 0x7f738015c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7384c5e640 1 -- 172.21.15.119:0/3852555191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700094d0 con 0x7f73800ae250 2023-11-08T19:28:28.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7384c5e640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 0x7f73801f2ab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f737400c820 tx=0x7f737400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374004220 con 0x7f73800ae250 2023-11-08T19:28:28.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73740043a0 con 0x7f73800ae250 2023-11-08T19:28:28.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73801f3ad0 con 0x7f73800ae250 2023-11-08T19:28:28.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7374007a70 con 0x7f73800ae250 2023-11-08T19:28:28.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.489+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73801f3790 con 0x7f73800ae250 2023-11-08T19:28:28.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.491+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7380074430 con 0x7f73800ae250 2023-11-08T19:28:28.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.491+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7374007bd0 con 0x7f73800ae250 2023-11-08T19:28:28.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.493+0000 7f737d7fa640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f736c06ae70 0x7f736c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.493+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f7374089910 con 0x7f73800ae250 2023-11-08T19:28:28.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.493+0000 7f737ffff640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f736c06ae70 0x7f736c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.494+0000 7f737ffff640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f736c06ae70 0x7f736c06d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7370036000 tx=0x7f737003d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.497+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7374054440 con 0x7f73800ae250 2023-11-08T19:28:28.690 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.688+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f738006cd50 con 0x7f73800ae250 2023-11-08T19:28:28.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.689+0000 7f737d7fa640 1 -- 172.21.15.119:0/3852555191 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f737401e020 con 0x7f73800ae250 2023-11-08T19:28:28.691 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:28.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f736c06ae70 msgr2=0x7f736c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f736c06ae70 0x7f736c06d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f7370036000 tx=0x7f737003d040 comp rx=0 tx=0).stop 2023-11-08T19:28:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 msgr2=0x7f73801f2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 0x7f73801f2ab0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f737400c820 tx=0x7f737400cce0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 shutdown_connections 2023-11-08T19:28:28.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f736c06ae70 0x7f736c06d330 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f738015d000 0x7f7380153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73800c3a50 0x7f738015c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 --2- 172.21.15.119:0/3852555191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73800ae250 0x7f73801f2ab0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 >> 172.21.15.119:0/3852555191 conn(0x7f73800cb610 msgr2=0x7f7380157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 shutdown_connections 2023-11-08T19:28:28.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.692+0000 7f7386ec2640 1 -- 172.21.15.119:0/3852555191 wait complete. 2023-11-08T19:28:28.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:28:28.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.926+0000 7f9dce727640 1 Processor -- start 2023-11-08T19:28:28.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.926+0000 7f9dce727640 1 -- start start 2023-11-08T19:28:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.927+0000 7f9dce727640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 0x7f9dc80ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.927+0000 7f9dce727640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dc80ce5b0 0x7f9dc80b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.927+0000 7f9dce727640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80d0830 con 0x7f9dc8160530 2023-11-08T19:28:28.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dce727640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80b27c0 con 0x7f9dc80ce5b0 2023-11-08T19:28:28.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dce727640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80b2aa0 con 0x7f9dc8160910 2023-11-08T19:28:28.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dcdf26640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9dc80ce5b0 0x7f9dc80b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dccf24640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 0x7f9dc80ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dccf24640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 0x7f9dc80ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:40304/0 (socket says 172.21.15.119:40304) 2023-11-08T19:28:28.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dccf24640 1 -- 172.21.15.119:0/4092725448 learned_addr learned my addr 172.21.15.119:0/4092725448 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.928+0000 7f9dcd725640 1 --1- 172.21.15.119:0/4092725448 >> v1:172.21.15.188:6789/0 conn(0x7f9dc8160530 0x7f9dc80cd960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:59590/0 (socket says 172.21.15.119:59590) 2023-11-08T19:28:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.929+0000 7f9dccf24640 1 -- 172.21.15.119:0/4092725448 >> v2:172.21.15.188:3300/0 conn(0x7f9dc80ce5b0 msgr2=0x7f9dc80b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.929+0000 7f9dccf24640 1 --2- 172.21.15.119:0/4092725448 >> v2:172.21.15.188:3300/0 conn(0x7f9dc80ce5b0 0x7f9dc80b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.929+0000 7f9dccf24640 1 -- 172.21.15.119:0/4092725448 >> v1:172.21.15.188:6789/0 conn(0x7f9dc8160530 legacy=0x7f9dc80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.929+0000 7f9dccf24640 1 -- 172.21.15.119:0/4092725448 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80d1640 con 0x7f9dc8160910 2023-11-08T19:28:28.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.930+0000 7f9dccf24640 1 --2- 172.21.15.119:0/4092725448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 0x7f9dc80ce070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009870 tx=0x7f9db802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8645e0c69270391 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.931+0000 7f9dbffff640 1 -- 172.21.15.119:0/4092725448 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8030840 con 0x7f9dc8160910 2023-11-08T19:28:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.931+0000 7f9dbffff640 1 -- 172.21.15.119:0/4092725448 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db80309a0 con 0x7f9dc8160910 2023-11-08T19:28:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.931+0000 7f9dbffff640 1 -- 172.21.15.119:0/4092725448 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8030d30 con 0x7f9dc8160910 2023-11-08T19:28:28.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 -- 172.21.15.119:0/4092725448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 msgr2=0x7f9dc80ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 --2- 172.21.15.119:0/4092725448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 0x7f9dc80ce070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009870 tx=0x7f9db802b980 comp rx=0 tx=0).stop 2023-11-08T19:28:28.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 -- 172.21.15.119:0/4092725448 shutdown_connections 2023-11-08T19:28:28.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 --2- 172.21.15.119:0/4092725448 >> v2:172.21.15.188:3300/0 conn(0x7f9dc80ce5b0 0x7f9dc80b2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 --2- 172.21.15.119:0/4092725448 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc8160910 0x7f9dc80ce070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 -- 172.21.15.119:0/4092725448 >> 172.21.15.119:0/4092725448 conn(0x7f9dc80d2ec0 msgr2=0x7f9dc80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 -- 172.21.15.119:0/4092725448 shutdown_connections 2023-11-08T19:28:28.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.932+0000 7f9dce727640 1 -- 172.21.15.119:0/4092725448 wait complete. 2023-11-08T19:28:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.934+0000 7f9dce727640 1 Processor -- start 2023-11-08T19:28:28.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.934+0000 7f9dce727640 1 -- start start 2023-11-08T19:28:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.935+0000 7f9dce727640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc80ce5b0 0x7f9dc8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.935+0000 7f9dce727640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8160530 0x7f9dc80b8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dce727640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 0x7f9dc80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dce727640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80b0c00 con 0x7f9dc8160910 2023-11-08T19:28:28.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dce727640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80b0ee0 con 0x7f9dc8160530 2023-11-08T19:28:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dce727640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80b11c0 con 0x7f9dc80ce5b0 2023-11-08T19:28:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 0x7f9dc80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 0x7f9dc80cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34898/0 (socket says 172.21.15.119:34898) 2023-11-08T19:28:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 -- 172.21.15.119:0/1031206982 learned_addr learned my addr 172.21.15.119:0/1031206982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:28.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dccf24640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8160530 0x7f9dc80b8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcd725640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc80ce5b0 0x7f9dc8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 -- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc80ce5b0 msgr2=0x7f9dc8150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc80ce5b0 0x7f9dc8150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 -- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8160530 msgr2=0x7f9dc80b8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:28.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8160530 0x7f9dc80b8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:28.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dcdf26640 1 -- 172.21.15.119:0/1031206982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db80094d0 con 0x7f9dc8160910 2023-11-08T19:28:28.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.936+0000 7f9dccf24640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8160530 0x7f9dc80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:28.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.937+0000 7f9dcdf26640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 0x7f9dc80cb690 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000efc0 tx=0x7f9dc000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.937+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc0009ca0 con 0x7f9dc8160910 2023-11-08T19:28:28.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.937+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dc000f040 con 0x7f9dc8160910 2023-11-08T19:28:28.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.937+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc00146c0 con 0x7f9dc8160910 2023-11-08T19:28:28.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.937+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc80b14a0 con 0x7f9dc8160910 2023-11-08T19:28:28.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.937+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc8150870 con 0x7f9dc8160910 2023-11-08T19:28:28.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.938+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc804ece0 con 0x7f9dc8160910 2023-11-08T19:28:28.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.939+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9dc0014820 con 0x7f9dc8160910 2023-11-08T19:28:28.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.940+0000 7f9dbdffb640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9dac06ae70 0x7f9dac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:28.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.940+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f9dc008ac20 con 0x7f9dc8160910 2023-11-08T19:28:28.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.940+0000 7f9dcd725640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9dac06ae70 0x7f9dac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:28.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.941+0000 7f9dcd725640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9dac06ae70 0x7f9dac06d330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4000fb0 tx=0x7f9dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:28.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:28.945+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9dc00556c0 con 0x7f9dc8160910 2023-11-08T19:28:29.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.135+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9dc80736c0 con 0x7f9dc8160910 2023-11-08T19:28:29.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.135+0000 7f9dbdffb640 1 -- 172.21.15.119:0/1031206982 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9dc0018020 con 0x7f9dc8160910 2023-11-08T19:28:29.137 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:29.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9dac06ae70 msgr2=0x7f9dac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9dac06ae70 0x7f9dac06d330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4000fb0 tx=0x7f9dc4009040 comp rx=0 tx=0).stop 2023-11-08T19:28:29.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 msgr2=0x7f9dc80cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 0x7f9dc80cb690 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f9dc000efc0 tx=0x7f9dc000c490 comp rx=0 tx=0).stop 2023-11-08T19:28:29.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 shutdown_connections 2023-11-08T19:28:29.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9dac06ae70 0x7f9dac06d330 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9dc8160910 0x7f9dc80cb690 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9dc8160530 0x7f9dc80b8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 --2- 172.21.15.119:0/1031206982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9dc80ce5b0 0x7f9dc8150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.138+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 >> 172.21.15.119:0/1031206982 conn(0x7f9dc80d2ec0 msgr2=0x7f9dc806b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.139+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 shutdown_connections 2023-11-08T19:28:29.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.139+0000 7f9dce727640 1 -- 172.21.15.119:0/1031206982 wait complete. 2023-11-08T19:28:29.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:28:29.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.365+0000 7f01023c9640 1 Processor -- start 2023-11-08T19:28:29.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.365+0000 7f01023c9640 1 -- start start 2023-11-08T19:28:29.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f01023c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00fc0c3a50 0x7f00fc0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f01023c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc06d220 con 0x7f00fc06a830 2023-11-08T19:28:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f01023c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc0aca40 con 0x7f00fc15d000 2023-11-08T19:28:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f01023c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc0ccd50 con 0x7f00fc0c3a50 2023-11-08T19:28:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f0100966640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f00fc0c3a50 0x7f00fc0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f00fb7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f00fc06a830 0x7f00fc160e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40064/0 (socket says 172.21.15.119:40064) 2023-11-08T19:28:29.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.367+0000 7f00fb7fe640 1 -- 172.21.15.119:0/2363100196 learned_addr learned my addr 172.21.15.119:0/2363100196 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:29.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.368+0000 7f0100966640 1 -- 172.21.15.119:0/2363100196 >> v1:172.21.15.188:6789/0 conn(0x7f00fc15d000 legacy=0x7f00fc06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:29.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.368+0000 7f0100966640 1 -- 172.21.15.119:0/2363100196 >> v1:172.21.15.119:6789/0 conn(0x7f00fc06a830 legacy=0x7f00fc160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.368+0000 7f0100966640 1 -- 172.21.15.119:0/2363100196 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0acd20 con 0x7f00fc0c3a50 2023-11-08T19:28:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.368+0000 7f00faffd640 1 -- 172.21.15.119:0/2363100196 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654722297 0 0) 0x7f00fc06d220 con 0x7f00fc06a830 2023-11-08T19:28:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.369+0000 7f0100966640 1 --2- 172.21.15.119:0/2363100196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc0c3a50 0x7f00fc0cc810 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009d90 tx=0x7f00f402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14caecd2d9d8c4f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.369+0000 7f00faffd640 1 -- 172.21.15.119:0/2363100196 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f402cbd0 con 0x7f00fc0c3a50 2023-11-08T19:28:29.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.369+0000 7f00faffd640 1 -- 172.21.15.119:0/2363100196 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f402cd30 con 0x7f00fc0c3a50 2023-11-08T19:28:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.369+0000 7f00faffd640 1 -- 172.21.15.119:0/2363100196 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f402cea0 con 0x7f00fc0c3a50 2023-11-08T19:28:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 -- 172.21.15.119:0/2363100196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc0c3a50 msgr2=0x7f00fc0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 --2- 172.21.15.119:0/2363100196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc0c3a50 0x7f00fc0cc810 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009d90 tx=0x7f00f402be20 comp rx=0 tx=0).stop 2023-11-08T19:28:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 -- 172.21.15.119:0/2363100196 shutdown_connections 2023-11-08T19:28:29.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 --2- 172.21.15.119:0/2363100196 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc0c3a50 0x7f00fc0cc810 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 -- 172.21.15.119:0/2363100196 >> 172.21.15.119:0/2363100196 conn(0x7f00fc0cb610 msgr2=0x7f00fc0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 -- 172.21.15.119:0/2363100196 shutdown_connections 2023-11-08T19:28:29.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.370+0000 7f01023c9640 1 -- 172.21.15.119:0/2363100196 wait complete. 2023-11-08T19:28:29.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.372+0000 7f01023c9640 1 Processor -- start 2023-11-08T19:28:29.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.372+0000 7f01023c9640 1 -- start start 2023-11-08T19:28:29.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f01023c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 0x7f00fc1600c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f01023c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc0c3a50 0x7f00fc15f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f01023c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc15d000 0x7f00fc0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f01023c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc15f7b0 con 0x7f00fc06a830 2023-11-08T19:28:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f01023c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc15e210 con 0x7f00fc0c3a50 2023-11-08T19:28:29.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f01023c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc15e4b0 con 0x7f00fc15d000 2023-11-08T19:28:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 0x7f00fc1600c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 0x7f00fc1600c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34924/0 (socket says 172.21.15.119:34924) 2023-11-08T19:28:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 -- 172.21.15.119:0/3149318115 learned_addr learned my addr 172.21.15.119:0/3149318115 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 -- 172.21.15.119:0/3149318115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc15d000 msgr2=0x7f00fc0b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc15d000 0x7f00fc0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 -- 172.21.15.119:0/3149318115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc0c3a50 msgr2=0x7f00fc15f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc0c3a50 0x7f00fc15f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.373+0000 7f00fbfff640 1 -- 172.21.15.119:0/3149318115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009a70 con 0x7f00fc06a830 2023-11-08T19:28:29.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.374+0000 7f00fbfff640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 0x7f00fc1600c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00b3f0 tx=0x7f00ec00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.374+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec011830 con 0x7f00fc06a830 2023-11-08T19:28:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.374+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec011990 con 0x7f00fc06a830 2023-11-08T19:28:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.374+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec011d40 con 0x7f00fc06a830 2023-11-08T19:28:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.374+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc15e790 con 0x7f00fc06a830 2023-11-08T19:28:29.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.374+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc15ed50 con 0x7f00fc06a830 2023-11-08T19:28:29.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.375+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc06cd50 con 0x7f00fc06a830 2023-11-08T19:28:29.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.376+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00ec01a400 con 0x7f00fc06a830 2023-11-08T19:28:29.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.377+0000 7f00f8ff9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00e806aea0 0x7f00e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.377+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f00ec089a00 con 0x7f00fc06a830 2023-11-08T19:28:29.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.377+0000 7f00fb7fe640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00e806aea0 0x7f00e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.378+0000 7f00fb7fe640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00e806aea0 0x7f00e806d360 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f00f00011e0 tx=0x7f00f0007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.381+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00ec054540 con 0x7f00fc06a830 2023-11-08T19:28:29.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.570+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f00fc0733a0 con 0x7f00fc06a830 2023-11-08T19:28:29.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.570+0000 7f00f8ff9640 1 -- 172.21.15.119:0/3149318115 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f00ec057b60 con 0x7f00fc06a830 2023-11-08T19:28:29.572 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:28:29.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00e806aea0 msgr2=0x7f00e806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00e806aea0 0x7f00e806d360 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f00f00011e0 tx=0x7f00f0007950 comp rx=0 tx=0).stop 2023-11-08T19:28:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 msgr2=0x7f00fc1600c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 0x7f00fc1600c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f00ec00b3f0 tx=0x7f00ec00b8b0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 shutdown_connections 2023-11-08T19:28:29.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00e806aea0 0x7f00e806d360 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00fc15d000 0x7f00fc0b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00fc0c3a50 0x7f00fc15f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 --2- 172.21.15.119:0/3149318115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00fc06a830 0x7f00fc1600c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 >> 172.21.15.119:0/3149318115 conn(0x7f00fc0cb610 msgr2=0x7f00fc15ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 shutdown_connections 2023-11-08T19:28:29.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.573+0000 7f01023c9640 1 -- 172.21.15.119:0/3149318115 wait complete. 2023-11-08T19:28:29.592 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:28:29.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.802+0000 7f4325289640 1 Processor -- start 2023-11-08T19:28:29.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.803+0000 7f4325289640 1 -- start start 2023-11-08T19:28:29.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.804+0000 7f4325289640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f432015d9f0 0x7f43200cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.804+0000 7f4325289640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43200c4960 con 0x7f4320142f80 2023-11-08T19:28:29.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.804+0000 7f4325289640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432015df60 con 0x7f432006cb40 2023-11-08T19:28:29.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.804+0000 7f4325289640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4320143a40 con 0x7f432015d9f0 2023-11-08T19:28:29.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.804+0000 7f431effd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4320142f80 0x7f43200c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40066/0 (socket says 172.21.15.119:40066) 2023-11-08T19:28:29.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.804+0000 7f431e7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f432015d9f0 0x7f43200cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.805+0000 7f431effd640 1 -- 172.21.15.119:0/2648148735 learned_addr learned my addr 172.21.15.119:0/2648148735 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:29.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.805+0000 7f431dffb640 1 -- 172.21.15.119:0/2648148735 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2707313160 0 0) 0x7f43200c4960 con 0x7f4320142f80 2023-11-08T19:28:29.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.805+0000 7f431dffb640 1 -- 172.21.15.119:0/2648148735 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f430c005730 con 0x7f4320142f80 2023-11-08T19:28:29.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.805+0000 7f431e7fc640 1 -- 172.21.15.119:0/2648148735 >> v1:172.21.15.188:6789/0 conn(0x7f432006cb40 legacy=0x7f43200c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:29.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.805+0000 7f431e7fc640 1 -- 172.21.15.119:0/2648148735 >> v1:172.21.15.119:6789/0 conn(0x7f4320142f80 legacy=0x7f43200c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.805+0000 7f431e7fc640 1 -- 172.21.15.119:0/2648148735 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432015e240 con 0x7f432015d9f0 2023-11-08T19:28:29.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.806+0000 7f431e7fc640 1 --2- 172.21.15.119:0/2648148735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f432015d9f0 0x7f43200cc6d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f43080097a0 tx=0x7f430802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5862717fecea16 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f431dffb640 1 -- 172.21.15.119:0/2648148735 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308031830 con 0x7f432015d9f0 2023-11-08T19:28:29.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f431dffb640 1 -- 172.21.15.119:0/2648148735 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4308031990 con 0x7f432015d9f0 2023-11-08T19:28:29.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f431dffb640 1 -- 172.21.15.119:0/2648148735 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308031d20 con 0x7f432015d9f0 2023-11-08T19:28:29.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f4325289640 1 -- 172.21.15.119:0/2648148735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f432015d9f0 msgr2=0x7f43200cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f4325289640 1 --2- 172.21.15.119:0/2648148735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f432015d9f0 0x7f43200cc6d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f43080097a0 tx=0x7f430802b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f4325289640 1 -- 172.21.15.119:0/2648148735 shutdown_connections 2023-11-08T19:28:29.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f4325289640 1 --2- 172.21.15.119:0/2648148735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f432015d9f0 0x7f43200cc6d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.807+0000 7f4325289640 1 -- 172.21.15.119:0/2648148735 >> 172.21.15.119:0/2648148735 conn(0x7f43200cb610 msgr2=0x7f43200cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:29.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.808+0000 7f4325289640 1 -- 172.21.15.119:0/2648148735 shutdown_connections 2023-11-08T19:28:29.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.808+0000 7f4325289640 1 -- 172.21.15.119:0/2648148735 wait complete. 2023-11-08T19:28:29.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.809+0000 7f4325289640 1 Processor -- start 2023-11-08T19:28:29.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.809+0000 7f4325289640 1 -- start start 2023-11-08T19:28:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.810+0000 7f4325289640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f432006cb40 0x7f43200b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.810+0000 7f4325289640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 0x7f43200afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.810+0000 7f4325289640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f432015d9f0 0x7f43200b3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.810+0000 7f4325289640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b31f0 con 0x7f4320142f80 2023-11-08T19:28:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.810+0000 7f4325289640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b1c10 con 0x7f432006cb40 2023-11-08T19:28:29.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.810+0000 7f4325289640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200b1ef0 con 0x7f432015d9f0 2023-11-08T19:28:29.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 0x7f43200afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 0x7f43200afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34948/0 (socket says 172.21.15.119:34948) 2023-11-08T19:28:29.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 -- 172.21.15.119:0/1232637249 learned_addr learned my addr 172.21.15.119:0/1232637249 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:29.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 -- 172.21.15.119:0/1232637249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f432015d9f0 msgr2=0x7f43200b3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f432015d9f0 0x7f43200b3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 -- 172.21.15.119:0/1232637249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f432006cb40 msgr2=0x7f43200b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:29.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f432006cb40 0x7f43200b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:29.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 -- 172.21.15.119:0/1232637249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4308009400 con 0x7f4320142f80 2023-11-08T19:28:29.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.811+0000 7f431e7fc640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 0x7f43200afab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f4308034040 tx=0x7f43080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.812+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430803a630 con 0x7f4320142f80 2023-11-08T19:28:29.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.812+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430803a7d0 con 0x7f4320142f80 2023-11-08T19:28:29.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.812+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430803ab80 con 0x7f4320142f80 2023-11-08T19:28:29.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.812+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43200b21d0 con 0x7f4320142f80 2023-11-08T19:28:29.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.812+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43200b2790 con 0x7f4320142f80 2023-11-08T19:28:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.813+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43200b29d0 con 0x7f4320142f80 2023-11-08T19:28:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.814+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f430803ace0 con 0x7f4320142f80 2023-11-08T19:28:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.815+0000 7f43177fe640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f430006ae70 0x7f430006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.815+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f43080ab5f0 con 0x7f4320142f80 2023-11-08T19:28:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.815+0000 7f431effd640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f430006ae70 0x7f430006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:29.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.816+0000 7f431effd640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f430006ae70 0x7f430006d330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f43100020a0 tx=0x7f4310001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:29.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:29.819+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43200b29d0 con 0x7f4320142f80 2023-11-08T19:28:30.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.023+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f43200b29d0 con 0x7f4320142f80 2023-11-08T19:28:30.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.024+0000 7f43177fe640 1 -- 172.21.15.119:0/1232637249 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7f43200b29d0 con 0x7f4320142f80 2023-11-08T19:28:30.026 INFO:teuthology.orchestra.run.smithi119.stderr:listed 0 entries 2023-11-08T19:28:30.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f430006ae70 msgr2=0x7f430006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f430006ae70 0x7f430006d330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f43100020a0 tx=0x7f4310001690 comp rx=0 tx=0).stop 2023-11-08T19:28:30.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 msgr2=0x7f43200afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 0x7f43200afab0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f4308034040 tx=0x7f43080046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 shutdown_connections 2023-11-08T19:28:30.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f430006ae70 0x7f430006d330 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f432015d9f0 0x7f43200b3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4320142f80 0x7f43200afab0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 --2- 172.21.15.119:0/1232637249 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f432006cb40 0x7f43200b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 >> 172.21.15.119:0/1232637249 conn(0x7f43200cb610 msgr2=0x7f43200cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:30.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 shutdown_connections 2023-11-08T19:28:30.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.026+0000 7f4325289640 1 -- 172.21.15.119:0/1232637249 wait complete. 2023-11-08T19:28:30.045 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:28:30.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:30.046 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:28:30.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:28:30.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:28:30.151 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:28:30.202 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:28:30.203 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:30.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:28:30.260 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:30.261 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:30.319 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:28:30.319 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:28:30.320 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:30.378 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:28:30.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:30.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.595+0000 7f448a316640 1 Processor -- start 2023-11-08T19:28:30.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.595+0000 7f448a316640 1 -- start start 2023-11-08T19:28:30.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f448a316640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44840adf20 0x7f448415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f448a316640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44840ad9d0 0x7f448415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f448a316640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44840ca2e0 con 0x7f44840cac30 2023-11-08T19:28:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f448a316640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4484071c70 con 0x7f44840ad9d0 2023-11-08T19:28:30.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f448a316640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448415b390 con 0x7f44840adf20 2023-11-08T19:28:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f4489b15640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f44840ad9d0 0x7f448415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.596+0000 7f4489314640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f44840cac30 0x7f448415f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40076/0 (socket says 172.21.15.119:40076) 2023-11-08T19:28:30.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.597+0000 7f4489314640 1 -- 172.21.15.119:0/4001240198 learned_addr learned my addr 172.21.15.119:0/4001240198 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.597+0000 7f4488b13640 1 --2- 172.21.15.119:0/4001240198 >> v2:172.21.15.188:3300/0 conn(0x7f44840adf20 0x7f448415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.597+0000 7f4489b15640 1 -- 172.21.15.119:0/4001240198 >> v2:172.21.15.188:3300/0 conn(0x7f44840adf20 msgr2=0x7f448415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.597+0000 7f4489b15640 1 --2- 172.21.15.119:0/4001240198 >> v2:172.21.15.188:3300/0 conn(0x7f44840adf20 0x7f448415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.597+0000 7f4489b15640 1 -- 172.21.15.119:0/4001240198 >> v1:172.21.15.119:6789/0 conn(0x7f44840cac30 legacy=0x7f448415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.597+0000 7f4489b15640 1 -- 172.21.15.119:0/4001240198 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840ca5c0 con 0x7f44840ad9d0 2023-11-08T19:28:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.598+0000 7f4489b15640 1 --2- 172.21.15.119:0/4001240198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840ad9d0 0x7f448415ae50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f44840ce940 tx=0x7f447c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a768dc2b0d0d4b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.598+0000 7f447bfff640 1 -- 172.21.15.119:0/4001240198 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c02cc70 con 0x7f44840ad9d0 2023-11-08T19:28:30.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.598+0000 7f447bfff640 1 -- 172.21.15.119:0/4001240198 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c02cdd0 con 0x7f44840ad9d0 2023-11-08T19:28:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.598+0000 7f447bfff640 1 -- 172.21.15.119:0/4001240198 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c034660 con 0x7f44840ad9d0 2023-11-08T19:28:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 -- 172.21.15.119:0/4001240198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840ad9d0 msgr2=0x7f448415ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 --2- 172.21.15.119:0/4001240198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840ad9d0 0x7f448415ae50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f44840ce940 tx=0x7f447c02bec0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 -- 172.21.15.119:0/4001240198 shutdown_connections 2023-11-08T19:28:30.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 --2- 172.21.15.119:0/4001240198 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840ad9d0 0x7f448415ae50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 --2- 172.21.15.119:0/4001240198 >> v2:172.21.15.188:3300/0 conn(0x7f44840adf20 0x7f448415a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 -- 172.21.15.119:0/4001240198 >> 172.21.15.119:0/4001240198 conn(0x7f44840b0560 msgr2=0x7f44840b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:30.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 -- 172.21.15.119:0/4001240198 shutdown_connections 2023-11-08T19:28:30.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.599+0000 7f448a316640 1 -- 172.21.15.119:0/4001240198 wait complete. 2023-11-08T19:28:30.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.601+0000 7f448a316640 1 Processor -- start 2023-11-08T19:28:30.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.601+0000 7f448a316640 1 -- start start 2023-11-08T19:28:30.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f448a316640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44840ad9d0 0x7f4484145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f448a316640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 0x7f44840cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f448a316640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44840cac30 0x7f44840cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f448a316640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44841463c0 con 0x7f44840adf20 2023-11-08T19:28:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f448a316640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44840ce4e0 con 0x7f44840cac30 2023-11-08T19:28:30.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f448a316640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448415b4e0 con 0x7f44840ad9d0 2023-11-08T19:28:30.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 0x7f44840cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:30.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 0x7f44840cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34966/0 (socket says 172.21.15.119:34966) 2023-11-08T19:28:30.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 -- 172.21.15.119:0/1397026956 learned_addr learned my addr 172.21.15.119:0/1397026956 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:30.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 -- 172.21.15.119:0/1397026956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44840ad9d0 msgr2=0x7f4484145e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44840ad9d0 0x7f4484145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 -- 172.21.15.119:0/1397026956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44840cac30 msgr2=0x7f44840cdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44840cac30 0x7f44840cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.602+0000 7f4488b13640 1 -- 172.21.15.119:0/1397026956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009a70 con 0x7f44840adf20 2023-11-08T19:28:30.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.603+0000 7f4488b13640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 0x7f44840cd970 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f447400c7f0 tx=0x7f447400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:30.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.603+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474004220 con 0x7f44840adf20 2023-11-08T19:28:30.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.603+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474004380 con 0x7f44840adf20 2023-11-08T19:28:30.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.603+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474007840 con 0x7f44840adf20 2023-11-08T19:28:30.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.603+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448415b7c0 con 0x7f44840adf20 2023-11-08T19:28:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.603+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f448415bd80 con 0x7f44840adf20 2023-11-08T19:28:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.604+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f448415bfa0 con 0x7f44840adf20 2023-11-08T19:28:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.605+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4474007bb0 con 0x7f44840adf20 2023-11-08T19:28:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.606+0000 7f4479ffb640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f446806ae70 0x7f446806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:30.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.606+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f4474089b40 con 0x7f44840adf20 2023-11-08T19:28:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.606+0000 7f4489314640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f446806ae70 0x7f446806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.607+0000 7f4489314640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f446806ae70 0x7f446806d330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f44840af000 tx=0x7f4480007a40 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:30.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.610+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4474054670 con 0x7f44840adf20 2023-11-08T19:28:30.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.845+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f448404ece0 con 0x7f44840adf20 2023-11-08T19:28:30.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.847+0000 7f4479ffb640 1 -- 172.21.15.119:0/1397026956 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f4474007e60 con 0x7f44840adf20 2023-11-08T19:28:30.850 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:30.851 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:15.191469+0000","modified":"2023-11-08T19:28:25.344541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/4016780496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4016780496},{"type":"v1","addr":"172.21.15.119:6837","nonce":4016780496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:28:30.852 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 46 2023-11-08T19:28:30.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f446806ae70 msgr2=0x7f446806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f446806ae70 0x7f446806d330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f44840af000 tx=0x7f4480007a40 comp rx=0 tx=0).stop 2023-11-08T19:28:30.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 msgr2=0x7f44840cd970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:30.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 0x7f44840cd970 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f447400c7f0 tx=0x7f447400ccb0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 shutdown_connections 2023-11-08T19:28:30.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f446806ae70 0x7f446806d330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44840cac30 0x7f44840cdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f44840adf20 0x7f44840cd970 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 --2- 172.21.15.119:0/1397026956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44840ad9d0 0x7f4484145e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:30.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 >> 172.21.15.119:0/1397026956 conn(0x7f44840b0560 msgr2=0x7f44840c4b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:30.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 shutdown_connections 2023-11-08T19:28:30.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:30.850+0000 7f448a316640 1 -- 172.21.15.119:0/1397026956 wait complete. 2023-11-08T19:28:30.871 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:31.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.079+0000 7fa0b42f2640 1 Processor -- start 2023-11-08T19:28:31.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.080+0000 7fa0b42f2640 1 -- start start 2023-11-08T19:28:31.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.081+0000 7fa0b42f2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0ac0ccde0 0x7fa0ac06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.081+0000 7fa0b42f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac1653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.081+0000 7fa0b42f2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0ac06d4b0 con 0x7fa0ac06a970 2023-11-08T19:28:31.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.081+0000 7fa0b42f2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0ac1619b0 con 0x7fa0ac0ccde0 2023-11-08T19:28:31.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.081+0000 7fa0b42f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac147b80 con 0x7fa0ac0c9150 2023-11-08T19:28:31.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0ac0ccde0 0x7fa0ac06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa0ac0ccde0 0x7fa0ac06cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34970/0 (socket says 172.21.15.119:34970) 2023-11-08T19:28:31.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 -- 172.21.15.119:0/4016242397 learned_addr learned my addr 172.21.15.119:0/4016242397 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:31.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 -- 172.21.15.119:0/4016242397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 msgr2=0x7fa0ac1653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 --2- 172.21.15.119:0/4016242397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac1653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 -- 172.21.15.119:0/4016242397 >> v1:172.21.15.188:6789/0 conn(0x7fa0ac06a970 legacy=0x7fa0ac0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.082+0000 7fa0b188d640 1 -- 172.21.15.119:0/4016242397 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ac147000 con 0x7fa0ac0ccde0 2023-11-08T19:28:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.083+0000 7fa0b188d640 1 --2- 172.21.15.119:0/4016242397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac06cf70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009870 tx=0x7fa09c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7af120c4f01301 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b108c640 1 -- 172.21.15.119:0/4016242397 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c0317c0 con 0x7fa0ac0ccde0 2023-11-08T19:28:31.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b108c640 1 -- 172.21.15.119:0/4016242397 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c031920 con 0x7fa0ac0ccde0 2023-11-08T19:28:31.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b108c640 1 -- 172.21.15.119:0/4016242397 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c031cb0 con 0x7fa0ac0ccde0 2023-11-08T19:28:31.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 -- 172.21.15.119:0/4016242397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac0ccde0 msgr2=0x7fa0ac06cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/4016242397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac06cf70 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009870 tx=0x7fa09c02b860 comp rx=0 tx=0).stop 2023-11-08T19:28:31.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 -- 172.21.15.119:0/4016242397 shutdown_connections 2023-11-08T19:28:31.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/4016242397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac06cf70 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/4016242397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac1653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 -- 172.21.15.119:0/4016242397 >> 172.21.15.119:0/4016242397 conn(0x7fa0ac0cb610 msgr2=0x7fa0ac0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 -- 172.21.15.119:0/4016242397 shutdown_connections 2023-11-08T19:28:31.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.084+0000 7fa0b42f2640 1 -- 172.21.15.119:0/4016242397 wait complete. 2023-11-08T19:28:31.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.086+0000 7fa0b42f2640 1 Processor -- start 2023-11-08T19:28:31.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.086+0000 7fa0b42f2640 1 -- start start 2023-11-08T19:28:31.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.087+0000 7fa0b42f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 0x7fa0ac0b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.087+0000 7fa0b42f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.087+0000 7fa0b42f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac1631e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.087+0000 7fa0b42f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac145fa0 con 0x7fa0ac06a970 2023-11-08T19:28:31.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.087+0000 7fa0b42f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac153ae0 con 0x7fa0ac0ccde0 2023-11-08T19:28:31.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.087+0000 7fa0b42f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0ac153dc0 con 0x7fa0ac0c9150 2023-11-08T19:28:31.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 0x7fa0ac0b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 0x7fa0ac0b18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:34982/0 (socket says 172.21.15.119:34982) 2023-11-08T19:28:31.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 -- 172.21.15.119:0/2430353603 learned_addr learned my addr 172.21.15.119:0/2430353603 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:31.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b288f640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac1631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b188d640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 -- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 msgr2=0x7fa0ac145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 -- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0ac0ccde0 msgr2=0x7fa0ac1631e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac1631e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b208e640 1 -- 172.21.15.119:0/2430353603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0094d0 con 0x7fa0ac06a970 2023-11-08T19:28:31.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.088+0000 7fa0b288f640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac1631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.089+0000 7fa0b208e640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 0x7fa0ac0b18d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c410 tx=0x7fa0a800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.089+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a80035f0 con 0x7fa0ac06a970 2023-11-08T19:28:31.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.089+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a8003750 con 0x7fa0ac06a970 2023-11-08T19:28:31.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.089+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8003b40 con 0x7fa0ac06a970 2023-11-08T19:28:31.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.089+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0ac1540a0 con 0x7fa0ac06a970 2023-11-08T19:28:31.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.089+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0ac154660 con 0x7fa0ac06a970 2023-11-08T19:28:31.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.090+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0ac1fa860 con 0x7fa0ac06a970 2023-11-08T19:28:31.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.091+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0a8003ca0 con 0x7fa0ac06a970 2023-11-08T19:28:31.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.092+0000 7fa0a2ffd640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa09006aea0 0x7fa09006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.092+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7fa0a8088e10 con 0x7fa0ac06a970 2023-11-08T19:28:31.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.092+0000 7fa0b188d640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa09006aea0 0x7fa09006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.093+0000 7fa0b188d640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa09006aea0 0x7fa09006d360 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa09c036000 tx=0x7fa09c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.097+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0a808c030 con 0x7fa0ac06a970 2023-11-08T19:28:31.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.286+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0ac074430 con 0x7fa0ac06a970 2023-11-08T19:28:31.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.290+0000 7fa0a2ffd640 1 -- 172.21.15.119:0/2430353603 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+21953 (secure 0 0 0) 0x7fa0a80538b0 con 0x7fa0ac06a970 2023-11-08T19:28:31.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:31.292 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":83,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:16.741197+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:31.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa09006aea0 msgr2=0x7fa09006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa09006aea0 0x7fa09006d360 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa09c036000 tx=0x7fa09c03d040 comp rx=0 tx=0).stop 2023-11-08T19:28:31.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 msgr2=0x7fa0ac0b18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 0x7fa0ac0b18d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c410 tx=0x7fa0a800c8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 shutdown_connections 2023-11-08T19:28:31.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa09006aea0 0x7fa09006d360 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa0ac0ccde0 0x7fa0ac1631e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.292+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa0ac0c9150 0x7fa0ac145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.293+0000 7fa0b42f2640 1 --2- 172.21.15.119:0/2430353603 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa0ac06a970 0x7fa0ac0b18d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.293+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 >> 172.21.15.119:0/2430353603 conn(0x7fa0ac0cb610 msgr2=0x7fa0ac06f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.293+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 shutdown_connections 2023-11-08T19:28:31.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.293+0000 7fa0b42f2640 1 -- 172.21.15.119:0/2430353603 wait complete. 2023-11-08T19:28:31.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:28:13.287561+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:28:14.184954+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:28:31.314 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:28:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.527+0000 7fab13f4c640 1 Processor -- start 2023-11-08T19:28:31.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.528+0000 7fab13f4c640 1 -- start start 2023-11-08T19:28:31.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.529+0000 7fab13f4c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7fab0c15bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.529+0000 7fab13f4c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fab0c15f1d0 0x7fab0c0c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.529+0000 7fab13f4c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab0c15ea60 con 0x7fab0c144580 2023-11-08T19:28:31.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.529+0000 7fab13f4c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab0c0c9150 con 0x7fab0c15f1d0 2023-11-08T19:28:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.529+0000 7fab13f4c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0c9360 con 0x7fab0c144960 2023-11-08T19:28:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab11ce8640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fab0c144580 0x7fab0c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40086/0 (socket says 172.21.15.119:40086) 2023-11-08T19:28:31.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab11ce8640 1 -- 172.21.15.119:0/1887568824 learned_addr learned my addr 172.21.15.119:0/1887568824 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206955776 0 0) 0x7fab0c15ea60 con 0x7fab0c144580 2023-11-08T19:28:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab114e7640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7fab0c15bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab124e9640 1 --2- 172.21.15.119:0/1887568824 >> v2:172.21.15.188:3300/0 conn(0x7fab0c15f1d0 0x7fab0c0c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faaf8003520 con 0x7fab0c144580 2023-11-08T19:28:31.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1514834019 0 0) 0x7faaf8003520 con 0x7fab0c144580 2023-11-08T19:28:31.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 msgr2=0x7fab0c15bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7fab0c15bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 >> v2:172.21.15.188:3300/0 conn(0x7fab0c15f1d0 msgr2=0x7fab0c0c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 --2- 172.21.15.119:0/1887568824 >> v2:172.21.15.188:3300/0 conn(0x7fab0c15f1d0 0x7fab0c0c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0c9640 con 0x7fab0c144580 2023-11-08T19:28:31.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fab08001670 con 0x7fab0c144580 2023-11-08T19:28:31.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.530+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 >> v1:172.21.15.119:6789/0 conn(0x7fab0c144580 legacy=0x7fab0c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab10ce6640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7faaf8003990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab10ce6640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf8004010 0x7faaf80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab10ce6640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaf8004d00 0x7faaf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0c9150 con 0x7faaf8004010 2023-11-08T19:28:31.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0c9360 con 0x7faaf8004d00 2023-11-08T19:28:31.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf800bbc0 con 0x7fab0c144960 2023-11-08T19:28:31.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab114e7640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf8004010 0x7faaf80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab11ce8640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7faaf8003990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab124e9640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaf8004d00 0x7faaf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab114e7640 1 -- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 msgr2=0x7faaf8003990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab114e7640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7faaf8003990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab114e7640 1 -- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaf8004d00 msgr2=0x7faaf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab114e7640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaf8004d00 0x7faaf8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab114e7640 1 -- 172.21.15.119:0/1887568824 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf8003520 con 0x7faaf8004010 2023-11-08T19:28:31.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab11ce8640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab0c144960 0x7faaf8003990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.531+0000 7fab124e9640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faaf8004d00 0x7faaf8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.532+0000 7fab114e7640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf8004010 0x7faaf80047c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faafc00ba30 tx=0x7faaf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f138f0785d86f5e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc032640 con 0x7faaf8004010 2023-11-08T19:28:31.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc0327a0 con 0x7faaf8004010 2023-11-08T19:28:31.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab10ce6640 1 -- 172.21.15.119:0/1887568824 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc032b30 con 0x7faaf8004010 2023-11-08T19:28:31.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab13f4c640 1 -- 172.21.15.119:0/1887568824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf8004010 msgr2=0x7faaf80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab13f4c640 1 --2- 172.21.15.119:0/1887568824 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faaf8004010 0x7faaf80047c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7faafc00ba30 tx=0x7faaf80057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab124e9640 1 -- 172.21.15.119:0/1887568824 reap_dead start 2023-11-08T19:28:31.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab13f4c640 1 -- 172.21.15.119:0/1887568824 shutdown_connections 2023-11-08T19:28:31.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab13f4c640 1 -- 172.21.15.119:0/1887568824 >> 172.21.15.119:0/1887568824 conn(0x7fab0c0cb610 msgr2=0x7fab0c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.533+0000 7fab13f4c640 1 -- 172.21.15.119:0/1887568824 shutdown_connections 2023-11-08T19:28:31.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.534+0000 7fab13f4c640 1 -- 172.21.15.119:0/1887568824 wait complete. 2023-11-08T19:28:31.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.535+0000 7fab13f4c640 1 Processor -- start 2023-11-08T19:28:31.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.535+0000 7fab13f4c640 1 -- start start 2023-11-08T19:28:31.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.536+0000 7fab13f4c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaf8004010 0x7fab0c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.536+0000 7fab13f4c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 0x7fab0c0c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.536+0000 7fab13f4c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab0c0c7bc0 0x7fab0c06ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.536+0000 7fab13f4c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c12c3b0 con 0x7fab0c13f5b0 2023-11-08T19:28:31.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.536+0000 7fab13f4c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0c8040 con 0x7fab0c0c7bc0 2023-11-08T19:28:31.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.536+0000 7fab13f4c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab0c0c8320 con 0x7faaf8004010 2023-11-08T19:28:31.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 0x7fab0c0c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 0x7fab0c0c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35004/0 (socket says 172.21.15.119:35004) 2023-11-08T19:28:31.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 -- 172.21.15.119:0/2554464766 learned_addr learned my addr 172.21.15.119:0/2554464766 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:31.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab11ce8640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaf8004010 0x7fab0c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab124e9640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab0c0c7bc0 0x7fab0c06ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 -- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaf8004010 msgr2=0x7fab0c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaf8004010 0x7fab0c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 -- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab0c0c7bc0 msgr2=0x7fab0c06ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab0c0c7bc0 0x7fab0c06ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab114e7640 1 -- 172.21.15.119:0/2554464766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab0c0c8600 con 0x7fab0c13f5b0 2023-11-08T19:28:31.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.537+0000 7fab124e9640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab0c0c7bc0 0x7fab0c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.538+0000 7fab114e7640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 0x7fab0c0c7680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7faafc00bb50 tx=0x7faafc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.538+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc03ca80 con 0x7fab0c13f5b0 2023-11-08T19:28:31.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.538+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faafc03cbe0 con 0x7fab0c13f5b0 2023-11-08T19:28:31.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.538+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab0c0c88e0 con 0x7fab0c13f5b0 2023-11-08T19:28:31.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.538+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc030410 con 0x7fab0c13f5b0 2023-11-08T19:28:31.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.538+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c15d280 con 0x7fab0c13f5b0 2023-11-08T19:28:31.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.539+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab0c0733c0 con 0x7fab0c13f5b0 2023-11-08T19:28:31.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.540+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faafc030570 con 0x7fab0c13f5b0 2023-11-08T19:28:31.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.541+0000 7fab027fc640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf006ae70 0x7faaf006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.541+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7faafc0ac290 con 0x7fab0c13f5b0 2023-11-08T19:28:31.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.542+0000 7fab11ce8640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf006ae70 0x7faaf006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.542+0000 7fab11ce8640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf006ae70 0x7faaf006d330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fab08000c00 tx=0x7fab08009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.546+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faafc07a8e0 con 0x7fab0c13f5b0 2023-11-08T19:28:31.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.734+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab0c15d280 con 0x7fab0c13f5b0 2023-11-08T19:28:31.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.735+0000 7fab027fc640 1 -- 172.21.15.119:0/2554464766 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7faafc006c30 con 0x7fab0c13f5b0 2023-11-08T19:28:31.737 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:28:31.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf006ae70 msgr2=0x7faaf006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf006ae70 0x7faaf006d330 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fab08000c00 tx=0x7fab08009040 comp rx=0 tx=0).stop 2023-11-08T19:28:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 msgr2=0x7fab0c0c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 0x7fab0c0c7680 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7faafc00bb50 tx=0x7faafc004730 comp rx=0 tx=0).stop 2023-11-08T19:28:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 shutdown_connections 2023-11-08T19:28:31.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faaf006ae70 0x7faaf006d330 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab0c0c7bc0 0x7fab0c06ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab0c13f5b0 0x7fab0c0c7680 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 --2- 172.21.15.119:0/2554464766 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faaf8004010 0x7fab0c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 >> 172.21.15.119:0/2554464766 conn(0x7fab0c0cb610 msgr2=0x7fab0c15db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 shutdown_connections 2023-11-08T19:28:31.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.738+0000 7fab13f4c640 1 -- 172.21.15.119:0/2554464766 wait complete. 2023-11-08T19:28:31.760 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:28:31.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.967+0000 7f50c07ee640 1 Processor -- start 2023-11-08T19:28:31.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.967+0000 7f50c07ee640 1 -- start start 2023-11-08T19:28:31.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50c07ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50b80ccb40 0x7f50b81621e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50c07ee640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b806ac60 con 0x7f50b80cc6d0 2023-11-08T19:28:31.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50c07ee640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50b80ccfd0 con 0x7f50b8144690 2023-11-08T19:28:31.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50c07ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50b8165de0 con 0x7f50b80ccb40 2023-11-08T19:28:31.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50bdd89640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f50b80cc6d0 0x7f50b8161ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40102/0 (socket says 172.21.15.119:40102) 2023-11-08T19:28:31.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50bdd89640 1 -- 172.21.15.119:0/2976774917 learned_addr learned my addr 172.21.15.119:0/2976774917 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:31.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.969+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160326640 0 0) 0x7f50b806ac60 con 0x7f50b80cc6d0 2023-11-08T19:28:31.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bed8b640 1 --2- 172.21.15.119:0/2976774917 >> v2:172.21.15.188:3300/0 conn(0x7f50b80ccb40 0x7f50b81621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50a4003520 con 0x7f50b80cc6d0 2023-11-08T19:28:31.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1356193545 0 0) 0x7f50a4003520 con 0x7f50b80cc6d0 2023-11-08T19:28:31.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 >> v2:172.21.15.188:3300/0 conn(0x7f50b80ccb40 msgr2=0x7f50b81621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 --2- 172.21.15.119:0/2976774917 >> v2:172.21.15.188:3300/0 conn(0x7f50b80ccb40 0x7f50b81621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 >> v1:172.21.15.188:6789/0 conn(0x7f50b8144690 legacy=0x7f50b806a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b8165160 con 0x7f50b80cc6d0 2023-11-08T19:28:31.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f50a8001670 con 0x7f50b80cc6d0 2023-11-08T19:28:31.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 >> v1:172.21.15.119:6789/0 conn(0x7f50b80cc6d0 legacy=0x7f50b8161ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50a4003970 0x7f50a4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a40047c0 0x7f50a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a4005180 0x7f50a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8165de0 con 0x7f50a40047c0 2023-11-08T19:28:31.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a400c0d0 con 0x7f50a4003970 2023-11-08T19:28:31.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50a400c290 con 0x7f50a4005180 2023-11-08T19:28:31.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.970+0000 7f50bdd89640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a40047c0 0x7f50a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50be58a640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50a4003970 0x7f50a4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bed8b640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a4005180 0x7f50a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bdd89640 1 -- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a4005180 msgr2=0x7f50a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bdd89640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a4005180 0x7f50a400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bdd89640 1 -- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50a4003970 msgr2=0x7f50a4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bdd89640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50a4003970 0x7f50a4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bdd89640 1 -- 172.21.15.119:0/2976774917 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a4003520 con 0x7f50a40047c0 2023-11-08T19:28:31.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50bed8b640 1 -- 172.21.15.119:0/2976774917 reap_dead start 2023-11-08T19:28:31.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.971+0000 7f50be58a640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50a4003970 0x7f50a4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.972+0000 7f50bdd89640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a40047c0 0x7f50a4004c40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f50a800c240 tx=0x7f50a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e701df58bfb259 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.972+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8005ad0 con 0x7f50a40047c0 2023-11-08T19:28:31.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.972+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a8005c30 con 0x7f50a40047c0 2023-11-08T19:28:31.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.972+0000 7f50bd588640 1 -- 172.21.15.119:0/2976774917 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a802cab0 con 0x7f50a40047c0 2023-11-08T19:28:31.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 -- 172.21.15.119:0/2976774917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a40047c0 msgr2=0x7f50a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a40047c0 0x7f50a4004c40 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f50a800c240 tx=0x7f50a40057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 -- 172.21.15.119:0/2976774917 shutdown_connections 2023-11-08T19:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 --2- 172.21.15.119:0/2976774917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50a40047c0 0x7f50a4004c40 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 -- 172.21.15.119:0/2976774917 >> 172.21.15.119:0/2976774917 conn(0x7f50b80cb610 msgr2=0x7f50b80cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 -- 172.21.15.119:0/2976774917 shutdown_connections 2023-11-08T19:28:31.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.973+0000 7f50c07ee640 1 -- 172.21.15.119:0/2976774917 wait complete. 2023-11-08T19:28:31.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.975+0000 7f50c07ee640 1 Processor -- start 2023-11-08T19:28:31.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.975+0000 7f50c07ee640 1 -- start start 2023-11-08T19:28:31.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50c07ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a40047c0 0x7f50b8142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50c07ee640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 0x7f50b80b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50c07ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b8070980 0x7f50b8070e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50c07ee640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b8071340 con 0x7f50b80b4550 2023-11-08T19:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50c07ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80c5f00 con 0x7f50b8070980 2023-11-08T19:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50c07ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50b80c61e0 con 0x7f50a40047c0 2023-11-08T19:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50bdd89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 0x7f50b80b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50bdd89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 0x7f50b80b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35026/0 (socket says 172.21.15.119:35026) 2023-11-08T19:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.976+0000 7f50bdd89640 1 -- 172.21.15.119:0/4284363728 learned_addr learned my addr 172.21.15.119:0/4284363728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50be58a640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a40047c0 0x7f50b8142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bed8b640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b8070980 0x7f50b8070e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bdd89640 1 -- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a40047c0 msgr2=0x7f50b8142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bdd89640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a40047c0 0x7f50b8142c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bdd89640 1 -- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b8070980 msgr2=0x7f50b8070e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bdd89640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b8070980 0x7f50b8070e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bdd89640 1 -- 172.21.15.119:0/4284363728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b80c64c0 con 0x7f50b80b4550 2023-11-08T19:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bed8b640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b8070980 0x7f50b8070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50bdd89640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 0x7f50b80b49d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f50a800c240 tx=0x7f50a8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a8044070 con 0x7f50b80b4550 2023-11-08T19:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a803a050 con 0x7f50b80b4550 2023-11-08T19:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.977+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a80049e0 con 0x7f50b80b4550 2023-11-08T19:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.978+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b80c67a0 con 0x7f50b80b4550 2023-11-08T19:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.978+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b80c5bd0 con 0x7f50b80b4550 2023-11-08T19:28:31.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.979+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b806ae90 con 0x7f50b80b4550 2023-11-08T19:28:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.980+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f50a8032070 con 0x7f50b80b4550 2023-11-08T19:28:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.980+0000 7f50aeffd640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f509c06ae70 0x7f509c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.981+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f50a80ab600 con 0x7f50b80b4550 2023-11-08T19:28:31.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.981+0000 7f50be58a640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f509c06ae70 0x7f509c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.981+0000 7f50be58a640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f509c06ae70 0x7f509c06d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000c00 tx=0x7f50b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:31.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:31.985+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50a8076130 con 0x7f50b80b4550 2023-11-08T19:28:32.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.174+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f50b806cd50 con 0x7f50b80b4550 2023-11-08T19:28:32.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.175+0000 7f50aeffd640 1 -- 172.21.15.119:0/4284363728 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f50a8079750 con 0x7f50b80b4550 2023-11-08T19:28:32.177 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:32.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f509c06ae70 msgr2=0x7f509c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f509c06ae70 0x7f509c06d330 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f50b4000c00 tx=0x7f50b4009040 comp rx=0 tx=0).stop 2023-11-08T19:28:32.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 msgr2=0x7f50b80b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 0x7f50b80b49d0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f50a800c240 tx=0x7f50a8001a00 comp rx=0 tx=0).stop 2023-11-08T19:28:32.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 shutdown_connections 2023-11-08T19:28:32.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f509c06ae70 0x7f509c06d330 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50b8070980 0x7f50b8070e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f50b80b4550 0x7f50b80b49d0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 --2- 172.21.15.119:0/4284363728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50a40047c0 0x7f50b8142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 >> 172.21.15.119:0/4284363728 conn(0x7f50b80cb610 msgr2=0x7f50b806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:32.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 shutdown_connections 2023-11-08T19:28:32.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.178+0000 7f50c07ee640 1 -- 172.21.15.119:0/4284363728 wait complete. 2023-11-08T19:28:32.197 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:32.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.415+0000 7f712e155640 1 Processor -- start 2023-11-08T19:28:32.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.415+0000 7f712e155640 1 -- start start 2023-11-08T19:28:32.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f712e155640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f71280c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f712e155640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f712806d250 0x7f712815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f712e155640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712815e440 con 0x7f712815d9f0 2023-11-08T19:28:32.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f712e155640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71280c4f50 con 0x7f712806d250 2023-11-08T19:28:32.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f712e155640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71280c4960 con 0x7f7128143600 2023-11-08T19:28:32.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f7127fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f712806d250 0x7f712815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f7127fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f712806d250 0x7f712815de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35034/0 (socket says 172.21.15.119:35034) 2023-11-08T19:28:32.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.417+0000 7f7127fff640 1 -- 172.21.15.119:0/4139841017 learned_addr learned my addr 172.21.15.119:0/4139841017 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:32.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.418+0000 7f71277fe640 1 --2- 172.21.15.119:0/4139841017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f71280c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.418+0000 7f7127fff640 1 -- 172.21.15.119:0/4139841017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 msgr2=0x7f71280c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:32.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.418+0000 7f7127fff640 1 --2- 172.21.15.119:0/4139841017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f71280c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.418+0000 7f7127fff640 1 -- 172.21.15.119:0/4139841017 >> v1:172.21.15.188:6789/0 conn(0x7f712815d9f0 legacy=0x7f712806cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:32.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.418+0000 7f7127fff640 1 -- 172.21.15.119:0/4139841017 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712815e720 con 0x7f712806d250 2023-11-08T19:28:32.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.419+0000 7f7127fff640 1 --2- 172.21.15.119:0/4139841017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f712815de30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7120009e50 tx=0x7f712002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f92bf41384a2f79 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:32.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.419+0000 7f71267fc640 1 -- 172.21.15.119:0/4139841017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712002c9e0 con 0x7f712806d250 2023-11-08T19:28:32.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.419+0000 7f71267fc640 1 -- 172.21.15.119:0/4139841017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712002cb40 con 0x7f712806d250 2023-11-08T19:28:32.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.419+0000 7f71267fc640 1 -- 172.21.15.119:0/4139841017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200343f0 con 0x7f712806d250 2023-11-08T19:28:32.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 -- 172.21.15.119:0/4139841017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 msgr2=0x7f712815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 --2- 172.21.15.119:0/4139841017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f712815de30 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7120009e50 tx=0x7f712002be40 comp rx=0 tx=0).stop 2023-11-08T19:28:32.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 -- 172.21.15.119:0/4139841017 shutdown_connections 2023-11-08T19:28:32.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 --2- 172.21.15.119:0/4139841017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f712815de30 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 --2- 172.21.15.119:0/4139841017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f71280c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 -- 172.21.15.119:0/4139841017 >> 172.21.15.119:0/4139841017 conn(0x7f71280cb610 msgr2=0x7f71280cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:32.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 -- 172.21.15.119:0/4139841017 shutdown_connections 2023-11-08T19:28:32.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.420+0000 7f712e155640 1 -- 172.21.15.119:0/4139841017 wait complete. 2023-11-08T19:28:32.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.422+0000 7f712e155640 1 Processor -- start 2023-11-08T19:28:32.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.422+0000 7f712e155640 1 -- start start 2023-11-08T19:28:32.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f712e155640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f71281f1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f712e155640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f712813ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f712e155640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f712815d9f0 0x7f712813d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f712e155640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71281f1790 con 0x7f712806d250 2023-11-08T19:28:32.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f712e155640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712813c6b0 con 0x7f712815d9f0 2023-11-08T19:28:32.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f712e155640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712813f170 con 0x7f7128143600 2023-11-08T19:28:32.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f71281f1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f71281f1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35040/0 (socket says 172.21.15.119:35040) 2023-11-08T19:28:32.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 -- 172.21.15.119:0/2522950736 learned_addr learned my addr 172.21.15.119:0/2522950736 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:32.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 -- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 msgr2=0x7f712813ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f7127fff640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f712815d9f0 0x7f712813d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f7126ffd640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f712813ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f712813ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 -- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f712815d9f0 msgr2=0x7f712813d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f712815d9f0 0x7f712813d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.423+0000 7f71277fe640 1 -- 172.21.15.119:0/2522950736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120009b30 con 0x7f712806d250 2023-11-08T19:28:32.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.424+0000 7f71277fe640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f71281f1250 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f711800d830 tx=0x7f711800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:32.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.424+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800b6c0 con 0x7f712806d250 2023-11-08T19:28:32.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.424+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800b820 con 0x7f712806d250 2023-11-08T19:28:32.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.424+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712813c900 con 0x7f712806d250 2023-11-08T19:28:32.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.424+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711800bbb0 con 0x7f712806d250 2023-11-08T19:28:32.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.424+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712813cec0 con 0x7f712806d250 2023-11-08T19:28:32.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.425+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128074430 con 0x7f712806d250 2023-11-08T19:28:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.426+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f711801e460 con 0x7f712806d250 2023-11-08T19:28:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.427+0000 7f710ffff640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f711406ae70 0x7f711406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.427+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f711808a330 con 0x7f712806d250 2023-11-08T19:28:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.428+0000 7f7126ffd640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f711406ae70 0x7f711406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.428+0000 7f7126ffd640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f711406ae70 0x7f711406d330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f712813f370 tx=0x7f711c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:32.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.433+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7118054e70 con 0x7f712806d250 2023-11-08T19:28:32.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.623+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f712813f3a0 con 0x7f712806d250 2023-11-08T19:28:32.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.623+0000 7f710ffff640 1 -- 172.21.15.119:0/2522950736 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v83) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7118058490 con 0x7f712806d250 2023-11-08T19:28:32.625 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:32.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f711406ae70 msgr2=0x7f711406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f711406ae70 0x7f711406d330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f712813f370 tx=0x7f711c008040 comp rx=0 tx=0).stop 2023-11-08T19:28:32.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 msgr2=0x7f71281f1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f71281f1250 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f711800d830 tx=0x7f711800dcf0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 shutdown_connections 2023-11-08T19:28:32.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f711406ae70 0x7f711406d330 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f712815d9f0 0x7f712813d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7128143600 0x7f712813ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 --2- 172.21.15.119:0/2522950736 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f712806d250 0x7f71281f1250 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 >> 172.21.15.119:0/2522950736 conn(0x7f71280cb610 msgr2=0x7f71280cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:32.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 shutdown_connections 2023-11-08T19:28:32.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.626+0000 7f712e155640 1 -- 172.21.15.119:0/2522950736 wait complete. 2023-11-08T19:28:32.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:32.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.864+0000 7f24db2e5640 1 Processor -- start 2023-11-08T19:28:32.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.864+0000 7f24db2e5640 1 -- start start 2023-11-08T19:28:32.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.866+0000 7f24db2e5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24d4161870 0x7f24d40cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.866+0000 7f24db2e5640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d415df50 con 0x7f24d415d4a0 2023-11-08T19:28:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.866+0000 7f24db2e5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d40d1e60 con 0x7f24d40cfa00 2023-11-08T19:28:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.866+0000 7f24db2e5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24d40d03b0 con 0x7f24d4161870 2023-11-08T19:28:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.866+0000 7f24d9081640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f24d415d4a0 0x7f24d40d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40106/0 (socket says 172.21.15.119:40106) 2023-11-08T19:28:32.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.866+0000 7f24d9081640 1 -- 172.21.15.119:0/2073848065 learned_addr learned my addr 172.21.15.119:0/2073848065 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777250727 0 0) 0x7f24d415df50 con 0x7f24d415d4a0 2023-11-08T19:28:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c0003520 con 0x7f24d415d4a0 2023-11-08T19:28:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24d9882640 1 --2- 172.21.15.119:0/2073848065 >> v2:172.21.15.188:3300/0 conn(0x7f24d4161870 0x7f24d40cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2752395988 0 0) 0x7f24c0003520 con 0x7f24d415d4a0 2023-11-08T19:28:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 >> v2:172.21.15.188:3300/0 conn(0x7f24d4161870 msgr2=0x7f24d40cfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:32.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 --2- 172.21.15.119:0/2073848065 >> v2:172.21.15.188:3300/0 conn(0x7f24d4161870 0x7f24d40cfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 >> v1:172.21.15.188:6789/0 conn(0x7f24d40cfa00 legacy=0x7f24d4161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d4161d40 con 0x7f24d415d4a0 2023-11-08T19:28:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f24d0001670 con 0x7f24d415d4a0 2023-11-08T19:28:32.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 >> v1:172.21.15.119:6789/0 conn(0x7f24d415d4a0 legacy=0x7f24d40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24c0003970 0x7f24c0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24c00047c0 0x7f24c0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24c000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40d03b0 con 0x7f24c00052a0 2023-11-08T19:28:32.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c000c150 con 0x7f24c0003970 2023-11-08T19:28:32.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.867+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c000c310 con 0x7f24c00047c0 2023-11-08T19:28:32.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9882640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24c000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9081640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24c0003970 0x7f24c0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d8880640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24c00047c0 0x7f24c0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9882640 1 -- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24c00047c0 msgr2=0x7f24c0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:32.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9882640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24c00047c0 0x7f24c0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9882640 1 -- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24c0003970 msgr2=0x7f24c0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:32.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9882640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24c0003970 0x7f24c0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.868+0000 7f24d9882640 1 -- 172.21.15.119:0/2073848065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0003520 con 0x7f24c00052a0 2023-11-08T19:28:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.869+0000 7f24d9882640 1 -- 172.21.15.119:0/2073848065 reap_dead start 2023-11-08T19:28:32.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.869+0000 7f24d9882640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24c000b980 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ba00 tx=0x7f24c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22c7a51c37615a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:32.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.869+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc004020 con 0x7f24c00052a0 2023-11-08T19:28:32.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.869+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc004180 con 0x7f24c00052a0 2023-11-08T19:28:32.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.869+0000 7f24c7fff640 1 -- 172.21.15.119:0/2073848065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc003610 con 0x7f24c00052a0 2023-11-08T19:28:32.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 -- 172.21.15.119:0/2073848065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 msgr2=0x7f24c000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24c000b980 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f24cc00ba00 tx=0x7f24c00057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 -- 172.21.15.119:0/2073848065 shutdown_connections 2023-11-08T19:28:32.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 --2- 172.21.15.119:0/2073848065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24c000b980 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 -- 172.21.15.119:0/2073848065 >> 172.21.15.119:0/2073848065 conn(0x7f24d406a9f0 msgr2=0x7f24d406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:32.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 -- 172.21.15.119:0/2073848065 shutdown_connections 2023-11-08T19:28:32.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.870+0000 7f24db2e5640 1 -- 172.21.15.119:0/2073848065 wait complete. 2023-11-08T19:28:32.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.872+0000 7f24db2e5640 1 Processor -- start 2023-11-08T19:28:32.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.872+0000 7f24db2e5640 1 -- start start 2023-11-08T19:28:32.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24db2e5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24d41f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24db2e5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40ab3f0 0x7f24d40cc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24db2e5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d414f670 0x7f24d40cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24db2e5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d41f6f50 con 0x7f24c00052a0 2023-11-08T19:28:32.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24db2e5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d41f7230 con 0x7f24d414f670 2023-11-08T19:28:32.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24db2e5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d41f7510 con 0x7f24d40ab3f0 2023-11-08T19:28:32.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24d9081640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24d41f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24d9081640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24d41f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35062/0 (socket says 172.21.15.119:35062) 2023-11-08T19:28:32.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.873+0000 7f24d9081640 1 -- 172.21.15.119:0/1775240050 learned_addr learned my addr 172.21.15.119:0/1775240050 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:32.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d8880640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40ab3f0 0x7f24d40cc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9882640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d414f670 0x7f24d40cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9081640 1 -- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40ab3f0 msgr2=0x7f24d40cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9081640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40ab3f0 0x7f24d40cc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9081640 1 -- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d414f670 msgr2=0x7f24d40cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:32.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9081640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d414f670 0x7f24d40cc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:32.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9081640 1 -- 172.21.15.119:0/1775240050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d40abdb0 con 0x7f24c00052a0 2023-11-08T19:28:32.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24d9081640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24d41f6650 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f24d000e7b0 tx=0x7f24d000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:32.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d000eec0 con 0x7f24c00052a0 2023-11-08T19:28:32.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d000c8d0 con 0x7f24c00052a0 2023-11-08T19:28:32.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d40ac090 con 0x7f24c00052a0 2023-11-08T19:28:32.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.874+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d000cc60 con 0x7f24c00052a0 2023-11-08T19:28:32.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.875+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d40a9980 con 0x7f24c00052a0 2023-11-08T19:28:32.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.876+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d40ac2d0 con 0x7f24c00052a0 2023-11-08T19:28:32.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.877+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24d001d460 con 0x7f24c00052a0 2023-11-08T19:28:32.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.877+0000 7f24c5ffb640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24b806aea0 0x7f24b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:32.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.878+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f24d0089c20 con 0x7f24c00052a0 2023-11-08T19:28:32.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.878+0000 7f24d8880640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24b806aea0 0x7f24b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:32.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.878+0000 7f24d8880640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24b806aea0 0x7f24b806d360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007fd0 tx=0x7f24c8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:32.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:32.882+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24d40ac2d0 con 0x7f24c00052a0 2023-11-08T19:28:33.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.071+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f24d40ac2d0 con 0x7f24c00052a0 2023-11-08T19:28:33.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.072+0000 7f24c5ffb640 1 -- 172.21.15.119:0/1775240050 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f24d40ac2d0 con 0x7f24c00052a0 2023-11-08T19:28:33.074 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:33.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24b806aea0 msgr2=0x7f24b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24b806aea0 0x7f24b806d360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f24c8007fd0 tx=0x7f24c8009040 comp rx=0 tx=0).stop 2023-11-08T19:28:33.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 msgr2=0x7f24d41f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24d41f6650 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f24d000e7b0 tx=0x7f24d000ec70 comp rx=0 tx=0).stop 2023-11-08T19:28:33.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 shutdown_connections 2023-11-08T19:28:33.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24b806aea0 0x7f24b806d360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24d414f670 0x7f24d40cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24d40ab3f0 0x7f24d40cc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 --2- 172.21.15.119:0/1775240050 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24c00052a0 0x7f24d41f6650 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.075+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 >> 172.21.15.119:0/1775240050 conn(0x7f24d406a9f0 msgr2=0x7f24d406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:33.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.076+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 shutdown_connections 2023-11-08T19:28:33.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.076+0000 7f24db2e5640 1 -- 172.21.15.119:0/1775240050 wait complete. 2023-11-08T19:28:33.096 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:28:33.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.313+0000 7f81c6264640 1 Processor -- start 2023-11-08T19:28:33.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.313+0000 7f81c6264640 1 -- start start 2023-11-08T19:28:33.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.314+0000 7f81c6264640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81c01596b0 0x7f81c006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.314+0000 7f81c6264640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c006bf30 con 0x7f81c0070040 2023-11-08T19:28:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.314+0000 7f81c6264640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81c006c210 con 0x7f81c006ba30 2023-11-08T19:28:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.314+0000 7f81c6264640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81c006cbb0 con 0x7f81c01596b0 2023-11-08T19:28:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81c4801640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f81c0070040 0x7f81c00c9360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40118/0 (socket says 172.21.15.119:40118) 2023-11-08T19:28:33.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81c4801640 1 -- 172.21.15.119:0/2831135437 learned_addr learned my addr 172.21.15.119:0/2831135437 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bffff640 1 --2- 172.21.15.119:0/2831135437 >> v2:172.21.15.188:3300/0 conn(0x7f81c01596b0 0x7f81c006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569586310 0 0) 0x7f81c006bf30 con 0x7f81c0070040 2023-11-08T19:28:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81ac003520 con 0x7f81c0070040 2023-11-08T19:28:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172736017 0 0) 0x7f81ac003520 con 0x7f81c0070040 2023-11-08T19:28:33.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 >> v2:172.21.15.188:3300/0 conn(0x7f81c01596b0 msgr2=0x7f81c006cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 --2- 172.21.15.119:0/2831135437 >> v2:172.21.15.188:3300/0 conn(0x7f81c01596b0 0x7f81c006cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 >> v1:172.21.15.188:6789/0 conn(0x7f81c006ba30 legacy=0x7f81c006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c006c4f0 con 0x7f81c0070040 2023-11-08T19:28:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.315+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f81b80019f0 con 0x7f81c0070040 2023-11-08T19:28:33.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 >> v1:172.21.15.119:6789/0 conn(0x7f81c0070040 legacy=0x7f81c00c9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81ac0039c0 0x7f81ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0047c0 0x7f81ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0051d0 0x7f81ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c006cbb0 con 0x7f81ac0047c0 2023-11-08T19:28:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac00c120 con 0x7f81ac0039c0 2023-11-08T19:28:33.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac00e4f0 con 0x7f81ac0051d0 2023-11-08T19:28:33.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81b7fff640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0047c0 0x7f81ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81bffff640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81ac0039c0 0x7f81ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81c4801640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0051d0 0x7f81ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81b7fff640 1 -- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0051d0 msgr2=0x7f81ac00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:33.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81b7fff640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0051d0 0x7f81ac00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81b7fff640 1 -- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81ac0039c0 msgr2=0x7f81ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:33.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81b7fff640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81ac0039c0 0x7f81ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81b7fff640 1 -- 172.21.15.119:0/2831135437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac003520 con 0x7f81ac0047c0 2023-11-08T19:28:33.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81c4801640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0051d0 0x7f81ac00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:33.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.316+0000 7f81c4801640 1 -- 172.21.15.119:0/2831135437 reap_dead start 2023-11-08T19:28:33.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81b7fff640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0047c0 0x7f81ac004c40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f81a8001720 tx=0x7f81ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48b400fd20b28f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:33.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a80052d0 con 0x7f81ac0047c0 2023-11-08T19:28:33.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a802e050 con 0x7f81ac0047c0 2023-11-08T19:28:33.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81bf7fe640 1 -- 172.21.15.119:0/2831135437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8003830 con 0x7f81ac0047c0 2023-11-08T19:28:33.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81c6264640 1 -- 172.21.15.119:0/2831135437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0047c0 msgr2=0x7f81ac004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81c6264640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0047c0 0x7f81ac004c40 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f81a8001720 tx=0x7f81ac0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81c6264640 1 -- 172.21.15.119:0/2831135437 shutdown_connections 2023-11-08T19:28:33.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81c6264640 1 --2- 172.21.15.119:0/2831135437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81ac0047c0 0x7f81ac004c40 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.318+0000 7f81c6264640 1 -- 172.21.15.119:0/2831135437 >> 172.21.15.119:0/2831135437 conn(0x7f81c00cb610 msgr2=0x7f81c00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:33.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.319+0000 7f81c6264640 1 -- 172.21.15.119:0/2831135437 shutdown_connections 2023-11-08T19:28:33.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.319+0000 7f81c6264640 1 -- 172.21.15.119:0/2831135437 wait complete. 2023-11-08T19:28:33.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.320+0000 7f81c6264640 1 Processor -- start 2023-11-08T19:28:33.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.320+0000 7f81c6264640 1 -- start start 2023-11-08T19:28:33.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.321+0000 7f81c6264640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0047c0 0x7f81c00b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.321+0000 7f81c6264640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 0x7f81c0147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.321+0000 7f81c6264640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c00c4aa0 0x7f81c00c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.321+0000 7f81c6264640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c015e170 con 0x7f81c0159a90 2023-11-08T19:28:33.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.321+0000 7f81c6264640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c015ca80 con 0x7f81c00c4aa0 2023-11-08T19:28:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.321+0000 7f81c6264640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81c00b2840 con 0x7f81ac0047c0 2023-11-08T19:28:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 0x7f81c0147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 0x7f81c0147510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35088/0 (socket says 172.21.15.119:35088) 2023-11-08T19:28:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 -- 172.21.15.119:0/713356720 learned_addr learned my addr 172.21.15.119:0/713356720 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:33.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81c4801640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c00c4aa0 0x7f81c00c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81bffff640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0047c0 0x7f81c00b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 -- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0047c0 msgr2=0x7f81c00b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0047c0 0x7f81c00b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 -- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c00c4aa0 msgr2=0x7f81c00c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c00c4aa0 0x7f81c00c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 -- 172.21.15.119:0/713356720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00b1d70 con 0x7f81c0159a90 2023-11-08T19:28:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.322+0000 7f81b7fff640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 0x7f81c0147510 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f81a802bbc0 tx=0x7f81a8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.323+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8007640 con 0x7f81c0159a90 2023-11-08T19:28:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.323+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a80069e0 con 0x7f81c0159a90 2023-11-08T19:28:33.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.323+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a8006d70 con 0x7f81c0159a90 2023-11-08T19:28:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.323+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c00b2c40 con 0x7f81c0159a90 2023-11-08T19:28:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.323+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c00b2280 con 0x7f81c0159a90 2023-11-08T19:28:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.324+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c00b2a10 con 0x7f81c0159a90 2023-11-08T19:28:33.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.325+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81a80035f0 con 0x7f81c0159a90 2023-11-08T19:28:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.326+0000 7f81bd7fa640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a006aea0 0x7f81a006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.326+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f81a80ab8b0 con 0x7f81c0159a90 2023-11-08T19:28:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.326+0000 7f81bffff640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a006aea0 0x7f81a006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.327+0000 7f81bffff640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a006aea0 0x7f81a006d360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f81b0006fd0 tx=0x7f81b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:33.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.331+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81c00b2a10 con 0x7f81c0159a90 2023-11-08T19:28:33.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.520+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81c00b2a10 con 0x7f81c0159a90 2023-11-08T19:28:33.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.521+0000 7f81bd7fa640 1 -- 172.21.15.119:0/713356720 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81c00b2a10 con 0x7f81c0159a90 2023-11-08T19:28:33.522 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:33.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a006aea0 msgr2=0x7f81a006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a006aea0 0x7f81a006d360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f81b0006fd0 tx=0x7f81b0008040 comp rx=0 tx=0).stop 2023-11-08T19:28:33.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 msgr2=0x7f81c0147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 0x7f81c0147510 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f81a802bbc0 tx=0x7f81a8001120 comp rx=0 tx=0).stop 2023-11-08T19:28:33.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 shutdown_connections 2023-11-08T19:28:33.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81a006aea0 0x7f81a006d360 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81c00c4aa0 0x7f81c00c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81c0159a90 0x7f81c0147510 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 --2- 172.21.15.119:0/713356720 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81ac0047c0 0x7f81c00b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.523+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 >> 172.21.15.119:0/713356720 conn(0x7f81c00cb610 msgr2=0x7f81c0071230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:33.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.524+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 shutdown_connections 2023-11-08T19:28:33.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.524+0000 7f81c6264640 1 -- 172.21.15.119:0/713356720 wait complete. 2023-11-08T19:28:33.544 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:28:33.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.762+0000 7efe92658640 1 Processor -- start 2023-11-08T19:28:33.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.762+0000 7efe92658640 1 -- start start 2023-11-08T19:28:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe92658640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe8c143600 0x7efe8c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe92658640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe8c15d9f0 0x7efe8c15de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe92658640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c06cb40 0x7efe8c06cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe92658640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8c0c4f50 con 0x7efe8c143600 2023-11-08T19:28:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe92658640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8c0c4960 con 0x7efe8c15d9f0 2023-11-08T19:28:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe92658640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c06d4f0 con 0x7efe8c06cb40 2023-11-08T19:28:33.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe8c143600 0x7efe8c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe8c143600 0x7efe8c0c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35092/0 (socket says 172.21.15.119:35092) 2023-11-08T19:28:33.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 -- 172.21.15.119:0/1758865559 learned_addr learned my addr 172.21.15.119:0/1758865559 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:33.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 -- 172.21.15.119:0/1758865559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c06cb40 msgr2=0x7efe8c06cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:33.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 --2- 172.21.15.119:0/1758865559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c06cb40 0x7efe8c06cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 -- 172.21.15.119:0/1758865559 >> v2:172.21.15.188:3300/0 conn(0x7efe8c15d9f0 msgr2=0x7efe8c15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:33.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 --2- 172.21.15.119:0/1758865559 >> v2:172.21.15.188:3300/0 conn(0x7efe8c15d9f0 0x7efe8c15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.764+0000 7efe8bfff640 1 -- 172.21.15.119:0/1758865559 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c15e440 con 0x7efe8c143600 2023-11-08T19:28:33.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.765+0000 7efe8bfff640 1 --2- 172.21.15.119:0/1758865559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c143600 0x7efe8c0c5370 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0c91e0 tx=0x7efe7c02b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9234618d9111fcea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:33.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.766+0000 7efe8b7fe640 1 -- 172.21.15.119:0/1758865559 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0317d0 con 0x7efe8c143600 2023-11-08T19:28:33.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.766+0000 7efe8b7fe640 1 -- 172.21.15.119:0/1758865559 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c031930 con 0x7efe8c143600 2023-11-08T19:28:33.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.766+0000 7efe8b7fe640 1 -- 172.21.15.119:0/1758865559 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c031cc0 con 0x7efe8c143600 2023-11-08T19:28:33.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.766+0000 7efe92658640 1 -- 172.21.15.119:0/1758865559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c143600 msgr2=0x7efe8c0c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.766+0000 7efe92658640 1 --2- 172.21.15.119:0/1758865559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c143600 0x7efe8c0c5370 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0c91e0 tx=0x7efe7c02b870 comp rx=0 tx=0).stop 2023-11-08T19:28:33.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 -- 172.21.15.119:0/1758865559 shutdown_connections 2023-11-08T19:28:33.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 --2- 172.21.15.119:0/1758865559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c06cb40 0x7efe8c06cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 --2- 172.21.15.119:0/1758865559 >> v2:172.21.15.188:3300/0 conn(0x7efe8c15d9f0 0x7efe8c15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 --2- 172.21.15.119:0/1758865559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c143600 0x7efe8c0c5370 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 -- 172.21.15.119:0/1758865559 >> 172.21.15.119:0/1758865559 conn(0x7efe8c0cb610 msgr2=0x7efe8c0cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:33.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 -- 172.21.15.119:0/1758865559 shutdown_connections 2023-11-08T19:28:33.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.767+0000 7efe92658640 1 -- 172.21.15.119:0/1758865559 wait complete. 2023-11-08T19:28:33.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.768+0000 7efe92658640 1 Processor -- start 2023-11-08T19:28:33.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.768+0000 7efe92658640 1 -- start start 2023-11-08T19:28:33.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.769+0000 7efe92658640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 0x7efe8c1f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.769+0000 7efe92658640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c15d9f0 0x7efe8c0b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.769+0000 7efe92658640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe8c0c85d0 0x7efe8c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.769+0000 7efe92658640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c1f50c0 con 0x7efe8c06cb40 2023-11-08T19:28:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.769+0000 7efe92658640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c1f53a0 con 0x7efe8c0c85d0 2023-11-08T19:28:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.769+0000 7efe92658640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c1f5680 con 0x7efe8c15d9f0 2023-11-08T19:28:33.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 0x7efe8c1f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 0x7efe8c1f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35106/0 (socket says 172.21.15.119:35106) 2023-11-08T19:28:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 -- 172.21.15.119:0/2683364609 learned_addr learned my addr 172.21.15.119:0/2683364609 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 -- 172.21.15.119:0/2683364609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c15d9f0 msgr2=0x7efe8c0b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c15d9f0 0x7efe8c0b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 -- 172.21.15.119:0/2683364609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe8c0c85d0 msgr2=0x7efe8c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe8c0c85d0 0x7efe8c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.770+0000 7efe8bfff640 1 -- 172.21.15.119:0/2683364609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7c0094d0 con 0x7efe8c06cb40 2023-11-08T19:28:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.771+0000 7efe8bfff640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 0x7efe8c1f4890 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7efe7c000c00 tx=0x7efe7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:33.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.771+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0329f0 con 0x7efe8c06cb40 2023-11-08T19:28:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.771+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c032b50 con 0x7efe8c06cb40 2023-11-08T19:28:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.771+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0453f0 con 0x7efe8c06cb40 2023-11-08T19:28:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.771+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c1f5960 con 0x7efe8c06cb40 2023-11-08T19:28:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.771+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c1f4dd0 con 0x7efe8c06cb40 2023-11-08T19:28:33.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.772+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c1f5e80 con 0x7efe8c06cb40 2023-11-08T19:28:33.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.773+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe7c030070 con 0x7efe8c06cb40 2023-11-08T19:28:33.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.774+0000 7efe897fa640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe6c06ae70 0x7efe6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:33.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.774+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7efe7c0ab900 con 0x7efe8c06cb40 2023-11-08T19:28:33.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.774+0000 7efe83fff640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe6c06ae70 0x7efe6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:33.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.775+0000 7efe83fff640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe6c06ae70 0x7efe6c06d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0b4650 tx=0x7efe74006bf0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:33.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.778+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe7c0763a0 con 0x7efe8c06cb40 2023-11-08T19:28:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.970+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efe8c0c8a20 con 0x7efe8c06cb40 2023-11-08T19:28:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.971+0000 7efe897fa640 1 -- 172.21.15.119:0/2683364609 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7efe7c0799c0 con 0x7efe8c06cb40 2023-11-08T19:28:33.973 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:28:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe6c06ae70 msgr2=0x7efe6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe6c06ae70 0x7efe6c06d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0b4650 tx=0x7efe74006bf0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 msgr2=0x7efe8c1f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 0x7efe8c1f4890 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7efe7c000c00 tx=0x7efe7c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 shutdown_connections 2023-11-08T19:28:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe6c06ae70 0x7efe6c06d330 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe8c0c85d0 0x7efe8c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe8c15d9f0 0x7efe8c0b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 --2- 172.21.15.119:0/2683364609 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe8c06cb40 0x7efe8c1f4890 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.974+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 >> 172.21.15.119:0/2683364609 conn(0x7efe8c0cb610 msgr2=0x7efe8c0c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.975+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 shutdown_connections 2023-11-08T19:28:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:33.975+0000 7efe92658640 1 -- 172.21.15.119:0/2683364609 wait complete. 2023-11-08T19:28:33.994 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:34.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.202+0000 7f121b933640 1 Processor -- start 2023-11-08T19:28:34.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.202+0000 7f121b933640 1 -- start start 2023-11-08T19:28:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.204+0000 7f121b933640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415b080 0x7f12140ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.204+0000 7f121b933640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1214161250 0x7f12141616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.204+0000 7f121b933640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12140cda70 con 0x7f12140cf8c0 2023-11-08T19:28:34.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.204+0000 7f121b933640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1214161fc0 con 0x7f1214161250 2023-11-08T19:28:34.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.204+0000 7f121b933640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140cdcc0 con 0x7f121415b080 2023-11-08T19:28:34.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.204+0000 7f1219ed0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1214161250 0x7f12141616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f1219ed0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1214161250 0x7f12141616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35108/0 (socket says 172.21.15.119:35108) 2023-11-08T19:28:34.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f1219ed0640 1 -- 172.21.15.119:0/949428461 learned_addr learned my addr 172.21.15.119:0/949428461 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:34.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f12196cf640 1 --2- 172.21.15.119:0/949428461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415b080 0x7f12140ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f1219ed0640 1 -- 172.21.15.119:0/949428461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415b080 msgr2=0x7f12140ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:34.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f1219ed0640 1 --2- 172.21.15.119:0/949428461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415b080 0x7f12140ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f1219ed0640 1 -- 172.21.15.119:0/949428461 >> v1:172.21.15.188:6789/0 conn(0x7f12140cf8c0 legacy=0x7f12140cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.205+0000 7f1219ed0640 1 -- 172.21.15.119:0/949428461 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140cdfa0 con 0x7f1214161250 2023-11-08T19:28:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.206+0000 7f1219ed0640 1 --2- 172.21.15.119:0/949428461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1214161250 0x7f12141616c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f120c009e50 tx=0x7f120c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184490e299f62384 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:34.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.207+0000 7f1207fff640 1 -- 172.21.15.119:0/949428461 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c02c9e0 con 0x7f1214161250 2023-11-08T19:28:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.207+0000 7f1207fff640 1 -- 172.21.15.119:0/949428461 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c02cb40 con 0x7f1214161250 2023-11-08T19:28:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.207+0000 7f1207fff640 1 -- 172.21.15.119:0/949428461 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c0343f0 con 0x7f1214161250 2023-11-08T19:28:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.207+0000 7f121b933640 1 -- 172.21.15.119:0/949428461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1214161250 msgr2=0x7f12141616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.207+0000 7f121b933640 1 --2- 172.21.15.119:0/949428461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1214161250 0x7f12141616c0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f120c009e50 tx=0x7f120c02be40 comp rx=0 tx=0).stop 2023-11-08T19:28:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.207+0000 7f121b933640 1 -- 172.21.15.119:0/949428461 shutdown_connections 2023-11-08T19:28:34.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.208+0000 7f121b933640 1 --2- 172.21.15.119:0/949428461 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1214161250 0x7f12141616c0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.208+0000 7f121b933640 1 --2- 172.21.15.119:0/949428461 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f121415b080 0x7f12140ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.208+0000 7f121b933640 1 -- 172.21.15.119:0/949428461 >> 172.21.15.119:0/949428461 conn(0x7f121406a9f0 msgr2=0x7f121406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:34.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.208+0000 7f121b933640 1 -- 172.21.15.119:0/949428461 shutdown_connections 2023-11-08T19:28:34.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.208+0000 7f121b933640 1 -- 172.21.15.119:0/949428461 wait complete. 2023-11-08T19:28:34.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.210+0000 7f121b933640 1 Processor -- start 2023-11-08T19:28:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.210+0000 7f121b933640 1 -- start start 2023-11-08T19:28:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.211+0000 7f121b933640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140cf8c0 0x7f121414dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.211+0000 7f121b933640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 0x7f1214143300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.211+0000 7f121b933640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1214161250 0x7f121415dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.211+0000 7f121b933640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1214156380 con 0x7f121415b080 2023-11-08T19:28:34.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.211+0000 7f121b933640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1214143840 con 0x7f12140cf8c0 2023-11-08T19:28:34.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.211+0000 7f121b933640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140c9890 con 0x7f1214161250 2023-11-08T19:28:34.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 0x7f1214143300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 0x7f1214143300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35124/0 (socket says 172.21.15.119:35124) 2023-11-08T19:28:34.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 -- 172.21.15.119:0/3476746646 learned_addr learned my addr 172.21.15.119:0/3476746646 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f12196cf640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140cf8c0 0x7f121414dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 -- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1214161250 msgr2=0x7f121415dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1214161250 0x7f121415dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 -- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140cf8c0 msgr2=0x7f121414dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140cf8c0 0x7f121414dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 -- 172.21.15.119:0/3476746646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c009b30 con 0x7f121415b080 2023-11-08T19:28:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f12196cf640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140cf8c0 0x7f121414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.212+0000 7f1218ece640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 0x7f1214143300 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f120800c430 tx=0x7f120800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.213+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208011790 con 0x7f121415b080 2023-11-08T19:28:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.213+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12080118f0 con 0x7f121415b080 2023-11-08T19:28:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.213+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1214155a50 con 0x7f121415b080 2023-11-08T19:28:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.213+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208011ce0 con 0x7f121415b080 2023-11-08T19:28:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.213+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1214156010 con 0x7f121415b080 2023-11-08T19:28:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.214+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1214156530 con 0x7f121415b080 2023-11-08T19:28:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.215+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f120801c460 con 0x7f121415b080 2023-11-08T19:28:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.216+0000 7f1205ffb640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11f806ae70 0x7f11f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.216+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f1208089ad0 con 0x7f121415b080 2023-11-08T19:28:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.216+0000 7f12196cf640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11f806ae70 0x7f11f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.217+0000 7f12196cf640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11f806ae70 0x7f11f806d330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1210006fd0 tx=0x7f1210008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:34.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.220+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1208054610 con 0x7f121415b080 2023-11-08T19:28:34.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.450+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1214074430 con 0x7f121415b080 2023-11-08T19:28:34.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.452+0000 7f1205ffb640 1 -- 172.21.15.119:0/3476746646 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f1208057c30 con 0x7f121415b080 2023-11-08T19:28:34.454 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:34.454 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:15.191469+0000","modified":"2023-11-08T19:28:25.344541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/4016780496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4016780496},{"type":"v1","addr":"172.21.15.119:6837","nonce":4016780496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:28:34.455 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 46 2023-11-08T19:28:34.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11f806ae70 msgr2=0x7f11f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11f806ae70 0x7f11f806d330 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1210006fd0 tx=0x7f1210008040 comp rx=0 tx=0).stop 2023-11-08T19:28:34.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 msgr2=0x7f1214143300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 0x7f1214143300 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f120800c430 tx=0x7f120800c8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 shutdown_connections 2023-11-08T19:28:34.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f11f806ae70 0x7f11f806d330 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1214161250 0x7f121415dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f121415b080 0x7f1214143300 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 --2- 172.21.15.119:0/3476746646 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f12140cf8c0 0x7f121414dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 >> 172.21.15.119:0/3476746646 conn(0x7f121406a9f0 msgr2=0x7f12140d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:34.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 shutdown_connections 2023-11-08T19:28:34.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.455+0000 7f121b933640 1 -- 172.21.15.119:0/3476746646 wait complete. 2023-11-08T19:28:34.475 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:34.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.679+0000 7f91a6111640 1 Processor -- start 2023-11-08T19:28:34.681 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.679+0000 7f91a6111640 1 -- start start 2023-11-08T19:28:34.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a6111640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91a00ad7d0 0x7f91a0070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a6111640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a00710a0 0x7f91a00c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a6111640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a006cff0 con 0x7f91a015b0d0 2023-11-08T19:28:34.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a6111640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a006d2d0 con 0x7f91a00ad7d0 2023-11-08T19:28:34.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a6111640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a006bd20 con 0x7f91a00710a0 2023-11-08T19:28:34.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a490e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91a00ad7d0 0x7f91a0070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a5910640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a00710a0 0x7f91a00c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a490e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f91a00ad7d0 0x7f91a0070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:47298/0 (socket says 172.21.15.119:47298) 2023-11-08T19:28:34.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.681+0000 7f91a490e640 1 -- 172.21.15.119:0/395537751 learned_addr learned my addr 172.21.15.119:0/395537751 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:34.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.682+0000 7f91a490e640 1 -- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a00710a0 msgr2=0x7f91a00c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:34.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.682+0000 7f91a490e640 1 --2- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a00710a0 0x7f91a00c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.682+0000 7f91a490e640 1 -- 172.21.15.119:0/395537751 >> v1:172.21.15.188:6789/0 conn(0x7f91a015b0d0 legacy=0x7f91a015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.682+0000 7f91a490e640 1 -- 172.21.15.119:0/395537751 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a006d760 con 0x7f91a00ad7d0 2023-11-08T19:28:34.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.683+0000 7f91a490e640 1 --2- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a0070b60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9190009870 tx=0x7f919002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=317d24d9580b7ba server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:34.686 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f9197fff640 1 -- 172.21.15.119:0/395537751 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190031640 con 0x7f91a00ad7d0 2023-11-08T19:28:34.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f9197fff640 1 -- 172.21.15.119:0/395537751 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91900317a0 con 0x7f91a00ad7d0 2023-11-08T19:28:34.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f9197fff640 1 -- 172.21.15.119:0/395537751 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190031b30 con 0x7f91a00ad7d0 2023-11-08T19:28:34.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f91a6111640 1 -- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 msgr2=0x7f91a0070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f91a6111640 1 --2- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a0070b60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9190009870 tx=0x7f919002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.687 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f91a6111640 1 -- 172.21.15.119:0/395537751 shutdown_connections 2023-11-08T19:28:34.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f91a6111640 1 --2- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a0070b60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f91a6111640 1 --2- 172.21.15.119:0/395537751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a00710a0 0x7f91a00c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.684+0000 7f91a6111640 1 -- 172.21.15.119:0/395537751 >> 172.21.15.119:0/395537751 conn(0x7f91a00b0560 msgr2=0x7f91a00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:34.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.685+0000 7f91a6111640 1 -- 172.21.15.119:0/395537751 shutdown_connections 2023-11-08T19:28:34.688 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.685+0000 7f91a6111640 1 -- 172.21.15.119:0/395537751 wait complete. 2023-11-08T19:28:34.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.686+0000 7f91a6111640 1 Processor -- start 2023-11-08T19:28:34.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.686+0000 7f91a6111640 1 -- start start 2023-11-08T19:28:34.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.687+0000 7f91a6111640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 0x7f91a014c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.687+0000 7f91a6111640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a015c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.687+0000 7f91a6111640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a015b0d0 0x7f91a00cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.687+0000 7f91a6111640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a00b1c00 con 0x7f91a00710a0 2023-11-08T19:28:34.692 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.687+0000 7f91a6111640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a015cb90 con 0x7f91a00ad7d0 2023-11-08T19:28:34.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.687+0000 7f91a6111640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a015ce70 con 0x7f91a015b0d0 2023-11-08T19:28:34.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 0x7f91a014c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 0x7f91a014c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35126/0 (socket says 172.21.15.119:35126) 2023-11-08T19:28:34.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 -- 172.21.15.119:0/442766717 learned_addr learned my addr 172.21.15.119:0/442766717 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:34.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a490e640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a015c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a5910640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a015b0d0 0x7f91a00cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 -- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a015b0d0 msgr2=0x7f91a00cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a015b0d0 0x7f91a00cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 -- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 msgr2=0x7f91a015c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a015c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 -- 172.21.15.119:0/442766717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91900094d0 con 0x7f91a00710a0 2023-11-08T19:28:34.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.688+0000 7f91a510f640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 0x7f91a014c870 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f919c00c400 tx=0x7f919c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:34.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.689+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c0052d0 con 0x7f91a00710a0 2023-11-08T19:28:34.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.689+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c005450 con 0x7f91a00710a0 2023-11-08T19:28:34.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.689+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c007260 con 0x7f91a00710a0 2023-11-08T19:28:34.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.689+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a015d150 con 0x7f91a00710a0 2023-11-08T19:28:34.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.689+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a00c87e0 con 0x7f91a00710a0 2023-11-08T19:28:34.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.690+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a004ece0 con 0x7f91a00710a0 2023-11-08T19:28:34.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.691+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f919c0073c0 con 0x7f91a00710a0 2023-11-08T19:28:34.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.692+0000 7f9195ffb640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f918406ae70 0x7f918406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:34.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.692+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7092+0+0 (secure 0 0 0) 0x7f919c088f90 con 0x7f91a00710a0 2023-11-08T19:28:34.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.692+0000 7f91a490e640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f918406ae70 0x7f918406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:34.698 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.693+0000 7f91a490e640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f918406ae70 0x7f918406d330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9190036000 tx=0x7f919003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:34.699 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.696+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f919c053ac0 con 0x7f91a00710a0 2023-11-08T19:28:34.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.897+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91a00b1320 con 0x7f91a00710a0 2023-11-08T19:28:34.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.899+0000 7f9195ffb640 1 -- 172.21.15.119:0/442766717 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22063 (secure 0 0 0) 0x7f919c0570e0 con 0x7f91a00710a0 2023-11-08T19:28:34.901 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:34.901 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":84,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:34.857295+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:34.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.901+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f918406ae70 msgr2=0x7f918406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.901+0000 7f91a6111640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f918406ae70 0x7f918406d330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9190036000 tx=0x7f919003c040 comp rx=0 tx=0).stop 2023-11-08T19:28:34.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.901+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 msgr2=0x7f91a014c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:34.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.901+0000 7f91a6111640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 0x7f91a014c870 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f919c00c400 tx=0x7f919c00c8c0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 shutdown_connections 2023-11-08T19:28:34.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f918406ae70 0x7f918406d330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f91a015b0d0 0x7f91a00cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91a00ad7d0 0x7f91a015c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 --2- 172.21.15.119:0/442766717 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91a00710a0 0x7f91a014c870 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:34.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 >> 172.21.15.119:0/442766717 conn(0x7f91a00b0560 msgr2=0x7f91a00c4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:34.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 shutdown_connections 2023-11-08T19:28:34.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:34.902+0000 7f91a6111640 1 -- 172.21.15.119:0/442766717 wait complete. 2023-11-08T19:28:34.921 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:28:34.922 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:35.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.117+0000 7f280e864640 1 Processor -- start 2023-11-08T19:28:35.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.117+0000 7f280e864640 1 -- start start 2023-11-08T19:28:35.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f280e864640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2808142f80 0x7f28080cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f280e864640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080c4960 con 0x7f280806cb40 2023-11-08T19:28:35.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f280e864640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280815df60 con 0x7f280815d9f0 2023-11-08T19:28:35.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f280e864640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2808143a40 con 0x7f2808142f80 2023-11-08T19:28:35.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f2807fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2808142f80 0x7f28080cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f2807fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2808142f80 0x7f28080cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35132/0 (socket says 172.21.15.119:35132) 2023-11-08T19:28:35.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f2807fff640 1 -- 172.21.15.119:0/972979745 learned_addr learned my addr 172.21.15.119:0/972979745 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:35.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.119+0000 7f280ce01640 1 --1- 172.21.15.119:0/972979745 >> v1:172.21.15.119:6789/0 conn(0x7f280806cb40 0x7f28080c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40132/0 (socket says 172.21.15.119:40132) 2023-11-08T19:28:35.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.120+0000 7f2807fff640 1 -- 172.21.15.119:0/972979745 >> v1:172.21.15.188:6789/0 conn(0x7f280815d9f0 legacy=0x7f28080c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:35.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.120+0000 7f2807fff640 1 -- 172.21.15.119:0/972979745 >> v1:172.21.15.119:6789/0 conn(0x7f280806cb40 legacy=0x7f28080c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.120+0000 7f2807fff640 1 -- 172.21.15.119:0/972979745 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280815e240 con 0x7f2808142f80 2023-11-08T19:28:35.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.121+0000 7f2807fff640 1 --2- 172.21.15.119:0/972979745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808142f80 0x7f28080cc6d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bde4b270eb76efb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:35.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.121+0000 7f28077fe640 1 -- 172.21.15.119:0/972979745 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8030860 con 0x7f2808142f80 2023-11-08T19:28:35.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.121+0000 7f28077fe640 1 -- 172.21.15.119:0/972979745 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f80309c0 con 0x7f2808142f80 2023-11-08T19:28:35.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.121+0000 7f28077fe640 1 -- 172.21.15.119:0/972979745 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8030d50 con 0x7f2808142f80 2023-11-08T19:28:35.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 -- 172.21.15.119:0/972979745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808142f80 msgr2=0x7f28080cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 --2- 172.21.15.119:0/972979745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808142f80 0x7f28080cc6d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802b9a0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 -- 172.21.15.119:0/972979745 shutdown_connections 2023-11-08T19:28:35.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 --2- 172.21.15.119:0/972979745 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808142f80 0x7f28080cc6d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 -- 172.21.15.119:0/972979745 >> 172.21.15.119:0/972979745 conn(0x7f28080cb610 msgr2=0x7f28080cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:35.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 -- 172.21.15.119:0/972979745 shutdown_connections 2023-11-08T19:28:35.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.122+0000 7f280e864640 1 -- 172.21.15.119:0/972979745 wait complete. 2023-11-08T19:28:35.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.123+0000 7f280e864640 1 Processor -- start 2023-11-08T19:28:35.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.124+0000 7f280e864640 1 -- start start 2023-11-08T19:28:35.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.124+0000 7f280e864640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280806cb40 0x7f280815ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.124+0000 7f280e864640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2808142f80 0x7f28080cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280e864640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 0x7f2808134da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280e864640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080b46d0 con 0x7f280815d9f0 2023-11-08T19:28:35.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280e864640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2808135910 con 0x7f280806cb40 2023-11-08T19:28:35.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280e864640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2808135bf0 con 0x7f2808142f80 2023-11-08T19:28:35.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 0x7f2808134da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 0x7f2808134da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35140/0 (socket says 172.21.15.119:35140) 2023-11-08T19:28:35.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 -- 172.21.15.119:0/2429860060 learned_addr learned my addr 172.21.15.119:0/2429860060 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:35.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 -- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2808142f80 msgr2=0x7f28080cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f2807fff640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280806cb40 0x7f280815ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f27fffff640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2808142f80 0x7f28080cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2808142f80 0x7f28080cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 -- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280806cb40 msgr2=0x7f280815ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280806cb40 0x7f280815ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.125+0000 7f280ce01640 1 -- 172.21.15.119:0/2429860060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80094d0 con 0x7f280815d9f0 2023-11-08T19:28:35.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.126+0000 7f280ce01640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 0x7f2808134da0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f280000caa0 tx=0x7f280000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:35.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.126+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000a730 con 0x7f280815d9f0 2023-11-08T19:28:35.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.126+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280000a890 con 0x7f280815d9f0 2023-11-08T19:28:35.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.126+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280000ac40 con 0x7f280815d9f0 2023-11-08T19:28:35.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.126+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280815a330 con 0x7f280815d9f0 2023-11-08T19:28:35.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.126+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280815a8f0 con 0x7f280815d9f0 2023-11-08T19:28:35.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.127+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2808074430 con 0x7f280815d9f0 2023-11-08T19:28:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.128+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2800006070 con 0x7f280815d9f0 2023-11-08T19:28:35.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.129+0000 7f28057fa640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.129+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f2800089590 con 0x7f280815d9f0 2023-11-08T19:28:35.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.129+0000 7f2807fff640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.130+0000 7f2807fff640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f27f8036000 tx=0x7f27f803b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:35.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.133+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28000540b0 con 0x7f280815d9f0 2023-11-08T19:28:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.351+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f280815e520 con 0x7f280815d9f0 2023-11-08T19:28:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.352+0000 7f28057fa640 1 -- 172.21.15.119:0/2429860060 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f28000576d0 con 0x7f280815d9f0 2023-11-08T19:28:35.356 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:35.356 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:15.191469+0000","modified":"2023-11-08T19:28:25.344541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/4016780496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4016780496},{"type":"v1","addr":"172.21.15.119:6837","nonce":4016780496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:28:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 46 2023-11-08T19:28:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.355+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 msgr2=0x7f27e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.355+0000 7f280e864640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f27f8036000 tx=0x7f27f803b040 comp rx=0 tx=0).stop 2023-11-08T19:28:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.355+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 msgr2=0x7f2808134da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.355+0000 7f280e864640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 0x7f2808134da0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f280000caa0 tx=0x7f280000cea0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 shutdown_connections 2023-11-08T19:28:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f280815d9f0 0x7f2808134da0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2808142f80 0x7f28080cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 --2- 172.21.15.119:0/2429860060 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280806cb40 0x7f280815ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 >> 172.21.15.119:0/2429860060 conn(0x7f28080cb610 msgr2=0x7f2808070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:35.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 shutdown_connections 2023-11-08T19:28:35.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.356+0000 7f280e864640 1 -- 172.21.15.119:0/2429860060 wait complete. 2023-11-08T19:28:35.380 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:35.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.576+0000 7f8a5aa0e640 1 Processor -- start 2023-11-08T19:28:35.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.576+0000 7f8a5aa0e640 1 -- start start 2023-11-08T19:28:35.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a5aa0e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a541447a0 0x7f8a5415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a5aa0e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8a540cad30 0x7f8a5406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a5aa0e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a540c3a50 con 0x7f8a5406a120 2023-11-08T19:28:35.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a5aa0e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a54144b80 con 0x7f8a541447a0 2023-11-08T19:28:35.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a5aa0e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a540c9150 con 0x7f8a540cad30 2023-11-08T19:28:35.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a53fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a541447a0 0x7f8a5415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a58fab640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8a5406a120 0x7f8a5406a530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40146/0 (socket says 172.21.15.119:40146) 2023-11-08T19:28:35.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.578+0000 7f8a58fab640 1 -- 172.21.15.119:0/4122146323 learned_addr learned my addr 172.21.15.119:0/4122146323 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:35.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a537fe640 1 --2- 172.21.15.119:0/4122146323 >> v2:172.21.15.188:3300/0 conn(0x7f8a540cad30 0x7f8a5406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a52ffd640 1 -- 172.21.15.119:0/4122146323 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437578404 0 0) 0x7f8a540c3a50 con 0x7f8a5406a120 2023-11-08T19:28:35.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a52ffd640 1 -- 172.21.15.119:0/4122146323 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a40003520 con 0x7f8a5406a120 2023-11-08T19:28:35.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a53fff640 1 -- 172.21.15.119:0/4122146323 >> v2:172.21.15.188:3300/0 conn(0x7f8a540cad30 msgr2=0x7f8a5406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:35.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a53fff640 1 --2- 172.21.15.119:0/4122146323 >> v2:172.21.15.188:3300/0 conn(0x7f8a540cad30 0x7f8a5406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a53fff640 1 -- 172.21.15.119:0/4122146323 >> v1:172.21.15.119:6789/0 conn(0x7f8a5406a120 legacy=0x7f8a5406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.579+0000 7f8a53fff640 1 -- 172.21.15.119:0/4122146323 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a540c9410 con 0x7f8a541447a0 2023-11-08T19:28:35.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.580+0000 7f8a53fff640 1 --2- 172.21.15.119:0/4122146323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a541447a0 0x7f8a5415f4c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8a5406df20 tx=0x7f8a4402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7411bee421d19935 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:35.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.581+0000 7f8a52ffd640 1 -- 172.21.15.119:0/4122146323 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44031830 con 0x7f8a541447a0 2023-11-08T19:28:35.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.581+0000 7f8a52ffd640 1 -- 172.21.15.119:0/4122146323 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a44031990 con 0x7f8a541447a0 2023-11-08T19:28:35.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.581+0000 7f8a52ffd640 1 -- 172.21.15.119:0/4122146323 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44031d20 con 0x7f8a541447a0 2023-11-08T19:28:35.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.581+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/4122146323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a541447a0 msgr2=0x7f8a5415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.581+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/4122146323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a541447a0 0x7f8a5415f4c0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f8a5406df20 tx=0x7f8a4402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.582+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/4122146323 shutdown_connections 2023-11-08T19:28:35.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.582+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/4122146323 >> v2:172.21.15.188:3300/0 conn(0x7f8a540cad30 0x7f8a5406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.582+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/4122146323 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a541447a0 0x7f8a5415f4c0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.582+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/4122146323 >> 172.21.15.119:0/4122146323 conn(0x7f8a540cb610 msgr2=0x7f8a540cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:35.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.582+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/4122146323 shutdown_connections 2023-11-08T19:28:35.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.582+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/4122146323 wait complete. 2023-11-08T19:28:35.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.583+0000 7f8a5aa0e640 1 Processor -- start 2023-11-08T19:28:35.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.584+0000 7f8a5aa0e640 1 -- start start 2023-11-08T19:28:35.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a5aa0e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 0x7f8a540c7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a5aa0e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a540cad30 0x7f8a54142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a5aa0e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a541447a0 0x7f8a54136f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a5aa0e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54141a90 con 0x7f8a5406a120 2023-11-08T19:28:35.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a5aa0e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54140e80 con 0x7f8a540cad30 2023-11-08T19:28:35.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a5aa0e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a54141160 con 0x7f8a541447a0 2023-11-08T19:28:35.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 0x7f8a540c7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 0x7f8a540c7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35164/0 (socket says 172.21.15.119:35164) 2023-11-08T19:28:35.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 -- 172.21.15.119:0/2654825064 learned_addr learned my addr 172.21.15.119:0/2654825064 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:35.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 -- 172.21.15.119:0/2654825064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a541447a0 msgr2=0x7f8a54136f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a541447a0 0x7f8a54136f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 -- 172.21.15.119:0/2654825064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a540cad30 msgr2=0x7f8a54142900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a540cad30 0x7f8a54142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.585+0000 7f8a53fff640 1 -- 172.21.15.119:0/2654825064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a44009400 con 0x7f8a5406a120 2023-11-08T19:28:35.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.586+0000 7f8a53fff640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 0x7f8a540c7e60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8a44000c00 tx=0x7f8a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:35.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.586+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4403a830 con 0x7f8a5406a120 2023-11-08T19:28:35.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.586+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4403a9b0 con 0x7f8a5406a120 2023-11-08T19:28:35.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.586+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4403ad60 con 0x7f8a5406a120 2023-11-08T19:28:35.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.586+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a54141440 con 0x7f8a5406a120 2023-11-08T19:28:35.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.586+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a540c83a0 con 0x7f8a5406a120 2023-11-08T19:28:35.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.587+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a54074430 con 0x7f8a5406a120 2023-11-08T19:28:35.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.588+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a4404d450 con 0x7f8a5406a120 2023-11-08T19:28:35.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.589+0000 7f8a50ff9640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a3406ae70 0x7f8a3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:35.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.589+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f8a440abce0 con 0x7f8a5406a120 2023-11-08T19:28:35.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.589+0000 7f8a537fe640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a3406ae70 0x7f8a3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:35.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.590+0000 7f8a537fe640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a3406ae70 0x7f8a3406d330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006a90 tx=0x7f8a3c006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:35.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.594+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a44076750 con 0x7f8a5406a120 2023-11-08T19:28:35.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.826+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a540b2cc0 con 0x7f8a5406a120 2023-11-08T19:28:35.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.827+0000 7f8a50ff9640 1 -- 172.21.15.119:0/2654825064 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f8a44035070 con 0x7f8a5406a120 2023-11-08T19:28:35.829 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:35.829 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":46,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:15.191469+0000","modified":"2023-11-08T19:28:25.344541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":40,"state":"up:active","state_seq":3,"addr":"172.21.15.119:6837/4016780496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":4016780496},{"type":"v1","addr":"172.21.15.119:6837","nonce":4016780496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T19:28:35.829 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 46 2023-11-08T19:28:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a3406ae70 msgr2=0x7f8a3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a3406ae70 0x7f8a3406d330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c006a90 tx=0x7f8a3c006a20 comp rx=0 tx=0).stop 2023-11-08T19:28:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 msgr2=0x7f8a540c7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 0x7f8a540c7e60 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f8a44000c00 tx=0x7f8a440046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 shutdown_connections 2023-11-08T19:28:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a3406ae70 0x7f8a3406d330 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a541447a0 0x7f8a54136f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a540cad30 0x7f8a54142900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 --2- 172.21.15.119:0/2654825064 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5406a120 0x7f8a540c7e60 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 >> 172.21.15.119:0/2654825064 conn(0x7f8a540cb610 msgr2=0x7f8a540cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 shutdown_connections 2023-11-08T19:28:35.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:35.830+0000 7f8a5aa0e640 1 -- 172.21.15.119:0/2654825064 wait complete. 2023-11-08T19:28:35.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:36.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.056+0000 7faced105640 1 Processor -- start 2023-11-08T19:28:36.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.056+0000 7faced105640 1 -- start start 2023-11-08T19:28:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.057+0000 7faced105640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7face8142f80 0x7face80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.057+0000 7faced105640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7face806cb40 0x7face806cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.057+0000 7faced105640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face806d4d0 con 0x7face815d9f0 2023-11-08T19:28:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.057+0000 7faced105640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face8143a40 con 0x7face8142f80 2023-11-08T19:28:36.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.057+0000 7faced105640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face80ca3e0 con 0x7face806cb40 2023-11-08T19:28:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7face8142f80 0x7face80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7face815d9f0 0x7face80c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40150/0 (socket says 172.21.15.119:40150) 2023-11-08T19:28:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face659c640 1 -- 172.21.15.119:0/910239422 learned_addr learned my addr 172.21.15.119:0/910239422 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face759e640 1 --2- 172.21.15.119:0/910239422 >> v2:172.21.15.188:3300/0 conn(0x7face806cb40 0x7face806cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face6d9d640 1 -- 172.21.15.119:0/910239422 >> v2:172.21.15.188:3300/0 conn(0x7face806cb40 msgr2=0x7face806cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face6d9d640 1 --2- 172.21.15.119:0/910239422 >> v2:172.21.15.188:3300/0 conn(0x7face806cb40 0x7face806cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face6d9d640 1 -- 172.21.15.119:0/910239422 >> v1:172.21.15.119:6789/0 conn(0x7face815d9f0 legacy=0x7face80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face6d9d640 1 -- 172.21.15.119:0/910239422 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face80c4960 con 0x7face8142f80 2023-11-08T19:28:36.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.058+0000 7face5d9b640 1 -- 172.21.15.119:0/910239422 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670897105 0 0) 0x7face806d4d0 con 0x7face815d9f0 2023-11-08T19:28:36.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.059+0000 7face6d9d640 1 --2- 172.21.15.119:0/910239422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80cc6d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7face80c41e0 tx=0x7facd802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478a1681576fd444 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7face5d9b640 1 -- 172.21.15.119:0/910239422 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8031830 con 0x7face8142f80 2023-11-08T19:28:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7face5d9b640 1 -- 172.21.15.119:0/910239422 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd8031990 con 0x7face8142f80 2023-11-08T19:28:36.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7face5d9b640 1 -- 172.21.15.119:0/910239422 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd8031d20 con 0x7face8142f80 2023-11-08T19:28:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7faced105640 1 -- 172.21.15.119:0/910239422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 msgr2=0x7face80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7faced105640 1 --2- 172.21.15.119:0/910239422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80cc6d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7face80c41e0 tx=0x7facd802b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7faced105640 1 -- 172.21.15.119:0/910239422 shutdown_connections 2023-11-08T19:28:36.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7faced105640 1 --2- 172.21.15.119:0/910239422 >> v2:172.21.15.188:3300/0 conn(0x7face806cb40 0x7face806cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7faced105640 1 --2- 172.21.15.119:0/910239422 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80cc6d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.060+0000 7faced105640 1 -- 172.21.15.119:0/910239422 >> 172.21.15.119:0/910239422 conn(0x7face80cb610 msgr2=0x7face80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.061+0000 7faced105640 1 -- 172.21.15.119:0/910239422 shutdown_connections 2023-11-08T19:28:36.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.061+0000 7faced105640 1 -- 172.21.15.119:0/910239422 wait complete. 2023-11-08T19:28:36.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.062+0000 7faced105640 1 Processor -- start 2023-11-08T19:28:36.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.062+0000 7faced105640 1 -- start start 2023-11-08T19:28:36.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.063+0000 7faced105640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face806cb40 0x7face8141950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.063+0000 7faced105640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.063+0000 7faced105640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815d9f0 0x7face80b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.063+0000 7faced105640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80b2330 con 0x7face8142f80 2023-11-08T19:28:36.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.063+0000 7faced105640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c89e0 con 0x7face815d9f0 2023-11-08T19:28:36.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.063+0000 7faced105640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face80b25d0 con 0x7face806cb40 2023-11-08T19:28:36.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35184/0 (socket says 172.21.15.119:35184) 2023-11-08T19:28:36.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 -- 172.21.15.119:0/2579769716 learned_addr learned my addr 172.21.15.119:0/2579769716 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:36.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face6d9d640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face806cb40 0x7face8141950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face759e640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815d9f0 0x7face80b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 -- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face806cb40 msgr2=0x7face8141950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face806cb40 0x7face8141950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 -- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815d9f0 msgr2=0x7face80b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815d9f0 0x7face80b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 -- 172.21.15.119:0/2579769716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd8009400 con 0x7face8142f80 2023-11-08T19:28:36.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.064+0000 7face659c640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80c84a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c4d0 tx=0x7facdc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.065+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0074e0 con 0x7face8142f80 2023-11-08T19:28:36.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.065+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc007660 con 0x7face8142f80 2023-11-08T19:28:36.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.065+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face80b28b0 con 0x7face8142f80 2023-11-08T19:28:36.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.065+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc004a20 con 0x7face8142f80 2023-11-08T19:28:36.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.065+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face80b2e70 con 0x7face8142f80 2023-11-08T19:28:36.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.066+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face8074430 con 0x7face8142f80 2023-11-08T19:28:36.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.067+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7facdc004b80 con 0x7face8142f80 2023-11-08T19:28:36.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.068+0000 7faccf7fe640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facd406aea0 0x7facd406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.069+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7facdc088db0 con 0x7face8142f80 2023-11-08T19:28:36.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.069+0000 7face6d9d640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facd406aea0 0x7facd406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.070+0000 7face6d9d640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facd406aea0 0x7facd406d360 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7facd8036000 tx=0x7facd803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.073+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facdc053830 con 0x7face8142f80 2023-11-08T19:28:36.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.265+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7face8159ee0 con 0x7face8142f80 2023-11-08T19:28:36.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.267+0000 7faccf7fe640 1 -- 172.21.15.119:0/2579769716 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22063 (secure 0 0 0) 0x7facdc018070 con 0x7face8142f80 2023-11-08T19:28:36.269 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:36.269 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":84,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:34.857295+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:13.287561+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:14.184954+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":84,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facd406aea0 msgr2=0x7facd406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facd406aea0 0x7facd406d360 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7facd8036000 tx=0x7facd803c040 comp rx=0 tx=0).stop 2023-11-08T19:28:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 msgr2=0x7face80c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80c84a0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7facdc00c4d0 tx=0x7facdc00c990 comp rx=0 tx=0).stop 2023-11-08T19:28:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 shutdown_connections 2023-11-08T19:28:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7facd406aea0 0x7facd406d360 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7face815d9f0 0x7face80b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.269+0000 7faced105640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7face8142f80 0x7face80c84a0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.270+0000 7faced105640 1 --2- 172.21.15.119:0/2579769716 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7face806cb40 0x7face8141950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.270+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 >> 172.21.15.119:0/2579769716 conn(0x7face80cb610 msgr2=0x7face80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.270+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 shutdown_connections 2023-11-08T19:28:36.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.270+0000 7faced105640 1 -- 172.21.15.119:0/2579769716 wait complete. 2023-11-08T19:28:36.290 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:28:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.503+0000 7f5ec5e56640 1 Processor -- start 2023-11-08T19:28:36.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.504+0000 7f5ec5e56640 1 -- start start 2023-11-08T19:28:36.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ec5e56640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ec0159550 0x7f5ec00ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ec5e56640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec00cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ec5e56640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec00cf0d0 con 0x7f5ec0072170 2023-11-08T19:28:36.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ec5e56640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec00cd3c0 con 0x7f5ec0159550 2023-11-08T19:28:36.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ec5e56640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00cd6a0 con 0x7f5ec0159930 2023-11-08T19:28:36.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ebf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ec0159550 0x7f5ec00ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ebf7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5ec0159550 0x7f5ec00ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35198/0 (socket says 172.21.15.119:35198) 2023-11-08T19:28:36.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.505+0000 7f5ebf7fe640 1 -- 172.21.15.119:0/1309409141 learned_addr learned my addr 172.21.15.119:0/1309409141 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.506+0000 7f5ebeffd640 1 --2- 172.21.15.119:0/1309409141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec00cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.506+0000 7f5ebf7fe640 1 -- 172.21.15.119:0/1309409141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 msgr2=0x7f5ec00cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.506+0000 7f5ebf7fe640 1 --2- 172.21.15.119:0/1309409141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec00cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.506+0000 7f5ebf7fe640 1 -- 172.21.15.119:0/1309409141 >> v1:172.21.15.188:6789/0 conn(0x7f5ec0072170 legacy=0x7f5ec006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:36.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.506+0000 7f5ebf7fe640 1 -- 172.21.15.119:0/1309409141 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00cd980 con 0x7f5ec0159550 2023-11-08T19:28:36.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.507+0000 7f5ebf7fe640 1 --2- 172.21.15.119:0/1309409141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec00ceb90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009870 tx=0x7f5eb002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68300e3db52f4964 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ebe7fc640 1 -- 172.21.15.119:0/1309409141 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0030840 con 0x7f5ec0159550 2023-11-08T19:28:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ebe7fc640 1 -- 172.21.15.119:0/1309409141 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb00309a0 con 0x7f5ec0159550 2023-11-08T19:28:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ebe7fc640 1 -- 172.21.15.119:0/1309409141 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0030d30 con 0x7f5ec0159550 2023-11-08T19:28:36.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1309409141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 msgr2=0x7f5ec00ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1309409141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec00ceb90 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0009870 tx=0x7f5eb002b980 comp rx=0 tx=0).stop 2023-11-08T19:28:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1309409141 shutdown_connections 2023-11-08T19:28:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1309409141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec00cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1309409141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec00ceb90 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.508+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1309409141 >> 172.21.15.119:0/1309409141 conn(0x7f5ec006c280 msgr2=0x7f5ec006c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.509+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1309409141 shutdown_connections 2023-11-08T19:28:36.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.509+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1309409141 wait complete. 2023-11-08T19:28:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.510+0000 7f5ec5e56640 1 Processor -- start 2023-11-08T19:28:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.511+0000 7f5ec5e56640 1 -- start start 2023-11-08T19:28:36.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.511+0000 7f5ec5e56640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ec0072170 0x7f5ec00b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.511+0000 7f5ec5e56640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec014a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.511+0000 7f5ec5e56640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec014adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ec5e56640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec014b470 con 0x7f5ec0159550 2023-11-08T19:28:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ec5e56640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00c42a0 con 0x7f5ec0072170 2023-11-08T19:28:36.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ec5e56640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00c4580 con 0x7f5ec0159930 2023-11-08T19:28:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec014a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec014a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35204/0 (socket says 172.21.15.119:35204) 2023-11-08T19:28:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 -- 172.21.15.119:0/1248094826 learned_addr learned my addr 172.21.15.119:0/1248094826 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebf7fe640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ec0072170 0x7f5ec00b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 -- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 msgr2=0x7f5ec014adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec014adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 -- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ec0072170 msgr2=0x7f5ec00b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ec0072170 0x7f5ec00b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebeffd640 1 -- 172.21.15.119:0/1248094826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb00094d0 con 0x7f5ec0159550 2023-11-08T19:28:36.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.512+0000 7f5ebf7fe640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ec0072170 0x7f5ec00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.513+0000 7f5ebeffd640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec014a880 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400d5f0 tx=0x7f5eb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.513+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40125f0 con 0x7f5ec0159550 2023-11-08T19:28:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.513+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb4012750 con 0x7f5ec0159550 2023-11-08T19:28:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.513+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec014b710 con 0x7f5ec0159550 2023-11-08T19:28:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.513+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4012b00 con 0x7f5ec0159550 2023-11-08T19:28:36.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.513+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec00b0950 con 0x7f5ec0159550 2023-11-08T19:28:36.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.514+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec00b0b40 con 0x7f5ec0159550 2023-11-08T19:28:36.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.515+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5eb4012c60 con 0x7f5ec0159550 2023-11-08T19:28:36.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.516+0000 7f5ea7fff640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5eac06aea0 0x7f5eac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.516+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f5eb4089ff0 con 0x7f5ec0159550 2023-11-08T19:28:36.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.516+0000 7f5ebf7fe640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5eac06aea0 0x7f5eac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.517+0000 7f5ebf7fe640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5eac06aea0 0x7f5eac06d360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0036000 tx=0x7f5eb003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.520+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5eb40bcca0 con 0x7f5ec0159550 2023-11-08T19:28:36.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.749+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5ec00cf2a0 con 0x7f5ec0159550 2023-11-08T19:28:36.752 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-11-08T19:28:36.750+0000 7f8fd7f7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:28:36.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.754+0000 7f5ea7fff640 1 -- 172.21.15.119:0/1248094826 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5eb4054b30 con 0x7f5ec0159550 2023-11-08T19:28:36.757 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:28:36.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.757+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5eac06aea0 msgr2=0x7f5eac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.757+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5eac06aea0 0x7f5eac06d360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0036000 tx=0x7f5eb003b040 comp rx=0 tx=0).stop 2023-11-08T19:28:36.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.757+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 msgr2=0x7f5ec014a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.757+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec014a880 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400d5f0 tx=0x7f5eb400dab0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.758+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 shutdown_connections 2023-11-08T19:28:36.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.758+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5eac06aea0 0x7f5eac06d360 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.758+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5ec0159930 0x7f5ec014adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.758+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5ec0159550 0x7f5ec014a880 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.758+0000 7f5ec5e56640 1 --2- 172.21.15.119:0/1248094826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5ec0072170 0x7f5ec00b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.758+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 >> 172.21.15.119:0/1248094826 conn(0x7f5ec006c280 msgr2=0x7f5ec0072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.760+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 shutdown_connections 2023-11-08T19:28:36.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.760+0000 7f5ec5e56640 1 -- 172.21.15.119:0/1248094826 wait complete. 2023-11-08T19:28:36.768 INFO:tasks.ceph.mds.c.smithi119.stderr: -2432> 2023-11-08T19:28:20.683+0000 7f803bb8e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:36.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:28:36.822 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:28:36.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.966+0000 7f4e4f59e640 1 Processor -- start 2023-11-08T19:28:36.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.966+0000 7f4e4f59e640 1 -- start start 2023-11-08T19:28:36.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.967+0000 7f4e4f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e5015d310 0x7f4e50071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.967+0000 7f4e4f59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 0x7f4e500ccad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5006d6e0 con 0x7f4e500ad8b0 2023-11-08T19:28:36.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4f59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e500cd010 con 0x7f4e500cc640 2023-11-08T19:28:36.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500c90c0 con 0x7f4e5015d310 2023-11-08T19:28:36.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 0x7f4e500ccad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4e59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e5015d310 0x7f4e50071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4ed9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 0x7f4e500ccad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:47362/0 (socket says 172.21.15.119:47362) 2023-11-08T19:28:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.968+0000 7f4e4ed9d640 1 -- 172.21.15.119:0/1376350179 learned_addr learned my addr 172.21.15.119:0/1376350179 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.969+0000 7f4e4e59c640 1 -- 172.21.15.119:0/1376350179 >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 msgr2=0x7f4e500ccad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:36.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.969+0000 7f4e4e59c640 1 --2- 172.21.15.119:0/1376350179 >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 0x7f4e500ccad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.969+0000 7f4e4e59c640 1 -- 172.21.15.119:0/1376350179 >> v1:172.21.15.188:6789/0 conn(0x7f4e500ad8b0 legacy=0x7f4e50070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.969+0000 7f4e4e59c640 1 -- 172.21.15.119:0/1376350179 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500c93a0 con 0x7f4e5015d310 2023-11-08T19:28:36.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.969+0000 7f4e4ed9d640 1 --2- 172.21.15.119:0/1376350179 >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 0x7f4e500ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:36.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.971+0000 7f4e4e59c640 1 --2- 172.21.15.119:0/1376350179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e5015d310 0x7f4e50071840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e400097a0 tx=0x7f4e4002dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432a47c40cf35626 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.971+0000 7f4e4d59a640 1 -- 172.21.15.119:0/1376350179 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4002f880 con 0x7f4e5015d310 2023-11-08T19:28:36.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.971+0000 7f4e4d59a640 1 -- 172.21.15.119:0/1376350179 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4002f9e0 con 0x7f4e5015d310 2023-11-08T19:28:36.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.971+0000 7f4e4d59a640 1 -- 172.21.15.119:0/1376350179 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4002fd70 con 0x7f4e5015d310 2023-11-08T19:28:36.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 -- 172.21.15.119:0/1376350179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e5015d310 msgr2=0x7f4e50071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/1376350179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e5015d310 0x7f4e50071840 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4e400097a0 tx=0x7f4e4002dac0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 -- 172.21.15.119:0/1376350179 shutdown_connections 2023-11-08T19:28:36.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/1376350179 >> v2:172.21.15.188:3300/0 conn(0x7f4e500cc640 0x7f4e500ccad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/1376350179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e5015d310 0x7f4e50071840 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 -- 172.21.15.119:0/1376350179 >> 172.21.15.119:0/1376350179 conn(0x7f4e500b0560 msgr2=0x7f4e500b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:36.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 -- 172.21.15.119:0/1376350179 shutdown_connections 2023-11-08T19:28:36.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.972+0000 7f4e4f59e640 1 -- 172.21.15.119:0/1376350179 wait complete. 2023-11-08T19:28:36.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.974+0000 7f4e4f59e640 1 Processor -- start 2023-11-08T19:28:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.974+0000 7f4e4f59e640 1 -- start start 2023-11-08T19:28:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 0x7f4e5014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e500cc640 0x7f4e500b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5015d310 0x7f4e500b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500b4360 con 0x7f4e500ad8b0 2023-11-08T19:28:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500b3000 con 0x7f4e5015d310 2023-11-08T19:28:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e500b32c0 con 0x7f4e500cc640 2023-11-08T19:28:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 0x7f4e5014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 0x7f4e5014a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35236/0 (socket says 172.21.15.119:35236) 2023-11-08T19:28:36.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.975+0000 7f4e4e59c640 1 -- 172.21.15.119:0/4081149728 learned_addr learned my addr 172.21.15.119:0/4081149728 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:36.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4ed9d640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5015d310 0x7f4e500b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4e59c640 1 -- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e500cc640 msgr2=0x7f4e500b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4dd9b640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e500cc640 0x7f4e500b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4e59c640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e500cc640 0x7f4e500b2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4e59c640 1 -- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5015d310 msgr2=0x7f4e500b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4e59c640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5015d310 0x7f4e500b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4e59c640 1 -- 172.21.15.119:0/4081149728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e48009b30 con 0x7f4e500ad8b0 2023-11-08T19:28:36.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4ed9d640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5015d310 0x7f4e500b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4e59c640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 0x7f4e5014a390 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40030a90 con 0x7f4e500ad8b0 2023-11-08T19:28:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e40030c10 con 0x7f4e500ad8b0 2023-11-08T19:28:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.976+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e40009400 con 0x7f4e500ad8b0 2023-11-08T19:28:36.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.977+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e40042410 con 0x7f4e500ad8b0 2023-11-08T19:28:36.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.977+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e500b3880 con 0x7f4e500ad8b0 2023-11-08T19:28:36.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.978+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5004ece0 con 0x7f4e500ad8b0 2023-11-08T19:28:36.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.979+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e40039400 con 0x7f4e500ad8b0 2023-11-08T19:28:36.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.980+0000 7f4e3effd640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4e3006ae70 0x7f4e3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:36.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.980+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f4e400ab6d0 con 0x7f4e500ad8b0 2023-11-08T19:28:36.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.980+0000 7f4e4dd9b640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4e3006ae70 0x7f4e3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:36.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.981+0000 7f4e4dd9b640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4e3006ae70 0x7f4e3006d330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e500cd8e0 tx=0x7f4e44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:36.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:36.984+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e40076100 con 0x7f4e500ad8b0 2023-11-08T19:28:37.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.212+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4e500c5890 con 0x7f4e500ad8b0 2023-11-08T19:28:37.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.758+0000 7f4e3effd640 1 -- 172.21.15.119:0/4081149728 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4e40079720 con 0x7f4e500ad8b0 2023-11-08T19:28:37.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4e3006ae70 msgr2=0x7f4e3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:37.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4e3006ae70 0x7f4e3006d330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e500cd8e0 tx=0x7f4e44009040 comp rx=0 tx=0).stop 2023-11-08T19:28:37.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 msgr2=0x7f4e5014a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:37.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 0x7f4e5014a390 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4e40000c00 tx=0x7f4e400046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:37.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 shutdown_connections 2023-11-08T19:28:37.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4e3006ae70 0x7f4e3006d330 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:37.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4e5015d310 0x7f4e500b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4e500cc640 0x7f4e500b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 --2- 172.21.15.119:0/4081149728 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4e500ad8b0 0x7f4e5014a390 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:37.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 >> 172.21.15.119:0/4081149728 conn(0x7f4e500b0560 msgr2=0x7f4e500ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 shutdown_connections 2023-11-08T19:28:37.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:37.762+0000 7f4e4f59e640 1 -- 172.21.15.119:0/4081149728 wait complete. 2023-11-08T19:28:37.787 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:28:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.049+0000 7f8c9b7e2640 1 Processor -- start 2023-11-08T19:28:38.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.049+0000 7f8c9b7e2640 1 -- start start 2023-11-08T19:28:38.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c9b7e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c9b7e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c94144960 0x7f8c940cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c9b7e2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c9415f1d0 0x7f8c940c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c9b7e2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c940c9150 con 0x7f8c94144960 2023-11-08T19:28:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c9b7e2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c9415ea60 con 0x7f8c9415f1d0 2023-11-08T19:28:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c9b7e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940cb270 con 0x7f8c94144580 2023-11-08T19:28:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c98d7d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c94144960 0x7f8c940cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c98d7d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8c94144960 0x7f8c940cad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35242/0 (socket says 172.21.15.119:35242) 2023-11-08T19:28:38.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.051+0000 7f8c98d7d640 1 -- 172.21.15.119:0/1107223567 learned_addr learned my addr 172.21.15.119:0/1107223567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:38.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c9957e640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c98d7d640 1 -- 172.21.15.119:0/1107223567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 msgr2=0x7f8c9415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:38.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c99d7f640 1 --2- 172.21.15.119:0/1107223567 >> v2:172.21.15.188:3300/0 conn(0x7f8c9415f1d0 0x7f8c940c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c98d7d640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c98d7d640 1 -- 172.21.15.119:0/1107223567 >> v2:172.21.15.188:3300/0 conn(0x7f8c9415f1d0 msgr2=0x7f8c940c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:38.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c98d7d640 1 --2- 172.21.15.119:0/1107223567 >> v2:172.21.15.188:3300/0 conn(0x7f8c9415f1d0 0x7f8c940c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c98d7d640 1 -- 172.21.15.119:0/1107223567 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940c9360 con 0x7f8c94144960 2023-11-08T19:28:38.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c9957e640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:38.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.052+0000 7f8c99d7f640 1 --2- 172.21.15.119:0/1107223567 >> v2:172.21.15.188:3300/0 conn(0x7f8c9415f1d0 0x7f8c940c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:38.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.053+0000 7f8c98d7d640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c940cad30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8c940c90e0 tx=0x7f8c8002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1006a3a007aa572 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:38.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.053+0000 7f8c93fff640 1 -- 172.21.15.119:0/1107223567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800317e0 con 0x7f8c94144960 2023-11-08T19:28:38.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.053+0000 7f8c93fff640 1 -- 172.21.15.119:0/1107223567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80031940 con 0x7f8c94144960 2023-11-08T19:28:38.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.053+0000 7f8c93fff640 1 -- 172.21.15.119:0/1107223567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80031cd0 con 0x7f8c94144960 2023-11-08T19:28:38.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/1107223567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 msgr2=0x7f8c940cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:38.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c940cad30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8c940c90e0 tx=0x7f8c8002b880 comp rx=0 tx=0).stop 2023-11-08T19:28:38.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/1107223567 shutdown_connections 2023-11-08T19:28:38.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/1107223567 >> v2:172.21.15.188:3300/0 conn(0x7f8c9415f1d0 0x7f8c940c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c940cad30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/1107223567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/1107223567 >> 172.21.15.119:0/1107223567 conn(0x7f8c940cb610 msgr2=0x7f8c940cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:38.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/1107223567 shutdown_connections 2023-11-08T19:28:38.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.054+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/1107223567 wait complete. 2023-11-08T19:28:38.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.056+0000 7f8c9b7e2640 1 Processor -- start 2023-11-08T19:28:38.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.056+0000 7f8c9b7e2640 1 -- start start 2023-11-08T19:28:38.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c9b7e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9413ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c9b7e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c9413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c9b7e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c9415f1d0 0x7f8c940c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c9b7e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c5bd0 con 0x7f8c94144960 2023-11-08T19:28:38.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c9b7e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c5e70 con 0x7f8c9415f1d0 2023-11-08T19:28:38.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c9b7e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c940c6150 con 0x7f8c94144580 2023-11-08T19:28:38.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.057+0000 7f8c98d7d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c9413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c9413bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35252/0 (socket says 172.21.15.119:35252) 2023-11-08T19:28:38.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 -- 172.21.15.119:0/707303030 learned_addr learned my addr 172.21.15.119:0/707303030 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:38.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c9957e640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9413ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c99d7f640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c9415f1d0 0x7f8c940c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 -- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 msgr2=0x7f8c9413ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:38.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9413ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 -- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c9415f1d0 msgr2=0x7f8c940c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:38.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c9415f1d0 0x7f8c940c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 -- 172.21.15.119:0/707303030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c800094d0 con 0x7f8c94144960 2023-11-08T19:28:38.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.058+0000 7f8c98d7d640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c9413bea0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8c8002b8c0 tx=0x7f8c800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:38.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.059+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80032980 con 0x7f8c94144960 2023-11-08T19:28:38.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.059+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80032b00 con 0x7f8c94144960 2023-11-08T19:28:38.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.059+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c940c6430 con 0x7f8c94144960 2023-11-08T19:28:38.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.059+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800453f0 con 0x7f8c94144960 2023-11-08T19:28:38.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.059+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9415a8c0 con 0x7f8c94144960 2023-11-08T19:28:38.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.060+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c94074430 con 0x7f8c94144960 2023-11-08T19:28:38.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.061+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c8004d3f0 con 0x7f8c94144960 2023-11-08T19:28:38.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.062+0000 7f8c91ffb640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c7806ae70 0x7f8c7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:38.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.062+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7206+0+0 (secure 0 0 0) 0x7f8c800abdd0 con 0x7f8c94144960 2023-11-08T19:28:38.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.062+0000 7f8c9957e640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c7806ae70 0x7f8c7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:38.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.063+0000 7f8c9957e640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c7806ae70 0x7f8c7806d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c88007130 tx=0x7f8c880079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:38.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.065+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c800769a0 con 0x7f8c94144960 2023-11-08T19:28:38.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.260+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c9406cb10 con 0x7f8c94144960 2023-11-08T19:28:38.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.898+0000 7f8c91ffb640 1 -- 172.21.15.119:0/707303030 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8c80079fc0 con 0x7f8c94144960 2023-11-08T19:28:38.901 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:28:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c7806ae70 msgr2=0x7f8c7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c7806ae70 0x7f8c7806d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c88007130 tx=0x7f8c880079c0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 msgr2=0x7f8c9413bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c9413bea0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8c8002b8c0 tx=0x7f8c800046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 shutdown_connections 2023-11-08T19:28:38.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8c7806ae70 0x7f8c7806d330 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c9415f1d0 0x7f8c940c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8c94144960 0x7f8c9413bea0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 --2- 172.21.15.119:0/707303030 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c94144580 0x7f8c9413ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:38.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.902+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 >> 172.21.15.119:0/707303030 conn(0x7f8c940cb610 msgr2=0x7f8c9406f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:38.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.904+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 shutdown_connections 2023-11-08T19:28:38.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:38.904+0000 7f8c9b7e2640 1 -- 172.21.15.119:0/707303030 wait complete. 2023-11-08T19:28:38.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:28:39.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.144+0000 7fdc5359e640 1 Processor -- start 2023-11-08T19:28:39.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.144+0000 7fdc5359e640 1 -- start start 2023-11-08T19:28:39.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.145+0000 7fdc5359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc540cc400 0x7fdc540cb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:39.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.145+0000 7fdc5359e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc540cc7e0 0x7fdc540cb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:39.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.145+0000 7fdc5359e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc540cce80 con 0x7fdc540ce7c0 2023-11-08T19:28:39.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.145+0000 7fdc5359e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc540b3070 con 0x7fdc540cc7e0 2023-11-08T19:28:39.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.145+0000 7fdc5359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc540b35a0 con 0x7fdc540cc400 2023-11-08T19:28:39.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc52d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc540cc7e0 0x7fdc540cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:39.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc49d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdc540ce7c0 0x7fdc54161390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45764/0 (socket says 172.21.15.119:45764) 2023-11-08T19:28:39.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc49d9b640 1 -- 172.21.15.119:0/1650242967 learned_addr learned my addr 172.21.15.119:0/1650242967 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:39.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc52d9d640 1 -- 172.21.15.119:0/1650242967 >> v2:172.21.15.188:3300/0 conn(0x7fdc540cc400 msgr2=0x7fdc540cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:39.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc5259c640 1 --2- 172.21.15.119:0/1650242967 >> v2:172.21.15.188:3300/0 conn(0x7fdc540cc400 0x7fdc540cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:39.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc52d9d640 1 --2- 172.21.15.119:0/1650242967 >> v2:172.21.15.188:3300/0 conn(0x7fdc540cc400 0x7fdc540cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc52d9d640 1 -- 172.21.15.119:0/1650242967 >> v1:172.21.15.119:6789/0 conn(0x7fdc540ce7c0 legacy=0x7fdc54161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:39.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc52d9d640 1 -- 172.21.15.119:0/1650242967 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc540ce0c0 con 0x7fdc540cc7e0 2023-11-08T19:28:39.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.146+0000 7fdc51d9b640 1 -- 172.21.15.119:0/1650242967 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382988344 0 0) 0x7fdc540cce80 con 0x7fdc540ce7c0 2023-11-08T19:28:39.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.147+0000 7fdc52d9d640 1 --2- 172.21.15.119:0/1650242967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc7e0 0x7fdc540cb780 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdc540d1c90 tx=0x7fdc4c02bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe96109be20f8d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:39.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc51d9b640 1 -- 172.21.15.119:0/1650242967 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c02cb50 con 0x7fdc540cc7e0 2023-11-08T19:28:39.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc51d9b640 1 -- 172.21.15.119:0/1650242967 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc4c02ccb0 con 0x7fdc540cc7e0 2023-11-08T19:28:39.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc51d9b640 1 -- 172.21.15.119:0/1650242967 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c034660 con 0x7fdc540cc7e0 2023-11-08T19:28:39.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 -- 172.21.15.119:0/1650242967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc7e0 msgr2=0x7fdc540cb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:39.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 --2- 172.21.15.119:0/1650242967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc7e0 0x7fdc540cb780 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fdc540d1c90 tx=0x7fdc4c02bda0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 -- 172.21.15.119:0/1650242967 shutdown_connections 2023-11-08T19:28:39.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 --2- 172.21.15.119:0/1650242967 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc7e0 0x7fdc540cb780 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 --2- 172.21.15.119:0/1650242967 >> v2:172.21.15.188:3300/0 conn(0x7fdc540cc400 0x7fdc540cb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 -- 172.21.15.119:0/1650242967 >> 172.21.15.119:0/1650242967 conn(0x7fdc5406a120 msgr2=0x7fdc5406a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:39.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 -- 172.21.15.119:0/1650242967 shutdown_connections 2023-11-08T19:28:39.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.148+0000 7fdc5359e640 1 -- 172.21.15.119:0/1650242967 wait complete. 2023-11-08T19:28:39.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.150+0000 7fdc5359e640 1 Processor -- start 2023-11-08T19:28:39.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.150+0000 7fdc5359e640 1 -- start start 2023-11-08T19:28:39.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 0x7fdc5414ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:39.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc540cc7e0 0x7fdc540b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:39.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc540ce7c0 0x7fdc54155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:39.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc54150490 con 0x7fdc540cc400 2023-11-08T19:28:39.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc540b5d60 con 0x7fdc540cc7e0 2023-11-08T19:28:39.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc540b4cc0 con 0x7fdc540ce7c0 2023-11-08T19:28:39.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.151+0000 7fdc5259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 0x7fdc5414ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:39.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 0x7fdc5414ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36270/0 (socket says 172.21.15.119:36270) 2023-11-08T19:28:39.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 -- 172.21.15.119:0/3201651873 learned_addr learned my addr 172.21.15.119:0/3201651873 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:39.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc49d9b640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc540cc7e0 0x7fdc540b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:39.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 -- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc540ce7c0 msgr2=0x7fdc54155520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:39.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc540ce7c0 0x7fdc54155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 -- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc540cc7e0 msgr2=0x7fdc540b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:39.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc540cc7e0 0x7fdc540b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 -- 172.21.15.119:0/3201651873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c009a70 con 0x7fdc540cc400 2023-11-08T19:28:39.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.152+0000 7fdc5259c640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 0x7fdc5414ff50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c7f0 tx=0x7fdc4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:39.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.153+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44004220 con 0x7fdc540cc400 2023-11-08T19:28:39.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.153+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc44004380 con 0x7fdc540cc400 2023-11-08T19:28:39.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.153+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc44007840 con 0x7fdc540cc400 2023-11-08T19:28:39.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.153+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc540b4fa0 con 0x7fdc540cc400 2023-11-08T19:28:39.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.153+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc540b5560 con 0x7fdc540cc400 2023-11-08T19:28:39.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.154+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5404ece0 con 0x7fdc540cc400 2023-11-08T19:28:39.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.155+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc44007bb0 con 0x7fdc540cc400 2023-11-08T19:28:39.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.156+0000 7fdc4b7fe640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3406ae70 0x7fdc3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:39.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.156+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6768+0+0 (secure 0 0 0) 0x7fdc44089990 con 0x7fdc540cc400 2023-11-08T19:28:39.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.156+0000 7fdc49d9b640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3406ae70 0x7fdc3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:39.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.157+0000 7fdc49d9b640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3406ae70 0x7fdc3406d330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc54156170 tx=0x7fdc3c007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:39.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.160+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc44054620 con 0x7fdc540cc400 2023-11-08T19:28:39.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.349+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc540b6b70 con 0x7fdc540cc400 2023-11-08T19:28:39.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.901+0000 7fdc4b7fe640 1 -- 172.21.15.119:0/3201651873 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdc4401e0d0 con 0x7fdc540cc400 2023-11-08T19:28:39.904 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-11-08T19:28:39.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3406ae70 msgr2=0x7fdc3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:39.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3406ae70 0x7fdc3406d330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fdc54156170 tx=0x7fdc3c007c00 comp rx=0 tx=0).stop 2023-11-08T19:28:39.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 msgr2=0x7fdc5414ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:39.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 0x7fdc5414ff50 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400c7f0 tx=0x7fdc4400ccb0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 shutdown_connections 2023-11-08T19:28:39.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc3406ae70 0x7fdc3406d330 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc540ce7c0 0x7fdc54155520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc540cc7e0 0x7fdc540b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 --2- 172.21.15.119:0/3201651873 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc540cc400 0x7fdc5414ff50 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:39.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.905+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 >> 172.21.15.119:0/3201651873 conn(0x7fdc5406a120 msgr2=0x7fdc540b0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:39.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.906+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 shutdown_connections 2023-11-08T19:28:39.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:39.908+0000 7fdc5359e640 1 -- 172.21.15.119:0/3201651873 wait complete. 2023-11-08T19:28:39.935 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:28:40.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.144+0000 7fd0d2444640 1 Processor -- start 2023-11-08T19:28:40.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.144+0000 7fd0d2444640 1 -- start start 2023-11-08T19:28:40.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0d2444640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0cc144550 0x7fd0cc0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0d2444640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0cbd20 con 0x7fd0cc15bce0 2023-11-08T19:28:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0d2444640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0cc0c43b0 con 0x7fd0cc0cb8c0 2023-11-08T19:28:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0d2444640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0cc15c150 con 0x7fd0cc144550 2023-11-08T19:28:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0cbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd0cc144550 0x7fd0cc0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0d09e1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd0cc15bce0 0x7fd0cc0c9340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45768/0 (socket says 172.21.15.119:45768) 2023-11-08T19:28:40.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.146+0000 7fd0d09e1640 1 -- 172.21.15.119:0/769642910 learned_addr learned my addr 172.21.15.119:0/769642910 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:40.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.147+0000 7fd0cbfff640 1 -- 172.21.15.119:0/769642910 >> v1:172.21.15.188:6789/0 conn(0x7fd0cc0cb8c0 legacy=0x7fd0cc15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:40.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.147+0000 7fd0cbfff640 1 -- 172.21.15.119:0/769642910 >> v1:172.21.15.119:6789/0 conn(0x7fd0cc15bce0 legacy=0x7fd0cc0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.147+0000 7fd0cbfff640 1 -- 172.21.15.119:0/769642910 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc15c340 con 0x7fd0cc144550 2023-11-08T19:28:40.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.148+0000 7fd0cbfff640 1 --2- 172.21.15.119:0/769642910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc144550 0x7fd0cc0cb380 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009d90 tx=0x7fd0bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48abc5226788f49d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:40.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.148+0000 7fd0cb7fe640 1 -- 172.21.15.119:0/769642910 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc02cc60 con 0x7fd0cc144550 2023-11-08T19:28:40.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.148+0000 7fd0cb7fe640 1 -- 172.21.15.119:0/769642910 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc02cdc0 con 0x7fd0cc144550 2023-11-08T19:28:40.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.148+0000 7fd0cb7fe640 1 -- 172.21.15.119:0/769642910 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc034660 con 0x7fd0cc144550 2023-11-08T19:28:40.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 -- 172.21.15.119:0/769642910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc144550 msgr2=0x7fd0cc0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 --2- 172.21.15.119:0/769642910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc144550 0x7fd0cc0cb380 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009d90 tx=0x7fd0bc02beb0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 -- 172.21.15.119:0/769642910 shutdown_connections 2023-11-08T19:28:40.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 --2- 172.21.15.119:0/769642910 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc144550 0x7fd0cc0cb380 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 -- 172.21.15.119:0/769642910 >> 172.21.15.119:0/769642910 conn(0x7fd0cc0b0950 msgr2=0x7fd0cc0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:40.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 -- 172.21.15.119:0/769642910 shutdown_connections 2023-11-08T19:28:40.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.149+0000 7fd0d2444640 1 -- 172.21.15.119:0/769642910 wait complete. 2023-11-08T19:28:40.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.151+0000 7fd0d2444640 1 Processor -- start 2023-11-08T19:28:40.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.151+0000 7fd0d2444640 1 -- start start 2023-11-08T19:28:40.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d2444640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0cc0cb8c0 0x7fd0cc13db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d2444640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0cc144550 0x7fd0cc13f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d2444640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 0x7fd0cc15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d2444640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc13d680 con 0x7fd0cc15bce0 2023-11-08T19:28:40.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d2444640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc1411b0 con 0x7fd0cc144550 2023-11-08T19:28:40.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d2444640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc13f8b0 con 0x7fd0cc0cb8c0 2023-11-08T19:28:40.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d09e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 0x7fd0cc15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d09e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 0x7fd0cc15a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36296/0 (socket says 172.21.15.119:36296) 2023-11-08T19:28:40.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0d09e1640 1 -- 172.21.15.119:0/61393776 learned_addr learned my addr 172.21.15.119:0/61393776 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0cbfff640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0cc0cb8c0 0x7fd0cc13db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.153+0000 7fd0c37fe640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0cc144550 0x7fd0cc13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d09e1640 1 -- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0cc0cb8c0 msgr2=0x7fd0cc13db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d09e1640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0cc0cb8c0 0x7fd0cc13db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d09e1640 1 -- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0cc144550 msgr2=0x7fd0cc13f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d09e1640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0cc144550 0x7fd0cc13f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d09e1640 1 -- 172.21.15.119:0/61393776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc009a70 con 0x7fd0cc15bce0 2023-11-08T19:28:40.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0c37fe640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0cc144550 0x7fd0cc13f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:40.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d09e1640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 0x7fd0cc15a9c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c370 tx=0x7fd0c400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:40.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c4005ce0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c4005e40 con 0x7fd0cc15bce0 2023-11-08T19:28:40.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c40072d0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc140ed0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.154+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc13d310 con 0x7fd0cc15bce0 2023-11-08T19:28:40.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.156+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0cc13d7f0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.157+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0c40074f0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.158+0000 7fd0c97fa640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd0ac06ae70 0x7fd0ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.158+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fd0c4088d50 con 0x7fd0cc15bce0 2023-11-08T19:28:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.158+0000 7fd0cbfff640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd0ac06ae70 0x7fd0ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.159+0000 7fd0cbfff640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd0ac06ae70 0x7fd0ac06d330 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc036000 tx=0x7fd0bc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:40.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.162+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0c4053b80 con 0x7fd0cc15bce0 2023-11-08T19:28:40.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.352+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd0cc04ece0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.358+0000 7fd0c97fa640 1 -- 172.21.15.119:0/61393776 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd0c40571a0 con 0x7fd0cc15bce0 2023-11-08T19:28:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.360+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd0ac06ae70 msgr2=0x7fd0ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.360+0000 7fd0d2444640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd0ac06ae70 0x7fd0ac06d330 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc036000 tx=0x7fd0bc030000 comp rx=0 tx=0).stop 2023-11-08T19:28:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.360+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 msgr2=0x7fd0cc15a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.360+0000 7fd0d2444640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 0x7fd0cc15a9c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fd0c400c370 tx=0x7fd0c400c830 comp rx=0 tx=0).stop 2023-11-08T19:28:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 shutdown_connections 2023-11-08T19:28:40.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd0ac06ae70 0x7fd0ac06d330 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd0cc15bce0 0x7fd0cc15a9c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd0cc144550 0x7fd0cc13f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 --2- 172.21.15.119:0/61393776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd0cc0cb8c0 0x7fd0cc13db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 >> 172.21.15.119:0/61393776 conn(0x7fd0cc0b0950 msgr2=0x7fd0cc0ca040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:40.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 shutdown_connections 2023-11-08T19:28:40.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.361+0000 7fd0d2444640 1 -- 172.21.15.119:0/61393776 wait complete. 2023-11-08T19:28:40.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config' 2023-11-08T19:28:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.587+0000 7fc114d2c640 1 Processor -- start 2023-11-08T19:28:40.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.588+0000 7fc114d2c640 1 -- start start 2023-11-08T19:28:40.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc114d2c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc11015d410 0x7fc1100ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc114d2c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc11015e5f0 0x7fc11015d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc114d2c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11006b570 con 0x7fc11015dd30 2023-11-08T19:28:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc114d2c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11006c1f0 con 0x7fc11015d410 2023-11-08T19:28:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc114d2c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11006b850 con 0x7fc11015e5f0 2023-11-08T19:28:40.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc10e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc11015d410 0x7fc1100ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc10e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc11015d410 0x7fc1100ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36302/0 (socket says 172.21.15.119:36302) 2023-11-08T19:28:40.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.589+0000 7fc10e59c640 1 -- 172.21.15.119:0/1712149554 learned_addr learned my addr 172.21.15.119:0/1712149554 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.590+0000 7fc10e59c640 1 -- 172.21.15.119:0/1712149554 >> v2:172.21.15.188:3300/0 conn(0x7fc11015e5f0 msgr2=0x7fc11015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.590+0000 7fc10dd9b640 1 --2- 172.21.15.119:0/1712149554 >> v2:172.21.15.188:3300/0 conn(0x7fc11015e5f0 0x7fc11015d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.590+0000 7fc10e59c640 1 --2- 172.21.15.119:0/1712149554 >> v2:172.21.15.188:3300/0 conn(0x7fc11015e5f0 0x7fc11015d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.590+0000 7fc10e59c640 1 -- 172.21.15.119:0/1712149554 >> v1:172.21.15.188:6789/0 conn(0x7fc11015dd30 legacy=0x7fc110160e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.590+0000 7fc10e59c640 1 -- 172.21.15.119:0/1712149554 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11006bb30 con 0x7fc11015d410 2023-11-08T19:28:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.591+0000 7fc10e59c640 1 --2- 172.21.15.119:0/1712149554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015d410 0x7fc1100ce040 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc110070060 tx=0x7fc10002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a6d36cec6cfc0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.591+0000 7fc10d59a640 1 -- 172.21.15.119:0/1712149554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10002ca40 con 0x7fc11015d410 2023-11-08T19:28:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.591+0000 7fc10d59a640 1 -- 172.21.15.119:0/1712149554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10002cba0 con 0x7fc11015d410 2023-11-08T19:28:40.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.591+0000 7fc10d59a640 1 -- 172.21.15.119:0/1712149554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1000343f0 con 0x7fc11015d410 2023-11-08T19:28:40.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 -- 172.21.15.119:0/1712149554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015d410 msgr2=0x7fc1100ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 --2- 172.21.15.119:0/1712149554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015d410 0x7fc1100ce040 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc110070060 tx=0x7fc10002bea0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 -- 172.21.15.119:0/1712149554 shutdown_connections 2023-11-08T19:28:40.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 --2- 172.21.15.119:0/1712149554 >> v2:172.21.15.188:3300/0 conn(0x7fc11015e5f0 0x7fc11015d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 --2- 172.21.15.119:0/1712149554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015d410 0x7fc1100ce040 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 -- 172.21.15.119:0/1712149554 >> 172.21.15.119:0/1712149554 conn(0x7fc1100d0db0 msgr2=0x7fc1100d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:40.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 -- 172.21.15.119:0/1712149554 shutdown_connections 2023-11-08T19:28:40.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.592+0000 7fc114d2c640 1 -- 172.21.15.119:0/1712149554 wait complete. 2023-11-08T19:28:40.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.594+0000 7fc114d2c640 1 Processor -- start 2023-11-08T19:28:40.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.594+0000 7fc114d2c640 1 -- start start 2023-11-08T19:28:40.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc114d2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc11015d410 0x7fc11015ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc114d2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc11015dd30 0x7fc1100c7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc114d2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 0x7fc1100cef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc114d2c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100c4ea0 con 0x7fc11015e5f0 2023-11-08T19:28:40.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc114d2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1100c75a0 con 0x7fc11015d410 2023-11-08T19:28:40.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc114d2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11015fcb0 con 0x7fc11015dd30 2023-11-08T19:28:40.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc10ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 0x7fc1100cef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.595+0000 7fc10ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 0x7fc1100cef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36304/0 (socket says 172.21.15.119:36304) 2023-11-08T19:28:40.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 -- 172.21.15.119:0/3903648391 learned_addr learned my addr 172.21.15.119:0/3903648391 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:40.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10e59c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc11015d410 0x7fc11015ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10dd9b640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc11015dd30 0x7fc1100c7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 -- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc11015dd30 msgr2=0x7fc1100c7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc11015dd30 0x7fc1100c7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 -- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc11015d410 msgr2=0x7fc11015ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc11015d410 0x7fc11015ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 -- 172.21.15.119:0/3903648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc100009b30 con 0x7fc11015e5f0 2023-11-08T19:28:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc10ed9d640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 0x7fc1100cef40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc10800c410 tx=0x7fc10800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108011790 con 0x7fc11015e5f0 2023-11-08T19:28:40.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1080118f0 con 0x7fc11015e5f0 2023-11-08T19:28:40.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.596+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc108011ca0 con 0x7fc11015e5f0 2023-11-08T19:28:40.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.597+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1100cf480 con 0x7fc11015e5f0 2023-11-08T19:28:40.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.597+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11015f380 con 0x7fc11015e5f0 2023-11-08T19:28:40.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.598+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1100c4fe0 con 0x7fc11015e5f0 2023-11-08T19:28:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.599+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc10801c460 con 0x7fc11015e5f0 2023-11-08T19:28:40.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.600+0000 7fc106ffd640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0f006ae70 0x7fc0f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:40.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.600+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fc108089680 con 0x7fc11015e5f0 2023-11-08T19:28:40.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.600+0000 7fc10e59c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0f006ae70 0x7fc0f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:40.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.601+0000 7fc10e59c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0f006ae70 0x7fc0f006d330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc100035000 tx=0x7fc100030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:40.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.604+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc108054370 con 0x7fc11015e5f0 2023-11-08T19:28:40.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.866+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]} v 0) v1 -- 0x7fc110074430 con 0x7fc11015e5f0 2023-11-08T19:28:40.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.916+0000 7fc106ffd640 1 -- 172.21.15.119:0/3903648391 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_allow_new_snaps_config"]}]=0 v236) v1 ==== 148+0+0 (secure 0 0 0) 0x7fc108057990 con 0x7fc11015e5f0 2023-11-08T19:28:40.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0f006ae70 msgr2=0x7fc0f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0f006ae70 0x7fc0f006d330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc100035000 tx=0x7fc100030000 comp rx=0 tx=0).stop 2023-11-08T19:28:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 msgr2=0x7fc1100cef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 0x7fc1100cef40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc10800c410 tx=0x7fc10800c8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 shutdown_connections 2023-11-08T19:28:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc0f006ae70 0x7fc0f006d330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc11015e5f0 0x7fc1100cef40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc11015dd30 0x7fc1100c7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 --2- 172.21.15.119:0/3903648391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc11015d410 0x7fc11015ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:40.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 >> 172.21.15.119:0/3903648391 conn(0x7fc1100d0db0 msgr2=0x7fc11015b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 shutdown_connections 2023-11-08T19:28:40.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:40.919+0000 7fc114d2c640 1 -- 172.21.15.119:0/3903648391 wait complete. 2023-11-08T19:28:40.940 INFO:tasks.cephfs_test_runner:Check whether 'allow_new_snaps' setting works ... ok 2023-11-08T19:28:40.940 INFO:tasks.cephfs_test_runner:Starting test: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) check snaptable transcation 2023-11-08T19:28:40.941 INFO:tasks.cephfs_test_runner:test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:28:40.941 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:41.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.132+0000 7f1af7fff640 1 Processor -- start 2023-11-08T19:28:41.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.132+0000 7f1af7fff640 1 -- start start 2023-11-08T19:28:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af7fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1af80adee0 0x7f1af815d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af806bc10 0x7f1af806c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af7fff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1af806b780 con 0x7f1af80ac8d0 2023-11-08T19:28:41.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af7fff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1af815dd30 con 0x7f1af80adee0 2023-11-08T19:28:41.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af7fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815e010 con 0x7f1af806bc10 2023-11-08T19:28:41.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af6ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1af80ac8d0 0x7f1af80accb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45784/0 (socket says 172.21.15.119:45784) 2023-11-08T19:28:41.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af6ffd640 1 -- 172.21.15.119:0/2699688498 learned_addr learned my addr 172.21.15.119:0/2699688498 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1aeffff640 1 --2- 172.21.15.119:0/2699688498 >> v2:172.21.15.188:3300/0 conn(0x7f1af80adee0 0x7f1af815d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.134+0000 7f1af77fe640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af806bc10 0x7f1af806c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247784905 0 0) 0x7f1af806b780 con 0x7f1af80ac8d0 2023-11-08T19:28:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ae4003520 con 0x7f1af80ac8d0 2023-11-08T19:28:41.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3338565950 0 0) 0x7f1ae4003520 con 0x7f1af80ac8d0 2023-11-08T19:28:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af806bc10 msgr2=0x7f1af806c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af806bc10 0x7f1af806c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 >> v2:172.21.15.188:3300/0 conn(0x7f1af80adee0 msgr2=0x7f1af815d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 --2- 172.21.15.119:0/2699688498 >> v2:172.21.15.188:3300/0 conn(0x7f1af80adee0 0x7f1af815d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af815e2f0 con 0x7f1af80ac8d0 2023-11-08T19:28:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f1ae8001670 con 0x7f1af80ac8d0 2023-11-08T19:28:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1aeffff640 1 --2- 172.21.15.119:0/2699688498 >> v2:172.21.15.188:3300/0 conn(0x7f1af80adee0 0x7f1af815d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:41.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 >> v1:172.21.15.119:6789/0 conn(0x7f1af80ac8d0 legacy=0x7f1af80accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ae4003970 0x7f1ae4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae40047c0 0x7f1ae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815dd30 con 0x7f1ae4005180 2023-11-08T19:28:41.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815e010 con 0x7f1ae40047c0 2023-11-08T19:28:41.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae400bff0 con 0x7f1ae4003970 2023-11-08T19:28:41.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.135+0000 7f1af77fe640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af806bc10 0x7f1af806c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:41.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af77fe640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af6ffd640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ae4003970 0x7f1ae4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1aeffff640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae40047c0 0x7f1ae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af77fe640 1 -- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ae4003970 msgr2=0x7f1ae4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:41.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af77fe640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ae4003970 0x7f1ae4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af77fe640 1 -- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae40047c0 msgr2=0x7f1ae4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:41.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af77fe640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae40047c0 0x7f1ae4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af77fe640 1 -- 172.21.15.119:0/2699688498 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae4003520 con 0x7f1ae4005180 2023-11-08T19:28:41.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1af6ffd640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ae4003970 0x7f1ae4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:41.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.136+0000 7f1aeffff640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ae40047c0 0x7f1ae4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:41.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.137+0000 7f1af77fe640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1ae400b980 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009f00 tx=0x7f1ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cbc9b66b2326d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:41.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.137+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af002cc70 con 0x7f1ae4005180 2023-11-08T19:28:41.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.137+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af002cdd0 con 0x7f1ae4005180 2023-11-08T19:28:41.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.137+0000 7f1af67fc640 1 -- 172.21.15.119:0/2699688498 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0003bb0 con 0x7f1ae4005180 2023-11-08T19:28:41.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af7fff640 1 -- 172.21.15.119:0/2699688498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 msgr2=0x7f1ae400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af7fff640 1 --2- 172.21.15.119:0/2699688498 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1ae400b980 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f1af0009f00 tx=0x7f1ae40057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af77fe640 1 -- 172.21.15.119:0/2699688498 reap_dead start 2023-11-08T19:28:41.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af7fff640 1 -- 172.21.15.119:0/2699688498 shutdown_connections 2023-11-08T19:28:41.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af7fff640 1 -- 172.21.15.119:0/2699688498 >> 172.21.15.119:0/2699688498 conn(0x7f1af80b0560 msgr2=0x7f1af80b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:41.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af7fff640 1 -- 172.21.15.119:0/2699688498 shutdown_connections 2023-11-08T19:28:41.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.138+0000 7f1af7fff640 1 -- 172.21.15.119:0/2699688498 wait complete. 2023-11-08T19:28:41.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.140+0000 7f1af7fff640 1 Processor -- start 2023-11-08T19:28:41.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.140+0000 7f1af7fff640 1 -- start start 2023-11-08T19:28:41.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1af815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1af8159c50 0x7f1af814de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af80b3710 0x7f1af80b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af7fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af80b40d0 con 0x7f1ae4005180 2023-11-08T19:28:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af7fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815ca80 con 0x7f1af8159c50 2023-11-08T19:28:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af7fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1af815c7a0 con 0x7f1af80b3710 2023-11-08T19:28:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1af815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1af815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36316/0 (socket says 172.21.15.119:36316) 2023-11-08T19:28:41.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 -- 172.21.15.119:0/4103230447 learned_addr learned my addr 172.21.15.119:0/4103230447 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:41.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 -- 172.21.15.119:0/4103230447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af80b3710 msgr2=0x7f1af80b3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af80b3710 0x7f1af80b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 -- 172.21.15.119:0/4103230447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1af8159c50 msgr2=0x7f1af814de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1af8159c50 0x7f1af814de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.141+0000 7f1af6ffd640 1 -- 172.21.15.119:0/4103230447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af80cd4b0 con 0x7f1ae4005180 2023-11-08T19:28:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.142+0000 7f1af6ffd640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1af815c1b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800d860 tx=0x7f1ae800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.142+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80074e0 con 0x7f1ae4005180 2023-11-08T19:28:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.142+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae8007640 con 0x7f1ae4005180 2023-11-08T19:28:41.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.142+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae80049e0 con 0x7f1ae4005180 2023-11-08T19:28:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.142+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af80c3340 con 0x7f1ae4005180 2023-11-08T19:28:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.142+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af80c3900 con 0x7f1ae4005180 2023-11-08T19:28:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.143+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af80c3a90 con 0x7f1ae4005180 2023-11-08T19:28:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.144+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ae8004b40 con 0x7f1ae4005180 2023-11-08T19:28:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.145+0000 7f1aef7fe640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ad806aea0 0x7f1ad806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.145+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f1ae8089580 con 0x7f1ae4005180 2023-11-08T19:28:41.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.145+0000 7f1aeffff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ad806aea0 0x7f1ad806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.146+0000 7f1aeffff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ad806aea0 0x7f1ad806d360 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0006e80 tx=0x7f1ae0006df0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:41.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.149+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ae8054320 con 0x7f1ae4005180 2023-11-08T19:28:41.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.325+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1af8070720 con 0x7f1ae4005180 2023-11-08T19:28:41.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.326+0000 7f1aef7fe640 1 -- 172.21.15.119:0/4103230447 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+18478 (secure 0 0 0) 0x7f1ae8019090 con 0x7f1ae4005180 2023-11-08T19:28:41.329 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:41.330 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":87,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:39.899695+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/4016780496":"2023-11-09T19:28:36.751683+0000","172.21.15.119:6835/4016780496":"2023-11-09T19:28:36.751683+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:41.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ad806aea0 msgr2=0x7f1ad806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ad806aea0 0x7f1ad806d360 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0006e80 tx=0x7f1ae0006df0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 msgr2=0x7f1af815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1af815c1b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800d860 tx=0x7f1ae800dd20 comp rx=0 tx=0).stop 2023-11-08T19:28:41.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 shutdown_connections 2023-11-08T19:28:41.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ad806aea0 0x7f1ad806d360 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1af80b3710 0x7f1af80b3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1af8159c50 0x7f1af814de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 --2- 172.21.15.119:0/4103230447 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ae4005180 0x7f1af815c1b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 >> 172.21.15.119:0/4103230447 conn(0x7f1af80b0560 msgr2=0x7f1af806f890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 shutdown_connections 2023-11-08T19:28:41.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.329+0000 7f1af7fff640 1 -- 172.21.15.119:0/4103230447 wait complete. 2023-11-08T19:28:41.350 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:28:41.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:28:41.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.561+0000 7fd973fff640 1 Processor -- start 2023-11-08T19:28:41.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.561+0000 7fd973fff640 1 -- start start 2023-11-08T19:28:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd973fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd97415deb0 0x7fd9740cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd973fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd97415e290 0x7fd97415e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd973fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd974071720 con 0x7fd97415a8c0 2023-11-08T19:28:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd973fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9740cf5e0 con 0x7fd97415e290 2023-11-08T19:28:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd973fff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9740cf8c0 con 0x7fd97415deb0 2023-11-08T19:28:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd97415e290 0x7fd97415e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd97415e290 0x7fd97415e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36326/0 (socket says 172.21.15.119:36326) 2023-11-08T19:28:41.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 -- 172.21.15.119:0/1422286017 learned_addr learned my addr 172.21.15.119:0/1422286017 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:41.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd972ffd640 1 --2- 172.21.15.119:0/1422286017 >> v2:172.21.15.188:3300/0 conn(0x7fd97415deb0 0x7fd9740cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 -- 172.21.15.119:0/1422286017 >> v2:172.21.15.188:3300/0 conn(0x7fd97415deb0 msgr2=0x7fd9740cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 --2- 172.21.15.119:0/1422286017 >> v2:172.21.15.188:3300/0 conn(0x7fd97415deb0 0x7fd9740cccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 -- 172.21.15.119:0/1422286017 >> v1:172.21.15.188:6789/0 conn(0x7fd97415a8c0 legacy=0x7fd97415acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.562+0000 7fd9727fc640 1 -- 172.21.15.119:0/1422286017 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740710d0 con 0x7fd97415e290 2023-11-08T19:28:41.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.564+0000 7fd9727fc640 1 --2- 172.21.15.119:0/1422286017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd97415e700 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd9740cc600 tx=0x7fd96802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=182b957d4a18c939 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.564+0000 7fd971ffb640 1 -- 172.21.15.119:0/1422286017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9680317e0 con 0x7fd97415e290 2023-11-08T19:28:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.564+0000 7fd971ffb640 1 -- 172.21.15.119:0/1422286017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968031940 con 0x7fd97415e290 2023-11-08T19:28:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.564+0000 7fd971ffb640 1 -- 172.21.15.119:0/1422286017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968031cd0 con 0x7fd97415e290 2023-11-08T19:28:41.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 -- 172.21.15.119:0/1422286017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 msgr2=0x7fd97415e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 --2- 172.21.15.119:0/1422286017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd97415e700 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd9740cc600 tx=0x7fd96802b880 comp rx=0 tx=0).stop 2023-11-08T19:28:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 -- 172.21.15.119:0/1422286017 shutdown_connections 2023-11-08T19:28:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 --2- 172.21.15.119:0/1422286017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd97415e700 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 --2- 172.21.15.119:0/1422286017 >> v2:172.21.15.188:3300/0 conn(0x7fd97415deb0 0x7fd9740cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 -- 172.21.15.119:0/1422286017 >> 172.21.15.119:0/1422286017 conn(0x7fd9740ceac0 msgr2=0x7fd9740ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 -- 172.21.15.119:0/1422286017 shutdown_connections 2023-11-08T19:28:41.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.565+0000 7fd973fff640 1 -- 172.21.15.119:0/1422286017 wait complete. 2023-11-08T19:28:41.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.567+0000 7fd973fff640 1 Processor -- start 2023-11-08T19:28:41.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.567+0000 7fd973fff640 1 -- start start 2023-11-08T19:28:41.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.568+0000 7fd973fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97415a8c0 0x7fd97415d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.568+0000 7fd973fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97415deb0 0x7fd9740af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.568+0000 7fd973fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd974146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.568+0000 7fd973fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97415cd40 con 0x7fd97415e290 2023-11-08T19:28:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.568+0000 7fd973fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97415d710 con 0x7fd97415a8c0 2023-11-08T19:28:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.568+0000 7fd973fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9741503e0 con 0x7fd97415deb0 2023-11-08T19:28:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd974146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd974146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36330/0 (socket says 172.21.15.119:36330) 2023-11-08T19:28:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 -- 172.21.15.119:0/1278061097 learned_addr learned my addr 172.21.15.119:0/1278061097 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9727fc640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97415deb0 0x7fd9740af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd972ffd640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97415a8c0 0x7fd97415d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 -- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97415deb0 msgr2=0x7fd9740af7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97415deb0 0x7fd9740af7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 -- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97415a8c0 msgr2=0x7fd97415d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97415a8c0 0x7fd97415d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 -- 172.21.15.119:0/1278061097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9680094d0 con 0x7fd97415e290 2023-11-08T19:28:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.569+0000 7fd9737fe640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd974146920 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b990 tx=0x7fd96c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.570+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00cba0 con 0x7fd97415e290 2023-11-08T19:28:41.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.570+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c00cd00 con 0x7fd97415e290 2023-11-08T19:28:41.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.570+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c014680 con 0x7fd97415e290 2023-11-08T19:28:41.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.570+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97415c410 con 0x7fd97415e290 2023-11-08T19:28:41.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.570+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97415c9d0 con 0x7fd97415e290 2023-11-08T19:28:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.571+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97415ceb0 con 0x7fd97415e290 2023-11-08T19:28:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.572+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd96c01d3b0 con 0x7fd97415e290 2023-11-08T19:28:41.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.573+0000 7fd95b7fe640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd96006aea0 0x7fd96006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.573+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fd96c089800 con 0x7fd97415e290 2023-11-08T19:28:41.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.573+0000 7fd972ffd640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd96006aea0 0x7fd96006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:41.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.574+0000 7fd972ffd640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd96006aea0 0x7fd96006d360 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd96400a970 tx=0x7fd964008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:41.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.577+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd96c054620 con 0x7fd97415e290 2023-11-08T19:28:41.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.769+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd97404ece0 con 0x7fd97415e290 2023-11-08T19:28:41.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.770+0000 7fd95b7fe640 1 -- 172.21.15.119:0/1278061097 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd96c0190a0 con 0x7fd97415e290 2023-11-08T19:28:41.773 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:28:41.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd96006aea0 msgr2=0x7fd96006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd96006aea0 0x7fd96006d360 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd96400a970 tx=0x7fd964008040 comp rx=0 tx=0).stop 2023-11-08T19:28:41.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 msgr2=0x7fd974146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:41.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd974146920 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00b990 tx=0x7fd96c00be50 comp rx=0 tx=0).stop 2023-11-08T19:28:41.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 shutdown_connections 2023-11-08T19:28:41.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd96006aea0 0x7fd96006d360 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd97415e290 0x7fd974146920 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd97415deb0 0x7fd9740af7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 --2- 172.21.15.119:0/1278061097 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd97415a8c0 0x7fd97415d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:41.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.773+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 >> 172.21.15.119:0/1278061097 conn(0x7fd9740ceac0 msgr2=0x7fd9740ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:41.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.774+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 shutdown_connections 2023-11-08T19:28:41.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.774+0000 7fd973fff640 1 -- 172.21.15.119:0/1278061097 wait complete. 2023-11-08T19:28:41.794 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:28:41.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.994+0000 7f667db5e640 1 Processor -- start 2023-11-08T19:28:41.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.994+0000 7f667db5e640 1 -- start start 2023-11-08T19:28:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.995+0000 7f667db5e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6678144960 0x7f667815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.995+0000 7f667db5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f667815f1d0 0x7f66780c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.995+0000 7f667db5e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667815ea60 con 0x7f6678144580 2023-11-08T19:28:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.995+0000 7f667db5e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66780c9150 con 0x7f6678144960 2023-11-08T19:28:41.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.995+0000 7f667db5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66780c9360 con 0x7f667815f1d0 2023-11-08T19:28:41.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.996+0000 7f6676ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6678144960 0x7f667815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.996+0000 7f6676ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6678144960 0x7f667815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36334/0 (socket says 172.21.15.119:36334) 2023-11-08T19:28:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.996+0000 7f6676ffd640 1 -- 172.21.15.119:0/714622708 learned_addr learned my addr 172.21.15.119:0/714622708 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.996+0000 7f6677fff640 1 --2- 172.21.15.119:0/714622708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f667815f1d0 0x7f66780c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.997+0000 7f6676ffd640 1 -- 172.21.15.119:0/714622708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f667815f1d0 msgr2=0x7f66780c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.997+0000 7f6676ffd640 1 --2- 172.21.15.119:0/714622708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f667815f1d0 0x7f66780c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.997+0000 7f6676ffd640 1 -- 172.21.15.119:0/714622708 >> v1:172.21.15.188:6789/0 conn(0x7f6678144580 legacy=0x7f66780cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.997+0000 7f6676ffd640 1 -- 172.21.15.119:0/714622708 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66780c9640 con 0x7f6678144960 2023-11-08T19:28:42.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.998+0000 7f6676ffd640 1 --2- 172.21.15.119:0/714622708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6678144960 0x7f667815bff0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f66780c9290 tx=0x7f666c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf86d1c69a8e5f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.998+0000 7f66767fc640 1 -- 172.21.15.119:0/714622708 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c031640 con 0x7f6678144960 2023-11-08T19:28:42.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.998+0000 7f66767fc640 1 -- 172.21.15.119:0/714622708 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c0317a0 con 0x7f6678144960 2023-11-08T19:28:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.998+0000 7f66767fc640 1 -- 172.21.15.119:0/714622708 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c031b30 con 0x7f6678144960 2023-11-08T19:28:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 -- 172.21.15.119:0/714622708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6678144960 msgr2=0x7f667815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 --2- 172.21.15.119:0/714622708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6678144960 0x7f667815bff0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f66780c9290 tx=0x7f666c02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 -- 172.21.15.119:0/714622708 shutdown_connections 2023-11-08T19:28:42.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 --2- 172.21.15.119:0/714622708 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6678144960 0x7f667815bff0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 --2- 172.21.15.119:0/714622708 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f667815f1d0 0x7f66780c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 -- 172.21.15.119:0/714622708 >> 172.21.15.119:0/714622708 conn(0x7f66780cb610 msgr2=0x7f66780cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 -- 172.21.15.119:0/714622708 shutdown_connections 2023-11-08T19:28:42.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:41.999+0000 7f667db5e640 1 -- 172.21.15.119:0/714622708 wait complete. 2023-11-08T19:28:42.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.001+0000 7f667db5e640 1 Processor -- start 2023-11-08T19:28:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.001+0000 7f667db5e640 1 -- start start 2023-11-08T19:28:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f667db5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6678144580 0x7f66781f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f667db5e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6678144960 0x7f66780ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f667db5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 0x7f66780ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f667db5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780ade70 con 0x7f667815f1d0 2023-11-08T19:28:42.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f667db5e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780c8300 con 0x7f6678144960 2023-11-08T19:28:42.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f667db5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66781f3a00 con 0x7f6678144580 2023-11-08T19:28:42.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f6677fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 0x7f66780ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f6677fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 0x7f66780ae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36338/0 (socket says 172.21.15.119:36338) 2023-11-08T19:28:42.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f6677fff640 1 -- 172.21.15.119:0/2656415579 learned_addr learned my addr 172.21.15.119:0/2656415579 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:42.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f6676ffd640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6678144960 0x7f66780ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f6677fff640 1 -- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6678144580 msgr2=0x7f66781f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f66777fe640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6678144580 0x7f66781f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.002+0000 7f6677fff640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6678144580 0x7f66781f34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f6677fff640 1 -- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6678144960 msgr2=0x7f66780ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f6677fff640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6678144960 0x7f66780ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f6677fff640 1 -- 172.21.15.119:0/2656415579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0094d0 con 0x7f667815f1d0 2023-11-08T19:28:42.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f6676ffd640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6678144960 0x7f66780ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:42.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f6677fff640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 0x7f66780ae8b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6670007d20 tx=0x7f667000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670014420 con 0x7f667815f1d0 2023-11-08T19:28:42.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66700145a0 con 0x7f667815f1d0 2023-11-08T19:28:42.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6670014950 con 0x7f667815f1d0 2023-11-08T19:28:42.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66780acdd0 con 0x7f667815f1d0 2023-11-08T19:28:42.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.003+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66780ad390 con 0x7f667815f1d0 2023-11-08T19:28:42.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.005+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6678074430 con 0x7f667815f1d0 2023-11-08T19:28:42.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.005+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6670014ab0 con 0x7f667815f1d0 2023-11-08T19:28:42.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.006+0000 7f665ffff640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f666406ae70 0x7f666406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.006+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f667008a1e0 con 0x7f667815f1d0 2023-11-08T19:28:42.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.006+0000 7f66777fe640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f666406ae70 0x7f666406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.007+0000 7f66777fe640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f666406ae70 0x7f666406d330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f66780b4d40 tx=0x7f6668009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.010+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6670054ff0 con 0x7f667815f1d0 2023-11-08T19:28:42.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.201+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f667806cb10 con 0x7f667815f1d0 2023-11-08T19:28:42.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.201+0000 7f665ffff640 1 -- 172.21.15.119:0/2656415579 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6670058610 con 0x7f667815f1d0 2023-11-08T19:28:42.204 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:42.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f666406ae70 msgr2=0x7f666406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f666406ae70 0x7f666406d330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f66780b4d40 tx=0x7f6668009040 comp rx=0 tx=0).stop 2023-11-08T19:28:42.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 msgr2=0x7f66780ae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 0x7f66780ae8b0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6670007d20 tx=0x7f667000e4b0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 shutdown_connections 2023-11-08T19:28:42.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f666406ae70 0x7f666406d330 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f667815f1d0 0x7f66780ae8b0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6678144960 0x7f66780ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 --2- 172.21.15.119:0/2656415579 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6678144580 0x7f66781f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 >> 172.21.15.119:0/2656415579 conn(0x7f66780cb610 msgr2=0x7f6678072e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 shutdown_connections 2023-11-08T19:28:42.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.204+0000 7f667db5e640 1 -- 172.21.15.119:0/2656415579 wait complete. 2023-11-08T19:28:42.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:42.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.442+0000 7f7e0afc0640 1 Processor -- start 2023-11-08T19:28:42.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.442+0000 7f7e0afc0640 1 -- start start 2023-11-08T19:28:42.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.443+0000 7f7e0afc0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7e040c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.443+0000 7f7e0afc0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0415e440 con 0x7f7e0415d9f0 2023-11-08T19:28:42.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.443+0000 7f7e0afc0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0415e720 con 0x7f7e0406d250 2023-11-08T19:28:42.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.443+0000 7f7e0afc0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040c4f50 con 0x7f7e04143600 2023-11-08T19:28:42.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.444+0000 7f7e03fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7e0415d9f0 0x7f7e0406cb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45798/0 (socket says 172.21.15.119:45798) 2023-11-08T19:28:42.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.444+0000 7f7e03fff640 1 -- 172.21.15.119:0/1540259438 learned_addr learned my addr 172.21.15.119:0/1540259438 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:42.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.444+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292803681 0 0) 0x7f7e0415e440 con 0x7f7e0415d9f0 2023-11-08T19:28:42.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.444+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7e040c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.444+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df4003520 con 0x7f7e0415d9f0 2023-11-08T19:28:42.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2397158616 0 0) 0x7f7df4003520 con 0x7f7e0415d9f0 2023-11-08T19:28:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 msgr2=0x7f7e040c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7e040c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 >> v1:172.21.15.188:6789/0 conn(0x7f7e0406d250 legacy=0x7f7e040c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040c4b70 con 0x7f7e0415d9f0 2023-11-08T19:28:42.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f7df0001670 con 0x7f7e0415d9f0 2023-11-08T19:28:42.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 >> v1:172.21.15.119:6789/0 conn(0x7f7e0415d9f0 legacy=0x7f7e0406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df40039a0 0x7f7df4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40047c0 0x7f7df4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7df4005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040c4f50 con 0x7f7df40039a0 2023-11-08T19:28:42.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df400bb10 con 0x7f7df40047c0 2023-11-08T19:28:42.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df400dee0 con 0x7f7e04143600 2023-11-08T19:28:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df40039a0 0x7f7df4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e03fff640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40047c0 0x7f7df4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.445+0000 7f7e0955d640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7df4005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e08d5c640 1 -- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 msgr2=0x7f7df4005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7df4005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e08d5c640 1 -- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40047c0 msgr2=0x7f7df4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40047c0 0x7f7df4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e08d5c640 1 -- 172.21.15.119:0/1540259438 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df4003520 con 0x7f7df40039a0 2023-11-08T19:28:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e0955d640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e04143600 0x7f7df4005230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.446+0000 7f7e03fff640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40047c0 0x7f7df4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.447+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df40039a0 0x7f7df4003dc0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ac0 tx=0x7f7df40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2e4754f3e14ae3c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.447+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df802c6b0 con 0x7f7df40039a0 2023-11-08T19:28:42.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.447+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df802c810 con 0x7f7df40039a0 2023-11-08T19:28:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.447+0000 7f7e037fe640 1 -- 172.21.15.119:0/1540259438 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df802cba0 con 0x7f7df40039a0 2023-11-08T19:28:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0afc0640 1 -- 172.21.15.119:0/1540259438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df40039a0 msgr2=0x7f7df4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/1540259438 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7df40039a0 0x7f7df4003dc0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ac0 tx=0x7f7df40057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0955d640 1 -- 172.21.15.119:0/1540259438 reap_dead start 2023-11-08T19:28:42.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0afc0640 1 -- 172.21.15.119:0/1540259438 shutdown_connections 2023-11-08T19:28:42.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0afc0640 1 -- 172.21.15.119:0/1540259438 >> 172.21.15.119:0/1540259438 conn(0x7f7e040cb610 msgr2=0x7f7e040cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0afc0640 1 -- 172.21.15.119:0/1540259438 shutdown_connections 2023-11-08T19:28:42.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.448+0000 7f7e0afc0640 1 -- 172.21.15.119:0/1540259438 wait complete. 2023-11-08T19:28:42.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.450+0000 7f7e0afc0640 1 Processor -- start 2023-11-08T19:28:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.450+0000 7f7e0afc0640 1 -- start start 2023-11-08T19:28:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0afc0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40039a0 0x7f7e0415d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0afc0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e040ae460 0x7f7e040ae8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0afc0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 0x7f7e040704d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0afc0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e04070a10 con 0x7f7e04070050 2023-11-08T19:28:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0afc0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0415d7f0 con 0x7f7df40039a0 2023-11-08T19:28:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0afc0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e040aef60 con 0x7f7e040ae460 2023-11-08T19:28:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0955d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 0x7f7e040704d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0955d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 0x7f7e040704d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36362/0 (socket says 172.21.15.119:36362) 2023-11-08T19:28:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e0955d640 1 -- 172.21.15.119:0/3012318622 learned_addr learned my addr 172.21.15.119:0/3012318622 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40039a0 0x7f7e0415d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.451+0000 7f7e03fff640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e040ae460 0x7f7e040ae8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0955d640 1 -- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e040ae460 msgr2=0x7f7e040ae8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0955d640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e040ae460 0x7f7e040ae8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0955d640 1 -- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40039a0 msgr2=0x7f7e0415d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0955d640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40039a0 0x7f7e0415d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0955d640 1 -- 172.21.15.119:0/3012318622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e040af240 con 0x7f7e04070050 2023-11-08T19:28:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e03fff640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e040ae460 0x7f7e040ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0955d640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 0x7f7e040704d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7df4004740 tx=0x7f7dfc00d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003b70 con 0x7f7e04070050 2023-11-08T19:28:42.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc003cd0 con 0x7f7e04070050 2023-11-08T19:28:42.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc005310 con 0x7f7e04070050 2023-11-08T19:28:42.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e040af520 con 0x7f7e04070050 2023-11-08T19:28:42.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.452+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e040b0400 con 0x7f7e04070050 2023-11-08T19:28:42.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.453+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e040afa40 con 0x7f7e04070050 2023-11-08T19:28:42.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.454+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7dfc005470 con 0x7f7e04070050 2023-11-08T19:28:42.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.455+0000 7f7e017fa640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7de806ae70 0x7f7de806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.455+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f7dfc089440 con 0x7f7e04070050 2023-11-08T19:28:42.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.456+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7de806ae70 0x7f7de806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.456+0000 7f7e08d5c640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7de806ae70 0x7f7de806d330 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ac0 tx=0x7f7df8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.460+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7dfc0541c0 con 0x7f7e04070050 2023-11-08T19:28:42.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.651+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f7e040c4230 con 0x7f7e04070050 2023-11-08T19:28:42.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.652+0000 7f7e017fa640 1 -- 172.21.15.119:0/3012318622 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v87) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7dfc0577e0 con 0x7f7e04070050 2023-11-08T19:28:42.653 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:42.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.654+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7de806ae70 msgr2=0x7f7de806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.654+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7de806ae70 0x7f7de806d330 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f7df8009ac0 tx=0x7f7df8030000 comp rx=0 tx=0).stop 2023-11-08T19:28:42.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.654+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 msgr2=0x7f7e040704d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.654+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 0x7f7e040704d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7df4004740 tx=0x7f7dfc00d920 comp rx=0 tx=0).stop 2023-11-08T19:28:42.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 shutdown_connections 2023-11-08T19:28:42.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7de806ae70 0x7f7de806d330 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7e04070050 0x7f7e040704d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e040ae460 0x7f7e040ae8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 --2- 172.21.15.119:0/3012318622 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7df40039a0 0x7f7e0415d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 >> 172.21.15.119:0/3012318622 conn(0x7f7e040cb610 msgr2=0x7f7e040cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 shutdown_connections 2023-11-08T19:28:42.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.655+0000 7f7e0afc0640 1 -- 172.21.15.119:0/3012318622 wait complete. 2023-11-08T19:28:42.676 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:42.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.894+0000 7fb00af7c640 1 Processor -- start 2023-11-08T19:28:42.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.894+0000 7fb00af7c640 1 -- start start 2023-11-08T19:28:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.896+0000 7fb00af7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 0x7fb00415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.896+0000 7fb00af7c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb004144960 0x7fb0040cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.896+0000 7fb00af7c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00415ea60 con 0x7fb00415f1d0 2023-11-08T19:28:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.896+0000 7fb00af7c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0040c9150 con 0x7fb004144960 2023-11-08T19:28:42.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.896+0000 7fb00af7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040cb270 con 0x7fb004144580 2023-11-08T19:28:42.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.896+0000 7fb008d18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 0x7fb00415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb003fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb004144960 0x7fb0040cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb008d18640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 0x7fb00415bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:43932/0 (socket says 172.21.15.119:43932) 2023-11-08T19:28:42.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb008d18640 1 -- 172.21.15.119:0/2640308353 learned_addr learned my addr 172.21.15.119:0/2640308353 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:42.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb008d18640 1 -- 172.21.15.119:0/2640308353 >> v2:172.21.15.188:3300/0 conn(0x7fb004144960 msgr2=0x7fb0040cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:42.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb008d18640 1 --2- 172.21.15.119:0/2640308353 >> v2:172.21.15.188:3300/0 conn(0x7fb004144960 0x7fb0040cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb008d18640 1 -- 172.21.15.119:0/2640308353 >> v1:172.21.15.188:6789/0 conn(0x7fb00415f1d0 legacy=0x7fb0040c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.897+0000 7fb008d18640 1 -- 172.21.15.119:0/2640308353 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040c9360 con 0x7fb004144580 2023-11-08T19:28:42.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.899+0000 7fb008d18640 1 --2- 172.21.15.119:0/2640308353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 0x7fb00415bff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1241ae2d631df608 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.899+0000 7fb0037fe640 1 -- 172.21.15.119:0/2640308353 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8031640 con 0x7fb004144580 2023-11-08T19:28:42.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.899+0000 7fb0037fe640 1 -- 172.21.15.119:0/2640308353 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff80317a0 con 0x7fb004144580 2023-11-08T19:28:42.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.899+0000 7fb0037fe640 1 -- 172.21.15.119:0/2640308353 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8031b30 con 0x7fb004144580 2023-11-08T19:28:42.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 -- 172.21.15.119:0/2640308353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 msgr2=0x7fb00415bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2640308353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 0x7fb00415bff0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 -- 172.21.15.119:0/2640308353 shutdown_connections 2023-11-08T19:28:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2640308353 >> v2:172.21.15.188:3300/0 conn(0x7fb004144960 0x7fb0040cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2640308353 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144580 0x7fb00415bff0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 -- 172.21.15.119:0/2640308353 >> 172.21.15.119:0/2640308353 conn(0x7fb0040cb610 msgr2=0x7fb0040cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 -- 172.21.15.119:0/2640308353 shutdown_connections 2023-11-08T19:28:42.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.900+0000 7fb00af7c640 1 -- 172.21.15.119:0/2640308353 wait complete. 2023-11-08T19:28:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.902+0000 7fb00af7c640 1 Processor -- start 2023-11-08T19:28:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.902+0000 7fb00af7c640 1 -- start start 2023-11-08T19:28:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb00af7c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb004144580 0x7fb0040c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb00af7c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144960 0x7fb0041404e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb00af7c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 0x7fb0040c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb00af7c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004140ec0 con 0x7fb00415f1d0 2023-11-08T19:28:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb00af7c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0041411a0 con 0x7fb004144580 2023-11-08T19:28:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb00af7c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb004141480 con 0x7fb004144960 2023-11-08T19:28:42.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb009519640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 0x7fb0040c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb009519640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 0x7fb0040c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36370/0 (socket says 172.21.15.119:36370) 2023-11-08T19:28:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb009519640 1 -- 172.21.15.119:0/2192633674 learned_addr learned my addr 172.21.15.119:0/2192633674 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb008d18640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb004144580 0x7fb0040c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.903+0000 7fb003fff640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144960 0x7fb0041404e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb009519640 1 -- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144960 msgr2=0x7fb0041404e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb009519640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144960 0x7fb0041404e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb009519640 1 -- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb004144580 msgr2=0x7fb0040c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:42.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb009519640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb004144580 0x7fb0040c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:42.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb009519640 1 -- 172.21.15.119:0/2192633674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80094d0 con 0x7fb00415f1d0 2023-11-08T19:28:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb008d18640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb004144580 0x7fb0040c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb009519640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 0x7fb0040c7f70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c730 tx=0x7faffc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc003b70 con 0x7fb00415f1d0 2023-11-08T19:28:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc003cf0 con 0x7fb00415f1d0 2023-11-08T19:28:42.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb004141760 con 0x7fb00415f1d0 2023-11-08T19:28:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc003ea0 con 0x7fb00415f1d0 2023-11-08T19:28:42.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.904+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0040c7850 con 0x7fb00415f1d0 2023-11-08T19:28:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.905+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0040a8790 con 0x7fb00415f1d0 2023-11-08T19:28:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.906+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faffc005310 con 0x7fb00415f1d0 2023-11-08T19:28:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.907+0000 7fb0017fa640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fafe806ae70 0x7fafe806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:42.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.907+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7faffc089670 con 0x7fb00415f1d0 2023-11-08T19:28:42.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.907+0000 7fb008d18640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fafe806ae70 0x7fafe806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:42.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.908+0000 7fb008d18640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fafe806ae70 0x7fafe806d330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff803c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:42.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:42.912+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0040a8790 con 0x7fb00415f1d0 2023-11-08T19:28:43.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.105+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fb0040a8790 con 0x7fb00415f1d0 2023-11-08T19:28:43.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.106+0000 7fb0017fa640 1 -- 172.21.15.119:0/2192633674 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v87) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb0040a8790 con 0x7fb00415f1d0 2023-11-08T19:28:43.107 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:43.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fafe806ae70 msgr2=0x7fafe806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fafe806ae70 0x7fafe806d330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faff8036000 tx=0x7faff803c040 comp rx=0 tx=0).stop 2023-11-08T19:28:43.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 msgr2=0x7fb0040c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 0x7fb0040c7f70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7faffc00c730 tx=0x7faffc00cbf0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 shutdown_connections 2023-11-08T19:28:43.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fafe806ae70 0x7fafe806d330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb00415f1d0 0x7fb0040c7f70 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb004144960 0x7fb0041404e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 --2- 172.21.15.119:0/2192633674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb004144580 0x7fb0040c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 >> 172.21.15.119:0/2192633674 conn(0x7fb0040cb610 msgr2=0x7fb00406f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:43.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 shutdown_connections 2023-11-08T19:28:43.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.109+0000 7fb00af7c640 1 -- 172.21.15.119:0/2192633674 wait complete. 2023-11-08T19:28:43.130 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2023-11-08T19:28:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.345+0000 7f899f59e640 1 Processor -- start 2023-11-08T19:28:43.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.345+0000 7f899f59e640 1 -- start start 2023-11-08T19:28:43.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.346+0000 7f899f59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89a00adf20 0x7f89a015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.346+0000 7f899f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 0x7f89a015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.346+0000 7f899f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a00ca2e0 con 0x7f89a00cac30 2023-11-08T19:28:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.346+0000 7f899f59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a0071c70 con 0x7f89a00adf20 2023-11-08T19:28:43.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.346+0000 7f899f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a015b390 con 0x7f89a00ad9d0 2023-11-08T19:28:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 0x7f89a015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:43.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f8997fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f89a00adf20 0x7f89a015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 0x7f89a015ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:43950/0 (socket says 172.21.15.119:43950) 2023-11-08T19:28:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 -- 172.21.15.119:0/2659625635 learned_addr learned my addr 172.21.15.119:0/2659625635 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 -- 172.21.15.119:0/2659625635 >> v2:172.21.15.188:3300/0 conn(0x7f89a00adf20 msgr2=0x7f89a015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 --2- 172.21.15.119:0/2659625635 >> v2:172.21.15.188:3300/0 conn(0x7f89a00adf20 0x7f89a015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 -- 172.21.15.119:0/2659625635 >> v1:172.21.15.188:6789/0 conn(0x7f89a00cac30 legacy=0x7f89a015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.347+0000 7f899ed9d640 1 -- 172.21.15.119:0/2659625635 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a00ca5c0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.349+0000 7f899ed9d640 1 --2- 172.21.15.119:0/2659625635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 0x7f89a015ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8998009e50 tx=0x7f899802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac731340bc32855 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.349+0000 7f899dd9b640 1 -- 172.21.15.119:0/2659625635 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899802cb00 con 0x7f89a00ad9d0 2023-11-08T19:28:43.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.349+0000 7f899dd9b640 1 -- 172.21.15.119:0/2659625635 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899802cc60 con 0x7f89a00ad9d0 2023-11-08T19:28:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.349+0000 7f899dd9b640 1 -- 172.21.15.119:0/2659625635 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89980343f0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 -- 172.21.15.119:0/2659625635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 msgr2=0x7f89a015ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 --2- 172.21.15.119:0/2659625635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 0x7f89a015ae50 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8998009e50 tx=0x7f899802bf60 comp rx=0 tx=0).stop 2023-11-08T19:28:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 -- 172.21.15.119:0/2659625635 shutdown_connections 2023-11-08T19:28:43.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 --2- 172.21.15.119:0/2659625635 >> v2:172.21.15.188:3300/0 conn(0x7f89a00adf20 0x7f89a015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 --2- 172.21.15.119:0/2659625635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00ad9d0 0x7f89a015ae50 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 -- 172.21.15.119:0/2659625635 >> 172.21.15.119:0/2659625635 conn(0x7f89a00b0560 msgr2=0x7f89a00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 -- 172.21.15.119:0/2659625635 shutdown_connections 2023-11-08T19:28:43.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.350+0000 7f899f59e640 1 -- 172.21.15.119:0/2659625635 wait complete. 2023-11-08T19:28:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.352+0000 7f899f59e640 1 Processor -- start 2023-11-08T19:28:43.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.352+0000 7f899f59e640 1 -- start start 2023-11-08T19:28:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 0x7f89a0148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a00adf20 0x7f89a015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00cac30 0x7f89a00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01493f0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a015e1e0 con 0x7f89a00adf20 2023-11-08T19:28:43.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a01e47e0 con 0x7f89a00cac30 2023-11-08T19:28:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 0x7f89a0148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 0x7f89a0148eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36372/0 (socket says 172.21.15.119:36372) 2023-11-08T19:28:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.353+0000 7f899e59c640 1 -- 172.21.15.119:0/3356747982 learned_addr learned my addr 172.21.15.119:0/3356747982 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f8997fff640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a00adf20 0x7f89a015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:43.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899ed9d640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00cac30 0x7f89a00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899e59c640 1 -- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00cac30 msgr2=0x7f89a00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899e59c640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00cac30 0x7f89a00ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899e59c640 1 -- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a00adf20 msgr2=0x7f89a015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899e59c640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a00adf20 0x7f89a015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899e59c640 1 -- 172.21.15.119:0/3356747982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998009b30 con 0x7f89a00ad9d0 2023-11-08T19:28:43.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f8997fff640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a00adf20 0x7f89a015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:43.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899e59c640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 0x7f89a0148eb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f899000c3c0 tx=0x7f899000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:43.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900052d0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8990005450 con 0x7f89a00ad9d0 2023-11-08T19:28:43.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990007260 con 0x7f89a00ad9d0 2023-11-08T19:28:43.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a01e4ac0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.354+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a01e5080 con 0x7f89a00ad9d0 2023-11-08T19:28:43.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.356+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a004ece0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.357+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89900073c0 con 0x7f89a00ad9d0 2023-11-08T19:28:43.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.357+0000 7f8996ffd640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f898006ae70 0x7f898006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:43.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.358+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f8990088c40 con 0x7f89a00ad9d0 2023-11-08T19:28:43.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.358+0000 7f8997fff640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f898006ae70 0x7f898006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:43.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.358+0000 7f8997fff640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f898006ae70 0x7f898006d330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f89a00ad5d0 tx=0x7f8988008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:43.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.362+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8990053a60 con 0x7f89a00ad9d0 2023-11-08T19:28:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.622+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) v1 -- 0x7f89a015e510 con 0x7f89a00ad9d0 2023-11-08T19:28:43.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.933+0000 7f8996ffd640 1 -- 172.21.15.119:0/3356747982 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v239) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8990057080 con 0x7f89a00ad9d0 2023-11-08T19:28:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.936+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f898006ae70 msgr2=0x7f898006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.936+0000 7f899f59e640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f898006ae70 0x7f898006d330 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f89a00ad5d0 tx=0x7f8988008040 comp rx=0 tx=0).stop 2023-11-08T19:28:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.936+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 msgr2=0x7f89a0148eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:43.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.936+0000 7f899f59e640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 0x7f89a0148eb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f899000c3c0 tx=0x7f899000c880 comp rx=0 tx=0).stop 2023-11-08T19:28:43.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 shutdown_connections 2023-11-08T19:28:43.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f898006ae70 0x7f898006d330 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a00cac30 0x7f89a00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a00adf20 0x7f89a015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 --2- 172.21.15.119:0/3356747982 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f89a00ad9d0 0x7f89a0148eb0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 >> 172.21.15.119:0/3356747982 conn(0x7f89a00b0560 msgr2=0x7f89a006b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 shutdown_connections 2023-11-08T19:28:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:43.937+0000 7f899f59e640 1 -- 172.21.15.119:0/3356747982 wait complete. 2023-11-08T19:28:43.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T19:28:44.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.179+0000 7fa51cae0640 1 Processor -- start 2023-11-08T19:28:44.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.180+0000 7fa51cae0640 1 -- start start 2023-11-08T19:28:44.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.181+0000 7fa51cae0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa51815d4a0 0x7fa5180d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.181+0000 7fa51cae0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa5180cfa00 0x7fa5180cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.181+0000 7fa51cae0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa518161160 0x7fa518161570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.181+0000 7fa51cae0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5180d03d0 con 0x7fa5180cfa00 2023-11-08T19:28:44.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.181+0000 7fa51cae0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa51815df50 con 0x7fa51815d4a0 2023-11-08T19:28:44.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.181+0000 7fa51cae0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180d1e60 con 0x7fa518161160 2023-11-08T19:28:44.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa5180cfa00 0x7fa5180cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa5180cfa00 0x7fa5180cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36380/0 (socket says 172.21.15.119:36380) 2023-11-08T19:28:44.185 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 -- 172.21.15.119:0/739444497 learned_addr learned my addr 172.21.15.119:0/739444497 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa51659c640 1 --2- 172.21.15.119:0/739444497 >> v2:172.21.15.188:3300/0 conn(0x7fa51815d4a0 0x7fa5180d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa516d9d640 1 --2- 172.21.15.119:0/739444497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa518161160 0x7fa518161570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 -- 172.21.15.119:0/739444497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa518161160 msgr2=0x7fa518161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 --2- 172.21.15.119:0/739444497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa518161160 0x7fa518161570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 -- 172.21.15.119:0/739444497 >> v2:172.21.15.188:3300/0 conn(0x7fa51815d4a0 msgr2=0x7fa5180d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 --2- 172.21.15.119:0/739444497 >> v2:172.21.15.188:3300/0 conn(0x7fa51815d4a0 0x7fa5180d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa50ffff640 1 -- 172.21.15.119:0/739444497 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5180d2830 con 0x7fa5180cfa00 2023-11-08T19:28:44.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.182+0000 7fa51659c640 1 --2- 172.21.15.119:0/739444497 >> v2:172.21.15.188:3300/0 conn(0x7fa51815d4a0 0x7fa5180d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.183+0000 7fa50ffff640 1 --2- 172.21.15.119:0/739444497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180cfa00 0x7fa5180cfe90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa518072830 tx=0x7fa50002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd562df8d4728461 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.183+0000 7fa515d9b640 1 -- 172.21.15.119:0/739444497 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000317e0 con 0x7fa5180cfa00 2023-11-08T19:28:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.183+0000 7fa515d9b640 1 -- 172.21.15.119:0/739444497 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa500031940 con 0x7fa5180cfa00 2023-11-08T19:28:44.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.183+0000 7fa515d9b640 1 -- 172.21.15.119:0/739444497 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500031cd0 con 0x7fa5180cfa00 2023-11-08T19:28:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 -- 172.21.15.119:0/739444497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180cfa00 msgr2=0x7fa5180cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 --2- 172.21.15.119:0/739444497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180cfa00 0x7fa5180cfe90 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa518072830 tx=0x7fa50002b880 comp rx=0 tx=0).stop 2023-11-08T19:28:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 -- 172.21.15.119:0/739444497 shutdown_connections 2023-11-08T19:28:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 --2- 172.21.15.119:0/739444497 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa518161160 0x7fa518161570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 --2- 172.21.15.119:0/739444497 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5180cfa00 0x7fa5180cfe90 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 --2- 172.21.15.119:0/739444497 >> v2:172.21.15.188:3300/0 conn(0x7fa51815d4a0 0x7fa5180d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 -- 172.21.15.119:0/739444497 >> 172.21.15.119:0/739444497 conn(0x7fa51806a9f0 msgr2=0x7fa51806add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 -- 172.21.15.119:0/739444497 shutdown_connections 2023-11-08T19:28:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.184+0000 7fa51cae0640 1 -- 172.21.15.119:0/739444497 wait complete. 2023-11-08T19:28:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.186+0000 7fa51cae0640 1 Processor -- start 2023-11-08T19:28:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.186+0000 7fa51cae0640 1 -- start start 2023-11-08T19:28:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51cae0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5180cfa00 0x7fa51814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51cae0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa51815d4a0 0x7fa5180a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51cae0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 0x7fa5180ab130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51cae0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180c6590 con 0x7fa518161160 2023-11-08T19:28:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51cae0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa51814d9e0 con 0x7fa5180cfa00 2023-11-08T19:28:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51cae0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5180aa400 con 0x7fa51815d4a0 2023-11-08T19:28:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa516d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 0x7fa5180ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa516d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 0x7fa5180ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36388/0 (socket says 172.21.15.119:36388) 2023-11-08T19:28:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa516d9d640 1 -- 172.21.15.119:0/3485371699 learned_addr learned my addr 172.21.15.119:0/3485371699 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa51659c640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5180cfa00 0x7fa51814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.187+0000 7fa50ffff640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa51815d4a0 0x7fa5180a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa516d9d640 1 -- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa51815d4a0 msgr2=0x7fa5180a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa516d9d640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa51815d4a0 0x7fa5180a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa516d9d640 1 -- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5180cfa00 msgr2=0x7fa51814d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa516d9d640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5180cfa00 0x7fa51814d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa516d9d640 1 -- 172.21.15.119:0/3485371699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5000094d0 con 0x7fa518161160 2023-11-08T19:28:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa50ffff640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa51815d4a0 0x7fa5180a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa516d9d640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 0x7fa5180ab130 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa51000cc30 tx=0x7fa51000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51000ce40 con 0x7fa518161160 2023-11-08T19:28:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510003990 con 0x7fa518161160 2023-11-08T19:28:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5180aa6e0 con 0x7fa518161160 2023-11-08T19:28:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510003d20 con 0x7fa518161160 2023-11-08T19:28:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.188+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5180aaca0 con 0x7fa518161160 2023-11-08T19:28:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.189+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5180c6790 con 0x7fa518161160 2023-11-08T19:28:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.190+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa510003e80 con 0x7fa518161160 2023-11-08T19:28:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.191+0000 7fa50effd640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa4f806ae70 0x7fa4f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.191+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fa510089c80 con 0x7fa518161160 2023-11-08T19:28:44.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.191+0000 7fa51659c640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa4f806ae70 0x7fa4f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.192+0000 7fa51659c640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa4f806ae70 0x7fa4f806d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa508006b10 tx=0x7fa508008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.196+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5180c6790 con 0x7fa518161160 2023-11-08T19:28:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.390+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa518067f80 con 0x7fa518161160 2023-11-08T19:28:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.397+0000 7fa50effd640 1 -- 172.21.15.119:0/3485371699 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v19)=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa518067f80 con 0x7fa518161160 2023-11-08T19:28:44.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa4f806ae70 msgr2=0x7fa4f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa4f806ae70 0x7fa4f806d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa508006b10 tx=0x7fa508008040 comp rx=0 tx=0).stop 2023-11-08T19:28:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 msgr2=0x7fa5180ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 0x7fa5180ab130 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fa51000cc30 tx=0x7fa51000a5c0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 shutdown_connections 2023-11-08T19:28:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa4f806ae70 0x7fa4f806d330 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa518161160 0x7fa5180ab130 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa51815d4a0 0x7fa5180a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 --2- 172.21.15.119:0/3485371699 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5180cfa00 0x7fa51814d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 >> 172.21.15.119:0/3485371699 conn(0x7fa51806a9f0 msgr2=0x7fa51815e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 shutdown_connections 2023-11-08T19:28:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:44.400+0000 7fa51cae0640 1 -- 172.21.15.119:0/3485371699 wait complete. 2023-11-08T19:28:44.421 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:28:44.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:28:44.532 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:28:44.653 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:28:44.653 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:44.654 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:44.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:28:44.714 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:28:44.714 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:44.773 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:28:44.774 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:28:44.774 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:44.834 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:28:44.834 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:28:44.835 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:28:44.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:28:44.896 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T19:28:44.896 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:45.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.116+0000 7f694bfa9640 1 Processor -- start 2023-11-08T19:28:45.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.116+0000 7f694bfa9640 1 -- start start 2023-11-08T19:28:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f694bfa9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69441447a0 0x7f694415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f694bfa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69440cad30 0x7f694406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f694bfa9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440c3a50 con 0x7f69440cb170 2023-11-08T19:28:45.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f694bfa9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6944144b80 con 0x7f69441447a0 2023-11-08T19:28:45.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f694bfa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440c9150 con 0x7f69440cad30 2023-11-08T19:28:45.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f6949d45640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69441447a0 0x7f694415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.118+0000 7f6949544640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69440cad30 0x7f694406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.119+0000 7f6949d45640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69441447a0 0x7f694415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:40054/0 (socket says 172.21.15.119:40054) 2023-11-08T19:28:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.119+0000 7f6949d45640 1 -- 172.21.15.119:0/956386630 learned_addr learned my addr 172.21.15.119:0/956386630 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.119+0000 7f6949d45640 1 -- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69440cad30 msgr2=0x7f694406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.119+0000 7f6949d45640 1 --2- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69440cad30 0x7f694406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.119+0000 7f6949d45640 1 -- 172.21.15.119:0/956386630 >> v1:172.21.15.188:6789/0 conn(0x7f69440cb170 legacy=0x7f694406a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.119+0000 7f6949d45640 1 -- 172.21.15.119:0/956386630 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440c9410 con 0x7f69441447a0 2023-11-08T19:28:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.121+0000 7f6949d45640 1 --2- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69441447a0 0x7f694415f4c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6940009870 tx=0x7f694002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=874ad91768bc1d9d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.121+0000 7f6948d43640 1 -- 172.21.15.119:0/956386630 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940030840 con 0x7f69441447a0 2023-11-08T19:28:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.122+0000 7f6948d43640 1 -- 172.21.15.119:0/956386630 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69400309a0 con 0x7f69441447a0 2023-11-08T19:28:45.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.122+0000 7f6948d43640 1 -- 172.21.15.119:0/956386630 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940030d30 con 0x7f69441447a0 2023-11-08T19:28:45.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.122+0000 7f694bfa9640 1 -- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69441447a0 msgr2=0x7f694415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.122+0000 7f694bfa9640 1 --2- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69441447a0 0x7f694415f4c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6940009870 tx=0x7f694002b980 comp rx=0 tx=0).stop 2023-11-08T19:28:45.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.122+0000 7f694bfa9640 1 -- 172.21.15.119:0/956386630 shutdown_connections 2023-11-08T19:28:45.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.123+0000 7f694bfa9640 1 --2- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69440cad30 0x7f694406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.123+0000 7f694bfa9640 1 --2- 172.21.15.119:0/956386630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69441447a0 0x7f694415f4c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.123+0000 7f694bfa9640 1 -- 172.21.15.119:0/956386630 >> 172.21.15.119:0/956386630 conn(0x7f69440cb610 msgr2=0x7f69440cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:45.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.123+0000 7f694bfa9640 1 -- 172.21.15.119:0/956386630 shutdown_connections 2023-11-08T19:28:45.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.123+0000 7f694bfa9640 1 -- 172.21.15.119:0/956386630 wait complete. 2023-11-08T19:28:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.125+0000 7f694bfa9640 1 Processor -- start 2023-11-08T19:28:45.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.125+0000 7f694bfa9640 1 -- start start 2023-11-08T19:28:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f694bfa9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 0x7f694415ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f694bfa9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69440cb170 0x7f69440b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f694bfa9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69441447a0 0x7f69440b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f694bfa9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69441f6df0 con 0x7f69440cad30 2023-11-08T19:28:45.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f694bfa9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694413f9b0 con 0x7f69440cb170 2023-11-08T19:28:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f694bfa9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69441f7210 con 0x7f69441447a0 2023-11-08T19:28:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.126+0000 7f6949d45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 0x7f694415ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 0x7f694415ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36394/0 (socket says 172.21.15.119:36394) 2023-11-08T19:28:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 -- 172.21.15.119:0/794766957 learned_addr learned my addr 172.21.15.119:0/794766957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:45.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949544640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69440cb170 0x7f69440b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f694a546640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69441447a0 0x7f69440b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 -- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69441447a0 msgr2=0x7f69440b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69441447a0 0x7f69440b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 -- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69440cb170 msgr2=0x7f69440b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69440cb170 0x7f69440b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949d45640 1 -- 172.21.15.119:0/794766957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69400094d0 con 0x7f69440cad30 2023-11-08T19:28:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.127+0000 7f6949544640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69440cb170 0x7f69440b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:45.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.128+0000 7f6949d45640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 0x7f694415ce90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6940000c00 tx=0x7f6940004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.128+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940039a90 con 0x7f69440cad30 2023-11-08T19:28:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.128+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6940039bf0 con 0x7f69440cad30 2023-11-08T19:28:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.128+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6940043410 con 0x7f69440cad30 2023-11-08T19:28:45.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.128+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69441f6450 con 0x7f69440cad30 2023-11-08T19:28:45.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.128+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69441f6a10 con 0x7f69440cad30 2023-11-08T19:28:45.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.129+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6944074430 con 0x7f69440cad30 2023-11-08T19:28:45.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.130+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f694002f390 con 0x7f69440cad30 2023-11-08T19:28:45.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.131+0000 7f693a7fc640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f692806ae70 0x7f692806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.131+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7f69400ab980 con 0x7f69440cad30 2023-11-08T19:28:45.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.132+0000 7f6949544640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f692806ae70 0x7f692806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.132+0000 7f6949544640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f692806ae70 0x7f692806d330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f69441f6280 tx=0x7f6934008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:45.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.136+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6940076790 con 0x7f69440cad30 2023-11-08T19:28:45.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.362+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f69441f7060 con 0x7f69440cad30 2023-11-08T19:28:45.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.363+0000 7f693a7fc640 1 -- 172.21.15.119:0/794766957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f6940048e90 con 0x7f69440cad30 2023-11-08T19:28:45.365 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:45.365 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3483870145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3483870145},{"type":"v1","addr":"172.21.15.188:6833","nonce":3483870145}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":5362,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/632097327","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":632097327},{"type":"v1","addr":"172.21.15.188:6835","nonce":632097327}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":39},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/2895920019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":2895920019},{"type":"v1","addr":"172.21.15.119:6836","nonce":2895920019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6972,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2962718914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2962718914},{"type":"v1","addr":"172.21.15.119:6837","nonce":2962718914}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48}],"filesystems":[]} 2023-11-08T19:28:45.366 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 48 2023-11-08T19:28:45.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f692806ae70 msgr2=0x7f692806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f692806ae70 0x7f692806d330 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f69441f6280 tx=0x7f6934008040 comp rx=0 tx=0).stop 2023-11-08T19:28:45.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 msgr2=0x7f694415ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 0x7f694415ce90 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6940000c00 tx=0x7f6940004730 comp rx=0 tx=0).stop 2023-11-08T19:28:45.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 shutdown_connections 2023-11-08T19:28:45.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f692806ae70 0x7f692806d330 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69441447a0 0x7f69440b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69440cb170 0x7f69440b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 --2- 172.21.15.119:0/794766957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f69440cad30 0x7f694415ce90 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 >> 172.21.15.119:0/794766957 conn(0x7f69440cb610 msgr2=0x7f6944142c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 shutdown_connections 2023-11-08T19:28:45.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.366+0000 7f694bfa9640 1 -- 172.21.15.119:0/794766957 wait complete. 2023-11-08T19:28:45.390 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:28:45.390 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:28:45.390 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:28:45.391 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:45.391 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T19:28:45.391 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T19:28:45.392 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:28:45.392 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:45.392 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:28:45.393 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:28:45.393 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:28:45.393 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:45.394 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:28:45.394 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:28:45.395 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:28:45.395 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:28:45.406 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:28:45.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:28:45.408 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:28:45.409 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T19:28:45.415 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:28:45.415 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:28:45.416 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:28:45.417 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:28:45.457 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:28:45.457 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:45.461 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:28:45.461 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:45.465 INFO:tasks.ceph.mds.c:Started 2023-11-08T19:28:45.465 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:45.468 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:28:45.468 DEBUG:teuthology.parallel:result is None 2023-11-08T19:28:45.469 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T19:28:45.617 INFO:tasks.ceph.mds.c.smithi119.stdout:starting mds.c at 2023-11-08T19:28:45.618 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:28:45.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-08T19:28:45.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-08T19:28:45.619 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-08T19:28:45.622 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:28:45.623 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:28:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.656+0000 7fb78064b640 1 Processor -- start 2023-11-08T19:28:45.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.657+0000 7fb78064b640 1 -- start start 2023-11-08T19:28:45.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb78064b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb778159d90 0x7fb77815a170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb78064b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7780c9360 con 0x7fb77815a6b0 2023-11-08T19:28:45.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb78064b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7780ca260 con 0x7fb7780c9e30 2023-11-08T19:28:45.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb78064b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb77806d3c0 con 0x7fb778159d90 2023-11-08T19:28:45.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb77e3e7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb778159d90 0x7fb77815a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb77dbe6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb77815a6b0 0x7fb7780c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45818/0 (socket says 172.21.15.119:45818) 2023-11-08T19:28:45.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.658+0000 7fb77dbe6640 1 -- 172.21.15.119:0/974032121 learned_addr learned my addr 172.21.15.119:0/974032121 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:45.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.659+0000 7fb77e3e7640 1 -- 172.21.15.119:0/974032121 >> v1:172.21.15.188:6789/0 conn(0x7fb7780c9e30 legacy=0x7fb7781630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.659+0000 7fb77e3e7640 1 -- 172.21.15.119:0/974032121 >> v1:172.21.15.119:6789/0 conn(0x7fb77815a6b0 legacy=0x7fb7780c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.659+0000 7fb77e3e7640 1 -- 172.21.15.119:0/974032121 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780ca490 con 0x7fb778159d90 2023-11-08T19:28:45.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.659+0000 7fb77d3e5640 1 -- 172.21.15.119:0/974032121 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203622595 0 0) 0x7fb7780c9360 con 0x7fb77815a6b0 2023-11-08T19:28:45.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.660+0000 7fb77e3e7640 1 --2- 172.21.15.119:0/974032121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb778159d90 0x7fb77815a170 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c6a0ba87aba25c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:45.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.660+0000 7fb77d3e5640 1 -- 172.21.15.119:0/974032121 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77402f810 con 0x7fb778159d90 2023-11-08T19:28:45.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.660+0000 7fb77d3e5640 1 -- 172.21.15.119:0/974032121 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77402f970 con 0x7fb778159d90 2023-11-08T19:28:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.660+0000 7fb77d3e5640 1 -- 172.21.15.119:0/974032121 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77402fd00 con 0x7fb778159d90 2023-11-08T19:28:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 -- 172.21.15.119:0/974032121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb778159d90 msgr2=0x7fb77815a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 --2- 172.21.15.119:0/974032121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb778159d90 0x7fb77815a170 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fb7740097a0 tx=0x7fb77402b9f0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 -- 172.21.15.119:0/974032121 shutdown_connections 2023-11-08T19:28:45.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 --2- 172.21.15.119:0/974032121 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb778159d90 0x7fb77815a170 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 -- 172.21.15.119:0/974032121 >> 172.21.15.119:0/974032121 conn(0x7fb7780cb610 msgr2=0x7fb7780cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 -- 172.21.15.119:0/974032121 shutdown_connections 2023-11-08T19:28:45.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.661+0000 7fb78064b640 1 -- 172.21.15.119:0/974032121 wait complete. 2023-11-08T19:28:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.663+0000 7fb78064b640 1 Processor -- start 2023-11-08T19:28:45.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.663+0000 7fb78064b640 1 -- start start 2023-11-08T19:28:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb78064b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 0x7fb77813a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb78064b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb778159d90 0x7fb778150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb78064b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb77815a6b0 0x7fb778152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb78064b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb778152100 con 0x7fb7780c9e30 2023-11-08T19:28:45.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb78064b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77813a980 con 0x7fb778159d90 2023-11-08T19:28:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb78064b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb778151480 con 0x7fb77815a6b0 2023-11-08T19:28:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 0x7fb77813a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 0x7fb77813a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36460/0 (socket says 172.21.15.119:36460) 2023-11-08T19:28:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 -- 172.21.15.119:0/2898460028 learned_addr learned my addr 172.21.15.119:0/2898460028 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:45.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77dbe6640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb778159d90 0x7fb778150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 -- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb77815a6b0 msgr2=0x7fb778152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb77815a6b0 0x7fb778152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 -- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb778159d90 msgr2=0x7fb778150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.664+0000 7fb77e3e7640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb778159d90 0x7fb778150aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:45.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb77e3e7640 1 -- 172.21.15.119:0/2898460028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb774009400 con 0x7fb7780c9e30 2023-11-08T19:28:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb77dbe6640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb778159d90 0x7fb778150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb77e3e7640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 0x7fb77813a440 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb7740098c0 tx=0x7fb7740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774046070 con 0x7fb7780c9e30 2023-11-08T19:28:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774037850 con 0x7fb7780c9e30 2023-11-08T19:28:45.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77403b040 con 0x7fb7780c9e30 2023-11-08T19:28:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb778151760 con 0x7fb7780c9e30 2023-11-08T19:28:45.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.665+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb778151d20 con 0x7fb7780c9e30 2023-11-08T19:28:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.667+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778152240 con 0x7fb7780c9e30 2023-11-08T19:28:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.668+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb774037c00 con 0x7fb7780c9e30 2023-11-08T19:28:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.668+0000 7fb76effd640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb75c06ae70 0x7fb75c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:45.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.669+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6354+0+0 (secure 0 0 0) 0x7fb7740ac170 con 0x7fb7780c9e30 2023-11-08T19:28:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.669+0000 7fb77dbe6640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb75c06ae70 0x7fb75c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.669+0000 7fb77dbe6640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb75c06ae70 0x7fb75c06d330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb77813ac60 tx=0x7fb768009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:45.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.673+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb774076f90 con 0x7fb7780c9e30 2023-11-08T19:28:45.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:45.876+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb778074430 con 0x7fb7780c9e30 2023-11-08T19:28:46.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.975+0000 7fb76effd640 1 -- 172.21.15.119:0/2898460028 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb77407a5b0 con 0x7fb7780c9e30 2023-11-08T19:28:46.977 INFO:teuthology.orchestra.run.smithi119.stderr: removed all blocklist entries 2023-11-08T19:28:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb75c06ae70 msgr2=0x7fb75c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:46.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb75c06ae70 0x7fb75c06d330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb77813ac60 tx=0x7fb768009040 comp rx=0 tx=0).stop 2023-11-08T19:28:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 msgr2=0x7fb77813a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 0x7fb77813a440 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fb7740098c0 tx=0x7fb7740046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 shutdown_connections 2023-11-08T19:28:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb75c06ae70 0x7fb75c06d330 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:46.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb77815a6b0 0x7fb778152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:46.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb778159d90 0x7fb778150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:46.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 --2- 172.21.15.119:0/2898460028 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb7780c9e30 0x7fb77813a440 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:46.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.980+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 >> 172.21.15.119:0/2898460028 conn(0x7fb7780cb610 msgr2=0x7fb7780c3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:46.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.981+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 shutdown_connections 2023-11-08T19:28:46.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:46.981+0000 7fb78064b640 1 -- 172.21.15.119:0/2898460028 wait complete. 2023-11-08T19:28:47.000 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T19:28:47.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.208+0000 7f567d185640 1 Processor -- start 2023-11-08T19:28:47.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.208+0000 7f567d185640 1 -- start start 2023-11-08T19:28:47.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.209+0000 7f567d185640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f567815aea0 0x7f56780c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.209+0000 7f567d185640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56780b18c0 0x7f56780b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.209+0000 7f567d185640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f567806be20 con 0x7f567806b650 2023-11-08T19:28:47.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.209+0000 7f567d185640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f567806c100 con 0x7f567815aea0 2023-11-08T19:28:47.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.209+0000 7f567d185640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567806c3e0 con 0x7f56780b18c0 2023-11-08T19:28:47.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f567815aea0 0x7f56780c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f567815aea0 0x7f56780c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36462/0 (socket says 172.21.15.119:36462) 2023-11-08T19:28:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 -- 172.21.15.119:0/2942421333 learned_addr learned my addr 172.21.15.119:0/2942421333 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f567c984640 1 --2- 172.21.15.119:0/2942421333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56780b18c0 0x7f56780b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 -- 172.21.15.119:0/2942421333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56780b18c0 msgr2=0x7f56780b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:47.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 --2- 172.21.15.119:0/2942421333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56780b18c0 0x7f56780b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 -- 172.21.15.119:0/2942421333 >> v1:172.21.15.188:6789/0 conn(0x7f567806b650 legacy=0x7f567806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:47.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.210+0000 7f5677fff640 1 -- 172.21.15.119:0/2942421333 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f567806c6c0 con 0x7f567815aea0 2023-11-08T19:28:47.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.211+0000 7f5677fff640 1 --2- 172.21.15.119:0/2942421333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567815aea0 0x7f56780c9c90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5668009870 tx=0x7f566802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a620ac3dbd97f9df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.212+0000 7f56777fe640 1 -- 172.21.15.119:0/2942421333 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668031640 con 0x7f567815aea0 2023-11-08T19:28:47.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.212+0000 7f56777fe640 1 -- 172.21.15.119:0/2942421333 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56680317a0 con 0x7f567815aea0 2023-11-08T19:28:47.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.212+0000 7f56777fe640 1 -- 172.21.15.119:0/2942421333 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668031b30 con 0x7f567815aea0 2023-11-08T19:28:47.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.212+0000 7f567d185640 1 -- 172.21.15.119:0/2942421333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567815aea0 msgr2=0x7f56780c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.212+0000 7f567d185640 1 --2- 172.21.15.119:0/2942421333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567815aea0 0x7f56780c9c90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f5668009870 tx=0x7f566802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.213+0000 7f567d185640 1 -- 172.21.15.119:0/2942421333 shutdown_connections 2023-11-08T19:28:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.213+0000 7f567d185640 1 --2- 172.21.15.119:0/2942421333 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56780b18c0 0x7f56780b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.213+0000 7f567d185640 1 --2- 172.21.15.119:0/2942421333 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567815aea0 0x7f56780c9c90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.213+0000 7f567d185640 1 -- 172.21.15.119:0/2942421333 >> 172.21.15.119:0/2942421333 conn(0x7f56780b0560 msgr2=0x7f56780b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:47.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.213+0000 7f567d185640 1 -- 172.21.15.119:0/2942421333 shutdown_connections 2023-11-08T19:28:47.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.213+0000 7f567d185640 1 -- 172.21.15.119:0/2942421333 wait complete. 2023-11-08T19:28:47.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.214+0000 7f567d185640 1 Processor -- start 2023-11-08T19:28:47.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 -- start start 2023-11-08T19:28:47.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 0x7f56781555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56780b18c0 0x7f56780c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f567815aea0 0x7f567815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5678155e70 con 0x7f567806b650 2023-11-08T19:28:47.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56780c64e0 con 0x7f56780b18c0 2023-11-08T19:28:47.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.215+0000 7f567d185640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56780c67c0 con 0x7f567815aea0 2023-11-08T19:28:47.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 0x7f56781555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 0x7f56781555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36478/0 (socket says 172.21.15.119:36478) 2023-11-08T19:28:47.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 -- 172.21.15.119:0/2336866341 learned_addr learned my addr 172.21.15.119:0/2336866341 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:47.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f567c984640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f567815aea0 0x7f567815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 -- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f567815aea0 msgr2=0x7f567815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f566ffff640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56780b18c0 0x7f56780c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f567815aea0 0x7f567815bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 -- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56780b18c0 msgr2=0x7f56780c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56780b18c0 0x7f56780c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.216+0000 7f5677fff640 1 -- 172.21.15.119:0/2336866341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56680094d0 con 0x7f567806b650 2023-11-08T19:28:47.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.217+0000 7f5677fff640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 0x7f56781555f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5668009870 tx=0x7f56680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.217+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f566803ac60 con 0x7f567806b650 2023-11-08T19:28:47.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.217+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f566803ade0 con 0x7f567806b650 2023-11-08T19:28:47.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.217+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5668044630 con 0x7f567806b650 2023-11-08T19:28:47.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.217+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56781560c0 con 0x7f567806b650 2023-11-08T19:28:47.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.217+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5678155b30 con 0x7f567806b650 2023-11-08T19:28:47.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.218+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f567806baa0 con 0x7f567806b650 2023-11-08T19:28:47.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.219+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f566802f050 con 0x7f567806b650 2023-11-08T19:28:47.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.220+0000 7f56757fa640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f565806ae70 0x7f565806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.221+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f56680aba60 con 0x7f567806b650 2023-11-08T19:28:47.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.221+0000 7f566ffff640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f565806ae70 0x7f565806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.221+0000 7f566ffff640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f565806ae70 0x7f565806d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f56780b3040 tx=0x7f5660008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.225+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f567806baa0 con 0x7f567806b650 2023-11-08T19:28:47.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.482+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f56781565e0 con 0x7f567806b650 2023-11-08T19:28:47.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.484+0000 7f56757fa640 1 -- 172.21.15.119:0/2336866341 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f56781565e0 con 0x7f567806b650 2023-11-08T19:28:47.485 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:47.486 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:28:47.486 INFO:teuthology.orchestra.run.smithi119.stdout: "auth_dump": [ 2023-11-08T19:28:47.486 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.486 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.a", 2023-11-08T19:28:47.486 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlRbIyIhAAE65zhyOZt7sW/uEyyLoWrg==", 2023-11-08T19:28:47.487 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.487 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.487 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:47.487 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:47.488 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.488 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.488 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.488 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.488 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.b", 2023-11-08T19:28:47.488 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etluue/CBAAoAr3fBCMWZ6NySb8lBdVDQ==", 2023-11-08T19:28:47.489 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.489 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.489 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:47.489 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:47.489 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.490 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.490 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.490 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.490 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.c", 2023-11-08T19:28:47.491 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4Etl8BXtMhAA+2X0aW+axkqKudgdPjHWTw==", 2023-11-08T19:28:47.491 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.491 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.491 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:47.491 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:47.492 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.492 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.492 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.492 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.492 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mds.d", 2023-11-08T19:28:47.493 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4Etl2JS2GRAAT8aVIlJbVMwWgiy2nQ2vHQ==", 2023-11-08T19:28:47.493 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.493 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.493 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:47.493 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:47.494 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.494 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.494 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.494 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.494 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.0", 2023-11-08T19:28:47.495 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBO4EtlRMmIDBAAwOT7vjOwg7cf7ez0Cai3iA==", 2023-11-08T19:28:47.495 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.495 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.495 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.495 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.1", 2023-11-08T19:28:47.496 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBR4EtlD4icFhAAK87f2Ckq45JtWy/WLcc+2A==", 2023-11-08T19:28:47.497 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.497 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.497 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.498 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.498 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.498 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.498 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.2", 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBU4EtlpemQIBAAIkUZrLDuO2mSJAKUJ6R/oA==", 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.499 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.500 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.3", 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBX4Etlw8EMOhAAqtcjwuKe81fPoQaCUld/yQ==", 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.501 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.4", 2023-11-08T19:28:47.502 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBf4EtlH5OSDRAAdzqbb+hBruu/CDod2WcZFQ==", 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.503 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.5", 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBi4EtlhMVLGBAAFMl7pc8Vn07GJcm7ok60pw==", 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.504 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.505 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.505 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.505 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.505 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.505 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.506 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.6", 2023-11-08T19:28:47.506 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBl4EtlnrEJIhAAzAEQ/Y/WdhqpwnNWOvzK+g==", 2023-11-08T19:28:47.506 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.506 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.506 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.507 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "osd.7", 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBo4EtlUusULBAAZ8203E39eGhU2YEg/vvuyQ==", 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow profile osd", 2023-11-08T19:28:47.508 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile osd", 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.0", 2023-11-08T19:28:47.509 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw==", 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow r", 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T19:28:47.510 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.1", 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etld5NwAhAArFffuZccqUQfTwR4sNDJXQ==", 2023-11-08T19:28:47.511 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:28:47.512 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.2", 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBJ4EtlIZw1MxAAitvGXmPChQ5U0c6n4IWy+Q==", 2023-11-08T19:28:47.513 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:28:47.514 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.515 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.515 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.515 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.3", 2023-11-08T19:28:47.515 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBK4Etljm5oDBAAtElxFxvCDVIrQM48NpHucQ==", 2023-11-08T19:28:47.515 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.515 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow", 2023-11-08T19:28:47.516 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow r", 2023-11-08T19:28:47.516 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow rw", 2023-11-08T19:28:47.516 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow rwx" 2023-11-08T19:28:47.516 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.516 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.517 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.517 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "client.admin", 2023-11-08T19:28:47.517 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBH4Etl5+oDFBAAMV8QbfI7L35tUuiNz9bvdw==", 2023-11-08T19:28:47.517 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.517 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:28:47.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mgr": "allow *", 2023-11-08T19:28:47.518 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow *", 2023-11-08T19:28:47.518 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.518 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.518 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.519 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.519 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.x", 2023-11-08T19:28:47.519 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlezsPFhAAOIsAed+u9ltzvIOF1Uy3PA==", 2023-11-08T19:28:47.519 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.519 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:28:47.519 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:28:47.520 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.520 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.520 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:47.520 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:47.520 INFO:teuthology.orchestra.run.smithi119.stdout: "entity": "mgr.y", 2023-11-08T19:28:47.521 INFO:teuthology.orchestra.run.smithi119.stdout: "key": "AQBI4EtlpYPmBxAAZ22sIrvr37HZYTARag3haQ==", 2023-11-08T19:28:47.521 INFO:teuthology.orchestra.run.smithi119.stdout: "caps": { 2023-11-08T19:28:47.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mds": "allow *", 2023-11-08T19:28:47.521 INFO:teuthology.orchestra.run.smithi119.stdout: "mon": "allow profile mgr", 2023-11-08T19:28:47.521 INFO:teuthology.orchestra.run.smithi119.stdout: "osd": "allow *" 2023-11-08T19:28:47.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.522 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:47.522 INFO:teuthology.orchestra.run.smithi119.stdout: ] 2023-11-08T19:28:47.522 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:28:47.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f565806ae70 msgr2=0x7f565806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f565806ae70 0x7f565806d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f56780b3040 tx=0x7f5660008040 comp rx=0 tx=0).stop 2023-11-08T19:28:47.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 msgr2=0x7f56781555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 0x7f56781555f0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f5668009870 tx=0x7f56680046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 shutdown_connections 2023-11-08T19:28:47.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f565806ae70 0x7f565806d330 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f567815aea0 0x7f567815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56780b18c0 0x7f56780c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 --2- 172.21.15.119:0/2336866341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f567806b650 0x7f56781555f0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.486+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 >> 172.21.15.119:0/2336866341 conn(0x7f56780b0560 msgr2=0x7f567806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:47.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.487+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 shutdown_connections 2023-11-08T19:28:47.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.487+0000 7f567d185640 1 -- 172.21.15.119:0/2336866341 wait complete. 2023-11-08T19:28:47.526 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:47.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.744+0000 7fc8727d0640 1 Processor -- start 2023-11-08T19:28:47.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.744+0000 7fc8727d0640 1 -- start start 2023-11-08T19:28:47.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc8727d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15d9f0 0x7fc86c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc8727d0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc86c06cb40 0x7fc86c06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc8727d0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc86c0c4960 con 0x7fc86c142f80 2023-11-08T19:28:47.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc8727d0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86c143a40 con 0x7fc86c06cb40 2023-11-08T19:28:47.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc8727d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0ca3e0 con 0x7fc86c15d9f0 2023-11-08T19:28:47.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc86bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc86c142f80 0x7fc86c0c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45834/0 (socket says 172.21.15.119:45834) 2023-11-08T19:28:47.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc86bfff640 1 -- 172.21.15.119:0/2963434764 learned_addr learned my addr 172.21.15.119:0/2963434764 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:47.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc870d6d640 1 --2- 172.21.15.119:0/2963434764 >> v2:172.21.15.119:3300/0 conn(0x7fc86c06cb40 0x7fc86c06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc863fff640 1 --2- 172.21.15.119:0/2963434764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15d9f0 0x7fc86c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc86b7fe640 1 -- 172.21.15.119:0/2963434764 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 474629145 0 0) 0x7fc86c0c4960 con 0x7fc86c142f80 2023-11-08T19:28:47.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.746+0000 7fc86b7fe640 1 -- 172.21.15.119:0/2963434764 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc858003520 con 0x7fc86c142f80 2023-11-08T19:28:47.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.747+0000 7fc870d6d640 1 -- 172.21.15.119:0/2963434764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15d9f0 msgr2=0x7fc86c0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:47.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.747+0000 7fc870d6d640 1 --2- 172.21.15.119:0/2963434764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15d9f0 0x7fc86c0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.747+0000 7fc870d6d640 1 -- 172.21.15.119:0/2963434764 >> v1:172.21.15.119:6789/0 conn(0x7fc86c142f80 legacy=0x7fc86c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.747+0000 7fc870d6d640 1 -- 172.21.15.119:0/2963434764 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc86c15df60 con 0x7fc86c06cb40 2023-11-08T19:28:47.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.748+0000 7fc870d6d640 1 --2- 172.21.15.119:0/2963434764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c06cb40 0x7fc86c06cfd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc864009d90 tx=0x7fc86402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8f9646e56cb2d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.748+0000 7fc86b7fe640 1 -- 172.21.15.119:0/2963434764 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86402cbc0 con 0x7fc86c06cb40 2023-11-08T19:28:47.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.748+0000 7fc86b7fe640 1 -- 172.21.15.119:0/2963434764 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86402cd20 con 0x7fc86c06cb40 2023-11-08T19:28:47.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.748+0000 7fc86b7fe640 1 -- 172.21.15.119:0/2963434764 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864034660 con 0x7fc86c06cb40 2023-11-08T19:28:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 -- 172.21.15.119:0/2963434764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c06cb40 msgr2=0x7fc86c06cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 --2- 172.21.15.119:0/2963434764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c06cb40 0x7fc86c06cfd0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fc864009d90 tx=0x7fc86402be10 comp rx=0 tx=0).stop 2023-11-08T19:28:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 -- 172.21.15.119:0/2963434764 shutdown_connections 2023-11-08T19:28:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 --2- 172.21.15.119:0/2963434764 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c06cb40 0x7fc86c06cfd0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 --2- 172.21.15.119:0/2963434764 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c15d9f0 0x7fc86c0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 -- 172.21.15.119:0/2963434764 >> 172.21.15.119:0/2963434764 conn(0x7fc86c0cb610 msgr2=0x7fc86c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 -- 172.21.15.119:0/2963434764 shutdown_connections 2023-11-08T19:28:47.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.749+0000 7fc8727d0640 1 -- 172.21.15.119:0/2963434764 wait complete. 2023-11-08T19:28:47.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.751+0000 7fc8727d0640 1 Processor -- start 2023-11-08T19:28:47.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.751+0000 7fc8727d0640 1 -- start start 2023-11-08T19:28:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc8727d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c06cb40 0x7fc86c13d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc8727d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c142f80 0x7fc86c13c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc8727d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 0x7fc86c13ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc8727d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0b1e60 con 0x7fc86c15d9f0 2023-11-08T19:28:47.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc8727d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0c8530 con 0x7fc86c06cb40 2023-11-08T19:28:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc8727d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc86c0b2100 con 0x7fc86c142f80 2023-11-08T19:28:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc870d6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 0x7fc86c13ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc870d6d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 0x7fc86c13ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36498/0 (socket says 172.21.15.119:36498) 2023-11-08T19:28:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc870d6d640 1 -- 172.21.15.119:0/3460325614 learned_addr learned my addr 172.21.15.119:0/3460325614 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:47.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc86bfff640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c06cb40 0x7fc86c13d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc863fff640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c142f80 0x7fc86c13c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.752+0000 7fc870d6d640 1 -- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c142f80 msgr2=0x7fc86c13c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc870d6d640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c142f80 0x7fc86c13c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc870d6d640 1 -- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c06cb40 msgr2=0x7fc86c13d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc870d6d640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c06cb40 0x7fc86c13d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc870d6d640 1 -- 172.21.15.119:0/3460325614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864009a70 con 0x7fc86c15d9f0 2023-11-08T19:28:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc86bfff640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c06cb40 0x7fc86c13d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc870d6d640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 0x7fc86c13ea90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc864009d90 tx=0x7fc864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc864040650 con 0x7fc86c15d9f0 2023-11-08T19:28:47.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86402ce90 con 0x7fc86c15d9f0 2023-11-08T19:28:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8640409f0 con 0x7fc86c15d9f0 2023-11-08T19:28:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86c0b23e0 con 0x7fc86c15d9f0 2023-11-08T19:28:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.753+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86c13cb40 con 0x7fc86c15d9f0 2023-11-08T19:28:47.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.755+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc86c074430 con 0x7fc86c15d9f0 2023-11-08T19:28:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.756+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc864040b50 con 0x7fc86c15d9f0 2023-11-08T19:28:47.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.757+0000 7fc8697fa640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc84c06ae70 0x7fc84c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.757+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fc8640ac580 con 0x7fc86c15d9f0 2023-11-08T19:28:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.757+0000 7fc86bfff640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc84c06ae70 0x7fc84c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.758+0000 7fc86bfff640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc84c06ae70 0x7fc84c06d330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0011e0 tx=0x7fc85c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:47.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.762+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8640772d0 con 0x7fc86c15d9f0 2023-11-08T19:28:47.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.955+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc86c0c9f80 con 0x7fc86c15d9f0 2023-11-08T19:28:47.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.957+0000 7fc8697fa640 1 -- 172.21.15.119:0/3460325614 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+18347 (secure 0 0 0) 0x7fc864009450 con 0x7fc86c15d9f0 2023-11-08T19:28:47.959 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:47.960 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:46.972709+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:47.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc84c06ae70 msgr2=0x7fc84c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc84c06ae70 0x7fc84c06d330 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0011e0 tx=0x7fc85c009040 comp rx=0 tx=0).stop 2023-11-08T19:28:47.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 msgr2=0x7fc86c13ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:47.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 0x7fc86c13ea90 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fc864009d90 tx=0x7fc864004a40 comp rx=0 tx=0).stop 2023-11-08T19:28:47.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 shutdown_connections 2023-11-08T19:28:47.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc84c06ae70 0x7fc84c06d330 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc86c15d9f0 0x7fc86c13ea90 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc86c142f80 0x7fc86c13c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 --2- 172.21.15.119:0/3460325614 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc86c06cb40 0x7fc86c13d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:47.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.960+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 >> 172.21.15.119:0/3460325614 conn(0x7fc86c0cb610 msgr2=0x7fc86c0c5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:47.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.961+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 shutdown_connections 2023-11-08T19:28:47.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:47.961+0000 7fc8727d0640 1 -- 172.21.15.119:0/3460325614 wait complete. 2023-11-08T19:28:47.982 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:28:47.983 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:28:48.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.200+0000 7f80fab97640 1 Processor -- start 2023-11-08T19:28:48.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.200+0000 7f80fab97640 1 -- start start 2023-11-08T19:28:48.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.201+0000 7f80fab97640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80f4143600 0x7f80f40c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80fab97640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f415e440 con 0x7f80f415d9f0 2023-11-08T19:28:48.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80fab97640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f415e720 con 0x7f80f406d250 2023-11-08T19:28:48.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80fab97640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80f40c4f50 con 0x7f80f4143600 2023-11-08T19:28:48.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80f8933640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80f4143600 0x7f80f40c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80f8933640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f80f4143600 0x7f80f40c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36504/0 (socket says 172.21.15.119:36504) 2023-11-08T19:28:48.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80f8933640 1 -- 172.21.15.119:0/4072318576 learned_addr learned my addr 172.21.15.119:0/4072318576 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:48.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80f8933640 1 -- 172.21.15.119:0/4072318576 >> v1:172.21.15.188:6789/0 conn(0x7f80f406d250 legacy=0x7f80f40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:48.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80f8933640 1 -- 172.21.15.119:0/4072318576 >> v1:172.21.15.119:6789/0 conn(0x7f80f415d9f0 legacy=0x7f80f406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.202+0000 7f80f8933640 1 -- 172.21.15.119:0/4072318576 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40c4b70 con 0x7f80f4143600 2023-11-08T19:28:48.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80f8933640 1 --2- 172.21.15.119:0/4072318576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f40c5370 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f80e400bb10 tx=0x7f80e402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea42f9f47d48209b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80f37fe640 1 -- 172.21.15.119:0/4072318576 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e402ee80 con 0x7f80f4143600 2023-11-08T19:28:48.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80f37fe640 1 -- 172.21.15.119:0/4072318576 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e4036440 con 0x7f80f4143600 2023-11-08T19:28:48.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80f37fe640 1 -- 172.21.15.119:0/4072318576 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e40367b0 con 0x7f80f4143600 2023-11-08T19:28:48.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80fab97640 1 -- 172.21.15.119:0/4072318576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 msgr2=0x7f80f40c5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80fab97640 1 --2- 172.21.15.119:0/4072318576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f40c5370 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f80e400bb10 tx=0x7f80e402e0d0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80fab97640 1 -- 172.21.15.119:0/4072318576 shutdown_connections 2023-11-08T19:28:48.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80fab97640 1 --2- 172.21.15.119:0/4072318576 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f40c5370 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.204+0000 7f80fab97640 1 -- 172.21.15.119:0/4072318576 >> 172.21.15.119:0/4072318576 conn(0x7f80f40cb610 msgr2=0x7f80f40cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.205+0000 7f80fab97640 1 -- 172.21.15.119:0/4072318576 shutdown_connections 2023-11-08T19:28:48.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.205+0000 7f80fab97640 1 -- 172.21.15.119:0/4072318576 wait complete. 2023-11-08T19:28:48.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.206+0000 7f80fab97640 1 Processor -- start 2023-11-08T19:28:48.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.206+0000 7f80fab97640 1 -- start start 2023-11-08T19:28:48.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.207+0000 7f80fab97640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f406d250 0x7f80f4141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.207+0000 7f80fab97640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f4141b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.207+0000 7f80fab97640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f415d9f0 0x7f80f40c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.207+0000 7f80fab97640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4139060 con 0x7f80f4143600 2023-11-08T19:28:48.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.207+0000 7f80fab97640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4139340 con 0x7f80f415d9f0 2023-11-08T19:28:48.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.207+0000 7f80fab97640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4139620 con 0x7f80f406d250 2023-11-08T19:28:48.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f4141b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f4141b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36520/0 (socket says 172.21.15.119:36520) 2023-11-08T19:28:48.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 -- 172.21.15.119:0/3601067273 learned_addr learned my addr 172.21.15.119:0/3601067273 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:48.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f8933640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f406d250 0x7f80f4141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 -- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f406d250 msgr2=0x7f80f4141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f406d250 0x7f80f4141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 -- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f415d9f0 msgr2=0x7f80f40c6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f9134640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f415d9f0 0x7f80f40c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f415d9f0 0x7f80f40c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f3fff640 1 -- 172.21.15.119:0/3601067273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e400b770 con 0x7f80f4143600 2023-11-08T19:28:48.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.208+0000 7f80f8933640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f406d250 0x7f80f4141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:48.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.209+0000 7f80f3fff640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f4141b80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00c730 tx=0x7f80dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.209+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc003b70 con 0x7f80f4143600 2023-11-08T19:28:48.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.209+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc003cf0 con 0x7f80f4143600 2023-11-08T19:28:48.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.209+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc003ea0 con 0x7f80f4143600 2023-11-08T19:28:48.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.209+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f4139900 con 0x7f80f4143600 2023-11-08T19:28:48.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.209+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f4139ec0 con 0x7f80f4143600 2023-11-08T19:28:48.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.210+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f40c4230 con 0x7f80f4143600 2023-11-08T19:28:48.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.211+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f80dc00aa90 con 0x7f80f4143600 2023-11-08T19:28:48.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.212+0000 7f80f17fa640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.212+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f80dc0896d0 con 0x7f80f4143600 2023-11-08T19:28:48.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.212+0000 7f80f8933640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.213+0000 7f80f8933640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f80e4038000 tx=0x7f80e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.217+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80dc0544b0 con 0x7f80f4143600 2023-11-08T19:28:48.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.404+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80f40ce320 con 0x7f80f4143600 2023-11-08T19:28:48.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.405+0000 7f80f17fa640 1 -- 172.21.15.119:0/3601067273 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80dc057ad0 con 0x7f80f4143600 2023-11-08T19:28:48.407 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:28:48.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.407+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 msgr2=0x7f80d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.407+0000 7f80fab97640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f80e4038000 tx=0x7f80e4008040 comp rx=0 tx=0).stop 2023-11-08T19:28:48.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.407+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 msgr2=0x7f80f4141b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.407+0000 7f80fab97640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f4141b80 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f80dc00c730 tx=0x7f80dc00cbf0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 shutdown_connections 2023-11-08T19:28:48.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f80d406aea0 0x7f80d406d360 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f80f415d9f0 0x7f80f40c6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f80f4143600 0x7f80f4141b80 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 --2- 172.21.15.119:0/3601067273 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f80f406d250 0x7f80f4141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 >> 172.21.15.119:0/3601067273 conn(0x7f80f40cb610 msgr2=0x7f80f40708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 shutdown_connections 2023-11-08T19:28:48.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.408+0000 7f80fab97640 1 -- 172.21.15.119:0/3601067273 wait complete. 2023-11-08T19:28:48.428 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:28:48.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.641+0000 7f3521685640 1 Processor -- start 2023-11-08T19:28:48.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.641+0000 7f3521685640 1 -- start start 2023-11-08T19:28:48.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.642+0000 7f3521685640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f351c15b2b0 0x7f351c071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.642+0000 7f3521685640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f351c0cf5e0 0x7f351c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.642+0000 7f3521685640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f351c0c9440 con 0x7f351c15b690 2023-11-08T19:28:48.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.642+0000 7f3521685640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0c9690 con 0x7f351c0cf5e0 2023-11-08T19:28:48.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.642+0000 7f3521685640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f351c0c9970 con 0x7f351c15b2b0 2023-11-08T19:28:48.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f351c0cf5e0 0x7f351c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f351c0cf5e0 0x7f351c0cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35406/0 (socket says 172.21.15.119:35406) 2023-11-08T19:28:48.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 -- 172.21.15.119:0/1397795081 learned_addr learned my addr 172.21.15.119:0/1397795081 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:48.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f351bfff640 1 --2- 172.21.15.119:0/1397795081 >> v2:172.21.15.188:3300/0 conn(0x7f351c15b2b0 0x7f351c071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 -- 172.21.15.119:0/1397795081 >> v2:172.21.15.188:3300/0 conn(0x7f351c15b2b0 msgr2=0x7f351c071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:48.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 --2- 172.21.15.119:0/1397795081 >> v2:172.21.15.188:3300/0 conn(0x7f351c15b2b0 0x7f351c071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 -- 172.21.15.119:0/1397795081 >> v1:172.21.15.188:6789/0 conn(0x7f351c15b690 legacy=0x7f351c0ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:48.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.643+0000 7f3520e84640 1 -- 172.21.15.119:0/1397795081 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0c9c50 con 0x7f351c0cf5e0 2023-11-08T19:28:48.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.644+0000 7f3520e84640 1 --2- 172.21.15.119:0/1397795081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c0cf5e0 0x7f351c0cab60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f351c06b3a0 tx=0x7f351402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=499bee3b11fc6028 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f351affd640 1 -- 172.21.15.119:0/1397795081 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351402c9f0 con 0x7f351c0cf5e0 2023-11-08T19:28:48.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f351affd640 1 -- 172.21.15.119:0/1397795081 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351402cb50 con 0x7f351c0cf5e0 2023-11-08T19:28:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f351affd640 1 -- 172.21.15.119:0/1397795081 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35140343f0 con 0x7f351c0cf5e0 2023-11-08T19:28:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 -- 172.21.15.119:0/1397795081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c0cf5e0 msgr2=0x7f351c0cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 --2- 172.21.15.119:0/1397795081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c0cf5e0 0x7f351c0cab60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f351c06b3a0 tx=0x7f351402be50 comp rx=0 tx=0).stop 2023-11-08T19:28:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 -- 172.21.15.119:0/1397795081 shutdown_connections 2023-11-08T19:28:48.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 --2- 172.21.15.119:0/1397795081 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c0cf5e0 0x7f351c0cab60 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 --2- 172.21.15.119:0/1397795081 >> v2:172.21.15.188:3300/0 conn(0x7f351c15b2b0 0x7f351c071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 -- 172.21.15.119:0/1397795081 >> 172.21.15.119:0/1397795081 conn(0x7f351c0ceac0 msgr2=0x7f351c0ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 -- 172.21.15.119:0/1397795081 shutdown_connections 2023-11-08T19:28:48.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.645+0000 7f3521685640 1 -- 172.21.15.119:0/1397795081 wait complete. 2023-11-08T19:28:48.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.647+0000 7f3521685640 1 Processor -- start 2023-11-08T19:28:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.647+0000 7f3521685640 1 -- start start 2023-11-08T19:28:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3521685640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f351c0cf5e0 0x7f351c0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3521685640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f351c15b2b0 0x7f351c0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3521685640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 0x7f351c06b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3521685640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c0adef0 con 0x7f351c15b690 2023-11-08T19:28:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3521685640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c15bb40 con 0x7f351c15b2b0 2023-11-08T19:28:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3521685640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351c15be20 con 0x7f351c0cf5e0 2023-11-08T19:28:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3520e84640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 0x7f351c06b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.648+0000 7f3520e84640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 0x7f351c06b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35408/0 (socket says 172.21.15.119:35408) 2023-11-08T19:28:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 -- 172.21.15.119:0/3739176346 learned_addr learned my addr 172.21.15.119:0/3739176346 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:48.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f351bfff640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f351c0cf5e0 0x7f351c0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f351b7fe640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f351c15b2b0 0x7f351c0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 -- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f351c0cf5e0 msgr2=0x7f351c0ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f351c0cf5e0 0x7f351c0ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 -- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f351c15b2b0 msgr2=0x7f351c0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f351c15b2b0 0x7f351c0acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 -- 172.21.15.119:0/3739176346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3514009b30 con 0x7f351c15b690 2023-11-08T19:28:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.649+0000 7f3520e84640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 0x7f351c06b5e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3514036040 tx=0x7f3514004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.650+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514009510 con 0x7f351c15b690 2023-11-08T19:28:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.650+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351403e450 con 0x7f351c15b690 2023-11-08T19:28:48.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.650+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351403e7e0 con 0x7f351c15b690 2023-11-08T19:28:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.650+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c15c100 con 0x7f351c15b690 2023-11-08T19:28:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.650+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c0adb50 con 0x7f351c15b690 2023-11-08T19:28:48.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.651+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351c15c5c0 con 0x7f351c15b690 2023-11-08T19:28:48.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.652+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f351403e940 con 0x7f351c15b690 2023-11-08T19:28:48.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.653+0000 7f3518ff9640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f350806ae70 0x7f350806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:48.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.653+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f35140ac620 con 0x7f351c15b690 2023-11-08T19:28:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.653+0000 7f351bfff640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f350806ae70 0x7f350806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.654+0000 7f351bfff640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f350806ae70 0x7f350806d330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f350c006fd0 tx=0x7f350c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:48.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.657+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3514077490 con 0x7f351c15b690 2023-11-08T19:28:48.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.851+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f351c04ece0 con 0x7f351c15b690 2023-11-08T19:28:48.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.852+0000 7f3518ff9640 1 -- 172.21.15.119:0/3739176346 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f351407aab0 con 0x7f351c15b690 2023-11-08T19:28:48.854 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:48.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f350806ae70 msgr2=0x7f350806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f350806ae70 0x7f350806d330 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f350c006fd0 tx=0x7f350c008040 comp rx=0 tx=0).stop 2023-11-08T19:28:48.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 msgr2=0x7f351c06b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:48.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 0x7f351c06b5e0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f3514036040 tx=0x7f3514004a40 comp rx=0 tx=0).stop 2023-11-08T19:28:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 shutdown_connections 2023-11-08T19:28:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f350806ae70 0x7f350806d330 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f351c15b690 0x7f351c06b5e0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f351c15b2b0 0x7f351c0acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 --2- 172.21.15.119:0/3739176346 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f351c0cf5e0 0x7f351c0ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 >> 172.21.15.119:0/3739176346 conn(0x7f351c0ceac0 msgr2=0x7f351c1598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:48.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 shutdown_connections 2023-11-08T19:28:48.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:48.855+0000 7f3521685640 1 -- 172.21.15.119:0/3739176346 wait complete. 2023-11-08T19:28:48.875 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:28:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.093+0000 7f2424db7640 1 Processor -- start 2023-11-08T19:28:49.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.093+0000 7f2424db7640 1 -- start start 2023-11-08T19:28:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f2424db7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f24200cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f2424db7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f242006a830 0x7f242015c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f2424db7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24200ced30 con 0x7f24200cf670 2023-11-08T19:28:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f2424db7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24200ce120 con 0x7f242006a830 2023-11-08T19:28:49.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f2424db7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200ce400 con 0x7f24200cade0 2023-11-08T19:28:49.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f241dd9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f24200cf670 0x7f242006a120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39598/0 (socket says 172.21.15.119:39598) 2023-11-08T19:28:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.095+0000 7f241dd9b640 1 -- 172.21.15.119:0/1445879019 learned_addr learned my addr 172.21.15.119:0/1445879019 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:49.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241e59c640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f24200cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241ed9d640 1 --2- 172.21.15.119:0/1445879019 >> v2:172.21.15.188:3300/0 conn(0x7f242006a830 0x7f242015c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 913353723 0 0) 0x7f24200ced30 con 0x7f24200cf670 2023-11-08T19:28:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f240c003520 con 0x7f24200cf670 2023-11-08T19:28:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1645930808 0 0) 0x7f240c003520 con 0x7f24200cf670 2023-11-08T19:28:49.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 msgr2=0x7f24200cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f24200cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 >> v2:172.21.15.188:3300/0 conn(0x7f242006a830 msgr2=0x7f242015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 --2- 172.21.15.119:0/1445879019 >> v2:172.21.15.188:3300/0 conn(0x7f242006a830 0x7f242015c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200ce6e0 con 0x7f24200cf670 2023-11-08T19:28:49.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2408001670 con 0x7f24200cf670 2023-11-08T19:28:49.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241ed9d640 1 --2- 172.21.15.119:0/1445879019 >> v2:172.21.15.188:3300/0 conn(0x7f242006a830 0x7f242015c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:49.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 >> v1:172.21.15.119:6789/0 conn(0x7f24200cf670 legacy=0x7f242006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c003970 0x7f240c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0047c0 0x7f240c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f240c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200ce120 con 0x7f240c0047c0 2023-11-08T19:28:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200ce400 con 0x7f240c003970 2023-11-08T19:28:49.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.096+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240c00ba30 con 0x7f24200cade0 2023-11-08T19:28:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241dd9b640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0047c0 0x7f240c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241e59c640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c003970 0x7f240c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241ed9d640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f240c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241dd9b640 1 -- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 msgr2=0x7f240c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241dd9b640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f240c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241dd9b640 1 -- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c003970 msgr2=0x7f240c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241dd9b640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c003970 0x7f240c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241dd9b640 1 -- 172.21.15.119:0/1445879019 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240c003520 con 0x7f240c0047c0 2023-11-08T19:28:49.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.097+0000 7f241ed9d640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24200cade0 0x7f240c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:49.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.098+0000 7f241dd9b640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0047c0 0x7f240c004c40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f240800c210 tx=0x7f240c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b986c61f986dee1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.098+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240802cae0 con 0x7f240c0047c0 2023-11-08T19:28:49.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.098+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240802cc40 con 0x7f240c0047c0 2023-11-08T19:28:49.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.098+0000 7f241d59a640 1 -- 172.21.15.119:0/1445879019 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080058d0 con 0x7f240c0047c0 2023-11-08T19:28:49.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f2424db7640 1 -- 172.21.15.119:0/1445879019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0047c0 msgr2=0x7f240c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f2424db7640 1 --2- 172.21.15.119:0/1445879019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f240c0047c0 0x7f240c004c40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f240800c210 tx=0x7f240c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f241ed9d640 1 -- 172.21.15.119:0/1445879019 reap_dead start 2023-11-08T19:28:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f2424db7640 1 -- 172.21.15.119:0/1445879019 shutdown_connections 2023-11-08T19:28:49.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f2424db7640 1 -- 172.21.15.119:0/1445879019 >> 172.21.15.119:0/1445879019 conn(0x7f24200cfd00 msgr2=0x7f24200d00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f2424db7640 1 -- 172.21.15.119:0/1445879019 shutdown_connections 2023-11-08T19:28:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.099+0000 7f2424db7640 1 -- 172.21.15.119:0/1445879019 wait complete. 2023-11-08T19:28:49.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.101+0000 7f2424db7640 1 Processor -- start 2023-11-08T19:28:49.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.101+0000 7f2424db7640 1 -- start start 2023-11-08T19:28:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f2424db7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0047c0 0x7f24200b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f2424db7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 0x7f24200c54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f2424db7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24201d4a10 0x7f24201d4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f2424db7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24200b2a30 con 0x7f24200c3f00 2023-11-08T19:28:49.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f2424db7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24201d53d0 con 0x7f240c0047c0 2023-11-08T19:28:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f2424db7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24200c44a0 con 0x7f24201d4a10 2023-11-08T19:28:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f241dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 0x7f24200c54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f241dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 0x7f24200c54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35418/0 (socket says 172.21.15.119:35418) 2023-11-08T19:28:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.102+0000 7f241dd9b640 1 -- 172.21.15.119:0/394416259 learned_addr learned my addr 172.21.15.119:0/394416259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241e59c640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0047c0 0x7f24200b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241ed9d640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24201d4a10 0x7f24201d4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241dd9b640 1 -- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24201d4a10 msgr2=0x7f24201d4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241dd9b640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24201d4a10 0x7f24201d4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241dd9b640 1 -- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0047c0 msgr2=0x7f24200b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241dd9b640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0047c0 0x7f24200b5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241dd9b640 1 -- 172.21.15.119:0/394416259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24200c4780 con 0x7f24200c3f00 2023-11-08T19:28:49.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.103+0000 7f241dd9b640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 0x7f24200c54a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f240800c210 tx=0x7f242006cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.104+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408001630 con 0x7f24200c3f00 2023-11-08T19:28:49.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.104+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2408007070 con 0x7f24200c3f00 2023-11-08T19:28:49.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.104+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24080073e0 con 0x7f24200c3f00 2023-11-08T19:28:49.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.104+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24200c4a60 con 0x7f24200c3f00 2023-11-08T19:28:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.104+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24200b58a0 con 0x7f24200c3f00 2023-11-08T19:28:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.105+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24200af2a0 con 0x7f24200c3f00 2023-11-08T19:28:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.106+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2408032070 con 0x7f24200c3f00 2023-11-08T19:28:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.107+0000 7f2416ffd640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f240006ae70 0x7f240006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.107+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f24080aacb0 con 0x7f24200c3f00 2023-11-08T19:28:49.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.107+0000 7f241e59c640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f240006ae70 0x7f240006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.108+0000 7f241e59c640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f240006ae70 0x7f240006d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2410007440 tx=0x7f241000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.111+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2408075b40 con 0x7f24200c3f00 2023-11-08T19:28:49.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.301+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7f24200b0af0 con 0x7f24200c3f00 2023-11-08T19:28:49.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.302+0000 7f2416ffd640 1 -- 172.21.15.119:0/394416259 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v92) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2408045070 con 0x7f24200c3f00 2023-11-08T19:28:49.304 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:49.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.304+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f240006ae70 msgr2=0x7f240006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.304+0000 7f2424db7640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f240006ae70 0x7f240006d330 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f2410007440 tx=0x7f241000b040 comp rx=0 tx=0).stop 2023-11-08T19:28:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.304+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 msgr2=0x7f24200c54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.304+0000 7f2424db7640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 0x7f24200c54a0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f240800c210 tx=0x7f242006cbb0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.304+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 shutdown_connections 2023-11-08T19:28:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f240006ae70 0x7f240006d330 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24201d4a10 0x7f24201d4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24200c3f00 0x7f24200c54a0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 --2- 172.21.15.119:0/394416259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f240c0047c0 0x7f24200b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 >> 172.21.15.119:0/394416259 conn(0x7f24200cfd00 msgr2=0x7f24200ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:49.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 shutdown_connections 2023-11-08T19:28:49.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.305+0000 7f2424db7640 1 -- 172.21.15.119:0/394416259 wait complete. 2023-11-08T19:28:49.326 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:28:49.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.543+0000 7fcc6c644640 1 Processor -- start 2023-11-08T19:28:49.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.543+0000 7fcc6c644640 1 -- start start 2023-11-08T19:28:49.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.544+0000 7fcc6c644640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcc64143600 0x7fcc640c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6c644640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc6415d9f0 0x7fcc6415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6c644640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6415e440 con 0x7fcc6406cb40 2023-11-08T19:28:49.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6c644640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640c4f50 con 0x7fcc6415d9f0 2023-11-08T19:28:49.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6c644640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640c4960 con 0x7fcc64143600 2023-11-08T19:28:49.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc69bdf640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcc6415d9f0 0x7fcc6415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6abe1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcc6406cb40 0x7fcc6406cf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39600/0 (socket says 172.21.15.119:39600) 2023-11-08T19:28:49.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6abe1640 1 -- 172.21.15.119:0/1507380224 learned_addr learned my addr 172.21.15.119:0/1507380224 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:49.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc6a3e0640 1 --2- 172.21.15.119:0/1507380224 >> v2:172.21.15.188:3300/0 conn(0x7fcc64143600 0x7fcc640c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1507380224 >> v2:172.21.15.188:3300/0 conn(0x7fcc64143600 msgr2=0x7fcc640c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc69bdf640 1 --2- 172.21.15.119:0/1507380224 >> v2:172.21.15.188:3300/0 conn(0x7fcc64143600 0x7fcc640c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.545+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1507380224 >> v1:172.21.15.119:6789/0 conn(0x7fcc6406cb40 legacy=0x7fcc6406cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.546+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1507380224 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6415e720 con 0x7fcc6415d9f0 2023-11-08T19:28:49.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.546+0000 7fcc693de640 1 -- 172.21.15.119:0/1507380224 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435919856 0 0) 0x7fcc6415e440 con 0x7fcc6406cb40 2023-11-08T19:28:49.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.547+0000 7fcc69bdf640 1 --2- 172.21.15.119:0/1507380224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6415d9f0 0x7fcc6415de60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcc640c3f30 tx=0x7fcc5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef23308c2227820 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.547+0000 7fcc693de640 1 -- 172.21.15.119:0/1507380224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5402e070 con 0x7fcc6415d9f0 2023-11-08T19:28:49.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.547+0000 7fcc693de640 1 -- 172.21.15.119:0/1507380224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc540315a0 con 0x7fcc6415d9f0 2023-11-08T19:28:49.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.547+0000 7fcc693de640 1 -- 172.21.15.119:0/1507380224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54031930 con 0x7fcc6415d9f0 2023-11-08T19:28:49.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.547+0000 7fcc6c644640 1 -- 172.21.15.119:0/1507380224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6415d9f0 msgr2=0x7fcc6415de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.547+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1507380224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6415d9f0 0x7fcc6415de60 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fcc640c3f30 tx=0x7fcc5402b840 comp rx=0 tx=0).stop 2023-11-08T19:28:49.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.548+0000 7fcc6c644640 1 -- 172.21.15.119:0/1507380224 shutdown_connections 2023-11-08T19:28:49.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.548+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1507380224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc6415d9f0 0x7fcc6415de60 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.548+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1507380224 >> v2:172.21.15.188:3300/0 conn(0x7fcc64143600 0x7fcc640c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.548+0000 7fcc6c644640 1 -- 172.21.15.119:0/1507380224 >> 172.21.15.119:0/1507380224 conn(0x7fcc640cb610 msgr2=0x7fcc640cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:49.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.548+0000 7fcc6c644640 1 -- 172.21.15.119:0/1507380224 shutdown_connections 2023-11-08T19:28:49.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.548+0000 7fcc6c644640 1 -- 172.21.15.119:0/1507380224 wait complete. 2023-11-08T19:28:49.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.549+0000 7fcc6c644640 1 Processor -- start 2023-11-08T19:28:49.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.550+0000 7fcc6c644640 1 -- start start 2023-11-08T19:28:49.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.550+0000 7fcc6c644640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6406cb40 0x7fcc640b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc6c644640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 0x7fcc64152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc6c644640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6415d9f0 0x7fcc640af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc6c644640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64154480 con 0x7fcc64143600 2023-11-08T19:28:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc6c644640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64153800 con 0x7fcc6415d9f0 2023-11-08T19:28:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc6c644640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64153ae0 con 0x7fcc6406cb40 2023-11-08T19:28:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 0x7fcc64152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 0x7fcc64152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35442/0 (socket says 172.21.15.119:35442) 2023-11-08T19:28:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1056264594 learned_addr learned my addr 172.21.15.119:0/1056264594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1056264594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6406cb40 msgr2=0x7fcc640b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6406cb40 0x7fcc640b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1056264594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6415d9f0 msgr2=0x7fcc640af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6415d9f0 0x7fcc640af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.551+0000 7fcc69bdf640 1 -- 172.21.15.119:0/1056264594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54009400 con 0x7fcc64143600 2023-11-08T19:28:49.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.552+0000 7fcc69bdf640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 0x7fcc64152e90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009890 tx=0x7fcc540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.552+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54032840 con 0x7fcc64143600 2023-11-08T19:28:49.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.552+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc540329a0 con 0x7fcc64143600 2023-11-08T19:28:49.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.552+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54032d50 con 0x7fcc64143600 2023-11-08T19:28:49.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.552+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64153dc0 con 0x7fcc64143600 2023-11-08T19:28:49.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.552+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc641533d0 con 0x7fcc64143600 2023-11-08T19:28:49.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.553+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc641535c0 con 0x7fcc64143600 2023-11-08T19:28:49.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.554+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc5403b3f0 con 0x7fcc64143600 2023-11-08T19:28:49.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.555+0000 7fcc5affd640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc4806ae70 0x7fcc4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.555+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fcc540ab920 con 0x7fcc64143600 2023-11-08T19:28:49.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.555+0000 7fcc6a3e0640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc4806ae70 0x7fcc4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.556+0000 7fcc6a3e0640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc4806ae70 0x7fcc4806d330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcc600098f0 tx=0x7fcc60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.559+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc641535c0 con 0x7fcc64143600 2023-11-08T19:28:49.679 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:49.688 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:28:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.748+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7fcc641535c0 con 0x7fcc64143600 2023-11-08T19:28:49.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.749+0000 7fcc5affd640 1 -- 172.21.15.119:0/1056264594 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v92) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcc641535c0 con 0x7fcc64143600 2023-11-08T19:28:49.750 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:28:49.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.751+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc4806ae70 msgr2=0x7fcc4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.751+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc4806ae70 0x7fcc4806d330 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcc600098f0 tx=0x7fcc60008040 comp rx=0 tx=0).stop 2023-11-08T19:28:49.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.751+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 msgr2=0x7fcc64152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.751+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 0x7fcc64152e90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009890 tx=0x7fcc540046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 shutdown_connections 2023-11-08T19:28:49.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcc4806ae70 0x7fcc4806d330 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcc6415d9f0 0x7fcc640af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcc64143600 0x7fcc64152e90 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 --2- 172.21.15.119:0/1056264594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcc6406cb40 0x7fcc640b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 >> 172.21.15.119:0/1056264594 conn(0x7fcc640cb610 msgr2=0x7fcc640cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:49.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 shutdown_connections 2023-11-08T19:28:49.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.752+0000 7fcc6c644640 1 -- 172.21.15.119:0/1056264594 wait complete. 2023-11-08T19:28:49.771 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T19:28:49.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.959+0000 7fc52d856640 1 Processor -- start 2023-11-08T19:28:49.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.959+0000 7fc52d856640 1 -- start start 2023-11-08T19:28:49.962 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc52d856640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5280c9720 0x7fc5280c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc52d856640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5280c9360 con 0x7fc5281610c0 2023-11-08T19:28:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc52d856640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc52806d3c0 con 0x7fc528159d90 2023-11-08T19:28:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc52d856640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5280ca490 con 0x7fc5280c9720 2023-11-08T19:28:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc5277fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc5281610c0 0x7fc5281634b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39612/0 (socket says 172.21.15.119:39612) 2023-11-08T19:28:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc5277fe640 1 -- 172.21.15.119:0/443186555 learned_addr learned my addr 172.21.15.119:0/443186555 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.961+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/443186555 >> v2:172.21.15.188:3300/0 conn(0x7fc5280c9720 0x7fc5280c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4142560019 0 0) 0x7fc5280c9360 con 0x7fc5281610c0 2023-11-08T19:28:49.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc514003440 con 0x7fc5281610c0 2023-11-08T19:28:49.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3701411369 0 0) 0x7fc514003440 con 0x7fc5281610c0 2023-11-08T19:28:49.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 >> v2:172.21.15.188:3300/0 conn(0x7fc5280c9720 msgr2=0x7fc5280c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 --2- 172.21.15.119:0/443186555 >> v2:172.21.15.188:3300/0 conn(0x7fc5280c9720 0x7fc5280c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 >> v1:172.21.15.188:6789/0 conn(0x7fc528159d90 legacy=0x7fc52815a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5280ca770 con 0x7fc5281610c0 2023-11-08T19:28:49.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fc520001670 con 0x7fc5281610c0 2023-11-08T19:28:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 >> v1:172.21.15.119:6789/0 conn(0x7fc5281610c0 legacy=0x7fc5281634b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc514003a60 0x7fc514003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc514008850 0x7fc51400ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc51400b480 0x7fc51400d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280ca490 con 0x7fc514008850 2023-11-08T19:28:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51400e220 con 0x7fc51400b480 2023-11-08T19:28:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc51400e3e0 con 0x7fc514003a60 2023-11-08T19:28:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.962+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc514008850 0x7fc51400ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc5277fe640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc51400b480 0x7fc51400d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc526ffd640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc514003a60 0x7fc514003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc51e7fc640 1 -- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc514003a60 msgr2=0x7fc514003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc514003a60 0x7fc514003e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc51e7fc640 1 -- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc51400b480 msgr2=0x7fc51400d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc51400b480 0x7fc51400d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc51e7fc640 1 -- 172.21.15.119:0/443186555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514003440 con 0x7fc514008850 2023-11-08T19:28:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.963+0000 7fc5277fe640 1 -- 172.21.15.119:0/443186555 reap_dead start 2023-11-08T19:28:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.964+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc514008850 0x7fc51400ac80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc510009fc0 tx=0x7fc5140036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a0d2be4057e4a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.964+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51002cc20 con 0x7fc514008850 2023-11-08T19:28:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.964+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000f040 con 0x7fc514008850 2023-11-08T19:28:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.964+0000 7fc5267fc640 1 -- 172.21.15.119:0/443186555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510034440 con 0x7fc514008850 2023-11-08T19:28:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 -- 172.21.15.119:0/443186555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc514008850 msgr2=0x7fc51400ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc514008850 0x7fc51400ac80 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fc510009fc0 tx=0x7fc5140036f0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 -- 172.21.15.119:0/443186555 shutdown_connections 2023-11-08T19:28:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 --2- 172.21.15.119:0/443186555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc514008850 0x7fc51400ac80 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 -- 172.21.15.119:0/443186555 >> 172.21.15.119:0/443186555 conn(0x7fc5280cb610 msgr2=0x7fc5280cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 -- 172.21.15.119:0/443186555 shutdown_connections 2023-11-08T19:28:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.965+0000 7fc52d856640 1 -- 172.21.15.119:0/443186555 wait complete. 2023-11-08T19:28:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.967+0000 7fc52d856640 1 Processor -- start 2023-11-08T19:28:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.967+0000 7fc52d856640 1 -- start start 2023-11-08T19:28:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc52d856640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514008850 0x7fc5281f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc52d856640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 0x7fc528153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc52d856640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280ca9b0 0x7fc5280cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc52d856640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5280cb370 con 0x7fc528153a70 2023-11-08T19:28:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc52d856640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281f3710 con 0x7fc514008850 2023-11-08T19:28:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc52d856640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5281f6d30 con 0x7fc5280ca9b0 2023-11-08T19:28:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc51e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 0x7fc528153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc51e7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 0x7fc528153040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35466/0 (socket says 172.21.15.119:35466) 2023-11-08T19:28:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc51e7fc640 1 -- 172.21.15.119:0/1895871400 learned_addr learned my addr 172.21.15.119:0/1895871400 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc526ffd640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514008850 0x7fc5281f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.968+0000 7fc5277fe640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280ca9b0 0x7fc5280cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc51e7fc640 1 -- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280ca9b0 msgr2=0x7fc5280cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280ca9b0 0x7fc5280cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc51e7fc640 1 -- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514008850 msgr2=0x7fc5281f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514008850 0x7fc5281f67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc51e7fc640 1 -- 172.21.15.119:0/1895871400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5281f2b00 con 0x7fc528153a70 2023-11-08T19:28:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc5277fe640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280ca9b0 0x7fc5280cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc51e7fc640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 0x7fc528153040 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc51002c690 tx=0x7fc510009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510034c50 con 0x7fc528153a70 2023-11-08T19:28:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc510034dd0 con 0x7fc528153a70 2023-11-08T19:28:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5281f2de0 con 0x7fc528153a70 2023-11-08T19:28:49.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510003bd0 con 0x7fc528153a70 2023-11-08T19:28:49.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.969+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5281f33a0 con 0x7fc528153a70 2023-11-08T19:28:49.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.971+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc528155200 con 0x7fc528153a70 2023-11-08T19:28:49.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.971+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc510040450 con 0x7fc528153a70 2023-11-08T19:28:49.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.972+0000 7fc524ff9640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc50806ae70 0x7fc50806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:49.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.972+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fc5100abfe0 con 0x7fc528153a70 2023-11-08T19:28:49.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.972+0000 7fc526ffd640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc50806ae70 0x7fc50806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:49.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.973+0000 7fc526ffd640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc50806ae70 0x7fc50806d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc5281f3960 tx=0x7fc518009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:49.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:49.977+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc510076e70 con 0x7fc528153a70 2023-11-08T19:28:50.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.167+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc52806a860 con 0x7fc528153a70 2023-11-08T19:28:50.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.168+0000 7fc524ff9640 1 -- 172.21.15.119:0/1895871400 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+287 (secure 0 0 0) 0x7fc51007a490 con 0x7fc528153a70 2023-11-08T19:28:50.170 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:50.170 INFO:teuthology.orchestra.run.smithi119.stdout:[ 2023-11-08T19:28:50.170 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:50.170 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 1, 2023-11-08T19:28:50.171 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": ".mgr" 2023-11-08T19:28:50.171 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:50.171 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:50.171 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 7, 2023-11-08T19:28:50.171 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "snap-test-pool" 2023-11-08T19:28:50.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:50.172 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:50.172 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 12, 2023-11-08T19:28:50.172 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "test_data_pool" 2023-11-08T19:28:50.172 INFO:teuthology.orchestra.run.smithi119.stdout: }, 2023-11-08T19:28:50.173 INFO:teuthology.orchestra.run.smithi119.stdout: { 2023-11-08T19:28:50.173 INFO:teuthology.orchestra.run.smithi119.stdout: "poolnum": 13, 2023-11-08T19:28:50.173 INFO:teuthology.orchestra.run.smithi119.stdout: "poolname": "test_metadata_pool" 2023-11-08T19:28:50.173 INFO:teuthology.orchestra.run.smithi119.stdout: } 2023-11-08T19:28:50.173 INFO:teuthology.orchestra.run.smithi119.stdout:] 2023-11-08T19:28:50.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc50806ae70 msgr2=0x7fc50806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc50806ae70 0x7fc50806d330 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc5281f3960 tx=0x7fc518009040 comp rx=0 tx=0).stop 2023-11-08T19:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 msgr2=0x7fc528153040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:50.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 0x7fc528153040 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fc51002c690 tx=0x7fc510009750 comp rx=0 tx=0).stop 2023-11-08T19:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 shutdown_connections 2023-11-08T19:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc50806ae70 0x7fc50806d330 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5280ca9b0 0x7fc5280cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc528153a70 0x7fc528153040 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 --2- 172.21.15.119:0/1895871400 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc514008850 0x7fc5281f67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 >> 172.21.15.119:0/1895871400 conn(0x7fc5280cb610 msgr2=0x7fc5280ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:50.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 shutdown_connections 2023-11-08T19:28:50.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.171+0000 7fc52d856640 1 -- 172.21.15.119:0/1895871400 wait complete. 2023-11-08T19:28:50.191 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T19:28:50.192 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T19:28:50.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.410+0000 7f4bc57d0640 1 Processor -- start 2023-11-08T19:28:50.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.410+0000 7f4bc57d0640 1 -- start start 2023-11-08T19:28:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bc57d0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4bc0144580 0x7f4bc015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bc57d0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc015ea60 con 0x7f4bc0144960 2023-11-08T19:28:50.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bc57d0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc00c9360 con 0x7f4bc015f1d0 2023-11-08T19:28:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bc57d0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc00c9150 con 0x7f4bc0144580 2023-11-08T19:28:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bbeffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4bc0144580 0x7f4bc015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bb67fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4bc0144960 0x7f4bc00cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39620/0 (socket says 172.21.15.119:39620) 2023-11-08T19:28:50.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.412+0000 7f4bb67fc640 1 -- 172.21.15.119:0/1243693698 learned_addr learned my addr 172.21.15.119:0/1243693698 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:50.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.413+0000 7f4bbeffd640 1 -- 172.21.15.119:0/1243693698 >> v1:172.21.15.188:6789/0 conn(0x7f4bc015f1d0 legacy=0x7f4bc00c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:50.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.413+0000 7f4bbeffd640 1 -- 172.21.15.119:0/1243693698 >> v1:172.21.15.119:6789/0 conn(0x7f4bc0144960 legacy=0x7f4bc00cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:50.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.413+0000 7f4bbeffd640 1 -- 172.21.15.119:0/1243693698 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00c9640 con 0x7f4bc0144580 2023-11-08T19:28:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.413+0000 7f4bbe7fc640 1 -- 172.21.15.119:0/1243693698 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1324814691 0 0) 0x7f4bc015ea60 con 0x7f4bc0144960 2023-11-08T19:28:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.414+0000 7f4bbeffd640 1 --2- 172.21.15.119:0/1243693698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc0144580 0x7f4bc015bff0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc627e745b320032 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.414+0000 7f4bbe7fc640 1 -- 172.21.15.119:0/1243693698 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb002e070 con 0x7f4bc0144580 2023-11-08T19:28:50.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.414+0000 7f4bbe7fc640 1 -- 172.21.15.119:0/1243693698 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb00315a0 con 0x7f4bc0144580 2023-11-08T19:28:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.414+0000 7f4bbe7fc640 1 -- 172.21.15.119:0/1243693698 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb0031930 con 0x7f4bc0144580 2023-11-08T19:28:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 -- 172.21.15.119:0/1243693698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc0144580 msgr2=0x7f4bc015bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/1243693698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc0144580 0x7f4bc015bff0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00097a0 tx=0x7f4bb002b840 comp rx=0 tx=0).stop 2023-11-08T19:28:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 -- 172.21.15.119:0/1243693698 shutdown_connections 2023-11-08T19:28:50.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/1243693698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc0144580 0x7f4bc015bff0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 -- 172.21.15.119:0/1243693698 >> 172.21.15.119:0/1243693698 conn(0x7f4bc00cb610 msgr2=0x7f4bc00cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:50.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 -- 172.21.15.119:0/1243693698 shutdown_connections 2023-11-08T19:28:50.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.415+0000 7f4bc57d0640 1 -- 172.21.15.119:0/1243693698 wait complete. 2023-11-08T19:28:50.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.416+0000 7f4bc57d0640 1 Processor -- start 2023-11-08T19:28:50.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.417+0000 7f4bc57d0640 1 -- start start 2023-11-08T19:28:50.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.417+0000 7f4bc57d0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0144580 0x7f4bc00c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:50.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bc57d0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc0144960 0x7f4bc00c81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bc57d0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 0x7f4bc01f2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bc57d0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00a84b0 con 0x7f4bc015f1d0 2023-11-08T19:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bc57d0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc00a8a70 con 0x7f4bc0144580 2023-11-08T19:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bc57d0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc01f3450 con 0x7f4bc0144960 2023-11-08T19:28:50.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 0x7f4bc01f2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:50.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 0x7f4bc01f2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35484/0 (socket says 172.21.15.119:35484) 2023-11-08T19:28:50.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 -- 172.21.15.119:0/2663607410 learned_addr learned my addr 172.21.15.119:0/2663607410 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:50.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 -- 172.21.15.119:0/2663607410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc0144960 msgr2=0x7f4bc00c81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:50.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc0144960 0x7f4bc00c81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 -- 172.21.15.119:0/2663607410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0144580 msgr2=0x7f4bc00c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:50.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0144580 0x7f4bc00c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:50.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.418+0000 7f4bbf7fe640 1 -- 172.21.15.119:0/2663607410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0009400 con 0x7f4bc015f1d0 2023-11-08T19:28:50.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.419+0000 7f4bbf7fe640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 0x7f4bc01f2990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800ba10 tx=0x7f4bb800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:50.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.419+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb800cac0 con 0x7f4bc015f1d0 2023-11-08T19:28:50.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.419+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb800cc20 con 0x7f4bc015f1d0 2023-11-08T19:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.419+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb8014440 con 0x7f4bc015f1d0 2023-11-08T19:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.419+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc01f3730 con 0x7f4bc015f1d0 2023-11-08T19:28:50.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.419+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc00affe0 con 0x7f4bc015f1d0 2023-11-08T19:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.420+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc01f3920 con 0x7f4bc015f1d0 2023-11-08T19:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.421+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bb801d3b0 con 0x7f4bc015f1d0 2023-11-08T19:28:50.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.422+0000 7f4bb7fff640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4ba006ae70 0x7f4ba006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:50.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.423+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f4bb80896f0 con 0x7f4bc015f1d0 2023-11-08T19:28:50.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.423+0000 7f4bbeffd640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4ba006ae70 0x7f4ba006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:50.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.423+0000 7f4bbeffd640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4ba006ae70 0x7f4ba006d330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0036000 tx=0x7f4bb0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:50.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.427+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4bc01f3920 con 0x7f4bc015f1d0 2023-11-08T19:28:50.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.617+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f4bc01f3920 con 0x7f4bc015f1d0 2023-11-08T19:28:50.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:50.996+0000 7f4bb7fff640 1 -- 172.21.15.119:0/2663607410 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4bc01f3920 con 0x7f4bc015f1d0 2023-11-08T19:28:50.998 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' created 2023-11-08T19:28:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.001+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4ba006ae70 msgr2=0x7f4ba006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.001+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4ba006ae70 0x7f4ba006d330 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0036000 tx=0x7f4bb0030000 comp rx=0 tx=0).stop 2023-11-08T19:28:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.001+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 msgr2=0x7f4bc01f2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.001+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 0x7f4bc01f2990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f4bb800ba10 tx=0x7f4bb800bed0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.002+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 shutdown_connections 2023-11-08T19:28:51.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.002+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4ba006ae70 0x7f4ba006d330 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.002+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4bc015f1d0 0x7f4bc01f2990 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.002+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bc0144960 0x7f4bc00c81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.002+0000 7f4bc57d0640 1 --2- 172.21.15.119:0/2663607410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bc0144580 0x7f4bc00c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.002+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 >> 172.21.15.119:0/2663607410 conn(0x7f4bc00cb610 msgr2=0x7f4bc006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.004+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 shutdown_connections 2023-11-08T19:28:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.004+0000 7f4bc57d0640 1 -- 172.21.15.119:0/2663607410 wait complete. 2023-11-08T19:28:51.032 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T19:28:51.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.239+0000 7f21ef026640 1 Processor -- start 2023-11-08T19:28:51.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.239+0000 7f21ef026640 1 -- start start 2023-11-08T19:28:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ef026640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f21e80cb4b0 0x7f21e806f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ef026640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e80cd020 0x7f21e80ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ef026640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e806feb0 con 0x7f21e80cb890 2023-11-08T19:28:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ef026640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e80c9400 con 0x7f21e80cb4b0 2023-11-08T19:28:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ef026640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80cd8c0 con 0x7f21e80cd020 2023-11-08T19:28:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ed5c3640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f21e80cb890 0x7f21e80ca160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39630/0 (socket says 172.21.15.119:39630) 2023-11-08T19:28:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.241+0000 7f21ed5c3640 1 -- 172.21.15.119:0/2731490415 learned_addr learned my addr 172.21.15.119:0/2731490415 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e7fff640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e80cd020 0x7f21e80ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/2731490415 >> v2:172.21.15.188:3300/0 conn(0x7f21e80cb4b0 0x7f21e806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275729732 0 0) 0x7f21e806feb0 con 0x7f21e80cb890 2023-11-08T19:28:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d8003520 con 0x7f21e80cb890 2023-11-08T19:28:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1494655959 0 0) 0x7f21d8003520 con 0x7f21e80cb890 2023-11-08T19:28:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e80cd020 msgr2=0x7f21e80ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:51.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e80cd020 0x7f21e80ad2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 >> v2:172.21.15.188:3300/0 conn(0x7f21e80cb4b0 msgr2=0x7f21e806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:51.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 --2- 172.21.15.119:0/2731490415 >> v2:172.21.15.188:3300/0 conn(0x7f21e80cb4b0 0x7f21e806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80ad810 con 0x7f21e80cb890 2023-11-08T19:28:51.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f21e00019f0 con 0x7f21e80cb890 2023-11-08T19:28:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 >> v1:172.21.15.119:6789/0 conn(0x7f21e80cb890 legacy=0x7f21e80ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.242+0000 7f21e77fe640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21e77fe640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d80047c0 0x7f21d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21e77fe640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d8005180 0x7f21d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80c9400 con 0x7f21d8003970 2023-11-08T19:28:51.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80cd8c0 con 0x7f21d80047c0 2023-11-08T19:28:51.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d800bef0 con 0x7f21d8005180 2023-11-08T19:28:51.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21e7fff640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e80cd020 0x7f21e80ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:51.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ed5c3640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d8005180 0x7f21d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ecdc2640 1 -- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d8005180 msgr2=0x7f21d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:51.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21d8005180 0x7f21d800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ecdc2640 1 -- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d80047c0 msgr2=0x7f21d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:51.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21d80047c0 0x7f21d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.243+0000 7f21ecdc2640 1 -- 172.21.15.119:0/2731490415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d8003520 con 0x7f21d8003970 2023-11-08T19:28:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.244+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21d8003d90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009ac0 tx=0x7f21d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9ba9414a9cda1a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.244+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc02c6d0 con 0x7f21d8003970 2023-11-08T19:28:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc02c830 con 0x7f21d8003970 2023-11-08T19:28:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21e77fe640 1 -- 172.21.15.119:0/2731490415 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc02cbc0 con 0x7f21d8003970 2023-11-08T19:28:51.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ef026640 1 -- 172.21.15.119:0/2731490415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 msgr2=0x7f21d8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ef026640 1 --2- 172.21.15.119:0/2731490415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21d8003d90 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f21dc009ac0 tx=0x7f21d80057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ed5c3640 1 -- 172.21.15.119:0/2731490415 reap_dead start 2023-11-08T19:28:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ef026640 1 -- 172.21.15.119:0/2731490415 shutdown_connections 2023-11-08T19:28:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ef026640 1 -- 172.21.15.119:0/2731490415 >> 172.21.15.119:0/2731490415 conn(0x7f21e80b0950 msgr2=0x7f21e80b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:51.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ef026640 1 -- 172.21.15.119:0/2731490415 shutdown_connections 2023-11-08T19:28:51.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.245+0000 7f21ef026640 1 -- 172.21.15.119:0/2731490415 wait complete. 2023-11-08T19:28:51.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.247+0000 7f21ef026640 1 Processor -- start 2023-11-08T19:28:51.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.247+0000 7f21ef026640 1 -- start start 2023-11-08T19:28:51.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.248+0000 7f21ef026640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21e80b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.248+0000 7f21ef026640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8070c10 0x7f21e80c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.248+0000 7f21ef026640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e8071070 0x7f21e80c6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.248+0000 7f21ef026640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80b3070 con 0x7f21d8003970 2023-11-08T19:28:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.248+0000 7f21ef026640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e81f6190 con 0x7f21e8071070 2023-11-08T19:28:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.248+0000 7f21ef026640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80c6810 con 0x7f21e8070c10 2023-11-08T19:28:51.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21e80b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21e80b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35506/0 (socket says 172.21.15.119:35506) 2023-11-08T19:28:51.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 -- 172.21.15.119:0/1165368083 learned_addr learned my addr 172.21.15.119:0/1165368083 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:51.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21e7fff640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8070c10 0x7f21e80c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 -- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8070c10 msgr2=0x7f21e80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:51.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8070c10 0x7f21e80c5c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 -- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e8071070 msgr2=0x7f21e80c6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ed5c3640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e8071070 0x7f21e80c6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e8071070 0x7f21e80c6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 -- 172.21.15.119:0/1165368083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80c6af0 con 0x7f21d8003970 2023-11-08T19:28:51.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21e7fff640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8070c10 0x7f21e80c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.249+0000 7f21ecdc2640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21e80b2b30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f21dc02c2d0 tx=0x7f21dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.250+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc003810 con 0x7f21d8003970 2023-11-08T19:28:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.250+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc003970 con 0x7f21d8003970 2023-11-08T19:28:51.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.250+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc003d20 con 0x7f21d8003970 2023-11-08T19:28:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.250+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e80b2050 con 0x7f21d8003970 2023-11-08T19:28:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.250+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e80b2610 con 0x7f21d8003970 2023-11-08T19:28:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.251+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e80b27a0 con 0x7f21d8003970 2023-11-08T19:28:51.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.252+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21dc040450 con 0x7f21d8003970 2023-11-08T19:28:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.253+0000 7f21e57fa640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21cc06aea0 0x7f21cc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.253+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 6638+0+0 (secure 0 0 0) 0x7f21dc0ab950 con 0x7f21d8003970 2023-11-08T19:28:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.253+0000 7f21e7fff640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21cc06aea0 0x7f21cc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:51.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.254+0000 7f21e7fff640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21cc06aea0 0x7f21cc06d360 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f21e80c6dd0 tx=0x7f21d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:51.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.258+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21dc0765c0 con 0x7f21d8003970 2023-11-08T19:28:51.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:51.448+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f21e804ece0 con 0x7f21d8003970 2023-11-08T19:28:52.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.013+0000 7f21e57fa640 1 -- 172.21.15.119:0/1165368083 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7f21dc0a9e60 con 0x7f21d8003970 2023-11-08T19:28:52.016 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' created 2023-11-08T19:28:52.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21cc06aea0 msgr2=0x7f21cc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:52.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21cc06aea0 0x7f21cc06d360 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f21e80c6dd0 tx=0x7f21d4008040 comp rx=0 tx=0).stop 2023-11-08T19:28:52.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 msgr2=0x7f21e80b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:52.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21e80b2b30 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f21dc02c2d0 tx=0x7f21dc004730 comp rx=0 tx=0).stop 2023-11-08T19:28:52.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 shutdown_connections 2023-11-08T19:28:52.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21cc06aea0 0x7f21cc06d360 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f21e8071070 0x7f21e80c6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f21e8070c10 0x7f21e80c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 --2- 172.21.15.119:0/1165368083 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f21d8003970 0x7f21e80b2b30 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 >> 172.21.15.119:0/1165368083 conn(0x7f21e80b0950 msgr2=0x7f21e80af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:52.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.017+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 shutdown_connections 2023-11-08T19:28:52.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.018+0000 7f21ef026640 1 -- 172.21.15.119:0/1165368083 wait complete. 2023-11-08T19:28:52.055 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T19:28:52.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.240+0000 7fca490a9640 1 Processor -- start 2023-11-08T19:28:52.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.240+0000 7fca490a9640 1 -- start start 2023-11-08T19:28:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.241+0000 7fca490a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fca44144580 0x7fca4415bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca490a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca44144960 0x7fca440cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca490a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4415ea60 con 0x7fca4415f1d0 2023-11-08T19:28:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca490a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440c9150 con 0x7fca44144960 2023-11-08T19:28:52.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca490a9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440cb270 con 0x7fca44144580 2023-11-08T19:28:52.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca3bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fca44144960 0x7fca440cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:52.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca4359e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fca4415f1d0 0x7fca440c3a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39642/0 (socket says 172.21.15.119:39642) 2023-11-08T19:28:52.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca4359e640 1 -- 172.21.15.119:0/88056162 learned_addr learned my addr 172.21.15.119:0/88056162 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:52.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca42d9d640 1 --2- 172.21.15.119:0/88056162 >> v2:172.21.15.188:3300/0 conn(0x7fca44144580 0x7fca4415bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:52.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca3bfff640 1 -- 172.21.15.119:0/88056162 >> v2:172.21.15.188:3300/0 conn(0x7fca44144580 msgr2=0x7fca4415bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:52.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca3bfff640 1 --2- 172.21.15.119:0/88056162 >> v2:172.21.15.188:3300/0 conn(0x7fca44144580 0x7fca4415bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.242+0000 7fca3bfff640 1 -- 172.21.15.119:0/88056162 >> v1:172.21.15.119:6789/0 conn(0x7fca4415f1d0 legacy=0x7fca440c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:52.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.243+0000 7fca3bfff640 1 -- 172.21.15.119:0/88056162 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440c9360 con 0x7fca44144960 2023-11-08T19:28:52.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.243+0000 7fca4259c640 1 -- 172.21.15.119:0/88056162 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708399960 0 0) 0x7fca4415ea60 con 0x7fca4415f1d0 2023-11-08T19:28:52.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.244+0000 7fca3bfff640 1 --2- 172.21.15.119:0/88056162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440cad30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fca440ceab0 tx=0x7fca2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b922b95248f2c2f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:52.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.244+0000 7fca4259c640 1 -- 172.21.15.119:0/88056162 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c02e070 con 0x7fca44144960 2023-11-08T19:28:52.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.244+0000 7fca4259c640 1 -- 172.21.15.119:0/88056162 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c0315a0 con 0x7fca44144960 2023-11-08T19:28:52.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.244+0000 7fca4259c640 1 -- 172.21.15.119:0/88056162 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c031930 con 0x7fca44144960 2023-11-08T19:28:52.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 -- 172.21.15.119:0/88056162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 msgr2=0x7fca440cad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:52.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 --2- 172.21.15.119:0/88056162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440cad30 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fca440ceab0 tx=0x7fca2c02b840 comp rx=0 tx=0).stop 2023-11-08T19:28:52.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 -- 172.21.15.119:0/88056162 shutdown_connections 2023-11-08T19:28:52.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 --2- 172.21.15.119:0/88056162 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440cad30 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 --2- 172.21.15.119:0/88056162 >> v2:172.21.15.188:3300/0 conn(0x7fca44144580 0x7fca4415bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 -- 172.21.15.119:0/88056162 >> 172.21.15.119:0/88056162 conn(0x7fca440cb610 msgr2=0x7fca440cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:52.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 -- 172.21.15.119:0/88056162 shutdown_connections 2023-11-08T19:28:52.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.245+0000 7fca490a9640 1 -- 172.21.15.119:0/88056162 wait complete. 2023-11-08T19:28:52.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.246+0000 7fca490a9640 1 Processor -- start 2023-11-08T19:28:52.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.247+0000 7fca490a9640 1 -- start start 2023-11-08T19:28:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.247+0000 7fca490a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44144580 0x7fca441f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca490a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440b1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca490a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415f1d0 0x7fca440b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca490a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44133de0 con 0x7fca44144960 2023-11-08T19:28:52.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca490a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca441300b0 con 0x7fca44144580 2023-11-08T19:28:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca490a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca44130390 con 0x7fca4415f1d0 2023-11-08T19:28:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440b1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440b1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35522/0 (socket says 172.21.15.119:35522) 2023-11-08T19:28:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 -- 172.21.15.119:0/1796152861 learned_addr learned my addr 172.21.15.119:0/1796152861 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:52.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 -- 172.21.15.119:0/1796152861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415f1d0 msgr2=0x7fca440b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:52.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415f1d0 0x7fca440b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 -- 172.21.15.119:0/1796152861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44144580 msgr2=0x7fca441f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:52.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44144580 0x7fca441f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:52.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.248+0000 7fca3bfff640 1 -- 172.21.15.119:0/1796152861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009400 con 0x7fca44144960 2023-11-08T19:28:52.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.249+0000 7fca3bfff640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440b1870 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009890 tx=0x7fca2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.249+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c032840 con 0x7fca44144960 2023-11-08T19:28:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.249+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c0329a0 con 0x7fca44144960 2023-11-08T19:28:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.249+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c032d50 con 0x7fca44144960 2023-11-08T19:28:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.249+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca44130670 con 0x7fca44144960 2023-11-08T19:28:52.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.249+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca441f0550 con 0x7fca44144960 2023-11-08T19:28:52.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.250+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44130e70 con 0x7fca44144960 2023-11-08T19:28:52.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.252+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca2c04e470 con 0x7fca44144960 2023-11-08T19:28:52.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.253+0000 7fca3b7fe640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca2806ae70 0x7fca2806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:52.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.253+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 7020+0+0 (secure 0 0 0) 0x7fca2c0ab9a0 con 0x7fca44144960 2023-11-08T19:28:52.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.253+0000 7fca42d9d640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca2806ae70 0x7fca2806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:52.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.254+0000 7fca42d9d640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca2806ae70 0x7fca2806d330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fca340098f0 tx=0x7fca34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:52.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.257+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca2c0765a0 con 0x7fca44144960 2023-11-08T19:28:52.331 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-08T19:28:52.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-08T19:28:52.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-08T19:28:52.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:52.481+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca44130e70 con 0x7fca44144960 2023-11-08T19:28:53.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.046+0000 7fca3b7fe640 1 -- 172.21.15.119:0/1796152861 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7fca44130e70 con 0x7fca44144960 2023-11-08T19:28:53.048 INFO:teuthology.orchestra.run.smithi119.stderr:new fs with metadata pool 16 and data pool 17 2023-11-08T19:28:53.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.058+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca2806ae70 msgr2=0x7fca2806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.058+0000 7fca490a9640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca2806ae70 0x7fca2806d330 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fca340098f0 tx=0x7fca34008040 comp rx=0 tx=0).stop 2023-11-08T19:28:53.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.058+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 msgr2=0x7fca440b1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.058+0000 7fca490a9640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440b1870 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009890 tx=0x7fca2c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.059+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 shutdown_connections 2023-11-08T19:28:53.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.059+0000 7fca490a9640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fca2806ae70 0x7fca2806d330 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.059+0000 7fca490a9640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fca4415f1d0 0x7fca440b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.059+0000 7fca490a9640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fca44144960 0x7fca440b1870 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.059+0000 7fca490a9640 1 --2- 172.21.15.119:0/1796152861 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fca44144580 0x7fca441f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.059+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 >> 172.21.15.119:0/1796152861 conn(0x7fca440cb610 msgr2=0x7fca4406a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:53.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.061+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 shutdown_connections 2023-11-08T19:28:53.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.061+0000 7fca490a9640 1 -- 172.21.15.119:0/1796152861 wait complete. 2023-11-08T19:28:53.093 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:53.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.271+0000 7faa5911a640 1 Processor -- start 2023-11-08T19:28:53.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.272+0000 7faa5911a640 1 -- start start 2023-11-08T19:28:53.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa5911a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa540adf20 0x7faa5415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa5911a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faa540ad9d0 0x7faa5415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa5911a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa540ca2e0 con 0x7faa540cac30 2023-11-08T19:28:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa5911a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa54071c70 con 0x7faa540adf20 2023-11-08T19:28:53.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa5911a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5415b390 con 0x7faa540ad9d0 2023-11-08T19:28:53.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa4bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faa540adf20 0x7faa5415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa53fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7faa540cac30 0x7faa5415f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39648/0 (socket says 172.21.15.119:39648) 2023-11-08T19:28:53.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa53fff640 1 -- 172.21.15.119:0/1217936437 learned_addr learned my addr 172.21.15.119:0/1217936437 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:53.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.274+0000 7faa58919640 1 --2- 172.21.15.119:0/1217936437 >> v2:172.21.15.188:3300/0 conn(0x7faa540ad9d0 0x7faa5415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.273+0000 7faa537fe640 1 -- 172.21.15.119:0/1217936437 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005268854 0 0) 0x7faa540ca2e0 con 0x7faa540cac30 2023-11-08T19:28:53.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.274+0000 7faa4bfff640 1 -- 172.21.15.119:0/1217936437 >> v2:172.21.15.188:3300/0 conn(0x7faa540ad9d0 msgr2=0x7faa5415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:53.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.274+0000 7faa4bfff640 1 --2- 172.21.15.119:0/1217936437 >> v2:172.21.15.188:3300/0 conn(0x7faa540ad9d0 0x7faa5415ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.274+0000 7faa4bfff640 1 -- 172.21.15.119:0/1217936437 >> v1:172.21.15.119:6789/0 conn(0x7faa540cac30 legacy=0x7faa5415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.274+0000 7faa4bfff640 1 -- 172.21.15.119:0/1217936437 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540ca5c0 con 0x7faa540adf20 2023-11-08T19:28:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.275+0000 7faa4bfff640 1 --2- 172.21.15.119:0/1217936437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa5415a910 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7faa5406f570 tx=0x7faa3c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee91e7cc9fce6d76 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.275+0000 7faa537fe640 1 -- 172.21.15.119:0/1217936437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c031830 con 0x7faa540adf20 2023-11-08T19:28:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.275+0000 7faa537fe640 1 -- 172.21.15.119:0/1217936437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c031990 con 0x7faa540adf20 2023-11-08T19:28:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.275+0000 7faa537fe640 1 -- 172.21.15.119:0/1217936437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c031d20 con 0x7faa540adf20 2023-11-08T19:28:53.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 -- 172.21.15.119:0/1217936437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 msgr2=0x7faa5415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 --2- 172.21.15.119:0/1217936437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa5415a910 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7faa5406f570 tx=0x7faa3c02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 -- 172.21.15.119:0/1217936437 shutdown_connections 2023-11-08T19:28:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 --2- 172.21.15.119:0/1217936437 >> v2:172.21.15.188:3300/0 conn(0x7faa540ad9d0 0x7faa5415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 --2- 172.21.15.119:0/1217936437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa5415a910 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 -- 172.21.15.119:0/1217936437 >> 172.21.15.119:0/1217936437 conn(0x7faa540b0560 msgr2=0x7faa540b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 -- 172.21.15.119:0/1217936437 shutdown_connections 2023-11-08T19:28:53.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.276+0000 7faa5911a640 1 -- 172.21.15.119:0/1217936437 wait complete. 2023-11-08T19:28:53.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.277+0000 7faa5911a640 1 Processor -- start 2023-11-08T19:28:53.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.278+0000 7faa5911a640 1 -- start start 2023-11-08T19:28:53.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa5911a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa540ad9d0 0x7faa541565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa5911a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa54152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa5911a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa540cac30 0x7faa541539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa5911a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5415be40 con 0x7faa540adf20 2023-11-08T19:28:53.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa5911a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5415b580 con 0x7faa540cac30 2023-11-08T19:28:53.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa5911a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5415b7f0 con 0x7faa540ad9d0 2023-11-08T19:28:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa54152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa54152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35550/0 (socket says 172.21.15.119:35550) 2023-11-08T19:28:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 -- 172.21.15.119:0/2620519101 learned_addr learned my addr 172.21.15.119:0/2620519101 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 -- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa540ad9d0 msgr2=0x7faa541565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa53fff640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa540ad9d0 0x7faa541565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa58919640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa540cac30 0x7faa541539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa540ad9d0 0x7faa541565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 -- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa540cac30 msgr2=0x7faa541539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa540cac30 0x7faa541539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.279+0000 7faa4bfff640 1 -- 172.21.15.119:0/2620519101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c009400 con 0x7faa540adf20 2023-11-08T19:28:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.280+0000 7faa4bfff640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa54152250 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7faa3c02bee0 tx=0x7faa3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.280+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c004230 con 0x7faa540adf20 2023-11-08T19:28:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.280+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c004390 con 0x7faa540adf20 2023-11-08T19:28:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.280+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c03a940 con 0x7faa540adf20 2023-11-08T19:28:53.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.280+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa5415bad0 con 0x7faa540adf20 2023-11-08T19:28:53.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.280+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa54152ad0 con 0x7faa540adf20 2023-11-08T19:28:53.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.281+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa5415bff0 con 0x7faa540adf20 2023-11-08T19:28:53.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.282+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa3c03aaa0 con 0x7faa540adf20 2023-11-08T19:28:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.283+0000 7faa517fa640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa3406ae70 0x7faa3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.283+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7faa3c0ac380 con 0x7faa540adf20 2023-11-08T19:28:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.283+0000 7faa53fff640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa3406ae70 0x7faa3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.284+0000 7faa53fff640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa3406ae70 0x7faa3406d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faa44000f60 tx=0x7faa44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.287+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa5415bff0 con 0x7faa540adf20 2023-11-08T19:28:53.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.472+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa5415bff0 con 0x7faa540adf20 2023-11-08T19:28:53.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.475+0000 7faa517fa640 1 -- 172.21.15.119:0/2620519101 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+22018 (secure 0 0 0) 0x7faa5415bff0 con 0x7faa540adf20 2023-11-08T19:28:53.477 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:53.477 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:52.483531+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa3406ae70 msgr2=0x7faa3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa3406ae70 0x7faa3406d330 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faa44000f60 tx=0x7faa44009040 comp rx=0 tx=0).stop 2023-11-08T19:28:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 msgr2=0x7faa54152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa54152250 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7faa3c02bee0 tx=0x7faa3c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 shutdown_connections 2023-11-08T19:28:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faa3406ae70 0x7faa3406d330 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faa540cac30 0x7faa541539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faa540adf20 0x7faa54152250 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 --2- 172.21.15.119:0/2620519101 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faa540ad9d0 0x7faa541565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 >> 172.21.15.119:0/2620519101 conn(0x7faa540b0560 msgr2=0x7faa540cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 shutdown_connections 2023-11-08T19:28:53.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.477+0000 7faa5911a640 1 -- 172.21.15.119:0/2620519101 wait complete. 2023-11-08T19:28:53.496 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:53.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.702+0000 7fefb78f2640 1 Processor -- start 2023-11-08T19:28:53.704 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.702+0000 7fefb78f2640 1 -- start start 2023-11-08T19:28:53.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb78f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 0x7fefb0161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb78f2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fefb006efb0 0x7fefb0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb78f2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb00cf3e0 con 0x7fefb006a500 2023-11-08T19:28:53.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb78f2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb00cfbd0 con 0x7fefb006efb0 2023-11-08T19:28:53.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb78f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0161e40 con 0x7fefb006ebd0 2023-11-08T19:28:53.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb5e8f640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fefb006a500 0x7fefb006a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39650/0 (socket says 172.21.15.119:39650) 2023-11-08T19:28:53.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb5e8f640 1 -- 172.21.15.119:0/2395166223 learned_addr learned my addr 172.21.15.119:0/2395166223 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:53.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.704+0000 7fefb4e8d640 1 --2- 172.21.15.119:0/2395166223 >> v2:172.21.15.188:3300/0 conn(0x7fefb006efb0 0x7fefb0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.707 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefb568e640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 0x7fefb0161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040665692 0 0) 0x7fefb00cf3e0 con 0x7fefb006a500 2023-11-08T19:28:53.708 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa0003520 con 0x7fefb006a500 2023-11-08T19:28:53.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1754599505 0 0) 0x7fefa0003520 con 0x7fefb006a500 2023-11-08T19:28:53.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 msgr2=0x7fefb0161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:53.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 0x7fefb0161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 >> v2:172.21.15.188:3300/0 conn(0x7fefb006efb0 msgr2=0x7fefb0070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:53.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 --2- 172.21.15.119:0/2395166223 >> v2:172.21.15.188:3300/0 conn(0x7fefb006efb0 0x7fefb0070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb006b0a0 con 0x7fefb006a500 2023-11-08T19:28:53.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fefa8001670 con 0x7fefb006a500 2023-11-08T19:28:53.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 >> v1:172.21.15.119:6789/0 conn(0x7fefb006a500 legacy=0x7fefb006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefa0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa00047c0 0x7fefa0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 0x7fefa0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0161e40 con 0x7fefa0003970 2023-11-08T19:28:53.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa000b980 con 0x7fefa00047c0 2023-11-08T19:28:53.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa000bb10 con 0x7fefb006ebd0 2023-11-08T19:28:53.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.705+0000 7fefb568e640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefa0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb4e8d640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa00047c0 0x7fefa0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb568e640 1 -- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 msgr2=0x7fefa0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:53.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb568e640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb006ebd0 0x7fefa0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb568e640 1 -- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa00047c0 msgr2=0x7fefa0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:53.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb568e640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa00047c0 0x7fefa0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb568e640 1 -- 172.21.15.119:0/2395166223 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0003520 con 0x7fefa0003970 2023-11-08T19:28:53.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.706+0000 7fefb4e8d640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefa00047c0 0x7fefa0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:53.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.707+0000 7fefb568e640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefa0003d90 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e170 tx=0x7fefa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b64a6d77c78ad4dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.707+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa402cc20 con 0x7fefa0003970 2023-11-08T19:28:53.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.707+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa400f040 con 0x7fefa0003970 2023-11-08T19:28:53.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.707+0000 7fefaffff640 1 -- 172.21.15.119:0/2395166223 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4034440 con 0x7fefa0003970 2023-11-08T19:28:53.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb78f2640 1 -- 172.21.15.119:0/2395166223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 msgr2=0x7fefa0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb78f2640 1 --2- 172.21.15.119:0/2395166223 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefa0003d90 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fefa400e170 tx=0x7fefa00057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb5e8f640 1 -- 172.21.15.119:0/2395166223 reap_dead start 2023-11-08T19:28:53.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb78f2640 1 -- 172.21.15.119:0/2395166223 shutdown_connections 2023-11-08T19:28:53.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb78f2640 1 -- 172.21.15.119:0/2395166223 >> 172.21.15.119:0/2395166223 conn(0x7fefb00cfd20 msgr2=0x7fefb00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:53.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb78f2640 1 -- 172.21.15.119:0/2395166223 shutdown_connections 2023-11-08T19:28:53.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.708+0000 7fefb78f2640 1 -- 172.21.15.119:0/2395166223 wait complete. 2023-11-08T19:28:53.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.710+0000 7fefb78f2640 1 Processor -- start 2023-11-08T19:28:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.710+0000 7fefb78f2640 1 -- start start 2023-11-08T19:28:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb78f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefb015de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb78f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefb014d520 0x7fefb014fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb78f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb015f2b0 0x7fefb015f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb78f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb015fca0 con 0x7fefa0003970 2023-11-08T19:28:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb78f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb014dd90 con 0x7fefb014d520 2023-11-08T19:28:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb78f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb0150600 con 0x7fefb015f2b0 2023-11-08T19:28:53.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefb015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefb015de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35568/0 (socket says 172.21.15.119:35568) 2023-11-08T19:28:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 -- 172.21.15.119:0/133361517 learned_addr learned my addr 172.21.15.119:0/133361517 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:53.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 -- 172.21.15.119:0/133361517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb015f2b0 msgr2=0x7fefb015f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb015f2b0 0x7fefb015f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 -- 172.21.15.119:0/133361517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefb014d520 msgr2=0x7fefb014fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefb014d520 0x7fefb014fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.711+0000 7fefb568e640 1 -- 172.21.15.119:0/133361517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb01508e0 con 0x7fefa0003970 2023-11-08T19:28:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.712+0000 7fefb568e640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefb015de60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fefa4037040 tx=0x7fefa4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.712+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4034d50 con 0x7fefa0003970 2023-11-08T19:28:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.712+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4040450 con 0x7fefa0003970 2023-11-08T19:28:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.712+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa40407e0 con 0x7fefa0003970 2023-11-08T19:28:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.712+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb0150bc0 con 0x7fefa0003970 2023-11-08T19:28:53.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.712+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb0150300 con 0x7fefa0003970 2023-11-08T19:28:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.713+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb0150d50 con 0x7fefa0003970 2023-11-08T19:28:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.714+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fefa4040940 con 0x7fefa0003970 2023-11-08T19:28:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.715+0000 7fefadffb640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef9406ae70 0x7fef9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.715+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fefa40ac4e0 con 0x7fefa0003970 2023-11-08T19:28:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.715+0000 7fefb4e8d640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef9406ae70 0x7fef9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:53.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.716+0000 7fefb4e8d640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef9406ae70 0x7fef9406d330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefb01f6600 tx=0x7fef9c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:53.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.720+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefa4077010 con 0x7fefa0003970 2023-11-08T19:28:53.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.909+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb0074430 con 0x7fefa0003970 2023-11-08T19:28:53.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.914+0000 7fefadffb640 1 -- 172.21.15.119:0/133361517 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+22018 (secure 0 0 0) 0x7fefa407a630 con 0x7fefa0003970 2023-11-08T19:28:53.916 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:53.916 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:52.483531+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:53.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef9406ae70 msgr2=0x7fef9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef9406ae70 0x7fef9406d330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fefb01f6600 tx=0x7fef9c008040 comp rx=0 tx=0).stop 2023-11-08T19:28:53.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 msgr2=0x7fefb015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:53.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefb015de60 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fefa4037040 tx=0x7fefa4009750 comp rx=0 tx=0).stop 2023-11-08T19:28:53.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 shutdown_connections 2023-11-08T19:28:53.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef9406ae70 0x7fef9406d330 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fefb015f2b0 0x7fefb015f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fefb014d520 0x7fefb014fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 --2- 172.21.15.119:0/133361517 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fefa0003970 0x7fefb015de60 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:53.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.916+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 >> 172.21.15.119:0/133361517 conn(0x7fefb00cfd20 msgr2=0x7fefb00ca680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:53.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.917+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 shutdown_connections 2023-11-08T19:28:53.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:53.917+0000 7fefb78f2640 1 -- 172.21.15.119:0/133361517 wait complete. 2023-11-08T19:28:53.938 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T19:28:54.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.148+0000 7f61c9dcb640 1 Processor -- start 2023-11-08T19:28:54.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.149+0000 7f61c9dcb640 1 -- start start 2023-11-08T19:28:54.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.150+0000 7f61c9dcb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 0x7f61c415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.150+0000 7f61c9dcb640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c406cdd0 con 0x7f61c40c9a30 2023-11-08T19:28:54.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.150+0000 7f61c9dcb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c406c530 con 0x7f61c40acf20 2023-11-08T19:28:54.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.150+0000 7f61c9dcb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415def0 con 0x7f61c40c9650 2023-11-08T19:28:54.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.150+0000 7f61c3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f61c40c9a30 0x7f61c406ba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39656/0 (socket says 172.21.15.119:39656) 2023-11-08T19:28:54.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.150+0000 7f61c3fff640 1 -- 172.21.15.119:0/3645238485 learned_addr learned my addr 172.21.15.119:0/3645238485 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:54.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 0x7f61c415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339365073 0 0) 0x7f61c406cdd0 con 0x7f61c40c9a30 2023-11-08T19:28:54.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b4003520 con 0x7f61c40c9a30 2023-11-08T19:28:54.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1547890264 0 0) 0x7f61b4003520 con 0x7f61c40c9a30 2023-11-08T19:28:54.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 msgr2=0x7f61c415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:54.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 0x7f61c415d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 >> v1:172.21.15.188:6789/0 conn(0x7f61c40acf20 legacy=0x7f61c40ad3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c406c810 con 0x7f61c40c9a30 2023-11-08T19:28:54.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f61b0001670 con 0x7f61c40c9a30 2023-11-08T19:28:54.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 >> v1:172.21.15.119:6789/0 conn(0x7f61c40c9a30 legacy=0x7f61c406ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61b40039a0 0x7f61b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 0x7f61b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b4004d00 0x7f61b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c415def0 con 0x7f61b40039a0 2023-11-08T19:28:54.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b400bab0 con 0x7f61b4004d00 2023-11-08T19:28:54.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.151+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b400bc70 con 0x7f61c40c9650 2023-11-08T19:28:54.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.152+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61b40039a0 0x7f61b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.152+0000 7f61c8dc9640 1 -- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 msgr2=0x7f61b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:54.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.152+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c40c9650 0x7f61b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.152+0000 7f61c8dc9640 1 -- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b4004d00 msgr2=0x7f61b4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:54.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.152+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b4004d00 0x7f61b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.152+0000 7f61c8dc9640 1 -- 172.21.15.119:0/3645238485 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b4003520 con 0x7f61b40039a0 2023-11-08T19:28:54.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.153+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61b40039a0 0x7f61b4003dc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009ac0 tx=0x7f61b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea562335d0f0e10 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:54.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.153+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b802c630 con 0x7f61b40039a0 2023-11-08T19:28:54.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.153+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b802c790 con 0x7f61b40039a0 2023-11-08T19:28:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.153+0000 7f61c37fe640 1 -- 172.21.15.119:0/3645238485 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b802cb20 con 0x7f61b40039a0 2023-11-08T19:28:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3645238485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61b40039a0 msgr2=0x7f61b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3645238485 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61b40039a0 0x7f61b4003dc0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f61b8009ac0 tx=0x7f61b40057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c95ca640 1 -- 172.21.15.119:0/3645238485 reap_dead start 2023-11-08T19:28:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3645238485 shutdown_connections 2023-11-08T19:28:54.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3645238485 >> 172.21.15.119:0/3645238485 conn(0x7f61c40ceac0 msgr2=0x7f61c40ceed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3645238485 shutdown_connections 2023-11-08T19:28:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.154+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3645238485 wait complete. 2023-11-08T19:28:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.156+0000 7f61c9dcb640 1 Processor -- start 2023-11-08T19:28:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.156+0000 7f61c9dcb640 1 -- start start 2023-11-08T19:28:54.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c9dcb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b40039a0 0x7f61c41e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c9dcb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c41e6f40 0x7f61c40c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c9dcb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 0x7f61c415ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c9dcb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40c6810 con 0x7f61c40c7170 2023-11-08T19:28:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c9dcb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c41e93b0 con 0x7f61b40039a0 2023-11-08T19:28:54.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c9dcb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40c5b90 con 0x7f61c41e6f40 2023-11-08T19:28:54.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c95ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 0x7f61c415ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c95ca640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 0x7f61c415ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35580/0 (socket says 172.21.15.119:35580) 2023-11-08T19:28:54.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.157+0000 7f61c95ca640 1 -- 172.21.15.119:0/3060055011 learned_addr learned my addr 172.21.15.119:0/3060055011 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:54.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b40039a0 0x7f61c41e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c3fff640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c41e6f40 0x7f61c40c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c95ca640 1 -- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c41e6f40 msgr2=0x7f61c40c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c95ca640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c41e6f40 0x7f61c40c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c95ca640 1 -- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b40039a0 msgr2=0x7f61c41e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:54.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c95ca640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b40039a0 0x7f61c41e8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:54.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c95ca640 1 -- 172.21.15.119:0/3060055011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40c5e70 con 0x7f61c40c7170 2023-11-08T19:28:54.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c95ca640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 0x7f61c415ec40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f61c404e630 tx=0x7f61bc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:54.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc005d40 con 0x7f61c40c7170 2023-11-08T19:28:54.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc00ced0 con 0x7f61c40c7170 2023-11-08T19:28:54.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00a850 con 0x7f61c40c7170 2023-11-08T19:28:54.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.158+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c40c6150 con 0x7f61c40c7170 2023-11-08T19:28:54.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.159+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c41e73a0 con 0x7f61c40c7170 2023-11-08T19:28:54.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.160+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c404ece0 con 0x7f61c40c7170 2023-11-08T19:28:54.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.161+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61bc00a9b0 con 0x7f61c40c7170 2023-11-08T19:28:54.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.161+0000 7f61c17fa640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f61a806ae70 0x7f61a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:54.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.162+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f61bc088b40 con 0x7f61c40c7170 2023-11-08T19:28:54.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.162+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f61a806ae70 0x7f61a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:54.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.162+0000 7f61c8dc9640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f61a806ae70 0x7f61a806d330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f61b8036000 tx=0x7f61b8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:54.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.166+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61bc0535e0 con 0x7f61c40c7170 2023-11-08T19:28:54.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:54.398+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f61c406d390 con 0x7f61c40c7170 2023-11-08T19:28:55.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.060+0000 7f61c17fa640 1 -- 172.21.15.119:0/3060055011 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7f61bc056c00 con 0x7f61c40c7170 2023-11-08T19:28:55.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.063+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f61a806ae70 msgr2=0x7f61a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.063+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f61a806ae70 0x7f61a806d330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f61b8036000 tx=0x7f61b8030000 comp rx=0 tx=0).stop 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 msgr2=0x7f61c415ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 0x7f61c415ec40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f61c404e630 tx=0x7f61bc00cb90 comp rx=0 tx=0).stop 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 shutdown_connections 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f61a806ae70 0x7f61a806d330 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f61c40c7170 0x7f61c415ec40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61c41e6f40 0x7f61c40c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 --2- 172.21.15.119:0/3060055011 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b40039a0 0x7f61c41e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 >> 172.21.15.119:0/3060055011 conn(0x7f61c40ceac0 msgr2=0x7f61c4160fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 shutdown_connections 2023-11-08T19:28:55.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.064+0000 7f61c9dcb640 1 -- 172.21.15.119:0/3060055011 wait complete. 2023-11-08T19:28:55.087 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T19:28:55.087 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T19:28:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.301+0000 7fd1f2747640 1 Processor -- start 2023-11-08T19:28:55.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.302+0000 7fd1f2747640 1 -- start start 2023-11-08T19:28:55.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.303+0000 7fd1f2747640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1ec160530 0x7fd1ec0cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.303+0000 7fd1f2747640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1ec0ce5b0 0x7fd1ec0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.303+0000 7fd1f2747640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ec0d0830 con 0x7fd1ec160910 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.303+0000 7fd1f2747640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b27c0 con 0x7fd1ec0ce5b0 2023-11-08T19:28:55.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.303+0000 7fd1f2747640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b2aa0 con 0x7fd1ec160530 2023-11-08T19:28:55.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1ec0ce5b0 0x7fd1ec0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1745640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd1ec160530 0x7fd1ec0cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd1ec0ce5b0 0x7fd1ec0b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35584/0 (socket says 172.21.15.119:35584) 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 -- 172.21.15.119:0/3940831352 learned_addr learned my addr 172.21.15.119:0/3940831352 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 -- 172.21.15.119:0/3940831352 >> v2:172.21.15.188:3300/0 conn(0x7fd1ec160530 msgr2=0x7fd1ec0cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 --2- 172.21.15.119:0/3940831352 >> v2:172.21.15.188:3300/0 conn(0x7fd1ec160530 0x7fd1ec0cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 -- 172.21.15.119:0/3940831352 >> v1:172.21.15.188:6789/0 conn(0x7fd1ec160910 legacy=0x7fd1ec0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.304+0000 7fd1f1f46640 1 -- 172.21.15.119:0/3940831352 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec0d1640 con 0x7fd1ec0ce5b0 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.305+0000 7fd1f1f46640 1 --2- 172.21.15.119:0/3940831352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec0b2f30 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec06a7d0 tx=0x7fd1e402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b687c01c3aaf779 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.306+0000 7fd1ebfff640 1 -- 172.21.15.119:0/3940831352 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4030860 con 0x7fd1ec0ce5b0 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.306+0000 7fd1ebfff640 1 -- 172.21.15.119:0/3940831352 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1e40309c0 con 0x7fd1ec0ce5b0 2023-11-08T19:28:55.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.306+0000 7fd1ebfff640 1 -- 172.21.15.119:0/3940831352 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1e4030d50 con 0x7fd1ec0ce5b0 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.306+0000 7fd1f2747640 1 -- 172.21.15.119:0/3940831352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec0ce5b0 msgr2=0x7fd1ec0b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.306+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3940831352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec0b2f30 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec06a7d0 tx=0x7fd1e402b9a0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.307+0000 7fd1f2747640 1 -- 172.21.15.119:0/3940831352 shutdown_connections 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.307+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3940831352 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec0b2f30 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.307+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3940831352 >> v2:172.21.15.188:3300/0 conn(0x7fd1ec160530 0x7fd1ec0cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.307+0000 7fd1f2747640 1 -- 172.21.15.119:0/3940831352 >> 172.21.15.119:0/3940831352 conn(0x7fd1ec0d2ec0 msgr2=0x7fd1ec0d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.307+0000 7fd1f2747640 1 -- 172.21.15.119:0/3940831352 shutdown_connections 2023-11-08T19:28:55.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.307+0000 7fd1f2747640 1 -- 172.21.15.119:0/3940831352 wait complete. 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.308+0000 7fd1f2747640 1 Processor -- start 2023-11-08T19:28:55.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 -- start start 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec1f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 0x7fd1ec0b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec160910 0x7fd1ec0caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b73e0 con 0x7fd1ec160530 2023-11-08T19:28:55.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0b6a00 con 0x7fd1ec0ce5b0 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.309+0000 7fd1f2747640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec0cb470 con 0x7fd1ec160910 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 0x7fd1ec0b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 0x7fd1ec0b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35596/0 (socket says 172.21.15.119:35596) 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 -- 172.21.15.119:0/3265882940 learned_addr learned my addr 172.21.15.119:0/3265882940 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:55.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f1745640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec1f2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 -- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec160910 msgr2=0x7fd1ec0caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec160910 0x7fd1ec0caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 -- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0ce5b0 msgr2=0x7fd1ec1f2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec1f2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.310+0000 7fd1f0f44640 1 -- 172.21.15.119:0/3265882940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40094d0 con 0x7fd1ec160530 2023-11-08T19:28:55.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.311+0000 7fd1f0f44640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 0x7fd1ec0b64c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800c400 tx=0x7fd1d800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.311+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80052d0 con 0x7fd1ec160530 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.311+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d8005430 con 0x7fd1ec160530 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.311+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec1f2eb0 con 0x7fd1ec160530 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.311+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8003830 con 0x7fd1ec160530 2023-11-08T19:28:55.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.311+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec1f3470 con 0x7fd1ec160530 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.312+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec0b7550 con 0x7fd1ec160530 2023-11-08T19:28:55.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.313+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1d8003990 con 0x7fd1ec160530 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.314+0000 7fd1e9ffb640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1d006ae70 0x7fd1d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.314+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fd1d8089190 con 0x7fd1ec160530 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.314+0000 7fd1f1745640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1d006ae70 0x7fd1d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.315+0000 7fd1f1745640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1d006ae70 0x7fd1d006d330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0cefd0 tx=0x7fd1e000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:55.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.318+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1d8053c40 con 0x7fd1ec160530 2023-11-08T19:28:55.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:55.545+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd1ec04ece0 con 0x7fd1ec160530 2023-11-08T19:28:56.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.064+0000 7fd1e9ffb640 1 -- 172.21.15.119:0/3265882940 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd1d8057260 con 0x7fd1ec160530 2023-11-08T19:28:56.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1d006ae70 msgr2=0x7fd1d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1d006ae70 0x7fd1d006d330 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec0cefd0 tx=0x7fd1e000b040 comp rx=0 tx=0).stop 2023-11-08T19:28:56.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 msgr2=0x7fd1ec0b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 0x7fd1ec0b64c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd1d800c400 tx=0x7fd1d800c8c0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 shutdown_connections 2023-11-08T19:28:56.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd1d006ae70 0x7fd1d006d330 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd1ec160910 0x7fd1ec0caf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd1ec160530 0x7fd1ec0b64c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 --2- 172.21.15.119:0/3265882940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd1ec0ce5b0 0x7fd1ec1f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.067+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 >> 172.21.15.119:0/3265882940 conn(0x7fd1ec0d2ec0 msgr2=0x7fd1ec15f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:56.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.068+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 shutdown_connections 2023-11-08T19:28:56.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.068+0000 7fd1f2747640 1 -- 172.21.15.119:0/3265882940 wait complete. 2023-11-08T19:28:56.089 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T19:28:56.089 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:28:56.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.299+0000 7f5c96eda640 1 Processor -- start 2023-11-08T19:28:56.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.299+0000 7f5c96eda640 1 -- start start 2023-11-08T19:28:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.300+0000 7f5c96eda640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c90142f80 0x7f5c900cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.300+0000 7f5c96eda640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c900c4960 con 0x7f5c9006cb40 2023-11-08T19:28:56.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c96eda640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9015df60 con 0x7f5c9015d9f0 2023-11-08T19:28:56.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c96eda640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c90143a40 con 0x7f5c90142f80 2023-11-08T19:28:56.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c94c76640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5c90142f80 0x7f5c900cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c95477640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f5c9006cb40 0x7f5c900c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39658/0 (socket says 172.21.15.119:39658) 2023-11-08T19:28:56.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c95477640 1 -- 172.21.15.119:0/4159360744 learned_addr learned my addr 172.21.15.119:0/4159360744 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:56.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c94c76640 1 -- 172.21.15.119:0/4159360744 >> v1:172.21.15.188:6789/0 conn(0x7f5c9015d9f0 legacy=0x7f5c900c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:56.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c94c76640 1 -- 172.21.15.119:0/4159360744 >> v1:172.21.15.119:6789/0 conn(0x7f5c9006cb40 legacy=0x7f5c900c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.301+0000 7f5c94c76640 1 -- 172.21.15.119:0/4159360744 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9015e240 con 0x7f5c90142f80 2023-11-08T19:28:56.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.302+0000 7f5c94c76640 1 --2- 172.21.15.119:0/4159360744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90142f80 0x7f5c900cc6d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5c840097a0 tx=0x7f5c8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce4f3155a9266753 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/4159360744 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84031830 con 0x7f5c90142f80 2023-11-08T19:28:56.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/4159360744 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c84031990 con 0x7f5c90142f80 2023-11-08T19:28:56.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c8f7fe640 1 -- 172.21.15.119:0/4159360744 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84031d20 con 0x7f5c90142f80 2023-11-08T19:28:56.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c96eda640 1 -- 172.21.15.119:0/4159360744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90142f80 msgr2=0x7f5c900cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c96eda640 1 --2- 172.21.15.119:0/4159360744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90142f80 0x7f5c900cc6d0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5c840097a0 tx=0x7f5c8402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c96eda640 1 -- 172.21.15.119:0/4159360744 shutdown_connections 2023-11-08T19:28:56.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c96eda640 1 --2- 172.21.15.119:0/4159360744 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c90142f80 0x7f5c900cc6d0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c96eda640 1 -- 172.21.15.119:0/4159360744 >> 172.21.15.119:0/4159360744 conn(0x7f5c900cb610 msgr2=0x7f5c900cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:56.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.303+0000 7f5c96eda640 1 -- 172.21.15.119:0/4159360744 shutdown_connections 2023-11-08T19:28:56.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.304+0000 7f5c96eda640 1 -- 172.21.15.119:0/4159360744 wait complete. 2023-11-08T19:28:56.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.305+0000 7f5c96eda640 1 Processor -- start 2023-11-08T19:28:56.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.305+0000 7f5c96eda640 1 -- start start 2023-11-08T19:28:56.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.306+0000 7f5c96eda640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c9006cb40 0x7f5c901511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.306+0000 7f5c96eda640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c90142f80 0x7f5c900c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.306+0000 7f5c96eda640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 0x7f5c9015c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.306+0000 7f5c96eda640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900c8110 con 0x7f5c9015d9f0 2023-11-08T19:28:56.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.306+0000 7f5c96eda640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900c72a0 con 0x7f5c90142f80 2023-11-08T19:28:56.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.306+0000 7f5c96eda640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900c7500 con 0x7f5c9006cb40 2023-11-08T19:28:56.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 0x7f5c9015c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 0x7f5c9015c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35618/0 (socket says 172.21.15.119:35618) 2023-11-08T19:28:56.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 -- 172.21.15.119:0/2256254862 learned_addr learned my addr 172.21.15.119:0/2256254862 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:56.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 -- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c9006cb40 msgr2=0x7f5c901511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c9006cb40 0x7f5c901511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 -- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c90142f80 msgr2=0x7f5c900c6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c8ffff640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c90142f80 0x7f5c900c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c90142f80 0x7f5c900c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 -- 172.21.15.119:0/2256254862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c84009400 con 0x7f5c9015d9f0 2023-11-08T19:28:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.307+0000 7f5c95477640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 0x7f5c9015c3e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800cbc0 tx=0x7f5c8800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.308+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800ad40 con 0x7f5c9015d9f0 2023-11-08T19:28:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.308+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c88012400 con 0x7f5c9015d9f0 2023-11-08T19:28:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.308+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8800aea0 con 0x7f5c9015d9f0 2023-11-08T19:28:56.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.308+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c900c77e0 con 0x7f5c9015d9f0 2023-11-08T19:28:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.308+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c900c7da0 con 0x7f5c9015d9f0 2023-11-08T19:28:56.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.309+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c90074430 con 0x7f5c9015d9f0 2023-11-08T19:28:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.310+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c88007070 con 0x7f5c9015d9f0 2023-11-08T19:28:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.311+0000 7f5c8d7fa640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c7406ae70 0x7f5c7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.311+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f5c88089a20 con 0x7f5c9015d9f0 2023-11-08T19:28:56.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.311+0000 7f5c94c76640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c7406ae70 0x7f5c7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.312+0000 7f5c94c76640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c7406ae70 0x7f5c7406d330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5c84036000 tx=0x7f5c84030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.315+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c88054580 con 0x7f5c9015d9f0 2023-11-08T19:28:56.417 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:28:56.416+0000 7f5f9fcb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:28:56.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.543+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c900c8250 con 0x7f5c9015d9f0 2023-11-08T19:28:56.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.545+0000 7f5c8d7fa640 1 -- 172.21.15.119:0/2256254862 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7f5c88017090 con 0x7f5c9015d9f0 2023-11-08T19:28:56.546 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:56.547 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:28:55.547548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:28:56.547 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 53 2023-11-08T19:28:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c7406ae70 msgr2=0x7f5c7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c7406ae70 0x7f5c7406d330 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5c84036000 tx=0x7f5c84030000 comp rx=0 tx=0).stop 2023-11-08T19:28:56.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 msgr2=0x7f5c9015c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 0x7f5c9015c3e0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800cbc0 tx=0x7f5c8800a430 comp rx=0 tx=0).stop 2023-11-08T19:28:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 shutdown_connections 2023-11-08T19:28:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5c7406ae70 0x7f5c7406d330 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5c9015d9f0 0x7f5c9015c3e0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c90142f80 0x7f5c900c6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 --2- 172.21.15.119:0/2256254862 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c9006cb40 0x7f5c901511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 >> 172.21.15.119:0/2256254862 conn(0x7f5c900cb610 msgr2=0x7f5c901424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.547+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 shutdown_connections 2023-11-08T19:28:56.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.548+0000 7f5c96eda640 1 -- 172.21.15.119:0/2256254862 wait complete. 2023-11-08T19:28:56.566 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:28:56.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.785+0000 7fb8956d1640 1 Processor -- start 2023-11-08T19:28:56.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.785+0000 7fb8956d1640 1 -- start start 2023-11-08T19:28:56.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb8956d1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8900adf20 0x7fb89015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb8956d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fb8900ad9d0 0x7fb89015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb8956d1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8900ca2e0 con 0x7fb8900cac30 2023-11-08T19:28:56.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb8956d1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb890071c70 con 0x7fb8900ad9d0 2023-11-08T19:28:56.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb8956d1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89015b390 con 0x7fb8900adf20 2023-11-08T19:28:56.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb88ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb8900cac30 0x7fb89015f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39660/0 (socket says 172.21.15.119:39660) 2023-11-08T19:28:56.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb88ffff640 1 -- 172.21.15.119:0/4207432642 learned_addr learned my addr 172.21.15.119:0/4207432642 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:56.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb894ed0640 1 --2- 172.21.15.119:0/4207432642 >> v2:172.21.15.119:3300/0 conn(0x7fb8900ad9d0 0x7fb89015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb88f7fe640 1 --2- 172.21.15.119:0/4207432642 >> v2:172.21.15.188:3300/0 conn(0x7fb8900adf20 0x7fb89015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.787+0000 7fb894ed0640 1 -- 172.21.15.119:0/4207432642 >> v2:172.21.15.188:3300/0 conn(0x7fb8900adf20 msgr2=0x7fb89015a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:56.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.788+0000 7fb894ed0640 1 --2- 172.21.15.119:0/4207432642 >> v2:172.21.15.188:3300/0 conn(0x7fb8900adf20 0x7fb89015a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.788+0000 7fb894ed0640 1 -- 172.21.15.119:0/4207432642 >> v1:172.21.15.119:6789/0 conn(0x7fb8900cac30 legacy=0x7fb89015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.788+0000 7fb88effd640 1 -- 172.21.15.119:0/4207432642 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786250037 0 0) 0x7fb8900ca2e0 con 0x7fb8900cac30 2023-11-08T19:28:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.788+0000 7fb894ed0640 1 -- 172.21.15.119:0/4207432642 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900ca5c0 con 0x7fb8900ad9d0 2023-11-08T19:28:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.788+0000 7fb88f7fe640 1 --2- 172.21.15.119:0/4207432642 >> v2:172.21.15.188:3300/0 conn(0x7fb8900adf20 0x7fb89015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:56.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.789+0000 7fb894ed0640 1 --2- 172.21.15.119:0/4207432642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb89015ae50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb8900ce940 tx=0x7fb88802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68ab0f90ca6b24e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.789+0000 7fb88effd640 1 -- 172.21.15.119:0/4207432642 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88802cc80 con 0x7fb8900ad9d0 2023-11-08T19:28:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.789+0000 7fb88effd640 1 -- 172.21.15.119:0/4207432642 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88802cde0 con 0x7fb8900ad9d0 2023-11-08T19:28:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.789+0000 7fb88effd640 1 -- 172.21.15.119:0/4207432642 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb888034660 con 0x7fb8900ad9d0 2023-11-08T19:28:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 -- 172.21.15.119:0/4207432642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 msgr2=0x7fb89015ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 --2- 172.21.15.119:0/4207432642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb89015ae50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fb8900ce940 tx=0x7fb88802bed0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 -- 172.21.15.119:0/4207432642 shutdown_connections 2023-11-08T19:28:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 --2- 172.21.15.119:0/4207432642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb89015ae50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 --2- 172.21.15.119:0/4207432642 >> v2:172.21.15.188:3300/0 conn(0x7fb8900adf20 0x7fb89015a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 -- 172.21.15.119:0/4207432642 >> 172.21.15.119:0/4207432642 conn(0x7fb8900b0560 msgr2=0x7fb8900b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:56.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 -- 172.21.15.119:0/4207432642 shutdown_connections 2023-11-08T19:28:56.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.790+0000 7fb8956d1640 1 -- 172.21.15.119:0/4207432642 wait complete. 2023-11-08T19:28:56.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.792+0000 7fb8956d1640 1 Processor -- start 2023-11-08T19:28:56.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.792+0000 7fb8956d1640 1 -- start start 2023-11-08T19:28:56.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.793+0000 7fb8956d1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb8900cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.793+0000 7fb8956d1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8900adf20 0x7fb8900acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.793+0000 7fb8956d1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900cac30 0x7fb8901f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.793+0000 7fb8956d1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8900cd3b0 con 0x7fb8900ad9d0 2023-11-08T19:28:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.793+0000 7fb8956d1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015c9a0 con 0x7fb8900cac30 2023-11-08T19:28:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.793+0000 7fb8956d1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89015cc80 con 0x7fb8900adf20 2023-11-08T19:28:56.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb8900cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb8900cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35628/0 (socket says 172.21.15.119:35628) 2023-11-08T19:28:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 -- 172.21.15.119:0/2158319907 learned_addr learned my addr 172.21.15.119:0/2158319907 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb894ed0640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900cac30 0x7fb8901f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 -- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8900adf20 msgr2=0x7fb8900acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88f7fe640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8900adf20 0x7fb8900acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8900adf20 0x7fb8900acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 -- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900cac30 msgr2=0x7fb8901f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900cac30 0x7fb8901f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.794+0000 7fb88ffff640 1 -- 172.21.15.119:0/2158319907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb888009a70 con 0x7fb8900ad9d0 2023-11-08T19:28:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.795+0000 7fb88ffff640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb8900cd5f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c430 tx=0x7fb88000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.795+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800074e0 con 0x7fb8900ad9d0 2023-11-08T19:28:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.795+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880007660 con 0x7fb8900ad9d0 2023-11-08T19:28:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.795+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89015cf60 con 0x7fb8900ad9d0 2023-11-08T19:28:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.795+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880004a20 con 0x7fb8900ad9d0 2023-11-08T19:28:56.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.795+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89015d520 con 0x7fb8900ad9d0 2023-11-08T19:28:56.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.796+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89015d740 con 0x7fb8900ad9d0 2023-11-08T19:28:56.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.797+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb880004b80 con 0x7fb8900ad9d0 2023-11-08T19:28:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.798+0000 7fb88cff9640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb87c06ae70 0x7fb87c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.798+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fb880089a40 con 0x7fb8900ad9d0 2023-11-08T19:28:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.798+0000 7fb88f7fe640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb87c06ae70 0x7fb87c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:56.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.799+0000 7fb88f7fe640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb87c06ae70 0x7fb87c06d330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb8900c5af0 tx=0x7fb884008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:56.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.802+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8800544f0 con 0x7fb8900ad9d0 2023-11-08T19:28:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.992+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb89004ece0 con 0x7fb8900ad9d0 2023-11-08T19:28:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.994+0000 7fb88cff9640 1 -- 172.21.15.119:0/2158319907 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+22005 (secure 0 0 0) 0x7fb880057b10 con 0x7fb8900ad9d0 2023-11-08T19:28:56.997 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:56.997 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":96,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:28:54.046302+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:28:56.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb87c06ae70 msgr2=0x7fb87c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb87c06ae70 0x7fb87c06d330 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb8900c5af0 tx=0x7fb884008040 comp rx=0 tx=0).stop 2023-11-08T19:28:57.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 msgr2=0x7fb8900cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb8900cd5f0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c430 tx=0x7fb88000c8f0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 shutdown_connections 2023-11-08T19:28:57.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb87c06ae70 0x7fb87c06d330 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8900cac30 0x7fb8901f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8900adf20 0x7fb8900acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 --2- 172.21.15.119:0/2158319907 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8900ad9d0 0x7fb8900cd5f0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 >> 172.21.15.119:0/2158319907 conn(0x7fb8900b0560 msgr2=0x7fb8900b1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:57.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 shutdown_connections 2023-11-08T19:28:57.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:56.997+0000 7fb8956d1640 1 -- 172.21.15.119:0/2158319907 wait complete. 2023-11-08T19:28:57.019 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T19:28:57.020 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.233+0000 7f4a5bfff640 1 Processor -- start 2023-11-08T19:28:57.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.233+0000 7f4a5bfff640 1 -- start start 2023-11-08T19:28:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.235+0000 7f4a5bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a5c0ad7d0 0x7f4a5c070b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.235+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.235+0000 7f4a5bfff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a5c06cff0 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.235+0000 7f4a5bfff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a5c06d2d0 con 0x7f4a5c0ad7d0 2023-11-08T19:28:57.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.235+0000 7f4a5bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c06bd20 con 0x7f4a5c0710a0 2023-11-08T19:28:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a5c0ad7d0 0x7f4a5c070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a5c0ad7d0 0x7f4a5c070b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:50626/0 (socket says 172.21.15.119:50626) 2023-11-08T19:28:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 -- 172.21.15.119:0/210364260 learned_addr learned my addr 172.21.15.119:0/210364260 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a5b7fe640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 -- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 msgr2=0x7f4a5c0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 -- 172.21.15.119:0/210364260 >> v1:172.21.15.188:6789/0 conn(0x7f4a5c15b0d0 legacy=0x7f4a5c15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a527fc640 1 -- 172.21.15.119:0/210364260 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c06d760 con 0x7f4a5c0ad7d0 2023-11-08T19:28:57.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.236+0000 7f4a5b7fe640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:28:57.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.238+0000 7f4a527fc640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c070b60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400bb10 tx=0x7f4a4402e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=635c9c529cc51583 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.238+0000 7f4a5a7fc640 1 -- 172.21.15.119:0/210364260 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4402ee70 con 0x7f4a5c0ad7d0 2023-11-08T19:28:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.238+0000 7f4a5a7fc640 1 -- 172.21.15.119:0/210364260 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a44036440 con 0x7f4a5c0ad7d0 2023-11-08T19:28:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.238+0000 7f4a5a7fc640 1 -- 172.21.15.119:0/210364260 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440367d0 con 0x7f4a5c0ad7d0 2023-11-08T19:28:57.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 -- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 msgr2=0x7f4a5c070b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c070b60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400bb10 tx=0x7f4a4402e0c0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 -- 172.21.15.119:0/210364260 shutdown_connections 2023-11-08T19:28:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c070b60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/210364260 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 -- 172.21.15.119:0/210364260 >> 172.21.15.119:0/210364260 conn(0x7f4a5c0b0560 msgr2=0x7f4a5c0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:57.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 -- 172.21.15.119:0/210364260 shutdown_connections 2023-11-08T19:28:57.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.239+0000 7f4a5bfff640 1 -- 172.21.15.119:0/210364260 wait complete. 2023-11-08T19:28:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.241+0000 7f4a5bfff640 1 Processor -- start 2023-11-08T19:28:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.241+0000 7f4a5bfff640 1 -- start start 2023-11-08T19:28:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c1511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 0x7f4a5c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5bfff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c1f3980 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5bfff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c1f2fe0 con 0x7f4a5c0ad7d0 2023-11-08T19:28:57.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5c1f32c0 con 0x7f4a5c0710a0 2023-11-08T19:28:57.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 0x7f4a5c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 0x7f4a5c0af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35638/0 (socket says 172.21.15.119:35638) 2023-11-08T19:28:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.242+0000 7f4a5b7fe640 1 -- 172.21.15.119:0/3506305117 learned_addr learned my addr 172.21.15.119:0/3506305117 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a527fc640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c1511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5affd640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5b7fe640 1 -- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 msgr2=0x7f4a5c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5b7fe640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5b7fe640 1 -- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 msgr2=0x7f4a5c1511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5b7fe640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c1511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5b7fe640 1 -- 172.21.15.119:0/3506305117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a54009a70 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a527fc640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:57.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5b7fe640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 0x7f4a5c0af680 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400efc0 tx=0x7f4a5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54009cd0 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a5400f040 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54003990 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.243+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4400b770 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.244+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5c1f2cd0 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.245+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a5c04ece0 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.245+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a54014480 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.246+0000 7f4a53fff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a3c06ae70 0x7f4a3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.246+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f4a5408a770 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.247+0000 7f4a5affd640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a3c06ae70 0x7f4a3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.247+0000 7f4a5affd640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a3c06ae70 0x7f4a3c06d330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000f60 tx=0x7f4a4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.251+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a54055330 con 0x7f4a5c15b0d0 2023-11-08T19:28:57.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.441+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a5c0c91f0 con 0x7f4a3c06ae70 2023-11-08T19:28:57.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.482+0000 7f4a53fff640 1 -- 172.21.15.119:0/3506305117 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+517488 (secure 0 0 0) 0x7f4a4c0012f0 con 0x7f4a3c06ae70 2023-11-08T19:28:57.491 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:57.495 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:28:57.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.492+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a3c06ae70 msgr2=0x7f4a3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.492+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a3c06ae70 0x7f4a3c06d330 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000f60 tx=0x7f4a4c009040 comp rx=0 tx=0).stop 2023-11-08T19:28:57.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.492+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 msgr2=0x7f4a5c0af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.492+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 0x7f4a5c0af680 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400efc0 tx=0x7f4a5400c490 comp rx=0 tx=0).stop 2023-11-08T19:28:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 shutdown_connections 2023-11-08T19:28:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a3c06ae70 0x7f4a3c06d330 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a5c15b0d0 0x7f4a5c0af680 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a5c0ad7d0 0x7f4a5c1511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 --2- 172.21.15.119:0/3506305117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a5c0710a0 0x7f4a5c0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 >> 172.21.15.119:0/3506305117 conn(0x7f4a5c0b0560 msgr2=0x7f4a5c1596a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:57.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 shutdown_connections 2023-11-08T19:28:57.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.493+0000 7f4a5bfff640 1 -- 172.21.15.119:0/3506305117 wait complete. 2023-11-08T19:28:57.515 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2023-11-08T19:28:56.741532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259548,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715172,"statfs":{"total":773094113280,"available":772828336128,"internally_reserved":0,"allocated":10342400,"data_stored":5691622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.741942"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.019781+0000","last_change":"2023-11-08T19:28:51.019781+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.019781+0000","last_undegraded":"2023-11-08T19:28:51.019781+0000","last_fullsized":"2023-11-08T19:28:51.019781+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898374+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:47.898374+0000","last_peered":"2023-11-08T19:28:47.898374+0000","last_clean":"2023-11-08T19:28:47.898374+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:47.898374+0000","last_undegraded":"2023-11-08T19:28:47.898374+0000","last_fullsized":"2023-11-08T19:28:47.898374+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897764+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:47.897764+0000","last_peered":"2023-11-08T19:28:47.897764+0000","last_clean":"2023-11-08T19:28:47.897764+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:47.897764+0000","last_undegraded":"2023-11-08T19:28:47.897764+0000","last_fullsized":"2023-11-08T19:28:47.897764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897826+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:47.897826+0000","last_peered":"2023-11-08T19:28:47.897826+0000","last_clean":"2023-11-08T19:28:47.897826+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:47.897826+0000","last_undegraded":"2023-11-08T19:28:47.897826+0000","last_fullsized":"2023-11-08T19:28:47.897826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.012025+0000","last_change":"2023-11-08T19:28:51.012025+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.012025+0000","last_undegraded":"2023-11-08T19:28:51.012025+0000","last_fullsized":"2023-11-08T19:28:51.012025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.981202+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:46.981202+0000","last_peered":"2023-11-08T19:28:46.981202+0000","last_clean":"2023-11-08T19:28:46.981202+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:46.981202+0000","last_undegraded":"2023-11-08T19:28:46.981202+0000","last_fullsized":"2023-11-08T19:28:46.981202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898666+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:47.898666+0000","last_peered":"2023-11-08T19:28:47.898666+0000","last_clean":"2023-11-08T19:28:47.898666+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:47.898666+0000","last_undegraded":"2023-11-08T19:28:47.898666+0000","last_fullsized":"2023-11-08T19:28:47.898666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898181+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:47.898181+0000","last_peered":"2023-11-08T19:28:47.898181+0000","last_clean":"2023-11-08T19:28:47.898181+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:47.898181+0000","last_undegraded":"2023-11-08T19:28:47.898181+0000","last_fullsized":"2023-11-08T19:28:47.898181+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.007233+0000","last_change":"2023-11-08T19:28:51.007233+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.007233+0000","last_undegraded":"2023-11-08T19:28:51.007233+0000","last_fullsized":"2023-11-08T19:28:51.007233+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984035+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:46.984035+0000","last_peered":"2023-11-08T19:28:46.984035+0000","last_clean":"2023-11-08T19:28:46.984035+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:46.984035+0000","last_undegraded":"2023-11-08T19:28:46.984035+0000","last_fullsized":"2023-11-08T19:28:46.984035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897644+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:47.897644+0000","last_peered":"2023-11-08T19:28:47.897644+0000","last_clean":"2023-11-08T19:28:47.897644+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:47.897644+0000","last_undegraded":"2023-11-08T19:28:47.897644+0000","last_fullsized":"2023-11-08T19:28:47.897644+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898929+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:47.898929+0000","last_peered":"2023-11-08T19:28:47.898929+0000","last_clean":"2023-11-08T19:28:47.898929+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:47.898929+0000","last_undegraded":"2023-11-08T19:28:47.898929+0000","last_fullsized":"2023-11-08T19:28:47.898929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.011921+0000","last_change":"2023-11-08T19:28:51.011921+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.011921+0000","last_undegraded":"2023-11-08T19:28:51.011921+0000","last_fullsized":"2023-11-08T19:28:51.011921+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073234+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:47.073234+0000","last_peered":"2023-11-08T19:28:47.073234+0000","last_clean":"2023-11-08T19:28:47.073234+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:47.073234+0000","last_undegraded":"2023-11-08T19:28:47.073234+0000","last_fullsized":"2023-11-08T19:28:47.073234+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897740+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:47.897740+0000","last_peered":"2023-11-08T19:28:47.897740+0000","last_clean":"2023-11-08T19:28:47.897740+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:47.897740+0000","last_undegraded":"2023-11-08T19:28:47.897740+0000","last_fullsized":"2023-11-08T19:28:47.897740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897705+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:47.897705+0000","last_peered":"2023-11-08T19:28:47.897705+0000","last_clean":"2023-11-08T19:28:47.897705+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:47.897705+0000","last_undegraded":"2023-11-08T19:28:47.897705+0000","last_fullsized":"2023-11-08T19:28:47.897705+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.021469+0000","last_change":"2023-11-08T19:28:51.021469+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.021469+0000","last_undegraded":"2023-11-08T19:28:51.021469+0000","last_fullsized":"2023-11-08T19:28:51.021469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982516+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:46.982516+0000","last_peered":"2023-11-08T19:28:46.982516+0000","last_clean":"2023-11-08T19:28:46.982516+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:46.982516+0000","last_undegraded":"2023-11-08T19:28:46.982516+0000","last_fullsized":"2023-11-08T19:28:46.982516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980580+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:46.980580+0000","last_peered":"2023-11-08T19:28:46.980580+0000","last_clean":"2023-11-08T19:28:46.980580+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:46.980580+0000","last_undegraded":"2023-11-08T19:28:46.980580+0000","last_fullsized":"2023-11-08T19:28:46.980580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898661+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:47.898661+0000","last_peered":"2023-11-08T19:28:47.898661+0000","last_clean":"2023-11-08T19:28:47.898661+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:47.898661+0000","last_undegraded":"2023-11-08T19:28:47.898661+0000","last_fullsized":"2023-11-08T19:28:47.898661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004832+0000","last_change":"2023-11-08T19:28:51.004832+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004832+0000","last_undegraded":"2023-11-08T19:28:51.004832+0000","last_fullsized":"2023-11-08T19:28:51.004832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073235+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:47.073235+0000","last_peered":"2023-11-08T19:28:47.073235+0000","last_clean":"2023-11-08T19:28:47.073235+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:47.073235+0000","last_undegraded":"2023-11-08T19:28:47.073235+0000","last_fullsized":"2023-11-08T19:28:47.073235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897438+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:47.897438+0000","last_peered":"2023-11-08T19:28:47.897438+0000","last_clean":"2023-11-08T19:28:47.897438+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:47.897438+0000","last_undegraded":"2023-11-08T19:28:47.897438+0000","last_fullsized":"2023-11-08T19:28:47.897438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073141+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:47.073141+0000","last_peered":"2023-11-08T19:28:47.073141+0000","last_clean":"2023-11-08T19:28:47.073141+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:47.073141+0000","last_undegraded":"2023-11-08T19:28:47.073141+0000","last_fullsized":"2023-11-08T19:28:47.073141+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004676+0000","last_change":"2023-11-08T19:28:51.004676+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004676+0000","last_undegraded":"2023-11-08T19:28:51.004676+0000","last_fullsized":"2023-11-08T19:28:51.004676+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897156+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:47.897156+0000","last_peered":"2023-11-08T19:28:47.897156+0000","last_clean":"2023-11-08T19:28:47.897156+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:47.897156+0000","last_undegraded":"2023-11-08T19:28:47.897156+0000","last_fullsized":"2023-11-08T19:28:47.897156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073059+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:47.073059+0000","last_peered":"2023-11-08T19:28:47.073059+0000","last_clean":"2023-11-08T19:28:47.073059+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:47.073059+0000","last_undegraded":"2023-11-08T19:28:47.073059+0000","last_fullsized":"2023-11-08T19:28:47.073059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073603+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:47.073603+0000","last_peered":"2023-11-08T19:28:47.073603+0000","last_clean":"2023-11-08T19:28:47.073603+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:47.073603+0000","last_undegraded":"2023-11-08T19:28:47.073603+0000","last_fullsized":"2023-11-08T19:28:47.073603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015351+0000","last_change":"2023-11-08T19:28:51.015351+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015351+0000","last_undegraded":"2023-11-08T19:28:51.015351+0000","last_fullsized":"2023-11-08T19:28:51.015351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897031+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:47.897031+0000","last_peered":"2023-11-08T19:28:47.897031+0000","last_clean":"2023-11-08T19:28:47.897031+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:47.897031+0000","last_undegraded":"2023-11-08T19:28:47.897031+0000","last_fullsized":"2023-11-08T19:28:47.897031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897639+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:47.897639+0000","last_peered":"2023-11-08T19:28:47.897639+0000","last_clean":"2023-11-08T19:28:47.897639+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:47.897639+0000","last_undegraded":"2023-11-08T19:28:47.897639+0000","last_fullsized":"2023-11-08T19:28:47.897639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983750+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:46.983750+0000","last_peered":"2023-11-08T19:28:46.983750+0000","last_clean":"2023-11-08T19:28:46.983750+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:46.983750+0000","last_undegraded":"2023-11-08T19:28:46.983750+0000","last_fullsized":"2023-11-08T19:28:46.983750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.009689+0000","last_change":"2023-11-08T19:28:51.009689+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.009689+0000","last_undegraded":"2023-11-08T19:28:51.009689+0000","last_fullsized":"2023-11-08T19:28:51.009689+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898820+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:47.898820+0000","last_peered":"2023-11-08T19:28:47.898820+0000","last_clean":"2023-11-08T19:28:47.898820+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:47.898820+0000","last_undegraded":"2023-11-08T19:28:47.898820+0000","last_fullsized":"2023-11-08T19:28:47.898820+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984389+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:46.984389+0000","last_peered":"2023-11-08T19:28:46.984389+0000","last_clean":"2023-11-08T19:28:46.984389+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:46.984389+0000","last_undegraded":"2023-11-08T19:28:46.984389+0000","last_fullsized":"2023-11-08T19:28:46.984389+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983524+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:46.983524+0000","last_peered":"2023-11-08T19:28:46.983524+0000","last_clean":"2023-11-08T19:28:46.983524+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:46.983524+0000","last_undegraded":"2023-11-08T19:28:46.983524+0000","last_fullsized":"2023-11-08T19:28:46.983524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005396+0000","last_change":"2023-11-08T19:28:51.005396+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005396+0000","last_undegraded":"2023-11-08T19:28:51.005396+0000","last_fullsized":"2023-11-08T19:28:51.005396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984268+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:46.984268+0000","last_peered":"2023-11-08T19:28:46.984268+0000","last_clean":"2023-11-08T19:28:46.984268+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:46.984268+0000","last_undegraded":"2023-11-08T19:28:46.984268+0000","last_fullsized":"2023-11-08T19:28:46.984268+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898588+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:47.898588+0000","last_peered":"2023-11-08T19:28:47.898588+0000","last_clean":"2023-11-08T19:28:47.898588+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:47.898588+0000","last_undegraded":"2023-11-08T19:28:47.898588+0000","last_fullsized":"2023-11-08T19:28:47.898588+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980541+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:46.980541+0000","last_peered":"2023-11-08T19:28:46.980541+0000","last_clean":"2023-11-08T19:28:46.980541+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:46.980541+0000","last_undegraded":"2023-11-08T19:28:46.980541+0000","last_fullsized":"2023-11-08T19:28:46.980541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005496+0000","last_change":"2023-11-08T19:28:51.005496+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005496+0000","last_undegraded":"2023-11-08T19:28:51.005496+0000","last_fullsized":"2023-11-08T19:28:51.005496+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982650+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:46.982650+0000","last_peered":"2023-11-08T19:28:46.982650+0000","last_clean":"2023-11-08T19:28:46.982650+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:46.982650+0000","last_undegraded":"2023-11-08T19:28:46.982650+0000","last_fullsized":"2023-11-08T19:28:46.982650+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073342+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:47.073342+0000","last_peered":"2023-11-08T19:28:47.073342+0000","last_clean":"2023-11-08T19:28:47.073342+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:47.073342+0000","last_undegraded":"2023-11-08T19:28:47.073342+0000","last_fullsized":"2023-11-08T19:28:47.073342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983351+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:46.983351+0000","last_peered":"2023-11-08T19:28:46.983351+0000","last_clean":"2023-11-08T19:28:46.983351+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:46.983351+0000","last_undegraded":"2023-11-08T19:28:46.983351+0000","last_fullsized":"2023-11-08T19:28:46.983351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.020306+0000","last_change":"2023-11-08T19:28:51.020306+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.020306+0000","last_undegraded":"2023-11-08T19:28:51.020306+0000","last_fullsized":"2023-11-08T19:28:51.020306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897863+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:47.897863+0000","last_peered":"2023-11-08T19:28:47.897863+0000","last_clean":"2023-11-08T19:28:47.897863+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:47.897863+0000","last_undegraded":"2023-11-08T19:28:47.897863+0000","last_fullsized":"2023-11-08T19:28:47.897863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073430+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:47.073430+0000","last_peered":"2023-11-08T19:28:47.073430+0000","last_clean":"2023-11-08T19:28:47.073430+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:47.073430+0000","last_undegraded":"2023-11-08T19:28:47.073430+0000","last_fullsized":"2023-11-08T19:28:47.073430+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984510+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:46.984510+0000","last_peered":"2023-11-08T19:28:46.984510+0000","last_clean":"2023-11-08T19:28:46.984510+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:46.984510+0000","last_undegraded":"2023-11-08T19:28:46.984510+0000","last_fullsized":"2023-11-08T19:28:46.984510+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015226+0000","last_change":"2023-11-08T19:28:51.015226+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015226+0000","last_undegraded":"2023-11-08T19:28:51.015226+0000","last_fullsized":"2023-11-08T19:28:51.015226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983868+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:46.983868+0000","last_peered":"2023-11-08T19:28:46.983868+0000","last_clean":"2023-11-08T19:28:46.983868+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:46.983868+0000","last_undegraded":"2023-11-08T19:28:46.983868+0000","last_fullsized":"2023-11-08T19:28:46.983868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983310+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:46.983310+0000","last_peered":"2023-11-08T19:28:46.983310+0000","last_clean":"2023-11-08T19:28:46.983310+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:46.983310+0000","last_undegraded":"2023-11-08T19:28:46.983310+0000","last_fullsized":"2023-11-08T19:28:46.983310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982243+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:46.982243+0000","last_peered":"2023-11-08T19:28:46.982243+0000","last_clean":"2023-11-08T19:28:46.982243+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:46.982243+0000","last_undegraded":"2023-11-08T19:28:46.982243+0000","last_fullsized":"2023-11-08T19:28:46.982243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.010187+0000","last_change":"2023-11-08T19:28:51.010187+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.010187+0000","last_undegraded":"2023-11-08T19:28:51.010187+0000","last_fullsized":"2023-11-08T19:28:51.010187+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897613+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:47.897613+0000","last_peered":"2023-11-08T19:28:47.897613+0000","last_clean":"2023-11-08T19:28:47.897613+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:47.897613+0000","last_undegraded":"2023-11-08T19:28:47.897613+0000","last_fullsized":"2023-11-08T19:28:47.897613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984004+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:46.984004+0000","last_peered":"2023-11-08T19:28:46.984004+0000","last_clean":"2023-11-08T19:28:46.984004+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:46.984004+0000","last_undegraded":"2023-11-08T19:28:46.984004+0000","last_fullsized":"2023-11-08T19:28:46.984004+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980151+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:46.980151+0000","last_peered":"2023-11-08T19:28:46.980151+0000","last_clean":"2023-11-08T19:28:46.980151+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:46.980151+0000","last_undegraded":"2023-11-08T19:28:46.980151+0000","last_fullsized":"2023-11-08T19:28:46.980151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.010260+0000","last_change":"2023-11-08T19:28:51.010260+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.010260+0000","last_undegraded":"2023-11-08T19:28:51.010260+0000","last_fullsized":"2023-11-08T19:28:51.010260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980042+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:46.980042+0000","last_peered":"2023-11-08T19:28:46.980042+0000","last_clean":"2023-11-08T19:28:46.980042+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:46.980042+0000","last_undegraded":"2023-11-08T19:28:46.980042+0000","last_fullsized":"2023-11-08T19:28:46.980042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897127+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:47.897127+0000","last_peered":"2023-11-08T19:28:47.897127+0000","last_clean":"2023-11-08T19:28:47.897127+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:47.897127+0000","last_undegraded":"2023-11-08T19:28:47.897127+0000","last_fullsized":"2023-11-08T19:28:47.897127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073742+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:47.073742+0000","last_peered":"2023-11-08T19:28:47.073742+0000","last_clean":"2023-11-08T19:28:47.073742+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:47.073742+0000","last_undegraded":"2023-11-08T19:28:47.073742+0000","last_fullsized":"2023-11-08T19:28:47.073742+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005943+0000","last_change":"2023-11-08T19:28:51.005943+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005943+0000","last_undegraded":"2023-11-08T19:28:51.005943+0000","last_fullsized":"2023-11-08T19:28:51.005943+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982027+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:46.982027+0000","last_peered":"2023-11-08T19:28:46.982027+0000","last_clean":"2023-11-08T19:28:46.982027+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:46.982027+0000","last_undegraded":"2023-11-08T19:28:46.982027+0000","last_fullsized":"2023-11-08T19:28:46.982027+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073176+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:47.073176+0000","last_peered":"2023-11-08T19:28:47.073176+0000","last_clean":"2023-11-08T19:28:47.073176+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:47.073176+0000","last_undegraded":"2023-11-08T19:28:47.073176+0000","last_fullsized":"2023-11-08T19:28:47.073176+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897468+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:47.897468+0000","last_peered":"2023-11-08T19:28:47.897468+0000","last_clean":"2023-11-08T19:28:47.897468+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:47.897468+0000","last_undegraded":"2023-11-08T19:28:47.897468+0000","last_fullsized":"2023-11-08T19:28:47.897468+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.009642+0000","last_change":"2023-11-08T19:28:51.009642+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.009642+0000","last_undegraded":"2023-11-08T19:28:51.009642+0000","last_fullsized":"2023-11-08T19:28:51.009642+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983068+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:46.983068+0000","last_peered":"2023-11-08T19:28:46.983068+0000","last_clean":"2023-11-08T19:28:46.983068+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:46.983068+0000","last_undegraded":"2023-11-08T19:28:46.983068+0000","last_fullsized":"2023-11-08T19:28:46.983068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897599+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:47.897599+0000","last_peered":"2023-11-08T19:28:47.897599+0000","last_clean":"2023-11-08T19:28:47.897599+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:47.897599+0000","last_undegraded":"2023-11-08T19:28:47.897599+0000","last_fullsized":"2023-11-08T19:28:47.897599+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898064+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:47.898064+0000","last_peered":"2023-11-08T19:28:47.898064+0000","last_clean":"2023-11-08T19:28:47.898064+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:47.898064+0000","last_undegraded":"2023-11-08T19:28:47.898064+0000","last_fullsized":"2023-11-08T19:28:47.898064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.006470+0000","last_change":"2023-11-08T19:28:51.006470+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.006470+0000","last_undegraded":"2023-11-08T19:28:51.006470+0000","last_fullsized":"2023-11-08T19:28:51.006470+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984345+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:46.984345+0000","last_peered":"2023-11-08T19:28:46.984345+0000","last_clean":"2023-11-08T19:28:46.984345+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:46.984345+0000","last_undegraded":"2023-11-08T19:28:46.984345+0000","last_fullsized":"2023-11-08T19:28:46.984345+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897497+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:47.897497+0000","last_peered":"2023-11-08T19:28:47.897497+0000","last_clean":"2023-11-08T19:28:47.897497+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:47.897497+0000","last_undegraded":"2023-11-08T19:28:47.897497+0000","last_fullsized":"2023-11-08T19:28:47.897497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898470+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:47.898470+0000","last_peered":"2023-11-08T19:28:47.898470+0000","last_clean":"2023-11-08T19:28:47.898470+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:47.898470+0000","last_undegraded":"2023-11-08T19:28:47.898470+0000","last_fullsized":"2023-11-08T19:28:47.898470+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.008922+0000","last_change":"2023-11-08T19:28:51.008922+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.008922+0000","last_undegraded":"2023-11-08T19:28:51.008922+0000","last_fullsized":"2023-11-08T19:28:51.008922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984247+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:46.984247+0000","last_peered":"2023-11-08T19:28:46.984247+0000","last_clean":"2023-11-08T19:28:46.984247+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:46.984247+0000","last_undegraded":"2023-11-08T19:28:46.984247+0000","last_fullsized":"2023-11-08T19:28:46.984247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898061+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:47.898061+0000","last_peered":"2023-11-08T19:28:47.898061+0000","last_clean":"2023-11-08T19:28:47.898061+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:47.898061+0000","last_undegraded":"2023-11-08T19:28:47.898061+0000","last_fullsized":"2023-11-08T19:28:47.898061+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073685+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:47.073685+0000","last_peered":"2023-11-08T19:28:47.073685+0000","last_clean":"2023-11-08T19:28:47.073685+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:47.073685+0000","last_undegraded":"2023-11-08T19:28:47.073685+0000","last_fullsized":"2023-11-08T19:28:47.073685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005345+0000","last_change":"2023-11-08T19:28:51.005345+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005345+0000","last_undegraded":"2023-11-08T19:28:51.005345+0000","last_fullsized":"2023-11-08T19:28:51.005345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898260+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:47.898260+0000","last_peered":"2023-11-08T19:28:47.898260+0000","last_clean":"2023-11-08T19:28:47.898260+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:47.898260+0000","last_undegraded":"2023-11-08T19:28:47.898260+0000","last_fullsized":"2023-11-08T19:28:47.898260+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897549+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:47.897549+0000","last_peered":"2023-11-08T19:28:47.897549+0000","last_clean":"2023-11-08T19:28:47.897549+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:47.897549+0000","last_undegraded":"2023-11-08T19:28:47.897549+0000","last_fullsized":"2023-11-08T19:28:47.897549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759631+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:47.759631+0000","last_peered":"2023-11-08T19:28:47.759631+0000","last_clean":"2023-11-08T19:28:47.759631+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:47.759631+0000","last_undegraded":"2023-11-08T19:28:47.759631+0000","last_fullsized":"2023-11-08T19:28:47.759631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.016804+0000","last_change":"2023-11-08T19:28:51.016804+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.016804+0000","last_undegraded":"2023-11-08T19:28:51.016804+0000","last_fullsized":"2023-11-08T19:28:51.016804+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073498+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:47.073498+0000","last_peered":"2023-11-08T19:28:47.073498+0000","last_clean":"2023-11-08T19:28:47.073498+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:47.073498+0000","last_undegraded":"2023-11-08T19:28:47.073498+0000","last_fullsized":"2023-11-08T19:28:47.073498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983546+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:46.983546+0000","last_peered":"2023-11-08T19:28:46.983546+0000","last_clean":"2023-11-08T19:28:46.983546+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:46.983546+0000","last_undegraded":"2023-11-08T19:28:46.983546+0000","last_fullsized":"2023-11-08T19:28:46.983546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.003018+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:51.003018+0000","last_peered":"2023-11-08T19:28:51.003018+0000","last_clean":"2023-11-08T19:28:51.003018+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:51.003018+0000","last_undegraded":"2023-11-08T19:28:51.003018+0000","last_fullsized":"2023-11-08T19:28:51.003018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004718+0000","last_change":"2023-11-08T19:28:51.004718+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004718+0000","last_undegraded":"2023-11-08T19:28:51.004718+0000","last_fullsized":"2023-11-08T19:28:51.004718+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897253+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:47.897253+0000","last_peered":"2023-11-08T19:28:47.897253+0000","last_clean":"2023-11-08T19:28:47.897253+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:47.897253+0000","last_undegraded":"2023-11-08T19:28:47.897253+0000","last_fullsized":"2023-11-08T19:28:47.897253+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073589+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:47.073589+0000","last_peered":"2023-11-08T19:28:47.073589+0000","last_clean":"2023-11-08T19:28:47.073589+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:47.073589+0000","last_undegraded":"2023-11-08T19:28:47.073589+0000","last_fullsized":"2023-11-08T19:28:47.073589+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.019899+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:51.019899+0000","last_peered":"2023-11-08T19:28:51.019899+0000","last_clean":"2023-11-08T19:28:51.019899+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:51.019899+0000","last_undegraded":"2023-11-08T19:28:51.019899+0000","last_fullsized":"2023-11-08T19:28:51.019899+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.020099+0000","last_change":"2023-11-08T19:28:51.020099+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.020099+0000","last_undegraded":"2023-11-08T19:28:51.020099+0000","last_fullsized":"2023-11-08T19:28:51.020099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897718+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:47.897718+0000","last_peered":"2023-11-08T19:28:47.897718+0000","last_clean":"2023-11-08T19:28:47.897718+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:47.897718+0000","last_undegraded":"2023-11-08T19:28:47.897718+0000","last_fullsized":"2023-11-08T19:28:47.897718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983268+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:46.983268+0000","last_peered":"2023-11-08T19:28:46.983268+0000","last_clean":"2023-11-08T19:28:46.983268+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:46.983268+0000","last_undegraded":"2023-11-08T19:28:46.983268+0000","last_fullsized":"2023-11-08T19:28:46.983268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897520+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:47.897520+0000","last_peered":"2023-11-08T19:28:47.897520+0000","last_clean":"2023-11-08T19:28:47.897520+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:47.897520+0000","last_undegraded":"2023-11-08T19:28:47.897520+0000","last_fullsized":"2023-11-08T19:28:47.897520+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.021701+0000","last_change":"2023-11-08T19:28:51.021701+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.021701+0000","last_undegraded":"2023-11-08T19:28:51.021701+0000","last_fullsized":"2023-11-08T19:28:51.021701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982262+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:46.982262+0000","last_peered":"2023-11-08T19:28:46.982262+0000","last_clean":"2023-11-08T19:28:46.982262+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:46.982262+0000","last_undegraded":"2023-11-08T19:28:46.982262+0000","last_fullsized":"2023-11-08T19:28:46.982262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898086+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:47.898086+0000","last_peered":"2023-11-08T19:28:47.898086+0000","last_clean":"2023-11-08T19:28:47.898086+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:47.898086+0000","last_undegraded":"2023-11-08T19:28:47.898086+0000","last_fullsized":"2023-11-08T19:28:47.898086+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.013666+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:51.013666+0000","last_peered":"2023-11-08T19:28:51.013666+0000","last_clean":"2023-11-08T19:28:51.013666+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:51.013666+0000","last_undegraded":"2023-11-08T19:28:51.013666+0000","last_fullsized":"2023-11-08T19:28:51.013666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015270+0000","last_change":"2023-11-08T19:28:51.015270+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015270+0000","last_undegraded":"2023-11-08T19:28:51.015270+0000","last_fullsized":"2023-11-08T19:28:51.015270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898860+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:47.898860+0000","last_peered":"2023-11-08T19:28:47.898860+0000","last_clean":"2023-11-08T19:28:47.898860+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:47.898860+0000","last_undegraded":"2023-11-08T19:28:47.898860+0000","last_fullsized":"2023-11-08T19:28:47.898860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073478+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:47.073478+0000","last_peered":"2023-11-08T19:28:47.073478+0000","last_clean":"2023-11-08T19:28:47.073478+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:47.073478+0000","last_undegraded":"2023-11-08T19:28:47.073478+0000","last_fullsized":"2023-11-08T19:28:47.073478+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983063+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:46.983063+0000","last_peered":"2023-11-08T19:28:46.983063+0000","last_clean":"2023-11-08T19:28:46.983063+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:46.983063+0000","last_undegraded":"2023-11-08T19:28:46.983063+0000","last_fullsized":"2023-11-08T19:28:46.983063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.012488+0000","last_change":"2023-11-08T19:28:51.012488+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.012488+0000","last_undegraded":"2023-11-08T19:28:51.012488+0000","last_fullsized":"2023-11-08T19:28:51.012488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982795+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:46.982795+0000","last_peered":"2023-11-08T19:28:46.982795+0000","last_clean":"2023-11-08T19:28:46.982795+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:46.982795+0000","last_undegraded":"2023-11-08T19:28:46.982795+0000","last_fullsized":"2023-11-08T19:28:46.982795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898720+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:47.898720+0000","last_peered":"2023-11-08T19:28:47.898720+0000","last_clean":"2023-11-08T19:28:47.898720+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:47.898720+0000","last_undegraded":"2023-11-08T19:28:47.898720+0000","last_fullsized":"2023-11-08T19:28:47.898720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073382+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:47.073382+0000","last_peered":"2023-11-08T19:28:47.073382+0000","last_clean":"2023-11-08T19:28:47.073382+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:47.073382+0000","last_undegraded":"2023-11-08T19:28:47.073382+0000","last_fullsized":"2023-11-08T19:28:47.073382+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.018696+0000","last_change":"2023-11-08T19:28:51.018696+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.018696+0000","last_undegraded":"2023-11-08T19:28:51.018696+0000","last_fullsized":"2023-11-08T19:28:51.018696+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984117+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:46.984117+0000","last_peered":"2023-11-08T19:28:46.984117+0000","last_clean":"2023-11-08T19:28:46.984117+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:46.984117+0000","last_undegraded":"2023-11-08T19:28:46.984117+0000","last_fullsized":"2023-11-08T19:28:46.984117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073301+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:47.073301+0000","last_peered":"2023-11-08T19:28:47.073301+0000","last_clean":"2023-11-08T19:28:47.073301+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:47.073301+0000","last_undegraded":"2023-11-08T19:28:47.073301+0000","last_fullsized":"2023-11-08T19:28:47.073301+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.017867+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:51.017867+0000","last_peered":"2023-11-08T19:28:51.017867+0000","last_clean":"2023-11-08T19:28:51.017867+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:51.017867+0000","last_undegraded":"2023-11-08T19:28:51.017867+0000","last_fullsized":"2023-11-08T19:28:51.017867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.008313+0000","last_change":"2023-11-08T19:28:51.008313+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.008313+0000","last_undegraded":"2023-11-08T19:28:51.008313+0000","last_fullsized":"2023-11-08T19:28:51.008313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897392+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:47.897392+0000","last_peered":"2023-11-08T19:28:47.897392+0000","last_clean":"2023-11-08T19:28:47.897392+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:47.897392+0000","last_undegraded":"2023-11-08T19:28:47.897392+0000","last_fullsized":"2023-11-08T19:28:47.897392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759820+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:47.759820+0000","last_peered":"2023-11-08T19:28:47.759820+0000","last_clean":"2023-11-08T19:28:47.759820+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:47.759820+0000","last_undegraded":"2023-11-08T19:28:47.759820+0000","last_fullsized":"2023-11-08T19:28:47.759820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983608+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:46.983608+0000","last_peered":"2023-11-08T19:28:46.983608+0000","last_clean":"2023-11-08T19:28:46.983608+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:46.983608+0000","last_undegraded":"2023-11-08T19:28:46.983608+0000","last_fullsized":"2023-11-08T19:28:46.983608+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.019995+0000","last_change":"2023-11-08T19:28:51.019995+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.019995+0000","last_undegraded":"2023-11-08T19:28:51.019995+0000","last_fullsized":"2023-11-08T19:28:51.019995+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759877+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:47.759877+0000","last_peered":"2023-11-08T19:28:47.759877+0000","last_clean":"2023-11-08T19:28:47.759877+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:47.759877+0000","last_undegraded":"2023-11-08T19:28:47.759877+0000","last_fullsized":"2023-11-08T19:28:47.759877+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980860+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:46.980860+0000","last_peered":"2023-11-08T19:28:46.980860+0000","last_clean":"2023-11-08T19:28:46.980860+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:46.980860+0000","last_undegraded":"2023-11-08T19:28:46.980860+0000","last_fullsized":"2023-11-08T19:28:46.980860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897035+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:47.897035+0000","last_peered":"2023-11-08T19:28:47.897035+0000","last_clean":"2023-11-08T19:28:47.897035+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:47.897035+0000","last_undegraded":"2023-11-08T19:28:47.897035+0000","last_fullsized":"2023-11-08T19:28:47.897035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004935+0000","last_change":"2023-11-08T19:28:51.004935+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004935+0000","last_undegraded":"2023-11-08T19:28:51.004935+0000","last_fullsized":"2023-11-08T19:28:51.004935+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897809+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:47.897809+0000","last_peered":"2023-11-08T19:28:47.897809+0000","last_clean":"2023-11-08T19:28:47.897809+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:47.897809+0000","last_undegraded":"2023-11-08T19:28:47.897809+0000","last_fullsized":"2023-11-08T19:28:47.897809+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980029+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:46.980029+0000","last_peered":"2023-11-08T19:28:46.980029+0000","last_clean":"2023-11-08T19:28:46.980029+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:46.980029+0000","last_undegraded":"2023-11-08T19:28:46.980029+0000","last_fullsized":"2023-11-08T19:28:46.980029+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.981009+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:46.981009+0000","last_peered":"2023-11-08T19:28:46.981009+0000","last_clean":"2023-11-08T19:28:46.981009+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:46.981009+0000","last_undegraded":"2023-11-08T19:28:46.981009+0000","last_fullsized":"2023-11-08T19:28:46.981009+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.007275+0000","last_change":"2023-11-08T19:28:51.007275+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.007275+0000","last_undegraded":"2023-11-08T19:28:51.007275+0000","last_fullsized":"2023-11-08T19:28:51.007275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759559+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:47.759559+0000","last_peered":"2023-11-08T19:28:47.759559+0000","last_clean":"2023-11-08T19:28:47.759559+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:47.759559+0000","last_undegraded":"2023-11-08T19:28:47.759559+0000","last_fullsized":"2023-11-08T19:28:47.759559+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759562+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:47.759562+0000","last_peered":"2023-11-08T19:28:47.759562+0000","last_clean":"2023-11-08T19:28:47.759562+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:47.759562+0000","last_undegraded":"2023-11-08T19:28:47.759562+0000","last_fullsized":"2023-11-08T19:28:47.759562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980880+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:46.980880+0000","last_peered":"2023-11-08T19:28:46.980880+0000","last_clean":"2023-11-08T19:28:46.980880+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:46.980880+0000","last_undegraded":"2023-11-08T19:28:46.980880+0000","last_fullsized":"2023-11-08T19:28:46.980880+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005157+0000","last_change":"2023-11-08T19:28:51.005157+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005157+0000","last_undegraded":"2023-11-08T19:28:51.005157+0000","last_fullsized":"2023-11-08T19:28:51.005157+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897246+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:47.897246+0000","last_peered":"2023-11-08T19:28:47.897246+0000","last_clean":"2023-11-08T19:28:47.897246+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:47.897246+0000","last_undegraded":"2023-11-08T19:28:47.897246+0000","last_fullsized":"2023-11-08T19:28:47.897246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982860+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:46.982860+0000","last_peered":"2023-11-08T19:28:46.982860+0000","last_clean":"2023-11-08T19:28:46.982860+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:46.982860+0000","last_undegraded":"2023-11-08T19:28:46.982860+0000","last_fullsized":"2023-11-08T19:28:46.982860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.004218+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:51.004218+0000","last_peered":"2023-11-08T19:28:51.004218+0000","last_clean":"2023-11-08T19:28:51.004218+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:51.004218+0000","last_undegraded":"2023-11-08T19:28:51.004218+0000","last_fullsized":"2023-11-08T19:28:51.004218+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.018351+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:51.018351+0000","last_peered":"2023-11-08T19:28:51.018351+0000","last_clean":"2023-11-08T19:28:51.018351+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:51.018351+0000","last_undegraded":"2023-11-08T19:28:51.018351+0000","last_fullsized":"2023-11-08T19:28:51.018351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":1155387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.52,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.141,"5min":1.25,"15min":1.3120000000000001},"min":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1360000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.74399999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.196,"15min":1.2569999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.518,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.46},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.125,"15min":1.1359999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0049999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.405,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.056,"5min":1.28,"15min":1.3620000000000001},"min":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.389,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.94999999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3129999999999999,"15min":1.367},"min":{"1min":0.79000000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.591,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.284},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.506,"15min":1.5700000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2200000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.167}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.798,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.056},{"interface":"front","average":{"1min":1.321,"5min":1.427,"15min":1.4970000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.774,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.73299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.359},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7829999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3220000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.546,"15min":1.579},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.9770000000000001,"5min":2.427,"15min":2.427},"last":1.234},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.5680000000000001,"15min":1.631},"min":{"1min":0.997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.8939999999999999,"5min":2.468,"15min":2.468},"last":1.401}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96999999999999997,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6279999999999999,"5min":1.752,"15min":1.752},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.1140000000000001,"15min":1.1060000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.581,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.095,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.47,"5min":1.679,"15min":1.679},"last":1.411},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.2789999999999999,"15min":1.337},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6559999999999999,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.999}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.238,"15min":1.292},"min":{"1min":0.68799999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4359999999999999,"5min":4.609,"15min":4.609},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.861,"15min":1.861},"last":1.177}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4350000000000001,"15min":1.4870000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.756,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.351},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1819999999999999,"15min":1.181},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.482,"5min":1.784,"15min":1.784},"last":1.2869999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.371,"15min":1.377},"min":{"1min":0.79600000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.732,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.518,"15min":1.5029999999999999},"min":{"1min":0.89500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9510000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1180000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7829999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.073},{"interface":"front","average":{"1min":1.403,"5min":1.347,"15min":1.335},"min":{"1min":0.83099999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8660000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5800000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5669999999999999,"15min":1.55},"min":{"1min":0.85899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8759999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.639},{"interface":"front","average":{"1min":1.512,"5min":1.454,"15min":1.454},"min":{"1min":0.81499999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8779999999999999,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.464}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.093,"15min":1.0740000000000001},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6990000000000001,"5min":1.76,"15min":1.76},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.234},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.258}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.141,"15min":1.1699999999999999},"min":{"1min":0.753,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5349999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.508},{"interface":"front","average":{"1min":1.181,"5min":1.343,"15min":1.4410000000000001},"min":{"1min":0.76200000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.617,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.0469999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1830000000000001,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.639,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3939999999999999,"15min":1.4990000000000001},"min":{"1min":0.94299999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.806,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.1000000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.288},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.804,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3109999999999999,"15min":1.3049999999999999},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.85399999999999998,"5min":0.755,"15min":0.755},"max":{"1min":1.635,"5min":2.347,"15min":2.347},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5149999999999999,"15min":1.5529999999999999},"min":{"1min":1.0589999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.097,"5min":3.153,"15min":3.153},"last":1.6319999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4930000000000001,"15min":1.528},"min":{"1min":0.89100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.8360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.5009999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.899,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.8029999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.6399999999999999,"15min":1.7},"min":{"1min":1.03,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0449999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.718},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.581,"15min":1.5900000000000001},"min":{"1min":1.1439999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.831,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.883}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0549999999999999,"15min":1.0469999999999999},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.496,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2549999999999999,"15min":1.2090000000000001},"min":{"1min":0.85699999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.696,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.069,"15min":1.091},"min":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1279999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.89100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.7450000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.28}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4610000000000001,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.423},"min":{"1min":0.82099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.391}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.696,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3600000000000001,"15min":1.387},"min":{"1min":0.83599999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.629,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4399999999999999,"15min":1.5129999999999999},"min":{"1min":0.96699999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.667,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.4970000000000001,"15min":1.5489999999999999},"min":{"1min":0.95899999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8560000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5229999999999999,"15min":1.5620000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.782,"5min":4.335,"15min":4.335},"last":1.218},{"interface":"front","average":{"1min":1.52,"5min":1.5880000000000001,"15min":1.6259999999999999},"min":{"1min":1.3140000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8080000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.605}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.6040000000000001,"15min":1.631},"min":{"1min":1.1799999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.77,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.6859999999999999,"15min":1.732},"min":{"1min":1.127,"5min":1.095,"15min":1.095},"max":{"1min":1.901,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.327}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.98599999999999999,"15min":1.0720000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":3.972,"15min":3.972},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.032,"15min":1.095},"min":{"1min":0.372,"5min":0.311,"15min":0.311},"max":{"1min":1.899,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.103,"15min":1.1850000000000001},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2609999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1080000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.8520000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2709999999999999,"15min":1.325},"min":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6739999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.448},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2669999999999999},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.754,"5min":2.73,"15min":2.73},"last":1.5029999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3740000000000001,"15min":1.448},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.663,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.143},{"interface":"front","average":{"1min":1.204,"5min":1.393,"15min":1.452},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":3.052,"15min":3.052},"last":1.204}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4339999999999999,"15min":1.498},"min":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0430000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.4670000000000001,"15min":1.5309999999999999},"min":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.948,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.878,"15min":0.878},"max":{"1min":3.21,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.5269999999999999,"15min":1.585},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0880000000000001,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.403}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6140000000000001,"15min":1.71},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.948,"5min":4.407,"15min":4.407},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.528,"15min":1.581},"min":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0059999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":564000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.86299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.879,"15min":0.92700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.171,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.198,"5min":1.393,"15min":1.393},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":1.016},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4690000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.032}]},{"osd":3,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.022,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0600000000000001,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6060000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.407},"min":{"1min":0.81499999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2969999999999999,"15min":1.365},"min":{"1min":0.81999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7,"5min":2.278,"15min":2.278},"last":1.4450000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.425},"min":{"1min":0.76600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1600000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.373,"15min":1.4610000000000001},"min":{"1min":0.78500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.954,"5min":2.536,"15min":2.536},"last":1.3799999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.347,"15min":1.389},"min":{"1min":0.93100000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9119999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.548},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.036,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.3460000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4930000000000001,"15min":1.577},"min":{"1min":0.85799999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.861,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.492,"15min":1.556},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.0030000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.639}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.745},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.304,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.123,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.1519999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.65000000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.0700000000000001,"15min":1.0860000000000001},"min":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2690000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.276}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.1539999999999999},"min":{"1min":0.78900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.556,"5min":2.165,"15min":2.165},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.252},"min":{"1min":0.77400000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.454,"5min":1.885,"15min":1.885},"last":1.7589999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2549999999999999},"min":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.274,"15min":1.321},"min":{"1min":0.78900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.579,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.5169999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3580000000000001,"15min":1.3660000000000001},"min":{"1min":0.84999999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1509999999999998,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.92400000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.724,"5min":2.282,"15min":2.282},"last":1.6279999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3200000000000001,"15min":1.3300000000000001},"min":{"1min":1.109,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.909,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4830000000000001,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.615}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1147140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.296,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.872,"15min":0.92800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3180000000000001,"5min":1.7,"15min":1.7},"last":0.92900000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.86599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.284,"5min":1.508,"15min":1.508},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.97199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.407,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.256}]},{"osd":3,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.6599999999999999,"5min":1.748,"15min":1.748},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0409999999999999,"15min":1.099},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.506,"5min":2.496,"15min":2.496},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5609999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.206},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3049999999999999,"15min":1.4199999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.54,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3979999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.286,"15min":1.3600000000000001},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.516},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.355,"15min":1.4330000000000001},"min":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.534}]},{"osd":6,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4870000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1619999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3540000000000001,"15min":1.405},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.3660000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.47},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.405,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.881,"5min":2.306,"15min":2.306},"last":1.702}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:57.527 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:57.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.773+0000 7f0b773d7640 1 Processor -- start 2023-11-08T19:28:57.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.773+0000 7f0b773d7640 1 -- start start 2023-11-08T19:28:57.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.774+0000 7f0b773d7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0b701447a0 0x7f0b7015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.774+0000 7f0b773d7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b700cad30 0x7f0b7006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.774+0000 7f0b773d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b7006a120 0x7f0b7006a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b773d7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7006aa90 con 0x7f0b700cad30 2023-11-08T19:28:57.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b773d7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b700c3a50 con 0x7f0b701447a0 2023-11-08T19:28:57.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b773d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b70144b80 con 0x7f0b7006a120 2023-11-08T19:28:57.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b700cad30 0x7f0b7006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0b700cad30 0x7f0b7006f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35650/0 (socket says 172.21.15.119:35650) 2023-11-08T19:28:57.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 -- 172.21.15.119:0/3510804026 learned_addr learned my addr 172.21.15.119:0/3510804026 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 -- 172.21.15.119:0/3510804026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b7006a120 msgr2=0x7f0b7006a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:57.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 --2- 172.21.15.119:0/3510804026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b7006a120 0x7f0b7006a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 -- 172.21.15.119:0/3510804026 >> v2:172.21.15.188:3300/0 conn(0x7f0b701447a0 msgr2=0x7f0b7015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:57.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 --2- 172.21.15.119:0/3510804026 >> v2:172.21.15.188:3300/0 conn(0x7f0b701447a0 0x7f0b7015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.775+0000 7f0b74972640 1 -- 172.21.15.119:0/3510804026 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700c9150 con 0x7f0b700cad30 2023-11-08T19:28:57.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.776+0000 7f0b74972640 1 --2- 172.21.15.119:0/3510804026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b7006f930 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0b700cfbd0 tx=0x7f0b6002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80115d24e7b0e8f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b67fff640 1 -- 172.21.15.119:0/3510804026 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60031870 con 0x7f0b700cad30 2023-11-08T19:28:57.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b67fff640 1 -- 172.21.15.119:0/3510804026 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b600319d0 con 0x7f0b700cad30 2023-11-08T19:28:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b67fff640 1 -- 172.21.15.119:0/3510804026 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60031d60 con 0x7f0b700cad30 2023-11-08T19:28:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 -- 172.21.15.119:0/3510804026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 msgr2=0x7f0b7006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 --2- 172.21.15.119:0/3510804026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b7006f930 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f0b700cfbd0 tx=0x7f0b6002b910 comp rx=0 tx=0).stop 2023-11-08T19:28:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 -- 172.21.15.119:0/3510804026 shutdown_connections 2023-11-08T19:28:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 --2- 172.21.15.119:0/3510804026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b7006a120 0x7f0b7006a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 --2- 172.21.15.119:0/3510804026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b7006f930 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 --2- 172.21.15.119:0/3510804026 >> v2:172.21.15.188:3300/0 conn(0x7f0b701447a0 0x7f0b7015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 -- 172.21.15.119:0/3510804026 >> 172.21.15.119:0/3510804026 conn(0x7f0b700cb610 msgr2=0x7f0b700cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:57.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 -- 172.21.15.119:0/3510804026 shutdown_connections 2023-11-08T19:28:57.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.777+0000 7f0b773d7640 1 -- 172.21.15.119:0/3510804026 wait complete. 2023-11-08T19:28:57.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.779+0000 7f0b773d7640 1 Processor -- start 2023-11-08T19:28:57.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.779+0000 7f0b773d7640 1 -- start start 2023-11-08T19:28:57.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.780+0000 7f0b773d7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7006a120 0x7f0b700c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.780+0000 7f0b773d7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b701425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.780+0000 7f0b773d7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b701447a0 0x7f0b70142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.780+0000 7f0b773d7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c7a70 con 0x7f0b700cad30 2023-11-08T19:28:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.780+0000 7f0b773d7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c6df0 con 0x7f0b7006a120 2023-11-08T19:28:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.780+0000 7f0b773d7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b700c70d0 con 0x7f0b701447a0 2023-11-08T19:28:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b701425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b701425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35668/0 (socket says 172.21.15.119:35668) 2023-11-08T19:28:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 -- 172.21.15.119:0/1494042776 learned_addr learned my addr 172.21.15.119:0/1494042776 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b75173640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7006a120 0x7f0b700c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 -- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b701447a0 msgr2=0x7f0b70142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b701447a0 0x7f0b70142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 -- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7006a120 msgr2=0x7f0b700c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7006a120 0x7f0b700c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 -- 172.21.15.119:0/1494042776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b600094d0 con 0x7f0b700cad30 2023-11-08T19:28:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.781+0000 7f0b74972640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b701425b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0b60000c00 tx=0x7f0b600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.782+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6003ad10 con 0x7f0b700cad30 2023-11-08T19:28:57.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.782+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6003ae70 con 0x7f0b700cad30 2023-11-08T19:28:57.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.782+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b700c73b0 con 0x7f0b700cad30 2023-11-08T19:28:57.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.782+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b60044610 con 0x7f0b700cad30 2023-11-08T19:28:57.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.782+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7013faf0 con 0x7f0b700cad30 2023-11-08T19:28:57.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.783+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b70074430 con 0x7f0b700cad30 2023-11-08T19:28:57.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.784+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b6004d450 con 0x7f0b700cad30 2023-11-08T19:28:57.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.785+0000 7f0b65ffb640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b5406ae70 0x7f0b5406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:57.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.785+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f0b6002d040 con 0x7f0b700cad30 2023-11-08T19:28:57.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.785+0000 7f0b75173640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b5406ae70 0x7f0b5406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:57.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.786+0000 7f0b75173640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b5406ae70 0x7f0b5406d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00a770 tx=0x7f0b6c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:57.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.789+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b60076d80 con 0x7f0b700cad30 2023-11-08T19:28:57.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:57.975+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b700c96f0 con 0x7f0b5406ae70 2023-11-08T19:28:58.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.001+0000 7f0b65ffb640 1 -- 172.21.15.119:0/1494042776 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+517488 (secure 0 0 0) 0x7f0b6c00c520 con 0x7f0b5406ae70 2023-11-08T19:28:58.010 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:58.013 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:28:58.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b5406ae70 msgr2=0x7f0b5406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b5406ae70 0x7f0b5406d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00a770 tx=0x7f0b6c008040 comp rx=0 tx=0).stop 2023-11-08T19:28:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 msgr2=0x7f0b701425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b701425b0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f0b60000c00 tx=0x7f0b600046c0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 shutdown_connections 2023-11-08T19:28:58.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0b5406ae70 0x7f0b5406d330 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0b701447a0 0x7f0b70142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0b700cad30 0x7f0b701425b0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 --2- 172.21.15.119:0/1494042776 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0b7006a120 0x7f0b700c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 >> 172.21.15.119:0/1494042776 conn(0x7f0b700cb610 msgr2=0x7f0b70143880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 shutdown_connections 2023-11-08T19:28:58.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.009+0000 7f0b773d7640 1 -- 172.21.15.119:0/1494042776 wait complete. 2023-11-08T19:28:58.030 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2023-11-08T19:28:56.741532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259548,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715172,"statfs":{"total":773094113280,"available":772828336128,"internally_reserved":0,"allocated":10342400,"data_stored":5691622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.741942"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.019781+0000","last_change":"2023-11-08T19:28:51.019781+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.019781+0000","last_undegraded":"2023-11-08T19:28:51.019781+0000","last_fullsized":"2023-11-08T19:28:51.019781+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898374+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:47.898374+0000","last_peered":"2023-11-08T19:28:47.898374+0000","last_clean":"2023-11-08T19:28:47.898374+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:47.898374+0000","last_undegraded":"2023-11-08T19:28:47.898374+0000","last_fullsized":"2023-11-08T19:28:47.898374+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897764+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:47.897764+0000","last_peered":"2023-11-08T19:28:47.897764+0000","last_clean":"2023-11-08T19:28:47.897764+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:47.897764+0000","last_undegraded":"2023-11-08T19:28:47.897764+0000","last_fullsized":"2023-11-08T19:28:47.897764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897826+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:47.897826+0000","last_peered":"2023-11-08T19:28:47.897826+0000","last_clean":"2023-11-08T19:28:47.897826+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:47.897826+0000","last_undegraded":"2023-11-08T19:28:47.897826+0000","last_fullsized":"2023-11-08T19:28:47.897826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.012025+0000","last_change":"2023-11-08T19:28:51.012025+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.012025+0000","last_undegraded":"2023-11-08T19:28:51.012025+0000","last_fullsized":"2023-11-08T19:28:51.012025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.981202+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:46.981202+0000","last_peered":"2023-11-08T19:28:46.981202+0000","last_clean":"2023-11-08T19:28:46.981202+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:46.981202+0000","last_undegraded":"2023-11-08T19:28:46.981202+0000","last_fullsized":"2023-11-08T19:28:46.981202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898666+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:47.898666+0000","last_peered":"2023-11-08T19:28:47.898666+0000","last_clean":"2023-11-08T19:28:47.898666+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:47.898666+0000","last_undegraded":"2023-11-08T19:28:47.898666+0000","last_fullsized":"2023-11-08T19:28:47.898666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898181+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:47.898181+0000","last_peered":"2023-11-08T19:28:47.898181+0000","last_clean":"2023-11-08T19:28:47.898181+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:47.898181+0000","last_undegraded":"2023-11-08T19:28:47.898181+0000","last_fullsized":"2023-11-08T19:28:47.898181+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.007233+0000","last_change":"2023-11-08T19:28:51.007233+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.007233+0000","last_undegraded":"2023-11-08T19:28:51.007233+0000","last_fullsized":"2023-11-08T19:28:51.007233+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984035+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:46.984035+0000","last_peered":"2023-11-08T19:28:46.984035+0000","last_clean":"2023-11-08T19:28:46.984035+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:46.984035+0000","last_undegraded":"2023-11-08T19:28:46.984035+0000","last_fullsized":"2023-11-08T19:28:46.984035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897644+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:47.897644+0000","last_peered":"2023-11-08T19:28:47.897644+0000","last_clean":"2023-11-08T19:28:47.897644+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:47.897644+0000","last_undegraded":"2023-11-08T19:28:47.897644+0000","last_fullsized":"2023-11-08T19:28:47.897644+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898929+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:47.898929+0000","last_peered":"2023-11-08T19:28:47.898929+0000","last_clean":"2023-11-08T19:28:47.898929+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:47.898929+0000","last_undegraded":"2023-11-08T19:28:47.898929+0000","last_fullsized":"2023-11-08T19:28:47.898929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.011921+0000","last_change":"2023-11-08T19:28:51.011921+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.011921+0000","last_undegraded":"2023-11-08T19:28:51.011921+0000","last_fullsized":"2023-11-08T19:28:51.011921+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073234+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:47.073234+0000","last_peered":"2023-11-08T19:28:47.073234+0000","last_clean":"2023-11-08T19:28:47.073234+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:47.073234+0000","last_undegraded":"2023-11-08T19:28:47.073234+0000","last_fullsized":"2023-11-08T19:28:47.073234+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897740+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:47.897740+0000","last_peered":"2023-11-08T19:28:47.897740+0000","last_clean":"2023-11-08T19:28:47.897740+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:47.897740+0000","last_undegraded":"2023-11-08T19:28:47.897740+0000","last_fullsized":"2023-11-08T19:28:47.897740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897705+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:47.897705+0000","last_peered":"2023-11-08T19:28:47.897705+0000","last_clean":"2023-11-08T19:28:47.897705+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:47.897705+0000","last_undegraded":"2023-11-08T19:28:47.897705+0000","last_fullsized":"2023-11-08T19:28:47.897705+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.021469+0000","last_change":"2023-11-08T19:28:51.021469+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.021469+0000","last_undegraded":"2023-11-08T19:28:51.021469+0000","last_fullsized":"2023-11-08T19:28:51.021469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982516+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:46.982516+0000","last_peered":"2023-11-08T19:28:46.982516+0000","last_clean":"2023-11-08T19:28:46.982516+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:46.982516+0000","last_undegraded":"2023-11-08T19:28:46.982516+0000","last_fullsized":"2023-11-08T19:28:46.982516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980580+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:46.980580+0000","last_peered":"2023-11-08T19:28:46.980580+0000","last_clean":"2023-11-08T19:28:46.980580+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:46.980580+0000","last_undegraded":"2023-11-08T19:28:46.980580+0000","last_fullsized":"2023-11-08T19:28:46.980580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898661+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:47.898661+0000","last_peered":"2023-11-08T19:28:47.898661+0000","last_clean":"2023-11-08T19:28:47.898661+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:47.898661+0000","last_undegraded":"2023-11-08T19:28:47.898661+0000","last_fullsized":"2023-11-08T19:28:47.898661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004832+0000","last_change":"2023-11-08T19:28:51.004832+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004832+0000","last_undegraded":"2023-11-08T19:28:51.004832+0000","last_fullsized":"2023-11-08T19:28:51.004832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073235+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:47.073235+0000","last_peered":"2023-11-08T19:28:47.073235+0000","last_clean":"2023-11-08T19:28:47.073235+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:47.073235+0000","last_undegraded":"2023-11-08T19:28:47.073235+0000","last_fullsized":"2023-11-08T19:28:47.073235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897438+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:47.897438+0000","last_peered":"2023-11-08T19:28:47.897438+0000","last_clean":"2023-11-08T19:28:47.897438+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:47.897438+0000","last_undegraded":"2023-11-08T19:28:47.897438+0000","last_fullsized":"2023-11-08T19:28:47.897438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073141+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:47.073141+0000","last_peered":"2023-11-08T19:28:47.073141+0000","last_clean":"2023-11-08T19:28:47.073141+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:47.073141+0000","last_undegraded":"2023-11-08T19:28:47.073141+0000","last_fullsized":"2023-11-08T19:28:47.073141+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004676+0000","last_change":"2023-11-08T19:28:51.004676+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004676+0000","last_undegraded":"2023-11-08T19:28:51.004676+0000","last_fullsized":"2023-11-08T19:28:51.004676+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897156+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:47.897156+0000","last_peered":"2023-11-08T19:28:47.897156+0000","last_clean":"2023-11-08T19:28:47.897156+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:47.897156+0000","last_undegraded":"2023-11-08T19:28:47.897156+0000","last_fullsized":"2023-11-08T19:28:47.897156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073059+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:47.073059+0000","last_peered":"2023-11-08T19:28:47.073059+0000","last_clean":"2023-11-08T19:28:47.073059+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:47.073059+0000","last_undegraded":"2023-11-08T19:28:47.073059+0000","last_fullsized":"2023-11-08T19:28:47.073059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073603+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:47.073603+0000","last_peered":"2023-11-08T19:28:47.073603+0000","last_clean":"2023-11-08T19:28:47.073603+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:47.073603+0000","last_undegraded":"2023-11-08T19:28:47.073603+0000","last_fullsized":"2023-11-08T19:28:47.073603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015351+0000","last_change":"2023-11-08T19:28:51.015351+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015351+0000","last_undegraded":"2023-11-08T19:28:51.015351+0000","last_fullsized":"2023-11-08T19:28:51.015351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897031+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:47.897031+0000","last_peered":"2023-11-08T19:28:47.897031+0000","last_clean":"2023-11-08T19:28:47.897031+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:47.897031+0000","last_undegraded":"2023-11-08T19:28:47.897031+0000","last_fullsized":"2023-11-08T19:28:47.897031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897639+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:47.897639+0000","last_peered":"2023-11-08T19:28:47.897639+0000","last_clean":"2023-11-08T19:28:47.897639+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:47.897639+0000","last_undegraded":"2023-11-08T19:28:47.897639+0000","last_fullsized":"2023-11-08T19:28:47.897639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983750+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:46.983750+0000","last_peered":"2023-11-08T19:28:46.983750+0000","last_clean":"2023-11-08T19:28:46.983750+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:46.983750+0000","last_undegraded":"2023-11-08T19:28:46.983750+0000","last_fullsized":"2023-11-08T19:28:46.983750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.009689+0000","last_change":"2023-11-08T19:28:51.009689+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.009689+0000","last_undegraded":"2023-11-08T19:28:51.009689+0000","last_fullsized":"2023-11-08T19:28:51.009689+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898820+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:47.898820+0000","last_peered":"2023-11-08T19:28:47.898820+0000","last_clean":"2023-11-08T19:28:47.898820+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:47.898820+0000","last_undegraded":"2023-11-08T19:28:47.898820+0000","last_fullsized":"2023-11-08T19:28:47.898820+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984389+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:46.984389+0000","last_peered":"2023-11-08T19:28:46.984389+0000","last_clean":"2023-11-08T19:28:46.984389+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:46.984389+0000","last_undegraded":"2023-11-08T19:28:46.984389+0000","last_fullsized":"2023-11-08T19:28:46.984389+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983524+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:46.983524+0000","last_peered":"2023-11-08T19:28:46.983524+0000","last_clean":"2023-11-08T19:28:46.983524+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:46.983524+0000","last_undegraded":"2023-11-08T19:28:46.983524+0000","last_fullsized":"2023-11-08T19:28:46.983524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005396+0000","last_change":"2023-11-08T19:28:51.005396+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005396+0000","last_undegraded":"2023-11-08T19:28:51.005396+0000","last_fullsized":"2023-11-08T19:28:51.005396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984268+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:46.984268+0000","last_peered":"2023-11-08T19:28:46.984268+0000","last_clean":"2023-11-08T19:28:46.984268+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:46.984268+0000","last_undegraded":"2023-11-08T19:28:46.984268+0000","last_fullsized":"2023-11-08T19:28:46.984268+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898588+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:47.898588+0000","last_peered":"2023-11-08T19:28:47.898588+0000","last_clean":"2023-11-08T19:28:47.898588+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:47.898588+0000","last_undegraded":"2023-11-08T19:28:47.898588+0000","last_fullsized":"2023-11-08T19:28:47.898588+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980541+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:46.980541+0000","last_peered":"2023-11-08T19:28:46.980541+0000","last_clean":"2023-11-08T19:28:46.980541+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:46.980541+0000","last_undegraded":"2023-11-08T19:28:46.980541+0000","last_fullsized":"2023-11-08T19:28:46.980541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005496+0000","last_change":"2023-11-08T19:28:51.005496+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005496+0000","last_undegraded":"2023-11-08T19:28:51.005496+0000","last_fullsized":"2023-11-08T19:28:51.005496+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982650+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:46.982650+0000","last_peered":"2023-11-08T19:28:46.982650+0000","last_clean":"2023-11-08T19:28:46.982650+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:46.982650+0000","last_undegraded":"2023-11-08T19:28:46.982650+0000","last_fullsized":"2023-11-08T19:28:46.982650+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073342+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:47.073342+0000","last_peered":"2023-11-08T19:28:47.073342+0000","last_clean":"2023-11-08T19:28:47.073342+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:47.073342+0000","last_undegraded":"2023-11-08T19:28:47.073342+0000","last_fullsized":"2023-11-08T19:28:47.073342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983351+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:46.983351+0000","last_peered":"2023-11-08T19:28:46.983351+0000","last_clean":"2023-11-08T19:28:46.983351+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:46.983351+0000","last_undegraded":"2023-11-08T19:28:46.983351+0000","last_fullsized":"2023-11-08T19:28:46.983351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.020306+0000","last_change":"2023-11-08T19:28:51.020306+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.020306+0000","last_undegraded":"2023-11-08T19:28:51.020306+0000","last_fullsized":"2023-11-08T19:28:51.020306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897863+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:47.897863+0000","last_peered":"2023-11-08T19:28:47.897863+0000","last_clean":"2023-11-08T19:28:47.897863+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:47.897863+0000","last_undegraded":"2023-11-08T19:28:47.897863+0000","last_fullsized":"2023-11-08T19:28:47.897863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073430+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:47.073430+0000","last_peered":"2023-11-08T19:28:47.073430+0000","last_clean":"2023-11-08T19:28:47.073430+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:47.073430+0000","last_undegraded":"2023-11-08T19:28:47.073430+0000","last_fullsized":"2023-11-08T19:28:47.073430+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984510+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:46.984510+0000","last_peered":"2023-11-08T19:28:46.984510+0000","last_clean":"2023-11-08T19:28:46.984510+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:46.984510+0000","last_undegraded":"2023-11-08T19:28:46.984510+0000","last_fullsized":"2023-11-08T19:28:46.984510+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015226+0000","last_change":"2023-11-08T19:28:51.015226+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015226+0000","last_undegraded":"2023-11-08T19:28:51.015226+0000","last_fullsized":"2023-11-08T19:28:51.015226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983868+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:46.983868+0000","last_peered":"2023-11-08T19:28:46.983868+0000","last_clean":"2023-11-08T19:28:46.983868+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:46.983868+0000","last_undegraded":"2023-11-08T19:28:46.983868+0000","last_fullsized":"2023-11-08T19:28:46.983868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983310+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:46.983310+0000","last_peered":"2023-11-08T19:28:46.983310+0000","last_clean":"2023-11-08T19:28:46.983310+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:46.983310+0000","last_undegraded":"2023-11-08T19:28:46.983310+0000","last_fullsized":"2023-11-08T19:28:46.983310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982243+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:46.982243+0000","last_peered":"2023-11-08T19:28:46.982243+0000","last_clean":"2023-11-08T19:28:46.982243+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:46.982243+0000","last_undegraded":"2023-11-08T19:28:46.982243+0000","last_fullsized":"2023-11-08T19:28:46.982243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.010187+0000","last_change":"2023-11-08T19:28:51.010187+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.010187+0000","last_undegraded":"2023-11-08T19:28:51.010187+0000","last_fullsized":"2023-11-08T19:28:51.010187+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897613+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:47.897613+0000","last_peered":"2023-11-08T19:28:47.897613+0000","last_clean":"2023-11-08T19:28:47.897613+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:47.897613+0000","last_undegraded":"2023-11-08T19:28:47.897613+0000","last_fullsized":"2023-11-08T19:28:47.897613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984004+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:46.984004+0000","last_peered":"2023-11-08T19:28:46.984004+0000","last_clean":"2023-11-08T19:28:46.984004+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:46.984004+0000","last_undegraded":"2023-11-08T19:28:46.984004+0000","last_fullsized":"2023-11-08T19:28:46.984004+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980151+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:46.980151+0000","last_peered":"2023-11-08T19:28:46.980151+0000","last_clean":"2023-11-08T19:28:46.980151+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:46.980151+0000","last_undegraded":"2023-11-08T19:28:46.980151+0000","last_fullsized":"2023-11-08T19:28:46.980151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.010260+0000","last_change":"2023-11-08T19:28:51.010260+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.010260+0000","last_undegraded":"2023-11-08T19:28:51.010260+0000","last_fullsized":"2023-11-08T19:28:51.010260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980042+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:46.980042+0000","last_peered":"2023-11-08T19:28:46.980042+0000","last_clean":"2023-11-08T19:28:46.980042+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:46.980042+0000","last_undegraded":"2023-11-08T19:28:46.980042+0000","last_fullsized":"2023-11-08T19:28:46.980042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897127+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:47.897127+0000","last_peered":"2023-11-08T19:28:47.897127+0000","last_clean":"2023-11-08T19:28:47.897127+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:47.897127+0000","last_undegraded":"2023-11-08T19:28:47.897127+0000","last_fullsized":"2023-11-08T19:28:47.897127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073742+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:47.073742+0000","last_peered":"2023-11-08T19:28:47.073742+0000","last_clean":"2023-11-08T19:28:47.073742+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:47.073742+0000","last_undegraded":"2023-11-08T19:28:47.073742+0000","last_fullsized":"2023-11-08T19:28:47.073742+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005943+0000","last_change":"2023-11-08T19:28:51.005943+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005943+0000","last_undegraded":"2023-11-08T19:28:51.005943+0000","last_fullsized":"2023-11-08T19:28:51.005943+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982027+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:46.982027+0000","last_peered":"2023-11-08T19:28:46.982027+0000","last_clean":"2023-11-08T19:28:46.982027+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:46.982027+0000","last_undegraded":"2023-11-08T19:28:46.982027+0000","last_fullsized":"2023-11-08T19:28:46.982027+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073176+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:47.073176+0000","last_peered":"2023-11-08T19:28:47.073176+0000","last_clean":"2023-11-08T19:28:47.073176+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:47.073176+0000","last_undegraded":"2023-11-08T19:28:47.073176+0000","last_fullsized":"2023-11-08T19:28:47.073176+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897468+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:47.897468+0000","last_peered":"2023-11-08T19:28:47.897468+0000","last_clean":"2023-11-08T19:28:47.897468+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:47.897468+0000","last_undegraded":"2023-11-08T19:28:47.897468+0000","last_fullsized":"2023-11-08T19:28:47.897468+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.009642+0000","last_change":"2023-11-08T19:28:51.009642+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.009642+0000","last_undegraded":"2023-11-08T19:28:51.009642+0000","last_fullsized":"2023-11-08T19:28:51.009642+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983068+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:46.983068+0000","last_peered":"2023-11-08T19:28:46.983068+0000","last_clean":"2023-11-08T19:28:46.983068+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:46.983068+0000","last_undegraded":"2023-11-08T19:28:46.983068+0000","last_fullsized":"2023-11-08T19:28:46.983068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897599+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:47.897599+0000","last_peered":"2023-11-08T19:28:47.897599+0000","last_clean":"2023-11-08T19:28:47.897599+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:47.897599+0000","last_undegraded":"2023-11-08T19:28:47.897599+0000","last_fullsized":"2023-11-08T19:28:47.897599+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898064+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:47.898064+0000","last_peered":"2023-11-08T19:28:47.898064+0000","last_clean":"2023-11-08T19:28:47.898064+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:47.898064+0000","last_undegraded":"2023-11-08T19:28:47.898064+0000","last_fullsized":"2023-11-08T19:28:47.898064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.006470+0000","last_change":"2023-11-08T19:28:51.006470+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.006470+0000","last_undegraded":"2023-11-08T19:28:51.006470+0000","last_fullsized":"2023-11-08T19:28:51.006470+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984345+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:46.984345+0000","last_peered":"2023-11-08T19:28:46.984345+0000","last_clean":"2023-11-08T19:28:46.984345+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:46.984345+0000","last_undegraded":"2023-11-08T19:28:46.984345+0000","last_fullsized":"2023-11-08T19:28:46.984345+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897497+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:47.897497+0000","last_peered":"2023-11-08T19:28:47.897497+0000","last_clean":"2023-11-08T19:28:47.897497+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:47.897497+0000","last_undegraded":"2023-11-08T19:28:47.897497+0000","last_fullsized":"2023-11-08T19:28:47.897497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898470+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:47.898470+0000","last_peered":"2023-11-08T19:28:47.898470+0000","last_clean":"2023-11-08T19:28:47.898470+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:47.898470+0000","last_undegraded":"2023-11-08T19:28:47.898470+0000","last_fullsized":"2023-11-08T19:28:47.898470+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.008922+0000","last_change":"2023-11-08T19:28:51.008922+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.008922+0000","last_undegraded":"2023-11-08T19:28:51.008922+0000","last_fullsized":"2023-11-08T19:28:51.008922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984247+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:46.984247+0000","last_peered":"2023-11-08T19:28:46.984247+0000","last_clean":"2023-11-08T19:28:46.984247+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:46.984247+0000","last_undegraded":"2023-11-08T19:28:46.984247+0000","last_fullsized":"2023-11-08T19:28:46.984247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898061+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:47.898061+0000","last_peered":"2023-11-08T19:28:47.898061+0000","last_clean":"2023-11-08T19:28:47.898061+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:47.898061+0000","last_undegraded":"2023-11-08T19:28:47.898061+0000","last_fullsized":"2023-11-08T19:28:47.898061+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073685+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:47.073685+0000","last_peered":"2023-11-08T19:28:47.073685+0000","last_clean":"2023-11-08T19:28:47.073685+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:47.073685+0000","last_undegraded":"2023-11-08T19:28:47.073685+0000","last_fullsized":"2023-11-08T19:28:47.073685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005345+0000","last_change":"2023-11-08T19:28:51.005345+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005345+0000","last_undegraded":"2023-11-08T19:28:51.005345+0000","last_fullsized":"2023-11-08T19:28:51.005345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898260+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:47.898260+0000","last_peered":"2023-11-08T19:28:47.898260+0000","last_clean":"2023-11-08T19:28:47.898260+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:47.898260+0000","last_undegraded":"2023-11-08T19:28:47.898260+0000","last_fullsized":"2023-11-08T19:28:47.898260+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897549+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:47.897549+0000","last_peered":"2023-11-08T19:28:47.897549+0000","last_clean":"2023-11-08T19:28:47.897549+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:47.897549+0000","last_undegraded":"2023-11-08T19:28:47.897549+0000","last_fullsized":"2023-11-08T19:28:47.897549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759631+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:47.759631+0000","last_peered":"2023-11-08T19:28:47.759631+0000","last_clean":"2023-11-08T19:28:47.759631+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:47.759631+0000","last_undegraded":"2023-11-08T19:28:47.759631+0000","last_fullsized":"2023-11-08T19:28:47.759631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.016804+0000","last_change":"2023-11-08T19:28:51.016804+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.016804+0000","last_undegraded":"2023-11-08T19:28:51.016804+0000","last_fullsized":"2023-11-08T19:28:51.016804+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073498+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:47.073498+0000","last_peered":"2023-11-08T19:28:47.073498+0000","last_clean":"2023-11-08T19:28:47.073498+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:47.073498+0000","last_undegraded":"2023-11-08T19:28:47.073498+0000","last_fullsized":"2023-11-08T19:28:47.073498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983546+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:46.983546+0000","last_peered":"2023-11-08T19:28:46.983546+0000","last_clean":"2023-11-08T19:28:46.983546+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:46.983546+0000","last_undegraded":"2023-11-08T19:28:46.983546+0000","last_fullsized":"2023-11-08T19:28:46.983546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.003018+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:51.003018+0000","last_peered":"2023-11-08T19:28:51.003018+0000","last_clean":"2023-11-08T19:28:51.003018+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:51.003018+0000","last_undegraded":"2023-11-08T19:28:51.003018+0000","last_fullsized":"2023-11-08T19:28:51.003018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004718+0000","last_change":"2023-11-08T19:28:51.004718+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004718+0000","last_undegraded":"2023-11-08T19:28:51.004718+0000","last_fullsized":"2023-11-08T19:28:51.004718+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897253+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:47.897253+0000","last_peered":"2023-11-08T19:28:47.897253+0000","last_clean":"2023-11-08T19:28:47.897253+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:47.897253+0000","last_undegraded":"2023-11-08T19:28:47.897253+0000","last_fullsized":"2023-11-08T19:28:47.897253+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073589+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:47.073589+0000","last_peered":"2023-11-08T19:28:47.073589+0000","last_clean":"2023-11-08T19:28:47.073589+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:47.073589+0000","last_undegraded":"2023-11-08T19:28:47.073589+0000","last_fullsized":"2023-11-08T19:28:47.073589+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.019899+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:51.019899+0000","last_peered":"2023-11-08T19:28:51.019899+0000","last_clean":"2023-11-08T19:28:51.019899+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:51.019899+0000","last_undegraded":"2023-11-08T19:28:51.019899+0000","last_fullsized":"2023-11-08T19:28:51.019899+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.020099+0000","last_change":"2023-11-08T19:28:51.020099+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.020099+0000","last_undegraded":"2023-11-08T19:28:51.020099+0000","last_fullsized":"2023-11-08T19:28:51.020099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897718+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:47.897718+0000","last_peered":"2023-11-08T19:28:47.897718+0000","last_clean":"2023-11-08T19:28:47.897718+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:47.897718+0000","last_undegraded":"2023-11-08T19:28:47.897718+0000","last_fullsized":"2023-11-08T19:28:47.897718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983268+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:46.983268+0000","last_peered":"2023-11-08T19:28:46.983268+0000","last_clean":"2023-11-08T19:28:46.983268+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:46.983268+0000","last_undegraded":"2023-11-08T19:28:46.983268+0000","last_fullsized":"2023-11-08T19:28:46.983268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897520+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:47.897520+0000","last_peered":"2023-11-08T19:28:47.897520+0000","last_clean":"2023-11-08T19:28:47.897520+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:47.897520+0000","last_undegraded":"2023-11-08T19:28:47.897520+0000","last_fullsized":"2023-11-08T19:28:47.897520+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.021701+0000","last_change":"2023-11-08T19:28:51.021701+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.021701+0000","last_undegraded":"2023-11-08T19:28:51.021701+0000","last_fullsized":"2023-11-08T19:28:51.021701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982262+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:46.982262+0000","last_peered":"2023-11-08T19:28:46.982262+0000","last_clean":"2023-11-08T19:28:46.982262+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:46.982262+0000","last_undegraded":"2023-11-08T19:28:46.982262+0000","last_fullsized":"2023-11-08T19:28:46.982262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898086+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:47.898086+0000","last_peered":"2023-11-08T19:28:47.898086+0000","last_clean":"2023-11-08T19:28:47.898086+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:47.898086+0000","last_undegraded":"2023-11-08T19:28:47.898086+0000","last_fullsized":"2023-11-08T19:28:47.898086+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.013666+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:51.013666+0000","last_peered":"2023-11-08T19:28:51.013666+0000","last_clean":"2023-11-08T19:28:51.013666+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:51.013666+0000","last_undegraded":"2023-11-08T19:28:51.013666+0000","last_fullsized":"2023-11-08T19:28:51.013666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015270+0000","last_change":"2023-11-08T19:28:51.015270+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015270+0000","last_undegraded":"2023-11-08T19:28:51.015270+0000","last_fullsized":"2023-11-08T19:28:51.015270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898860+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:47.898860+0000","last_peered":"2023-11-08T19:28:47.898860+0000","last_clean":"2023-11-08T19:28:47.898860+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:47.898860+0000","last_undegraded":"2023-11-08T19:28:47.898860+0000","last_fullsized":"2023-11-08T19:28:47.898860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073478+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:47.073478+0000","last_peered":"2023-11-08T19:28:47.073478+0000","last_clean":"2023-11-08T19:28:47.073478+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:47.073478+0000","last_undegraded":"2023-11-08T19:28:47.073478+0000","last_fullsized":"2023-11-08T19:28:47.073478+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983063+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:46.983063+0000","last_peered":"2023-11-08T19:28:46.983063+0000","last_clean":"2023-11-08T19:28:46.983063+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:46.983063+0000","last_undegraded":"2023-11-08T19:28:46.983063+0000","last_fullsized":"2023-11-08T19:28:46.983063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.012488+0000","last_change":"2023-11-08T19:28:51.012488+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.012488+0000","last_undegraded":"2023-11-08T19:28:51.012488+0000","last_fullsized":"2023-11-08T19:28:51.012488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982795+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:46.982795+0000","last_peered":"2023-11-08T19:28:46.982795+0000","last_clean":"2023-11-08T19:28:46.982795+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:46.982795+0000","last_undegraded":"2023-11-08T19:28:46.982795+0000","last_fullsized":"2023-11-08T19:28:46.982795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898720+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:47.898720+0000","last_peered":"2023-11-08T19:28:47.898720+0000","last_clean":"2023-11-08T19:28:47.898720+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:47.898720+0000","last_undegraded":"2023-11-08T19:28:47.898720+0000","last_fullsized":"2023-11-08T19:28:47.898720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073382+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:47.073382+0000","last_peered":"2023-11-08T19:28:47.073382+0000","last_clean":"2023-11-08T19:28:47.073382+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:47.073382+0000","last_undegraded":"2023-11-08T19:28:47.073382+0000","last_fullsized":"2023-11-08T19:28:47.073382+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.018696+0000","last_change":"2023-11-08T19:28:51.018696+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.018696+0000","last_undegraded":"2023-11-08T19:28:51.018696+0000","last_fullsized":"2023-11-08T19:28:51.018696+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984117+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:46.984117+0000","last_peered":"2023-11-08T19:28:46.984117+0000","last_clean":"2023-11-08T19:28:46.984117+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:46.984117+0000","last_undegraded":"2023-11-08T19:28:46.984117+0000","last_fullsized":"2023-11-08T19:28:46.984117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073301+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:47.073301+0000","last_peered":"2023-11-08T19:28:47.073301+0000","last_clean":"2023-11-08T19:28:47.073301+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:47.073301+0000","last_undegraded":"2023-11-08T19:28:47.073301+0000","last_fullsized":"2023-11-08T19:28:47.073301+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.017867+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:51.017867+0000","last_peered":"2023-11-08T19:28:51.017867+0000","last_clean":"2023-11-08T19:28:51.017867+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:51.017867+0000","last_undegraded":"2023-11-08T19:28:51.017867+0000","last_fullsized":"2023-11-08T19:28:51.017867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.008313+0000","last_change":"2023-11-08T19:28:51.008313+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.008313+0000","last_undegraded":"2023-11-08T19:28:51.008313+0000","last_fullsized":"2023-11-08T19:28:51.008313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897392+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:47.897392+0000","last_peered":"2023-11-08T19:28:47.897392+0000","last_clean":"2023-11-08T19:28:47.897392+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:47.897392+0000","last_undegraded":"2023-11-08T19:28:47.897392+0000","last_fullsized":"2023-11-08T19:28:47.897392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759820+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:47.759820+0000","last_peered":"2023-11-08T19:28:47.759820+0000","last_clean":"2023-11-08T19:28:47.759820+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:47.759820+0000","last_undegraded":"2023-11-08T19:28:47.759820+0000","last_fullsized":"2023-11-08T19:28:47.759820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983608+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:46.983608+0000","last_peered":"2023-11-08T19:28:46.983608+0000","last_clean":"2023-11-08T19:28:46.983608+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:46.983608+0000","last_undegraded":"2023-11-08T19:28:46.983608+0000","last_fullsized":"2023-11-08T19:28:46.983608+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.019995+0000","last_change":"2023-11-08T19:28:51.019995+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.019995+0000","last_undegraded":"2023-11-08T19:28:51.019995+0000","last_fullsized":"2023-11-08T19:28:51.019995+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759877+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:47.759877+0000","last_peered":"2023-11-08T19:28:47.759877+0000","last_clean":"2023-11-08T19:28:47.759877+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:47.759877+0000","last_undegraded":"2023-11-08T19:28:47.759877+0000","last_fullsized":"2023-11-08T19:28:47.759877+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980860+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:46.980860+0000","last_peered":"2023-11-08T19:28:46.980860+0000","last_clean":"2023-11-08T19:28:46.980860+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:46.980860+0000","last_undegraded":"2023-11-08T19:28:46.980860+0000","last_fullsized":"2023-11-08T19:28:46.980860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897035+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:47.897035+0000","last_peered":"2023-11-08T19:28:47.897035+0000","last_clean":"2023-11-08T19:28:47.897035+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:47.897035+0000","last_undegraded":"2023-11-08T19:28:47.897035+0000","last_fullsized":"2023-11-08T19:28:47.897035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004935+0000","last_change":"2023-11-08T19:28:51.004935+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004935+0000","last_undegraded":"2023-11-08T19:28:51.004935+0000","last_fullsized":"2023-11-08T19:28:51.004935+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897809+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:47.897809+0000","last_peered":"2023-11-08T19:28:47.897809+0000","last_clean":"2023-11-08T19:28:47.897809+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:47.897809+0000","last_undegraded":"2023-11-08T19:28:47.897809+0000","last_fullsized":"2023-11-08T19:28:47.897809+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980029+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:46.980029+0000","last_peered":"2023-11-08T19:28:46.980029+0000","last_clean":"2023-11-08T19:28:46.980029+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:46.980029+0000","last_undegraded":"2023-11-08T19:28:46.980029+0000","last_fullsized":"2023-11-08T19:28:46.980029+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.981009+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:46.981009+0000","last_peered":"2023-11-08T19:28:46.981009+0000","last_clean":"2023-11-08T19:28:46.981009+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:46.981009+0000","last_undegraded":"2023-11-08T19:28:46.981009+0000","last_fullsized":"2023-11-08T19:28:46.981009+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.007275+0000","last_change":"2023-11-08T19:28:51.007275+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.007275+0000","last_undegraded":"2023-11-08T19:28:51.007275+0000","last_fullsized":"2023-11-08T19:28:51.007275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759559+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:47.759559+0000","last_peered":"2023-11-08T19:28:47.759559+0000","last_clean":"2023-11-08T19:28:47.759559+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:47.759559+0000","last_undegraded":"2023-11-08T19:28:47.759559+0000","last_fullsized":"2023-11-08T19:28:47.759559+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759562+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:47.759562+0000","last_peered":"2023-11-08T19:28:47.759562+0000","last_clean":"2023-11-08T19:28:47.759562+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:47.759562+0000","last_undegraded":"2023-11-08T19:28:47.759562+0000","last_fullsized":"2023-11-08T19:28:47.759562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980880+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:46.980880+0000","last_peered":"2023-11-08T19:28:46.980880+0000","last_clean":"2023-11-08T19:28:46.980880+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:46.980880+0000","last_undegraded":"2023-11-08T19:28:46.980880+0000","last_fullsized":"2023-11-08T19:28:46.980880+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005157+0000","last_change":"2023-11-08T19:28:51.005157+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005157+0000","last_undegraded":"2023-11-08T19:28:51.005157+0000","last_fullsized":"2023-11-08T19:28:51.005157+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897246+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:47.897246+0000","last_peered":"2023-11-08T19:28:47.897246+0000","last_clean":"2023-11-08T19:28:47.897246+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:47.897246+0000","last_undegraded":"2023-11-08T19:28:47.897246+0000","last_fullsized":"2023-11-08T19:28:47.897246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982860+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:46.982860+0000","last_peered":"2023-11-08T19:28:46.982860+0000","last_clean":"2023-11-08T19:28:46.982860+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:46.982860+0000","last_undegraded":"2023-11-08T19:28:46.982860+0000","last_fullsized":"2023-11-08T19:28:46.982860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.004218+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:51.004218+0000","last_peered":"2023-11-08T19:28:51.004218+0000","last_clean":"2023-11-08T19:28:51.004218+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:51.004218+0000","last_undegraded":"2023-11-08T19:28:51.004218+0000","last_fullsized":"2023-11-08T19:28:51.004218+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.018351+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:51.018351+0000","last_peered":"2023-11-08T19:28:51.018351+0000","last_clean":"2023-11-08T19:28:51.018351+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:51.018351+0000","last_undegraded":"2023-11-08T19:28:51.018351+0000","last_fullsized":"2023-11-08T19:28:51.018351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":1155387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.52,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.141,"5min":1.25,"15min":1.3120000000000001},"min":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1360000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.74399999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.196,"15min":1.2569999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.518,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.46},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.125,"15min":1.1359999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0049999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.405,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.056,"5min":1.28,"15min":1.3620000000000001},"min":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.389,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.94999999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3129999999999999,"15min":1.367},"min":{"1min":0.79000000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.591,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.284},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.506,"15min":1.5700000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2200000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.167}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.798,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.056},{"interface":"front","average":{"1min":1.321,"5min":1.427,"15min":1.4970000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.774,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.73299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.359},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7829999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3220000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.546,"15min":1.579},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.9770000000000001,"5min":2.427,"15min":2.427},"last":1.234},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.5680000000000001,"15min":1.631},"min":{"1min":0.997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.8939999999999999,"5min":2.468,"15min":2.468},"last":1.401}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96999999999999997,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6279999999999999,"5min":1.752,"15min":1.752},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.1140000000000001,"15min":1.1060000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.581,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.095,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.47,"5min":1.679,"15min":1.679},"last":1.411},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.2789999999999999,"15min":1.337},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6559999999999999,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.999}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.238,"15min":1.292},"min":{"1min":0.68799999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4359999999999999,"5min":4.609,"15min":4.609},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.861,"15min":1.861},"last":1.177}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4350000000000001,"15min":1.4870000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.756,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.351},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1819999999999999,"15min":1.181},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.482,"5min":1.784,"15min":1.784},"last":1.2869999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.371,"15min":1.377},"min":{"1min":0.79600000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.732,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.518,"15min":1.5029999999999999},"min":{"1min":0.89500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9510000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1180000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7829999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.073},{"interface":"front","average":{"1min":1.403,"5min":1.347,"15min":1.335},"min":{"1min":0.83099999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8660000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5800000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5669999999999999,"15min":1.55},"min":{"1min":0.85899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8759999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.639},{"interface":"front","average":{"1min":1.512,"5min":1.454,"15min":1.454},"min":{"1min":0.81499999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8779999999999999,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.464}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.093,"15min":1.0740000000000001},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6990000000000001,"5min":1.76,"15min":1.76},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.234},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.258}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.141,"15min":1.1699999999999999},"min":{"1min":0.753,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5349999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.508},{"interface":"front","average":{"1min":1.181,"5min":1.343,"15min":1.4410000000000001},"min":{"1min":0.76200000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.617,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.0469999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1830000000000001,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.639,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3939999999999999,"15min":1.4990000000000001},"min":{"1min":0.94299999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.806,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.1000000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.288},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.804,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3109999999999999,"15min":1.3049999999999999},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.85399999999999998,"5min":0.755,"15min":0.755},"max":{"1min":1.635,"5min":2.347,"15min":2.347},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5149999999999999,"15min":1.5529999999999999},"min":{"1min":1.0589999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.097,"5min":3.153,"15min":3.153},"last":1.6319999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4930000000000001,"15min":1.528},"min":{"1min":0.89100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.8360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.5009999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.899,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.8029999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.6399999999999999,"15min":1.7},"min":{"1min":1.03,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0449999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.718},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.581,"15min":1.5900000000000001},"min":{"1min":1.1439999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.831,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.883}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0549999999999999,"15min":1.0469999999999999},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.496,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2549999999999999,"15min":1.2090000000000001},"min":{"1min":0.85699999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.696,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.069,"15min":1.091},"min":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1279999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.89100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.7450000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.28}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4610000000000001,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.423},"min":{"1min":0.82099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.391}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.696,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3600000000000001,"15min":1.387},"min":{"1min":0.83599999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.629,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4399999999999999,"15min":1.5129999999999999},"min":{"1min":0.96699999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.667,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.4970000000000001,"15min":1.5489999999999999},"min":{"1min":0.95899999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8560000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5229999999999999,"15min":1.5620000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.782,"5min":4.335,"15min":4.335},"last":1.218},{"interface":"front","average":{"1min":1.52,"5min":1.5880000000000001,"15min":1.6259999999999999},"min":{"1min":1.3140000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8080000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.605}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.6040000000000001,"15min":1.631},"min":{"1min":1.1799999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.77,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.6859999999999999,"15min":1.732},"min":{"1min":1.127,"5min":1.095,"15min":1.095},"max":{"1min":1.901,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.327}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.98599999999999999,"15min":1.0720000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":3.972,"15min":3.972},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.032,"15min":1.095},"min":{"1min":0.372,"5min":0.311,"15min":0.311},"max":{"1min":1.899,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.103,"15min":1.1850000000000001},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2609999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1080000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.8520000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2709999999999999,"15min":1.325},"min":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6739999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.448},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2669999999999999},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.754,"5min":2.73,"15min":2.73},"last":1.5029999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3740000000000001,"15min":1.448},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.663,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.143},{"interface":"front","average":{"1min":1.204,"5min":1.393,"15min":1.452},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":3.052,"15min":3.052},"last":1.204}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4339999999999999,"15min":1.498},"min":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0430000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.4670000000000001,"15min":1.5309999999999999},"min":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.948,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.878,"15min":0.878},"max":{"1min":3.21,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.5269999999999999,"15min":1.585},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0880000000000001,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.403}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6140000000000001,"15min":1.71},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.948,"5min":4.407,"15min":4.407},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.528,"15min":1.581},"min":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0059999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":564000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.86299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.879,"15min":0.92700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.171,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.198,"5min":1.393,"15min":1.393},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":1.016},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4690000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.032}]},{"osd":3,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.022,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0600000000000001,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6060000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.407},"min":{"1min":0.81499999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2969999999999999,"15min":1.365},"min":{"1min":0.81999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7,"5min":2.278,"15min":2.278},"last":1.4450000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.425},"min":{"1min":0.76600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1600000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.373,"15min":1.4610000000000001},"min":{"1min":0.78500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.954,"5min":2.536,"15min":2.536},"last":1.3799999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.347,"15min":1.389},"min":{"1min":0.93100000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9119999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.548},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.036,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.3460000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4930000000000001,"15min":1.577},"min":{"1min":0.85799999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.861,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.492,"15min":1.556},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.0030000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.639}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.745},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.304,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.123,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.1519999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.65000000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.0700000000000001,"15min":1.0860000000000001},"min":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2690000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.276}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.1539999999999999},"min":{"1min":0.78900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.556,"5min":2.165,"15min":2.165},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.252},"min":{"1min":0.77400000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.454,"5min":1.885,"15min":1.885},"last":1.7589999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2549999999999999},"min":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.274,"15min":1.321},"min":{"1min":0.78900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.579,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.5169999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3580000000000001,"15min":1.3660000000000001},"min":{"1min":0.84999999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1509999999999998,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.92400000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.724,"5min":2.282,"15min":2.282},"last":1.6279999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3200000000000001,"15min":1.3300000000000001},"min":{"1min":1.109,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.909,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4830000000000001,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.615}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1147140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.296,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.872,"15min":0.92800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3180000000000001,"5min":1.7,"15min":1.7},"last":0.92900000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.86599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.284,"5min":1.508,"15min":1.508},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.97199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.407,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.256}]},{"osd":3,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.6599999999999999,"5min":1.748,"15min":1.748},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0409999999999999,"15min":1.099},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.506,"5min":2.496,"15min":2.496},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5609999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.206},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3049999999999999,"15min":1.4199999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.54,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3979999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.286,"15min":1.3600000000000001},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.516},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.355,"15min":1.4330000000000001},"min":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.534}]},{"osd":6,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4870000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1619999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3540000000000001,"15min":1.405},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.3660000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.47},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.405,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.881,"5min":2.306,"15min":2.306},"last":1.702}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:58.040 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:58.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.290+0000 7eff76075640 1 Processor -- start 2023-11-08T19:28:58.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.290+0000 7eff76075640 1 -- start start 2023-11-08T19:28:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.292+0000 7eff76075640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 0x7eff700c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.292+0000 7eff76075640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff700ae410 con 0x7eff700cc810 2023-11-08T19:28:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.292+0000 7eff76075640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff7006ba30 con 0x7eff7006a390 2023-11-08T19:28:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.292+0000 7eff76075640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7006bcf0 con 0x7eff7015d180 2023-11-08T19:28:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.292+0000 7eff66ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7eff700cc810 0x7eff70160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:39674/0 (socket says 172.21.15.119:39674) 2023-11-08T19:28:58.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.292+0000 7eff66ffd640 1 -- 172.21.15.119:0/3403642244 learned_addr learned my addr 172.21.15.119:0/3403642244 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:58.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6ffff640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 0x7eff700c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090277808 0 0) 0x7eff700ae410 con 0x7eff700cc810 2023-11-08T19:28:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff5c003520 con 0x7eff700cc810 2023-11-08T19:28:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2518946860 0 0) 0x7eff5c003520 con 0x7eff700cc810 2023-11-08T19:28:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 msgr2=0x7eff700c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:58.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 0x7eff700c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 >> v1:172.21.15.188:6789/0 conn(0x7eff7006a390 legacy=0x7eff7015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7006bfd0 con 0x7eff700cc810 2023-11-08T19:28:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7eff58001670 con 0x7eff700cc810 2023-11-08T19:28:58.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 >> v1:172.21.15.119:6789/0 conn(0x7eff700cc810 legacy=0x7eff70160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff5c0039a0 0x7eff5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 0x7eff5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c004d00 0x7eff5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7006bcf0 con 0x7eff5c0039a0 2023-11-08T19:28:58.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c00bb10 con 0x7eff5c004d00 2023-11-08T19:28:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.293+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c00bcd0 con 0x7eff7015d180 2023-11-08T19:28:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff5c0039a0 0x7eff5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff66ffd640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 0x7eff5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6f7fe640 1 -- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 msgr2=0x7eff5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:58.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff7015d180 0x7eff5c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6ffff640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c004d00 0x7eff5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6f7fe640 1 -- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c004d00 msgr2=0x7eff5c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff5c004d00 0x7eff5c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.294+0000 7eff6f7fe640 1 -- 172.21.15.119:0/3403642244 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c003520 con 0x7eff5c0039a0 2023-11-08T19:28:58.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.295+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff5c0039a0 0x7eff5c003dc0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7eff60001e50 tx=0x7eff5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de60ae19410cd477 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:58.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.295+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60003b70 con 0x7eff5c0039a0 2023-11-08T19:28:58.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.295+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff60003cd0 con 0x7eff5c0039a0 2023-11-08T19:28:58.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.295+0000 7eff6effd640 1 -- 172.21.15.119:0/3403642244 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff60005310 con 0x7eff5c0039a0 2023-11-08T19:28:58.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff76075640 1 -- 172.21.15.119:0/3403642244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff5c0039a0 msgr2=0x7eff5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff76075640 1 --2- 172.21.15.119:0/3403642244 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff5c0039a0 0x7eff5c003dc0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7eff60001e50 tx=0x7eff5c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff6ffff640 1 -- 172.21.15.119:0/3403642244 reap_dead start 2023-11-08T19:28:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff76075640 1 -- 172.21.15.119:0/3403642244 shutdown_connections 2023-11-08T19:28:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff76075640 1 -- 172.21.15.119:0/3403642244 >> 172.21.15.119:0/3403642244 conn(0x7eff700cb610 msgr2=0x7eff700cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff76075640 1 -- 172.21.15.119:0/3403642244 shutdown_connections 2023-11-08T19:28:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.296+0000 7eff76075640 1 -- 172.21.15.119:0/3403642244 wait complete. 2023-11-08T19:28:58.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.298+0000 7eff76075640 1 Processor -- start 2023-11-08T19:28:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.298+0000 7eff76075640 1 -- start start 2023-11-08T19:28:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff76075640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff5c0039a0 0x7eff70157da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff76075640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff7015fe40 0x7eff7015e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff76075640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 0x7eff700c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff76075640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7015f9b0 con 0x7eff701443e0 2023-11-08T19:28:58.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff76075640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff70158620 con 0x7eff7015fe40 2023-11-08T19:28:58.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff76075640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7015eda0 con 0x7eff5c0039a0 2023-11-08T19:28:58.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.299+0000 7eff6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 0x7eff700c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 0x7eff700c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35690/0 (socket says 172.21.15.119:35690) 2023-11-08T19:28:58.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 -- 172.21.15.119:0/2844472545 learned_addr learned my addr 172.21.15.119:0/2844472545 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:58.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff5c0039a0 0x7eff70157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff66ffd640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff7015fe40 0x7eff7015e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 -- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff5c0039a0 msgr2=0x7eff70157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff5c0039a0 0x7eff70157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 -- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff7015fe40 msgr2=0x7eff7015e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff7015fe40 0x7eff7015e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 -- 172.21.15.119:0/2844472545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7015f080 con 0x7eff701443e0 2023-11-08T19:28:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff66ffd640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff7015fe40 0x7eff7015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:28:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6ffff640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 0x7eff700c5bd0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7eff70160b30 tx=0x7eff68010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:58.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.300+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800e8d0 con 0x7eff701443e0 2023-11-08T19:28:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.301+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff6800ea50 con 0x7eff701443e0 2023-11-08T19:28:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.301+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff6800ee00 con 0x7eff701443e0 2023-11-08T19:28:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.301+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff7015f360 con 0x7eff701443e0 2023-11-08T19:28:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.301+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff70144870 con 0x7eff701443e0 2023-11-08T19:28:58.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.302+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7015fb60 con 0x7eff701443e0 2023-11-08T19:28:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.303+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7eff68020410 con 0x7eff701443e0 2023-11-08T19:28:58.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.304+0000 7eff6cff9640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7eff5006ae70 0x7eff5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.304+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7eff68089430 con 0x7eff701443e0 2023-11-08T19:28:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.304+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7eff5006ae70 0x7eff5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.305+0000 7eff6f7fe640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7eff5006ae70 0x7eff5006d330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7eff60034000 tx=0x7eff60009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:58.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.308+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7eff7015fb60 con 0x7eff701443e0 2023-11-08T19:28:58.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.494+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7eff7006c590 con 0x7eff5006ae70 2023-11-08T19:28:58.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.529+0000 7eff6cff9640 1 -- 172.21.15.119:0/2844472545 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+517488 (secure 0 0 0) 0x7eff6003f040 con 0x7eff5006ae70 2023-11-08T19:28:58.538 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:58.542 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:28:58.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7eff5006ae70 msgr2=0x7eff5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7eff5006ae70 0x7eff5006d330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7eff60034000 tx=0x7eff60009030 comp rx=0 tx=0).stop 2023-11-08T19:28:58.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 msgr2=0x7eff700c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 0x7eff700c5bd0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7eff70160b30 tx=0x7eff68010c20 comp rx=0 tx=0).stop 2023-11-08T19:28:58.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 shutdown_connections 2023-11-08T19:28:58.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7eff5006ae70 0x7eff5006d330 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7eff701443e0 0x7eff700c5bd0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7eff7015fe40 0x7eff7015e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 --2- 172.21.15.119:0/2844472545 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7eff5c0039a0 0x7eff70157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 >> 172.21.15.119:0/2844472545 conn(0x7eff700cb610 msgr2=0x7eff700cd520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:58.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 shutdown_connections 2023-11-08T19:28:58.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.540+0000 7eff76075640 1 -- 172.21.15.119:0/2844472545 wait complete. 2023-11-08T19:28:58.565 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2023-11-08T19:28:56.741532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":258,"acting":258,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":205,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259548,"kb_used_data":10100,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715172,"statfs":{"total":773094113280,"available":772828336128,"internally_reserved":0,"allocated":10342400,"data_stored":5691622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.741942"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.019781+0000","last_change":"2023-11-08T19:28:51.019781+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.019781+0000","last_undegraded":"2023-11-08T19:28:51.019781+0000","last_fullsized":"2023-11-08T19:28:51.019781+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898374+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:47.898374+0000","last_peered":"2023-11-08T19:28:47.898374+0000","last_clean":"2023-11-08T19:28:47.898374+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:47.898374+0000","last_undegraded":"2023-11-08T19:28:47.898374+0000","last_fullsized":"2023-11-08T19:28:47.898374+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897764+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:47.897764+0000","last_peered":"2023-11-08T19:28:47.897764+0000","last_clean":"2023-11-08T19:28:47.897764+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:47.897764+0000","last_undegraded":"2023-11-08T19:28:47.897764+0000","last_fullsized":"2023-11-08T19:28:47.897764+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897826+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:47.897826+0000","last_peered":"2023-11-08T19:28:47.897826+0000","last_clean":"2023-11-08T19:28:47.897826+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:47.897826+0000","last_undegraded":"2023-11-08T19:28:47.897826+0000","last_fullsized":"2023-11-08T19:28:47.897826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.012025+0000","last_change":"2023-11-08T19:28:51.012025+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.012025+0000","last_undegraded":"2023-11-08T19:28:51.012025+0000","last_fullsized":"2023-11-08T19:28:51.012025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.981202+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:46.981202+0000","last_peered":"2023-11-08T19:28:46.981202+0000","last_clean":"2023-11-08T19:28:46.981202+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:46.981202+0000","last_undegraded":"2023-11-08T19:28:46.981202+0000","last_fullsized":"2023-11-08T19:28:46.981202+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898666+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:47.898666+0000","last_peered":"2023-11-08T19:28:47.898666+0000","last_clean":"2023-11-08T19:28:47.898666+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:47.898666+0000","last_undegraded":"2023-11-08T19:28:47.898666+0000","last_fullsized":"2023-11-08T19:28:47.898666+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898181+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:47.898181+0000","last_peered":"2023-11-08T19:28:47.898181+0000","last_clean":"2023-11-08T19:28:47.898181+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:47.898181+0000","last_undegraded":"2023-11-08T19:28:47.898181+0000","last_fullsized":"2023-11-08T19:28:47.898181+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.007233+0000","last_change":"2023-11-08T19:28:51.007233+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.007233+0000","last_undegraded":"2023-11-08T19:28:51.007233+0000","last_fullsized":"2023-11-08T19:28:51.007233+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984035+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:46.984035+0000","last_peered":"2023-11-08T19:28:46.984035+0000","last_clean":"2023-11-08T19:28:46.984035+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:46.984035+0000","last_undegraded":"2023-11-08T19:28:46.984035+0000","last_fullsized":"2023-11-08T19:28:46.984035+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897644+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:47.897644+0000","last_peered":"2023-11-08T19:28:47.897644+0000","last_clean":"2023-11-08T19:28:47.897644+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:47.897644+0000","last_undegraded":"2023-11-08T19:28:47.897644+0000","last_fullsized":"2023-11-08T19:28:47.897644+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":61,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898929+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:47.898929+0000","last_peered":"2023-11-08T19:28:47.898929+0000","last_clean":"2023-11-08T19:28:47.898929+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:47.898929+0000","last_undegraded":"2023-11-08T19:28:47.898929+0000","last_fullsized":"2023-11-08T19:28:47.898929+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.011921+0000","last_change":"2023-11-08T19:28:51.011921+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.011921+0000","last_undegraded":"2023-11-08T19:28:51.011921+0000","last_fullsized":"2023-11-08T19:28:51.011921+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073234+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:47.073234+0000","last_peered":"2023-11-08T19:28:47.073234+0000","last_clean":"2023-11-08T19:28:47.073234+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:47.073234+0000","last_undegraded":"2023-11-08T19:28:47.073234+0000","last_fullsized":"2023-11-08T19:28:47.073234+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897740+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:47.897740+0000","last_peered":"2023-11-08T19:28:47.897740+0000","last_clean":"2023-11-08T19:28:47.897740+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:47.897740+0000","last_undegraded":"2023-11-08T19:28:47.897740+0000","last_fullsized":"2023-11-08T19:28:47.897740+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897705+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:47.897705+0000","last_peered":"2023-11-08T19:28:47.897705+0000","last_clean":"2023-11-08T19:28:47.897705+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:47.897705+0000","last_undegraded":"2023-11-08T19:28:47.897705+0000","last_fullsized":"2023-11-08T19:28:47.897705+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.021469+0000","last_change":"2023-11-08T19:28:51.021469+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.021469+0000","last_undegraded":"2023-11-08T19:28:51.021469+0000","last_fullsized":"2023-11-08T19:28:51.021469+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982516+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:46.982516+0000","last_peered":"2023-11-08T19:28:46.982516+0000","last_clean":"2023-11-08T19:28:46.982516+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:46.982516+0000","last_undegraded":"2023-11-08T19:28:46.982516+0000","last_fullsized":"2023-11-08T19:28:46.982516+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980580+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:46.980580+0000","last_peered":"2023-11-08T19:28:46.980580+0000","last_clean":"2023-11-08T19:28:46.980580+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:46.980580+0000","last_undegraded":"2023-11-08T19:28:46.980580+0000","last_fullsized":"2023-11-08T19:28:46.980580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898661+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:47.898661+0000","last_peered":"2023-11-08T19:28:47.898661+0000","last_clean":"2023-11-08T19:28:47.898661+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:47.898661+0000","last_undegraded":"2023-11-08T19:28:47.898661+0000","last_fullsized":"2023-11-08T19:28:47.898661+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004832+0000","last_change":"2023-11-08T19:28:51.004832+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004832+0000","last_undegraded":"2023-11-08T19:28:51.004832+0000","last_fullsized":"2023-11-08T19:28:51.004832+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073235+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:47.073235+0000","last_peered":"2023-11-08T19:28:47.073235+0000","last_clean":"2023-11-08T19:28:47.073235+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:47.073235+0000","last_undegraded":"2023-11-08T19:28:47.073235+0000","last_fullsized":"2023-11-08T19:28:47.073235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897438+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:47.897438+0000","last_peered":"2023-11-08T19:28:47.897438+0000","last_clean":"2023-11-08T19:28:47.897438+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:47.897438+0000","last_undegraded":"2023-11-08T19:28:47.897438+0000","last_fullsized":"2023-11-08T19:28:47.897438+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073141+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:47.073141+0000","last_peered":"2023-11-08T19:28:47.073141+0000","last_clean":"2023-11-08T19:28:47.073141+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:47.073141+0000","last_undegraded":"2023-11-08T19:28:47.073141+0000","last_fullsized":"2023-11-08T19:28:47.073141+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004676+0000","last_change":"2023-11-08T19:28:51.004676+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004676+0000","last_undegraded":"2023-11-08T19:28:51.004676+0000","last_fullsized":"2023-11-08T19:28:51.004676+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897156+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:47.897156+0000","last_peered":"2023-11-08T19:28:47.897156+0000","last_clean":"2023-11-08T19:28:47.897156+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:47.897156+0000","last_undegraded":"2023-11-08T19:28:47.897156+0000","last_fullsized":"2023-11-08T19:28:47.897156+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073059+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:47.073059+0000","last_peered":"2023-11-08T19:28:47.073059+0000","last_clean":"2023-11-08T19:28:47.073059+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:47.073059+0000","last_undegraded":"2023-11-08T19:28:47.073059+0000","last_fullsized":"2023-11-08T19:28:47.073059+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073603+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:47.073603+0000","last_peered":"2023-11-08T19:28:47.073603+0000","last_clean":"2023-11-08T19:28:47.073603+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:47.073603+0000","last_undegraded":"2023-11-08T19:28:47.073603+0000","last_fullsized":"2023-11-08T19:28:47.073603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015351+0000","last_change":"2023-11-08T19:28:51.015351+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015351+0000","last_undegraded":"2023-11-08T19:28:51.015351+0000","last_fullsized":"2023-11-08T19:28:51.015351+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897031+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:47.897031+0000","last_peered":"2023-11-08T19:28:47.897031+0000","last_clean":"2023-11-08T19:28:47.897031+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:47.897031+0000","last_undegraded":"2023-11-08T19:28:47.897031+0000","last_fullsized":"2023-11-08T19:28:47.897031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897639+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:47.897639+0000","last_peered":"2023-11-08T19:28:47.897639+0000","last_clean":"2023-11-08T19:28:47.897639+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:47.897639+0000","last_undegraded":"2023-11-08T19:28:47.897639+0000","last_fullsized":"2023-11-08T19:28:47.897639+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983750+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:46.983750+0000","last_peered":"2023-11-08T19:28:46.983750+0000","last_clean":"2023-11-08T19:28:46.983750+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:46.983750+0000","last_undegraded":"2023-11-08T19:28:46.983750+0000","last_fullsized":"2023-11-08T19:28:46.983750+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.009689+0000","last_change":"2023-11-08T19:28:51.009689+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.009689+0000","last_undegraded":"2023-11-08T19:28:51.009689+0000","last_fullsized":"2023-11-08T19:28:51.009689+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898820+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:47.898820+0000","last_peered":"2023-11-08T19:28:47.898820+0000","last_clean":"2023-11-08T19:28:47.898820+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:47.898820+0000","last_undegraded":"2023-11-08T19:28:47.898820+0000","last_fullsized":"2023-11-08T19:28:47.898820+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984389+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:46.984389+0000","last_peered":"2023-11-08T19:28:46.984389+0000","last_clean":"2023-11-08T19:28:46.984389+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:46.984389+0000","last_undegraded":"2023-11-08T19:28:46.984389+0000","last_fullsized":"2023-11-08T19:28:46.984389+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983524+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:46.983524+0000","last_peered":"2023-11-08T19:28:46.983524+0000","last_clean":"2023-11-08T19:28:46.983524+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:46.983524+0000","last_undegraded":"2023-11-08T19:28:46.983524+0000","last_fullsized":"2023-11-08T19:28:46.983524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005396+0000","last_change":"2023-11-08T19:28:51.005396+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005396+0000","last_undegraded":"2023-11-08T19:28:51.005396+0000","last_fullsized":"2023-11-08T19:28:51.005396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984268+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:46.984268+0000","last_peered":"2023-11-08T19:28:46.984268+0000","last_clean":"2023-11-08T19:28:46.984268+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:46.984268+0000","last_undegraded":"2023-11-08T19:28:46.984268+0000","last_fullsized":"2023-11-08T19:28:46.984268+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898588+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:47.898588+0000","last_peered":"2023-11-08T19:28:47.898588+0000","last_clean":"2023-11-08T19:28:47.898588+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:47.898588+0000","last_undegraded":"2023-11-08T19:28:47.898588+0000","last_fullsized":"2023-11-08T19:28:47.898588+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980541+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:46.980541+0000","last_peered":"2023-11-08T19:28:46.980541+0000","last_clean":"2023-11-08T19:28:46.980541+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:46.980541+0000","last_undegraded":"2023-11-08T19:28:46.980541+0000","last_fullsized":"2023-11-08T19:28:46.980541+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005496+0000","last_change":"2023-11-08T19:28:51.005496+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005496+0000","last_undegraded":"2023-11-08T19:28:51.005496+0000","last_fullsized":"2023-11-08T19:28:51.005496+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982650+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:46.982650+0000","last_peered":"2023-11-08T19:28:46.982650+0000","last_clean":"2023-11-08T19:28:46.982650+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:46.982650+0000","last_undegraded":"2023-11-08T19:28:46.982650+0000","last_fullsized":"2023-11-08T19:28:46.982650+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073342+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:47.073342+0000","last_peered":"2023-11-08T19:28:47.073342+0000","last_clean":"2023-11-08T19:28:47.073342+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:47.073342+0000","last_undegraded":"2023-11-08T19:28:47.073342+0000","last_fullsized":"2023-11-08T19:28:47.073342+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983351+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:46.983351+0000","last_peered":"2023-11-08T19:28:46.983351+0000","last_clean":"2023-11-08T19:28:46.983351+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:46.983351+0000","last_undegraded":"2023-11-08T19:28:46.983351+0000","last_fullsized":"2023-11-08T19:28:46.983351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.020306+0000","last_change":"2023-11-08T19:28:51.020306+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.020306+0000","last_undegraded":"2023-11-08T19:28:51.020306+0000","last_fullsized":"2023-11-08T19:28:51.020306+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897863+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:47.897863+0000","last_peered":"2023-11-08T19:28:47.897863+0000","last_clean":"2023-11-08T19:28:47.897863+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:47.897863+0000","last_undegraded":"2023-11-08T19:28:47.897863+0000","last_fullsized":"2023-11-08T19:28:47.897863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073430+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:47.073430+0000","last_peered":"2023-11-08T19:28:47.073430+0000","last_clean":"2023-11-08T19:28:47.073430+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:47.073430+0000","last_undegraded":"2023-11-08T19:28:47.073430+0000","last_fullsized":"2023-11-08T19:28:47.073430+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":64,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984510+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:46.984510+0000","last_peered":"2023-11-08T19:28:46.984510+0000","last_clean":"2023-11-08T19:28:46.984510+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:46.984510+0000","last_undegraded":"2023-11-08T19:28:46.984510+0000","last_fullsized":"2023-11-08T19:28:46.984510+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015226+0000","last_change":"2023-11-08T19:28:51.015226+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015226+0000","last_undegraded":"2023-11-08T19:28:51.015226+0000","last_fullsized":"2023-11-08T19:28:51.015226+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983868+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:46.983868+0000","last_peered":"2023-11-08T19:28:46.983868+0000","last_clean":"2023-11-08T19:28:46.983868+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:46.983868+0000","last_undegraded":"2023-11-08T19:28:46.983868+0000","last_fullsized":"2023-11-08T19:28:46.983868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983310+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:46.983310+0000","last_peered":"2023-11-08T19:28:46.983310+0000","last_clean":"2023-11-08T19:28:46.983310+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:46.983310+0000","last_undegraded":"2023-11-08T19:28:46.983310+0000","last_fullsized":"2023-11-08T19:28:46.983310+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982243+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:46.982243+0000","last_peered":"2023-11-08T19:28:46.982243+0000","last_clean":"2023-11-08T19:28:46.982243+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:46.982243+0000","last_undegraded":"2023-11-08T19:28:46.982243+0000","last_fullsized":"2023-11-08T19:28:46.982243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.010187+0000","last_change":"2023-11-08T19:28:51.010187+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.010187+0000","last_undegraded":"2023-11-08T19:28:51.010187+0000","last_fullsized":"2023-11-08T19:28:51.010187+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897613+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:47.897613+0000","last_peered":"2023-11-08T19:28:47.897613+0000","last_clean":"2023-11-08T19:28:47.897613+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:47.897613+0000","last_undegraded":"2023-11-08T19:28:47.897613+0000","last_fullsized":"2023-11-08T19:28:47.897613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984004+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:46.984004+0000","last_peered":"2023-11-08T19:28:46.984004+0000","last_clean":"2023-11-08T19:28:46.984004+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:46.984004+0000","last_undegraded":"2023-11-08T19:28:46.984004+0000","last_fullsized":"2023-11-08T19:28:46.984004+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":57,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980151+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:46.980151+0000","last_peered":"2023-11-08T19:28:46.980151+0000","last_clean":"2023-11-08T19:28:46.980151+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:46.980151+0000","last_undegraded":"2023-11-08T19:28:46.980151+0000","last_fullsized":"2023-11-08T19:28:46.980151+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.010260+0000","last_change":"2023-11-08T19:28:51.010260+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.010260+0000","last_undegraded":"2023-11-08T19:28:51.010260+0000","last_fullsized":"2023-11-08T19:28:51.010260+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980042+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:46.980042+0000","last_peered":"2023-11-08T19:28:46.980042+0000","last_clean":"2023-11-08T19:28:46.980042+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:46.980042+0000","last_undegraded":"2023-11-08T19:28:46.980042+0000","last_fullsized":"2023-11-08T19:28:46.980042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897127+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:47.897127+0000","last_peered":"2023-11-08T19:28:47.897127+0000","last_clean":"2023-11-08T19:28:47.897127+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:47.897127+0000","last_undegraded":"2023-11-08T19:28:47.897127+0000","last_fullsized":"2023-11-08T19:28:47.897127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073742+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:47.073742+0000","last_peered":"2023-11-08T19:28:47.073742+0000","last_clean":"2023-11-08T19:28:47.073742+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:47.073742+0000","last_undegraded":"2023-11-08T19:28:47.073742+0000","last_fullsized":"2023-11-08T19:28:47.073742+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005943+0000","last_change":"2023-11-08T19:28:51.005943+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005943+0000","last_undegraded":"2023-11-08T19:28:51.005943+0000","last_fullsized":"2023-11-08T19:28:51.005943+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982027+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:46.982027+0000","last_peered":"2023-11-08T19:28:46.982027+0000","last_clean":"2023-11-08T19:28:46.982027+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:46.982027+0000","last_undegraded":"2023-11-08T19:28:46.982027+0000","last_fullsized":"2023-11-08T19:28:46.982027+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073176+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:47.073176+0000","last_peered":"2023-11-08T19:28:47.073176+0000","last_clean":"2023-11-08T19:28:47.073176+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:47.073176+0000","last_undegraded":"2023-11-08T19:28:47.073176+0000","last_fullsized":"2023-11-08T19:28:47.073176+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897468+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:47.897468+0000","last_peered":"2023-11-08T19:28:47.897468+0000","last_clean":"2023-11-08T19:28:47.897468+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:47.897468+0000","last_undegraded":"2023-11-08T19:28:47.897468+0000","last_fullsized":"2023-11-08T19:28:47.897468+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.009642+0000","last_change":"2023-11-08T19:28:51.009642+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.009642+0000","last_undegraded":"2023-11-08T19:28:51.009642+0000","last_fullsized":"2023-11-08T19:28:51.009642+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983068+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:46.983068+0000","last_peered":"2023-11-08T19:28:46.983068+0000","last_clean":"2023-11-08T19:28:46.983068+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:46.983068+0000","last_undegraded":"2023-11-08T19:28:46.983068+0000","last_fullsized":"2023-11-08T19:28:46.983068+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897599+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:47.897599+0000","last_peered":"2023-11-08T19:28:47.897599+0000","last_clean":"2023-11-08T19:28:47.897599+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:47.897599+0000","last_undegraded":"2023-11-08T19:28:47.897599+0000","last_fullsized":"2023-11-08T19:28:47.897599+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898064+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:47.898064+0000","last_peered":"2023-11-08T19:28:47.898064+0000","last_clean":"2023-11-08T19:28:47.898064+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:47.898064+0000","last_undegraded":"2023-11-08T19:28:47.898064+0000","last_fullsized":"2023-11-08T19:28:47.898064+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.006470+0000","last_change":"2023-11-08T19:28:51.006470+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.006470+0000","last_undegraded":"2023-11-08T19:28:51.006470+0000","last_fullsized":"2023-11-08T19:28:51.006470+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984345+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:46.984345+0000","last_peered":"2023-11-08T19:28:46.984345+0000","last_clean":"2023-11-08T19:28:46.984345+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:46.984345+0000","last_undegraded":"2023-11-08T19:28:46.984345+0000","last_fullsized":"2023-11-08T19:28:46.984345+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897497+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:47.897497+0000","last_peered":"2023-11-08T19:28:47.897497+0000","last_clean":"2023-11-08T19:28:47.897497+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:47.897497+0000","last_undegraded":"2023-11-08T19:28:47.897497+0000","last_fullsized":"2023-11-08T19:28:47.897497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898470+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:47.898470+0000","last_peered":"2023-11-08T19:28:47.898470+0000","last_clean":"2023-11-08T19:28:47.898470+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:47.898470+0000","last_undegraded":"2023-11-08T19:28:47.898470+0000","last_fullsized":"2023-11-08T19:28:47.898470+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.008922+0000","last_change":"2023-11-08T19:28:51.008922+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.008922+0000","last_undegraded":"2023-11-08T19:28:51.008922+0000","last_fullsized":"2023-11-08T19:28:51.008922+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984247+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:46.984247+0000","last_peered":"2023-11-08T19:28:46.984247+0000","last_clean":"2023-11-08T19:28:46.984247+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:46.984247+0000","last_undegraded":"2023-11-08T19:28:46.984247+0000","last_fullsized":"2023-11-08T19:28:46.984247+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898061+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:47.898061+0000","last_peered":"2023-11-08T19:28:47.898061+0000","last_clean":"2023-11-08T19:28:47.898061+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:47.898061+0000","last_undegraded":"2023-11-08T19:28:47.898061+0000","last_fullsized":"2023-11-08T19:28:47.898061+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073685+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:47.073685+0000","last_peered":"2023-11-08T19:28:47.073685+0000","last_clean":"2023-11-08T19:28:47.073685+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:47.073685+0000","last_undegraded":"2023-11-08T19:28:47.073685+0000","last_fullsized":"2023-11-08T19:28:47.073685+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005345+0000","last_change":"2023-11-08T19:28:51.005345+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005345+0000","last_undegraded":"2023-11-08T19:28:51.005345+0000","last_fullsized":"2023-11-08T19:28:51.005345+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898260+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:47.898260+0000","last_peered":"2023-11-08T19:28:47.898260+0000","last_clean":"2023-11-08T19:28:47.898260+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:47.898260+0000","last_undegraded":"2023-11-08T19:28:47.898260+0000","last_fullsized":"2023-11-08T19:28:47.898260+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897549+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:47.897549+0000","last_peered":"2023-11-08T19:28:47.897549+0000","last_clean":"2023-11-08T19:28:47.897549+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:47.897549+0000","last_undegraded":"2023-11-08T19:28:47.897549+0000","last_fullsized":"2023-11-08T19:28:47.897549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759631+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:47.759631+0000","last_peered":"2023-11-08T19:28:47.759631+0000","last_clean":"2023-11-08T19:28:47.759631+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:47.759631+0000","last_undegraded":"2023-11-08T19:28:47.759631+0000","last_fullsized":"2023-11-08T19:28:47.759631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.016804+0000","last_change":"2023-11-08T19:28:51.016804+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.016804+0000","last_undegraded":"2023-11-08T19:28:51.016804+0000","last_fullsized":"2023-11-08T19:28:51.016804+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073498+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:47.073498+0000","last_peered":"2023-11-08T19:28:47.073498+0000","last_clean":"2023-11-08T19:28:47.073498+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:47.073498+0000","last_undegraded":"2023-11-08T19:28:47.073498+0000","last_fullsized":"2023-11-08T19:28:47.073498+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983546+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:46.983546+0000","last_peered":"2023-11-08T19:28:46.983546+0000","last_clean":"2023-11-08T19:28:46.983546+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:46.983546+0000","last_undegraded":"2023-11-08T19:28:46.983546+0000","last_fullsized":"2023-11-08T19:28:46.983546+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.003018+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:51.003018+0000","last_peered":"2023-11-08T19:28:51.003018+0000","last_clean":"2023-11-08T19:28:51.003018+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:51.003018+0000","last_undegraded":"2023-11-08T19:28:51.003018+0000","last_fullsized":"2023-11-08T19:28:51.003018+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004718+0000","last_change":"2023-11-08T19:28:51.004718+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004718+0000","last_undegraded":"2023-11-08T19:28:51.004718+0000","last_fullsized":"2023-11-08T19:28:51.004718+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897253+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:47.897253+0000","last_peered":"2023-11-08T19:28:47.897253+0000","last_clean":"2023-11-08T19:28:47.897253+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:47.897253+0000","last_undegraded":"2023-11-08T19:28:47.897253+0000","last_fullsized":"2023-11-08T19:28:47.897253+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073589+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:47.073589+0000","last_peered":"2023-11-08T19:28:47.073589+0000","last_clean":"2023-11-08T19:28:47.073589+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:47.073589+0000","last_undegraded":"2023-11-08T19:28:47.073589+0000","last_fullsized":"2023-11-08T19:28:47.073589+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.019899+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:51.019899+0000","last_peered":"2023-11-08T19:28:51.019899+0000","last_clean":"2023-11-08T19:28:51.019899+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:51.019899+0000","last_undegraded":"2023-11-08T19:28:51.019899+0000","last_fullsized":"2023-11-08T19:28:51.019899+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.020099+0000","last_change":"2023-11-08T19:28:51.020099+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.020099+0000","last_undegraded":"2023-11-08T19:28:51.020099+0000","last_fullsized":"2023-11-08T19:28:51.020099+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897718+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:47.897718+0000","last_peered":"2023-11-08T19:28:47.897718+0000","last_clean":"2023-11-08T19:28:47.897718+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:47.897718+0000","last_undegraded":"2023-11-08T19:28:47.897718+0000","last_fullsized":"2023-11-08T19:28:47.897718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983268+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:46.983268+0000","last_peered":"2023-11-08T19:28:46.983268+0000","last_clean":"2023-11-08T19:28:46.983268+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:46.983268+0000","last_undegraded":"2023-11-08T19:28:46.983268+0000","last_fullsized":"2023-11-08T19:28:46.983268+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897520+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:47.897520+0000","last_peered":"2023-11-08T19:28:47.897520+0000","last_clean":"2023-11-08T19:28:47.897520+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:47.897520+0000","last_undegraded":"2023-11-08T19:28:47.897520+0000","last_fullsized":"2023-11-08T19:28:47.897520+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.021701+0000","last_change":"2023-11-08T19:28:51.021701+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.021701+0000","last_undegraded":"2023-11-08T19:28:51.021701+0000","last_fullsized":"2023-11-08T19:28:51.021701+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982262+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:46.982262+0000","last_peered":"2023-11-08T19:28:46.982262+0000","last_clean":"2023-11-08T19:28:46.982262+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:46.982262+0000","last_undegraded":"2023-11-08T19:28:46.982262+0000","last_fullsized":"2023-11-08T19:28:46.982262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898086+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:47.898086+0000","last_peered":"2023-11-08T19:28:47.898086+0000","last_clean":"2023-11-08T19:28:47.898086+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:47.898086+0000","last_undegraded":"2023-11-08T19:28:47.898086+0000","last_fullsized":"2023-11-08T19:28:47.898086+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":58,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.013666+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:51.013666+0000","last_peered":"2023-11-08T19:28:51.013666+0000","last_clean":"2023-11-08T19:28:51.013666+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:51.013666+0000","last_undegraded":"2023-11-08T19:28:51.013666+0000","last_fullsized":"2023-11-08T19:28:51.013666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.015270+0000","last_change":"2023-11-08T19:28:51.015270+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.015270+0000","last_undegraded":"2023-11-08T19:28:51.015270+0000","last_fullsized":"2023-11-08T19:28:51.015270+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898860+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:47.898860+0000","last_peered":"2023-11-08T19:28:47.898860+0000","last_clean":"2023-11-08T19:28:47.898860+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:47.898860+0000","last_undegraded":"2023-11-08T19:28:47.898860+0000","last_fullsized":"2023-11-08T19:28:47.898860+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073478+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:47.073478+0000","last_peered":"2023-11-08T19:28:47.073478+0000","last_clean":"2023-11-08T19:28:47.073478+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:47.073478+0000","last_undegraded":"2023-11-08T19:28:47.073478+0000","last_fullsized":"2023-11-08T19:28:47.073478+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":60,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983063+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:46.983063+0000","last_peered":"2023-11-08T19:28:46.983063+0000","last_clean":"2023-11-08T19:28:46.983063+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:46.983063+0000","last_undegraded":"2023-11-08T19:28:46.983063+0000","last_fullsized":"2023-11-08T19:28:46.983063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.012488+0000","last_change":"2023-11-08T19:28:51.012488+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.012488+0000","last_undegraded":"2023-11-08T19:28:51.012488+0000","last_fullsized":"2023-11-08T19:28:51.012488+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982795+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:46.982795+0000","last_peered":"2023-11-08T19:28:46.982795+0000","last_clean":"2023-11-08T19:28:46.982795+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:46.982795+0000","last_undegraded":"2023-11-08T19:28:46.982795+0000","last_fullsized":"2023-11-08T19:28:46.982795+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.898720+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:47.898720+0000","last_peered":"2023-11-08T19:28:47.898720+0000","last_clean":"2023-11-08T19:28:47.898720+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:47.898720+0000","last_undegraded":"2023-11-08T19:28:47.898720+0000","last_fullsized":"2023-11-08T19:28:47.898720+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":59,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073382+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:47.073382+0000","last_peered":"2023-11-08T19:28:47.073382+0000","last_clean":"2023-11-08T19:28:47.073382+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:47.073382+0000","last_undegraded":"2023-11-08T19:28:47.073382+0000","last_fullsized":"2023-11-08T19:28:47.073382+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.018696+0000","last_change":"2023-11-08T19:28:51.018696+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.018696+0000","last_undegraded":"2023-11-08T19:28:51.018696+0000","last_fullsized":"2023-11-08T19:28:51.018696+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.984117+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:46.984117+0000","last_peered":"2023-11-08T19:28:46.984117+0000","last_clean":"2023-11-08T19:28:46.984117+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:46.984117+0000","last_undegraded":"2023-11-08T19:28:46.984117+0000","last_fullsized":"2023-11-08T19:28:46.984117+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.073301+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:47.073301+0000","last_peered":"2023-11-08T19:28:47.073301+0000","last_clean":"2023-11-08T19:28:47.073301+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:47.073301+0000","last_undegraded":"2023-11-08T19:28:47.073301+0000","last_fullsized":"2023-11-08T19:28:47.073301+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":61,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.017867+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:51.017867+0000","last_peered":"2023-11-08T19:28:51.017867+0000","last_clean":"2023-11-08T19:28:51.017867+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:51.017867+0000","last_undegraded":"2023-11-08T19:28:51.017867+0000","last_fullsized":"2023-11-08T19:28:51.017867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.008313+0000","last_change":"2023-11-08T19:28:51.008313+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.008313+0000","last_undegraded":"2023-11-08T19:28:51.008313+0000","last_fullsized":"2023-11-08T19:28:51.008313+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897392+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:47.897392+0000","last_peered":"2023-11-08T19:28:47.897392+0000","last_clean":"2023-11-08T19:28:47.897392+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:47.897392+0000","last_undegraded":"2023-11-08T19:28:47.897392+0000","last_fullsized":"2023-11-08T19:28:47.897392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759820+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:47.759820+0000","last_peered":"2023-11-08T19:28:47.759820+0000","last_clean":"2023-11-08T19:28:47.759820+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:47.759820+0000","last_undegraded":"2023-11-08T19:28:47.759820+0000","last_fullsized":"2023-11-08T19:28:47.759820+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.983608+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:46.983608+0000","last_peered":"2023-11-08T19:28:46.983608+0000","last_clean":"2023-11-08T19:28:46.983608+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:46.983608+0000","last_undegraded":"2023-11-08T19:28:46.983608+0000","last_fullsized":"2023-11-08T19:28:46.983608+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.019995+0000","last_change":"2023-11-08T19:28:51.019995+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.019995+0000","last_undegraded":"2023-11-08T19:28:51.019995+0000","last_fullsized":"2023-11-08T19:28:51.019995+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759877+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:47.759877+0000","last_peered":"2023-11-08T19:28:47.759877+0000","last_clean":"2023-11-08T19:28:47.759877+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:47.759877+0000","last_undegraded":"2023-11-08T19:28:47.759877+0000","last_fullsized":"2023-11-08T19:28:47.759877+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980860+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:46.980860+0000","last_peered":"2023-11-08T19:28:46.980860+0000","last_clean":"2023-11-08T19:28:46.980860+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:46.980860+0000","last_undegraded":"2023-11-08T19:28:46.980860+0000","last_fullsized":"2023-11-08T19:28:46.980860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":65,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897035+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:47.897035+0000","last_peered":"2023-11-08T19:28:47.897035+0000","last_clean":"2023-11-08T19:28:47.897035+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:47.897035+0000","last_undegraded":"2023-11-08T19:28:47.897035+0000","last_fullsized":"2023-11-08T19:28:47.897035+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.004935+0000","last_change":"2023-11-08T19:28:51.004935+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.004935+0000","last_undegraded":"2023-11-08T19:28:51.004935+0000","last_fullsized":"2023-11-08T19:28:51.004935+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897809+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:47.897809+0000","last_peered":"2023-11-08T19:28:47.897809+0000","last_clean":"2023-11-08T19:28:47.897809+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:47.897809+0000","last_undegraded":"2023-11-08T19:28:47.897809+0000","last_fullsized":"2023-11-08T19:28:47.897809+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980029+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:46.980029+0000","last_peered":"2023-11-08T19:28:46.980029+0000","last_clean":"2023-11-08T19:28:46.980029+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:46.980029+0000","last_undegraded":"2023-11-08T19:28:46.980029+0000","last_fullsized":"2023-11-08T19:28:46.980029+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.981009+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:46.981009+0000","last_peered":"2023-11-08T19:28:46.981009+0000","last_clean":"2023-11-08T19:28:46.981009+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:46.981009+0000","last_undegraded":"2023-11-08T19:28:46.981009+0000","last_fullsized":"2023-11-08T19:28:46.981009+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.007275+0000","last_change":"2023-11-08T19:28:51.007275+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.007275+0000","last_undegraded":"2023-11-08T19:28:51.007275+0000","last_fullsized":"2023-11-08T19:28:51.007275+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759559+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:47.759559+0000","last_peered":"2023-11-08T19:28:47.759559+0000","last_clean":"2023-11-08T19:28:47.759559+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:47.759559+0000","last_undegraded":"2023-11-08T19:28:47.759559+0000","last_fullsized":"2023-11-08T19:28:47.759559+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.759562+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:47.759562+0000","last_peered":"2023-11-08T19:28:47.759562+0000","last_clean":"2023-11-08T19:28:47.759562+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:47.759562+0000","last_undegraded":"2023-11-08T19:28:47.759562+0000","last_fullsized":"2023-11-08T19:28:47.759562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":58,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.980880+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:46.980880+0000","last_peered":"2023-11-08T19:28:46.980880+0000","last_clean":"2023-11-08T19:28:46.980880+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:46.980880+0000","last_undegraded":"2023-11-08T19:28:46.980880+0000","last_fullsized":"2023-11-08T19:28:46.980880+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T19:28:51.998042+0000","last_change":"2023-11-08T19:28:51.998042+0000","last_active":"2023-11-08T19:28:51.998042+0000","last_peered":"2023-11-08T19:28:51.998042+0000","last_clean":"2023-11-08T19:28:51.998042+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.998042+0000","last_undegraded":"2023-11-08T19:28:51.998042+0000","last_fullsized":"2023-11-08T19:28:51.998042+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":93,"state":"creating+peering","last_fresh":"2023-11-08T19:28:51.005157+0000","last_change":"2023-11-08T19:28:51.005157+0000","last_active":"2023-11-08T19:28:50.984789+0000","last_peered":"2023-11-08T19:28:50.984789+0000","last_clean":"2023-11-08T19:28:50.984789+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T19:28:51.005157+0000","last_undegraded":"2023-11-08T19:28:51.005157+0000","last_fullsized":"2023-11-08T19:28:51.005157+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:47.897246+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:47.897246+0000","last_peered":"2023-11-08T19:28:47.897246+0000","last_clean":"2023-11-08T19:28:47.897246+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:47.897246+0000","last_undegraded":"2023-11-08T19:28:47.897246+0000","last_fullsized":"2023-11-08T19:28:47.897246+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-08T19:28:46.982860+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:46.982860+0000","last_peered":"2023-11-08T19:28:46.982860+0000","last_clean":"2023-11-08T19:28:46.982860+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:46.982860+0000","last_undegraded":"2023-11-08T19:28:46.982860+0000","last_fullsized":"2023-11-08T19:28:46.982860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.004218+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:51.004218+0000","last_peered":"2023-11-08T19:28:51.004218+0000","last_clean":"2023-11-08T19:28:51.004218+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:51.004218+0000","last_undegraded":"2023-11-08T19:28:51.004218+0000","last_fullsized":"2023-11-08T19:28:51.004218+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":229,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-11-08T19:28:51.018351+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:51.018351+0000","last_peered":"2023-11-08T19:28:51.018351+0000","last_clean":"2023-11-08T19:28:51.018351+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:51.018351+0000","last_undegraded":"2023-11-08T19:28:51.018351+0000","last_fullsized":"2023-11-08T19:28:51.018351+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574901,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1740800,"data_stored":1155387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.075,"15min":1.1000000000000001},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.52,"5min":1.7509999999999999,"15min":1.7509999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.141,"5min":1.25,"15min":1.3120000000000001},"min":{"1min":0.58599999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":2.1360000000000001,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.74399999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.196,"15min":1.2569999999999999},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.518,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":1.46},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.125,"15min":1.1359999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0049999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.405,"15min":1.4970000000000001},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":0.91900000000000004},{"interface":"front","average":{"1min":1.056,"5min":1.28,"15min":1.3620000000000001},"min":{"1min":0.67200000000000004,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.389,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.94999999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.3129999999999999,"15min":1.367},"min":{"1min":0.79000000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.591,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.284},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.506,"15min":1.5700000000000001},"min":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":2.2200000000000002,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.167}]},{"osd":4,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3360000000000001,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.875,"15min":0.875},"max":{"1min":1.798,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.056},{"interface":"front","average":{"1min":1.321,"5min":1.427,"15min":1.4970000000000001},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":1.774,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.0960000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.444,"15min":1.464},"min":{"1min":0.73299999999999998,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.359},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.4490000000000001,"15min":1.4790000000000001},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.7829999999999999,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.3220000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:27:50 2023","interfaces":[{"interface":"back","average":{"1min":1.474,"5min":1.546,"15min":1.579},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":1.9770000000000001,"5min":2.427,"15min":2.427},"last":1.234},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.5680000000000001,"15min":1.631},"min":{"1min":0.997,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":1.8939999999999999,"5min":2.468,"15min":2.468},"last":1.401}]}]},{"osd":6,"up_from":13,"seq":55834574900,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.96999999999999997,"15min":0.93400000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6279999999999999,"5min":1.752,"15min":1.752},"last":0.67700000000000005},{"interface":"front","average":{"1min":1.083,"5min":1.1140000000000001,"15min":1.1060000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.581,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.72999999999999998}]},{"osd":1,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.095,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.47,"5min":1.679,"15min":1.679},"last":1.411},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.2789999999999999,"15min":1.337},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.6559999999999999,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":0.999}]},{"osd":2,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.238,"15min":1.292},"min":{"1min":0.68799999999999994,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4359999999999999,"5min":4.609,"15min":4.609},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.236,"15min":1.2190000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6879999999999999,"5min":1.861,"15min":1.861},"last":1.177}]},{"osd":3,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.4350000000000001,"15min":1.4870000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.756,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.351},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1819999999999999,"15min":1.181},"min":{"1min":0.76900000000000002,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.482,"5min":1.784,"15min":1.784},"last":1.2869999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.371,"15min":1.377},"min":{"1min":0.79600000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.732,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.585,"5min":1.518,"15min":1.5029999999999999},"min":{"1min":0.89500000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.9510000000000001,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.1180000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.2729999999999999,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.7829999999999999,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.073},{"interface":"front","average":{"1min":1.403,"5min":1.347,"15min":1.335},"min":{"1min":0.83099999999999996,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.8660000000000001,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.5800000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:53 2023","interfaces":[{"interface":"back","average":{"1min":1.6220000000000001,"5min":1.5669999999999999,"15min":1.55},"min":{"1min":0.85899999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8759999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.639},{"interface":"front","average":{"1min":1.512,"5min":1.454,"15min":1.454},"min":{"1min":0.81499999999999995,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8779999999999999,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.464}]}]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.093,"15min":1.0740000000000001},"min":{"1min":0.59999999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6990000000000001,"5min":1.76,"15min":1.76},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.2310000000000001,"15min":1.234},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":1.258}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.056,"5min":1.141,"15min":1.1699999999999999},"min":{"1min":0.753,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.5349999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.508},{"interface":"front","average":{"1min":1.181,"5min":1.343,"15min":1.4410000000000001},"min":{"1min":0.76200000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.617,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":1.0469999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1830000000000001,"15min":1.1719999999999999},"min":{"1min":0.81899999999999995,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.639,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.3939999999999999,"15min":1.4990000000000001},"min":{"1min":0.94299999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.806,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.1000000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.28,"15min":1.288},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.804,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3109999999999999,"15min":1.3049999999999999},"min":{"1min":0.91300000000000003,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5189999999999999,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.6699999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2769999999999999,"15min":1.2869999999999999},"min":{"1min":0.85399999999999998,"5min":0.755,"15min":0.755},"max":{"1min":1.635,"5min":2.347,"15min":2.347},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.5149999999999999,"15min":1.5529999999999999},"min":{"1min":1.0589999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.097,"5min":3.153,"15min":3.153},"last":1.6319999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4930000000000001,"15min":1.528},"min":{"1min":0.89100000000000001,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.7350000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.8360000000000001},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4770000000000001,"15min":1.5009999999999999},"min":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"max":{"1min":1.899,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.8029999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.514,"5min":1.6399999999999999,"15min":1.7},"min":{"1min":1.03,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0449999999999999,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.718},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.581,"15min":1.5900000000000001},"min":{"1min":1.1439999999999999,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.831,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.883}]}]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.0549999999999999,"15min":1.0469999999999999},"min":{"1min":0.67000000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.496,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2549999999999999,"15min":1.2090000000000001},"min":{"1min":0.85699999999999998,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.696,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":1.069,"15min":1.091},"min":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1279999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.111,"15min":1.1599999999999999},"min":{"1min":0.89100000000000001,"5min":0.499,"15min":0.499},"max":{"1min":1.7450000000000001,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.28}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.3169999999999999,"15min":1.345},"min":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.4610000000000001,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.306,"5min":1.367,"15min":1.423},"min":{"1min":0.82099999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.585,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.391}]},{"osd":3,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.2769999999999999,"15min":1.292},"min":{"1min":0.72399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.696,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3600000000000001,"15min":1.387},"min":{"1min":0.83599999999999997,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.629,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":0.96299999999999997}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.4399999999999999,"15min":1.5129999999999999},"min":{"1min":0.96699999999999997,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.667,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.4970000000000001,"15min":1.5489999999999999},"min":{"1min":0.95899999999999996,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.8560000000000001,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.1559999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.5229999999999999,"15min":1.5620000000000001},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.782,"5min":4.335,"15min":4.335},"last":1.218},{"interface":"front","average":{"1min":1.52,"5min":1.5880000000000001,"15min":1.6259999999999999},"min":{"1min":1.3140000000000001,"5min":0.89300000000000002,"15min":0.89300000000000002},"max":{"1min":1.8080000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.605}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.6040000000000001,"15min":1.631},"min":{"1min":1.1799999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.77,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.6859999999999999,"15min":1.732},"min":{"1min":1.127,"5min":1.095,"15min":1.095},"max":{"1min":1.901,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.327}]}]},{"osd":3,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.98599999999999999,"15min":1.0720000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.23,"5min":3.972,"15min":3.972},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.96099999999999997,"5min":1.032,"15min":1.095},"min":{"1min":0.372,"5min":0.311,"15min":0.311},"max":{"1min":1.899,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.67900000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":1.103,"15min":1.1850000000000001},"min":{"1min":0.41699999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.2609999999999999,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.93899999999999995,"5min":1.05,"15min":1.1080000000000001},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.8520000000000001,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":0.93400000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.2709999999999999,"15min":1.325},"min":{"1min":0.59199999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6739999999999999,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":1.448},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.2110000000000001,"15min":1.2669999999999999},"min":{"1min":0.54200000000000004,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.754,"5min":2.73,"15min":2.73},"last":1.5029999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.3740000000000001,"15min":1.448},"min":{"1min":0.77100000000000002,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.663,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.143},{"interface":"front","average":{"1min":1.204,"5min":1.393,"15min":1.452},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.5589999999999999,"5min":3.052,"15min":3.052},"last":1.204}]},{"osd":5,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.4339999999999999,"15min":1.498},"min":{"1min":0.74299999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0430000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":0.996},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.4670000000000001,"15min":1.5309999999999999},"min":{"1min":0.88300000000000001,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.948,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.2509999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.4299999999999999,"15min":1.4399999999999999},"min":{"1min":1.004,"5min":0.878,"15min":0.878},"max":{"1min":3.21,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.5269999999999999,"15min":1.585},"min":{"1min":0.79500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0880000000000001,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.403}]},{"osd":7,"last update":"Wed Nov 8 19:27:52 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.6140000000000001,"15min":1.71},"min":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":1.948,"5min":4.407,"15min":4.407},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.528,"15min":1.581},"min":{"1min":0.84399999999999997,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":2.0059999999999998,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.5489999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32296,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1142784,"data_stored":564000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.82999999999999996,"15min":0.86299999999999999},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.879,"15min":0.92700000000000005},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.171,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.73699999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.83499999999999996,"15min":0.86699999999999999},"min":{"1min":0.45900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.198,"5min":1.393,"15min":1.393},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97099999999999997,"15min":1.016},"min":{"1min":0.54400000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.4690000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":1.032}]},{"osd":3,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0229999999999999,"5min":1.022,"15min":1.05},"min":{"1min":0.55400000000000005,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.0600000000000001,"15min":1.1259999999999999},"min":{"1min":0.52000000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6060000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.3169999999999999,"15min":1.407},"min":{"1min":0.81499999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2969999999999999,"15min":1.365},"min":{"1min":0.81999999999999995,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.7,"5min":2.278,"15min":2.278},"last":1.4450000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3120000000000001,"5min":1.3600000000000001,"15min":1.425},"min":{"1min":0.76600000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":2.1600000000000001,"5min":2.2000000000000002,"15min":2.2000000000000002},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.373,"15min":1.4610000000000001},"min":{"1min":0.78500000000000003,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.954,"5min":2.536,"15min":2.536},"last":1.3799999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.347,"15min":1.389},"min":{"1min":0.93100000000000005,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9119999999999999,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.548},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.395,"15min":1.4590000000000001},"min":{"1min":0.83799999999999997,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.036,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.3460000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:27:54 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.4930000000000001,"15min":1.577},"min":{"1min":0.85799999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.861,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.403,"5min":1.492,"15min":1.556},"min":{"1min":0.84999999999999998,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.0030000000000001,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.639}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1146880,"data_stored":565019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.754,"15min":0.745},"min":{"1min":0.497,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.304,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.95099999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.74399999999999999,"15min":0.751},"min":{"1min":0.38700000000000001,"5min":0.374,"15min":0.374},"max":{"1min":1.123,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.78200000000000003,"15min":0.79200000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2709999999999999},"last":1.3500000000000001},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.56799999999999995,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.1519999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0600000000000001},"min":{"1min":0.65000000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.107,"5min":1.0700000000000001,"15min":1.0860000000000001},"min":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":2.2690000000000001,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.276}]},{"osd":4,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.147,"15min":1.1539999999999999},"min":{"1min":0.78900000000000003,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.556,"5min":2.165,"15min":2.165},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1899999999999999,"5min":1.226,"15min":1.252},"min":{"1min":0.77400000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.454,"5min":1.885,"15min":1.885},"last":1.7589999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":1.2290000000000001,"15min":1.2549999999999999},"min":{"1min":0.73599999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5660000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.274,"15min":1.321},"min":{"1min":0.78900000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.579,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.5169999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3580000000000001,"15min":1.3660000000000001},"min":{"1min":0.84999999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.1509999999999998,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.3540000000000001,"15min":1.3799999999999999},"min":{"1min":0.92400000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":1.724,"5min":2.282,"15min":2.282},"last":1.6279999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:27:49 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.3200000000000001,"15min":1.3300000000000001},"min":{"1min":1.109,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.909,"5min":2.0409999999999999,"15min":2.0409999999999999},"last":1.8200000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.367,"15min":1.3899999999999999},"min":{"1min":0.82399999999999995,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":2.4830000000000001,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.615}]}]},{"osd":0,"up_from":13,"seq":55834574901,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1724416,"data_stored":1147140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.296,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.872,"15min":0.92800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3180000000000001,"5min":1.7,"15min":1.7},"last":0.92900000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.86599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.284,"5min":1.508,"15min":1.508},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.97199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.407,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.256}]},{"osd":3,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.6599999999999999,"5min":1.748,"15min":1.748},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0409999999999999,"15min":1.099},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.506,"5min":2.496,"15min":2.496},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5609999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.206},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3049999999999999,"15min":1.4199999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.54,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3979999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.286,"15min":1.3600000000000001},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.516},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.355,"15min":1.4330000000000001},"min":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.534}]},{"osd":6,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4870000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1619999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3540000000000001,"15min":1.405},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.3660000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.47},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.405,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.881,"5min":2.306,"15min":2.306},"last":1.702}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:28:58.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3f is not active+clean 2023-11-08T19:28:58.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3e is not active+clean 2023-11-08T19:28:58.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3d is not active+clean 2023-11-08T19:28:58.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3c is not active+clean 2023-11-08T19:28:58.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3b is not active+clean 2023-11-08T19:28:58.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3a is not active+clean 2023-11-08T19:28:58.581 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.581 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.39 is not active+clean 2023-11-08T19:28:58.582 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.582 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.38 is not active+clean 2023-11-08T19:28:58.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.583 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.37 is not active+clean 2023-11-08T19:28:58.583 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.584 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.36 is not active+clean 2023-11-08T19:28:58.584 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.35 is not active+clean 2023-11-08T19:28:58.585 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.585 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.34 is not active+clean 2023-11-08T19:28:58.586 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.586 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.33 is not active+clean 2023-11-08T19:28:58.587 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.587 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.32 is not active+clean 2023-11-08T19:28:58.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.588 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.31 is not active+clean 2023-11-08T19:28:58.588 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.30 is not active+clean 2023-11-08T19:28:58.589 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.589 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2f is not active+clean 2023-11-08T19:28:58.590 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.590 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2e is not active+clean 2023-11-08T19:28:58.591 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.591 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2d is not active+clean 2023-11-08T19:28:58.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.592 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2c is not active+clean 2023-11-08T19:28:58.592 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.593 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2b is not active+clean 2023-11-08T19:28:58.593 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2a is not active+clean 2023-11-08T19:28:58.594 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.594 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.29 is not active+clean 2023-11-08T19:28:58.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.595 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.28 is not active+clean 2023-11-08T19:28:58.595 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.596 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.27 is not active+clean 2023-11-08T19:28:58.596 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.597 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.26 is not active+clean 2023-11-08T19:28:58.597 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.25 is not active+clean 2023-11-08T19:28:58.598 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.598 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.24 is not active+clean 2023-11-08T19:28:58.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.599 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.23 is not active+clean 2023-11-08T19:28:58.599 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.600 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.22 is not active+clean 2023-11-08T19:28:58.600 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.21 is not active+clean 2023-11-08T19:28:58.601 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.601 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.20 is not active+clean 2023-11-08T19:28:58.602 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.602 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.b is not active+clean 2023-11-08T19:28:58.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.603 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-11-08T19:28:58.603 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.019781+0000', 'last_change': '2023-11-08T19:28:51.019781+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.019781+0000', 'last_undegraded': '2023-11-08T19:28:51.019781+0000', 'last_fullsized': '2023-11-08T19:28:51.019781+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.a is not active+clean 2023-11-08T19:28:58.604 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.604 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-11-08T19:28:58.605 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.012025+0000', 'last_change': '2023-11-08T19:28:51.012025+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.012025+0000', 'last_undegraded': '2023-11-08T19:28:51.012025+0000', 'last_fullsized': '2023-11-08T19:28:51.012025+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:58.605 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.9 is not active+clean 2023-11-08T19:28:58.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.606 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-11-08T19:28:58.606 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.007233+0000', 'last_change': '2023-11-08T19:28:51.007233+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.007233+0000', 'last_undegraded': '2023-11-08T19:28:51.007233+0000', 'last_fullsized': '2023-11-08T19:28:51.007233+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.607 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.8 is not active+clean 2023-11-08T19:28:58.607 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-11-08T19:28:58.608 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.011921+0000', 'last_change': '2023-11-08T19:28:51.011921+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.011921+0000', 'last_undegraded': '2023-11-08T19:28:51.011921+0000', 'last_fullsized': '2023-11-08T19:28:51.011921+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:58.608 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.f is not active+clean 2023-11-08T19:28:58.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.609 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-11-08T19:28:58.609 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.021469+0000', 'last_change': '2023-11-08T19:28:51.021469+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.021469+0000', 'last_undegraded': '2023-11-08T19:28:51.021469+0000', 'last_fullsized': '2023-11-08T19:28:51.021469+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.610 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.e is not active+clean 2023-11-08T19:28:58.610 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-11-08T19:28:58.611 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.004832+0000', 'last_change': '2023-11-08T19:28:51.004832+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.004832+0000', 'last_undegraded': '2023-11-08T19:28:51.004832+0000', 'last_fullsized': '2023-11-08T19:28:51.004832+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.611 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.d is not active+clean 2023-11-08T19:28:58.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.612 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-11-08T19:28:58.612 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.004676+0000', 'last_change': '2023-11-08T19:28:51.004676+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.004676+0000', 'last_undegraded': '2023-11-08T19:28:51.004676+0000', 'last_fullsized': '2023-11-08T19:28:51.004676+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:58.613 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.c is not active+clean 2023-11-08T19:28:58.613 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-11-08T19:28:58.614 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.015351+0000', 'last_change': '2023-11-08T19:28:51.015351+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.015351+0000', 'last_undegraded': '2023-11-08T19:28:51.015351+0000', 'last_fullsized': '2023-11-08T19:28:51.015351+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:28:58.614 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.3 is not active+clean 2023-11-08T19:28:58.615 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.615 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-11-08T19:28:58.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.009689+0000', 'last_change': '2023-11-08T19:28:51.009689+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.009689+0000', 'last_undegraded': '2023-11-08T19:28:51.009689+0000', 'last_fullsized': '2023-11-08T19:28:51.009689+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:58.616 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.2 is not active+clean 2023-11-08T19:28:58.616 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.617 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-11-08T19:28:58.617 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.005396+0000', 'last_change': '2023-11-08T19:28:51.005396+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.005396+0000', 'last_undegraded': '2023-11-08T19:28:51.005396+0000', 'last_fullsized': '2023-11-08T19:28:51.005396+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:58.618 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1 is not active+clean 2023-11-08T19:28:58.618 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.619 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-11-08T19:28:58.619 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.005496+0000', 'last_change': '2023-11-08T19:28:51.005496+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.005496+0000', 'last_undegraded': '2023-11-08T19:28:51.005496+0000', 'last_fullsized': '2023-11-08T19:28:51.005496+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.0 is not active+clean 2023-11-08T19:28:58.620 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.620 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-11-08T19:28:58.621 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.020306+0000', 'last_change': '2023-11-08T19:28:51.020306+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.020306+0000', 'last_undegraded': '2023-11-08T19:28:51.020306+0000', 'last_fullsized': '2023-11-08T19:28:51.020306+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.621 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.7 is not active+clean 2023-11-08T19:28:58.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.622 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-11-08T19:28:58.622 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.015226+0000', 'last_change': '2023-11-08T19:28:51.015226+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.015226+0000', 'last_undegraded': '2023-11-08T19:28:51.015226+0000', 'last_fullsized': '2023-11-08T19:28:51.015226+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:28:58.623 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.6 is not active+clean 2023-11-08T19:28:58.623 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-11-08T19:28:58.624 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.010187+0000', 'last_change': '2023-11-08T19:28:51.010187+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.010187+0000', 'last_undegraded': '2023-11-08T19:28:51.010187+0000', 'last_fullsized': '2023-11-08T19:28:51.010187+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:58.624 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.5 is not active+clean 2023-11-08T19:28:58.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.625 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-11-08T19:28:58.625 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.010260+0000', 'last_change': '2023-11-08T19:28:51.010260+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.010260+0000', 'last_undegraded': '2023-11-08T19:28:51.010260+0000', 'last_fullsized': '2023-11-08T19:28:51.010260+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:58.626 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.4 is not active+clean 2023-11-08T19:28:58.626 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-11-08T19:28:58.627 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.005943+0000', 'last_change': '2023-11-08T19:28:51.005943+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.005943+0000', 'last_undegraded': '2023-11-08T19:28:51.005943+0000', 'last_fullsized': '2023-11-08T19:28:51.005943+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.627 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1b is not active+clean 2023-11-08T19:28:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.628 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-11-08T19:28:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.009642+0000', 'last_change': '2023-11-08T19:28:51.009642+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.009642+0000', 'last_undegraded': '2023-11-08T19:28:51.009642+0000', 'last_fullsized': '2023-11-08T19:28:51.009642+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1a is not active+clean 2023-11-08T19:28:58.629 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-11-08T19:28:58.630 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.006470+0000', 'last_change': '2023-11-08T19:28:51.006470+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.006470+0000', 'last_undegraded': '2023-11-08T19:28:51.006470+0000', 'last_fullsized': '2023-11-08T19:28:51.006470+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.19 is not active+clean 2023-11-08T19:28:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.631 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-11-08T19:28:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.008922+0000', 'last_change': '2023-11-08T19:28:51.008922+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.008922+0000', 'last_undegraded': '2023-11-08T19:28:51.008922+0000', 'last_fullsized': '2023-11-08T19:28:51.008922+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.632 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.18 is not active+clean 2023-11-08T19:28:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-11-08T19:28:58.633 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.005345+0000', 'last_change': '2023-11-08T19:28:51.005345+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.005345+0000', 'last_undegraded': '2023-11-08T19:28:51.005345+0000', 'last_fullsized': '2023-11-08T19:28:51.005345+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-08T19:28:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1f is not active+clean 2023-11-08T19:28:58.634 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-11-08T19:28:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.016804+0000', 'last_change': '2023-11-08T19:28:51.016804+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.016804+0000', 'last_undegraded': '2023-11-08T19:28:51.016804+0000', 'last_fullsized': '2023-11-08T19:28:51.016804+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-08T19:28:58.635 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1e is not active+clean 2023-11-08T19:28:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.636 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-11-08T19:28:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.004718+0000', 'last_change': '2023-11-08T19:28:51.004718+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.004718+0000', 'last_undegraded': '2023-11-08T19:28:51.004718+0000', 'last_fullsized': '2023-11-08T19:28:51.004718+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1d is not active+clean 2023-11-08T19:28:58.637 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-11-08T19:28:58.638 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.020099+0000', 'last_change': '2023-11-08T19:28:51.020099+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.020099+0000', 'last_undegraded': '2023-11-08T19:28:51.020099+0000', 'last_fullsized': '2023-11-08T19:28:51.020099+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.1c is not active+clean 2023-11-08T19:28:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.639 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-11-08T19:28:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.021701+0000', 'last_change': '2023-11-08T19:28:51.021701+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.021701+0000', 'last_undegraded': '2023-11-08T19:28:51.021701+0000', 'last_fullsized': '2023-11-08T19:28:51.021701+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.640 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.13 is not active+clean 2023-11-08T19:28:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-11-08T19:28:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.015270+0000', 'last_change': '2023-11-08T19:28:51.015270+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.015270+0000', 'last_undegraded': '2023-11-08T19:28:51.015270+0000', 'last_fullsized': '2023-11-08T19:28:51.015270+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-08T19:28:58.642 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.12 is not active+clean 2023-11-08T19:28:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-11-08T19:28:58.643 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.012488+0000', 'last_change': '2023-11-08T19:28:51.012488+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.012488+0000', 'last_undegraded': '2023-11-08T19:28:51.012488+0000', 'last_fullsized': '2023-11-08T19:28:51.012488+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-08T19:28:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.11 is not active+clean 2023-11-08T19:28:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.644 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-11-08T19:28:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.018696+0000', 'last_change': '2023-11-08T19:28:51.018696+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.018696+0000', 'last_undegraded': '2023-11-08T19:28:51.018696+0000', 'last_fullsized': '2023-11-08T19:28:51.018696+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.645 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.17 is not active+clean 2023-11-08T19:28:58.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.646 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-11-08T19:28:58.646 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.008313+0000', 'last_change': '2023-11-08T19:28:51.008313+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.008313+0000', 'last_undegraded': '2023-11-08T19:28:51.008313+0000', 'last_fullsized': '2023-11-08T19:28:51.008313+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-08T19:28:58.647 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.16 is not active+clean 2023-11-08T19:28:58.647 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-11-08T19:28:58.648 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.019995+0000', 'last_change': '2023-11-08T19:28:51.019995+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.019995+0000', 'last_undegraded': '2023-11-08T19:28:51.019995+0000', 'last_fullsized': '2023-11-08T19:28:51.019995+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-08T19:28:58.648 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.15 is not active+clean 2023-11-08T19:28:58.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.649 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-11-08T19:28:58.649 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.004935+0000', 'last_change': '2023-11-08T19:28:51.004935+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.004935+0000', 'last_undegraded': '2023-11-08T19:28:51.004935+0000', 'last_fullsized': '2023-11-08T19:28:51.004935+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.650 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.14 is not active+clean 2023-11-08T19:28:58.650 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-11-08T19:28:58.651 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.007275+0000', 'last_change': '2023-11-08T19:28:51.007275+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.007275+0000', 'last_undegraded': '2023-11-08T19:28:51.007275+0000', 'last_fullsized': '2023-11-08T19:28:51.007275+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-08T19:28:58.651 INFO:tasks.cephfs.filesystem.ceph_manager:PG 17.10 is not active+clean 2023-11-08T19:28:58.652 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '17.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-08T19:28:51.998042+0000', 'last_change': '2023-11-08T19:28:51.998042+0000', 'last_active': '2023-11-08T19:28:51.998042+0000', 'last_peered': '2023-11-08T19:28:51.998042+0000', 'last_clean': '2023-11-08T19:28:51.998042+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.998042+0000', 'last_undegraded': '2023-11-08T19:28:51.998042+0000', 'last_fullsized': '2023-11-08T19:28:51.998042+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:51.998042+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-08T19:28:58.652 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-11-08T19:28:58.653 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 93, 'state': 'creating+peering', 'last_fresh': '2023-11-08T19:28:51.005157+0000', 'last_change': '2023-11-08T19:28:51.005157+0000', 'last_active': '2023-11-08T19:28:50.984789+0000', 'last_peered': '2023-11-08T19:28:50.984789+0000', 'last_clean': '2023-11-08T19:28:50.984789+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-08T19:28:51.005157+0000', 'last_undegraded': '2023-11-08T19:28:51.005157+0000', 'last_fullsized': '2023-11-08T19:28:51.005157+0000', 'mapping_epoch': 93, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 93, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'last_clean_scrub_stamp': '2023-11-08T19:28:50.984789+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-08T19:28:58.654 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-08T19:28:58.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.872+0000 7fb404e00640 1 Processor -- start 2023-11-08T19:28:58.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.873+0000 7fb404e00640 1 -- start start 2023-11-08T19:28:58.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.874+0000 7fb404e00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb4000cc810 0x7fb4000c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.874+0000 7fb404e00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4000ae410 con 0x7fb40006a390 2023-11-08T19:28:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.874+0000 7fb404e00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40006ba30 con 0x7fb40015d180 2023-11-08T19:28:58.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.874+0000 7fb404e00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40006bcf0 con 0x7fb4000cc810 2023-11-08T19:28:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fe59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb40006a390 0x7fb40015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56560/0 (socket says 172.21.15.119:56560) 2023-11-08T19:28:58.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fe59c640 1 -- 172.21.15.119:0/147383822 learned_addr learned my addr 172.21.15.119:0/147383822 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:58.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3f7fff640 1 --2- 172.21.15.119:0/147383822 >> v2:172.21.15.188:3300/0 conn(0x7fb4000cc810 0x7fb4000c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786955211 0 0) 0x7fb4000ae410 con 0x7fb40006a390 2023-11-08T19:28:58.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3ec003520 con 0x7fb40006a390 2023-11-08T19:28:58.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274227745 0 0) 0x7fb3ec003520 con 0x7fb40006a390 2023-11-08T19:28:58.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 >> v2:172.21.15.188:3300/0 conn(0x7fb4000cc810 msgr2=0x7fb4000c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:58.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 --2- 172.21.15.119:0/147383822 >> v2:172.21.15.188:3300/0 conn(0x7fb4000cc810 0x7fb4000c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 >> v1:172.21.15.188:6789/0 conn(0x7fb40015d180 legacy=0x7fb400160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.875+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40006bfd0 con 0x7fb40006a390 2023-11-08T19:28:58.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fb3f0001670 con 0x7fb40006a390 2023-11-08T19:28:58.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3f7fff640 1 --2- 172.21.15.119:0/147383822 >> v2:172.21.15.188:3300/0 conn(0x7fb4000cc810 0x7fb4000c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:28:58.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 >> v1:172.21.15.119:6789/0 conn(0x7fb40006a390 legacy=0x7fb40015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb3ec003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3ec0051d0 0x7fb3ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40006bcf0 con 0x7fb3ec0039c0 2023-11-08T19:28:58.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec00c120 con 0x7fb3ec0051d0 2023-11-08T19:28:58.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3ec00c2e0 con 0x7fb3ec0047c0 2023-11-08T19:28:58.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb3ec003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fe59c640 1 -- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3ec0047c0 msgr2=0x7fb3ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:58.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb3ec0047c0 0x7fb3ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fe59c640 1 -- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3ec0051d0 msgr2=0x7fb3ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:58.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fed9d640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3ec0051d0 0x7fb3ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb3ec0051d0 0x7fb3ec00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fed9d640 1 -- 172.21.15.119:0/147383822 reap_dead start 2023-11-08T19:28:58.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.876+0000 7fb3fe59c640 1 -- 172.21.15.119:0/147383822 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3ec003520 con 0x7fb3ec0039c0 2023-11-08T19:28:58.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.878+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb3ec003de0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c0e0 tx=0x7fb3ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e184b69219d2985 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:58.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.878+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0005ad0 con 0x7fb3ec0039c0 2023-11-08T19:28:58.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.878+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f0005c30 con 0x7fb3ec0039c0 2023-11-08T19:28:58.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.878+0000 7fb3fdd9b640 1 -- 172.21.15.119:0/147383822 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f002ca50 con 0x7fb3ec0039c0 2023-11-08T19:28:58.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.878+0000 7fb404e00640 1 -- 172.21.15.119:0/147383822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 msgr2=0x7fb3ec003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.878+0000 7fb404e00640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb3ec003de0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c0e0 tx=0x7fb3ec0057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.879+0000 7fb404e00640 1 -- 172.21.15.119:0/147383822 shutdown_connections 2023-11-08T19:28:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.879+0000 7fb404e00640 1 --2- 172.21.15.119:0/147383822 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb3ec003de0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.879+0000 7fb404e00640 1 -- 172.21.15.119:0/147383822 >> 172.21.15.119:0/147383822 conn(0x7fb4000cb610 msgr2=0x7fb4000cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.879+0000 7fb404e00640 1 -- 172.21.15.119:0/147383822 shutdown_connections 2023-11-08T19:28:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.879+0000 7fb404e00640 1 -- 172.21.15.119:0/147383822 wait complete. 2023-11-08T19:28:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.880+0000 7fb404e00640 1 Processor -- start 2023-11-08T19:28:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.881+0000 7fb404e00640 1 -- start start 2023-11-08T19:28:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.881+0000 7fb404e00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb400153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.881+0000 7fb404e00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4001466c0 0x7fb4000c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb404e00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4000c8460 0x7fb4000c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb404e00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4000c88e0 con 0x7fb3ec0039c0 2023-11-08T19:28:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb404e00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb400153dd0 con 0x7fb4000c8460 2023-11-08T19:28:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb404e00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40015b9e0 con 0x7fb4001466c0 2023-11-08T19:28:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb400153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb400153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50234/0 (socket says 172.21.15.119:50234) 2023-11-08T19:28:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 -- 172.21.15.119:0/3811812466 learned_addr learned my addr 172.21.15.119:0/3811812466 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3f7fff640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4001466c0 0x7fb4000c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 -- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4001466c0 msgr2=0x7fb4000c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4001466c0 0x7fb4000c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 -- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4000c8460 msgr2=0x7fb4000c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:58.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fed9d640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4000c8460 0x7fb4000c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4000c8460 0x7fb4000c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.882+0000 7fb3fe59c640 1 -- 172.21.15.119:0/3811812466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400146b40 con 0x7fb3ec0039c0 2023-11-08T19:28:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.883+0000 7fb3fe59c640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb400153550 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c0e0 tx=0x7fb3f002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.883+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f003c040 con 0x7fb3ec0039c0 2023-11-08T19:28:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.883+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f002cd60 con 0x7fb3ec0039c0 2023-11-08T19:28:58.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.883+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40015b0b0 con 0x7fb3ec0039c0 2023-11-08T19:28:58.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.883+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00072b0 con 0x7fb3ec0039c0 2023-11-08T19:28:58.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.883+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40015b670 con 0x7fb3ec0039c0 2023-11-08T19:28:58.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.884+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40015b830 con 0x7fb3ec0039c0 2023-11-08T19:28:58.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.885+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb3f0032070 con 0x7fb3ec0039c0 2023-11-08T19:28:58.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.886+0000 7fb3f6ffd640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb3e006ae70 0x7fb3e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:58.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.886+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fb3f00ab080 con 0x7fb3ec0039c0 2023-11-08T19:28:58.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.886+0000 7fb3f7fff640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb3e006ae70 0x7fb3e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:58.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.887+0000 7fb3f7fff640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb3e006ae70 0x7fb3e006d330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006a90 tx=0x7fb3e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:58.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:58.890+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb3f00ddca0 con 0x7fb3ec0039c0 2023-11-08T19:28:59.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.147+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb400074430 con 0x7fb3ec0039c0 2023-11-08T19:28:59.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.148+0000 7fb3f6ffd640 1 -- 172.21.15.119:0/3811812466 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1673 (secure 0 0 0) 0x7fb3f0075bc0 con 0x7fb3ec0039c0 2023-11-08T19:28:59.149 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:59.149 INFO:teuthology.orchestra.run.smithi119.stdout:{"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","health":{"status":"HEALTH_WARN","checks":{"POOL_APP_NOT_ENABLED":{"severity":"HEALTH_WARN","summary":{"message":"3 pool(s) do not have an application enabled","count":3},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":265,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":96,"num_osds":8,"num_up_osds":8,"osd_up_since":1699471485,"num_in_osds":8,"osd_in_since":1699471477,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97},{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32}],"num_pgs":193,"num_pools":6,"num_objects":2,"data_bytes":590368,"bytes_used":265777152,"bytes_avail":772828336128,"bytes_total":773094113280,"unknown_pgs_ratio":0.33160620927810669,"inactive_pgs_ratio":0.16580310463905334},"fsmap":{"epoch":53,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"b","status":"up:active","gid":7068}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2023-11-08T19:28:00.721187+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-08T19:28:59.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb3e006ae70 msgr2=0x7fb3e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb3e006ae70 0x7fb3e006d330 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006a90 tx=0x7fb3e8008040 comp rx=0 tx=0).stop 2023-11-08T19:28:59.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 msgr2=0x7fb400153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb400153550 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c0e0 tx=0x7fb3f002e070 comp rx=0 tx=0).stop 2023-11-08T19:28:59.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 shutdown_connections 2023-11-08T19:28:59.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb3e006ae70 0x7fb3e006d330 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb4000c8460 0x7fb4000c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb4001466c0 0x7fb4000c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 --2- 172.21.15.119:0/3811812466 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb3ec0039c0 0x7fb400153550 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 >> 172.21.15.119:0/3811812466 conn(0x7fb4000cb610 msgr2=0x7fb4000c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 shutdown_connections 2023-11-08T19:28:59.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.150+0000 7fb404e00640 1 -- 172.21.15.119:0/3811812466 wait complete. 2023-11-08T19:28:59.168 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-08T19:28:59.168 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:28:59.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.378+0000 7fe05a99e640 1 Processor -- start 2023-11-08T19:28:59.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.378+0000 7fe05a99e640 1 -- start start 2023-11-08T19:28:59.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.379+0000 7fe05a99e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe05406a390 0x7fe0540c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.379+0000 7fe05a99e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0540ae410 con 0x7fe05415d180 2023-11-08T19:28:59.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.379+0000 7fe05a99e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe05406ba30 con 0x7fe0540cc810 2023-11-08T19:28:59.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.379+0000 7fe05a99e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe05406bcf0 con 0x7fe05406a390 2023-11-08T19:28:59.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.380+0000 7fe058f3b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe05415d180 0x7fe054160bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56570/0 (socket says 172.21.15.119:56570) 2023-11-08T19:28:59.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.380+0000 7fe058f3b640 1 -- 172.21.15.119:0/263678801 learned_addr learned my addr 172.21.15.119:0/263678801 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:28:59.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.380+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4067465536 0 0) 0x7fe0540ae410 con 0x7fe05415d180 2023-11-08T19:28:59.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.380+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe040003520 con 0x7fe05415d180 2023-11-08T19:28:59.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.380+0000 7fe053fff640 1 --2- 172.21.15.119:0/263678801 >> v2:172.21.15.188:3300/0 conn(0x7fe05406a390 0x7fe0540c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:59.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 439369122 0 0) 0x7fe040003520 con 0x7fe05415d180 2023-11-08T19:28:59.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 >> v2:172.21.15.188:3300/0 conn(0x7fe05406a390 msgr2=0x7fe0540c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:59.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 --2- 172.21.15.119:0/263678801 >> v2:172.21.15.188:3300/0 conn(0x7fe05406a390 0x7fe0540c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 >> v1:172.21.15.188:6789/0 conn(0x7fe0540cc810 legacy=0x7fe05415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:59.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05406bfd0 con 0x7fe05415d180 2023-11-08T19:28:59.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe04c0019f0 con 0x7fe05415d180 2023-11-08T19:28:59.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 >> v1:172.21.15.119:6789/0 conn(0x7fe05415d180 legacy=0x7fe054160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400039c0 0x7fe040003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe040004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400051d0 0x7fe04000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05406bcf0 con 0x7fe0400047c0 2023-11-08T19:28:59.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04000c100 con 0x7fe0400039c0 2023-11-08T19:28:59.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04000c2c0 con 0x7fe0400051d0 2023-11-08T19:28:59.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe04bfff640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe040004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:59.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe053fff640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400039c0 0x7fe040003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:59.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe058f3b640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400051d0 0x7fe04000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:59.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe04bfff640 1 -- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400051d0 msgr2=0x7fe04000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:59.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe04bfff640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0400051d0 0x7fe04000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe04bfff640 1 -- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400039c0 msgr2=0x7fe040003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:28:59.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.381+0000 7fe04bfff640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0400039c0 0x7fe040003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.382+0000 7fe04bfff640 1 -- 172.21.15.119:0/263678801 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040003520 con 0x7fe0400047c0 2023-11-08T19:28:59.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.382+0000 7fe058f3b640 1 -- 172.21.15.119:0/263678801 reap_dead start 2023-11-08T19:28:59.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.383+0000 7fe04bfff640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe040004c40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ccc60 tx=0x7fe0400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24e8a47bb7a9b7f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:59.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.383+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c032600 con 0x7fe0400047c0 2023-11-08T19:28:59.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.383+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c02e050 con 0x7fe0400047c0 2023-11-08T19:28:59.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.383+0000 7fe0537fe640 1 -- 172.21.15.119:0/263678801 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c0329e0 con 0x7fe0400047c0 2023-11-08T19:28:59.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.383+0000 7fe05a99e640 1 -- 172.21.15.119:0/263678801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 msgr2=0x7fe040004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.383+0000 7fe05a99e640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe040004c40 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fe0540ccc60 tx=0x7fe0400057f0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.384+0000 7fe05a99e640 1 -- 172.21.15.119:0/263678801 shutdown_connections 2023-11-08T19:28:59.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.384+0000 7fe05a99e640 1 --2- 172.21.15.119:0/263678801 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe040004c40 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.384+0000 7fe05a99e640 1 -- 172.21.15.119:0/263678801 >> 172.21.15.119:0/263678801 conn(0x7fe0540cb610 msgr2=0x7fe0540cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:59.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.384+0000 7fe05a99e640 1 -- 172.21.15.119:0/263678801 shutdown_connections 2023-11-08T19:28:59.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.384+0000 7fe05a99e640 1 -- 172.21.15.119:0/263678801 wait complete. 2023-11-08T19:28:59.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.385+0000 7fe05a99e640 1 Processor -- start 2023-11-08T19:28:59.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.386+0000 7fe05a99e640 1 -- start start 2023-11-08T19:28:59.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe05a99e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe0540b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe05a99e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0540ae0a0 0x7fe054144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe05a99e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0540afca0 0x7fe0540b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe05a99e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0540ae520 con 0x7fe0400047c0 2023-11-08T19:28:59.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe05a99e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415d780 con 0x7fe0540ae0a0 2023-11-08T19:28:59.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe05a99e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe05415da60 con 0x7fe0540afca0 2023-11-08T19:28:59.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe0540b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:59.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe0540b0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50238/0 (socket says 172.21.15.119:50238) 2023-11-08T19:28:59.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 -- 172.21.15.119:0/4110674118 learned_addr learned my addr 172.21.15.119:0/4110674118 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:28:59.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 -- 172.21.15.119:0/4110674118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0540afca0 msgr2=0x7fe0540b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0540afca0 0x7fe0540b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 -- 172.21.15.119:0/4110674118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0540ae0a0 msgr2=0x7fe054144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0540ae0a0 0x7fe054144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.387+0000 7fe053fff640 1 -- 172.21.15.119:0/4110674118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe05415dd40 con 0x7fe0400047c0 2023-11-08T19:28:59.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.388+0000 7fe053fff640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe0540b0a60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe04400f600 tx=0x7fe04400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:59.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.388+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044014870 con 0x7fe0400047c0 2023-11-08T19:28:59.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.388+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0440149d0 con 0x7fe0400047c0 2023-11-08T19:28:59.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.388+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044014d40 con 0x7fe0400047c0 2023-11-08T19:28:59.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.388+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05415e020 con 0x7fe0400047c0 2023-11-08T19:28:59.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.388+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0540b0660 con 0x7fe0400047c0 2023-11-08T19:28:59.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.389+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe05415e1b0 con 0x7fe0400047c0 2023-11-08T19:28:59.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.390+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe04400d490 con 0x7fe0400047c0 2023-11-08T19:28:59.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.391+0000 7fe0517fa640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe03406ae70 0x7fe03406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:28:59.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.392+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fe04408a580 con 0x7fe0400047c0 2023-11-08T19:28:59.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.392+0000 7fe04bfff640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe03406ae70 0x7fe03406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:28:59.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.392+0000 7fe04bfff640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe03406ae70 0x7fe03406d330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe03c034000 tx=0x7fe03c02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:28:59.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.396+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe044055140 con 0x7fe0400047c0 2023-11-08T19:28:59.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.568+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe05406c590 con 0x7fe03406ae70 2023-11-08T19:28:59.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.584+0000 7fe0517fa640 1 -- 172.21.15.119:0/4110674118 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529702 (secure 0 0 0) 0x7fe03c007a70 con 0x7fe03406ae70 2023-11-08T19:28:59.592 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:28:59.596 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:28:59.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe03406ae70 msgr2=0x7fe03406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe03406ae70 0x7fe03406d330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe03c034000 tx=0x7fe03c02f000 comp rx=0 tx=0).stop 2023-11-08T19:28:59.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 msgr2=0x7fe0540b0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:28:59.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe0540b0a60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fe04400f600 tx=0x7fe04400fac0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 shutdown_connections 2023-11-08T19:28:59.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe03406ae70 0x7fe03406d330 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe0540afca0 0x7fe0540b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe0540ae0a0 0x7fe054144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 --2- 172.21.15.119:0/4110674118 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0400047c0 0x7fe0540b0a60 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:28:59.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 >> 172.21.15.119:0/4110674118 conn(0x7fe0540cb610 msgr2=0x7fe05415a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:28:59.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.591+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 shutdown_connections 2023-11-08T19:28:59.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:28:59.592+0000 7fe05a99e640 1 -- 172.21.15.119:0/4110674118 wait complete. 2023-11-08T19:28:59.612 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2023-11-08T19:28:58.742008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":352,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259920,"kb_used_data":10464,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714800,"statfs":{"total":773094113280,"available":772827955200,"internally_reserved":0,"allocated":10715136,"data_stored":5892334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.725113"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.054351+0000","last_change":"2023-11-08T19:28:53.076247+0000","last_active":"2023-11-08T19:28:54.054351+0000","last_peered":"2023-11-08T19:28:54.054351+0000","last_clean":"2023-11-08T19:28:54.054351+0000","last_became_active":"2023-11-08T19:28:53.075822+0000","last_became_peered":"2023-11-08T19:28:53.075822+0000","last_unstale":"2023-11-08T19:28:54.054351+0000","last_undegraded":"2023-11-08T19:28:54.054351+0000","last_fullsized":"2023-11-08T19:28:54.054351+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:36:02.843749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060887+0000","last_change":"2023-11-08T19:28:53.056957+0000","last_active":"2023-11-08T19:28:54.060887+0000","last_peered":"2023-11-08T19:28:54.060887+0000","last_clean":"2023-11-08T19:28:54.060887+0000","last_became_active":"2023-11-08T19:28:53.056624+0000","last_became_peered":"2023-11-08T19:28:53.056624+0000","last_unstale":"2023-11-08T19:28:54.060887+0000","last_undegraded":"2023-11-08T19:28:54.060887+0000","last_fullsized":"2023-11-08T19:28:54.060887+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:00.567332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.201294+0000","last_change":"2023-11-08T19:28:53.069343+0000","last_active":"2023-11-08T19:28:54.201294+0000","last_peered":"2023-11-08T19:28:54.201294+0000","last_clean":"2023-11-08T19:28:54.201294+0000","last_became_active":"2023-11-08T19:28:53.068214+0000","last_became_peered":"2023-11-08T19:28:53.068214+0000","last_unstale":"2023-11-08T19:28:54.201294+0000","last_undegraded":"2023-11-08T19:28:54.201294+0000","last_fullsized":"2023-11-08T19:28:54.201294+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:08.926976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053347+0000","last_change":"2023-11-08T19:28:53.064559+0000","last_active":"2023-11-08T19:28:54.053347+0000","last_peered":"2023-11-08T19:28:54.053347+0000","last_clean":"2023-11-08T19:28:54.053347+0000","last_became_active":"2023-11-08T19:28:53.063879+0000","last_became_peered":"2023-11-08T19:28:53.063879+0000","last_unstale":"2023-11-08T19:28:54.053347+0000","last_undegraded":"2023-11-08T19:28:54.053347+0000","last_fullsized":"2023-11-08T19:28:54.053347+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:14.186139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308299+0000","last_change":"2023-11-08T19:28:53.057782+0000","last_active":"2023-11-08T19:28:54.308299+0000","last_peered":"2023-11-08T19:28:54.308299+0000","last_clean":"2023-11-08T19:28:54.308299+0000","last_became_active":"2023-11-08T19:28:53.057379+0000","last_became_peered":"2023-11-08T19:28:53.057379+0000","last_unstale":"2023-11-08T19:28:54.308299+0000","last_undegraded":"2023-11-08T19:28:54.308299+0000","last_fullsized":"2023-11-08T19:28:54.308299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:25.561839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309567+0000","last_change":"2023-11-08T19:28:53.064420+0000","last_active":"2023-11-08T19:28:54.309567+0000","last_peered":"2023-11-08T19:28:54.309567+0000","last_clean":"2023-11-08T19:28:54.309567+0000","last_became_active":"2023-11-08T19:28:53.064122+0000","last_became_peered":"2023-11-08T19:28:53.064122+0000","last_unstale":"2023-11-08T19:28:54.309567+0000","last_undegraded":"2023-11-08T19:28:54.309567+0000","last_fullsized":"2023-11-08T19:28:54.309567+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:33:17.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.058786+0000","last_change":"2023-11-08T19:28:53.048517+0000","last_active":"2023-11-08T19:28:54.058786+0000","last_peered":"2023-11-08T19:28:54.058786+0000","last_clean":"2023-11-08T19:28:54.058786+0000","last_became_active":"2023-11-08T19:28:53.048243+0000","last_became_peered":"2023-11-08T19:28:53.048243+0000","last_unstale":"2023-11-08T19:28:54.058786+0000","last_undegraded":"2023-11-08T19:28:54.058786+0000","last_fullsized":"2023-11-08T19:28:54.058786+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:33:17.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200346+0000","last_change":"2023-11-08T19:28:53.070795+0000","last_active":"2023-11-08T19:28:54.200346+0000","last_peered":"2023-11-08T19:28:54.200346+0000","last_clean":"2023-11-08T19:28:54.200346+0000","last_became_active":"2023-11-08T19:28:53.069476+0000","last_became_peered":"2023-11-08T19:28:53.069476+0000","last_unstale":"2023-11-08T19:28:54.200346+0000","last_undegraded":"2023-11-08T19:28:54.200346+0000","last_fullsized":"2023-11-08T19:28:54.200346+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:41:23.508534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.054535+0000","last_change":"2023-11-08T19:28:53.054293+0000","last_active":"2023-11-08T19:28:54.054535+0000","last_peered":"2023-11-08T19:28:54.054535+0000","last_clean":"2023-11-08T19:28:54.054535+0000","last_became_active":"2023-11-08T19:28:53.053826+0000","last_became_peered":"2023-11-08T19:28:53.053826+0000","last_unstale":"2023-11-08T19:28:54.054535+0000","last_undegraded":"2023-11-08T19:28:54.054535+0000","last_fullsized":"2023-11-08T19:28:54.054535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:35:19.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053644+0000","last_change":"2023-11-08T19:28:53.065525+0000","last_active":"2023-11-08T19:28:54.053644+0000","last_peered":"2023-11-08T19:28:54.053644+0000","last_clean":"2023-11-08T19:28:54.053644+0000","last_became_active":"2023-11-08T19:28:53.061285+0000","last_became_peered":"2023-11-08T19:28:53.061285+0000","last_unstale":"2023-11-08T19:28:54.053644+0000","last_undegraded":"2023-11-08T19:28:54.053644+0000","last_fullsized":"2023-11-08T19:28:54.053644+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:07.300630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.059605+0000","last_change":"2023-11-08T19:28:53.054567+0000","last_active":"2023-11-08T19:28:54.059605+0000","last_peered":"2023-11-08T19:28:54.059605+0000","last_clean":"2023-11-08T19:28:54.059605+0000","last_became_active":"2023-11-08T19:28:53.054267+0000","last_became_peered":"2023-11-08T19:28:53.054267+0000","last_unstale":"2023-11-08T19:28:54.059605+0000","last_undegraded":"2023-11-08T19:28:54.059605+0000","last_fullsized":"2023-11-08T19:28:54.059605+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:33:58.968852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053421+0000","last_change":"2023-11-08T19:28:53.062488+0000","last_active":"2023-11-08T19:28:54.053421+0000","last_peered":"2023-11-08T19:28:54.053421+0000","last_clean":"2023-11-08T19:28:54.053421+0000","last_became_active":"2023-11-08T19:28:53.060062+0000","last_became_peered":"2023-11-08T19:28:53.060062+0000","last_unstale":"2023-11-08T19:28:54.053421+0000","last_undegraded":"2023-11-08T19:28:54.053421+0000","last_fullsized":"2023-11-08T19:28:54.053421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:46:06.679410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199869+0000","last_change":"2023-11-08T19:28:53.062160+0000","last_active":"2023-11-08T19:28:54.199869+0000","last_peered":"2023-11-08T19:28:54.199869+0000","last_clean":"2023-11-08T19:28:54.199869+0000","last_became_active":"2023-11-08T19:28:53.061828+0000","last_became_peered":"2023-11-08T19:28:53.061828+0000","last_unstale":"2023-11-08T19:28:54.199869+0000","last_undegraded":"2023-11-08T19:28:54.199869+0000","last_fullsized":"2023-11-08T19:28:54.199869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:48.671222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199698+0000","last_change":"2023-11-08T19:28:53.065503+0000","last_active":"2023-11-08T19:28:54.199698+0000","last_peered":"2023-11-08T19:28:54.199698+0000","last_clean":"2023-11-08T19:28:54.199698+0000","last_became_active":"2023-11-08T19:28:53.063856+0000","last_became_peered":"2023-11-08T19:28:53.063856+0000","last_unstale":"2023-11-08T19:28:54.199698+0000","last_undegraded":"2023-11-08T19:28:54.199698+0000","last_fullsized":"2023-11-08T19:28:54.199698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:57.188860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307269+0000","last_change":"2023-11-08T19:28:53.059797+0000","last_active":"2023-11-08T19:28:54.307269+0000","last_peered":"2023-11-08T19:28:54.307269+0000","last_clean":"2023-11-08T19:28:54.307269+0000","last_became_active":"2023-11-08T19:28:53.059471+0000","last_became_peered":"2023-11-08T19:28:53.059471+0000","last_unstale":"2023-11-08T19:28:54.307269+0000","last_undegraded":"2023-11-08T19:28:54.307269+0000","last_fullsized":"2023-11-08T19:28:54.307269+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:46.316592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052473+0000","last_change":"2023-11-08T19:28:53.057927+0000","last_active":"2023-11-08T19:28:54.052473+0000","last_peered":"2023-11-08T19:28:54.052473+0000","last_clean":"2023-11-08T19:28:54.052473+0000","last_became_active":"2023-11-08T19:28:53.057350+0000","last_became_peered":"2023-11-08T19:28:53.057350+0000","last_unstale":"2023-11-08T19:28:54.052473+0000","last_undegraded":"2023-11-08T19:28:54.052473+0000","last_fullsized":"2023-11-08T19:28:54.052473+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:59.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060907+0000","last_change":"2023-11-08T19:28:53.058637+0000","last_active":"2023-11-08T19:28:54.060907+0000","last_peered":"2023-11-08T19:28:54.060907+0000","last_clean":"2023-11-08T19:28:54.060907+0000","last_became_active":"2023-11-08T19:28:53.058326+0000","last_became_peered":"2023-11-08T19:28:53.058326+0000","last_unstale":"2023-11-08T19:28:54.060907+0000","last_undegraded":"2023-11-08T19:28:54.060907+0000","last_fullsized":"2023-11-08T19:28:54.060907+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:40:32.190737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308249+0000","last_change":"2023-11-08T19:28:53.055902+0000","last_active":"2023-11-08T19:28:54.308249+0000","last_peered":"2023-11-08T19:28:54.308249+0000","last_clean":"2023-11-08T19:28:54.308249+0000","last_became_active":"2023-11-08T19:28:53.055390+0000","last_became_peered":"2023-11-08T19:28:53.055390+0000","last_unstale":"2023-11-08T19:28:54.308249+0000","last_undegraded":"2023-11-08T19:28:54.308249+0000","last_fullsized":"2023-11-08T19:28:54.308249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:37.239173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200445+0000","last_change":"2023-11-08T19:28:53.070459+0000","last_active":"2023-11-08T19:28:54.200445+0000","last_peered":"2023-11-08T19:28:54.200445+0000","last_clean":"2023-11-08T19:28:54.200445+0000","last_became_active":"2023-11-08T19:28:53.067727+0000","last_became_peered":"2023-11-08T19:28:53.067727+0000","last_unstale":"2023-11-08T19:28:54.200445+0000","last_undegraded":"2023-11-08T19:28:54.200445+0000","last_fullsized":"2023-11-08T19:28:54.200445+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:21:34.950327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.059668+0000","last_change":"2023-11-08T19:28:53.047320+0000","last_active":"2023-11-08T19:28:54.059668+0000","last_peered":"2023-11-08T19:28:54.059668+0000","last_clean":"2023-11-08T19:28:54.059668+0000","last_became_active":"2023-11-08T19:28:53.047054+0000","last_became_peered":"2023-11-08T19:28:53.047054+0000","last_unstale":"2023-11-08T19:28:54.059668+0000","last_undegraded":"2023-11-08T19:28:54.059668+0000","last_fullsized":"2023-11-08T19:28:54.059668+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:37.239173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.061624+0000","last_change":"2023-11-08T19:28:53.057226+0000","last_active":"2023-11-08T19:28:54.061624+0000","last_peered":"2023-11-08T19:28:54.061624+0000","last_clean":"2023-11-08T19:28:54.061624+0000","last_became_active":"2023-11-08T19:28:53.056886+0000","last_became_peered":"2023-11-08T19:28:53.056886+0000","last_unstale":"2023-11-08T19:28:54.061624+0000","last_undegraded":"2023-11-08T19:28:54.061624+0000","last_fullsized":"2023-11-08T19:28:54.061624+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:31.880618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309234+0000","last_change":"2023-11-08T19:28:53.066182+0000","last_active":"2023-11-08T19:28:54.309234+0000","last_peered":"2023-11-08T19:28:54.309234+0000","last_clean":"2023-11-08T19:28:54.309234+0000","last_became_active":"2023-11-08T19:28:53.065645+0000","last_became_peered":"2023-11-08T19:28:53.065645+0000","last_unstale":"2023-11-08T19:28:54.309234+0000","last_undegraded":"2023-11-08T19:28:54.309234+0000","last_fullsized":"2023-11-08T19:28:54.309234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:42.397408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062336+0000","last_change":"2023-11-08T19:28:53.048605+0000","last_active":"2023-11-08T19:28:54.062336+0000","last_peered":"2023-11-08T19:28:54.062336+0000","last_clean":"2023-11-08T19:28:54.062336+0000","last_became_active":"2023-11-08T19:28:53.048273+0000","last_became_peered":"2023-11-08T19:28:53.048273+0000","last_unstale":"2023-11-08T19:28:54.062336+0000","last_undegraded":"2023-11-08T19:28:54.062336+0000","last_fullsized":"2023-11-08T19:28:54.062336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:57.188860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308249+0000","last_change":"2023-11-08T19:28:53.067040+0000","last_active":"2023-11-08T19:28:54.308249+0000","last_peered":"2023-11-08T19:28:54.308249+0000","last_clean":"2023-11-08T19:28:54.308249+0000","last_became_active":"2023-11-08T19:28:53.066693+0000","last_became_peered":"2023-11-08T19:28:53.066693+0000","last_unstale":"2023-11-08T19:28:54.308249+0000","last_undegraded":"2023-11-08T19:28:54.308249+0000","last_fullsized":"2023-11-08T19:28:54.308249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:52.001841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060940+0000","last_change":"2023-11-08T19:28:53.064737+0000","last_active":"2023-11-08T19:28:54.060940+0000","last_peered":"2023-11-08T19:28:54.060940+0000","last_clean":"2023-11-08T19:28:54.060940+0000","last_became_active":"2023-11-08T19:28:53.064468+0000","last_became_peered":"2023-11-08T19:28:53.064468+0000","last_unstale":"2023-11-08T19:28:54.060940+0000","last_undegraded":"2023-11-08T19:28:54.060940+0000","last_fullsized":"2023-11-08T19:28:54.060940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:45.310422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053792+0000","last_change":"2023-11-08T19:28:53.058005+0000","last_active":"2023-11-08T19:28:54.053792+0000","last_peered":"2023-11-08T19:28:54.053792+0000","last_clean":"2023-11-08T19:28:54.053792+0000","last_became_active":"2023-11-08T19:28:53.057238+0000","last_became_peered":"2023-11-08T19:28:53.057238+0000","last_unstale":"2023-11-08T19:28:54.053792+0000","last_undegraded":"2023-11-08T19:28:54.053792+0000","last_fullsized":"2023-11-08T19:28:54.053792+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:06.683189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.311724+0000","last_change":"2023-11-08T19:28:53.063577+0000","last_active":"2023-11-08T19:28:54.311724+0000","last_peered":"2023-11-08T19:28:54.311724+0000","last_clean":"2023-11-08T19:28:54.311724+0000","last_became_active":"2023-11-08T19:28:53.061305+0000","last_became_peered":"2023-11-08T19:28:53.061305+0000","last_unstale":"2023-11-08T19:28:54.311724+0000","last_undegraded":"2023-11-08T19:28:54.311724+0000","last_fullsized":"2023-11-08T19:28:54.311724+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:04.246478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.312329+0000","last_change":"2023-11-08T19:28:53.060004+0000","last_active":"2023-11-08T19:28:54.312329+0000","last_peered":"2023-11-08T19:28:54.312329+0000","last_clean":"2023-11-08T19:28:54.312329+0000","last_became_active":"2023-11-08T19:28:53.056312+0000","last_became_peered":"2023-11-08T19:28:53.056312+0000","last_unstale":"2023-11-08T19:28:54.312329+0000","last_undegraded":"2023-11-08T19:28:54.312329+0000","last_fullsized":"2023-11-08T19:28:54.312329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:08.857088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060889+0000","last_change":"2023-11-08T19:28:53.052784+0000","last_active":"2023-11-08T19:28:54.060889+0000","last_peered":"2023-11-08T19:28:54.060889+0000","last_clean":"2023-11-08T19:28:54.060889+0000","last_became_active":"2023-11-08T19:28:53.052458+0000","last_became_peered":"2023-11-08T19:28:53.052458+0000","last_unstale":"2023-11-08T19:28:54.060889+0000","last_undegraded":"2023-11-08T19:28:54.060889+0000","last_fullsized":"2023-11-08T19:28:54.060889+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:38.678500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.061095+0000","last_change":"2023-11-08T19:28:53.058887+0000","last_active":"2023-11-08T19:28:54.061095+0000","last_peered":"2023-11-08T19:28:54.061095+0000","last_clean":"2023-11-08T19:28:54.061095+0000","last_became_active":"2023-11-08T19:28:53.058559+0000","last_became_peered":"2023-11-08T19:28:53.058559+0000","last_unstale":"2023-11-08T19:28:54.061095+0000","last_undegraded":"2023-11-08T19:28:54.061095+0000","last_fullsized":"2023-11-08T19:28:54.061095+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:04.246478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052655+0000","last_change":"2023-11-08T19:28:53.059522+0000","last_active":"2023-11-08T19:28:54.052655+0000","last_peered":"2023-11-08T19:28:54.052655+0000","last_clean":"2023-11-08T19:28:54.052655+0000","last_became_active":"2023-11-08T19:28:53.059240+0000","last_became_peered":"2023-11-08T19:28:53.059240+0000","last_unstale":"2023-11-08T19:28:54.052655+0000","last_undegraded":"2023-11-08T19:28:54.052655+0000","last_fullsized":"2023-11-08T19:28:54.052655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:57:20.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.306513+0000","last_change":"2023-11-08T19:28:53.059447+0000","last_active":"2023-11-08T19:28:54.306513+0000","last_peered":"2023-11-08T19:28:54.306513+0000","last_clean":"2023-11-08T19:28:54.306513+0000","last_became_active":"2023-11-08T19:28:53.058684+0000","last_became_peered":"2023-11-08T19:28:53.058684+0000","last_unstale":"2023-11-08T19:28:54.306513+0000","last_undegraded":"2023-11-08T19:28:54.306513+0000","last_fullsized":"2023-11-08T19:28:54.306513+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:41:08.788666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053106+0000","last_change":"2023-11-08T19:28:53.069510+0000","last_active":"2023-11-08T19:28:54.053106+0000","last_peered":"2023-11-08T19:28:54.053106+0000","last_clean":"2023-11-08T19:28:54.053106+0000","last_became_active":"2023-11-08T19:28:53.064402+0000","last_became_peered":"2023-11-08T19:28:53.064402+0000","last_unstale":"2023-11-08T19:28:54.053106+0000","last_undegraded":"2023-11-08T19:28:54.053106+0000","last_fullsized":"2023-11-08T19:28:54.053106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:09:36.457883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026409+0000","last_change":"2023-11-08T19:28:52.043696+0000","last_active":"2023-11-08T19:28:53.026409+0000","last_peered":"2023-11-08T19:28:53.026409+0000","last_clean":"2023-11-08T19:28:53.026409+0000","last_became_active":"2023-11-08T19:28:52.043366+0000","last_became_peered":"2023-11-08T19:28:52.043366+0000","last_unstale":"2023-11-08T19:28:53.026409+0000","last_undegraded":"2023-11-08T19:28:53.026409+0000","last_fullsized":"2023-11-08T19:28:53.026409+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:22.572198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.030333+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:53.030333+0000","last_peered":"2023-11-08T19:28:53.030333+0000","last_clean":"2023-11-08T19:28:53.030333+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:53.030333+0000","last_undegraded":"2023-11-08T19:28:53.030333+0000","last_fullsized":"2023-11-08T19:28:53.030333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051636+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:53.051636+0000","last_peered":"2023-11-08T19:28:53.051636+0000","last_clean":"2023-11-08T19:28:53.051636+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:53.051636+0000","last_undegraded":"2023-11-08T19:28:53.051636+0000","last_fullsized":"2023-11-08T19:28:53.051636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046022+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:53.046022+0000","last_peered":"2023-11-08T19:28:53.046022+0000","last_clean":"2023-11-08T19:28:53.046022+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:53.046022+0000","last_undegraded":"2023-11-08T19:28:53.046022+0000","last_fullsized":"2023-11-08T19:28:53.046022+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199511+0000","last_change":"2023-11-08T19:28:53.064116+0000","last_active":"2023-11-08T19:28:54.199511+0000","last_peered":"2023-11-08T19:28:54.199511+0000","last_clean":"2023-11-08T19:28:54.199511+0000","last_became_active":"2023-11-08T19:28:53.063523+0000","last_became_peered":"2023-11-08T19:28:53.063523+0000","last_unstale":"2023-11-08T19:28:54.199511+0000","last_undegraded":"2023-11-08T19:28:54.199511+0000","last_fullsized":"2023-11-08T19:28:54.199511+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:48:22.332555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.081364+0000","last_change":"2023-11-08T19:28:52.045814+0000","last_active":"2023-11-08T19:28:53.081364+0000","last_peered":"2023-11-08T19:28:53.081364+0000","last_clean":"2023-11-08T19:28:53.081364+0000","last_became_active":"2023-11-08T19:28:52.045537+0000","last_became_peered":"2023-11-08T19:28:52.045537+0000","last_unstale":"2023-11-08T19:28:53.081364+0000","last_undegraded":"2023-11-08T19:28:53.081364+0000","last_fullsized":"2023-11-08T19:28:53.081364+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:49.384557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045388+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:53.045388+0000","last_peered":"2023-11-08T19:28:53.045388+0000","last_clean":"2023-11-08T19:28:53.045388+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:53.045388+0000","last_undegraded":"2023-11-08T19:28:53.045388+0000","last_fullsized":"2023-11-08T19:28:53.045388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052599+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:53.052599+0000","last_peered":"2023-11-08T19:28:53.052599+0000","last_clean":"2023-11-08T19:28:53.052599+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:53.052599+0000","last_undegraded":"2023-11-08T19:28:53.052599+0000","last_fullsized":"2023-11-08T19:28:53.052599+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029600+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:53.029600+0000","last_peered":"2023-11-08T19:28:53.029600+0000","last_clean":"2023-11-08T19:28:53.029600+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:53.029600+0000","last_undegraded":"2023-11-08T19:28:53.029600+0000","last_fullsized":"2023-11-08T19:28:53.029600+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310090+0000","last_change":"2023-11-08T19:28:53.059116+0000","last_active":"2023-11-08T19:28:54.310090+0000","last_peered":"2023-11-08T19:28:54.310090+0000","last_clean":"2023-11-08T19:28:54.310090+0000","last_became_active":"2023-11-08T19:28:53.055228+0000","last_became_peered":"2023-11-08T19:28:53.055228+0000","last_unstale":"2023-11-08T19:28:54.310090+0000","last_undegraded":"2023-11-08T19:28:54.310090+0000","last_fullsized":"2023-11-08T19:28:54.310090+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:44:01.886297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051019+0000","last_change":"2023-11-08T19:28:52.045539+0000","last_active":"2023-11-08T19:28:53.051019+0000","last_peered":"2023-11-08T19:28:53.051019+0000","last_clean":"2023-11-08T19:28:53.051019+0000","last_became_active":"2023-11-08T19:28:52.045224+0000","last_became_peered":"2023-11-08T19:28:52.045224+0000","last_unstale":"2023-11-08T19:28:53.051019+0000","last_undegraded":"2023-11-08T19:28:53.051019+0000","last_fullsized":"2023-11-08T19:28:53.051019+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:44.936274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.035456+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:53.035456+0000","last_peered":"2023-11-08T19:28:53.035456+0000","last_clean":"2023-11-08T19:28:53.035456+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:53.035456+0000","last_undegraded":"2023-11-08T19:28:53.035456+0000","last_fullsized":"2023-11-08T19:28:53.035456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029091+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:53.029091+0000","last_peered":"2023-11-08T19:28:53.029091+0000","last_clean":"2023-11-08T19:28:53.029091+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:53.029091+0000","last_undegraded":"2023-11-08T19:28:53.029091+0000","last_fullsized":"2023-11-08T19:28:53.029091+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052865+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:53.052865+0000","last_peered":"2023-11-08T19:28:53.052865+0000","last_clean":"2023-11-08T19:28:53.052865+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:53.052865+0000","last_undegraded":"2023-11-08T19:28:53.052865+0000","last_fullsized":"2023-11-08T19:28:53.052865+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199827+0000","last_change":"2023-11-08T19:28:53.058964+0000","last_active":"2023-11-08T19:28:54.199827+0000","last_peered":"2023-11-08T19:28:54.199827+0000","last_clean":"2023-11-08T19:28:54.199827+0000","last_became_active":"2023-11-08T19:28:53.058635+0000","last_became_peered":"2023-11-08T19:28:53.058635+0000","last_unstale":"2023-11-08T19:28:54.199827+0000","last_undegraded":"2023-11-08T19:28:54.199827+0000","last_fullsized":"2023-11-08T19:28:54.199827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:20.732302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044712+0000","last_change":"2023-11-08T19:28:52.046108+0000","last_active":"2023-11-08T19:28:53.044712+0000","last_peered":"2023-11-08T19:28:53.044712+0000","last_clean":"2023-11-08T19:28:53.044712+0000","last_became_active":"2023-11-08T19:28:52.045803+0000","last_became_peered":"2023-11-08T19:28:52.045803+0000","last_unstale":"2023-11-08T19:28:53.044712+0000","last_undegraded":"2023-11-08T19:28:53.044712+0000","last_fullsized":"2023-11-08T19:28:53.044712+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:11:34.638844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051423+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:53.051423+0000","last_peered":"2023-11-08T19:28:53.051423+0000","last_clean":"2023-11-08T19:28:53.051423+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:53.051423+0000","last_undegraded":"2023-11-08T19:28:53.051423+0000","last_fullsized":"2023-11-08T19:28:53.051423+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051229+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:53.051229+0000","last_peered":"2023-11-08T19:28:53.051229+0000","last_clean":"2023-11-08T19:28:53.051229+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:53.051229+0000","last_undegraded":"2023-11-08T19:28:53.051229+0000","last_fullsized":"2023-11-08T19:28:53.051229+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045843+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:53.045843+0000","last_peered":"2023-11-08T19:28:53.045843+0000","last_clean":"2023-11-08T19:28:53.045843+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:53.045843+0000","last_undegraded":"2023-11-08T19:28:53.045843+0000","last_fullsized":"2023-11-08T19:28:53.045843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308613+0000","last_change":"2023-11-08T19:28:53.059631+0000","last_active":"2023-11-08T19:28:54.308613+0000","last_peered":"2023-11-08T19:28:54.308613+0000","last_clean":"2023-11-08T19:28:54.308613+0000","last_became_active":"2023-11-08T19:28:53.056804+0000","last_became_peered":"2023-11-08T19:28:53.056804+0000","last_unstale":"2023-11-08T19:28:54.308613+0000","last_undegraded":"2023-11-08T19:28:54.308613+0000","last_fullsized":"2023-11-08T19:28:54.308613+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:54.774274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.077133+0000","last_change":"2023-11-08T19:28:52.041583+0000","last_active":"2023-11-08T19:28:53.077133+0000","last_peered":"2023-11-08T19:28:53.077133+0000","last_clean":"2023-11-08T19:28:53.077133+0000","last_became_active":"2023-11-08T19:28:52.041227+0000","last_became_peered":"2023-11-08T19:28:52.041227+0000","last_unstale":"2023-11-08T19:28:53.077133+0000","last_undegraded":"2023-11-08T19:28:53.077133+0000","last_fullsized":"2023-11-08T19:28:53.077133+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:59.217822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043019+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:53.043019+0000","last_peered":"2023-11-08T19:28:53.043019+0000","last_clean":"2023-11-08T19:28:53.043019+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:53.043019+0000","last_undegraded":"2023-11-08T19:28:53.043019+0000","last_fullsized":"2023-11-08T19:28:53.043019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051112+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:53.051112+0000","last_peered":"2023-11-08T19:28:53.051112+0000","last_clean":"2023-11-08T19:28:53.051112+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:53.051112+0000","last_undegraded":"2023-11-08T19:28:53.051112+0000","last_fullsized":"2023-11-08T19:28:53.051112+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046062+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:53.046062+0000","last_peered":"2023-11-08T19:28:53.046062+0000","last_clean":"2023-11-08T19:28:53.046062+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:53.046062+0000","last_undegraded":"2023-11-08T19:28:53.046062+0000","last_fullsized":"2023-11-08T19:28:53.046062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310044+0000","last_change":"2023-11-08T19:28:53.054636+0000","last_active":"2023-11-08T19:28:54.310044+0000","last_peered":"2023-11-08T19:28:54.310044+0000","last_clean":"2023-11-08T19:28:54.310044+0000","last_became_active":"2023-11-08T19:28:53.054315+0000","last_became_peered":"2023-11-08T19:28:53.054315+0000","last_unstale":"2023-11-08T19:28:54.310044+0000","last_undegraded":"2023-11-08T19:28:54.310044+0000","last_fullsized":"2023-11-08T19:28:54.310044+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:51.820923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.081472+0000","last_change":"2023-11-08T19:28:52.032141+0000","last_active":"2023-11-08T19:28:53.081472+0000","last_peered":"2023-11-08T19:28:53.081472+0000","last_clean":"2023-11-08T19:28:53.081472+0000","last_became_active":"2023-11-08T19:28:52.031853+0000","last_became_peered":"2023-11-08T19:28:52.031853+0000","last_unstale":"2023-11-08T19:28:53.081472+0000","last_undegraded":"2023-11-08T19:28:53.081472+0000","last_fullsized":"2023-11-08T19:28:53.081472+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:24.561720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051033+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:53.051033+0000","last_peered":"2023-11-08T19:28:53.051033+0000","last_clean":"2023-11-08T19:28:53.051033+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:53.051033+0000","last_undegraded":"2023-11-08T19:28:53.051033+0000","last_fullsized":"2023-11-08T19:28:53.051033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046540+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:53.046540+0000","last_peered":"2023-11-08T19:28:53.046540+0000","last_clean":"2023-11-08T19:28:53.046540+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:53.046540+0000","last_undegraded":"2023-11-08T19:28:53.046540+0000","last_fullsized":"2023-11-08T19:28:53.046540+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051596+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:53.051596+0000","last_peered":"2023-11-08T19:28:53.051596+0000","last_clean":"2023-11-08T19:28:53.051596+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:53.051596+0000","last_undegraded":"2023-11-08T19:28:53.051596+0000","last_fullsized":"2023-11-08T19:28:53.051596+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.201351+0000","last_change":"2023-11-08T19:28:53.070395+0000","last_active":"2023-11-08T19:28:54.201351+0000","last_peered":"2023-11-08T19:28:54.201351+0000","last_clean":"2023-11-08T19:28:54.201351+0000","last_became_active":"2023-11-08T19:28:53.068773+0000","last_became_peered":"2023-11-08T19:28:53.068773+0000","last_unstale":"2023-11-08T19:28:54.201351+0000","last_undegraded":"2023-11-08T19:28:54.201351+0000","last_fullsized":"2023-11-08T19:28:54.201351+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:31:47.198763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029605+0000","last_change":"2023-11-08T19:28:52.040141+0000","last_active":"2023-11-08T19:28:53.029605+0000","last_peered":"2023-11-08T19:28:53.029605+0000","last_clean":"2023-11-08T19:28:53.029605+0000","last_became_active":"2023-11-08T19:28:52.039861+0000","last_became_peered":"2023-11-08T19:28:52.039861+0000","last_unstale":"2023-11-08T19:28:53.029605+0000","last_undegraded":"2023-11-08T19:28:53.029605+0000","last_fullsized":"2023-11-08T19:28:53.029605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:53.761021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044830+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:53.044830+0000","last_peered":"2023-11-08T19:28:53.044830+0000","last_clean":"2023-11-08T19:28:53.044830+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:53.044830+0000","last_undegraded":"2023-11-08T19:28:53.044830+0000","last_fullsized":"2023-11-08T19:28:53.044830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051516+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:53.051516+0000","last_peered":"2023-11-08T19:28:53.051516+0000","last_clean":"2023-11-08T19:28:53.051516+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:53.051516+0000","last_undegraded":"2023-11-08T19:28:53.051516+0000","last_fullsized":"2023-11-08T19:28:53.051516+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051733+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:53.051733+0000","last_peered":"2023-11-08T19:28:53.051733+0000","last_clean":"2023-11-08T19:28:53.051733+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:53.051733+0000","last_undegraded":"2023-11-08T19:28:53.051733+0000","last_fullsized":"2023-11-08T19:28:53.051733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200030+0000","last_change":"2023-11-08T19:28:53.057783+0000","last_active":"2023-11-08T19:28:54.200030+0000","last_peered":"2023-11-08T19:28:54.200030+0000","last_clean":"2023-11-08T19:28:54.200030+0000","last_became_active":"2023-11-08T19:28:53.057527+0000","last_became_peered":"2023-11-08T19:28:53.057527+0000","last_unstale":"2023-11-08T19:28:54.200030+0000","last_undegraded":"2023-11-08T19:28:54.200030+0000","last_fullsized":"2023-11-08T19:28:54.200030+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:14:51.507430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.075025+0000","last_change":"2023-11-08T19:28:52.053100+0000","last_active":"2023-11-08T19:28:53.075025+0000","last_peered":"2023-11-08T19:28:53.075025+0000","last_clean":"2023-11-08T19:28:53.075025+0000","last_became_active":"2023-11-08T19:28:52.052793+0000","last_became_peered":"2023-11-08T19:28:52.052793+0000","last_unstale":"2023-11-08T19:28:53.075025+0000","last_undegraded":"2023-11-08T19:28:53.075025+0000","last_fullsized":"2023-11-08T19:28:53.075025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:02:27.383273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044186+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:53.044186+0000","last_peered":"2023-11-08T19:28:53.044186+0000","last_clean":"2023-11-08T19:28:53.044186+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:53.044186+0000","last_undegraded":"2023-11-08T19:28:53.044186+0000","last_fullsized":"2023-11-08T19:28:53.044186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029884+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:53.029884+0000","last_peered":"2023-11-08T19:28:53.029884+0000","last_clean":"2023-11-08T19:28:53.029884+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:53.029884+0000","last_undegraded":"2023-11-08T19:28:53.029884+0000","last_fullsized":"2023-11-08T19:28:53.029884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.032019+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:53.032019+0000","last_peered":"2023-11-08T19:28:53.032019+0000","last_clean":"2023-11-08T19:28:53.032019+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:53.032019+0000","last_undegraded":"2023-11-08T19:28:53.032019+0000","last_fullsized":"2023-11-08T19:28:53.032019+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.061240+0000","last_change":"2023-11-08T19:28:53.057430+0000","last_active":"2023-11-08T19:28:54.061240+0000","last_peered":"2023-11-08T19:28:54.061240+0000","last_clean":"2023-11-08T19:28:54.061240+0000","last_became_active":"2023-11-08T19:28:53.056880+0000","last_became_peered":"2023-11-08T19:28:53.056880+0000","last_unstale":"2023-11-08T19:28:54.061240+0000","last_undegraded":"2023-11-08T19:28:54.061240+0000","last_fullsized":"2023-11-08T19:28:54.061240+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:48:22.332555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.031140+0000","last_change":"2023-11-08T19:28:52.048946+0000","last_active":"2023-11-08T19:28:53.031140+0000","last_peered":"2023-11-08T19:28:53.031140+0000","last_clean":"2023-11-08T19:28:53.031140+0000","last_became_active":"2023-11-08T19:28:52.048664+0000","last_became_peered":"2023-11-08T19:28:52.048664+0000","last_unstale":"2023-11-08T19:28:53.031140+0000","last_undegraded":"2023-11-08T19:28:53.031140+0000","last_fullsized":"2023-11-08T19:28:53.031140+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:44:36.892504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052750+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:53.052750+0000","last_peered":"2023-11-08T19:28:53.052750+0000","last_clean":"2023-11-08T19:28:53.052750+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:53.052750+0000","last_undegraded":"2023-11-08T19:28:53.052750+0000","last_fullsized":"2023-11-08T19:28:53.052750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036414+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:53.036414+0000","last_peered":"2023-11-08T19:28:53.036414+0000","last_clean":"2023-11-08T19:28:53.036414+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:53.036414+0000","last_undegraded":"2023-11-08T19:28:53.036414+0000","last_fullsized":"2023-11-08T19:28:53.036414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047634+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:53.047634+0000","last_peered":"2023-11-08T19:28:53.047634+0000","last_clean":"2023-11-08T19:28:53.047634+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:53.047634+0000","last_undegraded":"2023-11-08T19:28:53.047634+0000","last_fullsized":"2023-11-08T19:28:53.047634+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.058959+0000","last_change":"2023-11-08T19:28:53.042132+0000","last_active":"2023-11-08T19:28:54.058959+0000","last_peered":"2023-11-08T19:28:54.058959+0000","last_clean":"2023-11-08T19:28:54.058959+0000","last_became_active":"2023-11-08T19:28:53.040236+0000","last_became_peered":"2023-11-08T19:28:53.040236+0000","last_unstale":"2023-11-08T19:28:54.058959+0000","last_undegraded":"2023-11-08T19:28:54.058959+0000","last_fullsized":"2023-11-08T19:28:54.058959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:26.746010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028620+0000","last_change":"2023-11-08T19:28:52.045194+0000","last_active":"2023-11-08T19:28:53.028620+0000","last_peered":"2023-11-08T19:28:53.028620+0000","last_clean":"2023-11-08T19:28:53.028620+0000","last_became_active":"2023-11-08T19:28:52.044931+0000","last_became_peered":"2023-11-08T19:28:52.044931+0000","last_unstale":"2023-11-08T19:28:53.028620+0000","last_undegraded":"2023-11-08T19:28:53.028620+0000","last_fullsized":"2023-11-08T19:28:53.028620+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:41:07.775414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036199+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:53.036199+0000","last_peered":"2023-11-08T19:28:53.036199+0000","last_clean":"2023-11-08T19:28:53.036199+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:53.036199+0000","last_undegraded":"2023-11-08T19:28:53.036199+0000","last_fullsized":"2023-11-08T19:28:53.036199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051689+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:53.051689+0000","last_peered":"2023-11-08T19:28:53.051689+0000","last_clean":"2023-11-08T19:28:53.051689+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:53.051689+0000","last_undegraded":"2023-11-08T19:28:53.051689+0000","last_fullsized":"2023-11-08T19:28:53.051689+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043941+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:53.043941+0000","last_peered":"2023-11-08T19:28:53.043941+0000","last_clean":"2023-11-08T19:28:53.043941+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:53.043941+0000","last_undegraded":"2023-11-08T19:28:53.043941+0000","last_fullsized":"2023-11-08T19:28:53.043941+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052909+0000","last_change":"2023-11-08T19:28:53.068461+0000","last_active":"2023-11-08T19:28:54.052909+0000","last_peered":"2023-11-08T19:28:54.052909+0000","last_clean":"2023-11-08T19:28:54.052909+0000","last_became_active":"2023-11-08T19:28:53.068175+0000","last_became_peered":"2023-11-08T19:28:53.068175+0000","last_unstale":"2023-11-08T19:28:54.052909+0000","last_undegraded":"2023-11-08T19:28:54.052909+0000","last_fullsized":"2023-11-08T19:28:54.052909+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:43.837543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051822+0000","last_change":"2023-11-08T19:28:52.040374+0000","last_active":"2023-11-08T19:28:53.051822+0000","last_peered":"2023-11-08T19:28:53.051822+0000","last_clean":"2023-11-08T19:28:53.051822+0000","last_became_active":"2023-11-08T19:28:52.040068+0000","last_became_peered":"2023-11-08T19:28:52.040068+0000","last_unstale":"2023-11-08T19:28:53.051822+0000","last_undegraded":"2023-11-08T19:28:53.051822+0000","last_fullsized":"2023-11-08T19:28:53.051822+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:41.609435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046847+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:53.046847+0000","last_peered":"2023-11-08T19:28:53.046847+0000","last_clean":"2023-11-08T19:28:53.046847+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:53.046847+0000","last_undegraded":"2023-11-08T19:28:53.046847+0000","last_fullsized":"2023-11-08T19:28:53.046847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051582+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:53.051582+0000","last_peered":"2023-11-08T19:28:53.051582+0000","last_clean":"2023-11-08T19:28:53.051582+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:53.051582+0000","last_undegraded":"2023-11-08T19:28:53.051582+0000","last_fullsized":"2023-11-08T19:28:53.051582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.049993+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:53.049993+0000","last_peered":"2023-11-08T19:28:53.049993+0000","last_clean":"2023-11-08T19:28:53.049993+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:53.049993+0000","last_undegraded":"2023-11-08T19:28:53.049993+0000","last_fullsized":"2023-11-08T19:28:53.049993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199696+0000","last_change":"2023-11-08T19:28:53.063620+0000","last_active":"2023-11-08T19:28:54.199696+0000","last_peered":"2023-11-08T19:28:54.199696+0000","last_clean":"2023-11-08T19:28:54.199696+0000","last_became_active":"2023-11-08T19:28:53.063355+0000","last_became_peered":"2023-11-08T19:28:53.063355+0000","last_unstale":"2023-11-08T19:28:54.199696+0000","last_undegraded":"2023-11-08T19:28:54.199696+0000","last_fullsized":"2023-11-08T19:28:54.199696+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:31.880618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.025739+0000","last_change":"2023-11-08T19:28:52.042246+0000","last_active":"2023-11-08T19:28:53.025739+0000","last_peered":"2023-11-08T19:28:53.025739+0000","last_clean":"2023-11-08T19:28:53.025739+0000","last_became_active":"2023-11-08T19:28:52.041982+0000","last_became_peered":"2023-11-08T19:28:52.041982+0000","last_unstale":"2023-11-08T19:28:53.025739+0000","last_undegraded":"2023-11-08T19:28:53.025739+0000","last_fullsized":"2023-11-08T19:28:53.025739+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:20.473289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047818+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:53.047818+0000","last_peered":"2023-11-08T19:28:53.047818+0000","last_clean":"2023-11-08T19:28:53.047818+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:53.047818+0000","last_undegraded":"2023-11-08T19:28:53.047818+0000","last_fullsized":"2023-11-08T19:28:53.047818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052299+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:53.052299+0000","last_peered":"2023-11-08T19:28:53.052299+0000","last_clean":"2023-11-08T19:28:53.052299+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:53.052299+0000","last_undegraded":"2023-11-08T19:28:53.052299+0000","last_fullsized":"2023-11-08T19:28:53.052299+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036616+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:53.036616+0000","last_peered":"2023-11-08T19:28:53.036616+0000","last_clean":"2023-11-08T19:28:53.036616+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:53.036616+0000","last_undegraded":"2023-11-08T19:28:53.036616+0000","last_fullsized":"2023-11-08T19:28:53.036616+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.054712+0000","last_change":"2023-11-08T19:28:53.064804+0000","last_active":"2023-11-08T19:28:54.054712+0000","last_peered":"2023-11-08T19:28:54.054712+0000","last_clean":"2023-11-08T19:28:54.054712+0000","last_became_active":"2023-11-08T19:28:53.063209+0000","last_became_peered":"2023-11-08T19:28:53.063209+0000","last_unstale":"2023-11-08T19:28:54.054712+0000","last_undegraded":"2023-11-08T19:28:54.054712+0000","last_fullsized":"2023-11-08T19:28:54.054712+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:09:07.336617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308523+0000","last_change":"2023-11-08T19:28:52.042660+0000","last_active":"2023-11-08T19:28:54.308523+0000","last_peered":"2023-11-08T19:28:54.308523+0000","last_clean":"2023-11-08T19:28:54.308523+0000","last_became_active":"2023-11-08T19:28:52.042200+0000","last_became_peered":"2023-11-08T19:28:52.042200+0000","last_unstale":"2023-11-08T19:28:54.308523+0000","last_undegraded":"2023-11-08T19:28:54.308523+0000","last_fullsized":"2023-11-08T19:28:54.308523+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:25.732757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.037181+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:53.037181+0000","last_peered":"2023-11-08T19:28:53.037181+0000","last_clean":"2023-11-08T19:28:53.037181+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:53.037181+0000","last_undegraded":"2023-11-08T19:28:53.037181+0000","last_fullsized":"2023-11-08T19:28:53.037181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.031562+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:53.031562+0000","last_peered":"2023-11-08T19:28:53.031562+0000","last_clean":"2023-11-08T19:28:53.031562+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:53.031562+0000","last_undegraded":"2023-11-08T19:28:53.031562+0000","last_fullsized":"2023-11-08T19:28:53.031562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.042684+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:53.042684+0000","last_peered":"2023-11-08T19:28:53.042684+0000","last_clean":"2023-11-08T19:28:53.042684+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:53.042684+0000","last_undegraded":"2023-11-08T19:28:53.042684+0000","last_fullsized":"2023-11-08T19:28:53.042684+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308395+0000","last_change":"2023-11-08T19:28:53.059040+0000","last_active":"2023-11-08T19:28:54.308395+0000","last_peered":"2023-11-08T19:28:54.308395+0000","last_clean":"2023-11-08T19:28:54.308395+0000","last_became_active":"2023-11-08T19:28:53.058773+0000","last_became_peered":"2023-11-08T19:28:53.058773+0000","last_unstale":"2023-11-08T19:28:54.308395+0000","last_undegraded":"2023-11-08T19:28:54.308395+0000","last_fullsized":"2023-11-08T19:28:54.308395+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:00.567332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.081288+0000","last_change":"2023-11-08T19:28:52.046987+0000","last_active":"2023-11-08T19:28:53.081288+0000","last_peered":"2023-11-08T19:28:53.081288+0000","last_clean":"2023-11-08T19:28:53.081288+0000","last_became_active":"2023-11-08T19:28:52.046726+0000","last_became_peered":"2023-11-08T19:28:52.046726+0000","last_unstale":"2023-11-08T19:28:53.081288+0000","last_undegraded":"2023-11-08T19:28:53.081288+0000","last_fullsized":"2023-11-08T19:28:53.081288+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:06.666959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047003+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:53.047003+0000","last_peered":"2023-11-08T19:28:53.047003+0000","last_clean":"2023-11-08T19:28:53.047003+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:53.047003+0000","last_undegraded":"2023-11-08T19:28:53.047003+0000","last_fullsized":"2023-11-08T19:28:53.047003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.037329+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:53.037329+0000","last_peered":"2023-11-08T19:28:53.037329+0000","last_clean":"2023-11-08T19:28:53.037329+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:53.037329+0000","last_undegraded":"2023-11-08T19:28:53.037329+0000","last_fullsized":"2023-11-08T19:28:53.037329+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":60,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045079+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:53.045079+0000","last_peered":"2023-11-08T19:28:53.045079+0000","last_clean":"2023-11-08T19:28:53.045079+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:53.045079+0000","last_undegraded":"2023-11-08T19:28:53.045079+0000","last_fullsized":"2023-11-08T19:28:53.045079+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307753+0000","last_change":"2023-11-08T19:28:53.060576+0000","last_active":"2023-11-08T19:28:54.307753+0000","last_peered":"2023-11-08T19:28:54.307753+0000","last_clean":"2023-11-08T19:28:54.307753+0000","last_became_active":"2023-11-08T19:28:53.060300+0000","last_became_peered":"2023-11-08T19:28:53.060300+0000","last_unstale":"2023-11-08T19:28:54.307753+0000","last_undegraded":"2023-11-08T19:28:54.307753+0000","last_fullsized":"2023-11-08T19:28:54.307753+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:33:58.968852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043299+0000","last_change":"2023-11-08T19:28:52.043670+0000","last_active":"2023-11-08T19:28:53.043299+0000","last_peered":"2023-11-08T19:28:53.043299+0000","last_clean":"2023-11-08T19:28:53.043299+0000","last_became_active":"2023-11-08T19:28:52.043416+0000","last_became_peered":"2023-11-08T19:28:52.043416+0000","last_unstale":"2023-11-08T19:28:53.043299+0000","last_undegraded":"2023-11-08T19:28:53.043299+0000","last_fullsized":"2023-11-08T19:28:53.043299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:08:35.425691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043852+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:53.043852+0000","last_peered":"2023-11-08T19:28:53.043852+0000","last_clean":"2023-11-08T19:28:53.043852+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:53.043852+0000","last_undegraded":"2023-11-08T19:28:53.043852+0000","last_fullsized":"2023-11-08T19:28:53.043852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044294+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:53.044294+0000","last_peered":"2023-11-08T19:28:53.044294+0000","last_clean":"2023-11-08T19:28:53.044294+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:53.044294+0000","last_undegraded":"2023-11-08T19:28:53.044294+0000","last_fullsized":"2023-11-08T19:28:53.044294+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051816+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:53.051816+0000","last_peered":"2023-11-08T19:28:53.051816+0000","last_clean":"2023-11-08T19:28:53.051816+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:53.051816+0000","last_undegraded":"2023-11-08T19:28:53.051816+0000","last_fullsized":"2023-11-08T19:28:53.051816+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053473+0000","last_change":"2023-11-08T19:28:53.066271+0000","last_active":"2023-11-08T19:28:54.053473+0000","last_peered":"2023-11-08T19:28:54.053473+0000","last_clean":"2023-11-08T19:28:54.053473+0000","last_became_active":"2023-11-08T19:28:53.065964+0000","last_became_peered":"2023-11-08T19:28:53.065964+0000","last_unstale":"2023-11-08T19:28:54.053473+0000","last_undegraded":"2023-11-08T19:28:54.053473+0000","last_fullsized":"2023-11-08T19:28:54.053473+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:12.444402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052056+0000","last_change":"2023-11-08T19:28:52.046515+0000","last_active":"2023-11-08T19:28:53.052056+0000","last_peered":"2023-11-08T19:28:53.052056+0000","last_clean":"2023-11-08T19:28:53.052056+0000","last_became_active":"2023-11-08T19:28:52.045882+0000","last_became_peered":"2023-11-08T19:28:52.045882+0000","last_unstale":"2023-11-08T19:28:53.052056+0000","last_undegraded":"2023-11-08T19:28:53.052056+0000","last_fullsized":"2023-11-08T19:28:53.052056+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:25.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043575+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:53.043575+0000","last_peered":"2023-11-08T19:28:53.043575+0000","last_clean":"2023-11-08T19:28:53.043575+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:53.043575+0000","last_undegraded":"2023-11-08T19:28:53.043575+0000","last_fullsized":"2023-11-08T19:28:53.043575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052383+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:53.052383+0000","last_peered":"2023-11-08T19:28:53.052383+0000","last_clean":"2023-11-08T19:28:53.052383+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:53.052383+0000","last_undegraded":"2023-11-08T19:28:53.052383+0000","last_fullsized":"2023-11-08T19:28:53.052383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047450+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:53.047450+0000","last_peered":"2023-11-08T19:28:53.047450+0000","last_clean":"2023-11-08T19:28:53.047450+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:53.047450+0000","last_undegraded":"2023-11-08T19:28:53.047450+0000","last_fullsized":"2023-11-08T19:28:53.047450+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062425+0000","last_change":"2023-11-08T19:28:53.060619+0000","last_active":"2023-11-08T19:28:54.062425+0000","last_peered":"2023-11-08T19:28:54.062425+0000","last_clean":"2023-11-08T19:28:54.062425+0000","last_became_active":"2023-11-08T19:28:53.057240+0000","last_became_peered":"2023-11-08T19:28:53.057240+0000","last_unstale":"2023-11-08T19:28:54.062425+0000","last_undegraded":"2023-11-08T19:28:54.062425+0000","last_fullsized":"2023-11-08T19:28:54.062425+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:20.732302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.074064+0000","last_change":"2023-11-08T19:28:52.045287+0000","last_active":"2023-11-08T19:28:53.074064+0000","last_peered":"2023-11-08T19:28:53.074064+0000","last_clean":"2023-11-08T19:28:53.074064+0000","last_became_active":"2023-11-08T19:28:52.044596+0000","last_became_peered":"2023-11-08T19:28:52.044596+0000","last_unstale":"2023-11-08T19:28:53.074064+0000","last_undegraded":"2023-11-08T19:28:53.074064+0000","last_fullsized":"2023-11-08T19:28:53.074064+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:26.971329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.049696+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:53.049696+0000","last_peered":"2023-11-08T19:28:53.049696+0000","last_clean":"2023-11-08T19:28:53.049696+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:53.049696+0000","last_undegraded":"2023-11-08T19:28:53.049696+0000","last_fullsized":"2023-11-08T19:28:53.049696+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045647+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:53.045647+0000","last_peered":"2023-11-08T19:28:53.045647+0000","last_clean":"2023-11-08T19:28:53.045647+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:53.045647+0000","last_undegraded":"2023-11-08T19:28:53.045647+0000","last_fullsized":"2023-11-08T19:28:53.045647+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":64,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308293+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:54.308293+0000","last_peered":"2023-11-08T19:28:54.308293+0000","last_clean":"2023-11-08T19:28:54.308293+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:54.308293+0000","last_undegraded":"2023-11-08T19:28:54.308293+0000","last_fullsized":"2023-11-08T19:28:54.308293+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052987+0000","last_change":"2023-11-08T19:28:53.058832+0000","last_active":"2023-11-08T19:28:54.052987+0000","last_peered":"2023-11-08T19:28:54.052987+0000","last_clean":"2023-11-08T19:28:54.052987+0000","last_became_active":"2023-11-08T19:28:53.058550+0000","last_became_peered":"2023-11-08T19:28:53.058550+0000","last_unstale":"2023-11-08T19:28:54.052987+0000","last_undegraded":"2023-11-08T19:28:54.052987+0000","last_fullsized":"2023-11-08T19:28:54.052987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:34:25.378632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.078143+0000","last_change":"2023-11-08T19:28:52.046795+0000","last_active":"2023-11-08T19:28:53.078143+0000","last_peered":"2023-11-08T19:28:53.078143+0000","last_clean":"2023-11-08T19:28:53.078143+0000","last_became_active":"2023-11-08T19:28:52.041733+0000","last_became_peered":"2023-11-08T19:28:52.041733+0000","last_unstale":"2023-11-08T19:28:53.078143+0000","last_undegraded":"2023-11-08T19:28:53.078143+0000","last_fullsized":"2023-11-08T19:28:53.078143+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:49.384557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036369+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:53.036369+0000","last_peered":"2023-11-08T19:28:53.036369+0000","last_clean":"2023-11-08T19:28:53.036369+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:53.036369+0000","last_undegraded":"2023-11-08T19:28:53.036369+0000","last_fullsized":"2023-11-08T19:28:53.036369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.048228+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:53.048228+0000","last_peered":"2023-11-08T19:28:53.048228+0000","last_clean":"2023-11-08T19:28:53.048228+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:53.048228+0000","last_undegraded":"2023-11-08T19:28:53.048228+0000","last_fullsized":"2023-11-08T19:28:53.048228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":64,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307815+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:54.307815+0000","last_peered":"2023-11-08T19:28:54.307815+0000","last_clean":"2023-11-08T19:28:54.307815+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:54.307815+0000","last_undegraded":"2023-11-08T19:28:54.307815+0000","last_fullsized":"2023-11-08T19:28:54.307815+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060450+0000","last_change":"2023-11-08T19:28:53.045269+0000","last_active":"2023-11-08T19:28:54.060450+0000","last_peered":"2023-11-08T19:28:54.060450+0000","last_clean":"2023-11-08T19:28:54.060450+0000","last_became_active":"2023-11-08T19:28:53.045001+0000","last_became_peered":"2023-11-08T19:28:53.045001+0000","last_unstale":"2023-11-08T19:28:54.060450+0000","last_undegraded":"2023-11-08T19:28:54.060450+0000","last_fullsized":"2023-11-08T19:28:54.060450+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:37.416484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051364+0000","last_change":"2023-11-08T19:28:52.048205+0000","last_active":"2023-11-08T19:28:53.051364+0000","last_peered":"2023-11-08T19:28:53.051364+0000","last_clean":"2023-11-08T19:28:53.051364+0000","last_became_active":"2023-11-08T19:28:52.047743+0000","last_became_peered":"2023-11-08T19:28:52.047743+0000","last_unstale":"2023-11-08T19:28:53.051364+0000","last_undegraded":"2023-11-08T19:28:53.051364+0000","last_fullsized":"2023-11-08T19:28:53.051364+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:53.593046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.035783+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:53.035783+0000","last_peered":"2023-11-08T19:28:53.035783+0000","last_clean":"2023-11-08T19:28:53.035783+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:53.035783+0000","last_undegraded":"2023-11-08T19:28:53.035783+0000","last_fullsized":"2023-11-08T19:28:53.035783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047901+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:53.047901+0000","last_peered":"2023-11-08T19:28:53.047901+0000","last_clean":"2023-11-08T19:28:53.047901+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:53.047901+0000","last_undegraded":"2023-11-08T19:28:53.047901+0000","last_fullsized":"2023-11-08T19:28:53.047901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.055075+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:54.055075+0000","last_peered":"2023-11-08T19:28:54.055075+0000","last_clean":"2023-11-08T19:28:54.055075+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:54.055075+0000","last_undegraded":"2023-11-08T19:28:54.055075+0000","last_fullsized":"2023-11-08T19:28:54.055075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052529+0000","last_change":"2023-11-08T19:28:53.067819+0000","last_active":"2023-11-08T19:28:54.052529+0000","last_peered":"2023-11-08T19:28:54.052529+0000","last_clean":"2023-11-08T19:28:54.052529+0000","last_became_active":"2023-11-08T19:28:53.067548+0000","last_became_peered":"2023-11-08T19:28:53.067548+0000","last_unstale":"2023-11-08T19:28:54.052529+0000","last_undegraded":"2023-11-08T19:28:54.052529+0000","last_fullsized":"2023-11-08T19:28:54.052529+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:43.045684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.027936+0000","last_change":"2023-11-08T19:28:52.053460+0000","last_active":"2023-11-08T19:28:53.027936+0000","last_peered":"2023-11-08T19:28:53.027936+0000","last_clean":"2023-11-08T19:28:53.027936+0000","last_became_active":"2023-11-08T19:28:52.053009+0000","last_became_peered":"2023-11-08T19:28:52.053009+0000","last_unstale":"2023-11-08T19:28:53.027936+0000","last_undegraded":"2023-11-08T19:28:53.027936+0000","last_fullsized":"2023-11-08T19:28:53.027936+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:16.388823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047989+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:53.047989+0000","last_peered":"2023-11-08T19:28:53.047989+0000","last_clean":"2023-11-08T19:28:53.047989+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:53.047989+0000","last_undegraded":"2023-11-08T19:28:53.047989+0000","last_fullsized":"2023-11-08T19:28:53.047989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051219+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:53.051219+0000","last_peered":"2023-11-08T19:28:53.051219+0000","last_clean":"2023-11-08T19:28:53.051219+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:53.051219+0000","last_undegraded":"2023-11-08T19:28:53.051219+0000","last_fullsized":"2023-11-08T19:28:53.051219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":63,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310492+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:54.310492+0000","last_peered":"2023-11-08T19:28:54.310492+0000","last_clean":"2023-11-08T19:28:54.310492+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:54.310492+0000","last_undegraded":"2023-11-08T19:28:54.310492+0000","last_fullsized":"2023-11-08T19:28:54.310492+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308918+0000","last_change":"2023-11-08T19:28:53.062951+0000","last_active":"2023-11-08T19:28:54.308918+0000","last_peered":"2023-11-08T19:28:54.308918+0000","last_clean":"2023-11-08T19:28:54.308918+0000","last_became_active":"2023-11-08T19:28:53.062453+0000","last_became_peered":"2023-11-08T19:28:53.062453+0000","last_unstale":"2023-11-08T19:28:54.308918+0000","last_undegraded":"2023-11-08T19:28:54.308918+0000","last_fullsized":"2023-11-08T19:28:54.308918+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:40:32.190737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.070943+0000","last_change":"2023-11-08T19:28:52.054688+0000","last_active":"2023-11-08T19:28:53.070943+0000","last_peered":"2023-11-08T19:28:53.070943+0000","last_clean":"2023-11-08T19:28:53.070943+0000","last_became_active":"2023-11-08T19:28:52.054309+0000","last_became_peered":"2023-11-08T19:28:52.054309+0000","last_unstale":"2023-11-08T19:28:53.070943+0000","last_undegraded":"2023-11-08T19:28:53.070943+0000","last_fullsized":"2023-11-08T19:28:53.070943+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:14:59.554079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051522+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:53.051522+0000","last_peered":"2023-11-08T19:28:53.051522+0000","last_clean":"2023-11-08T19:28:53.051522+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:53.051522+0000","last_undegraded":"2023-11-08T19:28:53.051522+0000","last_fullsized":"2023-11-08T19:28:53.051522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.031801+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:53.031801+0000","last_peered":"2023-11-08T19:28:53.031801+0000","last_clean":"2023-11-08T19:28:53.031801+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:53.031801+0000","last_undegraded":"2023-11-08T19:28:53.031801+0000","last_fullsized":"2023-11-08T19:28:53.031801+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052466+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:53.052466+0000","last_peered":"2023-11-08T19:28:53.052466+0000","last_clean":"2023-11-08T19:28:53.052466+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:53.052466+0000","last_undegraded":"2023-11-08T19:28:53.052466+0000","last_fullsized":"2023-11-08T19:28:53.052466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308648+0000","last_change":"2023-11-08T19:28:53.058531+0000","last_active":"2023-11-08T19:28:54.308648+0000","last_peered":"2023-11-08T19:28:54.308648+0000","last_clean":"2023-11-08T19:28:54.308648+0000","last_became_active":"2023-11-08T19:28:53.058277+0000","last_became_peered":"2023-11-08T19:28:53.058277+0000","last_unstale":"2023-11-08T19:28:54.308648+0000","last_undegraded":"2023-11-08T19:28:54.308648+0000","last_fullsized":"2023-11-08T19:28:54.308648+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:38.277146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"95'4","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.075905+0000","last_change":"2023-11-08T19:28:52.048963+0000","last_active":"2023-11-08T19:28:53.075905+0000","last_peered":"2023-11-08T19:28:53.075905+0000","last_clean":"2023-11-08T19:28:53.075905+0000","last_became_active":"2023-11-08T19:28:52.048642+0000","last_became_peered":"2023-11-08T19:28:52.048642+0000","last_unstale":"2023-11-08T19:28:53.075905+0000","last_undegraded":"2023-11-08T19:28:53.075905+0000","last_fullsized":"2023-11-08T19:28:53.075905+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:37.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028048+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:53.028048+0000","last_peered":"2023-11-08T19:28:53.028048+0000","last_clean":"2023-11-08T19:28:53.028048+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:53.028048+0000","last_undegraded":"2023-11-08T19:28:53.028048+0000","last_fullsized":"2023-11-08T19:28:53.028048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051654+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:53.051654+0000","last_peered":"2023-11-08T19:28:53.051654+0000","last_clean":"2023-11-08T19:28:53.051654+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:53.051654+0000","last_undegraded":"2023-11-08T19:28:53.051654+0000","last_fullsized":"2023-11-08T19:28:53.051654+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026031+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:53.026031+0000","last_peered":"2023-11-08T19:28:53.026031+0000","last_clean":"2023-11-08T19:28:53.026031+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:53.026031+0000","last_undegraded":"2023-11-08T19:28:53.026031+0000","last_fullsized":"2023-11-08T19:28:53.026031+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200941+0000","last_change":"2023-11-08T19:28:53.069140+0000","last_active":"2023-11-08T19:28:54.200941+0000","last_peered":"2023-11-08T19:28:54.200941+0000","last_clean":"2023-11-08T19:28:54.200941+0000","last_became_active":"2023-11-08T19:28:53.066451+0000","last_became_peered":"2023-11-08T19:28:53.066451+0000","last_unstale":"2023-11-08T19:28:54.200941+0000","last_undegraded":"2023-11-08T19:28:54.200941+0000","last_fullsized":"2023-11-08T19:28:54.200941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:19:17.413409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.080417+0000","last_change":"2023-11-08T19:28:52.044642+0000","last_active":"2023-11-08T19:28:53.080417+0000","last_peered":"2023-11-08T19:28:53.080417+0000","last_clean":"2023-11-08T19:28:53.080417+0000","last_became_active":"2023-11-08T19:28:52.044374+0000","last_became_peered":"2023-11-08T19:28:52.044374+0000","last_unstale":"2023-11-08T19:28:53.080417+0000","last_undegraded":"2023-11-08T19:28:53.080417+0000","last_fullsized":"2023-11-08T19:28:53.080417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:28:33.626999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045726+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:53.045726+0000","last_peered":"2023-11-08T19:28:53.045726+0000","last_clean":"2023-11-08T19:28:53.045726+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:53.045726+0000","last_undegraded":"2023-11-08T19:28:53.045726+0000","last_fullsized":"2023-11-08T19:28:53.045726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.030448+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:53.030448+0000","last_peered":"2023-11-08T19:28:53.030448+0000","last_clean":"2023-11-08T19:28:53.030448+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:53.030448+0000","last_undegraded":"2023-11-08T19:28:53.030448+0000","last_fullsized":"2023-11-08T19:28:53.030448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310115+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:54.310115+0000","last_peered":"2023-11-08T19:28:54.310115+0000","last_clean":"2023-11-08T19:28:54.310115+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:54.310115+0000","last_undegraded":"2023-11-08T19:28:54.310115+0000","last_fullsized":"2023-11-08T19:28:54.310115+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.311170+0000","last_change":"2023-11-08T19:28:53.059033+0000","last_active":"2023-11-08T19:28:54.311170+0000","last_peered":"2023-11-08T19:28:54.311170+0000","last_clean":"2023-11-08T19:28:54.311170+0000","last_became_active":"2023-11-08T19:28:53.058679+0000","last_became_peered":"2023-11-08T19:28:53.058679+0000","last_unstale":"2023-11-08T19:28:54.311170+0000","last_undegraded":"2023-11-08T19:28:54.311170+0000","last_fullsized":"2023-11-08T19:28:54.311170+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:37.416484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.075754+0000","last_change":"2023-11-08T19:28:52.044999+0000","last_active":"2023-11-08T19:28:53.075754+0000","last_peered":"2023-11-08T19:28:53.075754+0000","last_clean":"2023-11-08T19:28:53.075754+0000","last_became_active":"2023-11-08T19:28:52.043311+0000","last_became_peered":"2023-11-08T19:28:52.043311+0000","last_unstale":"2023-11-08T19:28:53.075754+0000","last_undegraded":"2023-11-08T19:28:53.075754+0000","last_fullsized":"2023-11-08T19:28:53.075754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:16.954121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043743+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:53.043743+0000","last_peered":"2023-11-08T19:28:53.043743+0000","last_clean":"2023-11-08T19:28:53.043743+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:53.043743+0000","last_undegraded":"2023-11-08T19:28:53.043743+0000","last_fullsized":"2023-11-08T19:28:53.043743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045869+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:53.045869+0000","last_peered":"2023-11-08T19:28:53.045869+0000","last_clean":"2023-11-08T19:28:53.045869+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:53.045869+0000","last_undegraded":"2023-11-08T19:28:53.045869+0000","last_fullsized":"2023-11-08T19:28:53.045869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051254+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:53.051254+0000","last_peered":"2023-11-08T19:28:53.051254+0000","last_clean":"2023-11-08T19:28:53.051254+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:53.051254+0000","last_undegraded":"2023-11-08T19:28:53.051254+0000","last_fullsized":"2023-11-08T19:28:53.051254+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309337+0000","last_change":"2023-11-08T19:28:53.071095+0000","last_active":"2023-11-08T19:28:54.309337+0000","last_peered":"2023-11-08T19:28:54.309337+0000","last_clean":"2023-11-08T19:28:54.309337+0000","last_became_active":"2023-11-08T19:28:53.070753+0000","last_became_peered":"2023-11-08T19:28:53.070753+0000","last_unstale":"2023-11-08T19:28:54.309337+0000","last_undegraded":"2023-11-08T19:28:54.309337+0000","last_fullsized":"2023-11-08T19:28:54.309337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:04.234641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047167+0000","last_change":"2023-11-08T19:28:52.045459+0000","last_active":"2023-11-08T19:28:53.047167+0000","last_peered":"2023-11-08T19:28:53.047167+0000","last_clean":"2023-11-08T19:28:53.047167+0000","last_became_active":"2023-11-08T19:28:52.044147+0000","last_became_peered":"2023-11-08T19:28:52.044147+0000","last_unstale":"2023-11-08T19:28:53.047167+0000","last_undegraded":"2023-11-08T19:28:53.047167+0000","last_fullsized":"2023-11-08T19:28:53.047167+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:10.473698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052729+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:53.052729+0000","last_peered":"2023-11-08T19:28:53.052729+0000","last_clean":"2023-11-08T19:28:53.052729+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:53.052729+0000","last_undegraded":"2023-11-08T19:28:53.052729+0000","last_fullsized":"2023-11-08T19:28:53.052729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052531+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:53.052531+0000","last_peered":"2023-11-08T19:28:53.052531+0000","last_clean":"2023-11-08T19:28:53.052531+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:53.052531+0000","last_undegraded":"2023-11-08T19:28:53.052531+0000","last_fullsized":"2023-11-08T19:28:53.052531+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.048217+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:53.048217+0000","last_peered":"2023-11-08T19:28:53.048217+0000","last_clean":"2023-11-08T19:28:53.048217+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:53.048217+0000","last_undegraded":"2023-11-08T19:28:53.048217+0000","last_fullsized":"2023-11-08T19:28:53.048217+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309580+0000","last_change":"2023-11-08T19:28:53.067720+0000","last_active":"2023-11-08T19:28:54.309580+0000","last_peered":"2023-11-08T19:28:54.309580+0000","last_clean":"2023-11-08T19:28:54.309580+0000","last_became_active":"2023-11-08T19:28:53.066313+0000","last_became_peered":"2023-11-08T19:28:53.066313+0000","last_unstale":"2023-11-08T19:28:54.309580+0000","last_undegraded":"2023-11-08T19:28:54.309580+0000","last_fullsized":"2023-11-08T19:28:54.309580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:52.651470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.076506+0000","last_change":"2023-11-08T19:28:52.048063+0000","last_active":"2023-11-08T19:28:53.076506+0000","last_peered":"2023-11-08T19:28:53.076506+0000","last_clean":"2023-11-08T19:28:53.076506+0000","last_became_active":"2023-11-08T19:28:52.047783+0000","last_became_peered":"2023-11-08T19:28:52.047783+0000","last_unstale":"2023-11-08T19:28:53.076506+0000","last_undegraded":"2023-11-08T19:28:53.076506+0000","last_fullsized":"2023-11-08T19:28:53.076506+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:02.863247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.048793+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:53.048793+0000","last_peered":"2023-11-08T19:28:53.048793+0000","last_clean":"2023-11-08T19:28:53.048793+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:53.048793+0000","last_undegraded":"2023-11-08T19:28:53.048793+0000","last_fullsized":"2023-11-08T19:28:53.048793+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052658+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:53.052658+0000","last_peered":"2023-11-08T19:28:53.052658+0000","last_clean":"2023-11-08T19:28:53.052658+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:53.052658+0000","last_undegraded":"2023-11-08T19:28:53.052658+0000","last_fullsized":"2023-11-08T19:28:53.052658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051655+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:53.051655+0000","last_peered":"2023-11-08T19:28:53.051655+0000","last_clean":"2023-11-08T19:28:53.051655+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:53.051655+0000","last_undegraded":"2023-11-08T19:28:53.051655+0000","last_fullsized":"2023-11-08T19:28:53.051655+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310311+0000","last_change":"2023-11-08T19:28:53.059680+0000","last_active":"2023-11-08T19:28:54.310311+0000","last_peered":"2023-11-08T19:28:54.310311+0000","last_clean":"2023-11-08T19:28:54.310311+0000","last_became_active":"2023-11-08T19:28:53.059396+0000","last_became_peered":"2023-11-08T19:28:53.059396+0000","last_unstale":"2023-11-08T19:28:54.310311+0000","last_undegraded":"2023-11-08T19:28:54.310311+0000","last_fullsized":"2023-11-08T19:28:54.310311+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:47:40.633125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"95'8","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.082271+0000","last_change":"2023-11-08T19:28:52.041623+0000","last_active":"2023-11-08T19:28:53.082271+0000","last_peered":"2023-11-08T19:28:53.082271+0000","last_clean":"2023-11-08T19:28:53.082271+0000","last_became_active":"2023-11-08T19:28:52.040377+0000","last_became_peered":"2023-11-08T19:28:52.040377+0000","last_unstale":"2023-11-08T19:28:53.082271+0000","last_undegraded":"2023-11-08T19:28:53.082271+0000","last_fullsized":"2023-11-08T19:28:53.082271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:55.361254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.035645+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:53.035645+0000","last_peered":"2023-11-08T19:28:53.035645+0000","last_clean":"2023-11-08T19:28:53.035645+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:53.035645+0000","last_undegraded":"2023-11-08T19:28:53.035645+0000","last_fullsized":"2023-11-08T19:28:53.035645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051503+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:53.051503+0000","last_peered":"2023-11-08T19:28:53.051503+0000","last_clean":"2023-11-08T19:28:53.051503+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:53.051503+0000","last_undegraded":"2023-11-08T19:28:53.051503+0000","last_fullsized":"2023-11-08T19:28:53.051503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.054425+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:53.054425+0000","last_peered":"2023-11-08T19:28:53.054425+0000","last_clean":"2023-11-08T19:28:53.054425+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:53.054425+0000","last_undegraded":"2023-11-08T19:28:53.054425+0000","last_fullsized":"2023-11-08T19:28:53.054425+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308728+0000","last_change":"2023-11-08T19:28:53.062739+0000","last_active":"2023-11-08T19:28:54.308728+0000","last_peered":"2023-11-08T19:28:54.308728+0000","last_clean":"2023-11-08T19:28:54.308728+0000","last_became_active":"2023-11-08T19:28:53.061762+0000","last_became_peered":"2023-11-08T19:28:53.061762+0000","last_unstale":"2023-11-08T19:28:54.308728+0000","last_undegraded":"2023-11-08T19:28:54.308728+0000","last_fullsized":"2023-11-08T19:28:54.308728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:38.678500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045190+0000","last_change":"2023-11-08T19:28:52.044950+0000","last_active":"2023-11-08T19:28:53.045190+0000","last_peered":"2023-11-08T19:28:53.045190+0000","last_clean":"2023-11-08T19:28:53.045190+0000","last_became_active":"2023-11-08T19:28:52.044673+0000","last_became_peered":"2023-11-08T19:28:52.044673+0000","last_unstale":"2023-11-08T19:28:53.045190+0000","last_undegraded":"2023-11-08T19:28:53.045190+0000","last_fullsized":"2023-11-08T19:28:53.045190+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:33.317757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028619+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:53.028619+0000","last_peered":"2023-11-08T19:28:53.028619+0000","last_clean":"2023-11-08T19:28:53.028619+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:53.028619+0000","last_undegraded":"2023-11-08T19:28:53.028619+0000","last_fullsized":"2023-11-08T19:28:53.028619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026794+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:53.026794+0000","last_peered":"2023-11-08T19:28:53.026794+0000","last_clean":"2023-11-08T19:28:53.026794+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:53.026794+0000","last_undegraded":"2023-11-08T19:28:53.026794+0000","last_fullsized":"2023-11-08T19:28:53.026794+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028271+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:53.028271+0000","last_peered":"2023-11-08T19:28:53.028271+0000","last_clean":"2023-11-08T19:28:53.028271+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:53.028271+0000","last_undegraded":"2023-11-08T19:28:53.028271+0000","last_fullsized":"2023-11-08T19:28:53.028271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062923+0000","last_change":"2023-11-08T19:28:53.044672+0000","last_active":"2023-11-08T19:28:54.062923+0000","last_peered":"2023-11-08T19:28:54.062923+0000","last_clean":"2023-11-08T19:28:54.062923+0000","last_became_active":"2023-11-08T19:28:53.044392+0000","last_became_peered":"2023-11-08T19:28:53.044392+0000","last_unstale":"2023-11-08T19:28:54.062923+0000","last_undegraded":"2023-11-08T19:28:54.062923+0000","last_fullsized":"2023-11-08T19:28:54.062923+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:17.402075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.027629+0000","last_change":"2023-11-08T19:28:52.046207+0000","last_active":"2023-11-08T19:28:53.027629+0000","last_peered":"2023-11-08T19:28:53.027629+0000","last_clean":"2023-11-08T19:28:53.027629+0000","last_became_active":"2023-11-08T19:28:52.044677+0000","last_became_peered":"2023-11-08T19:28:52.044677+0000","last_unstale":"2023-11-08T19:28:53.027629+0000","last_undegraded":"2023-11-08T19:28:53.027629+0000","last_fullsized":"2023-11-08T19:28:53.027629+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:17.866949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026055+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:53.026055+0000","last_peered":"2023-11-08T19:28:53.026055+0000","last_clean":"2023-11-08T19:28:53.026055+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:53.026055+0000","last_undegraded":"2023-11-08T19:28:53.026055+0000","last_fullsized":"2023-11-08T19:28:53.026055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043076+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:53.043076+0000","last_peered":"2023-11-08T19:28:53.043076+0000","last_clean":"2023-11-08T19:28:53.043076+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:53.043076+0000","last_undegraded":"2023-11-08T19:28:53.043076+0000","last_fullsized":"2023-11-08T19:28:53.043076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.027226+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:53.027226+0000","last_peered":"2023-11-08T19:28:53.027226+0000","last_clean":"2023-11-08T19:28:53.027226+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:53.027226+0000","last_undegraded":"2023-11-08T19:28:53.027226+0000","last_fullsized":"2023-11-08T19:28:53.027226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062718+0000","last_change":"2023-11-08T19:28:53.057917+0000","last_active":"2023-11-08T19:28:54.062718+0000","last_peered":"2023-11-08T19:28:54.062718+0000","last_clean":"2023-11-08T19:28:54.062718+0000","last_became_active":"2023-11-08T19:28:53.057566+0000","last_became_peered":"2023-11-08T19:28:53.057566+0000","last_unstale":"2023-11-08T19:28:54.062718+0000","last_undegraded":"2023-11-08T19:28:54.062718+0000","last_fullsized":"2023-11-08T19:28:54.062718+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:18:11.023596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"95'6","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.083681+0000","last_change":"2023-11-08T19:28:52.050926+0000","last_active":"2023-11-08T19:28:53.083681+0000","last_peered":"2023-11-08T19:28:53.083681+0000","last_clean":"2023-11-08T19:28:53.083681+0000","last_became_active":"2023-11-08T19:28:52.050635+0000","last_became_peered":"2023-11-08T19:28:52.050635+0000","last_unstale":"2023-11-08T19:28:53.083681+0000","last_undegraded":"2023-11-08T19:28:53.083681+0000","last_fullsized":"2023-11-08T19:28:53.083681+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:33.458673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029477+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:53.029477+0000","last_peered":"2023-11-08T19:28:53.029477+0000","last_clean":"2023-11-08T19:28:53.029477+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:53.029477+0000","last_undegraded":"2023-11-08T19:28:53.029477+0000","last_fullsized":"2023-11-08T19:28:53.029477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044812+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:53.044812+0000","last_peered":"2023-11-08T19:28:53.044812+0000","last_clean":"2023-11-08T19:28:53.044812+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:53.044812+0000","last_undegraded":"2023-11-08T19:28:53.044812+0000","last_fullsized":"2023-11-08T19:28:53.044812+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043239+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:53.043239+0000","last_peered":"2023-11-08T19:28:53.043239+0000","last_clean":"2023-11-08T19:28:53.043239+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:53.043239+0000","last_undegraded":"2023-11-08T19:28:53.043239+0000","last_fullsized":"2023-11-08T19:28:53.043239+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.311591+0000","last_change":"2023-11-08T19:28:53.069425+0000","last_active":"2023-11-08T19:28:54.311591+0000","last_peered":"2023-11-08T19:28:54.311591+0000","last_clean":"2023-11-08T19:28:54.311591+0000","last_became_active":"2023-11-08T19:28:53.069032+0000","last_became_peered":"2023-11-08T19:28:53.069032+0000","last_unstale":"2023-11-08T19:28:54.311591+0000","last_undegraded":"2023-11-08T19:28:54.311591+0000","last_fullsized":"2023-11-08T19:28:54.311591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:27.399438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"95'3","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.082079+0000","last_change":"2023-11-08T19:28:52.046860+0000","last_active":"2023-11-08T19:28:53.082079+0000","last_peered":"2023-11-08T19:28:53.082079+0000","last_clean":"2023-11-08T19:28:53.082079+0000","last_became_active":"2023-11-08T19:28:52.046550+0000","last_became_peered":"2023-11-08T19:28:52.046550+0000","last_unstale":"2023-11-08T19:28:53.082079+0000","last_undegraded":"2023-11-08T19:28:53.082079+0000","last_fullsized":"2023-11-08T19:28:53.082079+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:32.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.025807+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:53.025807+0000","last_peered":"2023-11-08T19:28:53.025807+0000","last_clean":"2023-11-08T19:28:53.025807+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:53.025807+0000","last_undegraded":"2023-11-08T19:28:53.025807+0000","last_fullsized":"2023-11-08T19:28:53.025807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.025980+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:53.025980+0000","last_peered":"2023-11-08T19:28:53.025980+0000","last_clean":"2023-11-08T19:28:53.025980+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:53.025980+0000","last_undegraded":"2023-11-08T19:28:53.025980+0000","last_fullsized":"2023-11-08T19:28:53.025980+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051336+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:53.051336+0000","last_peered":"2023-11-08T19:28:53.051336+0000","last_clean":"2023-11-08T19:28:53.051336+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:53.051336+0000","last_undegraded":"2023-11-08T19:28:53.051336+0000","last_fullsized":"2023-11-08T19:28:53.051336+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307857+0000","last_change":"2023-11-08T19:28:53.058135+0000","last_active":"2023-11-08T19:28:54.307857+0000","last_peered":"2023-11-08T19:28:54.307857+0000","last_clean":"2023-11-08T19:28:54.307857+0000","last_became_active":"2023-11-08T19:28:53.056156+0000","last_became_peered":"2023-11-08T19:28:53.056156+0000","last_unstale":"2023-11-08T19:28:54.307857+0000","last_undegraded":"2023-11-08T19:28:54.307857+0000","last_fullsized":"2023-11-08T19:28:54.307857+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:45.112533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029271+0000","last_change":"2023-11-08T19:28:52.041200+0000","last_active":"2023-11-08T19:28:53.029271+0000","last_peered":"2023-11-08T19:28:53.029271+0000","last_clean":"2023-11-08T19:28:53.029271+0000","last_became_active":"2023-11-08T19:28:52.040920+0000","last_became_peered":"2023-11-08T19:28:52.040920+0000","last_unstale":"2023-11-08T19:28:53.029271+0000","last_undegraded":"2023-11-08T19:28:53.029271+0000","last_fullsized":"2023-11-08T19:28:53.029271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:45.303340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044802+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:53.044802+0000","last_peered":"2023-11-08T19:28:53.044802+0000","last_clean":"2023-11-08T19:28:53.044802+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:53.044802+0000","last_undegraded":"2023-11-08T19:28:53.044802+0000","last_fullsized":"2023-11-08T19:28:53.044802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046975+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:53.046975+0000","last_peered":"2023-11-08T19:28:53.046975+0000","last_clean":"2023-11-08T19:28:53.046975+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:53.046975+0000","last_undegraded":"2023-11-08T19:28:53.046975+0000","last_fullsized":"2023-11-08T19:28:53.046975+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052783+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:53.052783+0000","last_peered":"2023-11-08T19:28:53.052783+0000","last_clean":"2023-11-08T19:28:53.052783+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:53.052783+0000","last_undegraded":"2023-11-08T19:28:53.052783+0000","last_fullsized":"2023-11-08T19:28:53.052783+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044531+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:53.044531+0000","last_peered":"2023-11-08T19:28:53.044531+0000","last_clean":"2023-11-08T19:28:53.044531+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:53.044531+0000","last_undegraded":"2023-11-08T19:28:53.044531+0000","last_fullsized":"2023-11-08T19:28:53.044531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1785856,"data_stored":1179586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.081,"15min":1.1020000000000001},"min":{"1min":0.69099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.206,"15min":1.2969999999999999},"min":{"1min":0.72499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.581,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":0.81799999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.2529999999999999},"min":{"1min":0.75700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.728,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.202,"5min":1.137,"15min":1.1399999999999999},"min":{"1min":0.91500000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.4590000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.3480000000000001,"15min":1.478},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2330000000000001,"15min":1.347},"min":{"1min":0.78500000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.476,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.96199999999999997}]},{"osd":3,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2949999999999999,"15min":1.361},"min":{"1min":0.97499999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0459999999999998,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.034},{"interface":"front","average":{"1min":1.359,"5min":1.4570000000000001,"15min":1.5529999999999999},"min":{"1min":0.98699999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7270000000000001,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.3560000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3169999999999999,"15min":1.357},"min":{"1min":0.90700000000000003,"5min":0.875,"15min":0.875},"max":{"1min":1.75,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.429,"5min":1.4059999999999999,"15min":1.49},"min":{"1min":0.92400000000000004,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.141,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.494}]},{"osd":5,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.431,"15min":1.46},"min":{"1min":1.1020000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.784,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":1.1140000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.573}]},{"osd":6,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.524,"15min":1.5720000000000001},"min":{"1min":1.165,"5min":0.878,"15min":0.878},"max":{"1min":1.887,"5min":2.427,"15min":2.427},"last":1.6719999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.5429999999999999,"15min":1.623},"min":{"1min":1.0700000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1920000000000002,"5min":2.468,"15min":2.468},"last":1.3899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":588672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.97099999999999997,"15min":0.93500000000000005},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.329,"5min":1.752,"15min":1.752},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.109,"15min":1.1040000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":0.93100000000000005}]},{"osd":1,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0920000000000001,"15min":1.0900000000000001},"min":{"1min":0.71099999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4399999999999999,"5min":1.679,"15min":1.679},"last":1.272},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2390000000000001,"15min":1.323},"min":{"1min":0.69999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.476,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":1.292}]},{"osd":2,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1879999999999999,"15min":1.276},"min":{"1min":0.69299999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6419999999999999,"5min":4.609,"15min":4.609},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.234,"15min":1.218},"min":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.48,"5min":1.861,"15min":1.861},"last":1.476}]},{"osd":3,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.363,"15min":1.4630000000000001},"min":{"1min":0.77900000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.655,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.587,"5min":1.784,"15min":1.784},"last":1.083}]},{"osd":4,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3740000000000001,"15min":1.3779999999999999},"min":{"1min":0.64100000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.881,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5189999999999999,"15min":1.5029999999999999},"min":{"1min":0.75700000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.998,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.133}]},{"osd":5,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.288,"15min":1.254},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.835,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.3520000000000001,"15min":1.3360000000000001},"min":{"1min":0.47999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.671,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.417}]},{"osd":7,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5640000000000001,"15min":1.5489999999999999},"min":{"1min":0.85799999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.456,"5min":1.454,"15min":1.454},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.381}]}]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":589206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1100000000000001,"15min":1.0800000000000001},"min":{"1min":0.84899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4710000000000001,"5min":1.76,"15min":1.76},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.24,"15min":1.2370000000000001},"min":{"1min":0.80000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.694,"5min":1.954,"15min":1.954},"last":1.2050000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1279999999999999,"15min":1.1659999999999999},"min":{"1min":0.81100000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6359999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.085,"5min":1.262,"15min":1.4139999999999999},"min":{"1min":0.77400000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.462,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1970000000000001,"15min":1.177},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7729999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.3029999999999999,"15min":1.4690000000000001},"min":{"1min":0.75,"5min":0.747,"15min":0.747},"max":{"1min":1.53,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.0900000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2509999999999999,"15min":1.2789999999999999},"min":{"1min":0.81399999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.573,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":1.365},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.333,"15min":1.3129999999999999},"min":{"1min":1.02,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.776,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.776}]},{"osd":4,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.282},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7969999999999999,"5min":2.347,"15min":2.347},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4750000000000001,"15min":1.54},"min":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7989999999999999,"5min":3.153,"15min":3.153},"last":1.5820000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4610000000000001,"15min":1.5169999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8540000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.304},{"interface":"front","average":{"1min":1.47,"5min":1.4690000000000001,"15min":1.4990000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.879,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.879}]},{"osd":7,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.597,"15min":1.6859999999999999},"min":{"1min":0.94399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0630000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.9219999999999999,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.4279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574903,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1183744,"data_stored":588602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0620000000000001,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.635,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.2390000000000001,"15min":1.204},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.546,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.2649999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0620000000000001,"15min":1.0880000000000001},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4359999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.097,"15min":1.155},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.3879999999999999,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":0.83799999999999997}]},{"osd":2,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.284,"15min":1.3340000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.784,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3280000000000001,"15min":1.4099999999999999},"min":{"1min":0.70199999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6910000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.2190000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2230000000000001,"15min":1.274},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4470000000000001,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":1.069},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3300000000000001,"15min":1.377},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.738,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.181}]},{"osd":5,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.393,"15min":1.4970000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7250000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":1.129},{"interface":"front","average":{"1min":1.252,"5min":1.4319999999999999,"15min":1.528},"min":{"1min":0.66600000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.7529999999999999,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.407}]},{"osd":6,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4830000000000001,"15min":1.5489999999999999},"min":{"1min":1.022,"5min":0.746,"15min":0.746},"max":{"1min":1.8580000000000001,"5min":4.335,"15min":4.335},"last":1.448},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.552,"15min":1.6140000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9490000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.371}]},{"osd":7,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.546,"15min":1.6120000000000001},"min":{"1min":0.92300000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.552},{"interface":"front","average":{"1min":1.595,"5min":1.6539999999999999,"15min":1.722},"min":{"1min":1.2430000000000001,"5min":1.095,"15min":1.095},"max":{"1min":2.0819999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.337}]}]},{"osd":3,"up_from":13,"seq":55834574901,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":590284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.90500000000000003,"15min":1.0449999999999999},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.206,"5min":3.972,"15min":3.972},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.99099999999999999,"15min":1.081},"min":{"1min":0.38300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.47,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":0.96099999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.054,"15min":1.169},"min":{"1min":0.47499999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.8480000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":1.214},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.98599999999999999,"15min":1.0860000000000001},"min":{"1min":0.55000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.448,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":1.117}]},{"osd":2,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.212,"15min":1.306},"min":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.639,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.004,"5min":1.1519999999999999,"15min":1.248},"min":{"1min":0.628,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.621,"5min":2.73,"15min":2.73},"last":0.628}]},{"osd":4,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3220000000000001,"15min":1.431},"min":{"1min":0.72999999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.744,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":1.744},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3500000000000001,"15min":1.4379999999999999},"min":{"1min":0.82999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7030000000000001,"5min":3.052,"15min":3.052},"last":1.694}]},{"osd":5,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.375,"15min":1.478},"min":{"1min":0.83999999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6830000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3999999999999999,"15min":1.5089999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9950000000000001,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.54}]},{"osd":6,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.3839999999999999,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.625,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.268},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4810000000000001,"15min":1.569},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.944,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.4339999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5409999999999999,"15min":1.6850000000000001},"min":{"1min":0.90000000000000002,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1150000000000002,"5min":4.407,"15min":4.407},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.4730000000000001,"15min":1.5629999999999999},"min":{"1min":0.84699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8100000000000001,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.401}]}]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1179648,"data_stored":587583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.85799999999999998},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3460000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3779999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":0.79200000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83099999999999996,"15min":0.86599999999999999},"min":{"1min":0.40200000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.94899999999999995,"15min":1.0089999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5029999999999999,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.008,"15min":1.0449999999999999},"min":{"1min":0.46600000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.6559999999999999,"5min":1.71,"15min":1.71},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.02,"15min":1.113},"min":{"1min":0.58799999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.264,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.81000000000000005}]},{"osd":4,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.27,"15min":1.391},"min":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.883,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.276,"5min":1.272,"15min":1.3560000000000001},"min":{"1min":0.89400000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6639999999999999,"5min":2.278,"15min":2.278},"last":1.147}]},{"osd":5,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3400000000000001,"15min":1.4179999999999999},"min":{"1min":0.93200000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3260000000000001,"15min":1.4450000000000001},"min":{"1min":0.79800000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9450000000000001,"5min":2.536,"15min":2.536},"last":1.014}]},{"osd":6,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3839999999999999},"min":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9810000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3640000000000001,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0169999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":0.98599999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4419999999999999,"15min":1.5600000000000001},"min":{"1min":0.89000000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9339999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.5429999999999999},"min":{"1min":0.93500000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.748,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.0720000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":588726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.65500000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.746},{"interface":"front","average":{"1min":1.004,"5min":0.91100000000000003,"15min":0.89500000000000002},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0780000000000001,"15min":1.0640000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.405},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.085},"min":{"1min":0.5,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.778,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.524}]},{"osd":4,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.153},"min":{"1min":0.72099999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8620000000000001,"5min":2.165,"15min":2.165},"last":1.49},{"interface":"front","average":{"1min":1.272,"5min":1.2270000000000001,"15min":1.2529999999999999},"min":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.018}]},{"osd":5,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.238,"15min":1.258},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1930000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.254,"15min":1.3140000000000001},"min":{"1min":0.73199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.748,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.3500000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.363,"15min":1.3680000000000001},"min":{"1min":0.92900000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.097,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.3280000000000001,"15min":1.371},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":2.282,"15min":2.282},"last":1.2010000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.333,"15min":1.335},"min":{"1min":0.74199999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.163},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.355,"15min":1.3859999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.964,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4450000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1794048,"data_stored":1179675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.296,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.871,"5min":0.872,"15min":0.92800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3180000000000001,"5min":1.7,"15min":1.7},"last":0.92900000000000005}]},{"osd":2,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.86599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.284,"5min":1.508,"15min":1.508},"last":1.1499999999999999},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.97199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.407,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":1.256}]},{"osd":3,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.6599999999999999,"5min":1.748,"15min":1.748},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0409999999999999,"15min":1.099},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.506,"5min":2.496,"15min":2.496},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5609999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.206},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3049999999999999,"15min":1.4199999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.54,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.3979999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.286,"15min":1.3600000000000001},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.516},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.355,"15min":1.4330000000000001},"min":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.534}]},{"osd":6,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4870000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1619999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.3540000000000001,"15min":1.405},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.3660000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.47},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.405,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.881,"5min":2.306,"15min":2.306},"last":1.702}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:29:02.624 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:29:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.849+0000 7f7c1f59e640 1 Processor -- start 2023-11-08T19:29:02.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.850+0000 7f7c1f59e640 1 -- start start 2023-11-08T19:29:02.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c1f59e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c2006bee0 0x7f7c2006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:02.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c1f59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2015a870 con 0x7f7c2006d520 2023-11-08T19:29:02.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c1f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c2006cbc0 con 0x7f7c200cc640 2023-11-08T19:29:02.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c1f59e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c2015cfa0 con 0x7f7c2006bee0 2023-11-08T19:29:02.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c1e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c2006bee0 0x7f7c2006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:02.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c17fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7c2006d520 0x7f7c2006c2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56582/0 (socket says 172.21.15.119:56582) 2023-11-08T19:29:02.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.851+0000 7f7c17fff640 1 -- 172.21.15.119:0/3625303580 learned_addr learned my addr 172.21.15.119:0/3625303580 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:02.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.852+0000 7f7c1e59c640 1 -- 172.21.15.119:0/3625303580 >> v1:172.21.15.188:6789/0 conn(0x7f7c200cc640 legacy=0x7f7c200ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:02.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.852+0000 7f7c1dd9b640 1 -- 172.21.15.119:0/3625303580 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626925959 0 0) 0x7f7c08001110 con 0x7f7c2006d520 2023-11-08T19:29:02.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.852+0000 7f7c1e59c640 1 -- 172.21.15.119:0/3625303580 >> v1:172.21.15.119:6789/0 conn(0x7f7c2006d520 legacy=0x7f7c2006c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:02.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.852+0000 7f7c1e59c640 1 -- 172.21.15.119:0/3625303580 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c200c9ad0 con 0x7f7c2006bee0 2023-11-08T19:29:02.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.853+0000 7f7c1e59c640 1 --2- 172.21.15.119:0/3625303580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c2006cfb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000a940 tx=0x7f7c1002ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ab4ccf21d58b78 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:02.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1dd9b640 1 -- 172.21.15.119:0/3625303580 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1002dbe0 con 0x7f7c2006bee0 2023-11-08T19:29:02.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1dd9b640 1 -- 172.21.15.119:0/3625303580 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1002dd40 con 0x7f7c2006bee0 2023-11-08T19:29:02.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1dd9b640 1 -- 172.21.15.119:0/3625303580 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c10035660 con 0x7f7c2006bee0 2023-11-08T19:29:02.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 -- 172.21.15.119:0/3625303580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 msgr2=0x7f7c2006cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:02.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/3625303580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c2006cfb0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000a940 tx=0x7f7c1002ce30 comp rx=0 tx=0).stop 2023-11-08T19:29:02.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 -- 172.21.15.119:0/3625303580 shutdown_connections 2023-11-08T19:29:02.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/3625303580 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c2006cfb0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:02.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 -- 172.21.15.119:0/3625303580 >> 172.21.15.119:0/3625303580 conn(0x7f7c200b0560 msgr2=0x7f7c200b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:02.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 -- 172.21.15.119:0/3625303580 shutdown_connections 2023-11-08T19:29:02.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.854+0000 7f7c1f59e640 1 -- 172.21.15.119:0/3625303580 wait complete. 2023-11-08T19:29:02.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.856+0000 7f7c1f59e640 1 Processor -- start 2023-11-08T19:29:02.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.856+0000 7f7c1f59e640 1 -- start start 2023-11-08T19:29:02.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c201495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:02.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2006d520 0x7f7c2014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:02.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c200cc640 0x7f7c200ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:02.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c200c54b0 con 0x7f7c2006bee0 2023-11-08T19:29:02.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c20149b10 con 0x7f7c200cc640 2023-11-08T19:29:02.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c200c4410 con 0x7f7c2006d520 2023-11-08T19:29:02.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.857+0000 7f7c1e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c201495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:02.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c201495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50260/0 (socket says 172.21.15.119:50260) 2023-11-08T19:29:02.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 -- 172.21.15.119:0/759976103 learned_addr learned my addr 172.21.15.119:0/759976103 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:02.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c17fff640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2006d520 0x7f7c2014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:02.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1ed9d640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c200cc640 0x7f7c200ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:02.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 -- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2006d520 msgr2=0x7f7c2014c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2006d520 0x7f7c2014c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 -- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c200cc640 msgr2=0x7f7c200ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c200cc640 0x7f7c200ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 -- 172.21.15.119:0/759976103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1000a5a0 con 0x7f7c2006bee0 2023-11-08T19:29:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.858+0000 7f7c1e59c640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c201495d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000aa60 tx=0x7f7c100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:02.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.859+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1003e610 con 0x7f7c2006bee0 2023-11-08T19:29:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.859+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1003e770 con 0x7f7c2006bee0 2023-11-08T19:29:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.859+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1003eb20 con 0x7f7c2006bee0 2023-11-08T19:29:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.859+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c200c46f0 con 0x7f7c2006bee0 2023-11-08T19:29:02.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.859+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c200c4cb0 con 0x7f7c2006bee0 2023-11-08T19:29:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.860+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2004ece0 con 0x7f7c2006bee0 2023-11-08T19:29:02.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.861+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c1003c040 con 0x7f7c2006bee0 2023-11-08T19:29:02.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.862+0000 7f7c16ffd640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c0006aea0 0x7f7c0006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:02.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.862+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f7c100abe90 con 0x7f7c2006bee0 2023-11-08T19:29:02.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.862+0000 7f7c17fff640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c0006aea0 0x7f7c0006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:02.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.863+0000 7f7c17fff640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c0006aea0 0x7f7c0006d360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7c200b1030 tx=0x7f7c08007a10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:02.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:02.866+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c10076940 con 0x7f7c2006bee0 2023-11-08T19:29:03.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.050+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c2015d4f0 con 0x7f7c0006aea0 2023-11-08T19:29:03.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.089+0000 7f7c16ffd640 1 -- 172.21.15.119:0/759976103 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+529658 (secure 0 0 0) 0x7f7c08001440 con 0x7f7c0006aea0 2023-11-08T19:29:03.098 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:03.102 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:29:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c0006aea0 msgr2=0x7f7c0006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c0006aea0 0x7f7c0006d360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f7c200b1030 tx=0x7f7c08007a10 comp rx=0 tx=0).stop 2023-11-08T19:29:03.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 msgr2=0x7f7c201495d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c201495d0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f7c1000aa60 tx=0x7f7c100046c0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 shutdown_connections 2023-11-08T19:29:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c0006aea0 0x7f7c0006d360 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c200cc640 0x7f7c200ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c2006d520 0x7f7c2014c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 --2- 172.21.15.119:0/759976103 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c2006bee0 0x7f7c201495d0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 >> 172.21.15.119:0/759976103 conn(0x7f7c200b0560 msgr2=0x7f7c200ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 shutdown_connections 2023-11-08T19:29:03.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.100+0000 7f7c1f59e640 1 -- 172.21.15.119:0/759976103 wait complete. 2023-11-08T19:29:03.126 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2023-11-08T19:29:02.743901+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259920,"kb_used_data":10464,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714800,"statfs":{"total":773094113280,"available":772827955200,"internally_reserved":0,"allocated":10715136,"data_stored":5892334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.722761"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.054351+0000","last_change":"2023-11-08T19:28:53.076247+0000","last_active":"2023-11-08T19:28:54.054351+0000","last_peered":"2023-11-08T19:28:54.054351+0000","last_clean":"2023-11-08T19:28:54.054351+0000","last_became_active":"2023-11-08T19:28:53.075822+0000","last_became_peered":"2023-11-08T19:28:53.075822+0000","last_unstale":"2023-11-08T19:28:54.054351+0000","last_undegraded":"2023-11-08T19:28:54.054351+0000","last_fullsized":"2023-11-08T19:28:54.054351+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:36:02.843749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060887+0000","last_change":"2023-11-08T19:28:53.056957+0000","last_active":"2023-11-08T19:28:54.060887+0000","last_peered":"2023-11-08T19:28:54.060887+0000","last_clean":"2023-11-08T19:28:54.060887+0000","last_became_active":"2023-11-08T19:28:53.056624+0000","last_became_peered":"2023-11-08T19:28:53.056624+0000","last_unstale":"2023-11-08T19:28:54.060887+0000","last_undegraded":"2023-11-08T19:28:54.060887+0000","last_fullsized":"2023-11-08T19:28:54.060887+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:00.567332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.201294+0000","last_change":"2023-11-08T19:28:53.069343+0000","last_active":"2023-11-08T19:28:54.201294+0000","last_peered":"2023-11-08T19:28:54.201294+0000","last_clean":"2023-11-08T19:28:54.201294+0000","last_became_active":"2023-11-08T19:28:53.068214+0000","last_became_peered":"2023-11-08T19:28:53.068214+0000","last_unstale":"2023-11-08T19:28:54.201294+0000","last_undegraded":"2023-11-08T19:28:54.201294+0000","last_fullsized":"2023-11-08T19:28:54.201294+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:08.926976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053347+0000","last_change":"2023-11-08T19:28:53.064559+0000","last_active":"2023-11-08T19:28:54.053347+0000","last_peered":"2023-11-08T19:28:54.053347+0000","last_clean":"2023-11-08T19:28:54.053347+0000","last_became_active":"2023-11-08T19:28:53.063879+0000","last_became_peered":"2023-11-08T19:28:53.063879+0000","last_unstale":"2023-11-08T19:28:54.053347+0000","last_undegraded":"2023-11-08T19:28:54.053347+0000","last_fullsized":"2023-11-08T19:28:54.053347+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:00:14.186139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308299+0000","last_change":"2023-11-08T19:28:53.057782+0000","last_active":"2023-11-08T19:28:54.308299+0000","last_peered":"2023-11-08T19:28:54.308299+0000","last_clean":"2023-11-08T19:28:54.308299+0000","last_became_active":"2023-11-08T19:28:53.057379+0000","last_became_peered":"2023-11-08T19:28:53.057379+0000","last_unstale":"2023-11-08T19:28:54.308299+0000","last_undegraded":"2023-11-08T19:28:54.308299+0000","last_fullsized":"2023-11-08T19:28:54.308299+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:39:25.561839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309567+0000","last_change":"2023-11-08T19:28:53.064420+0000","last_active":"2023-11-08T19:28:54.309567+0000","last_peered":"2023-11-08T19:28:54.309567+0000","last_clean":"2023-11-08T19:28:54.309567+0000","last_became_active":"2023-11-08T19:28:53.064122+0000","last_became_peered":"2023-11-08T19:28:53.064122+0000","last_unstale":"2023-11-08T19:28:54.309567+0000","last_undegraded":"2023-11-08T19:28:54.309567+0000","last_fullsized":"2023-11-08T19:28:54.309567+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:33:17.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.058786+0000","last_change":"2023-11-08T19:28:53.048517+0000","last_active":"2023-11-08T19:28:54.058786+0000","last_peered":"2023-11-08T19:28:54.058786+0000","last_clean":"2023-11-08T19:28:54.058786+0000","last_became_active":"2023-11-08T19:28:53.048243+0000","last_became_peered":"2023-11-08T19:28:53.048243+0000","last_unstale":"2023-11-08T19:28:54.058786+0000","last_undegraded":"2023-11-08T19:28:54.058786+0000","last_fullsized":"2023-11-08T19:28:54.058786+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:33:17.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200346+0000","last_change":"2023-11-08T19:28:53.070795+0000","last_active":"2023-11-08T19:28:54.200346+0000","last_peered":"2023-11-08T19:28:54.200346+0000","last_clean":"2023-11-08T19:28:54.200346+0000","last_became_active":"2023-11-08T19:28:53.069476+0000","last_became_peered":"2023-11-08T19:28:53.069476+0000","last_unstale":"2023-11-08T19:28:54.200346+0000","last_undegraded":"2023-11-08T19:28:54.200346+0000","last_fullsized":"2023-11-08T19:28:54.200346+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:41:23.508534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.054535+0000","last_change":"2023-11-08T19:28:53.054293+0000","last_active":"2023-11-08T19:28:54.054535+0000","last_peered":"2023-11-08T19:28:54.054535+0000","last_clean":"2023-11-08T19:28:54.054535+0000","last_became_active":"2023-11-08T19:28:53.053826+0000","last_became_peered":"2023-11-08T19:28:53.053826+0000","last_unstale":"2023-11-08T19:28:54.054535+0000","last_undegraded":"2023-11-08T19:28:54.054535+0000","last_fullsized":"2023-11-08T19:28:54.054535+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:35:19.537209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053644+0000","last_change":"2023-11-08T19:28:53.065525+0000","last_active":"2023-11-08T19:28:54.053644+0000","last_peered":"2023-11-08T19:28:54.053644+0000","last_clean":"2023-11-08T19:28:54.053644+0000","last_became_active":"2023-11-08T19:28:53.061285+0000","last_became_peered":"2023-11-08T19:28:53.061285+0000","last_unstale":"2023-11-08T19:28:54.053644+0000","last_undegraded":"2023-11-08T19:28:54.053644+0000","last_fullsized":"2023-11-08T19:28:54.053644+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:50:07.300630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.059605+0000","last_change":"2023-11-08T19:28:53.054567+0000","last_active":"2023-11-08T19:28:54.059605+0000","last_peered":"2023-11-08T19:28:54.059605+0000","last_clean":"2023-11-08T19:28:54.059605+0000","last_became_active":"2023-11-08T19:28:53.054267+0000","last_became_peered":"2023-11-08T19:28:53.054267+0000","last_unstale":"2023-11-08T19:28:54.059605+0000","last_undegraded":"2023-11-08T19:28:54.059605+0000","last_fullsized":"2023-11-08T19:28:54.059605+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:33:58.968852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053421+0000","last_change":"2023-11-08T19:28:53.062488+0000","last_active":"2023-11-08T19:28:54.053421+0000","last_peered":"2023-11-08T19:28:54.053421+0000","last_clean":"2023-11-08T19:28:54.053421+0000","last_became_active":"2023-11-08T19:28:53.060062+0000","last_became_peered":"2023-11-08T19:28:53.060062+0000","last_unstale":"2023-11-08T19:28:54.053421+0000","last_undegraded":"2023-11-08T19:28:54.053421+0000","last_fullsized":"2023-11-08T19:28:54.053421+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:46:06.679410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199869+0000","last_change":"2023-11-08T19:28:53.062160+0000","last_active":"2023-11-08T19:28:54.199869+0000","last_peered":"2023-11-08T19:28:54.199869+0000","last_clean":"2023-11-08T19:28:54.199869+0000","last_became_active":"2023-11-08T19:28:53.061828+0000","last_became_peered":"2023-11-08T19:28:53.061828+0000","last_unstale":"2023-11-08T19:28:54.199869+0000","last_undegraded":"2023-11-08T19:28:54.199869+0000","last_fullsized":"2023-11-08T19:28:54.199869+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:38:48.671222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199698+0000","last_change":"2023-11-08T19:28:53.065503+0000","last_active":"2023-11-08T19:28:54.199698+0000","last_peered":"2023-11-08T19:28:54.199698+0000","last_clean":"2023-11-08T19:28:54.199698+0000","last_became_active":"2023-11-08T19:28:53.063856+0000","last_became_peered":"2023-11-08T19:28:53.063856+0000","last_unstale":"2023-11-08T19:28:54.199698+0000","last_undegraded":"2023-11-08T19:28:54.199698+0000","last_fullsized":"2023-11-08T19:28:54.199698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:57.188860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307269+0000","last_change":"2023-11-08T19:28:53.059797+0000","last_active":"2023-11-08T19:28:54.307269+0000","last_peered":"2023-11-08T19:28:54.307269+0000","last_clean":"2023-11-08T19:28:54.307269+0000","last_became_active":"2023-11-08T19:28:53.059471+0000","last_became_peered":"2023-11-08T19:28:53.059471+0000","last_unstale":"2023-11-08T19:28:54.307269+0000","last_undegraded":"2023-11-08T19:28:54.307269+0000","last_fullsized":"2023-11-08T19:28:54.307269+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:46.316592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052473+0000","last_change":"2023-11-08T19:28:53.057927+0000","last_active":"2023-11-08T19:28:54.052473+0000","last_peered":"2023-11-08T19:28:54.052473+0000","last_clean":"2023-11-08T19:28:54.052473+0000","last_became_active":"2023-11-08T19:28:53.057350+0000","last_became_peered":"2023-11-08T19:28:53.057350+0000","last_unstale":"2023-11-08T19:28:54.052473+0000","last_undegraded":"2023-11-08T19:28:54.052473+0000","last_fullsized":"2023-11-08T19:28:54.052473+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:59.396641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060907+0000","last_change":"2023-11-08T19:28:53.058637+0000","last_active":"2023-11-08T19:28:54.060907+0000","last_peered":"2023-11-08T19:28:54.060907+0000","last_clean":"2023-11-08T19:28:54.060907+0000","last_became_active":"2023-11-08T19:28:53.058326+0000","last_became_peered":"2023-11-08T19:28:53.058326+0000","last_unstale":"2023-11-08T19:28:54.060907+0000","last_undegraded":"2023-11-08T19:28:54.060907+0000","last_fullsized":"2023-11-08T19:28:54.060907+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:40:32.190737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308249+0000","last_change":"2023-11-08T19:28:53.055902+0000","last_active":"2023-11-08T19:28:54.308249+0000","last_peered":"2023-11-08T19:28:54.308249+0000","last_clean":"2023-11-08T19:28:54.308249+0000","last_became_active":"2023-11-08T19:28:53.055390+0000","last_became_peered":"2023-11-08T19:28:53.055390+0000","last_unstale":"2023-11-08T19:28:54.308249+0000","last_undegraded":"2023-11-08T19:28:54.308249+0000","last_fullsized":"2023-11-08T19:28:54.308249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:37.239173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200445+0000","last_change":"2023-11-08T19:28:53.070459+0000","last_active":"2023-11-08T19:28:54.200445+0000","last_peered":"2023-11-08T19:28:54.200445+0000","last_clean":"2023-11-08T19:28:54.200445+0000","last_became_active":"2023-11-08T19:28:53.067727+0000","last_became_peered":"2023-11-08T19:28:53.067727+0000","last_unstale":"2023-11-08T19:28:54.200445+0000","last_undegraded":"2023-11-08T19:28:54.200445+0000","last_fullsized":"2023-11-08T19:28:54.200445+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:21:34.950327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.059668+0000","last_change":"2023-11-08T19:28:53.047320+0000","last_active":"2023-11-08T19:28:54.059668+0000","last_peered":"2023-11-08T19:28:54.059668+0000","last_clean":"2023-11-08T19:28:54.059668+0000","last_became_active":"2023-11-08T19:28:53.047054+0000","last_became_peered":"2023-11-08T19:28:53.047054+0000","last_unstale":"2023-11-08T19:28:54.059668+0000","last_undegraded":"2023-11-08T19:28:54.059668+0000","last_fullsized":"2023-11-08T19:28:54.059668+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:23:37.239173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.061624+0000","last_change":"2023-11-08T19:28:53.057226+0000","last_active":"2023-11-08T19:28:54.061624+0000","last_peered":"2023-11-08T19:28:54.061624+0000","last_clean":"2023-11-08T19:28:54.061624+0000","last_became_active":"2023-11-08T19:28:53.056886+0000","last_became_peered":"2023-11-08T19:28:53.056886+0000","last_unstale":"2023-11-08T19:28:54.061624+0000","last_undegraded":"2023-11-08T19:28:54.061624+0000","last_fullsized":"2023-11-08T19:28:54.061624+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:31.880618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309234+0000","last_change":"2023-11-08T19:28:53.066182+0000","last_active":"2023-11-08T19:28:54.309234+0000","last_peered":"2023-11-08T19:28:54.309234+0000","last_clean":"2023-11-08T19:28:54.309234+0000","last_became_active":"2023-11-08T19:28:53.065645+0000","last_became_peered":"2023-11-08T19:28:53.065645+0000","last_unstale":"2023-11-08T19:28:54.309234+0000","last_undegraded":"2023-11-08T19:28:54.309234+0000","last_fullsized":"2023-11-08T19:28:54.309234+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:42.397408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062336+0000","last_change":"2023-11-08T19:28:53.048605+0000","last_active":"2023-11-08T19:28:54.062336+0000","last_peered":"2023-11-08T19:28:54.062336+0000","last_clean":"2023-11-08T19:28:54.062336+0000","last_became_active":"2023-11-08T19:28:53.048273+0000","last_became_peered":"2023-11-08T19:28:53.048273+0000","last_unstale":"2023-11-08T19:28:54.062336+0000","last_undegraded":"2023-11-08T19:28:54.062336+0000","last_fullsized":"2023-11-08T19:28:54.062336+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:57.188860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308249+0000","last_change":"2023-11-08T19:28:53.067040+0000","last_active":"2023-11-08T19:28:54.308249+0000","last_peered":"2023-11-08T19:28:54.308249+0000","last_clean":"2023-11-08T19:28:54.308249+0000","last_became_active":"2023-11-08T19:28:53.066693+0000","last_became_peered":"2023-11-08T19:28:53.066693+0000","last_unstale":"2023-11-08T19:28:54.308249+0000","last_undegraded":"2023-11-08T19:28:54.308249+0000","last_fullsized":"2023-11-08T19:28:54.308249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:52.001841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060940+0000","last_change":"2023-11-08T19:28:53.064737+0000","last_active":"2023-11-08T19:28:54.060940+0000","last_peered":"2023-11-08T19:28:54.060940+0000","last_clean":"2023-11-08T19:28:54.060940+0000","last_became_active":"2023-11-08T19:28:53.064468+0000","last_became_peered":"2023-11-08T19:28:53.064468+0000","last_unstale":"2023-11-08T19:28:54.060940+0000","last_undegraded":"2023-11-08T19:28:54.060940+0000","last_fullsized":"2023-11-08T19:28:54.060940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:45.310422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053792+0000","last_change":"2023-11-08T19:28:53.058005+0000","last_active":"2023-11-08T19:28:54.053792+0000","last_peered":"2023-11-08T19:28:54.053792+0000","last_clean":"2023-11-08T19:28:54.053792+0000","last_became_active":"2023-11-08T19:28:53.057238+0000","last_became_peered":"2023-11-08T19:28:53.057238+0000","last_unstale":"2023-11-08T19:28:54.053792+0000","last_undegraded":"2023-11-08T19:28:54.053792+0000","last_fullsized":"2023-11-08T19:28:54.053792+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:06.683189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.311724+0000","last_change":"2023-11-08T19:28:53.063577+0000","last_active":"2023-11-08T19:28:54.311724+0000","last_peered":"2023-11-08T19:28:54.311724+0000","last_clean":"2023-11-08T19:28:54.311724+0000","last_became_active":"2023-11-08T19:28:53.061305+0000","last_became_peered":"2023-11-08T19:28:53.061305+0000","last_unstale":"2023-11-08T19:28:54.311724+0000","last_undegraded":"2023-11-08T19:28:54.311724+0000","last_fullsized":"2023-11-08T19:28:54.311724+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:04.246478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.312329+0000","last_change":"2023-11-08T19:28:53.060004+0000","last_active":"2023-11-08T19:28:54.312329+0000","last_peered":"2023-11-08T19:28:54.312329+0000","last_clean":"2023-11-08T19:28:54.312329+0000","last_became_active":"2023-11-08T19:28:53.056312+0000","last_became_peered":"2023-11-08T19:28:53.056312+0000","last_unstale":"2023-11-08T19:28:54.312329+0000","last_undegraded":"2023-11-08T19:28:54.312329+0000","last_fullsized":"2023-11-08T19:28:54.312329+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:08.857088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060889+0000","last_change":"2023-11-08T19:28:53.052784+0000","last_active":"2023-11-08T19:28:54.060889+0000","last_peered":"2023-11-08T19:28:54.060889+0000","last_clean":"2023-11-08T19:28:54.060889+0000","last_became_active":"2023-11-08T19:28:53.052458+0000","last_became_peered":"2023-11-08T19:28:53.052458+0000","last_unstale":"2023-11-08T19:28:54.060889+0000","last_undegraded":"2023-11-08T19:28:54.060889+0000","last_fullsized":"2023-11-08T19:28:54.060889+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:38.678500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.061095+0000","last_change":"2023-11-08T19:28:53.058887+0000","last_active":"2023-11-08T19:28:54.061095+0000","last_peered":"2023-11-08T19:28:54.061095+0000","last_clean":"2023-11-08T19:28:54.061095+0000","last_became_active":"2023-11-08T19:28:53.058559+0000","last_became_peered":"2023-11-08T19:28:53.058559+0000","last_unstale":"2023-11-08T19:28:54.061095+0000","last_undegraded":"2023-11-08T19:28:54.061095+0000","last_fullsized":"2023-11-08T19:28:54.061095+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:04.246478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052655+0000","last_change":"2023-11-08T19:28:53.059522+0000","last_active":"2023-11-08T19:28:54.052655+0000","last_peered":"2023-11-08T19:28:54.052655+0000","last_clean":"2023-11-08T19:28:54.052655+0000","last_became_active":"2023-11-08T19:28:53.059240+0000","last_became_peered":"2023-11-08T19:28:53.059240+0000","last_unstale":"2023-11-08T19:28:54.052655+0000","last_undegraded":"2023-11-08T19:28:54.052655+0000","last_fullsized":"2023-11-08T19:28:54.052655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:57:20.775693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.306513+0000","last_change":"2023-11-08T19:28:53.059447+0000","last_active":"2023-11-08T19:28:54.306513+0000","last_peered":"2023-11-08T19:28:54.306513+0000","last_clean":"2023-11-08T19:28:54.306513+0000","last_became_active":"2023-11-08T19:28:53.058684+0000","last_became_peered":"2023-11-08T19:28:53.058684+0000","last_unstale":"2023-11-08T19:28:54.306513+0000","last_undegraded":"2023-11-08T19:28:54.306513+0000","last_fullsized":"2023-11-08T19:28:54.306513+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:41:08.788666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053106+0000","last_change":"2023-11-08T19:28:53.069510+0000","last_active":"2023-11-08T19:28:54.053106+0000","last_peered":"2023-11-08T19:28:54.053106+0000","last_clean":"2023-11-08T19:28:54.053106+0000","last_became_active":"2023-11-08T19:28:53.064402+0000","last_became_peered":"2023-11-08T19:28:53.064402+0000","last_unstale":"2023-11-08T19:28:54.053106+0000","last_undegraded":"2023-11-08T19:28:54.053106+0000","last_fullsized":"2023-11-08T19:28:54.053106+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:09:36.457883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026409+0000","last_change":"2023-11-08T19:28:52.043696+0000","last_active":"2023-11-08T19:28:53.026409+0000","last_peered":"2023-11-08T19:28:53.026409+0000","last_clean":"2023-11-08T19:28:53.026409+0000","last_became_active":"2023-11-08T19:28:52.043366+0000","last_became_peered":"2023-11-08T19:28:52.043366+0000","last_unstale":"2023-11-08T19:28:53.026409+0000","last_undegraded":"2023-11-08T19:28:53.026409+0000","last_fullsized":"2023-11-08T19:28:53.026409+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:22.572198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.030333+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:28:53.030333+0000","last_peered":"2023-11-08T19:28:53.030333+0000","last_clean":"2023-11-08T19:28:53.030333+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:53.030333+0000","last_undegraded":"2023-11-08T19:28:53.030333+0000","last_fullsized":"2023-11-08T19:28:53.030333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051636+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:28:53.051636+0000","last_peered":"2023-11-08T19:28:53.051636+0000","last_clean":"2023-11-08T19:28:53.051636+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:28:53.051636+0000","last_undegraded":"2023-11-08T19:28:53.051636+0000","last_fullsized":"2023-11-08T19:28:53.051636+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046022+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:28:53.046022+0000","last_peered":"2023-11-08T19:28:53.046022+0000","last_clean":"2023-11-08T19:28:53.046022+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:28:53.046022+0000","last_undegraded":"2023-11-08T19:28:53.046022+0000","last_fullsized":"2023-11-08T19:28:53.046022+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199511+0000","last_change":"2023-11-08T19:28:53.064116+0000","last_active":"2023-11-08T19:28:54.199511+0000","last_peered":"2023-11-08T19:28:54.199511+0000","last_clean":"2023-11-08T19:28:54.199511+0000","last_became_active":"2023-11-08T19:28:53.063523+0000","last_became_peered":"2023-11-08T19:28:53.063523+0000","last_unstale":"2023-11-08T19:28:54.199511+0000","last_undegraded":"2023-11-08T19:28:54.199511+0000","last_fullsized":"2023-11-08T19:28:54.199511+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:48:22.332555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.081364+0000","last_change":"2023-11-08T19:28:52.045814+0000","last_active":"2023-11-08T19:28:53.081364+0000","last_peered":"2023-11-08T19:28:53.081364+0000","last_clean":"2023-11-08T19:28:53.081364+0000","last_became_active":"2023-11-08T19:28:52.045537+0000","last_became_peered":"2023-11-08T19:28:52.045537+0000","last_unstale":"2023-11-08T19:28:53.081364+0000","last_undegraded":"2023-11-08T19:28:53.081364+0000","last_fullsized":"2023-11-08T19:28:53.081364+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:49.384557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045388+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:28:53.045388+0000","last_peered":"2023-11-08T19:28:53.045388+0000","last_clean":"2023-11-08T19:28:53.045388+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:28:53.045388+0000","last_undegraded":"2023-11-08T19:28:53.045388+0000","last_fullsized":"2023-11-08T19:28:53.045388+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052599+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:28:53.052599+0000","last_peered":"2023-11-08T19:28:53.052599+0000","last_clean":"2023-11-08T19:28:53.052599+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:28:53.052599+0000","last_undegraded":"2023-11-08T19:28:53.052599+0000","last_fullsized":"2023-11-08T19:28:53.052599+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029600+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:28:53.029600+0000","last_peered":"2023-11-08T19:28:53.029600+0000","last_clean":"2023-11-08T19:28:53.029600+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:28:53.029600+0000","last_undegraded":"2023-11-08T19:28:53.029600+0000","last_fullsized":"2023-11-08T19:28:53.029600+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310090+0000","last_change":"2023-11-08T19:28:53.059116+0000","last_active":"2023-11-08T19:28:54.310090+0000","last_peered":"2023-11-08T19:28:54.310090+0000","last_clean":"2023-11-08T19:28:54.310090+0000","last_became_active":"2023-11-08T19:28:53.055228+0000","last_became_peered":"2023-11-08T19:28:53.055228+0000","last_unstale":"2023-11-08T19:28:54.310090+0000","last_undegraded":"2023-11-08T19:28:54.310090+0000","last_fullsized":"2023-11-08T19:28:54.310090+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:44:01.886297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051019+0000","last_change":"2023-11-08T19:28:52.045539+0000","last_active":"2023-11-08T19:28:53.051019+0000","last_peered":"2023-11-08T19:28:53.051019+0000","last_clean":"2023-11-08T19:28:53.051019+0000","last_became_active":"2023-11-08T19:28:52.045224+0000","last_became_peered":"2023-11-08T19:28:52.045224+0000","last_unstale":"2023-11-08T19:28:53.051019+0000","last_undegraded":"2023-11-08T19:28:53.051019+0000","last_fullsized":"2023-11-08T19:28:53.051019+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:51:44.936274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.035456+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:28:53.035456+0000","last_peered":"2023-11-08T19:28:53.035456+0000","last_clean":"2023-11-08T19:28:53.035456+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:28:53.035456+0000","last_undegraded":"2023-11-08T19:28:53.035456+0000","last_fullsized":"2023-11-08T19:28:53.035456+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029091+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:28:53.029091+0000","last_peered":"2023-11-08T19:28:53.029091+0000","last_clean":"2023-11-08T19:28:53.029091+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:28:53.029091+0000","last_undegraded":"2023-11-08T19:28:53.029091+0000","last_fullsized":"2023-11-08T19:28:53.029091+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052865+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:28:53.052865+0000","last_peered":"2023-11-08T19:28:53.052865+0000","last_clean":"2023-11-08T19:28:53.052865+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:28:53.052865+0000","last_undegraded":"2023-11-08T19:28:53.052865+0000","last_fullsized":"2023-11-08T19:28:53.052865+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199827+0000","last_change":"2023-11-08T19:28:53.058964+0000","last_active":"2023-11-08T19:28:54.199827+0000","last_peered":"2023-11-08T19:28:54.199827+0000","last_clean":"2023-11-08T19:28:54.199827+0000","last_became_active":"2023-11-08T19:28:53.058635+0000","last_became_peered":"2023-11-08T19:28:53.058635+0000","last_unstale":"2023-11-08T19:28:54.199827+0000","last_undegraded":"2023-11-08T19:28:54.199827+0000","last_fullsized":"2023-11-08T19:28:54.199827+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:20.732302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044712+0000","last_change":"2023-11-08T19:28:52.046108+0000","last_active":"2023-11-08T19:28:53.044712+0000","last_peered":"2023-11-08T19:28:53.044712+0000","last_clean":"2023-11-08T19:28:53.044712+0000","last_became_active":"2023-11-08T19:28:52.045803+0000","last_became_peered":"2023-11-08T19:28:52.045803+0000","last_unstale":"2023-11-08T19:28:53.044712+0000","last_undegraded":"2023-11-08T19:28:53.044712+0000","last_fullsized":"2023-11-08T19:28:53.044712+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:11:34.638844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051423+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:28:53.051423+0000","last_peered":"2023-11-08T19:28:53.051423+0000","last_clean":"2023-11-08T19:28:53.051423+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:28:53.051423+0000","last_undegraded":"2023-11-08T19:28:53.051423+0000","last_fullsized":"2023-11-08T19:28:53.051423+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051229+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:28:53.051229+0000","last_peered":"2023-11-08T19:28:53.051229+0000","last_clean":"2023-11-08T19:28:53.051229+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:28:53.051229+0000","last_undegraded":"2023-11-08T19:28:53.051229+0000","last_fullsized":"2023-11-08T19:28:53.051229+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045843+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:28:53.045843+0000","last_peered":"2023-11-08T19:28:53.045843+0000","last_clean":"2023-11-08T19:28:53.045843+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:28:53.045843+0000","last_undegraded":"2023-11-08T19:28:53.045843+0000","last_fullsized":"2023-11-08T19:28:53.045843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308613+0000","last_change":"2023-11-08T19:28:53.059631+0000","last_active":"2023-11-08T19:28:54.308613+0000","last_peered":"2023-11-08T19:28:54.308613+0000","last_clean":"2023-11-08T19:28:54.308613+0000","last_became_active":"2023-11-08T19:28:53.056804+0000","last_became_peered":"2023-11-08T19:28:53.056804+0000","last_unstale":"2023-11-08T19:28:54.308613+0000","last_undegraded":"2023-11-08T19:28:54.308613+0000","last_fullsized":"2023-11-08T19:28:54.308613+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:54.774274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.077133+0000","last_change":"2023-11-08T19:28:52.041583+0000","last_active":"2023-11-08T19:28:53.077133+0000","last_peered":"2023-11-08T19:28:53.077133+0000","last_clean":"2023-11-08T19:28:53.077133+0000","last_became_active":"2023-11-08T19:28:52.041227+0000","last_became_peered":"2023-11-08T19:28:52.041227+0000","last_unstale":"2023-11-08T19:28:53.077133+0000","last_undegraded":"2023-11-08T19:28:53.077133+0000","last_fullsized":"2023-11-08T19:28:53.077133+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:59.217822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043019+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:28:53.043019+0000","last_peered":"2023-11-08T19:28:53.043019+0000","last_clean":"2023-11-08T19:28:53.043019+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:28:53.043019+0000","last_undegraded":"2023-11-08T19:28:53.043019+0000","last_fullsized":"2023-11-08T19:28:53.043019+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051112+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:28:53.051112+0000","last_peered":"2023-11-08T19:28:53.051112+0000","last_clean":"2023-11-08T19:28:53.051112+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:28:53.051112+0000","last_undegraded":"2023-11-08T19:28:53.051112+0000","last_fullsized":"2023-11-08T19:28:53.051112+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046062+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:28:53.046062+0000","last_peered":"2023-11-08T19:28:53.046062+0000","last_clean":"2023-11-08T19:28:53.046062+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:28:53.046062+0000","last_undegraded":"2023-11-08T19:28:53.046062+0000","last_fullsized":"2023-11-08T19:28:53.046062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310044+0000","last_change":"2023-11-08T19:28:53.054636+0000","last_active":"2023-11-08T19:28:54.310044+0000","last_peered":"2023-11-08T19:28:54.310044+0000","last_clean":"2023-11-08T19:28:54.310044+0000","last_became_active":"2023-11-08T19:28:53.054315+0000","last_became_peered":"2023-11-08T19:28:53.054315+0000","last_unstale":"2023-11-08T19:28:54.310044+0000","last_undegraded":"2023-11-08T19:28:54.310044+0000","last_fullsized":"2023-11-08T19:28:54.310044+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:34:51.820923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.081472+0000","last_change":"2023-11-08T19:28:52.032141+0000","last_active":"2023-11-08T19:28:53.081472+0000","last_peered":"2023-11-08T19:28:53.081472+0000","last_clean":"2023-11-08T19:28:53.081472+0000","last_became_active":"2023-11-08T19:28:52.031853+0000","last_became_peered":"2023-11-08T19:28:52.031853+0000","last_unstale":"2023-11-08T19:28:53.081472+0000","last_undegraded":"2023-11-08T19:28:53.081472+0000","last_fullsized":"2023-11-08T19:28:53.081472+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:54:24.561720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051033+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:28:53.051033+0000","last_peered":"2023-11-08T19:28:53.051033+0000","last_clean":"2023-11-08T19:28:53.051033+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:28:53.051033+0000","last_undegraded":"2023-11-08T19:28:53.051033+0000","last_fullsized":"2023-11-08T19:28:53.051033+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046540+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:28:53.046540+0000","last_peered":"2023-11-08T19:28:53.046540+0000","last_clean":"2023-11-08T19:28:53.046540+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:28:53.046540+0000","last_undegraded":"2023-11-08T19:28:53.046540+0000","last_fullsized":"2023-11-08T19:28:53.046540+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051596+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:28:53.051596+0000","last_peered":"2023-11-08T19:28:53.051596+0000","last_clean":"2023-11-08T19:28:53.051596+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:28:53.051596+0000","last_undegraded":"2023-11-08T19:28:53.051596+0000","last_fullsized":"2023-11-08T19:28:53.051596+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.201351+0000","last_change":"2023-11-08T19:28:53.070395+0000","last_active":"2023-11-08T19:28:54.201351+0000","last_peered":"2023-11-08T19:28:54.201351+0000","last_clean":"2023-11-08T19:28:54.201351+0000","last_became_active":"2023-11-08T19:28:53.068773+0000","last_became_peered":"2023-11-08T19:28:53.068773+0000","last_unstale":"2023-11-08T19:28:54.201351+0000","last_undegraded":"2023-11-08T19:28:54.201351+0000","last_fullsized":"2023-11-08T19:28:54.201351+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:31:47.198763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029605+0000","last_change":"2023-11-08T19:28:52.040141+0000","last_active":"2023-11-08T19:28:53.029605+0000","last_peered":"2023-11-08T19:28:53.029605+0000","last_clean":"2023-11-08T19:28:53.029605+0000","last_became_active":"2023-11-08T19:28:52.039861+0000","last_became_peered":"2023-11-08T19:28:52.039861+0000","last_unstale":"2023-11-08T19:28:53.029605+0000","last_undegraded":"2023-11-08T19:28:53.029605+0000","last_fullsized":"2023-11-08T19:28:53.029605+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:38:53.761021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044830+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:28:53.044830+0000","last_peered":"2023-11-08T19:28:53.044830+0000","last_clean":"2023-11-08T19:28:53.044830+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:28:53.044830+0000","last_undegraded":"2023-11-08T19:28:53.044830+0000","last_fullsized":"2023-11-08T19:28:53.044830+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051516+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:28:53.051516+0000","last_peered":"2023-11-08T19:28:53.051516+0000","last_clean":"2023-11-08T19:28:53.051516+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:28:53.051516+0000","last_undegraded":"2023-11-08T19:28:53.051516+0000","last_fullsized":"2023-11-08T19:28:53.051516+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051733+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:28:53.051733+0000","last_peered":"2023-11-08T19:28:53.051733+0000","last_clean":"2023-11-08T19:28:53.051733+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:28:53.051733+0000","last_undegraded":"2023-11-08T19:28:53.051733+0000","last_fullsized":"2023-11-08T19:28:53.051733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200030+0000","last_change":"2023-11-08T19:28:53.057783+0000","last_active":"2023-11-08T19:28:54.200030+0000","last_peered":"2023-11-08T19:28:54.200030+0000","last_clean":"2023-11-08T19:28:54.200030+0000","last_became_active":"2023-11-08T19:28:53.057527+0000","last_became_peered":"2023-11-08T19:28:53.057527+0000","last_unstale":"2023-11-08T19:28:54.200030+0000","last_undegraded":"2023-11-08T19:28:54.200030+0000","last_fullsized":"2023-11-08T19:28:54.200030+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:14:51.507430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.075025+0000","last_change":"2023-11-08T19:28:52.053100+0000","last_active":"2023-11-08T19:28:53.075025+0000","last_peered":"2023-11-08T19:28:53.075025+0000","last_clean":"2023-11-08T19:28:53.075025+0000","last_became_active":"2023-11-08T19:28:52.052793+0000","last_became_peered":"2023-11-08T19:28:52.052793+0000","last_unstale":"2023-11-08T19:28:53.075025+0000","last_undegraded":"2023-11-08T19:28:53.075025+0000","last_fullsized":"2023-11-08T19:28:53.075025+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:02:27.383273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044186+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:28:53.044186+0000","last_peered":"2023-11-08T19:28:53.044186+0000","last_clean":"2023-11-08T19:28:53.044186+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:28:53.044186+0000","last_undegraded":"2023-11-08T19:28:53.044186+0000","last_fullsized":"2023-11-08T19:28:53.044186+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":42,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029884+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:28:53.029884+0000","last_peered":"2023-11-08T19:28:53.029884+0000","last_clean":"2023-11-08T19:28:53.029884+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:28:53.029884+0000","last_undegraded":"2023-11-08T19:28:53.029884+0000","last_fullsized":"2023-11-08T19:28:53.029884+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.032019+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:28:53.032019+0000","last_peered":"2023-11-08T19:28:53.032019+0000","last_clean":"2023-11-08T19:28:53.032019+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:28:53.032019+0000","last_undegraded":"2023-11-08T19:28:53.032019+0000","last_fullsized":"2023-11-08T19:28:53.032019+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.061240+0000","last_change":"2023-11-08T19:28:53.057430+0000","last_active":"2023-11-08T19:28:54.061240+0000","last_peered":"2023-11-08T19:28:54.061240+0000","last_clean":"2023-11-08T19:28:54.061240+0000","last_became_active":"2023-11-08T19:28:53.056880+0000","last_became_peered":"2023-11-08T19:28:53.056880+0000","last_unstale":"2023-11-08T19:28:54.061240+0000","last_undegraded":"2023-11-08T19:28:54.061240+0000","last_fullsized":"2023-11-08T19:28:54.061240+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:48:22.332555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.031140+0000","last_change":"2023-11-08T19:28:52.048946+0000","last_active":"2023-11-08T19:28:53.031140+0000","last_peered":"2023-11-08T19:28:53.031140+0000","last_clean":"2023-11-08T19:28:53.031140+0000","last_became_active":"2023-11-08T19:28:52.048664+0000","last_became_peered":"2023-11-08T19:28:52.048664+0000","last_unstale":"2023-11-08T19:28:53.031140+0000","last_undegraded":"2023-11-08T19:28:53.031140+0000","last_fullsized":"2023-11-08T19:28:53.031140+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:44:36.892504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052750+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:28:53.052750+0000","last_peered":"2023-11-08T19:28:53.052750+0000","last_clean":"2023-11-08T19:28:53.052750+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:28:53.052750+0000","last_undegraded":"2023-11-08T19:28:53.052750+0000","last_fullsized":"2023-11-08T19:28:53.052750+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036414+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:28:53.036414+0000","last_peered":"2023-11-08T19:28:53.036414+0000","last_clean":"2023-11-08T19:28:53.036414+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:28:53.036414+0000","last_undegraded":"2023-11-08T19:28:53.036414+0000","last_fullsized":"2023-11-08T19:28:53.036414+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047634+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:28:53.047634+0000","last_peered":"2023-11-08T19:28:53.047634+0000","last_clean":"2023-11-08T19:28:53.047634+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:28:53.047634+0000","last_undegraded":"2023-11-08T19:28:53.047634+0000","last_fullsized":"2023-11-08T19:28:53.047634+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.058959+0000","last_change":"2023-11-08T19:28:53.042132+0000","last_active":"2023-11-08T19:28:54.058959+0000","last_peered":"2023-11-08T19:28:54.058959+0000","last_clean":"2023-11-08T19:28:54.058959+0000","last_became_active":"2023-11-08T19:28:53.040236+0000","last_became_peered":"2023-11-08T19:28:53.040236+0000","last_unstale":"2023-11-08T19:28:54.058959+0000","last_undegraded":"2023-11-08T19:28:54.058959+0000","last_fullsized":"2023-11-08T19:28:54.058959+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:26.746010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028620+0000","last_change":"2023-11-08T19:28:52.045194+0000","last_active":"2023-11-08T19:28:53.028620+0000","last_peered":"2023-11-08T19:28:53.028620+0000","last_clean":"2023-11-08T19:28:53.028620+0000","last_became_active":"2023-11-08T19:28:52.044931+0000","last_became_peered":"2023-11-08T19:28:52.044931+0000","last_unstale":"2023-11-08T19:28:53.028620+0000","last_undegraded":"2023-11-08T19:28:53.028620+0000","last_fullsized":"2023-11-08T19:28:53.028620+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:41:07.775414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036199+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:28:53.036199+0000","last_peered":"2023-11-08T19:28:53.036199+0000","last_clean":"2023-11-08T19:28:53.036199+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:28:53.036199+0000","last_undegraded":"2023-11-08T19:28:53.036199+0000","last_fullsized":"2023-11-08T19:28:53.036199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051689+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:28:53.051689+0000","last_peered":"2023-11-08T19:28:53.051689+0000","last_clean":"2023-11-08T19:28:53.051689+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:28:53.051689+0000","last_undegraded":"2023-11-08T19:28:53.051689+0000","last_fullsized":"2023-11-08T19:28:53.051689+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.312575+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:28:54.312575+0000","last_peered":"2023-11-08T19:28:54.312575+0000","last_clean":"2023-11-08T19:28:54.312575+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:28:54.312575+0000","last_undegraded":"2023-11-08T19:28:54.312575+0000","last_fullsized":"2023-11-08T19:28:54.312575+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052909+0000","last_change":"2023-11-08T19:28:53.068461+0000","last_active":"2023-11-08T19:28:54.052909+0000","last_peered":"2023-11-08T19:28:54.052909+0000","last_clean":"2023-11-08T19:28:54.052909+0000","last_became_active":"2023-11-08T19:28:53.068175+0000","last_became_peered":"2023-11-08T19:28:53.068175+0000","last_unstale":"2023-11-08T19:28:54.052909+0000","last_undegraded":"2023-11-08T19:28:54.052909+0000","last_fullsized":"2023-11-08T19:28:54.052909+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:04:43.837543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051822+0000","last_change":"2023-11-08T19:28:52.040374+0000","last_active":"2023-11-08T19:28:53.051822+0000","last_peered":"2023-11-08T19:28:53.051822+0000","last_clean":"2023-11-08T19:28:53.051822+0000","last_became_active":"2023-11-08T19:28:52.040068+0000","last_became_peered":"2023-11-08T19:28:52.040068+0000","last_unstale":"2023-11-08T19:28:53.051822+0000","last_undegraded":"2023-11-08T19:28:53.051822+0000","last_fullsized":"2023-11-08T19:28:53.051822+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:41.609435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046847+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:28:53.046847+0000","last_peered":"2023-11-08T19:28:53.046847+0000","last_clean":"2023-11-08T19:28:53.046847+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:28:53.046847+0000","last_undegraded":"2023-11-08T19:28:53.046847+0000","last_fullsized":"2023-11-08T19:28:53.046847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051582+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:28:53.051582+0000","last_peered":"2023-11-08T19:28:53.051582+0000","last_clean":"2023-11-08T19:28:53.051582+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:28:53.051582+0000","last_undegraded":"2023-11-08T19:28:53.051582+0000","last_fullsized":"2023-11-08T19:28:53.051582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.049993+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:28:53.049993+0000","last_peered":"2023-11-08T19:28:53.049993+0000","last_clean":"2023-11-08T19:28:53.049993+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:28:53.049993+0000","last_undegraded":"2023-11-08T19:28:53.049993+0000","last_fullsized":"2023-11-08T19:28:53.049993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199696+0000","last_change":"2023-11-08T19:28:53.063620+0000","last_active":"2023-11-08T19:28:54.199696+0000","last_peered":"2023-11-08T19:28:54.199696+0000","last_clean":"2023-11-08T19:28:54.199696+0000","last_became_active":"2023-11-08T19:28:53.063355+0000","last_became_peered":"2023-11-08T19:28:53.063355+0000","last_unstale":"2023-11-08T19:28:54.199696+0000","last_undegraded":"2023-11-08T19:28:54.199696+0000","last_fullsized":"2023-11-08T19:28:54.199696+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:31.880618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.025739+0000","last_change":"2023-11-08T19:28:52.042246+0000","last_active":"2023-11-08T19:28:53.025739+0000","last_peered":"2023-11-08T19:28:53.025739+0000","last_clean":"2023-11-08T19:28:53.025739+0000","last_became_active":"2023-11-08T19:28:52.041982+0000","last_became_peered":"2023-11-08T19:28:52.041982+0000","last_unstale":"2023-11-08T19:28:53.025739+0000","last_undegraded":"2023-11-08T19:28:53.025739+0000","last_fullsized":"2023-11-08T19:28:53.025739+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:20.473289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047818+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:28:53.047818+0000","last_peered":"2023-11-08T19:28:53.047818+0000","last_clean":"2023-11-08T19:28:53.047818+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:28:53.047818+0000","last_undegraded":"2023-11-08T19:28:53.047818+0000","last_fullsized":"2023-11-08T19:28:53.047818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052299+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:28:53.052299+0000","last_peered":"2023-11-08T19:28:53.052299+0000","last_clean":"2023-11-08T19:28:53.052299+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:28:53.052299+0000","last_undegraded":"2023-11-08T19:28:53.052299+0000","last_fullsized":"2023-11-08T19:28:53.052299+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":67,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036616+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:28:53.036616+0000","last_peered":"2023-11-08T19:28:53.036616+0000","last_clean":"2023-11-08T19:28:53.036616+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:28:53.036616+0000","last_undegraded":"2023-11-08T19:28:53.036616+0000","last_fullsized":"2023-11-08T19:28:53.036616+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.054712+0000","last_change":"2023-11-08T19:28:53.064804+0000","last_active":"2023-11-08T19:28:54.054712+0000","last_peered":"2023-11-08T19:28:54.054712+0000","last_clean":"2023-11-08T19:28:54.054712+0000","last_became_active":"2023-11-08T19:28:53.063209+0000","last_became_peered":"2023-11-08T19:28:53.063209+0000","last_unstale":"2023-11-08T19:28:54.054712+0000","last_undegraded":"2023-11-08T19:28:54.054712+0000","last_fullsized":"2023-11-08T19:28:54.054712+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:09:07.336617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"96'1","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308523+0000","last_change":"2023-11-08T19:28:52.042660+0000","last_active":"2023-11-08T19:28:54.308523+0000","last_peered":"2023-11-08T19:28:54.308523+0000","last_clean":"2023-11-08T19:28:54.308523+0000","last_became_active":"2023-11-08T19:28:52.042200+0000","last_became_peered":"2023-11-08T19:28:52.042200+0000","last_unstale":"2023-11-08T19:28:54.308523+0000","last_undegraded":"2023-11-08T19:28:54.308523+0000","last_fullsized":"2023-11-08T19:28:54.308523+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:25.732757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.037181+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:28:53.037181+0000","last_peered":"2023-11-08T19:28:53.037181+0000","last_clean":"2023-11-08T19:28:53.037181+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:28:53.037181+0000","last_undegraded":"2023-11-08T19:28:53.037181+0000","last_fullsized":"2023-11-08T19:28:53.037181+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.031562+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:28:53.031562+0000","last_peered":"2023-11-08T19:28:53.031562+0000","last_clean":"2023-11-08T19:28:53.031562+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:28:53.031562+0000","last_undegraded":"2023-11-08T19:28:53.031562+0000","last_fullsized":"2023-11-08T19:28:53.031562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":63,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.199918+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:28:54.199918+0000","last_peered":"2023-11-08T19:28:54.199918+0000","last_clean":"2023-11-08T19:28:54.199918+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:28:54.199918+0000","last_undegraded":"2023-11-08T19:28:54.199918+0000","last_fullsized":"2023-11-08T19:28:54.199918+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308395+0000","last_change":"2023-11-08T19:28:53.059040+0000","last_active":"2023-11-08T19:28:54.308395+0000","last_peered":"2023-11-08T19:28:54.308395+0000","last_clean":"2023-11-08T19:28:54.308395+0000","last_became_active":"2023-11-08T19:28:53.058773+0000","last_became_peered":"2023-11-08T19:28:53.058773+0000","last_unstale":"2023-11-08T19:28:54.308395+0000","last_undegraded":"2023-11-08T19:28:54.308395+0000","last_fullsized":"2023-11-08T19:28:54.308395+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:00.567332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.081288+0000","last_change":"2023-11-08T19:28:52.046987+0000","last_active":"2023-11-08T19:28:53.081288+0000","last_peered":"2023-11-08T19:28:53.081288+0000","last_clean":"2023-11-08T19:28:53.081288+0000","last_became_active":"2023-11-08T19:28:52.046726+0000","last_became_peered":"2023-11-08T19:28:52.046726+0000","last_unstale":"2023-11-08T19:28:53.081288+0000","last_undegraded":"2023-11-08T19:28:53.081288+0000","last_fullsized":"2023-11-08T19:28:53.081288+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:41:06.666959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047003+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:28:53.047003+0000","last_peered":"2023-11-08T19:28:53.047003+0000","last_clean":"2023-11-08T19:28:53.047003+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:28:53.047003+0000","last_undegraded":"2023-11-08T19:28:53.047003+0000","last_fullsized":"2023-11-08T19:28:53.047003+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.037329+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:28:53.037329+0000","last_peered":"2023-11-08T19:28:53.037329+0000","last_clean":"2023-11-08T19:28:53.037329+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:28:53.037329+0000","last_undegraded":"2023-11-08T19:28:53.037329+0000","last_fullsized":"2023-11-08T19:28:53.037329+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":61,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310563+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:28:54.310563+0000","last_peered":"2023-11-08T19:28:54.310563+0000","last_clean":"2023-11-08T19:28:54.310563+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:28:54.310563+0000","last_undegraded":"2023-11-08T19:28:54.310563+0000","last_fullsized":"2023-11-08T19:28:54.310563+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307753+0000","last_change":"2023-11-08T19:28:53.060576+0000","last_active":"2023-11-08T19:28:54.307753+0000","last_peered":"2023-11-08T19:28:54.307753+0000","last_clean":"2023-11-08T19:28:54.307753+0000","last_became_active":"2023-11-08T19:28:53.060300+0000","last_became_peered":"2023-11-08T19:28:53.060300+0000","last_unstale":"2023-11-08T19:28:54.307753+0000","last_undegraded":"2023-11-08T19:28:54.307753+0000","last_fullsized":"2023-11-08T19:28:54.307753+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:33:58.968852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043299+0000","last_change":"2023-11-08T19:28:52.043670+0000","last_active":"2023-11-08T19:28:53.043299+0000","last_peered":"2023-11-08T19:28:53.043299+0000","last_clean":"2023-11-08T19:28:53.043299+0000","last_became_active":"2023-11-08T19:28:52.043416+0000","last_became_peered":"2023-11-08T19:28:52.043416+0000","last_unstale":"2023-11-08T19:28:53.043299+0000","last_undegraded":"2023-11-08T19:28:53.043299+0000","last_fullsized":"2023-11-08T19:28:53.043299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:08:35.425691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":30,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043852+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:28:53.043852+0000","last_peered":"2023-11-08T19:28:53.043852+0000","last_clean":"2023-11-08T19:28:53.043852+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:28:53.043852+0000","last_undegraded":"2023-11-08T19:28:53.043852+0000","last_fullsized":"2023-11-08T19:28:53.043852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044294+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:28:53.044294+0000","last_peered":"2023-11-08T19:28:53.044294+0000","last_clean":"2023-11-08T19:28:53.044294+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:28:53.044294+0000","last_undegraded":"2023-11-08T19:28:53.044294+0000","last_fullsized":"2023-11-08T19:28:53.044294+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053364+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:28:54.053364+0000","last_peered":"2023-11-08T19:28:54.053364+0000","last_clean":"2023-11-08T19:28:54.053364+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:28:54.053364+0000","last_undegraded":"2023-11-08T19:28:54.053364+0000","last_fullsized":"2023-11-08T19:28:54.053364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.053473+0000","last_change":"2023-11-08T19:28:53.066271+0000","last_active":"2023-11-08T19:28:54.053473+0000","last_peered":"2023-11-08T19:28:54.053473+0000","last_clean":"2023-11-08T19:28:54.053473+0000","last_became_active":"2023-11-08T19:28:53.065964+0000","last_became_peered":"2023-11-08T19:28:53.065964+0000","last_unstale":"2023-11-08T19:28:54.053473+0000","last_undegraded":"2023-11-08T19:28:54.053473+0000","last_fullsized":"2023-11-08T19:28:54.053473+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:37:12.444402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052056+0000","last_change":"2023-11-08T19:28:52.046515+0000","last_active":"2023-11-08T19:28:53.052056+0000","last_peered":"2023-11-08T19:28:53.052056+0000","last_clean":"2023-11-08T19:28:53.052056+0000","last_became_active":"2023-11-08T19:28:52.045882+0000","last_became_peered":"2023-11-08T19:28:52.045882+0000","last_unstale":"2023-11-08T19:28:53.052056+0000","last_undegraded":"2023-11-08T19:28:53.052056+0000","last_fullsized":"2023-11-08T19:28:53.052056+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:25.103518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043575+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:28:53.043575+0000","last_peered":"2023-11-08T19:28:53.043575+0000","last_clean":"2023-11-08T19:28:53.043575+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:28:53.043575+0000","last_undegraded":"2023-11-08T19:28:53.043575+0000","last_fullsized":"2023-11-08T19:28:53.043575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052383+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:28:53.052383+0000","last_peered":"2023-11-08T19:28:53.052383+0000","last_clean":"2023-11-08T19:28:53.052383+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:28:53.052383+0000","last_undegraded":"2023-11-08T19:28:53.052383+0000","last_fullsized":"2023-11-08T19:28:53.052383+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":64,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307692+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:28:54.307692+0000","last_peered":"2023-11-08T19:28:54.307692+0000","last_clean":"2023-11-08T19:28:54.307692+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:28:54.307692+0000","last_undegraded":"2023-11-08T19:28:54.307692+0000","last_fullsized":"2023-11-08T19:28:54.307692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062425+0000","last_change":"2023-11-08T19:28:53.060619+0000","last_active":"2023-11-08T19:28:54.062425+0000","last_peered":"2023-11-08T19:28:54.062425+0000","last_clean":"2023-11-08T19:28:54.062425+0000","last_became_active":"2023-11-08T19:28:53.057240+0000","last_became_peered":"2023-11-08T19:28:53.057240+0000","last_unstale":"2023-11-08T19:28:54.062425+0000","last_undegraded":"2023-11-08T19:28:54.062425+0000","last_fullsized":"2023-11-08T19:28:54.062425+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:31:20.732302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.074064+0000","last_change":"2023-11-08T19:28:52.045287+0000","last_active":"2023-11-08T19:28:53.074064+0000","last_peered":"2023-11-08T19:28:53.074064+0000","last_clean":"2023-11-08T19:28:53.074064+0000","last_became_active":"2023-11-08T19:28:52.044596+0000","last_became_peered":"2023-11-08T19:28:52.044596+0000","last_unstale":"2023-11-08T19:28:53.074064+0000","last_undegraded":"2023-11-08T19:28:53.074064+0000","last_fullsized":"2023-11-08T19:28:53.074064+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:56:26.971329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.049696+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:28:53.049696+0000","last_peered":"2023-11-08T19:28:53.049696+0000","last_clean":"2023-11-08T19:28:53.049696+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:28:53.049696+0000","last_undegraded":"2023-11-08T19:28:53.049696+0000","last_fullsized":"2023-11-08T19:28:53.049696+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045647+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:28:53.045647+0000","last_peered":"2023-11-08T19:28:53.045647+0000","last_clean":"2023-11-08T19:28:53.045647+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:28:53.045647+0000","last_undegraded":"2023-11-08T19:28:53.045647+0000","last_fullsized":"2023-11-08T19:28:53.045647+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":64,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308293+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:28:54.308293+0000","last_peered":"2023-11-08T19:28:54.308293+0000","last_clean":"2023-11-08T19:28:54.308293+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:28:54.308293+0000","last_undegraded":"2023-11-08T19:28:54.308293+0000","last_fullsized":"2023-11-08T19:28:54.308293+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052987+0000","last_change":"2023-11-08T19:28:53.058832+0000","last_active":"2023-11-08T19:28:54.052987+0000","last_peered":"2023-11-08T19:28:54.052987+0000","last_clean":"2023-11-08T19:28:54.052987+0000","last_became_active":"2023-11-08T19:28:53.058550+0000","last_became_peered":"2023-11-08T19:28:53.058550+0000","last_unstale":"2023-11-08T19:28:54.052987+0000","last_undegraded":"2023-11-08T19:28:54.052987+0000","last_fullsized":"2023-11-08T19:28:54.052987+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:34:25.378632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.078143+0000","last_change":"2023-11-08T19:28:52.046795+0000","last_active":"2023-11-08T19:28:53.078143+0000","last_peered":"2023-11-08T19:28:53.078143+0000","last_clean":"2023-11-08T19:28:53.078143+0000","last_became_active":"2023-11-08T19:28:52.041733+0000","last_became_peered":"2023-11-08T19:28:52.041733+0000","last_unstale":"2023-11-08T19:28:53.078143+0000","last_undegraded":"2023-11-08T19:28:53.078143+0000","last_fullsized":"2023-11-08T19:28:53.078143+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:11:49.384557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.036369+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:28:53.036369+0000","last_peered":"2023-11-08T19:28:53.036369+0000","last_clean":"2023-11-08T19:28:53.036369+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:28:53.036369+0000","last_undegraded":"2023-11-08T19:28:53.036369+0000","last_fullsized":"2023-11-08T19:28:53.036369+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.048228+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:28:53.048228+0000","last_peered":"2023-11-08T19:28:53.048228+0000","last_clean":"2023-11-08T19:28:53.048228+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:28:53.048228+0000","last_undegraded":"2023-11-08T19:28:53.048228+0000","last_fullsized":"2023-11-08T19:28:53.048228+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":64,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307815+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:28:54.307815+0000","last_peered":"2023-11-08T19:28:54.307815+0000","last_clean":"2023-11-08T19:28:54.307815+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:28:54.307815+0000","last_undegraded":"2023-11-08T19:28:54.307815+0000","last_fullsized":"2023-11-08T19:28:54.307815+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.060450+0000","last_change":"2023-11-08T19:28:53.045269+0000","last_active":"2023-11-08T19:28:54.060450+0000","last_peered":"2023-11-08T19:28:54.060450+0000","last_clean":"2023-11-08T19:28:54.060450+0000","last_became_active":"2023-11-08T19:28:53.045001+0000","last_became_peered":"2023-11-08T19:28:53.045001+0000","last_unstale":"2023-11-08T19:28:54.060450+0000","last_undegraded":"2023-11-08T19:28:54.060450+0000","last_fullsized":"2023-11-08T19:28:54.060450+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:37.416484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051364+0000","last_change":"2023-11-08T19:28:52.048205+0000","last_active":"2023-11-08T19:28:53.051364+0000","last_peered":"2023-11-08T19:28:53.051364+0000","last_clean":"2023-11-08T19:28:53.051364+0000","last_became_active":"2023-11-08T19:28:52.047743+0000","last_became_peered":"2023-11-08T19:28:52.047743+0000","last_unstale":"2023-11-08T19:28:53.051364+0000","last_undegraded":"2023-11-08T19:28:53.051364+0000","last_fullsized":"2023-11-08T19:28:53.051364+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:53.593046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.035783+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:28:53.035783+0000","last_peered":"2023-11-08T19:28:53.035783+0000","last_clean":"2023-11-08T19:28:53.035783+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:28:53.035783+0000","last_undegraded":"2023-11-08T19:28:53.035783+0000","last_fullsized":"2023-11-08T19:28:53.035783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047901+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:28:53.047901+0000","last_peered":"2023-11-08T19:28:53.047901+0000","last_clean":"2023-11-08T19:28:53.047901+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:28:53.047901+0000","last_undegraded":"2023-11-08T19:28:53.047901+0000","last_fullsized":"2023-11-08T19:28:53.047901+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.055075+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:28:54.055075+0000","last_peered":"2023-11-08T19:28:54.055075+0000","last_clean":"2023-11-08T19:28:54.055075+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:28:54.055075+0000","last_undegraded":"2023-11-08T19:28:54.055075+0000","last_fullsized":"2023-11-08T19:28:54.055075+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.052529+0000","last_change":"2023-11-08T19:28:53.067819+0000","last_active":"2023-11-08T19:28:54.052529+0000","last_peered":"2023-11-08T19:28:54.052529+0000","last_clean":"2023-11-08T19:28:54.052529+0000","last_became_active":"2023-11-08T19:28:53.067548+0000","last_became_peered":"2023-11-08T19:28:53.067548+0000","last_unstale":"2023-11-08T19:28:54.052529+0000","last_undegraded":"2023-11-08T19:28:54.052529+0000","last_fullsized":"2023-11-08T19:28:54.052529+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:43.045684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.027936+0000","last_change":"2023-11-08T19:28:52.053460+0000","last_active":"2023-11-08T19:28:53.027936+0000","last_peered":"2023-11-08T19:28:53.027936+0000","last_clean":"2023-11-08T19:28:53.027936+0000","last_became_active":"2023-11-08T19:28:52.053009+0000","last_became_peered":"2023-11-08T19:28:52.053009+0000","last_unstale":"2023-11-08T19:28:53.027936+0000","last_undegraded":"2023-11-08T19:28:53.027936+0000","last_fullsized":"2023-11-08T19:28:53.027936+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:16.388823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047989+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:28:53.047989+0000","last_peered":"2023-11-08T19:28:53.047989+0000","last_clean":"2023-11-08T19:28:53.047989+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:28:53.047989+0000","last_undegraded":"2023-11-08T19:28:53.047989+0000","last_fullsized":"2023-11-08T19:28:53.047989+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051219+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:28:53.051219+0000","last_peered":"2023-11-08T19:28:53.051219+0000","last_clean":"2023-11-08T19:28:53.051219+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:28:53.051219+0000","last_undegraded":"2023-11-08T19:28:53.051219+0000","last_fullsized":"2023-11-08T19:28:53.051219+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":63,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310492+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:28:54.310492+0000","last_peered":"2023-11-08T19:28:54.310492+0000","last_clean":"2023-11-08T19:28:54.310492+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:28:54.310492+0000","last_undegraded":"2023-11-08T19:28:54.310492+0000","last_fullsized":"2023-11-08T19:28:54.310492+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308918+0000","last_change":"2023-11-08T19:28:53.062951+0000","last_active":"2023-11-08T19:28:54.308918+0000","last_peered":"2023-11-08T19:28:54.308918+0000","last_clean":"2023-11-08T19:28:54.308918+0000","last_became_active":"2023-11-08T19:28:53.062453+0000","last_became_peered":"2023-11-08T19:28:53.062453+0000","last_unstale":"2023-11-08T19:28:54.308918+0000","last_undegraded":"2023-11-08T19:28:54.308918+0000","last_fullsized":"2023-11-08T19:28:54.308918+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:40:32.190737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"95'1","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.070943+0000","last_change":"2023-11-08T19:28:52.054688+0000","last_active":"2023-11-08T19:28:53.070943+0000","last_peered":"2023-11-08T19:28:53.070943+0000","last_clean":"2023-11-08T19:28:53.070943+0000","last_became_active":"2023-11-08T19:28:52.054309+0000","last_became_peered":"2023-11-08T19:28:52.054309+0000","last_unstale":"2023-11-08T19:28:53.070943+0000","last_undegraded":"2023-11-08T19:28:53.070943+0000","last_fullsized":"2023-11-08T19:28:53.070943+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:14:59.554079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051522+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:28:53.051522+0000","last_peered":"2023-11-08T19:28:53.051522+0000","last_clean":"2023-11-08T19:28:53.051522+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:28:53.051522+0000","last_undegraded":"2023-11-08T19:28:53.051522+0000","last_fullsized":"2023-11-08T19:28:53.051522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.031801+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:28:53.031801+0000","last_peered":"2023-11-08T19:28:53.031801+0000","last_clean":"2023-11-08T19:28:53.031801+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:28:53.031801+0000","last_undegraded":"2023-11-08T19:28:53.031801+0000","last_fullsized":"2023-11-08T19:28:53.031801+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052466+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:28:53.052466+0000","last_peered":"2023-11-08T19:28:53.052466+0000","last_clean":"2023-11-08T19:28:53.052466+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:28:53.052466+0000","last_undegraded":"2023-11-08T19:28:53.052466+0000","last_fullsized":"2023-11-08T19:28:53.052466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308648+0000","last_change":"2023-11-08T19:28:53.058531+0000","last_active":"2023-11-08T19:28:54.308648+0000","last_peered":"2023-11-08T19:28:54.308648+0000","last_clean":"2023-11-08T19:28:54.308648+0000","last_became_active":"2023-11-08T19:28:53.058277+0000","last_became_peered":"2023-11-08T19:28:53.058277+0000","last_unstale":"2023-11-08T19:28:54.308648+0000","last_undegraded":"2023-11-08T19:28:54.308648+0000","last_fullsized":"2023-11-08T19:28:54.308648+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:38.277146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"95'4","reported_seq":16,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.075905+0000","last_change":"2023-11-08T19:28:52.048963+0000","last_active":"2023-11-08T19:28:53.075905+0000","last_peered":"2023-11-08T19:28:53.075905+0000","last_clean":"2023-11-08T19:28:53.075905+0000","last_became_active":"2023-11-08T19:28:52.048642+0000","last_became_peered":"2023-11-08T19:28:52.048642+0000","last_unstale":"2023-11-08T19:28:53.075905+0000","last_undegraded":"2023-11-08T19:28:53.075905+0000","last_fullsized":"2023-11-08T19:28:53.075905+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:11:37.263894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028048+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:28:53.028048+0000","last_peered":"2023-11-08T19:28:53.028048+0000","last_clean":"2023-11-08T19:28:53.028048+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:28:53.028048+0000","last_undegraded":"2023-11-08T19:28:53.028048+0000","last_fullsized":"2023-11-08T19:28:53.028048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051654+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:28:53.051654+0000","last_peered":"2023-11-08T19:28:53.051654+0000","last_clean":"2023-11-08T19:28:53.051654+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:28:53.051654+0000","last_undegraded":"2023-11-08T19:28:53.051654+0000","last_fullsized":"2023-11-08T19:28:53.051654+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026031+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:28:53.026031+0000","last_peered":"2023-11-08T19:28:53.026031+0000","last_clean":"2023-11-08T19:28:53.026031+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:28:53.026031+0000","last_undegraded":"2023-11-08T19:28:53.026031+0000","last_fullsized":"2023-11-08T19:28:53.026031+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.200941+0000","last_change":"2023-11-08T19:28:53.069140+0000","last_active":"2023-11-08T19:28:54.200941+0000","last_peered":"2023-11-08T19:28:54.200941+0000","last_clean":"2023-11-08T19:28:54.200941+0000","last_became_active":"2023-11-08T19:28:53.066451+0000","last_became_peered":"2023-11-08T19:28:53.066451+0000","last_unstale":"2023-11-08T19:28:54.200941+0000","last_undegraded":"2023-11-08T19:28:54.200941+0000","last_fullsized":"2023-11-08T19:28:54.200941+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:19:17.413409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.080417+0000","last_change":"2023-11-08T19:28:52.044642+0000","last_active":"2023-11-08T19:28:53.080417+0000","last_peered":"2023-11-08T19:28:53.080417+0000","last_clean":"2023-11-08T19:28:53.080417+0000","last_became_active":"2023-11-08T19:28:52.044374+0000","last_became_peered":"2023-11-08T19:28:52.044374+0000","last_unstale":"2023-11-08T19:28:53.080417+0000","last_undegraded":"2023-11-08T19:28:53.080417+0000","last_fullsized":"2023-11-08T19:28:53.080417+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:28:33.626999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045726+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:28:53.045726+0000","last_peered":"2023-11-08T19:28:53.045726+0000","last_clean":"2023-11-08T19:28:53.045726+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:28:53.045726+0000","last_undegraded":"2023-11-08T19:28:53.045726+0000","last_fullsized":"2023-11-08T19:28:53.045726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":41,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.030448+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:28:53.030448+0000","last_peered":"2023-11-08T19:28:53.030448+0000","last_clean":"2023-11-08T19:28:53.030448+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:28:53.030448+0000","last_undegraded":"2023-11-08T19:28:53.030448+0000","last_fullsized":"2023-11-08T19:28:53.030448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":62,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310115+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:28:54.310115+0000","last_peered":"2023-11-08T19:28:54.310115+0000","last_clean":"2023-11-08T19:28:54.310115+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:28:54.310115+0000","last_undegraded":"2023-11-08T19:28:54.310115+0000","last_fullsized":"2023-11-08T19:28:54.310115+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.311170+0000","last_change":"2023-11-08T19:28:53.059033+0000","last_active":"2023-11-08T19:28:54.311170+0000","last_peered":"2023-11-08T19:28:54.311170+0000","last_clean":"2023-11-08T19:28:54.311170+0000","last_became_active":"2023-11-08T19:28:53.058679+0000","last_became_peered":"2023-11-08T19:28:53.058679+0000","last_unstale":"2023-11-08T19:28:54.311170+0000","last_undegraded":"2023-11-08T19:28:54.311170+0000","last_fullsized":"2023-11-08T19:28:54.311170+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:38:37.416484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.075754+0000","last_change":"2023-11-08T19:28:52.044999+0000","last_active":"2023-11-08T19:28:53.075754+0000","last_peered":"2023-11-08T19:28:53.075754+0000","last_clean":"2023-11-08T19:28:53.075754+0000","last_became_active":"2023-11-08T19:28:52.043311+0000","last_became_peered":"2023-11-08T19:28:52.043311+0000","last_unstale":"2023-11-08T19:28:53.075754+0000","last_undegraded":"2023-11-08T19:28:53.075754+0000","last_fullsized":"2023-11-08T19:28:53.075754+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:56:16.954121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043743+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:28:53.043743+0000","last_peered":"2023-11-08T19:28:53.043743+0000","last_clean":"2023-11-08T19:28:53.043743+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:28:53.043743+0000","last_undegraded":"2023-11-08T19:28:53.043743+0000","last_fullsized":"2023-11-08T19:28:53.043743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045869+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:28:53.045869+0000","last_peered":"2023-11-08T19:28:53.045869+0000","last_clean":"2023-11-08T19:28:53.045869+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:28:53.045869+0000","last_undegraded":"2023-11-08T19:28:53.045869+0000","last_fullsized":"2023-11-08T19:28:53.045869+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051254+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:28:53.051254+0000","last_peered":"2023-11-08T19:28:53.051254+0000","last_clean":"2023-11-08T19:28:53.051254+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:28:53.051254+0000","last_undegraded":"2023-11-08T19:28:53.051254+0000","last_fullsized":"2023-11-08T19:28:53.051254+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309337+0000","last_change":"2023-11-08T19:28:53.071095+0000","last_active":"2023-11-08T19:28:54.309337+0000","last_peered":"2023-11-08T19:28:54.309337+0000","last_clean":"2023-11-08T19:28:54.309337+0000","last_became_active":"2023-11-08T19:28:53.070753+0000","last_became_peered":"2023-11-08T19:28:53.070753+0000","last_unstale":"2023-11-08T19:28:54.309337+0000","last_undegraded":"2023-11-08T19:28:54.309337+0000","last_fullsized":"2023-11-08T19:28:54.309337+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:04.234641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.047167+0000","last_change":"2023-11-08T19:28:52.045459+0000","last_active":"2023-11-08T19:28:53.047167+0000","last_peered":"2023-11-08T19:28:53.047167+0000","last_clean":"2023-11-08T19:28:53.047167+0000","last_became_active":"2023-11-08T19:28:52.044147+0000","last_became_peered":"2023-11-08T19:28:52.044147+0000","last_unstale":"2023-11-08T19:28:53.047167+0000","last_undegraded":"2023-11-08T19:28:53.047167+0000","last_fullsized":"2023-11-08T19:28:53.047167+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:45:10.473698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":33,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052729+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:28:53.052729+0000","last_peered":"2023-11-08T19:28:53.052729+0000","last_clean":"2023-11-08T19:28:53.052729+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:28:53.052729+0000","last_undegraded":"2023-11-08T19:28:53.052729+0000","last_fullsized":"2023-11-08T19:28:53.052729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052531+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:28:53.052531+0000","last_peered":"2023-11-08T19:28:53.052531+0000","last_clean":"2023-11-08T19:28:53.052531+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:28:53.052531+0000","last_undegraded":"2023-11-08T19:28:53.052531+0000","last_fullsized":"2023-11-08T19:28:53.052531+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":63,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.048217+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:28:53.048217+0000","last_peered":"2023-11-08T19:28:53.048217+0000","last_clean":"2023-11-08T19:28:53.048217+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:28:53.048217+0000","last_undegraded":"2023-11-08T19:28:53.048217+0000","last_fullsized":"2023-11-08T19:28:53.048217+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.309580+0000","last_change":"2023-11-08T19:28:53.067720+0000","last_active":"2023-11-08T19:28:54.309580+0000","last_peered":"2023-11-08T19:28:54.309580+0000","last_clean":"2023-11-08T19:28:54.309580+0000","last_became_active":"2023-11-08T19:28:53.066313+0000","last_became_peered":"2023-11-08T19:28:53.066313+0000","last_unstale":"2023-11-08T19:28:54.309580+0000","last_undegraded":"2023-11-08T19:28:54.309580+0000","last_fullsized":"2023-11-08T19:28:54.309580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:52.651470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"95'2","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.076506+0000","last_change":"2023-11-08T19:28:52.048063+0000","last_active":"2023-11-08T19:28:53.076506+0000","last_peered":"2023-11-08T19:28:53.076506+0000","last_clean":"2023-11-08T19:28:53.076506+0000","last_became_active":"2023-11-08T19:28:52.047783+0000","last_became_peered":"2023-11-08T19:28:52.047783+0000","last_unstale":"2023-11-08T19:28:53.076506+0000","last_undegraded":"2023-11-08T19:28:53.076506+0000","last_fullsized":"2023-11-08T19:28:53.076506+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:12:02.863247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.048793+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:28:53.048793+0000","last_peered":"2023-11-08T19:28:53.048793+0000","last_clean":"2023-11-08T19:28:53.048793+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:28:53.048793+0000","last_undegraded":"2023-11-08T19:28:53.048793+0000","last_fullsized":"2023-11-08T19:28:53.048793+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":40,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052658+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:28:53.052658+0000","last_peered":"2023-11-08T19:28:53.052658+0000","last_clean":"2023-11-08T19:28:53.052658+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:28:53.052658+0000","last_undegraded":"2023-11-08T19:28:53.052658+0000","last_fullsized":"2023-11-08T19:28:53.052658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051655+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:28:53.051655+0000","last_peered":"2023-11-08T19:28:53.051655+0000","last_clean":"2023-11-08T19:28:53.051655+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:28:53.051655+0000","last_undegraded":"2023-11-08T19:28:53.051655+0000","last_fullsized":"2023-11-08T19:28:53.051655+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.310311+0000","last_change":"2023-11-08T19:28:53.059680+0000","last_active":"2023-11-08T19:28:54.310311+0000","last_peered":"2023-11-08T19:28:54.310311+0000","last_clean":"2023-11-08T19:28:54.310311+0000","last_became_active":"2023-11-08T19:28:53.059396+0000","last_became_peered":"2023-11-08T19:28:53.059396+0000","last_unstale":"2023-11-08T19:28:54.310311+0000","last_undegraded":"2023-11-08T19:28:54.310311+0000","last_fullsized":"2023-11-08T19:28:54.310311+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:47:40.633125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"95'8","reported_seq":20,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.082271+0000","last_change":"2023-11-08T19:28:52.041623+0000","last_active":"2023-11-08T19:28:53.082271+0000","last_peered":"2023-11-08T19:28:53.082271+0000","last_clean":"2023-11-08T19:28:53.082271+0000","last_became_active":"2023-11-08T19:28:52.040377+0000","last_became_peered":"2023-11-08T19:28:52.040377+0000","last_unstale":"2023-11-08T19:28:53.082271+0000","last_undegraded":"2023-11-08T19:28:53.082271+0000","last_fullsized":"2023-11-08T19:28:53.082271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:55.361254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":34,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.035645+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:28:53.035645+0000","last_peered":"2023-11-08T19:28:53.035645+0000","last_clean":"2023-11-08T19:28:53.035645+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:28:53.035645+0000","last_undegraded":"2023-11-08T19:28:53.035645+0000","last_fullsized":"2023-11-08T19:28:53.035645+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051503+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:28:53.051503+0000","last_peered":"2023-11-08T19:28:53.051503+0000","last_clean":"2023-11-08T19:28:53.051503+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:28:53.051503+0000","last_undegraded":"2023-11-08T19:28:53.051503+0000","last_fullsized":"2023-11-08T19:28:53.051503+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":64,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.054425+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:28:53.054425+0000","last_peered":"2023-11-08T19:28:53.054425+0000","last_clean":"2023-11-08T19:28:53.054425+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:28:53.054425+0000","last_undegraded":"2023-11-08T19:28:53.054425+0000","last_fullsized":"2023-11-08T19:28:53.054425+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.308728+0000","last_change":"2023-11-08T19:28:53.062739+0000","last_active":"2023-11-08T19:28:54.308728+0000","last_peered":"2023-11-08T19:28:54.308728+0000","last_clean":"2023-11-08T19:28:54.308728+0000","last_became_active":"2023-11-08T19:28:53.061762+0000","last_became_peered":"2023-11-08T19:28:53.061762+0000","last_unstale":"2023-11-08T19:28:54.308728+0000","last_undegraded":"2023-11-08T19:28:54.308728+0000","last_fullsized":"2023-11-08T19:28:54.308728+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:38.678500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.045190+0000","last_change":"2023-11-08T19:28:52.044950+0000","last_active":"2023-11-08T19:28:53.045190+0000","last_peered":"2023-11-08T19:28:53.045190+0000","last_clean":"2023-11-08T19:28:53.045190+0000","last_became_active":"2023-11-08T19:28:52.044673+0000","last_became_peered":"2023-11-08T19:28:52.044673+0000","last_unstale":"2023-11-08T19:28:53.045190+0000","last_undegraded":"2023-11-08T19:28:53.045190+0000","last_fullsized":"2023-11-08T19:28:53.045190+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:40:33.317757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028619+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:28:53.028619+0000","last_peered":"2023-11-08T19:28:53.028619+0000","last_clean":"2023-11-08T19:28:53.028619+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:28:53.028619+0000","last_undegraded":"2023-11-08T19:28:53.028619+0000","last_fullsized":"2023-11-08T19:28:53.028619+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026794+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:28:53.026794+0000","last_peered":"2023-11-08T19:28:53.026794+0000","last_clean":"2023-11-08T19:28:53.026794+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:28:53.026794+0000","last_undegraded":"2023-11-08T19:28:53.026794+0000","last_fullsized":"2023-11-08T19:28:53.026794+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.028271+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:28:53.028271+0000","last_peered":"2023-11-08T19:28:53.028271+0000","last_clean":"2023-11-08T19:28:53.028271+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:28:53.028271+0000","last_undegraded":"2023-11-08T19:28:53.028271+0000","last_fullsized":"2023-11-08T19:28:53.028271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062923+0000","last_change":"2023-11-08T19:28:53.044672+0000","last_active":"2023-11-08T19:28:54.062923+0000","last_peered":"2023-11-08T19:28:54.062923+0000","last_clean":"2023-11-08T19:28:54.062923+0000","last_became_active":"2023-11-08T19:28:53.044392+0000","last_became_peered":"2023-11-08T19:28:53.044392+0000","last_unstale":"2023-11-08T19:28:54.062923+0000","last_undegraded":"2023-11-08T19:28:54.062923+0000","last_fullsized":"2023-11-08T19:28:54.062923+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:18:17.402075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.027629+0000","last_change":"2023-11-08T19:28:52.046207+0000","last_active":"2023-11-08T19:28:53.027629+0000","last_peered":"2023-11-08T19:28:53.027629+0000","last_clean":"2023-11-08T19:28:53.027629+0000","last_became_active":"2023-11-08T19:28:52.044677+0000","last_became_peered":"2023-11-08T19:28:52.044677+0000","last_unstale":"2023-11-08T19:28:53.027629+0000","last_undegraded":"2023-11-08T19:28:53.027629+0000","last_fullsized":"2023-11-08T19:28:53.027629+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:55:17.866949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.026055+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:28:53.026055+0000","last_peered":"2023-11-08T19:28:53.026055+0000","last_clean":"2023-11-08T19:28:53.026055+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:28:53.026055+0000","last_undegraded":"2023-11-08T19:28:53.026055+0000","last_fullsized":"2023-11-08T19:28:53.026055+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043076+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:28:53.043076+0000","last_peered":"2023-11-08T19:28:53.043076+0000","last_clean":"2023-11-08T19:28:53.043076+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:28:53.043076+0000","last_undegraded":"2023-11-08T19:28:53.043076+0000","last_fullsized":"2023-11-08T19:28:53.043076+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":68,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.027226+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:28:53.027226+0000","last_peered":"2023-11-08T19:28:53.027226+0000","last_clean":"2023-11-08T19:28:53.027226+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:28:53.027226+0000","last_undegraded":"2023-11-08T19:28:53.027226+0000","last_fullsized":"2023-11-08T19:28:53.027226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.062718+0000","last_change":"2023-11-08T19:28:53.057917+0000","last_active":"2023-11-08T19:28:54.062718+0000","last_peered":"2023-11-08T19:28:54.062718+0000","last_clean":"2023-11-08T19:28:54.062718+0000","last_became_active":"2023-11-08T19:28:53.057566+0000","last_became_peered":"2023-11-08T19:28:53.057566+0000","last_unstale":"2023-11-08T19:28:54.062718+0000","last_undegraded":"2023-11-08T19:28:54.062718+0000","last_fullsized":"2023-11-08T19:28:54.062718+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:18:11.023596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"95'6","reported_seq":18,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.083681+0000","last_change":"2023-11-08T19:28:52.050926+0000","last_active":"2023-11-08T19:28:53.083681+0000","last_peered":"2023-11-08T19:28:53.083681+0000","last_clean":"2023-11-08T19:28:53.083681+0000","last_became_active":"2023-11-08T19:28:52.050635+0000","last_became_peered":"2023-11-08T19:28:52.050635+0000","last_unstale":"2023-11-08T19:28:53.083681+0000","last_undegraded":"2023-11-08T19:28:53.083681+0000","last_fullsized":"2023-11-08T19:28:53.083681+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:48:33.458673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":35,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029477+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:28:53.029477+0000","last_peered":"2023-11-08T19:28:53.029477+0000","last_clean":"2023-11-08T19:28:53.029477+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:28:53.029477+0000","last_undegraded":"2023-11-08T19:28:53.029477+0000","last_fullsized":"2023-11-08T19:28:53.029477+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":37,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044812+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:28:53.044812+0000","last_peered":"2023-11-08T19:28:53.044812+0000","last_clean":"2023-11-08T19:28:53.044812+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:28:53.044812+0000","last_undegraded":"2023-11-08T19:28:53.044812+0000","last_fullsized":"2023-11-08T19:28:53.044812+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.043239+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:28:53.043239+0000","last_peered":"2023-11-08T19:28:53.043239+0000","last_clean":"2023-11-08T19:28:53.043239+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:28:53.043239+0000","last_undegraded":"2023-11-08T19:28:53.043239+0000","last_fullsized":"2023-11-08T19:28:53.043239+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.311591+0000","last_change":"2023-11-08T19:28:53.069425+0000","last_active":"2023-11-08T19:28:54.311591+0000","last_peered":"2023-11-08T19:28:54.311591+0000","last_clean":"2023-11-08T19:28:54.311591+0000","last_became_active":"2023-11-08T19:28:53.069032+0000","last_became_peered":"2023-11-08T19:28:53.069032+0000","last_unstale":"2023-11-08T19:28:54.311591+0000","last_undegraded":"2023-11-08T19:28:54.311591+0000","last_fullsized":"2023-11-08T19:28:54.311591+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:27.399438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"95'3","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.082079+0000","last_change":"2023-11-08T19:28:52.046860+0000","last_active":"2023-11-08T19:28:53.082079+0000","last_peered":"2023-11-08T19:28:53.082079+0000","last_clean":"2023-11-08T19:28:53.082079+0000","last_became_active":"2023-11-08T19:28:52.046550+0000","last_became_peered":"2023-11-08T19:28:52.046550+0000","last_unstale":"2023-11-08T19:28:53.082079+0000","last_undegraded":"2023-11-08T19:28:53.082079+0000","last_fullsized":"2023-11-08T19:28:53.082079+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:32.968042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":32,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.025807+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:28:53.025807+0000","last_peered":"2023-11-08T19:28:53.025807+0000","last_clean":"2023-11-08T19:28:53.025807+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:28:53.025807+0000","last_undegraded":"2023-11-08T19:28:53.025807+0000","last_fullsized":"2023-11-08T19:28:53.025807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":39,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.025980+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:28:53.025980+0000","last_peered":"2023-11-08T19:28:53.025980+0000","last_clean":"2023-11-08T19:28:53.025980+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:28:53.025980+0000","last_undegraded":"2023-11-08T19:28:53.025980+0000","last_fullsized":"2023-11-08T19:28:53.025980+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":61,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.051336+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:28:53.051336+0000","last_peered":"2023-11-08T19:28:53.051336+0000","last_clean":"2023-11-08T19:28:53.051336+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:28:53.051336+0000","last_undegraded":"2023-11-08T19:28:53.051336+0000","last_fullsized":"2023-11-08T19:28:53.051336+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-11-08T19:28:54.307857+0000","last_change":"2023-11-08T19:28:53.058135+0000","last_active":"2023-11-08T19:28:54.307857+0000","last_peered":"2023-11-08T19:28:54.307857+0000","last_clean":"2023-11-08T19:28:54.307857+0000","last_became_active":"2023-11-08T19:28:53.056156+0000","last_became_peered":"2023-11-08T19:28:53.056156+0000","last_unstale":"2023-11-08T19:28:54.307857+0000","last_undegraded":"2023-11-08T19:28:54.307857+0000","last_fullsized":"2023-11-08T19:28:54.307857+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:51.998042+0000","last_clean_scrub_stamp":"2023-11-08T19:28:51.998042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:18:45.112533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.029271+0000","last_change":"2023-11-08T19:28:52.041200+0000","last_active":"2023-11-08T19:28:53.029271+0000","last_peered":"2023-11-08T19:28:53.029271+0000","last_clean":"2023-11-08T19:28:53.029271+0000","last_became_active":"2023-11-08T19:28:52.040920+0000","last_became_peered":"2023-11-08T19:28:52.040920+0000","last_unstale":"2023-11-08T19:28:53.029271+0000","last_undegraded":"2023-11-08T19:28:53.029271+0000","last_fullsized":"2023-11-08T19:28:53.029271+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:28:50.984789+0000","last_clean_scrub_stamp":"2023-11-08T19:28:50.984789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:35:45.303340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":31,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044802+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:28:53.044802+0000","last_peered":"2023-11-08T19:28:53.044802+0000","last_clean":"2023-11-08T19:28:53.044802+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:28:53.044802+0000","last_undegraded":"2023-11-08T19:28:53.044802+0000","last_fullsized":"2023-11-08T19:28:53.044802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":38,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.046975+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:28:53.046975+0000","last_peered":"2023-11-08T19:28:53.046975+0000","last_clean":"2023-11-08T19:28:53.046975+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:28:53.046975+0000","last_undegraded":"2023-11-08T19:28:53.046975+0000","last_fullsized":"2023-11-08T19:28:53.046975+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":62,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.052783+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:28:53.052783+0000","last_peered":"2023-11-08T19:28:53.052783+0000","last_clean":"2023-11-08T19:28:53.052783+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:28:53.052783+0000","last_undegraded":"2023-11-08T19:28:53.052783+0000","last_fullsized":"2023-11-08T19:28:53.052783+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-08T19:28:53.044531+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:28:53.044531+0000","last_peered":"2023-11-08T19:28:53.044531+0000","last_clean":"2023-11-08T19:28:53.044531+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:28:53.044531+0000","last_undegraded":"2023-11-08T19:28:53.044531+0000","last_fullsized":"2023-11-08T19:28:53.044531+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32932,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338908,"statfs":{"total":96636764160,"available":96603041792,"internally_reserved":0,"allocated":1785856,"data_stored":1179586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.081,"15min":1.1020000000000001},"min":{"1min":0.69099999999999995,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.397},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.206,"15min":1.2969999999999999},"min":{"1min":0.72499999999999998,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.581,"5min":4.4589999999999996,"15min":4.4589999999999996},"last":1.0229999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1830000000000001,"15min":1.2529999999999999},"min":{"1min":0.75700000000000001,"5min":0.628,"15min":0.628},"max":{"1min":1.728,"5min":4.5330000000000004,"15min":4.5330000000000004},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.202,"5min":1.137,"15min":1.1399999999999999},"min":{"1min":0.91500000000000004,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.673,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.90900000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.3480000000000001,"15min":1.478},"min":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.6930000000000001,"5min":4.6980000000000004,"15min":4.6980000000000004},"last":1.891},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.2330000000000001,"15min":1.347},"min":{"1min":0.78500000000000003,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.476,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":1.3140000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.2949999999999999,"15min":1.361},"min":{"1min":0.97499999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0459999999999998,"5min":2.2240000000000002,"15min":2.2240000000000002},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.4570000000000001,"15min":1.5529999999999999},"min":{"1min":0.98699999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"max":{"1min":1.7270000000000001,"5min":4.6150000000000002,"15min":4.6150000000000002},"last":1.833}]},{"osd":4,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.3169999999999999,"15min":1.357},"min":{"1min":0.90700000000000003,"5min":0.875,"15min":0.875},"max":{"1min":1.75,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.71},{"interface":"front","average":{"1min":1.429,"5min":1.4059999999999999,"15min":1.49},"min":{"1min":0.92400000000000004,"5min":0.78600000000000003,"15min":0.78600000000000003},"max":{"1min":2.141,"5min":2.2869999999999999,"15min":2.2869999999999999},"last":1.105}]},{"osd":5,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.431,"15min":1.46},"min":{"1min":1.1020000000000001,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.784,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.752},{"interface":"front","average":{"1min":1.391,"5min":1.429,"15min":1.4730000000000001},"min":{"1min":1.1140000000000001,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.732,"5min":2.1680000000000001,"15min":2.1680000000000001},"last":1.516}]},{"osd":6,"last update":"Wed Nov 8 19:28:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4850000000000001,"5min":1.524,"15min":1.5720000000000001},"min":{"1min":1.165,"5min":0.878,"15min":0.878},"max":{"1min":1.887,"5min":2.427,"15min":2.427},"last":1.609},{"interface":"front","average":{"1min":1.542,"5min":1.5429999999999999,"15min":1.623},"min":{"1min":1.0700000000000001,"5min":0.94999999999999996,"15min":0.94999999999999996},"max":{"1min":2.1920000000000002,"5min":2.468,"15min":2.468},"last":1.4690000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32340,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339500,"statfs":{"total":96636764160,"available":96603648000,"internally_reserved":0,"allocated":1187840,"data_stored":588672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.97099999999999997,"15min":0.93500000000000005},"min":{"1min":0.65600000000000003,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.329,"5min":1.752,"15min":1.752},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.077,"5min":1.109,"15min":1.1040000000000001},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"last":1.6060000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.0920000000000001,"15min":1.0900000000000001},"min":{"1min":0.71099999999999997,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4399999999999999,"5min":1.679,"15min":1.679},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.2390000000000001,"15min":1.323},"min":{"1min":0.69999999999999996,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.476,"5min":4.6399999999999997,"15min":4.6399999999999997},"last":1.143}]},{"osd":2,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1879999999999999,"15min":1.276},"min":{"1min":0.69299999999999995,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6419999999999999,"5min":4.609,"15min":4.609},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.234,"15min":1.218},"min":{"1min":0.74299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.48,"5min":1.861,"15min":1.861},"last":1.417}]},{"osd":3,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.363,"15min":1.4630000000000001},"min":{"1min":0.77900000000000003,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.655,"5min":4.7190000000000003,"15min":4.7190000000000003},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1759999999999999,"15min":1.179},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.587,"5min":1.784,"15min":1.784},"last":1.579}]},{"osd":4,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3740000000000001,"15min":1.3779999999999999},"min":{"1min":0.64100000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.881,"5min":2.0449999999999999,"15min":2.0449999999999999},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5189999999999999,"15min":1.5029999999999999},"min":{"1min":0.75700000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.998,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":1.5449999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.288,"15min":1.254},"min":{"1min":0.54900000000000004,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.835,"5min":1.9079999999999999,"15min":1.9079999999999999},"last":1.506},{"interface":"front","average":{"1min":1.353,"5min":1.3520000000000001,"15min":1.3360000000000001},"min":{"1min":0.47999999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.671,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":1.48}]},{"osd":7,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.5640000000000001,"15min":1.5489999999999999},"min":{"1min":0.85799999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.9399999999999999,"5min":2.2519999999999998,"15min":2.2519999999999998},"last":1.667},{"interface":"front","average":{"1min":1.456,"5min":1.454,"15min":1.454},"min":{"1min":0.505,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.8,"5min":3.0640000000000001,"15min":3.0640000000000001},"last":1.6419999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":589206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1100000000000001,"15min":1.0800000000000001},"min":{"1min":0.84899999999999998,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.4710000000000001,"5min":1.76,"15min":1.76},"last":1.4550000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.24,"15min":1.2370000000000001},"min":{"1min":0.80000000000000004,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.694,"5min":1.954,"15min":1.954},"last":1.0920000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":1.1279999999999999,"15min":1.1659999999999999},"min":{"1min":0.81100000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.6359999999999999,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.085,"5min":1.262,"15min":1.4139999999999999},"min":{"1min":0.77400000000000002,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.462,"5min":7.6779999999999999,"15min":7.6779999999999999},"last":0.99399999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.1970000000000001,"15min":1.177},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":1.7729999999999999,"5min":2.0640000000000001,"15min":2.0640000000000001},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.097,"5min":1.3029999999999999,"15min":1.4690000000000001},"min":{"1min":0.75,"5min":0.747,"15min":0.747},"max":{"1min":1.53,"5min":7.6479999999999997,"15min":7.6479999999999997},"last":1.377}]},{"osd":3,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.2509999999999999,"15min":1.2789999999999999},"min":{"1min":0.81399999999999995,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.573,"5min":2.1920000000000002,"15min":2.1920000000000002},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.333,"15min":1.3129999999999999},"min":{"1min":1.02,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.776,"5min":2.2120000000000002,"15min":2.2120000000000002},"last":1.3009999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.26,"15min":1.282},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.7969999999999999,"5min":2.347,"15min":2.347},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.4750000000000001,"15min":1.54},"min":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7989999999999999,"5min":3.153,"15min":3.153},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4610000000000001,"15min":1.5169999999999999},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8540000000000001,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.47,"5min":1.4690000000000001,"15min":1.4990000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.879,"5min":2.6190000000000002,"15min":2.6190000000000002},"last":1.512}]},{"osd":7,"last update":"Wed Nov 8 19:28:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.597,"15min":1.6859999999999999},"min":{"1min":0.94399999999999995,"5min":0.85799999999999998,"15min":0.85799999999999998},"max":{"1min":2.0630000000000002,"5min":3.2730000000000001,"15min":3.2730000000000001},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.5700000000000001,"15min":1.5860000000000001},"min":{"1min":0.96799999999999997,"5min":0.95099999999999996,"15min":0.95099999999999996},"max":{"1min":1.9219999999999999,"5min":2.4670000000000001,"15min":2.4670000000000001},"last":1.419}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1183744,"data_stored":588602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.077,"5min":1.0620000000000001,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.635,"5min":2.5219999999999998,"15min":2.5219999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.2390000000000001,"15min":1.204},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.546,"5min":7.0119999999999996,"15min":7.0119999999999996},"last":1.536}]},{"osd":1,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0620000000000001,"15min":1.0880000000000001},"min":{"1min":0.71199999999999997,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.4359999999999999,"5min":2.9790000000000001,"15min":2.9790000000000001},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":1.097,"15min":1.155},"min":{"1min":0.67200000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.3879999999999999,"5min":2.9940000000000002,"15min":2.9940000000000002},"last":1.151}]},{"osd":2,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1970000000000001,"5min":1.284,"15min":1.3340000000000001},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.784,"5min":4.0410000000000004,"15min":4.0410000000000004},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.3280000000000001,"15min":1.4099999999999999},"min":{"1min":0.70199999999999996,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.6910000000000001,"5min":3.2069999999999999,"15min":3.2069999999999999},"last":1.4099999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.2230000000000001,"15min":1.274},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.4470000000000001,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":1.458},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.3300000000000001,"15min":1.377},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.738,"5min":3.5710000000000002,"15min":3.5710000000000002},"last":1.494}]},{"osd":5,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.393,"15min":1.4970000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.7250000000000001,"5min":4.2350000000000003,"15min":4.2350000000000003},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.252,"5min":1.4319999999999999,"15min":1.528},"min":{"1min":0.66600000000000004,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.7529999999999999,"5min":4.5880000000000001,"15min":4.5880000000000001},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.4830000000000001,"15min":1.5489999999999999},"min":{"1min":1.022,"5min":0.746,"15min":0.746},"max":{"1min":1.8580000000000001,"5min":4.335,"15min":4.335},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.552,"15min":1.6140000000000001},"min":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.9490000000000001,"5min":4.4130000000000003,"15min":4.4130000000000003},"last":1.6040000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.546,"15min":1.6120000000000001},"min":{"1min":0.92300000000000004,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.823,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.6539999999999999,"15min":1.722},"min":{"1min":1.2430000000000001,"5min":1.095,"15min":1.095},"max":{"1min":2.0819999999999999,"5min":4.4950000000000001,"15min":4.4950000000000001},"last":1.5740000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1200128,"data_stored":590284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.90500000000000003,"15min":1.0449999999999999},"min":{"1min":0.34799999999999998,"5min":0.312,"15min":0.312},"max":{"1min":1.206,"5min":3.972,"15min":3.972},"last":0.877},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.99099999999999999,"15min":1.081},"min":{"1min":0.38300000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.47,"5min":3.1520000000000001,"15min":3.1520000000000001},"last":1.014}]},{"osd":1,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.054,"15min":1.169},"min":{"1min":0.47499999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.8480000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.98599999999999999,"15min":1.0860000000000001},"min":{"1min":0.55000000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.448,"5min":4.5750000000000002,"15min":4.5750000000000002},"last":1.075}]},{"osd":2,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.212,"15min":1.306},"min":{"1min":0.56899999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.639,"5min":3.9079999999999999,"15min":3.9079999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.1519999999999999,"15min":1.248},"min":{"1min":0.628,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.621,"5min":2.73,"15min":2.73},"last":1.395}]},{"osd":4,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.3220000000000001,"15min":1.431},"min":{"1min":0.72999999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.744,"5min":4.2770000000000001,"15min":4.2770000000000001},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.3500000000000001,"15min":1.4379999999999999},"min":{"1min":0.82999999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7030000000000001,"5min":3.052,"15min":3.052},"last":1.573}]},{"osd":5,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.375,"15min":1.478},"min":{"1min":0.83999999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6830000000000001,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3999999999999999,"15min":1.5089999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.9950000000000001,"5min":4.2380000000000004,"15min":4.2380000000000004},"last":1.2869999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.3839999999999999,"15min":1.4239999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.625,"5min":4.2220000000000004,"15min":4.2220000000000004},"last":1.2},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.4810000000000001,"15min":1.569},"min":{"1min":0.86899999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.944,"5min":3.1230000000000002,"15min":3.1230000000000002},"last":1.6359999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:28:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.5409999999999999,"15min":1.6850000000000001},"min":{"1min":0.90000000000000002,"5min":0.73799999999999999,"15min":0.73799999999999999},"max":{"1min":2.1150000000000002,"5min":4.407,"15min":4.407},"last":1.157},{"interface":"front","average":{"1min":1.333,"5min":1.4730000000000001,"15min":1.5629999999999999},"min":{"1min":0.84699999999999998,"5min":0.79400000000000004,"15min":0.79400000000000004},"max":{"1min":1.8100000000000001,"5min":4.3849999999999998,"15min":4.3849999999999998},"last":1.3169999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32332,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339508,"statfs":{"total":96636764160,"available":96603656192,"internally_reserved":0,"allocated":1179648,"data_stored":587583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.81599999999999995,"15min":0.85799999999999998},"min":{"1min":0.38100000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.3460000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.83999999999999997,"15min":0.91400000000000003},"min":{"1min":0.41699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.3779999999999999,"5min":1.4890000000000001,"15min":1.4890000000000001},"last":1.3560000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83099999999999996,"15min":0.86599999999999999},"min":{"1min":0.40200000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.8759999999999999,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.94899999999999995,"15min":1.0089999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.5029999999999999,"5min":1.5820000000000001,"15min":1.5820000000000001},"last":0.80500000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.008,"15min":1.0449999999999999},"min":{"1min":0.46600000000000003,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.6559999999999999,"5min":1.71,"15min":1.71},"last":1.181},{"interface":"front","average":{"1min":0.95899999999999996,"5min":1.02,"15min":1.113},"min":{"1min":0.58799999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.264,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.242}]},{"osd":4,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.27,"15min":1.391},"min":{"1min":0.71099999999999997,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.883,"5min":3.2959999999999998,"15min":3.2959999999999998},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.272,"15min":1.3560000000000001},"min":{"1min":0.89400000000000002,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.6639999999999999,"5min":2.278,"15min":2.278},"last":1.3260000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3400000000000001,"15min":1.4179999999999999},"min":{"1min":0.93200000000000005,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":3.0529999999999999,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.3260000000000001,"15min":1.4450000000000001},"min":{"1min":0.79800000000000004,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.9450000000000001,"5min":2.536,"15min":2.536},"last":1.1459999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.3320000000000001,"15min":1.3839999999999999},"min":{"1min":0.75900000000000001,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":1.9810000000000001,"5min":2.0840000000000001,"15min":2.0840000000000001},"last":1.391},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.3640000000000001,"15min":1.448},"min":{"1min":0.90000000000000002,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0169999999999999,"5min":2.0910000000000002,"15min":2.0910000000000002},"last":1.5189999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:28:56 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.4419999999999999,"15min":1.5600000000000001},"min":{"1min":0.89000000000000001,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9339999999999999,"5min":2.6549999999999998,"15min":2.6549999999999998},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.4530000000000001,"15min":1.5429999999999999},"min":{"1min":0.93500000000000005,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.748,"5min":2.3439999999999999,"15min":2.3439999999999999},"last":1.4830000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574903,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":1191936,"data_stored":588726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.77000000000000002,"15min":0.75},"min":{"1min":0.47199999999999998,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.3069999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.76300000000000001,"15min":0.75700000000000001},"min":{"1min":0.42599999999999999,"5min":0.374,"15min":0.374},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.76300000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.77900000000000003,"15min":0.79100000000000004},"min":{"1min":0.41999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.004,"5min":0.91100000000000003,"15min":0.89500000000000002},"min":{"1min":0.47499999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.1890000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.0780000000000001,"15min":1.0640000000000001},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.978,"5min":1.978,"15min":1.978},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.079,"5min":1.0669999999999999,"15min":1.085},"min":{"1min":0.5,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.778,"5min":2.5369999999999999,"15min":2.5369999999999999},"last":1.1539999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.145,"15min":1.153},"min":{"1min":0.72099999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.8620000000000001,"5min":2.165,"15min":2.165},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.272,"5min":1.2270000000000001,"15min":1.2529999999999999},"min":{"1min":0.64900000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":2.0830000000000002,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.008}]},{"osd":5,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.238,"15min":1.258},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.1930000000000001,"5min":2.4569999999999999,"15min":2.4569999999999999},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.254,"15min":1.3140000000000001},"min":{"1min":0.73199999999999998,"5min":0.629,"15min":0.629},"max":{"1min":1.748,"5min":2.4089999999999998,"15min":2.4089999999999998},"last":1.454}]},{"osd":6,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.363,"15min":1.3680000000000001},"min":{"1min":0.92900000000000005,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.097,"5min":2.6019999999999999,"15min":2.6019999999999999},"last":1.407},{"interface":"front","average":{"1min":1.266,"5min":1.3280000000000001,"15min":1.371},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.649,"5min":2.282,"15min":2.282},"last":1.546}]},{"osd":7,"last update":"Wed Nov 8 19:28:52 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.333,"15min":1.335},"min":{"1min":0.74199999999999999,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.131},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.355,"15min":1.3859999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.964,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.2410000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32940,"kb_used_data":1752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338900,"statfs":{"total":96636764160,"available":96603033600,"internally_reserved":0,"allocated":1794048,"data_stored":1179675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.40600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.296,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.871,"5min":0.872,"15min":0.92800000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3180000000000001,"5min":1.7,"15min":1.7},"last":1.022}]},{"osd":2,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.82799999999999996,"15min":0.86599999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.284,"5min":1.508,"15min":1.508},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.97099999999999997,"5min":0.92400000000000004,"15min":0.97199999999999998},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.407,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.95799999999999996}]},{"osd":3,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97099999999999997,"15min":1.002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.6599999999999999,"5min":1.748,"15min":1.748},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.0409999999999999,"15min":1.099},"min":{"1min":0.48499999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.506,"5min":2.496,"15min":2.496},"last":0.98799999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.2230000000000001,"15min":1.2290000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.5609999999999999,"5min":1.9239999999999999,"15min":1.9239999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.3049999999999999,"15min":1.4199999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.54,"5min":2.4039999999999999,"15min":2.4039999999999999},"last":1.4350000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.286,"15min":1.3600000000000001},"min":{"1min":0.68899999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6399999999999999,"5min":2.4369999999999998,"15min":2.4369999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.355,"15min":1.4330000000000001},"min":{"1min":0.78100000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.089,"5min":2.4239999999999999,"15min":2.4239999999999999},"last":1.3540000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.395,"15min":1.4870000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1619999999999999,"5min":2.5710000000000002,"15min":2.5710000000000002},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.3540000000000001,"15min":1.405},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.4249999999999998,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.4610000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:28:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3680000000000001,"15min":1.3979999999999999},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.3239999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.405,"15min":1.4770000000000001},"min":{"1min":0.84799999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.881,"5min":2.306,"15min":2.306},"last":1.53}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:29:03.136 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T19:29:03.137 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T19:29:03.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.395+0000 7f1375c90640 1 Processor -- start 2023-11-08T19:29:03.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.395+0000 7f1375c90640 1 -- start start 2023-11-08T19:29:03.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.396+0000 7f1375c90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f137015d9f0 0x7f137015de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.396+0000 7f1375c90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137006cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f1375c90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f137006d4f0 con 0x7f1370143600 2023-11-08T19:29:03.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f1375c90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13700c4f50 con 0x7f137015d9f0 2023-11-08T19:29:03.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f1375c90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13700c4960 con 0x7f137006cb40 2023-11-08T19:29:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f136f7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1370143600 0x7f13700c5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:51258/0 (socket says 172.21.15.119:51258) 2023-11-08T19:29:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f136f7fe640 1 -- 172.21.15.119:0/2307902854 learned_addr learned my addr 172.21.15.119:0/2307902854 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:03.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f136effd640 1 --2- 172.21.15.119:0/2307902854 >> v2:172.21.15.188:3300/0 conn(0x7f137015d9f0 0x7f137015de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.397+0000 7f136ffff640 1 --2- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137006cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.398+0000 7f136e7fc640 1 -- 172.21.15.119:0/2307902854 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455326824 0 0) 0x7f137006d4f0 con 0x7f1370143600 2023-11-08T19:29:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.398+0000 7f136e7fc640 1 -- 172.21.15.119:0/2307902854 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f135c003520 con 0x7f1370143600 2023-11-08T19:29:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.398+0000 7f136effd640 1 -- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 msgr2=0x7f137006cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:03.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.398+0000 7f136effd640 1 --2- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137006cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.398+0000 7f136effd640 1 -- 172.21.15.119:0/2307902854 >> v1:172.21.15.188:6789/0 conn(0x7f1370143600 legacy=0x7f13700c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.398+0000 7f136effd640 1 -- 172.21.15.119:0/2307902854 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137015e440 con 0x7f137015d9f0 2023-11-08T19:29:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.399+0000 7f136effd640 1 --2- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137015d9f0 0x7f137015de60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f13580097a0 tx=0x7f135802b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5b721439cf8e5c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.400+0000 7f136e7fc640 1 -- 172.21.15.119:0/2307902854 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135802e070 con 0x7f137015d9f0 2023-11-08T19:29:03.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.400+0000 7f136e7fc640 1 -- 172.21.15.119:0/2307902854 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13580315a0 con 0x7f137015d9f0 2023-11-08T19:29:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.400+0000 7f136e7fc640 1 -- 172.21.15.119:0/2307902854 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358031930 con 0x7f137015d9f0 2023-11-08T19:29:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.400+0000 7f1375c90640 1 -- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137015d9f0 msgr2=0x7f137015de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.400+0000 7f1375c90640 1 --2- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137015d9f0 0x7f137015de60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f13580097a0 tx=0x7f135802b840 comp rx=0 tx=0).stop 2023-11-08T19:29:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.401+0000 7f1375c90640 1 -- 172.21.15.119:0/2307902854 shutdown_connections 2023-11-08T19:29:03.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.401+0000 7f1375c90640 1 --2- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f137015d9f0 0x7f137015de60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.401+0000 7f1375c90640 1 --2- 172.21.15.119:0/2307902854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137006cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.401+0000 7f1375c90640 1 -- 172.21.15.119:0/2307902854 >> 172.21.15.119:0/2307902854 conn(0x7f13700cb610 msgr2=0x7f13700cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:03.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.401+0000 7f1375c90640 1 -- 172.21.15.119:0/2307902854 shutdown_connections 2023-11-08T19:29:03.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.401+0000 7f1375c90640 1 -- 172.21.15.119:0/2307902854 wait complete. 2023-11-08T19:29:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.402+0000 7f1375c90640 1 Processor -- start 2023-11-08T19:29:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.403+0000 7f1375c90640 1 -- start start 2023-11-08T19:29:03.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.403+0000 7f1375c90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137012ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.403+0000 7f1375c90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1370143600 0x7f137012e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f1375c90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 0x7f13701e9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f1375c90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137012fb50 con 0x7f137015d9f0 2023-11-08T19:29:03.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f1375c90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137012ef40 con 0x7f1370143600 2023-11-08T19:29:03.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f1375c90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137012f220 con 0x7f137006cb40 2023-11-08T19:29:03.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 0x7f13701e9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 0x7f13701e9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50262/0 (socket says 172.21.15.119:50262) 2023-11-08T19:29:03.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 -- 172.21.15.119:0/76135838 learned_addr learned my addr 172.21.15.119:0/76135838 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:03.410 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136f7fe640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137012ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 -- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 msgr2=0x7f137012ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137012ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 -- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1370143600 msgr2=0x7f137012e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1370143600 0x7f137012e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.404+0000 7f136ffff640 1 -- 172.21.15.119:0/76135838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358009400 con 0x7f137015d9f0 2023-11-08T19:29:03.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.405+0000 7f136ffff640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 0x7f13701e9710 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f136800e860 tx=0x7f136800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.405+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1368009540 con 0x7f137015d9f0 2023-11-08T19:29:03.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.405+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136800c8e0 con 0x7f137015d9f0 2023-11-08T19:29:03.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.405+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f137012f500 con 0x7f137015d9f0 2023-11-08T19:29:03.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.405+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136800cc90 con 0x7f137015d9f0 2023-11-08T19:29:03.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.405+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f137012eaa0 con 0x7f137015d9f0 2023-11-08T19:29:03.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.406+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f137012ecf0 con 0x7f137015d9f0 2023-11-08T19:29:03.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.407+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1368013470 con 0x7f137015d9f0 2023-11-08T19:29:03.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.408+0000 7f1367fff640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f135006ae70 0x7f135006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.408+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f1368089ee0 con 0x7f137015d9f0 2023-11-08T19:29:03.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.408+0000 7f136f7fe640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f135006ae70 0x7f135006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.409+0000 7f136f7fe640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f135006ae70 0x7f135006d330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1360005c00 tx=0x7f1360009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.412+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1368054990 con 0x7f137015d9f0 2023-11-08T19:29:03.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.670+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1370074430 con 0x7f137015d9f0 2023-11-08T19:29:03.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.674+0000 7f1367fff640 1 -- 172.21.15.119:0/76135838 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7f13680210b0 con 0x7f137015d9f0 2023-11-08T19:29:03.676 INFO:teuthology.orchestra.run.smithi119.stdout:[client.0] 2023-11-08T19:29:03.676 INFO:teuthology.orchestra.run.smithi119.stdout: key = AQBJ4EtlA6S6JxAAMRfdzdScnYLRdwuonon0Iw== 2023-11-08T19:29:03.676 INFO:teuthology.orchestra.run.smithi119.stdout: caps mds = "allow" 2023-11-08T19:29:03.676 INFO:teuthology.orchestra.run.smithi119.stdout: caps mon = "allow r" 2023-11-08T19:29:03.676 INFO:teuthology.orchestra.run.smithi119.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T19:29:03.677 INFO:teuthology.orchestra.run.smithi119.stderr:updated caps for client.0 2023-11-08T19:29:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.676+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f135006ae70 msgr2=0x7f135006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.676+0000 7f1375c90640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f135006ae70 0x7f135006d330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1360005c00 tx=0x7f1360009040 comp rx=0 tx=0).stop 2023-11-08T19:29:03.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.676+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 msgr2=0x7f13701e9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.676+0000 7f1375c90640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 0x7f13701e9710 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f136800e860 tx=0x7f136800ed20 comp rx=0 tx=0).stop 2023-11-08T19:29:03.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 shutdown_connections 2023-11-08T19:29:03.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f135006ae70 0x7f135006d330 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f137015d9f0 0x7f13701e9710 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1370143600 0x7f137012e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 --2- 172.21.15.119:0/76135838 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f137006cb40 0x7f137012ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 >> 172.21.15.119:0/76135838 conn(0x7f13700cb610 msgr2=0x7f13701428c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:03.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 shutdown_connections 2023-11-08T19:29:03.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.677+0000 7f1375c90640 1 -- 172.21.15.119:0/76135838 wait complete. 2023-11-08T19:29:03.698 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:03.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.904+0000 7fd2dc95e640 1 Processor -- start 2023-11-08T19:29:03.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.905+0000 7fd2dc95e640 1 -- start start 2023-11-08T19:29:03.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.906+0000 7fd2dc95e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd2d80cef40 0x7fd2d80cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.906+0000 7fd2dc95e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d80ca200 con 0x7fd2d8071a50 2023-11-08T19:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.906+0000 7fd2dc95e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2d815d580 con 0x7fd2d80cc3f0 2023-11-08T19:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.906+0000 7fd2dc95e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2d80cc870 con 0x7fd2d80cef40 2023-11-08T19:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.906+0000 7fd2d659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd2d8071a50 0x7fd2d80c9af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56588/0 (socket says 172.21.15.119:56588) 2023-11-08T19:29:03.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.906+0000 7fd2d659c640 1 -- 172.21.15.119:0/1620823970 learned_addr learned my addr 172.21.15.119:0/1620823970 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:03.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1178676967 0 0) 0x7fd2d80ca200 con 0x7fd2d8071a50 2023-11-08T19:29:03.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2c4003520 con 0x7fd2d8071a50 2023-11-08T19:29:03.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d5d9b640 1 --2- 172.21.15.119:0/1620823970 >> v2:172.21.15.188:3300/0 conn(0x7fd2d80cef40 0x7fd2d80cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 566852065 0 0) 0x7fd2c4003520 con 0x7fd2d8071a50 2023-11-08T19:29:03.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 >> v2:172.21.15.188:3300/0 conn(0x7fd2d80cef40 msgr2=0x7fd2d80cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:03.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 --2- 172.21.15.119:0/1620823970 >> v2:172.21.15.188:3300/0 conn(0x7fd2d80cef40 0x7fd2d80cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 >> v1:172.21.15.188:6789/0 conn(0x7fd2d80cc3f0 legacy=0x7fd2d815d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:03.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8071e30 con 0x7fd2d8071a50 2023-11-08T19:29:03.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd2c0001670 con 0x7fd2d8071a50 2023-11-08T19:29:03.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 >> v1:172.21.15.119:6789/0 conn(0x7fd2d8071a50 legacy=0x7fd2d80c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4003970 0x7fd2c4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2c40047c0 0x7fd2c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4005180 0x7fd2c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80cc870 con 0x7fd2c4005180 2023-11-08T19:29:03.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c0b0 con 0x7fd2c4003970 2023-11-08T19:29:03.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.907+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c400c270 con 0x7fd2c40047c0 2023-11-08T19:29:03.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4005180 0x7fd2c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d5d9b640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2c40047c0 0x7fd2c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2c40047c0 msgr2=0x7fd2c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:03.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2c40047c0 0x7fd2c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4003970 msgr2=0x7fd2c4003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:03.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d659c640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4003970 0x7fd2c4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4003970 0x7fd2c4003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1620823970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c4003520 con 0x7fd2c4005180 2023-11-08T19:29:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.908+0000 7fd2d659c640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4003970 0x7fd2c4003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.909+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1620823970 reap_dead start 2023-11-08T19:29:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.909+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4005180 0x7fd2c400b980 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0003d80 tx=0x7fd2c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e8930c987ba84d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.909+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0007220 con 0x7fd2c4005180 2023-11-08T19:29:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.909+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d002e050 con 0x7fd2c4005180 2023-11-08T19:29:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.909+0000 7fd2d559a640 1 -- 172.21.15.119:0/1620823970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d00075e0 con 0x7fd2c4005180 2023-11-08T19:29:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1620823970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4005180 msgr2=0x7fd2c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4005180 0x7fd2c400b980 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0003d80 tx=0x7fd2c40057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1620823970 shutdown_connections 2023-11-08T19:29:03.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1620823970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2c4005180 0x7fd2c400b980 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1620823970 >> 172.21.15.119:0/1620823970 conn(0x7fd2d806c280 msgr2=0x7fd2d806c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:03.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1620823970 shutdown_connections 2023-11-08T19:29:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.910+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1620823970 wait complete. 2023-11-08T19:29:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.912+0000 7fd2dc95e640 1 Processor -- start 2023-11-08T19:29:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.912+0000 7fd2dc95e640 1 -- start start 2023-11-08T19:29:03.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2dc95e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4005180 0x7fd2d80c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2dc95e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d80c2fd0 0x7fd2d8154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2dc95e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 0x7fd2d80b0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2dc95e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d8154850 con 0x7fd2d815d750 2023-11-08T19:29:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2dc95e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d81f27b0 con 0x7fd2c4005180 2023-11-08T19:29:03.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2dc95e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80c5940 con 0x7fd2d80c2fd0 2023-11-08T19:29:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2d6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 0x7fd2d80b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2d6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 0x7fd2d80b0170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50274/0 (socket says 172.21.15.119:50274) 2023-11-08T19:29:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.913+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1158891571 learned_addr learned my addr 172.21.15.119:0/1158891571 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d659c640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4005180 0x7fd2d80c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d5d9b640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d80c2fd0 0x7fd2d8154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d80c2fd0 msgr2=0x7fd2d8154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d80c2fd0 0x7fd2d8154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4005180 msgr2=0x7fd2d80c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4005180 0x7fd2d80c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:03.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d6d9d640 1 -- 172.21.15.119:0/1158891571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d80c5c20 con 0x7fd2d815d750 2023-11-08T19:29:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d659c640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4005180 0x7fd2d80c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2d6d9d640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 0x7fd2d80b0170 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0000c00 tx=0x7fd2d00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d0007a70 con 0x7fd2d815d750 2023-11-08T19:29:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2d003e470 con 0x7fd2d815d750 2023-11-08T19:29:03.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2d003e820 con 0x7fd2d815d750 2023-11-08T19:29:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d80c5f00 con 0x7fd2d815d750 2023-11-08T19:29:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.914+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d815ed20 con 0x7fd2d815d750 2023-11-08T19:29:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.916+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d80cacf0 con 0x7fd2d815d750 2023-11-08T19:29:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.917+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2d003e980 con 0x7fd2d815d750 2023-11-08T19:29:03.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.917+0000 7fd2ceffd640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2b806aea0 0x7fd2b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.918+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fd2d00ab8f0 con 0x7fd2d815d750 2023-11-08T19:29:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.918+0000 7fd2d659c640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2b806aea0 0x7fd2b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.918+0000 7fd2d659c640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2b806aea0 0x7fd2b806d360 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8160460 tx=0x7fd2c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:03.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:03.922+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2d00765d0 con 0x7fd2d815d750 2023-11-08T19:29:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.150+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2d8074430 con 0x7fd2d815d750 2023-11-08T19:29:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.151+0000 7fd2ceffd640 1 -- 172.21.15.119:0/1158891571 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4924 (secure 0 0 0) 0x7fd2d0079bf0 con 0x7fd2d815d750 2023-11-08T19:29:04.152 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:04.153 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7065,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:28:55.547548+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7068},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:04.153 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 53 2023-11-08T19:29:04.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.153+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2b806aea0 msgr2=0x7fd2b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.153+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2b806aea0 0x7fd2b806d360 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd2d8160460 tx=0x7fd2c0009040 comp rx=0 tx=0).stop 2023-11-08T19:29:04.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.153+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 msgr2=0x7fd2d80b0170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.153+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 0x7fd2d80b0170 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0000c00 tx=0x7fd2d00036f0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 shutdown_connections 2023-11-08T19:29:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2b806aea0 0x7fd2b806d360 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d815d750 0x7fd2d80b0170 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2d80c2fd0 0x7fd2d8154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 --2- 172.21.15.119:0/1158891571 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2c4005180 0x7fd2d80c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 >> 172.21.15.119:0/1158891571 conn(0x7fd2d806c280 msgr2=0x7fd2d806ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 shutdown_connections 2023-11-08T19:29:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.154+0000 7fd2dc95e640 1 -- 172.21.15.119:0/1158891571 wait complete. 2023-11-08T19:29:04.576 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:28:55.547548+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7068}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7068': {'gid': 7068, 'name': 'b', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/168553637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 168553637}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 168553637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:29:04.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T19:29:04.577 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:29:04.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.803+0000 7f1427d25640 1 Processor -- start 2023-11-08T19:29:04.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.804+0000 7f1427d25640 1 -- start start 2023-11-08T19:29:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.805+0000 7f1427d25640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f14201447a0 0x7f142015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.805+0000 7f1427d25640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f142006a120 0x7f142006a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.805+0000 7f1427d25640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142006aa70 con 0x7f14200cad30 2023-11-08T19:29:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.805+0000 7f1427d25640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1420070040 con 0x7f142006a120 2023-11-08T19:29:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f1427d25640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14200c3a50 con 0x7f14201447a0 2023-11-08T19:29:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f142006a120 0x7f142006a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f142006a120 0x7f142006a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50280/0 (socket says 172.21.15.119:50280) 2023-11-08T19:29:04.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 -- 172.21.15.119:0/4111185688 learned_addr learned my addr 172.21.15.119:0/4111185688 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f1425ac1640 1 --2- 172.21.15.119:0/4111185688 >> v2:172.21.15.188:3300/0 conn(0x7f14201447a0 0x7f142015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 -- 172.21.15.119:0/4111185688 >> v2:172.21.15.188:3300/0 conn(0x7f14201447a0 msgr2=0x7f142015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 --2- 172.21.15.119:0/4111185688 >> v2:172.21.15.188:3300/0 conn(0x7f14201447a0 0x7f142015f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 -- 172.21.15.119:0/4111185688 >> v1:172.21.15.188:6789/0 conn(0x7f14200cad30 legacy=0x7f142006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.806+0000 7f14262c2640 1 -- 172.21.15.119:0/4111185688 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1420144b80 con 0x7f142006a120 2023-11-08T19:29:04.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.808+0000 7f14262c2640 1 --2- 172.21.15.119:0/4111185688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142006a530 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f14200b1850 tx=0x7f141802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e2d76a9bd07daf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.810+0000 7f1424abf640 1 -- 172.21.15.119:0/4111185688 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418031640 con 0x7f142006a120 2023-11-08T19:29:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.810+0000 7f1424abf640 1 -- 172.21.15.119:0/4111185688 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14180317a0 con 0x7f142006a120 2023-11-08T19:29:04.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.810+0000 7f1424abf640 1 -- 172.21.15.119:0/4111185688 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418031b30 con 0x7f142006a120 2023-11-08T19:29:04.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.810+0000 7f1427d25640 1 -- 172.21.15.119:0/4111185688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 msgr2=0x7f142006a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.810+0000 7f1427d25640 1 --2- 172.21.15.119:0/4111185688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142006a530 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f14200b1850 tx=0x7f141802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.811+0000 7f1427d25640 1 -- 172.21.15.119:0/4111185688 shutdown_connections 2023-11-08T19:29:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.811+0000 7f1427d25640 1 --2- 172.21.15.119:0/4111185688 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142006a530 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.811+0000 7f1427d25640 1 --2- 172.21.15.119:0/4111185688 >> v2:172.21.15.188:3300/0 conn(0x7f14201447a0 0x7f142015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.811+0000 7f1427d25640 1 -- 172.21.15.119:0/4111185688 >> 172.21.15.119:0/4111185688 conn(0x7f14200cb610 msgr2=0x7f14200cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.811+0000 7f1427d25640 1 -- 172.21.15.119:0/4111185688 shutdown_connections 2023-11-08T19:29:04.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.811+0000 7f1427d25640 1 -- 172.21.15.119:0/4111185688 wait complete. 2023-11-08T19:29:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.812+0000 7f1427d25640 1 Processor -- start 2023-11-08T19:29:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.813+0000 7f1427d25640 1 -- start start 2023-11-08T19:29:04.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1427d25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1427d25640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14200cad30 0x7f14200c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1427d25640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14201447a0 0x7f142015de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1427d25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142015e3b0 con 0x7f142006a120 2023-11-08T19:29:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1427d25640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14200c5dd0 con 0x7f14201447a0 2023-11-08T19:29:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1427d25640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142015cdd0 con 0x7f14200cad30 2023-11-08T19:29:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142015c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50288/0 (socket says 172.21.15.119:50288) 2023-11-08T19:29:04.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 -- 172.21.15.119:0/1827975489 learned_addr learned my addr 172.21.15.119:0/1827975489 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:04.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 -- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14200cad30 msgr2=0x7f14200c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f14262c2640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14201447a0 0x7f142015de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14200cad30 0x7f14200c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 -- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14201447a0 msgr2=0x7f142015de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14201447a0 0x7f142015de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.814+0000 7f1425ac1640 1 -- 172.21.15.119:0/1827975489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14180094d0 con 0x7f142006a120 2023-11-08T19:29:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.815+0000 7f1425ac1640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142015c760 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1410007ef0 tx=0x7f141000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.817+0000 7f141e7fc640 1 -- 172.21.15.119:0/1827975489 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410014620 con 0x7f142006a120 2023-11-08T19:29:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.817+0000 7f141e7fc640 1 -- 172.21.15.119:0/1827975489 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410014780 con 0x7f142006a120 2023-11-08T19:29:04.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.817+0000 7f141e7fc640 1 -- 172.21.15.119:0/1827975489 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410014b30 con 0x7f142006a120 2023-11-08T19:29:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.817+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142015d0b0 con 0x7f142006a120 2023-11-08T19:29:04.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.817+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142015d670 con 0x7f142006a120 2023-11-08T19:29:04.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.819+0000 7f141e7fc640 1 -- 172.21.15.119:0/1827975489 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1410014c90 con 0x7f142006a120 2023-11-08T19:29:04.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.820+0000 7f141e7fc640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f140c06ae70 0x7f140c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.820+0000 7f141e7fc640 1 -- 172.21.15.119:0/1827975489 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f141008a560 con 0x7f142006a120 2023-11-08T19:29:04.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.820+0000 7f14252c0640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f140c06ae70 0x7f140c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.821+0000 7f14252c0640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f140c06ae70 0x7f140c06d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.821+0000 7f142a990740 1 Processor -- start 2023-11-08T19:29:04.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 -- start start 2023-11-08T19:29:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8feec0 0x55ad7f8d1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8ff2a0 0x55ad7f901c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 0x55ad7f902190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f8447e0 con 0x55ad7f8d2300 2023-11-08T19:29:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f841ed0 con 0x55ad7f8ff2a0 2023-11-08T19:29:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.822+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f83d7c0 con 0x55ad7f8feec0 2023-11-08T19:29:04.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 0x55ad7f902190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 0x55ad7f902190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50296/0 (socket says 172.21.15.119:50296) 2023-11-08T19:29:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 -- 172.21.15.119:0/667200757 learned_addr learned my addr 172.21.15.119:0/667200757 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f1425ac1640 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8feec0 0x55ad7f8d1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14252c0640 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8ff2a0 0x55ad7f901c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 -- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8feec0 msgr2=0x55ad7f8d1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8feec0 0x55ad7f8d1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 -- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8ff2a0 msgr2=0x55ad7f901c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8ff2a0 0x55ad7f901c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.823+0000 7f14262c2640 1 -- 172.21.15.119:0/667200757 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad7f810b10 con 0x55ad7f8d2300 2023-11-08T19:29:04.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.824+0000 7f14262c2640 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 0x55ad7f902190 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f141802b930 tx=0x7f1418008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0604082dcee8f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.824+0000 7f14077fe640 1 -- 172.21.15.119:0/667200757 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14180315c0 con 0x55ad7f8d2300 2023-11-08T19:29:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.824+0000 7f14077fe640 1 -- 172.21.15.119:0/667200757 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1418031910 con 0x55ad7f8d2300 2023-11-08T19:29:04.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.824+0000 7f14077fe640 1 -- 172.21.15.119:0/667200757 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418031cc0 con 0x55ad7f8d2300 2023-11-08T19:29:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 -- 172.21.15.119:0/667200757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 msgr2=0x55ad7f902190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 0x55ad7f902190 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f141802b930 tx=0x7f1418008030 comp rx=0 tx=0).stop 2023-11-08T19:29:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 -- 172.21.15.119:0/667200757 shutdown_connections 2023-11-08T19:29:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8d2300 0x55ad7f902190 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8ff2a0 0x55ad7f901c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 --2- 172.21.15.119:0/667200757 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8feec0 0x55ad7f8d1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 -- 172.21.15.119:0/667200757 >> 172.21.15.119:0/667200757 conn(0x55ad7f8d40d0 msgr2=0x55ad7f91cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 -- 172.21.15.119:0/667200757 shutdown_connections 2023-11-08T19:29:04.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 -- 172.21.15.119:0/667200757 wait complete. 2023-11-08T19:29:04.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.825+0000 7f142a990740 1 -- start start 2023-11-08T19:29:04.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.826+0000 7f142a990740 1 Processor -- start 2023-11-08T19:29:04.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f915960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f915ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f913bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f7dac40 con 0x55ad7f8fcc20 2023-11-08T19:29:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f7da8e0 con 0x55ad7f8feec0 2023-11-08T19:29:04.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f5a0f20 con 0x55ad7f8ff2a0 2023-11-08T19:29:04.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f142a990740 10 client.0 _ll_register_callbacks cb 0x55ad7f7fccf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:04.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f913bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f913bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50308/0 (socket says 172.21.15.119:50308) 2023-11-08T19:29:04.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 -- 172.21.15.119:0/2970572392 learned_addr learned my addr 172.21.15.119:0/2970572392 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:04.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 -- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 msgr2=0x55ad7f915ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f915ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 -- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 msgr2=0x55ad7f915960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f1425ac1640 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f915960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f915960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.827+0000 7f14262c2640 1 -- 172.21.15.119:0/2970572392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1420144b80 con 0x55ad7f8fcc20 2023-11-08T19:29:04.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f14262c2640 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f913bb0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1418006da0 tx=0x7f1418004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168957a7cb8ea755 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f1404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:04.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f1404ff9640 1 -- 172.21.15.119:0/2970572392 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418003610 con 0x55ad7f8fcc20 2023-11-08T19:29:04.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f1404ff9640 1 -- 172.21.15.119:0/2970572392 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f1418003770 con 0x55ad7f8fcc20 2023-11-08T19:29:04.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad7f811c80 con 0x55ad7f8fcc20 2023-11-08T19:29:04.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f1404ff9640 1 -- 172.21.15.119:0/2970572392 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1418003930 con 0x55ad7f8fcc20 2023-11-08T19:29:04.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f142a990740 10 client.7254 fetch_fsmap learned FSMap version 53 2023-11-08T19:29:04.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.828+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55ad7f83be40 con 0x55ad7f8fcc20 2023-11-08T19:29:04.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f1404ff9640 1 -- 172.21.15.119:0/2970572392 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ad7f811c80 con 0x55ad7f8fcc20 2023-11-08T19:29:04.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f1404ff9640 1 -- 172.21.15.119:0/2970572392 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f141804b030 con 0x55ad7f8fcc20 2023-11-08T19:29:04.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f142a990740 10 client.7254 fetch_fsmap finished waiting for FSMap version 53 2023-11-08T19:29:04.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f142a990740 10 client.7254 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:04.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f142a990740 20 client.7254 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:04.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f142a990740 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92c800 0x55ad7f92cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f142a990740 4 client.7254 mds_command: new command op to 7068 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:04.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.829+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 --> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad7f8367a0 con 0x55ad7f92c800 2023-11-08T19:29:04.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.830+0000 7f1425ac1640 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92c800 0x55ad7f92cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.831+0000 7f1425ac1640 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92c800 0x55ad7f92cbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798c5ec2f712f24a server_cookie=f9a32f6faa75d894 in_seq=0 out_seq=0 2023-11-08T19:29:04.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.831+0000 7f1404ff9640 10 client.7254 ms_handle_connect on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:04.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f1404ff9640 1 -- 172.21.15.119:0/2970572392 <== mds.0 v2:172.21.15.188:6833/168553637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ad7f8367a0 con 0x55ad7f92c800 2023-11-08T19:29:04.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f1404ff9640 10 client.7254 handle_command_reply: tid=0 2023-11-08T19:29:04.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 client.7254 shutdown 2023-11-08T19:29:04.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 msgr2=0x55ad7f913bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f913bb0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1418006da0 tx=0x7f1418004730 comp rx=0 tx=0).stop 2023-11-08T19:29:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 shutdown_connections 2023-11-08T19:29:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92c800 0x55ad7f92cbe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f913bb0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f915ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 --2- 172.21.15.119:0/2970572392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f915960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f1404ff9640 0 client.7254 ms_handle_reset on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 >> 172.21.15.119:0/2970572392 conn(0x55ad7f8d2300 msgr2=0x55ad7f901c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 shutdown_connections 2023-11-08T19:29:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.833+0000 7f142a990740 1 -- 172.21.15.119:0/2970572392 wait complete. 2023-11-08T19:29:04.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.834+0000 7f142a990740 20 client.7254 trim_cache size 0 max 16384 2023-11-08T19:29:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.850+0000 7f142a990740 1 Processor -- start 2023-11-08T19:29:04.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 -- start start 2023-11-08T19:29:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f920350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f902c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f903180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f8d5780 con 0x55ad7f8fcc20 2023-11-08T19:29:04.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f7e25d0 con 0x55ad7f8feec0 2023-11-08T19:29:04.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.851+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f903890 con 0x55ad7f8ff2a0 2023-11-08T19:29:04.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f920350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f920350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50314/0 (socket says 172.21.15.119:50314) 2023-11-08T19:29:04.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 -- 172.21.15.119:0/1728246144 learned_addr learned my addr 172.21.15.119:0/1728246144 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 -- 172.21.15.119:0/1728246144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 msgr2=0x55ad7f903180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f903180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 -- 172.21.15.119:0/1728246144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 msgr2=0x55ad7f902c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f902c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 -- 172.21.15.119:0/1728246144 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad7f8d8190 con 0x55ad7f8fcc20 2023-11-08T19:29:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1425ac1640 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f920350 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1410055260 tx=0x7f14100883f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9c7055a8b7fd07 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1407fff640 1 -- 172.21.15.119:0/1728246144 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410058d60 con 0x55ad7f8fcc20 2023-11-08T19:29:04.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.852+0000 7f1407fff640 1 -- 172.21.15.119:0/1728246144 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141005ceb0 con 0x55ad7f8fcc20 2023-11-08T19:29:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f1407fff640 1 -- 172.21.15.119:0/1728246144 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410061060 con 0x55ad7f8fcc20 2023-11-08T19:29:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 -- 172.21.15.119:0/1728246144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 msgr2=0x55ad7f920350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f920350 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1410055260 tx=0x7f14100883f0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 -- 172.21.15.119:0/1728246144 shutdown_connections 2023-11-08T19:29:04.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f903180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f902c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 --2- 172.21.15.119:0/1728246144 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f920350 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 -- 172.21.15.119:0/1728246144 >> 172.21.15.119:0/1728246144 conn(0x55ad7f8d2300 msgr2=0x55ad7f8cd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 -- 172.21.15.119:0/1728246144 shutdown_connections 2023-11-08T19:29:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.853+0000 7f142a990740 1 -- 172.21.15.119:0/1728246144 wait complete. 2023-11-08T19:29:04.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.854+0000 7f142a990740 1 -- start start 2023-11-08T19:29:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.854+0000 7f142a990740 1 Processor -- start 2023-11-08T19:29:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f91db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f91e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f92cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f91e950 con 0x55ad7f8fcc20 2023-11-08T19:29:04.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f902b90 con 0x55ad7f8feec0 2023-11-08T19:29:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ad7f92e3c0 con 0x55ad7f8ff2a0 2023-11-08T19:29:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f142a990740 10 client.0 _ll_register_callbacks cb 0x7f1418047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f1425ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f91db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.855+0000 7f1425ac1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f91db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50330/0 (socket says 172.21.15.119:50330) 2023-11-08T19:29:04.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 -- 172.21.15.119:0/3632290750 learned_addr learned my addr 172.21.15.119:0/3632290750 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:04.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f14252c0640 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f91e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 -- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 msgr2=0x55ad7f92cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f92cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 -- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 msgr2=0x55ad7f91e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:04.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f91e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 -- 172.21.15.119:0/3632290750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142015d670 con 0x55ad7f8fcc20 2023-11-08T19:29:04.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1425ac1640 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f91db50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f141000e9b0 tx=0x7f141005f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec24d8ff1dddc13 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:04.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1404ff9640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1404ff9640 1 -- 172.21.15.119:0/3632290750 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410061060 con 0x55ad7f8fcc20 2023-11-08T19:29:04.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f1404ff9640 1 -- 172.21.15.119:0/3632290750 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f14100885a0 con 0x55ad7f8fcc20 2023-11-08T19:29:04.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.856+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1418031910 con 0x55ad7f8fcc20 2023-11-08T19:29:04.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f1404ff9640 1 -- 172.21.15.119:0/3632290750 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14100821f0 con 0x55ad7f8fcc20 2023-11-08T19:29:04.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 10 client.7260 fetch_fsmap learned FSMap version 53 2023-11-08T19:29:04.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x55ad7f7d9b90 con 0x55ad7f8fcc20 2023-11-08T19:29:04.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f1404ff9640 1 -- 172.21.15.119:0/3632290750 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1418031910 con 0x55ad7f8fcc20 2023-11-08T19:29:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f1404ff9640 1 -- 172.21.15.119:0/3632290750 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1410062030 con 0x55ad7f8fcc20 2023-11-08T19:29:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 10 client.7260 fetch_fsmap finished waiting for FSMap version 53 2023-11-08T19:29:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 10 client.7260 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 20 client.7260 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92d120 0x55ad7f8fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:04.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 4 client.7260 mds_command: new command op to 7068 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.857+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 --> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ad7f8364c0 con 0x55ad7f92d120 2023-11-08T19:29:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.858+0000 7f14252c0640 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92d120 0x55ad7f8fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.858+0000 7f14252c0640 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92d120 0x55ad7f8fb7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f645abe1571be26f server_cookie=b8d5489611ce8824 in_seq=0 out_seq=0 2023-11-08T19:29:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.858+0000 7f1404ff9640 10 client.7260 ms_handle_connect on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f1404ff9640 1 -- 172.21.15.119:0/3632290750 <== mds.0 v2:172.21.15.188:6833/168553637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ad7f8364c0 con 0x55ad7f92d120 2023-11-08T19:29:04.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f1404ff9640 10 client.7260 handle_command_reply: tid=0 2023-11-08T19:29:04.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 client.7260 shutdown 2023-11-08T19:29:04.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 msgr2=0x55ad7f91db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f91db50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f141000e9b0 tx=0x7f141005f040 comp rx=0 tx=0).stop 2023-11-08T19:29:04.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 shutdown_connections 2023-11-08T19:29:04.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55ad7f92d120 0x55ad7f8fb7b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ad7f8ff2a0 0x55ad7f92cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ad7f8feec0 0x55ad7f91e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 --2- 172.21.15.119:0/3632290750 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55ad7f8fcc20 0x55ad7f91db50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 >> 172.21.15.119:0/3632290750 conn(0x55ad7f8d2300 msgr2=0x55ad7f920350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f1404ff9640 0 client.7260 ms_handle_reset on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:04.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 shutdown_connections 2023-11-08T19:29:04.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.859+0000 7f142a990740 1 -- 172.21.15.119:0/3632290750 wait complete. 2023-11-08T19:29:04.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f142a990740 20 client.7260 trim_cache size 0 max 16384 2023-11-08T19:29:04.872 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:29:04.872 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:29:04.873 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:29:04.873 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7068, 2023-11-08T19:29:04.873 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:29:04.873 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:29:04.873 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:29:04.874 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 11.810236525000001, 2023-11-08T19:29:04.874 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 53, 2023-11-08T19:29:04.874 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 96, 2023-11-08T19:29:04.874 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 96, 2023-11-08T19:29:04.874 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 19.233090568000001 2023-11-08T19:29:04.874 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:29:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f140c06ae70 msgr2=0x7f140c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f140c06ae70 0x7f140c06d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006bc0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 msgr2=0x7f142015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:04.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142015c760 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f1410007ef0 tx=0x7f141000e4b0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 shutdown_connections 2023-11-08T19:29:04.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f140c06ae70 0x7f140c06d330 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f14201447a0 0x7f142015de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f14200cad30 0x7f14200c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 --2- 172.21.15.119:0/1827975489 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f142006a120 0x7f142015c760 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 >> 172.21.15.119:0/1827975489 conn(0x7f14200cb610 msgr2=0x7f1420072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.860+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 shutdown_connections 2023-11-08T19:29:04.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:04.861+0000 7f1427d25640 1 -- 172.21.15.119:0/1827975489 wait complete. 2023-11-08T19:29:04.878 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T19:29:05.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.085+0000 7fd3dbe34640 1 Processor -- start 2023-11-08T19:29:05.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.085+0000 7fd3dbe34640 1 -- start start 2023-11-08T19:29:05.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.086+0000 7fd3dbe34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.086+0000 7fd3dbe34640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd3d40cfa00 0x7fd3d40cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.086+0000 7fd3dbe34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd3d4161160 0x7fd3d41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.086+0000 7fd3dbe34640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d4161b30 con 0x7fd3d4161160 2023-11-08T19:29:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.086+0000 7fd3dbe34640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d415daa0 con 0x7fd3d40cfa00 2023-11-08T19:29:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.086+0000 7fd3dbe34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d415df70 con 0x7fd3d415d6c0 2023-11-08T19:29:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd3d4161160 0x7fd3d41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd3d4161160 0x7fd3d41615f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50342/0 (socket says 172.21.15.119:50342) 2023-11-08T19:29:05.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 -- 172.21.15.119:0/2651589434 learned_addr learned my addr 172.21.15.119:0/2651589434 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:05.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3d9bd0640 1 --2- 172.21.15.119:0/2651589434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3d93cf640 1 --2- 172.21.15.119:0/2651589434 >> v2:172.21.15.188:3300/0 conn(0x7fd3d40cfa00 0x7fd3d40cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 -- 172.21.15.119:0/2651589434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 msgr2=0x7fd3d40d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:05.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 --2- 172.21.15.119:0/2651589434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 -- 172.21.15.119:0/2651589434 >> v2:172.21.15.188:3300/0 conn(0x7fd3d40cfa00 msgr2=0x7fd3d40cfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:05.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 --2- 172.21.15.119:0/2651589434 >> v2:172.21.15.188:3300/0 conn(0x7fd3d40cfa00 0x7fd3d40cfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3da3d1640 1 -- 172.21.15.119:0/2651589434 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d40d03a0 con 0x7fd3d4161160 2023-11-08T19:29:05.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.087+0000 7fd3d93cf640 1 --2- 172.21.15.119:0/2651589434 >> v2:172.21.15.188:3300/0 conn(0x7fd3d40cfa00 0x7fd3d40cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:05.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.088+0000 7fd3da3d1640 1 --2- 172.21.15.119:0/2651589434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d4161160 0x7fd3d41615f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40d4060 tx=0x7fd3cc02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68fbcdd933213fa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:05.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3d8bce640 1 -- 172.21.15.119:0/2651589434 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc02c9e0 con 0x7fd3d4161160 2023-11-08T19:29:05.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3d8bce640 1 -- 172.21.15.119:0/2651589434 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc02cb40 con 0x7fd3d4161160 2023-11-08T19:29:05.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3d8bce640 1 -- 172.21.15.119:0/2651589434 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc0343f0 con 0x7fd3d4161160 2023-11-08T19:29:05.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 -- 172.21.15.119:0/2651589434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d4161160 msgr2=0x7fd3d41615f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:05.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/2651589434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d4161160 0x7fd3d41615f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40d4060 tx=0x7fd3cc02be40 comp rx=0 tx=0).stop 2023-11-08T19:29:05.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 -- 172.21.15.119:0/2651589434 shutdown_connections 2023-11-08T19:29:05.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/2651589434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d4161160 0x7fd3d41615f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/2651589434 >> v2:172.21.15.188:3300/0 conn(0x7fd3d40cfa00 0x7fd3d40cfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/2651589434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 -- 172.21.15.119:0/2651589434 >> 172.21.15.119:0/2651589434 conn(0x7fd3d406a9f0 msgr2=0x7fd3d406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:05.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 -- 172.21.15.119:0/2651589434 shutdown_connections 2023-11-08T19:29:05.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.089+0000 7fd3dbe34640 1 -- 172.21.15.119:0/2651589434 wait complete. 2023-11-08T19:29:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.091+0000 7fd3dbe34640 1 Processor -- start 2023-11-08T19:29:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.091+0000 7fd3dbe34640 1 -- start start 2023-11-08T19:29:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3dbe34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 0x7fd3d414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3dbe34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3dbe34640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3d4161160 0x7fd3d40ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3dbe34640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d4159d10 con 0x7fd3d40cfa00 2023-11-08T19:29:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3dbe34640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40c9a20 con 0x7fd3d4161160 2023-11-08T19:29:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3dbe34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d40c9d00 con 0x7fd3d415d6c0 2023-11-08T19:29:05.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3d9bd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 0x7fd3d414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3d9bd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 0x7fd3d414f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50344/0 (socket says 172.21.15.119:50344) 2023-11-08T19:29:05.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.092+0000 7fd3d9bd0640 1 -- 172.21.15.119:0/585051777 learned_addr learned my addr 172.21.15.119:0/585051777 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:05.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3da3d1640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3d4161160 0x7fd3d40ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d93cf640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d9bd0640 1 -- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 msgr2=0x7fd3d40cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:05.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d9bd0640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d9bd0640 1 -- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3d4161160 msgr2=0x7fd3d40ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:05.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d9bd0640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3d4161160 0x7fd3d40ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d9bd0640 1 -- 172.21.15.119:0/585051777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc009b30 con 0x7fd3d40cfa00 2023-11-08T19:29:05.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3da3d1640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3d4161160 0x7fd3d40ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3d9bd0640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 0x7fd3d414f5e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d5f0 tx=0x7fd3d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d00125f0 con 0x7fd3d40cfa00 2023-11-08T19:29:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.093+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d0012750 con 0x7fd3d40cfa00 2023-11-08T19:29:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.094+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d0012b00 con 0x7fd3d40cfa00 2023-11-08T19:29:05.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.094+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d40cbfe0 con 0x7fd3d40cfa00 2023-11-08T19:29:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.094+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d40cc5a0 con 0x7fd3d40cfa00 2023-11-08T19:29:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.095+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d40cc760 con 0x7fd3d40cfa00 2023-11-08T19:29:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.096+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3d0012c60 con 0x7fd3d40cfa00 2023-11-08T19:29:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.097+0000 7fd3ca7fc640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd3b806ae70 0x7fd3b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:05.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.097+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fd3d008a060 con 0x7fd3d40cfa00 2023-11-08T19:29:05.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.097+0000 7fd3d93cf640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd3b806ae70 0x7fd3b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:05.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.098+0000 7fd3d93cf640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd3b806ae70 0x7fd3b806d330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40d27c0 tx=0x7fd3c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:05.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.102+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3d0054b90 con 0x7fd3d40cfa00 2023-11-08T19:29:05.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.326+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd3d4074430 con 0x7fd3d40cfa00 2023-11-08T19:29:05.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.327+0000 7fd3ca7fc640 1 -- 172.21.15.119:0/585051777 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd3d00581b0 con 0x7fd3d40cfa00 2023-11-08T19:29:05.328 INFO:teuthology.orchestra.run.smithi119.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T19:29:05.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd3b806ae70 msgr2=0x7fd3b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd3b806ae70 0x7fd3b806d330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd3d40d27c0 tx=0x7fd3c4008040 comp rx=0 tx=0).stop 2023-11-08T19:29:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 msgr2=0x7fd3d414f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 0x7fd3d414f5e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fd3d000d5f0 tx=0x7fd3d000dab0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 shutdown_connections 2023-11-08T19:29:05.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd3b806ae70 0x7fd3b806d330 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd3d4161160 0x7fd3d40ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd3d415d6c0 0x7fd3d40cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 --2- 172.21.15.119:0/585051777 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd3d40cfa00 0x7fd3d414f5e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 >> 172.21.15.119:0/585051777 conn(0x7fd3d406a9f0 msgr2=0x7fd3d40b6770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 shutdown_connections 2023-11-08T19:29:05.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:05.329+0000 7fd3dbe34640 1 -- 172.21.15.119:0/585051777 wait complete. 2023-11-08T19:29:05.349 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-08T19:29:05.350 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T19:29:05.350 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi119.front.sepia.ceph.com 2023-11-08T19:29:05.350 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T19:29:05.351 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T19:29:05.351 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-08T19:29:05.351 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-08T19:29:05.352 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T19:29:05.352 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:29:05.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.353 DEBUG:teuthology.orchestra.run.smithi119:> ip addr 2023-11-08T19:29:05.405 INFO:teuthology.orchestra.run.smithi119.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T19:29:05.406 INFO:teuthology.orchestra.run.smithi119.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T19:29:05.406 INFO:teuthology.orchestra.run.smithi119.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T19:29:05.406 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.406 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 ::1/128 scope host 2023-11-08T19:29:05.407 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.407 INFO:teuthology.orchestra.run.smithi119.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:29:05.407 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:62 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:29:05.407 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f0 2023-11-08T19:29:05.407 INFO:teuthology.orchestra.run.smithi119.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:29:05.407 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:d8:da:63 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:29:05.408 INFO:teuthology.orchestra.run.smithi119.stdout: altname enp7s0f1 2023-11-08T19:29:05.408 INFO:teuthology.orchestra.run.smithi119.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T19:29:05.408 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ee brd ff:ff:ff:ff:ff:ff 2023-11-08T19:29:05.408 INFO:teuthology.orchestra.run.smithi119.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T19:29:05.408 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 0c:c4:7a:8f:d5:ef brd ff:ff:ff:ff:ff:ff 2023-11-08T19:29:05.408 INFO:teuthology.orchestra.run.smithi119.stdout: inet 172.21.15.119/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T19:29:05.409 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft 42125sec preferred_lft 42125sec 2023-11-08T19:29:05.409 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::ec4:7aff:fe8f:d5ef/64 scope link 2023-11-08T19:29:05.409 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.409 INFO:teuthology.orchestra.run.smithi119.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T19:29:05.409 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff 2023-11-08T19:29:05.410 INFO:teuthology.orchestra.run.smithi119.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T19:29:05.410 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.410 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::e06c:64ff:fe55:3f8b/64 scope link 2023-11-08T19:29:05.410 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.410 INFO:teuthology.orchestra.run.smithi119.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:29:05.411 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether c2:86:39:1e:91:42 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:29:05.411 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::c086:39ff:fe1e:9142/64 scope link 2023-11-08T19:29:05.411 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.411 INFO:teuthology.orchestra.run.smithi119.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T19:29:05.411 INFO:teuthology.orchestra.run.smithi119.stdout: link/ether 06:94:cb:d6:d5:60 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:29:05.411 INFO:teuthology.orchestra.run.smithi119.stdout: inet6 fe80::494:cbff:fed6:d560/64 scope link 2023-11-08T19:29:05.412 INFO:teuthology.orchestra.run.smithi119.stdout: valid_lft forever preferred_lft forever 2023-11-08T19:29:05.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.413 DEBUG:teuthology.orchestra.run.smithi119:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T19:29:05.504 INFO:teuthology.orchestra.run.smithi119.stdout:1 2023-11-08T19:29:05.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.508 DEBUG:teuthology.orchestra.run.smithi119:> route 2023-11-08T19:29:05.563 INFO:teuthology.orchestra.run.smithi119.stdout:Kernel IP routing table 2023-11-08T19:29:05.564 INFO:teuthology.orchestra.run.smithi119.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T19:29:05.564 INFO:teuthology.orchestra.run.smithi119.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T19:29:05.564 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T19:29:05.564 INFO:teuthology.orchestra.run.smithi119.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T19:29:05.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.566 DEBUG:teuthology.orchestra.run.smithi119:> (cd / && exec bash -c ' 2023-11-08T19:29:05.566 DEBUG:teuthology.orchestra.run.smithi119:> set -e 2023-11-08T19:29:05.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T19:29:05.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T19:29:05.567 DEBUG:teuthology.orchestra.run.smithi119:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T19:29:05.568 DEBUG:teuthology.orchestra.run.smithi119:> ') 2023-11-08T19:29:05.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.749 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:29:05.798 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:29:05.799 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:29:05.799 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:29:05.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.800 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:29:05.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:29:05.882 DEBUG:teuthology.orchestra.run.smithi119:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T19:29:05.929 INFO:teuthology.orchestra.run.smithi119.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:29:05.929 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T19:29:05.929 DEBUG:teuthology.orchestra.run.smithi119:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:29:05.977 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-08T19:29:05.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:05.978 DEBUG:teuthology.orchestra.run.smithi119:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:29:06.165 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:29:06.165 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:29:06.199 INFO:teuthology.orchestra.run.smithi119.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-08T19:29:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-08T19:29:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: resolved to: "172.21.15.119:3300,172.21.15.188:3300,172.21.15.188:3301" 2023-11-08T19:29:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: trying mount with new device syntax: 0@ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d.cephfs=/ 2023-11-08T19:29:06.200 INFO:teuthology.orchestra.run.smithi119.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.119:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2023-11-08T19:29:06.201 INFO:teuthology.orchestra.run.smithi119.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-08T19:29:06.202 INFO:teuthology.orchestra.run.smithi119.stdout: You just mounted a file system that supports labels which does not 2023-11-08T19:29:06.202 INFO:teuthology.orchestra.run.smithi119.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-08T19:29:06.202 INFO:teuthology.orchestra.run.smithi119.stdout: applications will generate AVC messages and not be allowed access to 2023-11-08T19:29:06.202 INFO:teuthology.orchestra.run.smithi119.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-08T19:29:06.206 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-08T19:29:06.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:06.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T19:29:06.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:06.292 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-08T19:29:06.367 INFO:teuthology.orchestra.run.smithi119.stdout:client5111 2023-11-08T19:29:06.367 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:29:06.368 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:29:06.423 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config get mds_max_snaps_per_dir 2023-11-08T19:29:06.645 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_max_snaps_per_dir":"100"} 2023-11-08T19:29:06.645 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_max_snaps_per_dir": "100" } 2023-11-08T19:29:06.646 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2023-11-08T19:29:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.867+0000 7f573676c640 1 Processor -- start 2023-11-08T19:29:06.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.868+0000 7f573676c640 1 -- start start 2023-11-08T19:29:06.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f573676c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f573015bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:06.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f573676c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f573015f1d0 0x7f57300c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:06.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f573676c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573015ea60 con 0x7f5730144960 2023-11-08T19:29:06.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f573676c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57300c9150 con 0x7f573015f1d0 2023-11-08T19:29:06.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f573676c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57300c9360 con 0x7f5730144580 2023-11-08T19:29:06.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f5734d09640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f573015f1d0 0x7f57300c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:06.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f5734d09640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f573015f1d0 0x7f57300c3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50350/0 (socket says 172.21.15.119:50350) 2023-11-08T19:29:06.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.869+0000 7f5734d09640 1 -- 172.21.15.119:0/3994847338 learned_addr learned my addr 172.21.15.119:0/3994847338 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:06.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.870+0000 7f572ffff640 1 --2- 172.21.15.119:0/3994847338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f573015bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.870+0000 7f5734d09640 1 -- 172.21.15.119:0/3994847338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 msgr2=0x7f573015bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.870+0000 7f5734d09640 1 --2- 172.21.15.119:0/3994847338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f573015bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.870+0000 7f5734d09640 1 -- 172.21.15.119:0/3994847338 >> v1:172.21.15.188:6789/0 conn(0x7f5730144960 legacy=0x7f57300cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:06.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.870+0000 7f5734d09640 1 -- 172.21.15.119:0/3994847338 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57300c9640 con 0x7f573015f1d0 2023-11-08T19:29:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.871+0000 7f5734d09640 1 --2- 172.21.15.119:0/3994847338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f573015f1d0 0x7f57300c3a50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f57300c9290 tx=0x7f572802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a1d3957a0cc0ee2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.871+0000 7f572f7fe640 1 -- 172.21.15.119:0/3994847338 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572802c9c0 con 0x7f573015f1d0 2023-11-08T19:29:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.871+0000 7f572f7fe640 1 -- 172.21.15.119:0/3994847338 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572802cb20 con 0x7f573015f1d0 2023-11-08T19:29:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.871+0000 7f572f7fe640 1 -- 172.21.15.119:0/3994847338 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280343f0 con 0x7f573015f1d0 2023-11-08T19:29:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 -- 172.21.15.119:0/3994847338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f573015f1d0 msgr2=0x7f57300c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:06.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 --2- 172.21.15.119:0/3994847338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f573015f1d0 0x7f57300c3a50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f57300c9290 tx=0x7f572802be20 comp rx=0 tx=0).stop 2023-11-08T19:29:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 -- 172.21.15.119:0/3994847338 shutdown_connections 2023-11-08T19:29:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 --2- 172.21.15.119:0/3994847338 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f573015f1d0 0x7f57300c3a50 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 --2- 172.21.15.119:0/3994847338 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f573015bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:06.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 -- 172.21.15.119:0/3994847338 >> 172.21.15.119:0/3994847338 conn(0x7f57300cb610 msgr2=0x7f57300cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:06.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 -- 172.21.15.119:0/3994847338 shutdown_connections 2023-11-08T19:29:06.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.872+0000 7f573676c640 1 -- 172.21.15.119:0/3994847338 wait complete. 2023-11-08T19:29:06.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.874+0000 7f573676c640 1 Processor -- start 2023-11-08T19:29:06.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.875+0000 7f573676c640 1 -- start start 2023-11-08T19:29:06.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f573676c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f5730140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:06.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f573676c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 0x7f57300afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f573676c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573015f1d0 0x7f57301e6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f573676c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57300c83a0 con 0x7f5730144960 2023-11-08T19:29:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f573676c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57300c8680 con 0x7f573015f1d0 2023-11-08T19:29:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f573676c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57300c5c70 con 0x7f5730144580 2023-11-08T19:29:06.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 0x7f57300afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:06.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 0x7f57300afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50352/0 (socket says 172.21.15.119:50352) 2023-11-08T19:29:06.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 -- 172.21.15.119:0/1894503505 learned_addr learned my addr 172.21.15.119:0/1894503505 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:06.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f572ffff640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f5730140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:06.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5734d09640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573015f1d0 0x7f57301e6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:06.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 -- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 msgr2=0x7f5730140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f5730140540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 -- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573015f1d0 msgr2=0x7f57301e6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573015f1d0 0x7f57301e6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.876+0000 7f5727fff640 1 -- 172.21.15.119:0/1894503505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5728009b30 con 0x7f5730144960 2023-11-08T19:29:06.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.877+0000 7f5727fff640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 0x7f57300afeb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f571800c430 tx=0x7f571800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.877+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718011790 con 0x7f5730144960 2023-11-08T19:29:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.877+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57180118f0 con 0x7f5730144960 2023-11-08T19:29:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.877+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5718011ce0 con 0x7f5730144960 2023-11-08T19:29:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.877+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57300c5f50 con 0x7f5730144960 2023-11-08T19:29:06.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.877+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57301412b0 con 0x7f5730144960 2023-11-08T19:29:06.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.878+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57300c6140 con 0x7f5730144960 2023-11-08T19:29:06.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.879+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f571801c460 con 0x7f5730144960 2023-11-08T19:29:06.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.880+0000 7f572d7fa640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f571006ae70 0x7f571006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:06.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.881+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f5718089b30 con 0x7f5730144960 2023-11-08T19:29:06.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.881+0000 7f572ffff640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f571006ae70 0x7f571006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:06.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.881+0000 7f572ffff640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f571006ae70 0x7f571006d330 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5720006fd0 tx=0x7f5720008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:06.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:06.885+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5718054660 con 0x7f5730144960 2023-11-08T19:29:07.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.118+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5730074430 con 0x7f5730144960 2023-11-08T19:29:07.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.822+0000 7f572d7fa640 1 -- 172.21.15.119:0/1894503505 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v54) v1 ==== 167+0+0 (secure 0 0 0) 0x7f5718057c80 con 0x7f5730144960 2023-11-08T19:29:07.824 INFO:teuthology.orchestra.run.smithi119.stderr:enabled new snapshots 2023-11-08T19:29:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f571006ae70 msgr2=0x7f571006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f571006ae70 0x7f571006d330 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f5720006fd0 tx=0x7f5720008040 comp rx=0 tx=0).stop 2023-11-08T19:29:07.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 msgr2=0x7f57300afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 0x7f57300afeb0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f571800c430 tx=0x7f571800c8f0 comp rx=0 tx=0).stop 2023-11-08T19:29:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 shutdown_connections 2023-11-08T19:29:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f571006ae70 0x7f571006d330 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f573015f1d0 0x7f57301e6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:07.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5730144960 0x7f57300afeb0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:07.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 --2- 172.21.15.119:0/1894503505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5730144580 0x7f5730140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:07.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.828+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 >> 172.21.15.119:0/1894503505 conn(0x7f57300cb610 msgr2=0x7f57300cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:07.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.829+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 shutdown_connections 2023-11-08T19:29:07.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:07.829+0000 7f573676c640 1 -- 172.21.15.119:0/1894503505 wait complete. 2023-11-08T19:29:07.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 2 2023-11-08T19:29:08.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.073+0000 7f8a5d316640 1 Processor -- start 2023-11-08T19:29:08.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.073+0000 7f8a5d316640 1 -- start start 2023-11-08T19:29:08.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a5d316640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a58144580 0x7f8a5815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a5d316640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8a58144960 0x7f8a580cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a5d316640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a5815f1d0 0x7f8a580c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a5d316640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580c9150 con 0x7f8a58144580 2023-11-08T19:29:08.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a5d316640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5815ea60 con 0x7f8a58144960 2023-11-08T19:29:08.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a5d316640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580cb270 con 0x7f8a5815f1d0 2023-11-08T19:29:08.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a56ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a58144580 0x7f8a5815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a56ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a58144580 0x7f8a5815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50362/0 (socket says 172.21.15.119:50362) 2023-11-08T19:29:08.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a56ffd640 1 -- 172.21.15.119:0/366834080 learned_addr learned my addr 172.21.15.119:0/366834080 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:08.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a577fe640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a5815f1d0 0x7f8a580c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.075+0000 7f8a567fc640 1 --2- 172.21.15.119:0/366834080 >> v2:172.21.15.188:3300/0 conn(0x7f8a58144960 0x7f8a580cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a56ffd640 1 -- 172.21.15.119:0/366834080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a5815f1d0 msgr2=0x7f8a580c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a56ffd640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a5815f1d0 0x7f8a580c3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a56ffd640 1 -- 172.21.15.119:0/366834080 >> v2:172.21.15.188:3300/0 conn(0x7f8a58144960 msgr2=0x7f8a580cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a56ffd640 1 --2- 172.21.15.119:0/366834080 >> v2:172.21.15.188:3300/0 conn(0x7f8a58144960 0x7f8a580cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a56ffd640 1 -- 172.21.15.119:0/366834080 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580c9360 con 0x7f8a58144580 2023-11-08T19:29:08.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a577fe640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a5815f1d0 0x7f8a580c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:08.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.076+0000 7f8a567fc640 1 --2- 172.21.15.119:0/366834080 >> v2:172.21.15.188:3300/0 conn(0x7f8a58144960 0x7f8a580cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:08.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.077+0000 7f8a56ffd640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a58144580 0x7f8a5815bff0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8a580ce900 tx=0x7f8a4802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcc0e993c7744bd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.077+0000 7f8a55ffb640 1 -- 172.21.15.119:0/366834080 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a480317d0 con 0x7f8a58144580 2023-11-08T19:29:08.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a55ffb640 1 -- 172.21.15.119:0/366834080 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a48031930 con 0x7f8a58144580 2023-11-08T19:29:08.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a55ffb640 1 -- 172.21.15.119:0/366834080 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48031cc0 con 0x7f8a58144580 2023-11-08T19:29:08.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 -- 172.21.15.119:0/366834080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a58144580 msgr2=0x7f8a5815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a58144580 0x7f8a5815bff0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f8a580ce900 tx=0x7f8a4802b870 comp rx=0 tx=0).stop 2023-11-08T19:29:08.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 -- 172.21.15.119:0/366834080 shutdown_connections 2023-11-08T19:29:08.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a5815f1d0 0x7f8a580c3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 --2- 172.21.15.119:0/366834080 >> v2:172.21.15.188:3300/0 conn(0x7f8a58144960 0x7f8a580cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 --2- 172.21.15.119:0/366834080 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a58144580 0x7f8a5815bff0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 -- 172.21.15.119:0/366834080 >> 172.21.15.119:0/366834080 conn(0x7f8a580cb610 msgr2=0x7f8a580cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:08.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 -- 172.21.15.119:0/366834080 shutdown_connections 2023-11-08T19:29:08.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.078+0000 7f8a5d316640 1 -- 172.21.15.119:0/366834080 wait complete. 2023-11-08T19:29:08.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.080+0000 7f8a5d316640 1 Processor -- start 2023-11-08T19:29:08.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.080+0000 7f8a5d316640 1 -- start start 2023-11-08T19:29:08.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.081+0000 7f8a5d316640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a58144580 0x7f8a580c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.081+0000 7f8a5d316640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a58144960 0x7f8a5815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.081+0000 7f8a5d316640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 0x7f8a5813d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.081+0000 7f8a5d316640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5815ad70 con 0x7f8a5815f1d0 2023-11-08T19:29:08.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.081+0000 7f8a5d316640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c6260 con 0x7f8a58144580 2023-11-08T19:29:08.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.081+0000 7f8a5d316640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580c6540 con 0x7f8a58144960 2023-11-08T19:29:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 0x7f8a5813d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 0x7f8a5813d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50378/0 (socket says 172.21.15.119:50378) 2023-11-08T19:29:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 -- 172.21.15.119:0/3552937343 learned_addr learned my addr 172.21.15.119:0/3552937343 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a567fc640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a58144960 0x7f8a5815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a56ffd640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a58144580 0x7f8a580c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 -- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a58144960 msgr2=0x7f8a5815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a58144960 0x7f8a5815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 -- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a58144580 msgr2=0x7f8a580c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a58144580 0x7f8a580c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 -- 172.21.15.119:0/3552937343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a480094d0 con 0x7f8a5815f1d0 2023-11-08T19:29:08.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.082+0000 7f8a577fe640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 0x7f8a5813d900 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000cca0 tx=0x7f8a5000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.083+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000f680 con 0x7f8a5815f1d0 2023-11-08T19:29:08.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.083+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5000f800 con 0x7f8a5815f1d0 2023-11-08T19:29:08.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.083+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000fbb0 con 0x7f8a5815f1d0 2023-11-08T19:29:08.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.083+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a580c6820 con 0x7f8a5815f1d0 2023-11-08T19:29:08.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.083+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a580c6de0 con 0x7f8a5815f1d0 2023-11-08T19:29:08.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.084+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a58074430 con 0x7f8a5815f1d0 2023-11-08T19:29:08.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.085+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a5000b040 con 0x7f8a5815f1d0 2023-11-08T19:29:08.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.086+0000 7f8a3f7fe640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a4406ae70 0x7f8a4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.086+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f8a5008a4d0 con 0x7f8a5815f1d0 2023-11-08T19:29:08.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.086+0000 7f8a56ffd640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a4406ae70 0x7f8a4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.087+0000 7f8a56ffd640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a4406ae70 0x7f8a4406d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8a48036000 tx=0x7f8a4803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.090+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a50055000 con 0x7f8a5815f1d0 2023-11-08T19:29:08.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.316+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f8a5815ecb0 con 0x7f8a5815f1d0 2023-11-08T19:29:08.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.677+0000 7f8a3f7fe640 1 -- 172.21.15.119:0/3552937343 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v55) v1 ==== 105+0+0 (secure 0 0 0) 0x7f8a50058620 con 0x7f8a5815f1d0 2023-11-08T19:29:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a4406ae70 msgr2=0x7f8a4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a4406ae70 0x7f8a4406d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8a48036000 tx=0x7f8a4803d040 comp rx=0 tx=0).stop 2023-11-08T19:29:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 msgr2=0x7f8a5813d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 0x7f8a5813d900 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000cca0 tx=0x7f8a5000a650 comp rx=0 tx=0).stop 2023-11-08T19:29:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 shutdown_connections 2023-11-08T19:29:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a4406ae70 0x7f8a4406d330 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a5815f1d0 0x7f8a5813d900 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a58144960 0x7f8a5815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 --2- 172.21.15.119:0/3552937343 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a58144580 0x7f8a580c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.680+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 >> 172.21.15.119:0/3552937343 conn(0x7f8a580cb610 msgr2=0x7f8a5806a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.681+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 shutdown_connections 2023-11-08T19:29:08.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.681+0000 7f8a5d316640 1 -- 172.21.15.119:0/3552937343 wait complete. 2023-11-08T19:29:08.710 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:08.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.901+0000 7fc315983640 1 Processor -- start 2023-11-08T19:29:08.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.902+0000 7fc315983640 1 -- start start 2023-11-08T19:29:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc315983640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b2d0 0x7fc3100cd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc315983640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc3100caf40 0x7fc3100cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc315983640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31015b6b0 con 0x7fc3100d18d0 2023-11-08T19:29:08.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc315983640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc31006eb20 con 0x7fc3100caf40 2023-11-08T19:29:08.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc315983640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100cc980 con 0x7fc31015b2d0 2023-11-08T19:29:08.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc30e7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc3100caf40 0x7fc3100cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc30e7fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc3100caf40 0x7fc3100cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52638/0 (socket says 172.21.15.119:52638) 2023-11-08T19:29:08.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.903+0000 7fc30e7fc640 1 -- 172.21.15.119:0/4087398814 learned_addr learned my addr 172.21.15.119:0/4087398814 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:08.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.904+0000 7fc30e7fc640 1 -- 172.21.15.119:0/4087398814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b2d0 msgr2=0x7fc3100cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.904+0000 7fc30effd640 1 --2- 172.21.15.119:0/4087398814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b2d0 0x7fc3100cd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.904+0000 7fc30e7fc640 1 --2- 172.21.15.119:0/4087398814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b2d0 0x7fc3100cd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.904+0000 7fc30e7fc640 1 -- 172.21.15.119:0/4087398814 >> v1:172.21.15.188:6789/0 conn(0x7fc3100d18d0 legacy=0x7fc31006d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:08.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.904+0000 7fc30e7fc640 1 -- 172.21.15.119:0/4087398814 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100ccc60 con 0x7fc3100caf40 2023-11-08T19:29:08.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.905+0000 7fc30e7fc640 1 --2- 172.21.15.119:0/4087398814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc3100cdd00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009870 tx=0x7fc2f802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570c12a357086da6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.905+0000 7fc30dffb640 1 -- 172.21.15.119:0/4087398814 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80317c0 con 0x7fc3100caf40 2023-11-08T19:29:08.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.905+0000 7fc30dffb640 1 -- 172.21.15.119:0/4087398814 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8031920 con 0x7fc3100caf40 2023-11-08T19:29:08.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.905+0000 7fc30dffb640 1 -- 172.21.15.119:0/4087398814 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8031cb0 con 0x7fc3100caf40 2023-11-08T19:29:08.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 -- 172.21.15.119:0/4087398814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 msgr2=0x7fc3100cdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 --2- 172.21.15.119:0/4087398814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc3100cdd00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8009870 tx=0x7fc2f802b860 comp rx=0 tx=0).stop 2023-11-08T19:29:08.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 -- 172.21.15.119:0/4087398814 shutdown_connections 2023-11-08T19:29:08.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 --2- 172.21.15.119:0/4087398814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc3100cdd00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 --2- 172.21.15.119:0/4087398814 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc31015b2d0 0x7fc3100cd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 -- 172.21.15.119:0/4087398814 >> 172.21.15.119:0/4087398814 conn(0x7fc3100d0db0 msgr2=0x7fc3100d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:08.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 -- 172.21.15.119:0/4087398814 shutdown_connections 2023-11-08T19:29:08.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.906+0000 7fc315983640 1 -- 172.21.15.119:0/4087398814 wait complete. 2023-11-08T19:29:08.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.908+0000 7fc315983640 1 Processor -- start 2023-11-08T19:29:08.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.909+0000 7fc315983640 1 -- start start 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc315983640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc31015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc315983640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100d18d0 0x7fc310156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc315983640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b2d0 0x7fc310157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc315983640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31015e920 con 0x7fc3100caf40 2023-11-08T19:29:08.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc315983640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31015bde0 con 0x7fc31015b2d0 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc315983640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100c7f20 con 0x7fc3100d18d0 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc31015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc31015b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52642/0 (socket says 172.21.15.119:52642) 2023-11-08T19:29:08.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30effd640 1 -- 172.21.15.119:0/3179152290 learned_addr learned my addr 172.21.15.119:0/3179152290 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:08.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30f7fe640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b2d0 0x7fc310157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30e7fc640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100d18d0 0x7fc310156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30effd640 1 -- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100d18d0 msgr2=0x7fc310156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.910+0000 7fc30effd640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100d18d0 0x7fc310156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc30effd640 1 -- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b2d0 msgr2=0x7fc310157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:08.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc30effd640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b2d0 0x7fc310157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:08.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc30effd640 1 -- 172.21.15.119:0/3179152290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f80094d0 con 0x7fc3100caf40 2023-11-08T19:29:08.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc30f7fe640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b2d0 0x7fc310157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:08.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc30effd640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc31015b8a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc30000cd70 tx=0x7fc30000a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300003b90 con 0x7fc3100caf40 2023-11-08T19:29:08.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300003cf0 con 0x7fc3100caf40 2023-11-08T19:29:08.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300004240 con 0x7fc3100caf40 2023-11-08T19:29:08.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc310156ac0 con 0x7fc3100caf40 2023-11-08T19:29:08.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.911+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31015c2d0 con 0x7fc3100caf40 2023-11-08T19:29:08.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.912+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc310074430 con 0x7fc3100caf40 2023-11-08T19:29:08.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.913+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3000043a0 con 0x7fc3100caf40 2023-11-08T19:29:08.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.915+0000 7fc307fff640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc2f006ae70 0x7fc2f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:08.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.915+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fc30008a3a0 con 0x7fc3100caf40 2023-11-08T19:29:08.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.915+0000 7fc30e7fc640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc2f006ae70 0x7fc2f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:08.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.916+0000 7fc30e7fc640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc2f006ae70 0x7fc2f006d330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8036000 tx=0x7fc2f803d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:08.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:08.920+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc300054ee0 con 0x7fc3100caf40 2023-11-08T19:29:09.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.153+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3100af760 con 0x7fc3100caf40 2023-11-08T19:29:09.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.154+0000 7fc307fff640 1 -- 172.21.15.119:0/3179152290 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fc30001d090 con 0x7fc3100caf40 2023-11-08T19:29:09.155 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:09.156 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":55,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:08.674421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:creating","state_seq":1,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:09.156 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 55 2023-11-08T19:29:09.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc2f006ae70 msgr2=0x7fc2f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc2f006ae70 0x7fc2f006d330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8036000 tx=0x7fc2f803d040 comp rx=0 tx=0).stop 2023-11-08T19:29:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 msgr2=0x7fc31015b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc31015b8a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fc30000cd70 tx=0x7fc30000a610 comp rx=0 tx=0).stop 2023-11-08T19:29:09.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 shutdown_connections 2023-11-08T19:29:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc2f006ae70 0x7fc2f006d330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc31015b2d0 0x7fc310157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3100d18d0 0x7fc310156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 --2- 172.21.15.119:0/3179152290 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc3100caf40 0x7fc31015b8a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:09.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.156+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 >> 172.21.15.119:0/3179152290 conn(0x7fc3100d0db0 msgr2=0x7fc3100cd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.157+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 shutdown_connections 2023-11-08T19:29:09.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:09.157+0000 7fc315983640 1 -- 172.21.15.119:0/3179152290 wait complete. 2023-11-08T19:29:09.579 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:29:08.674421+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7068, 'mds_1': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7065': {'gid': 7065, 'name': 'd', 'rank': 1, 'incarnation': 55, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.188:6834/4246462419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4246462419}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4246462419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7068': {'gid': 7068, 'name': 'b', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/168553637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 168553637}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 168553637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:29:09.580 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7065:up:creating 2023-11-08T19:29:10.581 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:10.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.804+0000 7fada2afc640 1 Processor -- start 2023-11-08T19:29:10.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.804+0000 7fada2afc640 1 -- start start 2023-11-08T19:29:10.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.806+0000 7fada2afc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 0x7fad9c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:10.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.806+0000 7fada2afc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c06a120 0x7fad9c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:10.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.806+0000 7fada2afc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad9c06aa90 con 0x7fad9c1447a0 2023-11-08T19:29:10.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.806+0000 7fada2afc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad9c15f930 con 0x7fad9c15f4c0 2023-11-08T19:29:10.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.806+0000 7fada2afc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0c3a50 con 0x7fad9c06a120 2023-11-08T19:29:10.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.807+0000 7fad9bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 0x7fad9c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:10.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.807+0000 7fada1099640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c06a120 0x7fad9c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:10.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.807+0000 7fad9bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 0x7fad9c06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:33540/0 (socket says 172.21.15.119:33540) 2023-11-08T19:29:10.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.807+0000 7fada0898640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fad9c1447a0 0x7fad9c0cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:56838/0 (socket says 172.21.15.119:56838) 2023-11-08T19:29:10.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.807+0000 7fad9bfff640 1 -- 172.21.15.119:0/1622249218 learned_addr learned my addr 172.21.15.119:0/1622249218 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:10.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.808+0000 7fada1099640 1 -- 172.21.15.119:0/1622249218 >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 msgr2=0x7fad9c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:10.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.808+0000 7fada1099640 1 --2- 172.21.15.119:0/1622249218 >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 0x7fad9c06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:10.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.808+0000 7fada1099640 1 -- 172.21.15.119:0/1622249218 >> v1:172.21.15.188:6789/0 conn(0x7fad9c1447a0 legacy=0x7fad9c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.808+0000 7fada1099640 1 -- 172.21.15.119:0/1622249218 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad9c144b80 con 0x7fad9c06a120 2023-11-08T19:29:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.808+0000 7fad9b7fe640 1 -- 172.21.15.119:0/1622249218 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175015648 0 0) 0x7fad9c06aa90 con 0x7fad9c1447a0 2023-11-08T19:29:10.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.808+0000 7fad9bfff640 1 --2- 172.21.15.119:0/1622249218 >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 0x7fad9c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:29:10.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.809+0000 7fada1099640 1 --2- 172.21.15.119:0/1622249218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c06a120 0x7fad9c06a550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad94009d90 tx=0x7fad9402beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1ee233a2b183c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:10.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.810+0000 7fad9b7fe640 1 -- 172.21.15.119:0/1622249218 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9402cc60 con 0x7fad9c06a120 2023-11-08T19:29:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.810+0000 7fad9b7fe640 1 -- 172.21.15.119:0/1622249218 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9402cdc0 con 0x7fad9c06a120 2023-11-08T19:29:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.810+0000 7fad9b7fe640 1 -- 172.21.15.119:0/1622249218 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94034660 con 0x7fad9c06a120 2023-11-08T19:29:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 -- 172.21.15.119:0/1622249218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c06a120 msgr2=0x7fad9c06a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 --2- 172.21.15.119:0/1622249218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c06a120 0x7fad9c06a550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fad94009d90 tx=0x7fad9402beb0 comp rx=0 tx=0).stop 2023-11-08T19:29:10.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 -- 172.21.15.119:0/1622249218 shutdown_connections 2023-11-08T19:29:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 --2- 172.21.15.119:0/1622249218 >> v2:172.21.15.188:3300/0 conn(0x7fad9c15f4c0 0x7fad9c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 --2- 172.21.15.119:0/1622249218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c06a120 0x7fad9c06a550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 -- 172.21.15.119:0/1622249218 >> 172.21.15.119:0/1622249218 conn(0x7fad9c0cb610 msgr2=0x7fad9c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 -- 172.21.15.119:0/1622249218 shutdown_connections 2023-11-08T19:29:10.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.811+0000 7fada2afc640 1 -- 172.21.15.119:0/1622249218 wait complete. 2023-11-08T19:29:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.813+0000 7fada2afc640 1 Processor -- start 2023-11-08T19:29:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.813+0000 7fada2afc640 1 -- start start 2023-11-08T19:29:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada2afc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad9c06a120 0x7fad9c13c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada2afc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 0x7fad9c13cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada2afc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c15f4c0 0x7fad9c13d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada2afc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0b1c10 con 0x7fad9c1447a0 2023-11-08T19:29:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada2afc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0b1ef0 con 0x7fad9c06a120 2023-11-08T19:29:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada2afc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad9c0b21d0 con 0x7fad9c15f4c0 2023-11-08T19:29:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 0x7fad9c13cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:10.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 0x7fad9c13cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52652/0 (socket says 172.21.15.119:52652) 2023-11-08T19:29:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 -- 172.21.15.119:0/1532124076 learned_addr learned my addr 172.21.15.119:0/1532124076 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada1099640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c15f4c0 0x7fad9c13d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fada0898640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad9c06a120 0x7fad9c13c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 -- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c15f4c0 msgr2=0x7fad9c13d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:10.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c15f4c0 0x7fad9c13d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 -- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad9c06a120 msgr2=0x7fad9c13c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.814+0000 7fad9bfff640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad9c06a120 0x7fad9c13c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fad9bfff640 1 -- 172.21.15.119:0/1532124076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad880094d0 con 0x7fad9c1447a0 2023-11-08T19:29:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fada1099640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c15f4c0 0x7fad9c13d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fad9bfff640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 0x7fad9c13cf90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fad8800e9d0 tx=0x7fad8800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad8800ca60 con 0x7fad9c1447a0 2023-11-08T19:29:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad8800cbc0 con 0x7fad9c1447a0 2023-11-08T19:29:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad88013470 con 0x7fad9c1447a0 2023-11-08T19:29:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad94009a70 con 0x7fad9c1447a0 2023-11-08T19:29:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.815+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad9c0b2790 con 0x7fad9c1447a0 2023-11-08T19:29:10.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.816+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad9c074430 con 0x7fad9c1447a0 2023-11-08T19:29:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.817+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad880135d0 con 0x7fad9c1447a0 2023-11-08T19:29:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.818+0000 7fad997fa640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad8006aea0 0x7fad8006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.818+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fad8808a1e0 con 0x7fad9c1447a0 2023-11-08T19:29:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.819+0000 7fada0898640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad8006aea0 0x7fad8006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.819+0000 7fada0898640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad8006aea0 0x7fad8006d360 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fad90000f60 tx=0x7fad90007b70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:10.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:10.823+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad88054d20 con 0x7fad9c1447a0 2023-11-08T19:29:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.052+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad9c06caa0 con 0x7fad9c1447a0 2023-11-08T19:29:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.054+0000 7fad997fa640 1 -- 172.21.15.119:0/1532124076 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4938 (secure 0 0 0) 0x7fad88058340 con 0x7fad9c1447a0 2023-11-08T19:29:11.056 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:11.056 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:08.704975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 56 2023-11-08T19:29:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad8006aea0 msgr2=0x7fad8006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad8006aea0 0x7fad8006d360 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fad90000f60 tx=0x7fad90007b70 comp rx=0 tx=0).stop 2023-11-08T19:29:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 msgr2=0x7fad9c13cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 0x7fad9c13cf90 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fad8800e9d0 tx=0x7fad8800ee90 comp rx=0 tx=0).stop 2023-11-08T19:29:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 shutdown_connections 2023-11-08T19:29:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fad8006aea0 0x7fad8006d360 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fad9c15f4c0 0x7fad9c13d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fad9c1447a0 0x7fad9c13cf90 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 --2- 172.21.15.119:0/1532124076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fad9c06a120 0x7fad9c13c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 >> 172.21.15.119:0/1532124076 conn(0x7fad9c0cb610 msgr2=0x7fad9c0c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.056+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 shutdown_connections 2023-11-08T19:29:11.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.057+0000 7fada2afc640 1 -- 172.21.15.119:0/1532124076 wait complete. 2023-11-08T19:29:11.419 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:29:11.418+0000 7f5f9fcb1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:11.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:29:08.704975+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7068, 'mds_1': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7065': {'gid': 7065, 'name': 'd', 'rank': 1, 'incarnation': 55, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6834/4246462419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4246462419}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4246462419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7068': {'gid': 7068, 'name': 'b', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6835/168553637', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 168553637}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 168553637}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:29:11.481 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:29:11.482 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:29:11.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.707+0000 7fdacb1b6640 1 Processor -- start 2023-11-08T19:29:11.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.708+0000 7fdacb1b6640 1 -- start start 2023-11-08T19:29:11.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.709+0000 7fdacb1b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 0x7fdac406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.709+0000 7fdacb1b6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdac40c9150 0x7fdac40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.709+0000 7fdacb1b6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac406d4b0 con 0x7fdac40ccc10 2023-11-08T19:29:11.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.709+0000 7fdacb1b6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac41619b0 con 0x7fdac40c9150 2023-11-08T19:29:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdacb1b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40cd0a0 con 0x7fdac406a970 2023-11-08T19:29:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdac9753640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fdac40ccc10 0x7fdac41653f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60846/0 (socket says 172.21.15.119:60846) 2023-11-08T19:29:11.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdac9753640 1 -- 172.21.15.119:0/1548188785 learned_addr learned my addr 172.21.15.119:0/1548188785 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 0x7fdac406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdac3fff640 1 --2- 172.21.15.119:0/1548188785 >> v2:172.21.15.188:3300/0 conn(0x7fdac40c9150 0x7fdac40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1756825029 0 0) 0x7fdabc001110 con 0x7fdac40ccc10 2023-11-08T19:29:11.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.710+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab0003520 con 0x7fdac40ccc10 2023-11-08T19:29:11.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630805395 0 0) 0x7fdab0003520 con 0x7fdac40ccc10 2023-11-08T19:29:11.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 msgr2=0x7fdac406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 0x7fdac406cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 >> v2:172.21.15.188:3300/0 conn(0x7fdac40c9150 msgr2=0x7fdac40cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 --2- 172.21.15.119:0/1548188785 >> v2:172.21.15.188:3300/0 conn(0x7fdac40c9150 0x7fdac40cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac4147b80 con 0x7fdac40ccc10 2023-11-08T19:29:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fdabc001800 con 0x7fdac40ccc10 2023-11-08T19:29:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 >> v1:172.21.15.119:6789/0 conn(0x7fdac40ccc10 legacy=0x7fdac41653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab0003970 0x7fdab0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 0x7fdab00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab0004d00 0x7fdab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac40cd0a0 con 0x7fdab0003970 2023-11-08T19:29:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab000b980 con 0x7fdab0004d00 2023-11-08T19:29:11.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdab000baf0 con 0x7fdac406a970 2023-11-08T19:29:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab0003970 0x7fdab0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.711+0000 7fdac3fff640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 0x7fdab00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.712+0000 7fdac9753640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab0004d00 0x7fdab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.712+0000 7fdac8f52640 1 -- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 msgr2=0x7fdab00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.712+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdac406a970 0x7fdab00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.712+0000 7fdac8f52640 1 -- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab0004d00 msgr2=0x7fdab0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.712+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdab0004d00 0x7fdab0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.712+0000 7fdac8f52640 1 -- 172.21.15.119:0/1548188785 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdab0003520 con 0x7fdab0003970 2023-11-08T19:29:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab0003970 0x7fdab0003d90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009fc0 tx=0x7fdab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e07d939926d872a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab402cc20 con 0x7fdab0003970 2023-11-08T19:29:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab400f040 con 0x7fdab0003970 2023-11-08T19:29:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdac37fe640 1 -- 172.21.15.119:0/1548188785 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4034440 con 0x7fdab0003970 2023-11-08T19:29:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdacb1b6640 1 -- 172.21.15.119:0/1548188785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab0003970 msgr2=0x7fdab0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/1548188785 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdab0003970 0x7fdab0003d90 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fdab4009fc0 tx=0x7fdab00057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.713+0000 7fdac9753640 1 -- 172.21.15.119:0/1548188785 reap_dead start 2023-11-08T19:29:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.714+0000 7fdacb1b6640 1 -- 172.21.15.119:0/1548188785 shutdown_connections 2023-11-08T19:29:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.714+0000 7fdacb1b6640 1 -- 172.21.15.119:0/1548188785 >> 172.21.15.119:0/1548188785 conn(0x7fdac40cb610 msgr2=0x7fdac40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.714+0000 7fdacb1b6640 1 -- 172.21.15.119:0/1548188785 shutdown_connections 2023-11-08T19:29:11.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.714+0000 7fdacb1b6640 1 -- 172.21.15.119:0/1548188785 wait complete. 2023-11-08T19:29:11.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.715+0000 7fdacb1b6640 1 Processor -- start 2023-11-08T19:29:11.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 -- start start 2023-11-08T19:29:11.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdab0003970 0x7fdac40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac40c6110 0x7fdac41f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 0x7fdac41f7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4149330 con 0x7fdac41f7880 2023-11-08T19:29:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4149150 con 0x7fdac40c6110 2023-11-08T19:29:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.716+0000 7fdacb1b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac41f8240 con 0x7fdab0003970 2023-11-08T19:29:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 0x7fdac41f7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 0x7fdac41f7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52662/0 (socket says 172.21.15.119:52662) 2023-11-08T19:29:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 -- 172.21.15.119:0/424023625 learned_addr learned my addr 172.21.15.119:0/424023625 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac8f52640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdab0003970 0x7fdac40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 -- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdab0003970 msgr2=0x7fdac40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdab0003970 0x7fdac40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 -- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac40c6110 msgr2=0x7fdac41f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac3fff640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac40c6110 0x7fdac41f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac40c6110 0x7fdac41f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 -- 172.21.15.119:0/424023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac40c6690 con 0x7fdac41f7880 2023-11-08T19:29:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.717+0000 7fdac9753640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 0x7fdac41f7d00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdac40ccff0 tx=0x7fdabc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.718+0000 7fdac17fa640 1 -- 172.21.15.119:0/424023625 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00c5b0 con 0x7fdac41f7880 2023-11-08T19:29:11.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.718+0000 7fdac17fa640 1 -- 172.21.15.119:0/424023625 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc00c710 con 0x7fdac41f7880 2023-11-08T19:29:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.718+0000 7fdac17fa640 1 -- 172.21.15.119:0/424023625 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc00cac0 con 0x7fdac41f7880 2023-11-08T19:29:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.718+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac40c6970 con 0x7fdac41f7880 2023-11-08T19:29:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.718+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac40c6f30 con 0x7fdac41f7880 2023-11-08T19:29:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.721+0000 7fdac17fa640 1 -- 172.21.15.119:0/424023625 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdabc00cc20 con 0x7fdac41f7880 2023-11-08T19:29:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.722+0000 7fdac17fa640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdaa406ae70 0x7fdaa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.722+0000 7fdac17fa640 1 -- 172.21.15.119:0/424023625 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fdabc089160 con 0x7fdac41f7880 2023-11-08T19:29:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.722+0000 7fdac8f52640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdaa406ae70 0x7fdaa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.722+0000 7fdacde23740 1 Processor -- start 2023-11-08T19:29:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.722+0000 7fdacde23740 1 -- start start 2023-11-08T19:29:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdac8f52640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdaa406ae70 0x7fdaa406d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdab4036000 tx=0x7fdab402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d96722c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdacde23740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 0x563d96723190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdacde23740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d966657e0 con 0x563d966f3300 2023-11-08T19:29:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d96662ed0 con 0x563d9671fec0 2023-11-08T19:29:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.723+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d9665e7c0 con 0x563d967202a0 2023-11-08T19:29:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 0x563d96723190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 0x563d96723190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52666/0 (socket says 172.21.15.119:52666) 2023-11-08T19:29:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 -- 172.21.15.119:0/49142530 learned_addr learned my addr 172.21.15.119:0/49142530 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 -- 172.21.15.119:0/49142530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 msgr2=0x563d96722c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d96722c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 -- 172.21.15.119:0/49142530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 msgr2=0x563d966f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 -- 172.21.15.119:0/49142530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d96631b10 con 0x563d966f3300 2023-11-08T19:29:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdac9753640 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 0x563d96723190 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdabc05f6a0 tx=0x7fdabc057540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f2d3d54297d87c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdaba7fc640 1 -- 172.21.15.119:0/49142530 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc05ba20 con 0x563d966f3300 2023-11-08T19:29:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdaba7fc640 1 -- 172.21.15.119:0/49142530 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc05fbd0 con 0x563d966f3300 2023-11-08T19:29:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.724+0000 7fdaba7fc640 1 -- 172.21.15.119:0/49142530 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc055c70 con 0x563d966f3300 2023-11-08T19:29:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 -- 172.21.15.119:0/49142530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 msgr2=0x563d96723190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 0x563d96723190 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fdabc05f6a0 tx=0x7fdabc057540 comp rx=0 tx=0).stop 2023-11-08T19:29:11.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 -- 172.21.15.119:0/49142530 shutdown_connections 2023-11-08T19:29:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d966f3300 0x563d96723190 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d96722c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 --2- 172.21.15.119:0/49142530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 -- 172.21.15.119:0/49142530 >> 172.21.15.119:0/49142530 conn(0x563d966f50d0 msgr2=0x563d9673dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 -- 172.21.15.119:0/49142530 shutdown_connections 2023-11-08T19:29:11.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.725+0000 7fdacde23740 1 -- 172.21.15.119:0/49142530 wait complete. 2023-11-08T19:29:11.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.726+0000 7fdacde23740 1 -- start start 2023-11-08T19:29:11.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.726+0000 7fdacde23740 1 Processor -- start 2023-11-08T19:29:11.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d9670dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96725940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d96738100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d965fbc40 con 0x563d96725e80 2023-11-08T19:29:11.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d965fb8e0 con 0x563d967202a0 2023-11-08T19:29:11.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d963c1f20 con 0x563d9671fec0 2023-11-08T19:29:11.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdacde23740 10 client.0 _ll_register_callbacks cb 0x563d9661dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:11.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d96738100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d96738100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52676/0 (socket says 172.21.15.119:52676) 2023-11-08T19:29:11.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.727+0000 7fdac9753640 1 -- 172.21.15.119:0/1461356522 learned_addr learned my addr 172.21.15.119:0/1461356522 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:11.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d9670dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac3fff640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96725940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac9753640 1 -- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 msgr2=0x563d9670dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac9753640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d9670dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac9753640 1 -- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 msgr2=0x563d96725940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac9753640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96725940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac9753640 1 -- 172.21.15.119:0/1461356522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac40c6f30 con 0x563d96725e80 2023-11-08T19:29:11.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdac9753640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d96738100 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdabc05f6a0 tx=0x7fdabc007070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b2104691d0807f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdaa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:11.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1461356522 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc053fe0 con 0x563d96725e80 2023-11-08T19:29:11.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1461356522 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fdabc054140 con 0x563d96725e80 2023-11-08T19:29:11.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d96632c80 con 0x563d96725e80 2023-11-08T19:29:11.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1461356522 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdabc05f740 con 0x563d96725e80 2023-11-08T19:29:11.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdacde23740 10 client.7299 fetch_fsmap learned FSMap version 56 2023-11-08T19:29:11.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.728+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x563d9665ce40 con 0x563d96725e80 2023-11-08T19:29:11.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1461356522 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563d96632c80 con 0x563d96725e80 2023-11-08T19:29:11.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1461356522 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 56) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fdabc07c050 con 0x563d96725e80 2023-11-08T19:29:11.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdacde23740 10 client.7299 fetch_fsmap finished waiting for FSMap version 56 2023-11-08T19:29:11.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdacde23740 10 client.7299 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:29:11.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdacde23740 20 client.7299 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:11.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdacde23740 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d9670fc80 0x563d967103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdacde23740 4 client.7299 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:11.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.729+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d966577a0 con 0x563d9670fc80 2023-11-08T19:29:11.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.730+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d9670fc80 0x563d967103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.731+0000 7fdaa37fe640 10 client.7299 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:11.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.731+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d9670fc80 0x563d967103b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffd863f1afb59286 server_cookie=70b8095141f921ed in_seq=0 out_seq=0 2023-11-08T19:29:11.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1461356522 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d966577a0 con 0x563d9670fc80 2023-11-08T19:29:11.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdaa37fe640 10 client.7299 handle_command_reply: tid=0 2023-11-08T19:29:11.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 client.7299 shutdown 2023-11-08T19:29:11.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 msgr2=0x563d96738100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d96738100 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fdabc05f6a0 tx=0x7fdabc007070 comp rx=0 tx=0).stop 2023-11-08T19:29:11.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 shutdown_connections 2023-11-08T19:29:11.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d96738100 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d9670fc80 0x563d967103b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96725940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 --2- 172.21.15.119:0/1461356522 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d9670dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdaa37fe640 0 client.7299 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:11.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 >> 172.21.15.119:0/1461356522 conn(0x563d966f3300 msgr2=0x563d96722c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 shutdown_connections 2023-11-08T19:29:11.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.733+0000 7fdacde23740 1 -- 172.21.15.119:0/1461356522 wait complete. 2023-11-08T19:29:11.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.734+0000 7fdacde23740 20 client.7299 trim_cache size 0 max 16384 2023-11-08T19:29:11.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.750+0000 7fdacde23740 1 Processor -- start 2023-11-08T19:29:11.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.750+0000 7fdacde23740 1 -- start start 2023-11-08T19:29:11.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d966f2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdacde23740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdacde23740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d966f4830 con 0x563d96725e80 2023-11-08T19:29:11.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d9670db10 con 0x563d9671fec0 2023-11-08T19:29:11.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d966f31c0 con 0x563d967202a0 2023-11-08T19:29:11.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.751+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52688/0 (socket says 172.21.15.119:52688) 2023-11-08T19:29:11.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 -- 172.21.15.119:0/148495600 learned_addr learned my addr 172.21.15.119:0/148495600 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:11.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac8f52640 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac3fff640 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d966f2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 -- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 msgr2=0x563d966f2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d966f2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 -- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 msgr2=0x563d966f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 -- 172.21.15.119:0/148495600 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d965fab90 con 0x563d96725e80 2023-11-08T19:29:11.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac3fff640 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d966f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:11.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdac9753640 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdabc055c20 tx=0x7fdabc081000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff86f51beca47df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdabaffd640 1 -- 172.21.15.119:0/148495600 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc057410 con 0x563d96725e80 2023-11-08T19:29:11.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdabaffd640 1 -- 172.21.15.119:0/148495600 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc05f740 con 0x563d96725e80 2023-11-08T19:29:11.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.752+0000 7fdabaffd640 1 -- 172.21.15.119:0/148495600 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc056830 con 0x563d96725e80 2023-11-08T19:29:11.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 -- 172.21.15.119:0/148495600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 msgr2=0x563d967404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fdabc055c20 tx=0x7fdabc081000 comp rx=0 tx=0).stop 2023-11-08T19:29:11.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 -- 172.21.15.119:0/148495600 shutdown_connections 2023-11-08T19:29:11.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d967202a0 0x563d966f2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 --2- 172.21.15.119:0/148495600 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d9671fec0 0x563d966f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 -- 172.21.15.119:0/148495600 >> 172.21.15.119:0/148495600 conn(0x563d966f3300 msgr2=0x563d96722c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 -- 172.21.15.119:0/148495600 shutdown_connections 2023-11-08T19:29:11.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.753+0000 7fdacde23740 1 -- 172.21.15.119:0/148495600 wait complete. 2023-11-08T19:29:11.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.754+0000 7fdacde23740 1 -- start start 2023-11-08T19:29:11.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.754+0000 7fdacde23740 1 Processor -- start 2023-11-08T19:29:11.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d967aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96733ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d967ab2f0 con 0x563d96725e80 2023-11-08T19:29:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d967ab490 con 0x563d967202a0 2023-11-08T19:29:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d966f23e0 con 0x563d9671fec0 2023-11-08T19:29:11.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdacde23740 10 client.0 _ll_register_callbacks cb 0x7fdabc06e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdac9753640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52692/0 (socket says 172.21.15.119:52692) 2023-11-08T19:29:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.755+0000 7fdac9753640 1 -- 172.21.15.119:0/1990587329 learned_addr learned my addr 172.21.15.119:0/1990587329 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d967aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac3fff640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96733ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac9753640 1 -- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 msgr2=0x563d967aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac9753640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d967aadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac9753640 1 -- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 msgr2=0x563d96733ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac9753640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96733ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac9753640 1 -- 172.21.15.119:0/1990587329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdabc079030 con 0x563d96725e80 2023-11-08T19:29:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdac9753640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdabc053dd0 tx=0x7fdabc089050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8772005e7d5b1da server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdaa37fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1990587329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc057410 con 0x563d96725e80 2023-11-08T19:29:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1990587329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fdabc0538d0 con 0x563d96725e80 2023-11-08T19:29:11.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdabc05fbd0 con 0x563d96725e80 2023-11-08T19:29:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1990587329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdabc056830 con 0x563d96725e80 2023-11-08T19:29:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdacde23740 10 client.7305 fetch_fsmap learned FSMap version 56 2023-11-08T19:29:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.756+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x563d96620180 con 0x563d96725e80 2023-11-08T19:29:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1990587329 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc05fbd0 con 0x563d96725e80 2023-11-08T19:29:11.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1990587329 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 56) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fdabc07c070 con 0x563d96725e80 2023-11-08T19:29:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdacde23740 10 client.7305 fetch_fsmap finished waiting for FSMap version 56 2023-11-08T19:29:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdacde23740 10 client.7305 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:29:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdacde23740 20 client.7305 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:11.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdacde23740 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d967a7320 0x563d96740a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdacde23740 4 client.7305 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:11.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d966574c0 con 0x563d967a7320 2023-11-08T19:29:11.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.757+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d967a7320 0x563d96740a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.759+0000 7fdac8f52640 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d967a7320 0x563d96740a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d4f2a185b711e5 server_cookie=f27c1e801b801bdd in_seq=0 out_seq=0 2023-11-08T19:29:11.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.759+0000 7fdaa37fe640 10 client.7305 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:11.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.759+0000 7fdaa37fe640 1 -- 172.21.15.119:0/1990587329 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x563d966574c0 con 0x563d967a7320 2023-11-08T19:29:11.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdaa37fe640 10 client.7305 handle_command_reply: tid=0 2023-11-08T19:29:11.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 client.7305 shutdown 2023-11-08T19:29:11.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 msgr2=0x563d967404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fdabc053dd0 tx=0x7fdabc089050 comp rx=0 tx=0).stop 2023-11-08T19:29:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 shutdown_connections 2023-11-08T19:29:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x563d96725e80 0x563d967404c0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x563d967a7320 0x563d96740a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x563d967202a0 0x563d96733ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 --2- 172.21.15.119:0/1990587329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x563d9671fec0 0x563d967aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 >> 172.21.15.119:0/1990587329 conn(0x563d966f3300 msgr2=0x563d96725710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdaa37fe640 0 client.7305 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:11.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 shutdown_connections 2023-11-08T19:29:11.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 1 -- 172.21.15.119:0/1990587329 wait complete. 2023-11-08T19:29:11.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.760+0000 7fdacde23740 20 client.7305 trim_cache size 0 max 16384 2023-11-08T19:29:11.775 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:29:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:29:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-11-08T19:29:11.775 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7065, 2023-11-08T19:29:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:29:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:29:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:29:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 3.0804062980000002, 2023-11-08T19:29:11.776 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 56, 2023-11-08T19:29:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 96, 2023-11-08T19:29:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 96, 2023-11-08T19:29:11.777 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 26.133806558 2023-11-08T19:29:11.777 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:29:11.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdaa406ae70 msgr2=0x7fdaa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdaa406ae70 0x7fdaa406d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdab4036000 tx=0x7fdab402e000 comp rx=0 tx=0).stop 2023-11-08T19:29:11.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 msgr2=0x7fdac41f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 0x7fdac41f7d00 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fdac40ccff0 tx=0x7fdabc00eb90 comp rx=0 tx=0).stop 2023-11-08T19:29:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 shutdown_connections 2023-11-08T19:29:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdaa406ae70 0x7fdaa406d330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdac41f7880 0x7fdac41f7d00 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdac40c6110 0x7fdac41f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 --2- 172.21.15.119:0/424023625 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdab0003970 0x7fdac40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 >> 172.21.15.119:0/424023625 conn(0x7fdac40cb610 msgr2=0x7fdac406f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 shutdown_connections 2023-11-08T19:29:11.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.761+0000 7fdacb1b6640 1 -- 172.21.15.119:0/424023625 wait complete. 2023-11-08T19:29:11.781 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:29:11.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.986+0000 7f88f4900640 1 Processor -- start 2023-11-08T19:29:11.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.987+0000 7f88f4900640 1 -- start start 2023-11-08T19:29:11.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.988+0000 7f88f4900640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f88f015d410 0x7f88f00ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.988+0000 7f88f4900640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f006b570 con 0x7f88f015e5f0 2023-11-08T19:29:11.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.988+0000 7f88f4900640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88f006b850 con 0x7f88f015df00 2023-11-08T19:29:11.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.988+0000 7f88f4900640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88f006c1f0 con 0x7f88f015d410 2023-11-08T19:29:11.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.988+0000 7f88edd9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f88f015e5f0 0x7f88f015d7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60850/0 (socket says 172.21.15.119:60850) 2023-11-08T19:29:11.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.988+0000 7f88edd9b640 1 -- 172.21.15.119:0/3884713881 learned_addr learned my addr 172.21.15.119:0/3884713881 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:11.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ee59c640 1 --2- 172.21.15.119:0/3884713881 >> v2:172.21.15.188:3300/0 conn(0x7f88f015d410 0x7f88f00ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105093969 0 0) 0x7f88f006b570 con 0x7f88f015e5f0 2023-11-08T19:29:11.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d0003520 con 0x7f88f015e5f0 2023-11-08T19:29:11.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1148539172 0 0) 0x7f88d0003520 con 0x7f88f015e5f0 2023-11-08T19:29:11.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 >> v2:172.21.15.188:3300/0 conn(0x7f88f015d410 msgr2=0x7f88f00ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 --2- 172.21.15.119:0/3884713881 >> v2:172.21.15.188:3300/0 conn(0x7f88f015d410 0x7f88f00ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 >> v1:172.21.15.188:6789/0 conn(0x7f88f015df00 legacy=0x7f88f006dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f006bb30 con 0x7f88f015e5f0 2023-11-08T19:29:11.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f88d80019f0 con 0x7f88f015e5f0 2023-11-08T19:29:11.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 >> v1:172.21.15.119:6789/0 conn(0x7f88f015e5f0 legacy=0x7f88f015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:11.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d0003970 0x7f88d0003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00047c0 0x7f88d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d00052a0 0x7f88d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:11.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f006c1f0 con 0x7f88d00052a0 2023-11-08T19:29:11.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d000c150 con 0x7f88d0003970 2023-11-08T19:29:11.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.989+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d000c310 con 0x7f88d00047c0 2023-11-08T19:29:11.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88eed9d640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d00052a0 0x7f88d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88edd9b640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00047c0 0x7f88d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88ee59c640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d0003970 0x7f88d0003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:11.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88eed9d640 1 -- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00047c0 msgr2=0x7f88d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88eed9d640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00047c0 0x7f88d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88eed9d640 1 -- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d0003970 msgr2=0x7f88d0003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:11.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88eed9d640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88d0003970 0x7f88d0003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:11.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.990+0000 7f88eed9d640 1 -- 172.21.15.119:0/3884713881 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0003520 con 0x7f88d00052a0 2023-11-08T19:29:12.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.991+0000 7f88eed9d640 1 -- 172.21.15.119:0/3884713881 reap_dead start 2023-11-08T19:29:12.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.992+0000 7f88eed9d640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d00052a0 0x7f88d000b980 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003d80 tx=0x7f88d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f349132abae404 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.992+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8007220 con 0x7f88d00052a0 2023-11-08T19:29:12.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.992+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e802e050 con 0x7f88d00052a0 2023-11-08T19:29:12.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.992+0000 7f88ed59a640 1 -- 172.21.15.119:0/3884713881 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e80075e0 con 0x7f88d00052a0 2023-11-08T19:29:12.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 -- 172.21.15.119:0/3884713881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d00052a0 msgr2=0x7f88d000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d00052a0 0x7f88d000b980 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003d80 tx=0x7f88d00057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 -- 172.21.15.119:0/3884713881 shutdown_connections 2023-11-08T19:29:12.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 --2- 172.21.15.119:0/3884713881 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88d00052a0 0x7f88d000b980 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 -- 172.21.15.119:0/3884713881 >> 172.21.15.119:0/3884713881 conn(0x7f88f00d0db0 msgr2=0x7f88f00d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 -- 172.21.15.119:0/3884713881 shutdown_connections 2023-11-08T19:29:12.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.993+0000 7f88f4900640 1 -- 172.21.15.119:0/3884713881 wait complete. 2023-11-08T19:29:12.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.995+0000 7f88f4900640 1 Processor -- start 2023-11-08T19:29:12.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.995+0000 7f88f4900640 1 -- start start 2023-11-08T19:29:12.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88f4900640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00052a0 0x7f88f0157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88f4900640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 0x7f88f00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88f4900640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f009af10 0x7f88f00cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88f4900640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f01f6670 con 0x7f88f009aa90 2023-11-08T19:29:12.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88f4900640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f01f59f0 con 0x7f88f009af10 2023-11-08T19:29:12.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88f4900640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88f01f5cd0 con 0x7f88d00052a0 2023-11-08T19:29:12.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 0x7f88f00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 0x7f88f00cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52722/0 (socket says 172.21.15.119:52722) 2023-11-08T19:29:12.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 -- 172.21.15.119:0/3577926120 learned_addr learned my addr 172.21.15.119:0/3577926120 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:12.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 -- 172.21.15.119:0/3577926120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00052a0 msgr2=0x7f88f0157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00052a0 0x7f88f0157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 -- 172.21.15.119:0/3577926120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f009af10 msgr2=0x7f88f00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f009af10 0x7f88f00cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.996+0000 7f88edd9b640 1 -- 172.21.15.119:0/3577926120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f01f5fb0 con 0x7f88f009aa90 2023-11-08T19:29:12.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.997+0000 7f88edd9b640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 0x7f88f00cd630 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f88d8001580 tx=0x7f88d800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.997+0000 7f88e6ffd640 1 -- 172.21.15.119:0/3577926120 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800ee50 con 0x7f88f009aa90 2023-11-08T19:29:12.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.997+0000 7f88e6ffd640 1 -- 172.21.15.119:0/3577926120 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8013420 con 0x7f88f009aa90 2023-11-08T19:29:12.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.997+0000 7f88e6ffd640 1 -- 172.21.15.119:0/3577926120 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d80137d0 con 0x7f88f009aa90 2023-11-08T19:29:12.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.997+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f01f6290 con 0x7f88f009aa90 2023-11-08T19:29:12.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.997+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f0157ed0 con 0x7f88f009aa90 2023-11-08T19:29:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:11.999+0000 7f88e6ffd640 1 -- 172.21.15.119:0/3577926120 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f88d8013930 con 0x7f88f009aa90 2023-11-08T19:29:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.001+0000 7f88e6ffd640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d406aec0 0x7f88d406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.001+0000 7f88e6ffd640 1 -- 172.21.15.119:0/3577926120 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f88d808a2d0 con 0x7f88f009aa90 2023-11-08T19:29:12.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.001+0000 7f88ee59c640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d406aec0 0x7f88d406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.002+0000 7f88f756d740 1 Processor -- start 2023-11-08T19:29:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.002+0000 7f88ee59c640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d406aec0 0x7f88d406d380 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f88f006e6f0 tx=0x7f88e000d240 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.002+0000 7f88f756d740 1 -- start start 2023-11-08T19:29:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.002+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791aec0 0x55d2f78eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f791dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88f756d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 0x55d2f791e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88f756d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f78607e0 con 0x55d2f78ee300 2023-11-08T19:29:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f785ded0 con 0x55d2f791b2a0 2023-11-08T19:29:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f78597c0 con 0x55d2f791aec0 2023-11-08T19:29:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 0x55d2f791e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 0x55d2f791e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52738/0 (socket says 172.21.15.119:52738) 2023-11-08T19:29:12.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 -- 172.21.15.119:0/2319907948 learned_addr learned my addr 172.21.15.119:0/2319907948 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:12.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 -- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791aec0 msgr2=0x55d2f78eddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88ee59c640 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791aec0 0x55d2f78eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f791dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791aec0 0x55d2f78eddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 -- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 msgr2=0x55d2f791dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f791dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.003+0000 7f88eed9d640 1 -- 172.21.15.119:0/2319907948 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f782cb10 con 0x55d2f78ee300 2023-11-08T19:29:12.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.004+0000 7f88eed9d640 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 0x55d2f791e190 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003ea0 tx=0x7f88e802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4da7583d14beefb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.004+0000 7f88dffff640 1 -- 172.21.15.119:0/2319907948 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8042070 con 0x55d2f78ee300 2023-11-08T19:29:12.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.004+0000 7f88dffff640 1 -- 172.21.15.119:0/2319907948 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e803e530 con 0x55d2f78ee300 2023-11-08T19:29:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.004+0000 7f88dffff640 1 -- 172.21.15.119:0/2319907948 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e803e8e0 con 0x55d2f78ee300 2023-11-08T19:29:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 -- 172.21.15.119:0/2319907948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 msgr2=0x55d2f791e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 0x55d2f791e190 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003ea0 tx=0x7f88e802d000 comp rx=0 tx=0).stop 2023-11-08T19:29:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 -- 172.21.15.119:0/2319907948 shutdown_connections 2023-11-08T19:29:12.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f78ee300 0x55d2f791e190 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f791dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 --2- 172.21.15.119:0/2319907948 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791aec0 0x55d2f78eddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 -- 172.21.15.119:0/2319907948 >> 172.21.15.119:0/2319907948 conn(0x55d2f78f00d0 msgr2=0x55d2f7938c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 -- 172.21.15.119:0/2319907948 shutdown_connections 2023-11-08T19:29:12.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.005+0000 7f88f756d740 1 -- 172.21.15.119:0/2319907948 wait complete. 2023-11-08T19:29:12.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.006+0000 7f88f756d740 1 -- start start 2023-11-08T19:29:12.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.006+0000 7f88f756d740 1 Processor -- start 2023-11-08T19:29:12.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f790b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f790b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f790a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f77f6c40 con 0x55d2f791aec0 2023-11-08T19:29:12.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f77f68e0 con 0x55d2f791b2a0 2023-11-08T19:29:12.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f75bcf20 con 0x55d2f7908f20 2023-11-08T19:29:12.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88f756d740 10 client.0 _ll_register_callbacks cb 0x55d2f7818cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:12.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88ee59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f790b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88ee59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f790b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52748/0 (socket says 172.21.15.119:52748) 2023-11-08T19:29:12.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88ee59c640 1 -- 172.21.15.119:0/2672634513 learned_addr learned my addr 172.21.15.119:0/2672634513 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:12.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88eed9d640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f790a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.007+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f790b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88ee59c640 1 -- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 msgr2=0x55d2f790a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88ee59c640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f790a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88ee59c640 1 -- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 msgr2=0x55d2f790b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88ee59c640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f790b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88ee59c640 1 -- 172.21.15.119:0/2672634513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d0003520 con 0x55d2f791aec0 2023-11-08T19:29:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88ee59c640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f790b4b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f88e000e000 tx=0x7f88e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80631d9a54ceabc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88dd7fa640 1 -- 172.21.15.119:0/2672634513 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e000de40 con 0x55d2f791aec0 2023-11-08T19:29:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88dd7fa640 1 -- 172.21.15.119:0/2672634513 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f88e0015820 con 0x55d2f791aec0 2023-11-08T19:29:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2f782dc80 con 0x55d2f791aec0 2023-11-08T19:29:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88dd7fa640 1 -- 172.21.15.119:0/2672634513 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88e00159e0 con 0x55d2f791aec0 2023-11-08T19:29:12.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88f756d740 10 client.7317 fetch_fsmap learned FSMap version 56 2023-11-08T19:29:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.008+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d2f7857e40 con 0x55d2f791aec0 2023-11-08T19:29:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88dd7fa640 1 -- 172.21.15.119:0/2672634513 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d2f782dc80 con 0x55d2f791aec0 2023-11-08T19:29:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88dd7fa640 1 -- 172.21.15.119:0/2672634513 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 56) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f88e0015b40 con 0x55d2f791aec0 2023-11-08T19:29:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88f756d740 10 client.7317 fetch_fsmap finished waiting for FSMap version 56 2023-11-08T19:29:12.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88f756d740 10 client.7317 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88f756d740 20 client.7317 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88f756d740 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f7954140 0x55d2f7956530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88f756d740 4 client.7317 mds_command: new command op to 7068 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.009+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 --> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2f78527a0 con 0x55d2f7954140 2023-11-08T19:29:12.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.010+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f7954140 0x55d2f7956530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.010+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f7954140 0x55d2f7956530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae626e6cc6f87293 server_cookie=f463105fd607a891 in_seq=0 out_seq=0 2023-11-08T19:29:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.010+0000 7f88dd7fa640 10 client.7317 ms_handle_connect on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:12.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.012+0000 7f88dd7fa640 1 -- 172.21.15.119:0/2672634513 <== mds.0 v2:172.21.15.188:6833/168553637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d2f78527a0 con 0x55d2f7954140 2023-11-08T19:29:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.012+0000 7f88dd7fa640 10 client.7317 handle_command_reply: tid=0 2023-11-08T19:29:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 client.7317 shutdown 2023-11-08T19:29:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 msgr2=0x55d2f790b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f790b4b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f88e000e000 tx=0x7f88e0008040 comp rx=0 tx=0).stop 2023-11-08T19:29:12.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 shutdown_connections 2023-11-08T19:29:12.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f7954140 0x55d2f7956530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f790a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791b2a0 0x55d2f790b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88dd7fa640 0 client.7317 ms_handle_reset on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:12.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 --2- 172.21.15.119:0/2672634513 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f790b4b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 >> 172.21.15.119:0/2672634513 conn(0x55d2f78ee300 msgr2=0x55d2f78f1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 shutdown_connections 2023-11-08T19:29:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 1 -- 172.21.15.119:0/2672634513 wait complete. 2023-11-08T19:29:12.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.013+0000 7f88f756d740 20 client.7317 trim_cache size 0 max 16384 2023-11-08T19:29:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.030+0000 7f88f756d740 1 Processor -- start 2023-11-08T19:29:12.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.030+0000 7f88f756d740 1 -- start start 2023-11-08T19:29:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f7908f20 0x55d2f78ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88f756d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f791e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791b2a0 0x55d2f795a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88f756d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7939380 con 0x55d2f791aec0 2023-11-08T19:29:12.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7930550 con 0x55d2f7908f20 2023-11-08T19:29:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f795b1b0 con 0x55d2f791b2a0 2023-11-08T19:29:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88edd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f791e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88edd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f791e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52756/0 (socket says 172.21.15.119:52756) 2023-11-08T19:29:12.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.031+0000 7f88edd9b640 1 -- 172.21.15.119:0/2669297818 learned_addr learned my addr 172.21.15.119:0/2669297818 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:12.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88ee59c640 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f7908f20 0x55d2f78ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88edd9b640 1 -- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791b2a0 msgr2=0x55d2f795a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791b2a0 0x55d2f795a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88edd9b640 1 -- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f7908f20 msgr2=0x55d2f78ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f7908f20 0x55d2f78ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88edd9b640 1 -- 172.21.15.119:0/2669297818 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2f77f5b90 con 0x55d2f791aec0 2023-11-08T19:29:12.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88ee59c640 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f7908f20 0x55d2f78ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:12.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88edd9b640 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f791e0a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f88d800ea30 tx=0x7f88d80881e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c7611706549588 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88e4ff9640 1 -- 172.21.15.119:0/2669297818 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8060e50 con 0x55d2f791aec0 2023-11-08T19:29:12.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88e4ff9640 1 -- 172.21.15.119:0/2669297818 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d8056c90 con 0x55d2f791aec0 2023-11-08T19:29:12.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.032+0000 7f88e4ff9640 1 -- 172.21.15.119:0/2669297818 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8056ef0 con 0x55d2f791aec0 2023-11-08T19:29:12.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 -- 172.21.15.119:0/2669297818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 msgr2=0x55d2f791e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f791e0a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f88d800ea30 tx=0x7f88d80881e0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 -- 172.21.15.119:0/2669297818 shutdown_connections 2023-11-08T19:29:12.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f791b2a0 0x55d2f795a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791aec0 0x55d2f791e0a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 --2- 172.21.15.119:0/2669297818 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f7908f20 0x55d2f78ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 -- 172.21.15.119:0/2669297818 >> 172.21.15.119:0/2669297818 conn(0x55d2f78ee300 msgr2=0x55d2f790a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 -- 172.21.15.119:0/2669297818 shutdown_connections 2023-11-08T19:29:12.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.033+0000 7f88f756d740 1 -- 172.21.15.119:0/2669297818 wait complete. 2023-11-08T19:29:12.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.034+0000 7f88f756d740 1 -- start start 2023-11-08T19:29:12.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.034+0000 7f88f756d740 1 Processor -- start 2023-11-08T19:29:12.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f7999ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791aec0 0x55d2f7936800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 0x55d2f7936d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f799a020 con 0x55d2f791b2a0 2023-11-08T19:29:12.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2f7937a30 con 0x55d2f791aec0 2023-11-08T19:29:12.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d2f79079c0 con 0x55d2f7908f20 2023-11-08T19:29:12.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88f756d740 10 client.0 _ll_register_callbacks cb 0x7f88e000fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:12.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 0x55d2f7936d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88eed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 0x55d2f7936d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52770/0 (socket says 172.21.15.119:52770) 2023-11-08T19:29:12.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.035+0000 7f88eed9d640 1 -- 172.21.15.119:0/1058130963 learned_addr learned my addr 172.21.15.119:0/1058130963 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:12.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88ee59c640 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f7999ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88eed9d640 1 -- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 msgr2=0x55d2f7999ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88eed9d640 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f7999ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88eed9d640 1 -- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791aec0 msgr2=0x55d2f7936800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88eed9d640 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791aec0 0x55d2f7936800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88eed9d640 1 -- 172.21.15.119:0/1058130963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f0157ed0 con 0x55d2f791b2a0 2023-11-08T19:29:12.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88eed9d640 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 0x55d2f7936d40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003ea0 tx=0x7f88e80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e93acc8a81c972f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:12.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88dd7fa640 1 -- 172.21.15.119:0/1058130963 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e8042040 con 0x55d2f791b2a0 2023-11-08T19:29:12.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88dd7fa640 1 -- 172.21.15.119:0/1058130963 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f88e804cca0 con 0x55d2f791b2a0 2023-11-08T19:29:12.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88e803e530 con 0x55d2f791b2a0 2023-11-08T19:29:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88dd7fa640 1 -- 172.21.15.119:0/1058130963 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88e804ce60 con 0x55d2f791b2a0 2023-11-08T19:29:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88f756d740 10 client.7323 fetch_fsmap learned FSMap version 56 2023-11-08T19:29:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.036+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55d2f781b180 con 0x55d2f791b2a0 2023-11-08T19:29:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88dd7fa640 1 -- 172.21.15.119:0/1058130963 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e803e530 con 0x55d2f791b2a0 2023-11-08T19:29:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88dd7fa640 1 -- 172.21.15.119:0/1058130963 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 56) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f88e803a040 con 0x55d2f791b2a0 2023-11-08T19:29:12.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88f756d740 10 client.7323 fetch_fsmap finished waiting for FSMap version 56 2023-11-08T19:29:12.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88f756d740 10 client.7323 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:29:12.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88f756d740 20 client.7323 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:12.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88f756d740 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f791d820 0x55d2f791dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88f756d740 4 client.7323 mds_command: new command op to 7068 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:12.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.037+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 --> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d2f78524c0 con 0x55d2f791d820 2023-11-08T19:29:12.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.038+0000 7f88ee59c640 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f791d820 0x55d2f791dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.038+0000 7f88ee59c640 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f791d820 0x55d2f791dc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=520ca63630b7085e server_cookie=f83493f426ecb2e9 in_seq=0 out_seq=0 2023-11-08T19:29:12.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.038+0000 7f88dd7fa640 10 client.7323 ms_handle_connect on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:12.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.039+0000 7f88dd7fa640 1 -- 172.21.15.119:0/1058130963 <== mds.0 v2:172.21.15.188:6833/168553637 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55d2f78524c0 con 0x55d2f791d820 2023-11-08T19:29:12.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.039+0000 7f88dd7fa640 10 client.7323 handle_command_reply: tid=0 2023-11-08T19:29:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.039+0000 7f88f756d740 1 client.7323 shutdown 2023-11-08T19:29:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.039+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 msgr2=0x55d2f7936d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.039+0000 7f88f756d740 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 0x55d2f7936d40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f88e8003ea0 tx=0x7f88e80036f0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 shutdown_connections 2023-11-08T19:29:12.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:6833/168553637,v1:172.21.15.188:6835/168553637] conn(0x55d2f791d820 0x55d2f791dc00 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55d2f791b2a0 0x55d2f7936d40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55d2f791aec0 0x55d2f7936800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 --2- 172.21.15.119:0/1058130963 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55d2f7908f20 0x55d2f7999ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88dd7fa640 0 client.7323 ms_handle_reset on v2:172.21.15.188:6833/168553637 2023-11-08T19:29:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 >> 172.21.15.119:0/1058130963 conn(0x55d2f78ee300 msgr2=0x55d2f78e9560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 shutdown_connections 2023-11-08T19:29:12.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 1 -- 172.21.15.119:0/1058130963 wait complete. 2023-11-08T19:29:12.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f756d740 20 client.7323 trim_cache size 0 max 16384 2023-11-08T19:29:12.053 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:29:12.054 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:29:12.054 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:29:12.054 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7068, 2023-11-08T19:29:12.054 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:29:12.054 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:29:12.055 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:29:12.055 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 18.990221835, 2023-11-08T19:29:12.055 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 56, 2023-11-08T19:29:12.055 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 96, 2023-11-08T19:29:12.055 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 96, 2023-11-08T19:29:12.056 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 26.413084168000001 2023-11-08T19:29:12.056 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:29:12.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d406aec0 msgr2=0x7f88d406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f4900640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d406aec0 0x7f88d406d380 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f88f006e6f0 tx=0x7f88e000d240 comp rx=0 tx=0).stop 2023-11-08T19:29:12.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 msgr2=0x7f88f00cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.040+0000 7f88f4900640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 0x7f88f00cd630 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f88d8001580 tx=0x7f88d800e530 comp rx=0 tx=0).stop 2023-11-08T19:29:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 shutdown_connections 2023-11-08T19:29:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f88d406aec0 0x7f88d406d380 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f88f009af10 0x7f88f00cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f88f009aa90 0x7f88f00cd630 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 --2- 172.21.15.119:0/3577926120 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f88d00052a0 0x7f88f0157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 >> 172.21.15.119:0/3577926120 conn(0x7f88f00d0db0 msgr2=0x7f88f015a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 shutdown_connections 2023-11-08T19:29:12.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.041+0000 7f88f4900640 1 -- 172.21.15.119:0/3577926120 wait complete. 2023-11-08T19:29:12.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:12.060 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p d1/dir) 2023-11-08T19:29:12.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:12.127 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.pin -v 1 d1) 2023-11-08T19:29:12.190 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:12.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.404+0000 7f2f0b373640 1 Processor -- start 2023-11-08T19:29:12.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.404+0000 7f2f0b373640 1 -- start start 2023-11-08T19:29:12.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0b373640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2f040cac30 0x7f2f0415f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0b373640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f040adf20 0x7f2f0415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0b373640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f0415f8c0 con 0x7f2f040ad9d0 2023-11-08T19:29:12.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0b373640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f04071c70 con 0x7f2f040adf20 2023-11-08T19:29:12.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0b373640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f040ca2e0 con 0x7f2f040cac30 2023-11-08T19:29:12.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0ab72640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2f040ad9d0 0x7f2f040c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60860/0 (socket says 172.21.15.119:60860) 2023-11-08T19:29:12.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f09b70640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2f040adf20 0x7f2f0415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.406+0000 7f2f0ab72640 1 -- 172.21.15.119:0/2901257654 learned_addr learned my addr 172.21.15.119:0/2901257654 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:12.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.407+0000 7f2f0a371640 1 --2- 172.21.15.119:0/2901257654 >> v2:172.21.15.188:3300/0 conn(0x7f2f040cac30 0x7f2f0415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.407+0000 7f2f09b70640 1 -- 172.21.15.119:0/2901257654 >> v2:172.21.15.188:3300/0 conn(0x7f2f040cac30 msgr2=0x7f2f0415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:12.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.407+0000 7f2f09b70640 1 --2- 172.21.15.119:0/2901257654 >> v2:172.21.15.188:3300/0 conn(0x7f2f040cac30 0x7f2f0415f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.407+0000 7f2f09b70640 1 -- 172.21.15.119:0/2901257654 >> v1:172.21.15.119:6789/0 conn(0x7f2f040ad9d0 legacy=0x7f2f040c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.407+0000 7f2f09b70640 1 -- 172.21.15.119:0/2901257654 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040ca5c0 con 0x7f2f040adf20 2023-11-08T19:29:12.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.408+0000 7f2f09b70640 1 --2- 172.21.15.119:0/2901257654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040adf20 0x7f2f0415a910 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406b410 tx=0x7f2ef402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4547c4364027a745 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.408+0000 7f2f0936f640 1 -- 172.21.15.119:0/2901257654 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4031830 con 0x7f2f040adf20 2023-11-08T19:29:12.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.408+0000 7f2f0936f640 1 -- 172.21.15.119:0/2901257654 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4031990 con 0x7f2f040adf20 2023-11-08T19:29:12.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.408+0000 7f2f0936f640 1 -- 172.21.15.119:0/2901257654 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4031d20 con 0x7f2f040adf20 2023-11-08T19:29:12.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 -- 172.21.15.119:0/2901257654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040adf20 msgr2=0x7f2f0415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 --2- 172.21.15.119:0/2901257654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040adf20 0x7f2f0415a910 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406b410 tx=0x7f2ef402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 -- 172.21.15.119:0/2901257654 shutdown_connections 2023-11-08T19:29:12.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 --2- 172.21.15.119:0/2901257654 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040adf20 0x7f2f0415a910 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 --2- 172.21.15.119:0/2901257654 >> v2:172.21.15.188:3300/0 conn(0x7f2f040cac30 0x7f2f0415f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 -- 172.21.15.119:0/2901257654 >> 172.21.15.119:0/2901257654 conn(0x7f2f040b0560 msgr2=0x7f2f040b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 -- 172.21.15.119:0/2901257654 shutdown_connections 2023-11-08T19:29:12.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.409+0000 7f2f0b373640 1 -- 172.21.15.119:0/2901257654 wait complete. 2023-11-08T19:29:12.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.411+0000 7f2f0b373640 1 Processor -- start 2023-11-08T19:29:12.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.411+0000 7f2f0b373640 1 -- start start 2023-11-08T19:29:12.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0b373640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040ad9d0 0x7f2f04151a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0b373640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f040adf20 0x7f2f04155570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0b373640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 0x7f2f04152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0b373640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04155ab0 con 0x7f2f040cac30 2023-11-08T19:29:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0b373640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04153fd0 con 0x7f2f040adf20 2023-11-08T19:29:12.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0b373640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04152ff0 con 0x7f2f040ad9d0 2023-11-08T19:29:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 0x7f2f04152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 0x7f2f04152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52792/0 (socket says 172.21.15.119:52792) 2023-11-08T19:29:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 -- 172.21.15.119:0/4102264113 learned_addr learned my addr 172.21.15.119:0/4102264113 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f09b70640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f040adf20 0x7f2f04155570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0a371640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040ad9d0 0x7f2f04151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 -- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040ad9d0 msgr2=0x7f2f04151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040ad9d0 0x7f2f04151a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 -- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f040adf20 msgr2=0x7f2f04155570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f040adf20 0x7f2f04155570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f0ab72640 1 -- 172.21.15.119:0/4102264113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef4009400 con 0x7f2f040cac30 2023-11-08T19:29:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.412+0000 7f2f09b70640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f040adf20 0x7f2f04155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.413+0000 7f2f0ab72640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 0x7f2f04152ab0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00cb20 tx=0x7f2efc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.413+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00aca0 con 0x7f2f040cac30 2023-11-08T19:29:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.413+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00ae00 con 0x7f2f040cac30 2023-11-08T19:29:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.413+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc012600 con 0x7f2f040cac30 2023-11-08T19:29:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.413+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f04153270 con 0x7f2f040cac30 2023-11-08T19:29:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.413+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f04153c60 con 0x7f2f040cac30 2023-11-08T19:29:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.414+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f041537e0 con 0x7f2f040cac30 2023-11-08T19:29:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.415+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2efc007070 con 0x7f2f040cac30 2023-11-08T19:29:12.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.416+0000 7f2efaffd640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ee806ae70 0x7f2ee806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:12.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.416+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f2efc089950 con 0x7f2f040cac30 2023-11-08T19:29:12.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.417+0000 7f2f0a371640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ee806ae70 0x7f2ee806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:12.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.417+0000 7f2f0a371640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ee806ae70 0x7f2ee806d330 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006fd0 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:12.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.421+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f041537e0 con 0x7f2f040cac30 2023-11-08T19:29:12.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.647+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f041537e0 con 0x7f2f040cac30 2023-11-08T19:29:12.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.648+0000 7f2efaffd640 1 -- 172.21.15.119:0/4102264113 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f2f041537e0 con 0x7f2f040cac30 2023-11-08T19:29:12.650 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:12.650 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:08.704975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:12.651 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 56 2023-11-08T19:29:12.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ee806ae70 msgr2=0x7f2ee806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ee806ae70 0x7f2ee806d330 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006fd0 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2023-11-08T19:29:12.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 msgr2=0x7f2f04152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:12.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 0x7f2f04152ab0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2efc00cb20 tx=0x7f2efc00a060 comp rx=0 tx=0).stop 2023-11-08T19:29:12.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 shutdown_connections 2023-11-08T19:29:12.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2ee806ae70 0x7f2ee806d330 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f040cac30 0x7f2f04152ab0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f040adf20 0x7f2f04155570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 --2- 172.21.15.119:0/4102264113 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2f040ad9d0 0x7f2f04151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 >> 172.21.15.119:0/4102264113 conn(0x7f2f040b0560 msgr2=0x7f2f0406c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 shutdown_connections 2023-11-08T19:29:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:12.651+0000 7f2f0b373640 1 -- 172.21.15.119:0/4102264113 wait complete. 2023-11-08T19:29:12.672 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-08T19:29:12.906 INFO:teuthology.orchestra.run.smithi188.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-08T19:29:12.907 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:29:12.907 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-08T19:29:14.908 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:15.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.130+0000 7f346ab35640 1 Processor -- start 2023-11-08T19:29:15.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.130+0000 7f346ab35640 1 -- start start 2023-11-08T19:29:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f346ab35640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34640ad9d0 0x7f346406c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f346ab35640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34640ca2e0 con 0x7f34640adf20 2023-11-08T19:29:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f346ab35640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34640ca5c0 con 0x7f34640cac30 2023-11-08T19:29:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f346ab35640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3464071c70 con 0x7f34640ad9d0 2023-11-08T19:29:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f3469332640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f34640adf20 0x7f346415a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60864/0 (socket says 172.21.15.119:60864) 2023-11-08T19:29:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f3469332640 1 -- 172.21.15.119:0/987085908 learned_addr learned my addr 172.21.15.119:0/987085908 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:15.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.132+0000 7f346a334640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f34640ad9d0 0x7f346406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.133+0000 7f346a334640 1 -- 172.21.15.119:0/987085908 >> v1:172.21.15.188:6789/0 conn(0x7f34640cac30 legacy=0x7f346415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.133+0000 7f346a334640 1 -- 172.21.15.119:0/987085908 >> v1:172.21.15.119:6789/0 conn(0x7f34640adf20 legacy=0x7f346415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.133+0000 7f346a334640 1 -- 172.21.15.119:0/987085908 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346415eb80 con 0x7f34640ad9d0 2023-11-08T19:29:15.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.134+0000 7f346a334640 1 --2- 172.21.15.119:0/987085908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640ad9d0 0x7f346406c660 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f345c009d90 tx=0x7f345c02bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2480ef8487fef56 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:15.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.134+0000 7f3468b31640 1 -- 172.21.15.119:0/987085908 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c02ccf0 con 0x7f34640ad9d0 2023-11-08T19:29:15.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.134+0000 7f3468b31640 1 -- 172.21.15.119:0/987085908 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c02ce50 con 0x7f34640ad9d0 2023-11-08T19:29:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.134+0000 7f3468b31640 1 -- 172.21.15.119:0/987085908 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c034660 con 0x7f34640ad9d0 2023-11-08T19:29:15.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 -- 172.21.15.119:0/987085908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640ad9d0 msgr2=0x7f346406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 --2- 172.21.15.119:0/987085908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640ad9d0 0x7f346406c660 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f345c009d90 tx=0x7f345c02bf40 comp rx=0 tx=0).stop 2023-11-08T19:29:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 -- 172.21.15.119:0/987085908 shutdown_connections 2023-11-08T19:29:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 --2- 172.21.15.119:0/987085908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640ad9d0 0x7f346406c660 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 -- 172.21.15.119:0/987085908 >> 172.21.15.119:0/987085908 conn(0x7f34640b0560 msgr2=0x7f34640b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:15.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 -- 172.21.15.119:0/987085908 shutdown_connections 2023-11-08T19:29:15.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.135+0000 7f346ab35640 1 -- 172.21.15.119:0/987085908 wait complete. 2023-11-08T19:29:15.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.137+0000 7f346ab35640 1 Processor -- start 2023-11-08T19:29:15.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.137+0000 7f346ab35640 1 -- start start 2023-11-08T19:29:15.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f346ab35640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34640ad9d0 0x7f34640c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f346ab35640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 0x7f346415c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f346ab35640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34640cac30 0x7f34640b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f346ab35640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346415d700 con 0x7f34640adf20 2023-11-08T19:29:15.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f346ab35640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464145c80 con 0x7f34640ad9d0 2023-11-08T19:29:15.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f346ab35640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34640c5920 con 0x7f34640cac30 2023-11-08T19:29:15.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f3469332640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 0x7f346415c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f3469332640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 0x7f346415c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52810/0 (socket says 172.21.15.119:52810) 2023-11-08T19:29:15.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f3469332640 1 -- 172.21.15.119:0/31965285 learned_addr learned my addr 172.21.15.119:0/31965285 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:15.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.138+0000 7f3469b33640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34640ad9d0 0x7f34640c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469332640 1 -- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34640cac30 msgr2=0x7f34640b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469332640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34640cac30 0x7f34640b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469332640 1 -- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34640ad9d0 msgr2=0x7f34640c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469332640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34640ad9d0 0x7f34640c53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469332640 1 -- 172.21.15.119:0/31965285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c009a70 con 0x7f34640adf20 2023-11-08T19:29:15.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469b33640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34640ad9d0 0x7f34640c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:15.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f3469332640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 0x7f346415c4f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f345400c500 tx=0x7f345400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:15.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454005ce0 con 0x7f34640adf20 2023-11-08T19:29:15.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3454005e40 con 0x7f34640adf20 2023-11-08T19:29:15.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34540072f0 con 0x7f34640adf20 2023-11-08T19:29:15.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346415ce60 con 0x7f34640adf20 2023-11-08T19:29:15.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.139+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346415d420 con 0x7f34640adf20 2023-11-08T19:29:15.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.140+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3464145ae0 con 0x7f34640adf20 2023-11-08T19:29:15.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.141+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3454007450 con 0x7f34640adf20 2023-11-08T19:29:15.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.142+0000 7f345a7fc640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f344806ae70 0x7f344806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:15.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.143+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f3454088fd0 con 0x7f34640adf20 2023-11-08T19:29:15.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.143+0000 7f3469b33640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f344806ae70 0x7f344806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:15.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.143+0000 7f3469b33640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f344806ae70 0x7f344806d330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3460000f60 tx=0x7f3460009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:15.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.147+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3464145ae0 con 0x7f34640adf20 2023-11-08T19:29:15.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.376+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3464145ae0 con 0x7f34640adf20 2023-11-08T19:29:15.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.377+0000 7f345a7fc640 1 -- 172.21.15.119:0/31965285 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f3464145ae0 con 0x7f34640adf20 2023-11-08T19:29:15.379 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:15.379 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:08.704975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:15.379 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 56 2023-11-08T19:29:15.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f344806ae70 msgr2=0x7f344806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f344806ae70 0x7f344806d330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3460000f60 tx=0x7f3460009040 comp rx=0 tx=0).stop 2023-11-08T19:29:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 msgr2=0x7f346415c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 0x7f346415c4f0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f345400c500 tx=0x7f345400c9c0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 shutdown_connections 2023-11-08T19:29:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f344806ae70 0x7f344806d330 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f34640cac30 0x7f34640b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f34640adf20 0x7f346415c4f0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 --2- 172.21.15.119:0/31965285 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f34640ad9d0 0x7f34640c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 >> 172.21.15.119:0/31965285 conn(0x7f34640b0560 msgr2=0x7f34640af230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 shutdown_connections 2023-11-08T19:29:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:15.380+0000 7f346ab35640 1 -- 172.21.15.119:0/31965285 wait complete. 2023-11-08T19:29:15.400 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-08T19:29:15.640 INFO:teuthology.orchestra.run.smithi188.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-08T19:29:15.641 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:29:15.641 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-08T19:29:17.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.863+0000 7f7d08e02640 1 Processor -- start 2023-11-08T19:29:17.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.863+0000 7f7d08e02640 1 -- start start 2023-11-08T19:29:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.864+0000 7f7d08e02640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7d040cac30 0x7f7d0415f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.864+0000 7f7d08e02640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7d040adf20 0x7f7d0415a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.864+0000 7f7d08e02640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040ad9d0 0x7f7d0415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.864+0000 7f7d08e02640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d04071c70 con 0x7f7d040adf20 2023-11-08T19:29:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.864+0000 7f7d08e02640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d0415b390 con 0x7f7d040cac30 2023-11-08T19:29:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.864+0000 7f7d08e02640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0415f8c0 con 0x7f7d040ad9d0 2023-11-08T19:29:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7d040adf20 0x7f7d0415a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7d040adf20 0x7f7d0415a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52824/0 (socket says 172.21.15.119:52824) 2023-11-08T19:29:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 -- 172.21.15.119:0/86411724 learned_addr learned my addr 172.21.15.119:0/86411724 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7d03fff640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040ad9d0 0x7f7d0415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7d037fe640 1 --2- 172.21.15.119:0/86411724 >> v2:172.21.15.188:3300/0 conn(0x7f7d040cac30 0x7f7d0415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 -- 172.21.15.119:0/86411724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040ad9d0 msgr2=0x7f7d0415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040ad9d0 0x7f7d0415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 -- 172.21.15.119:0/86411724 >> v2:172.21.15.188:3300/0 conn(0x7f7d040cac30 msgr2=0x7f7d0415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 --2- 172.21.15.119:0/86411724 >> v2:172.21.15.188:3300/0 conn(0x7f7d040cac30 0x7f7d0415f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7cfbfff640 1 -- 172.21.15.119:0/86411724 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040ca2e0 con 0x7f7d040adf20 2023-11-08T19:29:17.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.865+0000 7f7d03fff640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040ad9d0 0x7f7d0415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:17.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.866+0000 7f7cfbfff640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040adf20 0x7f7d0415a910 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406b410 tx=0x7f7cec02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f190370a9fee7c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:17.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.866+0000 7f7d02ffd640 1 -- 172.21.15.119:0/86411724 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec031880 con 0x7f7d040adf20 2023-11-08T19:29:17.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d02ffd640 1 -- 172.21.15.119:0/86411724 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec0319e0 con 0x7f7d040adf20 2023-11-08T19:29:17.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d02ffd640 1 -- 172.21.15.119:0/86411724 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec031d70 con 0x7f7d040adf20 2023-11-08T19:29:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 -- 172.21.15.119:0/86411724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040adf20 msgr2=0x7f7d0415a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040adf20 0x7f7d0415a910 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f7d0406b410 tx=0x7f7cec02b920 comp rx=0 tx=0).stop 2023-11-08T19:29:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 -- 172.21.15.119:0/86411724 shutdown_connections 2023-11-08T19:29:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040ad9d0 0x7f7d0415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 --2- 172.21.15.119:0/86411724 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040adf20 0x7f7d0415a910 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 --2- 172.21.15.119:0/86411724 >> v2:172.21.15.188:3300/0 conn(0x7f7d040cac30 0x7f7d0415f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 -- 172.21.15.119:0/86411724 >> 172.21.15.119:0/86411724 conn(0x7f7d040b0560 msgr2=0x7f7d040b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:17.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 -- 172.21.15.119:0/86411724 shutdown_connections 2023-11-08T19:29:17.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.867+0000 7f7d08e02640 1 -- 172.21.15.119:0/86411724 wait complete. 2023-11-08T19:29:17.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.869+0000 7f7d08e02640 1 Processor -- start 2023-11-08T19:29:17.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.869+0000 7f7d08e02640 1 -- start start 2023-11-08T19:29:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d08e02640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d040ad9d0 0x7f7d040acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d08e02640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040adf20 0x7f7d04155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d08e02640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 0x7f7d04155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d08e02640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0415beb0 con 0x7f7d040cac30 2023-11-08T19:29:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d08e02640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d04155590 con 0x7f7d040ad9d0 2023-11-08T19:29:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d08e02640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0415b580 con 0x7f7d040adf20 2023-11-08T19:29:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.870+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 0x7f7d04155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 0x7f7d04155fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52830/0 (socket says 172.21.15.119:52830) 2023-11-08T19:29:17.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 -- 172.21.15.119:0/2947352350 learned_addr learned my addr 172.21.15.119:0/2947352350 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:17.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d037fe640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d040ad9d0 0x7f7d040acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7cfbfff640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040adf20 0x7f7d04155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 -- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040adf20 msgr2=0x7f7d04155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:17.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040adf20 0x7f7d04155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 -- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d040ad9d0 msgr2=0x7f7d040acfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d040ad9d0 0x7f7d040acfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 -- 172.21.15.119:0/2947352350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cec0094d0 con 0x7f7d040cac30 2023-11-08T19:29:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d03fff640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 0x7f7d04155fb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00cd10 tx=0x7f7cfc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.871+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00f680 con 0x7f7d040cac30 2023-11-08T19:29:17.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.872+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc00f800 con 0x7f7d040cac30 2023-11-08T19:29:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.872+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00fbb0 con 0x7f7d040cac30 2023-11-08T19:29:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.872+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0415b7f0 con 0x7f7d040cac30 2023-11-08T19:29:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.872+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d041564f0 con 0x7f7d040cac30 2023-11-08T19:29:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.873+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0415bff0 con 0x7f7d040cac30 2023-11-08T19:29:17.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.874+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7cfc00b040 con 0x7f7d040cac30 2023-11-08T19:29:17.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.875+0000 7f7d00ff9640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7ce406ae70 0x7f7ce406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:17.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.875+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f7cfc08a570 con 0x7f7d040cac30 2023-11-08T19:29:17.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.875+0000 7f7d037fe640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7ce406ae70 0x7f7ce406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.876+0000 7f7d037fe640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7ce406ae70 0x7f7ce406d330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7d0415da50 tx=0x7f7cf4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:17.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:17.879+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d0415bff0 con 0x7f7d040cac30 2023-11-08T19:29:18.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.110+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d0415bff0 con 0x7f7d040cac30 2023-11-08T19:29:18.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.111+0000 7f7d00ff9640 1 -- 172.21.15.119:0/2947352350 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4938 (secure 0 0 0) 0x7f7d0415bff0 con 0x7f7d040cac30 2023-11-08T19:29:18.113 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:18.113 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":56,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:08.704975+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:18.113 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 56 2023-11-08T19:29:18.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7ce406ae70 msgr2=0x7f7ce406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:18.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7ce406ae70 0x7f7ce406d330 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f7d0415da50 tx=0x7f7cf4008040 comp rx=0 tx=0).stop 2023-11-08T19:29:18.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 msgr2=0x7f7d04155fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:18.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 0x7f7d04155fb0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00cd10 tx=0x7f7cfc00a650 comp rx=0 tx=0).stop 2023-11-08T19:29:18.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 shutdown_connections 2023-11-08T19:29:18.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7ce406ae70 0x7f7ce406d330 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:18.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7d040cac30 0x7f7d04155fb0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:18.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7d040adf20 0x7f7d04155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:18.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 --2- 172.21.15.119:0/2947352350 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7d040ad9d0 0x7f7d040acfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:18.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 >> 172.21.15.119:0/2947352350 conn(0x7f7d040b0560 msgr2=0x7f7d040b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:18.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 shutdown_connections 2023-11-08T19:29:18.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:18.114+0000 7f7d08e02640 1 -- 172.21.15.119:0/2947352350 wait complete. 2023-11-08T19:29:18.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-08T19:29:18.375 INFO:teuthology.orchestra.run.smithi188.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}}] 2023-11-08T19:29:18.376 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:29:18.376 INFO:tasks.cephfs.cephfs_test_case:[] =?= [('/d1', 1)] 2023-11-08T19:29:19.814 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:19.824 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:20.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:20.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.600+0000 7fcf3bfff640 1 Processor -- start 2023-11-08T19:29:20.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.600+0000 7fcf3bfff640 1 -- start start 2023-11-08T19:29:20.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.601+0000 7fcf3bfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf3c0adf20 0x7fcf3c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.601+0000 7fcf3bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf3c0ad9d0 0x7fcf3c15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.601+0000 7fcf3bfff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3c0ca2e0 con 0x7fcf3c0cac30 2023-11-08T19:29:20.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.601+0000 7fcf3bfff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c071c70 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.601+0000 7fcf3bfff640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3c15b390 con 0x7fcf3c0adf20 2023-11-08T19:29:20.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3b7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcf3c0ad9d0 0x7fcf3c15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3affd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcf3c0cac30 0x7fcf3c15f380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58718/0 (socket says 172.21.15.119:58718) 2023-11-08T19:29:20.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3affd640 1 -- 172.21.15.119:0/3462242499 learned_addr learned my addr 172.21.15.119:0/3462242499 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:20.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf33fff640 1 --2- 172.21.15.119:0/3462242499 >> v2:172.21.15.188:3300/0 conn(0x7fcf3c0adf20 0x7fcf3c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3b7fe640 1 -- 172.21.15.119:0/3462242499 >> v2:172.21.15.188:3300/0 conn(0x7fcf3c0adf20 msgr2=0x7fcf3c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:20.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3b7fe640 1 --2- 172.21.15.119:0/3462242499 >> v2:172.21.15.188:3300/0 conn(0x7fcf3c0adf20 0x7fcf3c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3b7fe640 1 -- 172.21.15.119:0/3462242499 >> v1:172.21.15.119:6789/0 conn(0x7fcf3c0cac30 legacy=0x7fcf3c15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3b7fe640 1 -- 172.21.15.119:0/3462242499 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3c0ca5c0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.602+0000 7fcf3a7fc640 1 -- 172.21.15.119:0/3462242499 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302080965 0 0) 0x7fcf3c0ca2e0 con 0x7fcf3c0cac30 2023-11-08T19:29:20.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.603+0000 7fcf3b7fe640 1 --2- 172.21.15.119:0/3462242499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c15ae50 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0ce940 tx=0x7fcf3402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746a37d50147e23a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3a7fc640 1 -- 172.21.15.119:0/3462242499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3402cc80 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3a7fc640 1 -- 172.21.15.119:0/3462242499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3402cde0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3a7fc640 1 -- 172.21.15.119:0/3462242499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34034660 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 -- 172.21.15.119:0/3462242499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 msgr2=0x7fcf3c15ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/3462242499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c15ae50 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0ce940 tx=0x7fcf3402bed0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 -- 172.21.15.119:0/3462242499 shutdown_connections 2023-11-08T19:29:20.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/3462242499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c15ae50 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/3462242499 >> v2:172.21.15.188:3300/0 conn(0x7fcf3c0adf20 0x7fcf3c15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 -- 172.21.15.119:0/3462242499 >> 172.21.15.119:0/3462242499 conn(0x7fcf3c0b0560 msgr2=0x7fcf3c0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:20.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 -- 172.21.15.119:0/3462242499 shutdown_connections 2023-11-08T19:29:20.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.604+0000 7fcf3bfff640 1 -- 172.21.15.119:0/3462242499 wait complete. 2023-11-08T19:29:20.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.606+0000 7fcf3bfff640 1 Processor -- start 2023-11-08T19:29:20.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.606+0000 7fcf3bfff640 1 -- start start 2023-11-08T19:29:20.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3c0adf20 0x7fcf3c156310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3bfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3c0cac30 0x7fcf3c152660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3bfff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c153a30 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3bfff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c153d10 con 0x7fcf3c0adf20 2023-11-08T19:29:20.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3bfff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3c153ff0 con 0x7fcf3c0cac30 2023-11-08T19:29:20.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53286/0 (socket says 172.21.15.119:53286) 2023-11-08T19:29:20.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.607+0000 7fcf3affd640 1 -- 172.21.15.119:0/2043997957 learned_addr learned my addr 172.21.15.119:0/2043997957 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:20.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3affd640 1 -- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3c0cac30 msgr2=0x7fcf3c152660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3b7fe640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3c0cac30 0x7fcf3c152660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf33fff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3c0adf20 0x7fcf3c156310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3affd640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3c0cac30 0x7fcf3c152660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3affd640 1 -- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3c0adf20 msgr2=0x7fcf3c156310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3affd640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3c0adf20 0x7fcf3c156310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3affd640 1 -- 172.21.15.119:0/2043997957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34009a70 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf3affd640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c155dd0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c430 tx=0x7fcf2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.608+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c0074e0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.609+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c007660 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.609+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c004a20 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.609+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3c1542d0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.609+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3c153700 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.610+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3c1547f0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.611+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf2c004b80 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.612+0000 7fcf337fe640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcf1c06ae70 0x7fcf1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:20.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.612+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fcf2c089aa0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.612+0000 7fcf33fff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcf1c06ae70 0x7fcf1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:20.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.613+0000 7fcf33fff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcf1c06ae70 0x7fcf1c06d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0c5860 tx=0x7fcf24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:20.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.616+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf2c054550 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.849+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf3c04ece0 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.850+0000 7fcf337fe640 1 -- 172.21.15.119:0/2043997957 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fcf2c057b70 con 0x7fcf3c0ad9d0 2023-11-08T19:29:20.852 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:20.853 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:20.631811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:20.853 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 57 2023-11-08T19:29:20.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcf1c06ae70 msgr2=0x7fcf1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcf1c06ae70 0x7fcf1c06d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0c5860 tx=0x7fcf24008040 comp rx=0 tx=0).stop 2023-11-08T19:29:20.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 msgr2=0x7fcf3c155dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:20.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c155dd0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c430 tx=0x7fcf2c00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 shutdown_connections 2023-11-08T19:29:20.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcf1c06ae70 0x7fcf1c06d330 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf3c0cac30 0x7fcf3c152660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf3c0adf20 0x7fcf3c156310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 --2- 172.21.15.119:0/2043997957 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcf3c0ad9d0 0x7fcf3c155dd0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:20.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.852+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 >> 172.21.15.119:0/2043997957 conn(0x7fcf3c0b0560 msgr2=0x7fcf3c15a750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:20.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.853+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 shutdown_connections 2023-11-08T19:29:20.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:20.853+0000 7fcf3bfff640 1 -- 172.21.15.119:0/2043997957 wait complete. 2023-11-08T19:29:20.874 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json get subtrees 2023-11-08T19:29:21.107 INFO:teuthology.orchestra.run.smithi188.stdout:[{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"~mds1","dirfrag":"0x101","snapid_first":2,"projected_version":"1","version":"1","committing_version":"1","committed_version":"1","is_rep":false,"dir_auth":"1","states":["auth","complete"],"is_auth":true,"auth_state":{"replicas":{}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1,"dirty":0,"authpin":0},"nref":2}},{"is_auth":false,"auth_first":0,"auth_second":-2,"export_pin":-1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":-1,"dir":{"path":"","dirfrag":"0x1","snapid_first":2,"projected_version":"0","version":"8","committing_version":"0","committed_version":"0","is_rep":true,"dir_auth":"0","states":[],"is_auth":false,"auth_state":{"replicas":{}},"replica_state":{"authority":[0,-2],"replica_nonce":1},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"child":1,"subtree":1},"nref":2}},{"is_auth":true,"auth_first":1,"auth_second":-2,"export_pin":1,"distributed_ephemeral_pin":false,"random_ephemeral_pin":false,"export_pin_target":1,"dir":{"path":"/d1","dirfrag":"0x10000000000","snapid_first":2,"projected_version":"3","version":"3","committing_version":"0","committed_version":"0","is_rep":false,"dir_auth":"1","states":["auth","dirty","complete"],"is_auth":true,"auth_state":{"replicas":{"0":1}},"replica_state":{"authority":[1,-2],"replica_nonce":0},"auth_pins":0,"is_frozen":false,"is_freezing":false,"pins":{"ptrwaiter":0,"child":1,"frozen":0,"subtree":1,"importing":0,"replicated":1,"dirty":1},"nref":4}}] 2023-11-08T19:29:21.108 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "1", "committing_version": "1", "dir_auth": "1", "dirfrag": "0x101", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 2, "path": "~mds1", "pins": { "authpin": 0, "child": 1, "dirty": 0, "subtree": 1 }, "projected_version": "1", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "complete" ], "version": "1" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": true, "random_ephemeral_pin": false }, { "auth_first": 0, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": {} }, "committed_version": "0", "committing_version": "0", "dir_auth": "0", "dirfrag": "0x1", "is_auth": false, "is_freezing": false, "is_frozen": false, "is_rep": true, "nref": 2, "path": "", "pins": { "child": 1, "subtree": 1 }, "projected_version": "0", "replica_state": { "authority": [ 0, -2 ], "replica_nonce": 1 }, "snapid_first": 2, "states": [], "version": "8" }, "distributed_ephemeral_pin": false, "export_pin": -1, "export_pin_target": -1, "is_auth": false, "random_ephemeral_pin": false }, { "auth_first": 1, "auth_second": -2, "dir": { "auth_pins": 0, "auth_state": { "replicas": { "0": 1 } }, "committed_version": "0", "committing_version": "0", "dir_auth": "1", "dirfrag": "0x10000000000", "is_auth": true, "is_freezing": false, "is_frozen": false, "is_rep": false, "nref": 4, "path": "/d1", "pins": { "child": 1, "dirty": 1, "frozen": 0, "importing": 0, "ptrwaiter": 0, "replicated": 1, "subtree": 1 }, "projected_version": "3", "replica_state": { "authority": [ 1, -2 ], "replica_nonce": 0 }, "snapid_first": 2, "states": [ "auth", "dirty", "complete" ], "version": "3" }, "distributed_ephemeral_pin": false, "export_pin": 1, "export_pin_target": 1, "is_auth": true, "random_ephemeral_pin": false } ] 2023-11-08T19:29:21.108 INFO:tasks.cephfs.cephfs_test_case:[('/d1', 1)] =?= [('/d1', 1)] 2023-11-08T19:29:21.109 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-08T19:29:21.339 INFO:teuthology.orchestra.run.smithi188.stdout:{"last_snap":1,"last_created":1,"last_destroyed":1,"pending_noop":[],"snaps":[],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:29:21.340 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 1, "last_destroyed": 1, "last_snap": 1, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-08T19:29:21.340 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=1 2023-11-08T19:29:21.340 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:21.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.554+0000 7f7804b89640 1 Processor -- start 2023-11-08T19:29:21.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.554+0000 7f7804b89640 1 -- start start 2023-11-08T19:29:21.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.555+0000 7f7804b89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78001447a0 0x7f780015f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.555+0000 7f7804b89640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78000cad30 0x7f780006f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f7804b89640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f78000cb170 0x7f780006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f7804b89640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78000c3a50 con 0x7f78000cad30 2023-11-08T19:29:21.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f7804b89640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7800144b80 con 0x7f78000cb170 2023-11-08T19:29:21.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f7804b89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78000c9150 con 0x7f78001447a0 2023-11-08T19:29:21.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78000cad30 0x7f780006f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f78000cad30 0x7f780006f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53294/0 (socket says 172.21.15.119:53294) 2023-11-08T19:29:21.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 -- 172.21.15.119:0/1813801039 learned_addr learned my addr 172.21.15.119:0/1813801039 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:21.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fe59c640 1 --2- 172.21.15.119:0/1813801039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78001447a0 0x7f780015f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fed9d640 1 --2- 172.21.15.119:0/1813801039 >> v2:172.21.15.188:3300/0 conn(0x7f78000cb170 0x7f780006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 -- 172.21.15.119:0/1813801039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78001447a0 msgr2=0x7f780015f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 --2- 172.21.15.119:0/1813801039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78001447a0 0x7f780015f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 -- 172.21.15.119:0/1813801039 >> v2:172.21.15.188:3300/0 conn(0x7f78000cb170 msgr2=0x7f780006a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:21.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 --2- 172.21.15.119:0/1813801039 >> v2:172.21.15.188:3300/0 conn(0x7f78000cb170 0x7f780006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.556+0000 7f77fdd9b640 1 -- 172.21.15.119:0/1813801039 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78000c9410 con 0x7f78000cad30 2023-11-08T19:29:21.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.558+0000 7f77fdd9b640 1 --2- 172.21.15.119:0/1813801039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cad30 0x7f780006f930 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f78000ca7a0 tx=0x7f77f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756ece8ba5bad786 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.558+0000 7f77fd59a640 1 -- 172.21.15.119:0/1813801039 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4031870 con 0x7f78000cad30 2023-11-08T19:29:21.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.558+0000 7f77fd59a640 1 -- 172.21.15.119:0/1813801039 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40319d0 con 0x7f78000cad30 2023-11-08T19:29:21.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.558+0000 7f77fd59a640 1 -- 172.21.15.119:0/1813801039 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4031d60 con 0x7f78000cad30 2023-11-08T19:29:21.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.558+0000 7f7804b89640 1 -- 172.21.15.119:0/1813801039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cad30 msgr2=0x7f780006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.558+0000 7f7804b89640 1 --2- 172.21.15.119:0/1813801039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cad30 0x7f780006f930 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f78000ca7a0 tx=0x7f77f402b910 comp rx=0 tx=0).stop 2023-11-08T19:29:21.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 -- 172.21.15.119:0/1813801039 shutdown_connections 2023-11-08T19:29:21.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 --2- 172.21.15.119:0/1813801039 >> v2:172.21.15.188:3300/0 conn(0x7f78000cb170 0x7f780006a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 --2- 172.21.15.119:0/1813801039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cad30 0x7f780006f930 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 --2- 172.21.15.119:0/1813801039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78001447a0 0x7f780015f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 -- 172.21.15.119:0/1813801039 >> 172.21.15.119:0/1813801039 conn(0x7f78000cb610 msgr2=0x7f78000cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 -- 172.21.15.119:0/1813801039 shutdown_connections 2023-11-08T19:29:21.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.559+0000 7f7804b89640 1 -- 172.21.15.119:0/1813801039 wait complete. 2023-11-08T19:29:21.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.560+0000 7f7804b89640 1 Processor -- start 2023-11-08T19:29:21.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.560+0000 7f7804b89640 1 -- start start 2023-11-08T19:29:21.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.561+0000 7f7804b89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cad30 0x7f78000c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.561+0000 7f7804b89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 0x7f7800150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.561+0000 7f7804b89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78001447a0 0x7f78000b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.561+0000 7f7804b89640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000b4620 con 0x7f78000cb170 2023-11-08T19:29:21.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.561+0000 7f7804b89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7800138e90 con 0x7f78001447a0 2023-11-08T19:29:21.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.561+0000 7f7804b89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78000b22b0 con 0x7f78000cad30 2023-11-08T19:29:21.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 0x7f7800150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 0x7f7800150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53298/0 (socket says 172.21.15.119:53298) 2023-11-08T19:29:21.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 -- 172.21.15.119:0/2019360455 learned_addr learned my addr 172.21.15.119:0/2019360455 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:21.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 -- 172.21.15.119:0/2019360455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cad30 msgr2=0x7f78000c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cad30 0x7f78000c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 -- 172.21.15.119:0/2019360455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78001447a0 msgr2=0x7f78000b40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78001447a0 0x7f78000b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 -- 172.21.15.119:0/2019360455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40094d0 con 0x7f78000cb170 2023-11-08T19:29:21.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.562+0000 7f77fdd9b640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 0x7f7800150720 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f77f402b950 tx=0x7f77f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.563+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4031860 con 0x7f78000cb170 2023-11-08T19:29:21.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.563+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f40443f0 con 0x7f78000cb170 2023-11-08T19:29:21.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.563+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f40447a0 con 0x7f78000cb170 2023-11-08T19:29:21.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.563+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78000b2500 con 0x7f78000cb170 2023-11-08T19:29:21.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.563+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78000b2ac0 con 0x7f78000cb170 2023-11-08T19:29:21.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.564+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7800074430 con 0x7f78000cb170 2023-11-08T19:29:21.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.565+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f77f404d450 con 0x7f78000cb170 2023-11-08T19:29:21.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.566+0000 7f77eeffd640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f77e006ae70 0x7f77e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:21.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.567+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f77f402d040 con 0x7f78000cb170 2023-11-08T19:29:21.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.567+0000 7f77fe59c640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f77e006ae70 0x7f77e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:21.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.567+0000 7f77fe59c640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f77e006ae70 0x7f77e006d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f77f0006b10 tx=0x7f77f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:21.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.571+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f77f4076e70 con 0x7f78000cb170 2023-11-08T19:29:21.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.801+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f780006caa0 con 0x7f78000cb170 2023-11-08T19:29:21.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.802+0000 7f77eeffd640 1 -- 172.21.15.119:0/2019360455 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f77f4035090 con 0x7f78000cb170 2023-11-08T19:29:21.805 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:21.805 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":57,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:20.631811+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:21.805 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 57 2023-11-08T19:29:21.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.804+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f77e006ae70 msgr2=0x7f77e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.804+0000 7f7804b89640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f77e006ae70 0x7f77e006d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f77f0006b10 tx=0x7f77f0008040 comp rx=0 tx=0).stop 2023-11-08T19:29:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.804+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 msgr2=0x7f7800150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.804+0000 7f7804b89640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 0x7f7800150720 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f77f402b950 tx=0x7f77f40046c0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 shutdown_connections 2023-11-08T19:29:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f77e006ae70 0x7f77e006d330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f78001447a0 0x7f78000b40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f78000cb170 0x7f7800150720 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 --2- 172.21.15.119:0/2019360455 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f78000cad30 0x7f78000c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 >> 172.21.15.119:0/2019360455 conn(0x7f78000cb610 msgr2=0x7f780006a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 shutdown_connections 2023-11-08T19:29:21.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:21.805+0000 7f7804b89640 1 -- 172.21.15.119:0/2019360455 wait complete. 2023-11-08T19:29:21.826 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:29:22.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.034+0000 7f8489258640 1 Processor -- start 2023-11-08T19:29:22.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.035+0000 7f8489258640 1 -- start start 2023-11-08T19:29:22.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.036+0000 7f8489258640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8484161870 0x7f84840cfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.036+0000 7f8489258640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848415df50 con 0x7f848415d4a0 2023-11-08T19:29:22.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.036+0000 7f8489258640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84840d1e60 con 0x7f84840cfa00 2023-11-08T19:29:22.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.036+0000 7f8489258640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84840d03b0 con 0x7f8484161870 2023-11-08T19:29:22.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.036+0000 7f8482d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f848415d4a0 0x7f84840d22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58726/0 (socket says 172.21.15.119:58726) 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.036+0000 7f8482d9d640 1 -- 172.21.15.119:0/823346320 learned_addr learned my addr 172.21.15.119:0/823346320 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:22.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.037+0000 7f848359e640 1 --2- 172.21.15.119:0/823346320 >> v2:172.21.15.119:3300/0 conn(0x7f8484161870 0x7f84840cfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.037+0000 7f8481d9b640 1 -- 172.21.15.119:0/823346320 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226007085 0 0) 0x7f848415df50 con 0x7f848415d4a0 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.037+0000 7f8481d9b640 1 -- 172.21.15.119:0/823346320 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8470003520 con 0x7f848415d4a0 2023-11-08T19:29:22.042 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.037+0000 7f848359e640 1 -- 172.21.15.119:0/823346320 >> v1:172.21.15.188:6789/0 conn(0x7f84840cfa00 legacy=0x7f8484161160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.037+0000 7f848359e640 1 -- 172.21.15.119:0/823346320 >> v1:172.21.15.119:6789/0 conn(0x7f848415d4a0 legacy=0x7f84840d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.037+0000 7f848359e640 1 -- 172.21.15.119:0/823346320 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8484161d40 con 0x7f8484161870 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.038+0000 7f848359e640 1 --2- 172.21.15.119:0/823346320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484161870 0x7f84840cfe70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097a0 tx=0x7f847c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303eed05257ac73b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8481d9b640 1 -- 172.21.15.119:0/823346320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c02e070 con 0x7f8484161870 2023-11-08T19:29:22.043 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8481d9b640 1 -- 172.21.15.119:0/823346320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c0315a0 con 0x7f8484161870 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8481d9b640 1 -- 172.21.15.119:0/823346320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c031910 con 0x7f8484161870 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 -- 172.21.15.119:0/823346320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484161870 msgr2=0x7f84840cfe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 --2- 172.21.15.119:0/823346320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484161870 0x7f84840cfe70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f847c0097a0 tx=0x7f847c02b840 comp rx=0 tx=0).stop 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 -- 172.21.15.119:0/823346320 shutdown_connections 2023-11-08T19:29:22.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 --2- 172.21.15.119:0/823346320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8484161870 0x7f84840cfe70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 -- 172.21.15.119:0/823346320 >> 172.21.15.119:0/823346320 conn(0x7f848406a9f0 msgr2=0x7f848406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 -- 172.21.15.119:0/823346320 shutdown_connections 2023-11-08T19:29:22.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.039+0000 7f8489258640 1 -- 172.21.15.119:0/823346320 wait complete. 2023-11-08T19:29:22.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.041+0000 7f8489258640 1 Processor -- start 2023-11-08T19:29:22.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.041+0000 7f8489258640 1 -- start start 2023-11-08T19:29:22.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f8489258640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84840cfa00 0x7f84840ccf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f8489258640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 0x7f8484156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f8489258640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8484161870 0x7f8484149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f8489258640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84841545a0 con 0x7f848415d4a0 2023-11-08T19:29:22.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f8489258640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8484156290 con 0x7f8484161870 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f8489258640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84841551f0 con 0x7f84840cfa00 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f848259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 0x7f8484156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f848259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 0x7f8484156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53310/0 (socket says 172.21.15.119:53310) 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.042+0000 7f848259c640 1 -- 172.21.15.119:0/3963485397 learned_addr learned my addr 172.21.15.119:0/3963485397 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:22.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f8482d9d640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84840cfa00 0x7f84840ccf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848359e640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8484161870 0x7f8484149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848259c640 1 -- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84840cfa00 msgr2=0x7f84840ccf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848259c640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84840cfa00 0x7f84840ccf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848259c640 1 -- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8484161870 msgr2=0x7f8484149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848259c640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8484161870 0x7f8484149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848259c640 1 -- 172.21.15.119:0/3963485397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c009400 con 0x7f848415d4a0 2023-11-08T19:29:22.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f8482d9d640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84840cfa00 0x7f84840ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848359e640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8484161870 0x7f8484149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f848259c640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 0x7f8484156860 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c430 tx=0x7f846c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.043+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c011790 con 0x7f848415d4a0 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.044+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c0118f0 con 0x7f848415d4a0 2023-11-08T19:29:22.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.044+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c011ce0 con 0x7f848415d4a0 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.044+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84841554d0 con 0x7f848415d4a0 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.044+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8484155a90 con 0x7f848415d4a0 2023-11-08T19:29:22.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.045+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84840d10f0 con 0x7f848415d4a0 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.046+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f846c01c460 con 0x7f848415d4a0 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.046+0000 7f847b7fe640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846406ae70 0x7f846406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:22.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.047+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f846c089aa0 con 0x7f848415d4a0 2023-11-08T19:29:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.047+0000 7f8482d9d640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846406ae70 0x7f846406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.047+0000 7f8482d9d640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846406ae70 0x7f846406d330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8474005c00 tx=0x7f8474009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:22.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.051+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f846c054660 con 0x7f848415d4a0 2023-11-08T19:29:22.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.311+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f84840c9920 con 0x7f848415d4a0 2023-11-08T19:29:22.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.845+0000 7f847b7fe640 1 -- 172.21.15.119:0/3963485397 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7068 v58) v1 ==== 112+0+0 (secure 0 0 0) 0x7f846c057c80 con 0x7f848415d4a0 2023-11-08T19:29:22.848 INFO:teuthology.orchestra.run.smithi119.stderr:freezing mds.7068 2023-11-08T19:29:22.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.848+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846406ae70 msgr2=0x7f846406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.848+0000 7f8489258640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846406ae70 0x7f846406d330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8474005c00 tx=0x7f8474009040 comp rx=0 tx=0).stop 2023-11-08T19:29:22.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.848+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 msgr2=0x7f8484156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:22.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.848+0000 7f8489258640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 0x7f8484156860 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c430 tx=0x7f846c00c8f0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 shutdown_connections 2023-11-08T19:29:22.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846406ae70 0x7f846406d330 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8484161870 0x7f8484149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f848415d4a0 0x7f8484156860 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 --2- 172.21.15.119:0/3963485397 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f84840cfa00 0x7f84840ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:22.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 >> 172.21.15.119:0/3963485397 conn(0x7f848406a9f0 msgr2=0x7f848415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:22.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 shutdown_connections 2023-11-08T19:29:22.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:22.849+0000 7f8489258640 1 -- 172.21.15.119:0/3963485397 wait complete. 2023-11-08T19:29:22.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 1 2023-11-08T19:29:23.102 INFO:teuthology.orchestra.run.smithi188.stdout:{"success":"mds_kill_mdstable_at = '1' (not observed, change may require restart) "} 2023-11-08T19:29:23.103 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '1' (not observed, change may require restart) " } 2023-11-08T19:29:23.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:23.104 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s11) 2023-11-08T19:29:23.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:29:23.171 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f5fa24b6640 time 2023-11-08T19:29:23.170800+0000 2023-11-08T19:29:23.171 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:29:23.172 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.172 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5fa7f6efaf] 2023-11-08T19:29:23.172 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.173 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.173 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.173 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.173 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.173 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.174 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.174 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.174 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.174 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.174 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:29:23.170+0000 7f5fa24b6640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f5fa24b6640 time 2023-11-08T19:29:23.170800+0000 2023-11-08T19:29:23.175 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:29:23.175 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.175 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.175 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5fa7f6efaf] 2023-11-08T19:29:23.175 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.176 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.176 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.176 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.176 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.176 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.176 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.177 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.177 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.177 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.177 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.177 INFO:tasks.ceph.mds.b.smithi188.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:29:23.178 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f5fa24b6640 thread_name:ms_dispatch 2023-11-08T19:29:23.178 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.178 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5fa7654db0] 2023-11-08T19:29:23.179 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5fa76a154c] 2023-11-08T19:29:23.179 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:29:23.179 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:29:23.179 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5fa7f6f009] 2023-11-08T19:29:23.179 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.180 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.180 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.180 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.180 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.180 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.181 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.181 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.181 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.181 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.181 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:29:23.173+0000 7f5fa24b6640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:23.182 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f5fa24b6640 thread_name:ms_dispatch 2023-11-08T19:29:23.182 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.182 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.182 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5fa7654db0] 2023-11-08T19:29:23.182 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5fa76a154c] 2023-11-08T19:29:23.183 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:29:23.183 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:29:23.183 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5fa7f6f009] 2023-11-08T19:29:23.183 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.183 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.184 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.184 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.184 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.184 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.184 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.185 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.185 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.185 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.185 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:23.185 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.199 INFO:tasks.ceph.mds.b.smithi188.stderr: -3520> 2023-11-08T19:28:56.416+0000 7f5f9fcb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:23.210 INFO:tasks.ceph.mds.b.smithi188.stderr: -1783> 2023-11-08T19:29:11.418+0000 7f5f9fcb1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:23.221 INFO:tasks.ceph.mds.b.smithi188.stderr: -1> 2023-11-08T19:29:23.170+0000 7f5fa24b6640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f5fa24b6640 time 2023-11-08T19:29:23.170800+0000 2023-11-08T19:29:23.221 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:29:23.221 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.222 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.222 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5fa7f6efaf] 2023-11-08T19:29:23.222 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.222 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.223 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.223 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.223 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.223 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.224 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.224 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.224 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.224 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.224 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.225 INFO:tasks.ceph.mds.b.smithi188.stderr: 0> 2023-11-08T19:29:23.173+0000 7f5fa24b6640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:23.225 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f5fa24b6640 thread_name:ms_dispatch 2023-11-08T19:29:23.225 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.225 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.225 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5fa7654db0] 2023-11-08T19:29:23.226 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5fa76a154c] 2023-11-08T19:29:23.226 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:29:23.226 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:29:23.226 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5fa7f6f009] 2023-11-08T19:29:23.227 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.227 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.227 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.227 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.227 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.228 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.228 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.228 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.228 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.229 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.229 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:23.229 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.229 INFO:tasks.ceph.mds.b.smithi188.stderr: -3520> 2023-11-08T19:28:56.416+0000 7f5f9fcb1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:23.231 INFO:tasks.ceph.mds.b.smithi188.stderr: -1783> 2023-11-08T19:29:11.418+0000 7f5f9fcb1640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:23.234 INFO:tasks.ceph.mds.b.smithi188.stderr: -1> 2023-11-08T19:29:23.170+0000 7f5fa24b6640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_prepare(ceph::cref_t&)' thread 7f5fa24b6640 time 2023-11-08T19:29:23.170800+0000 2023-11-08T19:29:23.234 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 61: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 1) 2023-11-08T19:29:23.235 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.235 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.235 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f5fa7f6efaf] 2023-11-08T19:29:23.235 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.236 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.236 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.236 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.236 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.236 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.237 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.237 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.237 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.237 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.237 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.238 INFO:tasks.ceph.mds.b.smithi188.stderr: 0> 2023-11-08T19:29:23.173+0000 7f5fa24b6640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:23.238 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f5fa24b6640 thread_name:ms_dispatch 2023-11-08T19:29:23.238 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.238 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:23.238 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f5fa7654db0] 2023-11-08T19:29:23.239 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f5fa76a154c] 2023-11-08T19:29:23.239 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:29:23.239 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:29:23.239 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f5fa7f6f009] 2023-11-08T19:29:23.239 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f5fa7f6f16d] 2023-11-08T19:29:23.240 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_prepare(boost::intrusive_ptr const&)+0x354) [0x55fc11cad254] 2023-11-08T19:29:23.240 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x55fc11a400a9] 2023-11-08T19:29:23.240 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x55fc11a42b2b] 2023-11-08T19:29:23.240 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x55fc11a4301c] 2023-11-08T19:29:23.241 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x55fc11a2c325] 2023-11-08T19:29:23.241 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f5fa815ff9a] 2023-11-08T19:29:23.241 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f5fa81f1741] 2023-11-08T19:29:23.241 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f5fa769f802] 2023-11-08T19:29:23.241 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f5fa763f450] 2023-11-08T19:29:23.242 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:23.242 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:29:23.381 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:29:23.381 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:29:23.382 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:23.445 INFO:tasks.ceph.mds.b.smithi188.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:29:23.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.593+0000 7f6d6277c640 1 Processor -- start 2023-11-08T19:29:23.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.593+0000 7f6d6277c640 1 -- start start 2023-11-08T19:29:23.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d6277c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d5c1447a0 0x7f6d5c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d6277c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d6277c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0cb170 0x7f6d5c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d6277c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c0c3a50 con 0x7f6d5c1447a0 2023-11-08T19:29:23.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d6277c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c144b80 con 0x7f6d5c0cb170 2023-11-08T19:29:23.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d6277c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0c9150 con 0x7f6d5c0cad30 2023-11-08T19:29:23.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d5c1447a0 0x7f6d5c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:23.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6d5c1447a0 0x7f6d5c15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53324/0 (socket says 172.21.15.119:53324) 2023-11-08T19:29:23.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 -- 172.21.15.119:0/1234899059 learned_addr learned my addr 172.21.15.119:0/1234899059 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:23.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 -- 172.21.15.119:0/1234899059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 msgr2=0x7f6d5c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:23.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 --2- 172.21.15.119:0/1234899059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 -- 172.21.15.119:0/1234899059 >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0cb170 msgr2=0x7f6d5c06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:23.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 --2- 172.21.15.119:0/1234899059 >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0cb170 0x7f6d5c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.596+0000 7f6d5bfff640 1 -- 172.21.15.119:0/1234899059 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c0c9410 con 0x7f6d5c1447a0 2023-11-08T19:29:23.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.597+0000 7f6d5bfff640 1 --2- 172.21.15.119:0/1234899059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15f4c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0ca7a0 tx=0x7f6d4c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d374d800ea525cae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:23.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.597+0000 7f6d5b7fe640 1 -- 172.21.15.119:0/1234899059 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c031870 con 0x7f6d5c1447a0 2023-11-08T19:29:23.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.597+0000 7f6d5b7fe640 1 -- 172.21.15.119:0/1234899059 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c0319d0 con 0x7f6d5c1447a0 2023-11-08T19:29:23.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.597+0000 7f6d5b7fe640 1 -- 172.21.15.119:0/1234899059 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c031d60 con 0x7f6d5c1447a0 2023-11-08T19:29:23.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 -- 172.21.15.119:0/1234899059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 msgr2=0x7f6d5c15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1234899059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15f4c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0ca7a0 tx=0x7f6d4c02b910 comp rx=0 tx=0).stop 2023-11-08T19:29:23.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 -- 172.21.15.119:0/1234899059 shutdown_connections 2023-11-08T19:29:23.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1234899059 >> v2:172.21.15.188:3300/0 conn(0x7f6d5c0cb170 0x7f6d5c06a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1234899059 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1234899059 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15f4c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 -- 172.21.15.119:0/1234899059 >> 172.21.15.119:0/1234899059 conn(0x7f6d5c0cb610 msgr2=0x7f6d5c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:23.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 -- 172.21.15.119:0/1234899059 shutdown_connections 2023-11-08T19:29:23.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.598+0000 7f6d6277c640 1 -- 172.21.15.119:0/1234899059 wait complete. 2023-11-08T19:29:23.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.600+0000 7f6d6277c640 1 Processor -- start 2023-11-08T19:29:23.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.600+0000 7f6d6277c640 1 -- start start 2023-11-08T19:29:23.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d6277c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c15a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d6277c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5c0cb170 0x7f6d5c0cdf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d6277c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d6277c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0cda70 con 0x7f6d5c1447a0 2023-11-08T19:29:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d6277c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0cd140 con 0x7f6d5c0cb170 2023-11-08T19:29:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d6277c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c0cd420 con 0x7f6d5c0cad30 2023-11-08T19:29:23.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d60d19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d60d19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:53326/0 (socket says 172.21.15.119:53326) 2023-11-08T19:29:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d60d19640 1 -- 172.21.15.119:0/1665510664 learned_addr learned my addr 172.21.15.119:0/1665510664 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d5bfff640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c15a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d53fff640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5c0cb170 0x7f6d5c0cdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:23.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.601+0000 7f6d60d19640 1 -- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 msgr2=0x7f6d5c15a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d60d19640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c15a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d60d19640 1 -- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5c0cb170 msgr2=0x7f6d5c0cdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d60d19640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5c0cb170 0x7f6d5c0cdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d60d19640 1 -- 172.21.15.119:0/1665510664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d4c0094d0 con 0x7f6d5c1447a0 2023-11-08T19:29:23.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d53fff640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5c0cb170 0x7f6d5c0cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:23.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d60d19640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15b8c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b9e0 tx=0x7f6d5400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:23.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5400cb20 con 0x7f6d5c1447a0 2023-11-08T19:29:23.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5400cc80 con 0x7f6d5c1447a0 2023-11-08T19:29:23.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d54014680 con 0x7f6d5c1447a0 2023-11-08T19:29:23.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c0cd700 con 0x7f6d5c1447a0 2023-11-08T19:29:23.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.602+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c0ce440 con 0x7f6d5c1447a0 2023-11-08T19:29:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.604+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c0cdc20 con 0x7f6d5c1447a0 2023-11-08T19:29:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.605+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d5401d3b0 con 0x7f6d5c1447a0 2023-11-08T19:29:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.606+0000 7f6d597fa640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.606+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f6d54089b30 con 0x7f6d5c1447a0 2023-11-08T19:29:23.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.606+0000 7f6d5bfff640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:23.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.607+0000 7f6d5bfff640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c036000 tx=0x7f6d4c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:23.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.612+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d5c0cdc20 con 0x7f6d5c1447a0 2023-11-08T19:29:23.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.844+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d5c074430 con 0x7f6d5c1447a0 2023-11-08T19:29:23.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.846+0000 7f6d597fa640 1 -- 172.21.15.119:0/1665510664 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f6d54019090 con 0x7f6d5c1447a0 2023-11-08T19:29:23.848 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:23.849 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:22.313909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:23.849 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 58 2023-11-08T19:29:23.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 msgr2=0x7f6d3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c036000 tx=0x7f6d4c03c040 comp rx=0 tx=0).stop 2023-11-08T19:29:23.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 msgr2=0x7f6d5c15b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:23.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15b8c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400b9e0 tx=0x7f6d5400bea0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 shutdown_connections 2023-11-08T19:29:23.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6d3c06ae70 0x7f6d3c06d330 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6d5c1447a0 0x7f6d5c15b8c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d5c0cb170 0x7f6d5c0cdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 --2- 172.21.15.119:0/1665510664 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d5c0cad30 0x7f6d5c15a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:23.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.849+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 >> 172.21.15.119:0/1665510664 conn(0x7f6d5c0cb610 msgr2=0x7f6d5c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:23.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.851+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 shutdown_connections 2023-11-08T19:29:23.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:23.851+0000 7f6d6277c640 1 -- 172.21.15.119:0/1665510664 wait complete. 2023-11-08T19:29:23.872 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:25.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-08T19:29:28.873 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:29.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.096+0000 7fb8d9ead640 1 Processor -- start 2023-11-08T19:29:29.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.096+0000 7fb8d9ead640 1 -- start start 2023-11-08T19:29:29.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d9ead640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 0x7fb8d415f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d9ead640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb8d40ad9d0 0x7fb8d406b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d9ead640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d415f8c0 con 0x7fb8d40adf20 2023-11-08T19:29:29.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d9ead640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d4071c70 con 0x7fb8d40ad9d0 2023-11-08T19:29:29.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d9ead640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40ca2e0 con 0x7fb8d40cac30 2023-11-08T19:29:29.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb8d40adf20 0x7fb8d415a910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34862/0 (socket says 172.21.15.119:34862) 2023-11-08T19:29:29.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d3fff640 1 -- 172.21.15.119:0/2970235319 learned_addr learned my addr 172.21.15.119:0/2970235319 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:29.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.098+0000 7fb8d8eab640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 0x7fb8d415f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d96ac640 1 --2- 172.21.15.119:0/2970235319 >> v2:172.21.15.188:3300/0 conn(0x7fb8d40ad9d0 0x7fb8d406b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830009980 0 0) 0x7fb8d415f8c0 con 0x7fb8d40adf20 2023-11-08T19:29:29.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8c4003520 con 0x7fb8d40adf20 2023-11-08T19:29:29.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2382620876 0 0) 0x7fb8c4003520 con 0x7fb8d40adf20 2023-11-08T19:29:29.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 msgr2=0x7fb8d415f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 0x7fb8d415f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 >> v2:172.21.15.188:3300/0 conn(0x7fb8d40ad9d0 msgr2=0x7fb8d406b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.104 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 --2- 172.21.15.119:0/2970235319 >> v2:172.21.15.188:3300/0 conn(0x7fb8d40ad9d0 0x7fb8d406b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40ca5c0 con 0x7fb8d40adf20 2023-11-08T19:29:29.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fb8c0001670 con 0x7fb8d40adf20 2023-11-08T19:29:29.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 >> v1:172.21.15.119:6789/0 conn(0x7fb8d40adf20 legacy=0x7fb8d415a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4003970 0x7fb8c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 0x7fb8c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4071c70 con 0x7fb8c40047c0 2023-11-08T19:29:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40ca2e0 con 0x7fb8c4003970 2023-11-08T19:29:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.099+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c400b980 con 0x7fb8d40cac30 2023-11-08T19:29:29.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d8eab640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4003970 0x7fb8c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d96ac640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 0x7fb8c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d3fff640 1 -- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 msgr2=0x7fb8c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40cac30 0x7fb8c4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d3fff640 1 -- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4003970 msgr2=0x7fb8c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4003970 0x7fb8c4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d3fff640 1 -- 172.21.15.119:0/2970235319 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c4003520 con 0x7fb8c40047c0 2023-11-08T19:29:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.100+0000 7fb8d8eab640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8c4003970 0x7fb8c4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.101+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8c4004c40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c0e0 tx=0x7fb8c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8986d3b2bf4fe1b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:29.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.101+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0005ad0 con 0x7fb8c40047c0 2023-11-08T19:29:29.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.101+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0005c50 con 0x7fb8c40047c0 2023-11-08T19:29:29.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.101+0000 7fb8d37fe640 1 -- 172.21.15.119:0/2970235319 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c002c7f0 con 0x7fb8c40047c0 2023-11-08T19:29:29.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d9ead640 1 -- 172.21.15.119:0/2970235319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 msgr2=0x7fb8c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/2970235319 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8c4004c40 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c0e0 tx=0x7fb8c40057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d96ac640 1 -- 172.21.15.119:0/2970235319 reap_dead start 2023-11-08T19:29:29.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d9ead640 1 -- 172.21.15.119:0/2970235319 shutdown_connections 2023-11-08T19:29:29.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d9ead640 1 -- 172.21.15.119:0/2970235319 >> 172.21.15.119:0/2970235319 conn(0x7fb8d40b0560 msgr2=0x7fb8d40b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:29.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d9ead640 1 -- 172.21.15.119:0/2970235319 shutdown_connections 2023-11-08T19:29:29.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.102+0000 7fb8d9ead640 1 -- 172.21.15.119:0/2970235319 wait complete. 2023-11-08T19:29:29.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.104+0000 7fb8d9ead640 1 Processor -- start 2023-11-08T19:29:29.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.104+0000 7fb8d9ead640 1 -- start start 2023-11-08T19:29:29.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.105+0000 7fb8d9ead640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8d40cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.105+0000 7fb8d9ead640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d41536a0 0x7fb8d4155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.105+0000 7fb8d9ead640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40c62e0 0x7fb8d40c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.105+0000 7fb8d9ead640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d414e9e0 con 0x7fb8c40047c0 2023-11-08T19:29:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.105+0000 7fb8d9ead640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d4155eb0 con 0x7fb8d41536a0 2023-11-08T19:29:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.105+0000 7fb8d9ead640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d40aebb0 con 0x7fb8d40c62e0 2023-11-08T19:29:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8d40cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8d40cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38404/0 (socket says 172.21.15.119:38404) 2023-11-08T19:29:29.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 -- 172.21.15.119:0/1811203568 learned_addr learned my addr 172.21.15.119:0/1811203568 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:29.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d41536a0 0x7fb8d4155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d96ac640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40c62e0 0x7fb8d40c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 -- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40c62e0 msgr2=0x7fb8d40c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40c62e0 0x7fb8d40c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 -- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d41536a0 msgr2=0x7fb8d4155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d41536a0 0x7fb8d4155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 -- 172.21.15.119:0/1811203568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d40ccc20 con 0x7fb8c40047c0 2023-11-08T19:29:29.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.106+0000 7fb8d8eab640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8d40cc640 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8010850 tx=0x7fb8c8010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:29.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.107+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c800eb00 con 0x7fb8c40047c0 2023-11-08T19:29:29.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.107+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c800ec60 con 0x7fb8c40047c0 2023-11-08T19:29:29.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.107+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c801c400 con 0x7fb8c40047c0 2023-11-08T19:29:29.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.107+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d40ccf00 con 0x7fb8c40047c0 2023-11-08T19:29:29.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.107+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d40cd4c0 con 0x7fb8c40047c0 2023-11-08T19:29:29.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.108+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d404ece0 con 0x7fb8c40047c0 2023-11-08T19:29:29.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.109+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8c801c560 con 0x7fb8c40047c0 2023-11-08T19:29:29.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.110+0000 7fb8d17fa640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8b806aea0 0x7fb8b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:29.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.110+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fb8c8089f00 con 0x7fb8c40047c0 2023-11-08T19:29:29.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.110+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8b806aea0 0x7fb8b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:29.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.111+0000 7fb8d3fff640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8b806aea0 0x7fb8b806d360 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0034000 tx=0x7fb8c0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:29.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.115+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8c8054a30 con 0x7fb8c40047c0 2023-11-08T19:29:29.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.342+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8d40b1110 con 0x7fb8c40047c0 2023-11-08T19:29:29.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.343+0000 7fb8d17fa640 1 -- 172.21.15.119:0/1811203568 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4939 (secure 0 0 0) 0x7fb8c8058050 con 0x7fb8c40047c0 2023-11-08T19:29:29.345 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:29.345 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:22.313909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:29.346 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 58 2023-11-08T19:29:29.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8b806aea0 msgr2=0x7fb8b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8b806aea0 0x7fb8b806d360 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0034000 tx=0x7fb8c0007000 comp rx=0 tx=0).stop 2023-11-08T19:29:29.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 msgr2=0x7fb8d40cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:29.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8d40cc640 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8010850 tx=0x7fb8c8010d10 comp rx=0 tx=0).stop 2023-11-08T19:29:29.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 shutdown_connections 2023-11-08T19:29:29.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb8b806aea0 0x7fb8b806d360 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb8d40c62e0 0x7fb8d40c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb8d41536a0 0x7fb8d4155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 --2- 172.21.15.119:0/1811203568 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb8c40047c0 0x7fb8d40cc640 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:29.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 >> 172.21.15.119:0/1811203568 conn(0x7fb8d40b0560 msgr2=0x7fb8d40af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:29.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 shutdown_connections 2023-11-08T19:29:29.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:29.346+0000 7fb8d9ead640 1 -- 172.21.15.119:0/1811203568 wait complete. 2023-11-08T19:29:29.367 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:32.426 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-08T19:29:34.368 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:34.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.589+0000 7f4fc71e9640 1 Processor -- start 2023-11-08T19:29:34.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.589+0000 7f4fc71e9640 1 -- start start 2023-11-08T19:29:34.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc71e9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4fc00adf20 0x7f4fc015a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc71e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00ad9d0 0x7f4fc015ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc71e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc00ca2e0 con 0x7f4fc00cac30 2023-11-08T19:29:34.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc71e9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc0071c70 con 0x7f4fc00adf20 2023-11-08T19:29:34.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc71e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc015b390 con 0x7f4fc00ad9d0 2023-11-08T19:29:34.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc59e6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4fc00adf20 0x7f4fc015a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc69e8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00ad9d0 0x7f4fc015ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc59e6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4fc00adf20 0x7f4fc015a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:34786/0 (socket says 172.21.15.119:34786) 2023-11-08T19:29:34.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.591+0000 7f4fc59e6640 1 -- 172.21.15.119:0/3144229318 learned_addr learned my addr 172.21.15.119:0/3144229318 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:34.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.592+0000 7f4fc59e6640 1 -- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00ad9d0 msgr2=0x7f4fc015ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:34.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.592+0000 7f4fc59e6640 1 --2- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00ad9d0 0x7f4fc015ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.592+0000 7f4fc59e6640 1 -- 172.21.15.119:0/3144229318 >> v1:172.21.15.188:6789/0 conn(0x7f4fc00cac30 legacy=0x7f4fc015f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.592+0000 7f4fc59e6640 1 -- 172.21.15.119:0/3144229318 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00ca5c0 con 0x7f4fc00adf20 2023-11-08T19:29:34.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.594+0000 7f4fc59e6640 1 --2- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00adf20 0x7f4fc015a910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009870 tx=0x7f4fb002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2b0315a50325824 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:34.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.594+0000 7f4fc51e5640 1 -- 172.21.15.119:0/3144229318 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0030840 con 0x7f4fc00adf20 2023-11-08T19:29:34.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.594+0000 7f4fc51e5640 1 -- 172.21.15.119:0/3144229318 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb00309a0 con 0x7f4fc00adf20 2023-11-08T19:29:34.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.594+0000 7f4fc51e5640 1 -- 172.21.15.119:0/3144229318 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb0030d30 con 0x7f4fc00adf20 2023-11-08T19:29:34.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 -- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00adf20 msgr2=0x7f4fc015a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00adf20 0x7f4fc015a910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4fb0009870 tx=0x7f4fb002b980 comp rx=0 tx=0).stop 2023-11-08T19:29:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 -- 172.21.15.119:0/3144229318 shutdown_connections 2023-11-08T19:29:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00adf20 0x7f4fc015a910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/3144229318 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00ad9d0 0x7f4fc015ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 -- 172.21.15.119:0/3144229318 >> 172.21.15.119:0/3144229318 conn(0x7f4fc00b0560 msgr2=0x7f4fc00b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 -- 172.21.15.119:0/3144229318 shutdown_connections 2023-11-08T19:29:34.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.595+0000 7f4fc71e9640 1 -- 172.21.15.119:0/3144229318 wait complete. 2023-11-08T19:29:34.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.597+0000 7f4fc71e9640 1 Processor -- start 2023-11-08T19:29:34.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.597+0000 7f4fc71e9640 1 -- start start 2023-11-08T19:29:34.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc71e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00ad9d0 0x7f4fc01e5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc71e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00adf20 0x7f4fc00c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc71e9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 0x7f4fc0140ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc71e9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01e6d00 con 0x7f4fc00cac30 2023-11-08T19:29:34.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc71e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01e6080 con 0x7f4fc00ad9d0 2023-11-08T19:29:34.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc71e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc01e6360 con 0x7f4fc00adf20 2023-11-08T19:29:34.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 0x7f4fc0140ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 0x7f4fc0140ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38406/0 (socket says 172.21.15.119:38406) 2023-11-08T19:29:34.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 -- 172.21.15.119:0/2887518310 learned_addr learned my addr 172.21.15.119:0/2887518310 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:34.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc59e6640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00adf20 0x7f4fc00c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 -- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00adf20 msgr2=0x7f4fc00c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc61e7640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00ad9d0 0x7f4fc01e5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00adf20 0x7f4fc00c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 -- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00ad9d0 msgr2=0x7f4fc01e5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00ad9d0 0x7f4fc01e5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.598+0000 7f4fc69e8640 1 -- 172.21.15.119:0/2887518310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb00094d0 con 0x7f4fc00cac30 2023-11-08T19:29:34.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.599+0000 7f4fc69e8640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 0x7f4fc0140ee0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800efc0 tx=0x7f4fb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:34.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.599+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8014480 con 0x7f4fc00cac30 2023-11-08T19:29:34.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.599+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb800f040 con 0x7f4fc00cac30 2023-11-08T19:29:34.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.599+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb8014820 con 0x7f4fc00cac30 2023-11-08T19:29:34.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.599+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc01e6640 con 0x7f4fc00cac30 2023-11-08T19:29:34.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.599+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc01e7120 con 0x7f4fc00cac30 2023-11-08T19:29:34.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.601+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc01e6e40 con 0x7f4fc00cac30 2023-11-08T19:29:34.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.601+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fb8014980 con 0x7f4fc00cac30 2023-11-08T19:29:34.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.603+0000 7f4fb6ffd640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4fa406ae70 0x7f4fa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:34.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.603+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f4fb808aa30 con 0x7f4fc00cac30 2023-11-08T19:29:34.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.603+0000 7f4fc61e7640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4fa406ae70 0x7f4fa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:34.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.604+0000 7f4fc61e7640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4fa406ae70 0x7f4fa406d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc001500 tx=0x7f4fbc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:34.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.607+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fc01e6e40 con 0x7f4fc00cac30 2023-11-08T19:29:34.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.835+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fc01e6e40 con 0x7f4fc00cac30 2023-11-08T19:29:34.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.836+0000 7f4fb6ffd640 1 -- 172.21.15.119:0/2887518310 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4939 (secure 0 0 0) 0x7f4fc01e6e40 con 0x7f4fc00cac30 2023-11-08T19:29:34.837 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:34.838 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":58,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:22.313909+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:34.838 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 58 2023-11-08T19:29:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.838+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4fa406ae70 msgr2=0x7f4fa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.838+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4fa406ae70 0x7f4fa406d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc001500 tx=0x7f4fbc009040 comp rx=0 tx=0).stop 2023-11-08T19:29:34.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.838+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 msgr2=0x7f4fc0140ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:34.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.838+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 0x7f4fc0140ee0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4fb800efc0 tx=0x7f4fb800c490 comp rx=0 tx=0).stop 2023-11-08T19:29:34.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 shutdown_connections 2023-11-08T19:29:34.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4fa406ae70 0x7f4fa406d330 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4fc00cac30 0x7f4fc0140ee0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4fc00adf20 0x7f4fc00c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 --2- 172.21.15.119:0/2887518310 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4fc00ad9d0 0x7f4fc01e5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:34.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 >> 172.21.15.119:0/2887518310 conn(0x7f4fc00b0560 msgr2=0x7f4fc006b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:34.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 shutdown_connections 2023-11-08T19:29:34.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:34.839+0000 7f4fc71e9640 1 -- 172.21.15.119:0/2887518310 wait complete. 2023-11-08T19:29:34.859 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:39.040 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-08T19:29:39.860 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:40.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.080+0000 7f00fa4c9640 1 Processor -- start 2023-11-08T19:29:40.082 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.080+0000 7f00fa4c9640 1 -- start start 2023-11-08T19:29:40.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00fa4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 0x7f00f406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00fa4c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f406d4b0 con 0x7f00f40ccde0 2023-11-08T19:29:40.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00fa4c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f4147b80 con 0x7f00f40c9150 2023-11-08T19:29:40.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00fa4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f41619b0 con 0x7f00f406a970 2023-11-08T19:29:40.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00f8a66640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f00f40ccde0 0x7f00f41653f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41220/0 (socket says 172.21.15.119:41220) 2023-11-08T19:29:40.084 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00f8a66640 1 -- 172.21.15.119:0/175588088 learned_addr learned my addr 172.21.15.119:0/175588088 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:40.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.082+0000 7f00f3fff640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 0x7f00f406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379466240 0 0) 0x7f00ec001490 con 0x7f00f40ccde0 2023-11-08T19:29:40.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e0003520 con 0x7f00f40ccde0 2023-11-08T19:29:40.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2671988251 0 0) 0x7f00e0003520 con 0x7f00f40ccde0 2023-11-08T19:29:40.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 msgr2=0x7f00f406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 0x7f00f406cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 >> v1:172.21.15.188:6789/0 conn(0x7f00f40c9150 legacy=0x7f00f40cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4147000 con 0x7f00f40ccde0 2023-11-08T19:29:40.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f00ec001b80 con 0x7f00f40ccde0 2023-11-08T19:29:40.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 >> v1:172.21.15.119:6789/0 conn(0x7f00f40ccde0 legacy=0x7f00f41653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 0x7f00e00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e0003f20 0x7f00e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e000bb10 con 0x7f00e0003f20 2023-11-08T19:29:40.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e000bc50 con 0x7f00e0004d00 2023-11-08T19:29:40.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.083+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00e000be10 con 0x7f00f406a970 2023-11-08T19:29:40.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e0003f20 0x7f00e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00f3fff640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 0x7f00e00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00f8a66640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00eb7fe640 1 -- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 msgr2=0x7f00e00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.091 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00f406a970 0x7f00e00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00eb7fe640 1 -- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e0004d00 msgr2=0x7f00e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00eb7fe640 1 -- 172.21.15.119:0/175588088 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0003520 con 0x7f00e0003f20 2023-11-08T19:29:40.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.084+0000 7f00f8a66640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00e0004d00 0x7f00e0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:40.092 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.085+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e0003f20 0x7f00e00047c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f00dc001750 tx=0x7f00e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c921694e3ca9e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.085+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0052d0 con 0x7f00e0003f20 2023-11-08T19:29:40.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.085+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc02e050 con 0x7f00e0003f20 2023-11-08T19:29:40.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.085+0000 7f00f37fe640 1 -- 172.21.15.119:0/175588088 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc003830 con 0x7f00e0003f20 2023-11-08T19:29:40.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00fa4c9640 1 -- 172.21.15.119:0/175588088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e0003f20 msgr2=0x7f00e00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/175588088 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00e0003f20 0x7f00e00047c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f00dc001750 tx=0x7f00e00057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00f8a66640 1 -- 172.21.15.119:0/175588088 reap_dead start 2023-11-08T19:29:40.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00fa4c9640 1 -- 172.21.15.119:0/175588088 shutdown_connections 2023-11-08T19:29:40.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00fa4c9640 1 -- 172.21.15.119:0/175588088 >> 172.21.15.119:0/175588088 conn(0x7f00f40cb610 msgr2=0x7f00f40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:40.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00fa4c9640 1 -- 172.21.15.119:0/175588088 shutdown_connections 2023-11-08T19:29:40.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.086+0000 7f00fa4c9640 1 -- 172.21.15.119:0/175588088 wait complete. 2023-11-08T19:29:40.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.088+0000 7f00fa4c9640 1 Processor -- start 2023-11-08T19:29:40.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.088+0000 7f00fa4c9640 1 -- start start 2023-11-08T19:29:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00fa4c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e0003f20 0x7f00f41487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00fa4c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 0x7f00f41fa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00fa4c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f41f8110 0x7f00f41f8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00fa4c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4148fc0 con 0x7f00f4162d90 2023-11-08T19:29:40.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00fa4c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f41fab30 con 0x7f00f41f8110 2023-11-08T19:29:40.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00fa4c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4163210 con 0x7f00e0003f20 2023-11-08T19:29:40.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00eb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 0x7f00f41fa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00eb7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 0x7f00f41fa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33404/0 (socket says 172.21.15.119:33404) 2023-11-08T19:29:40.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.089+0000 7f00eb7fe640 1 -- 172.21.15.119:0/1075267378 learned_addr learned my addr 172.21.15.119:0/1075267378 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:40.099 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00f3fff640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e0003f20 0x7f00f41487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00f8a66640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f41f8110 0x7f00f41f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00eb7fe640 1 -- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e0003f20 msgr2=0x7f00f41487b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e0003f20 0x7f00f41487b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00eb7fe640 1 -- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f41f8110 msgr2=0x7f00f41f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f41f8110 0x7f00f41f8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00eb7fe640 1 -- 172.21.15.119:0/1075267378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f41fad60 con 0x7f00f4162d90 2023-11-08T19:29:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00f3fff640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e0003f20 0x7f00f41487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00eb7fe640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 0x7f00f41fa5f0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f00dc001870 tx=0x7f00dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc007640 con 0x7f00f4162d90 2023-11-08T19:29:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc0069e0 con 0x7f00f4162d90 2023-11-08T19:29:40.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc006d90 con 0x7f00f4162d90 2023-11-08T19:29:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.090+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f41fb040 con 0x7f00f4162d90 2023-11-08T19:29:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.091+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f41491d0 con 0x7f00f4162d90 2023-11-08T19:29:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.092+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f4149360 con 0x7f00f4162d90 2023-11-08T19:29:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.092+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00dc0035f0 con 0x7f00f4162d90 2023-11-08T19:29:40.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.093+0000 7f00f17fa640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00d406aea0 0x7f00d406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.093+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7f00dc0ab7b0 con 0x7f00f4162d90 2023-11-08T19:29:40.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.094+0000 7f00f3fff640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00d406aea0 0x7f00d406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.094+0000 7f00f3fff640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00d406aea0 0x7f00d406d360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f00f414ad30 tx=0x7f00e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.098+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f00dc076400 con 0x7f00f4162d90 2023-11-08T19:29:40.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.323+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00f4074430 con 0x7f00f4162d90 2023-11-08T19:29:40.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.324+0000 7f00f17fa640 1 -- 172.21.15.119:0/1075267378 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+4987 (secure 0 0 0) 0x7f00dc079a20 con 0x7f00f4162d90 2023-11-08T19:29:40.326 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:40.326 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":59,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5539,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:38.686234+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7068,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7068":{"gid":7068,"name":"b","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6835/168553637","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":168553637},{"type":"v1","addr":"172.21.15.188:6835","nonce":168553637}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:29:38.686234+0000","export_targets":[1],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:40.326 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 59 2023-11-08T19:29:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.326+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00d406aea0 msgr2=0x7f00d406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.326+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00d406aea0 0x7f00d406d360 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f00f414ad30 tx=0x7f00e4008040 comp rx=0 tx=0).stop 2023-11-08T19:29:40.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.326+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 msgr2=0x7f00f41fa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 0x7f00f41fa5f0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f00dc001870 tx=0x7f00dc001120 comp rx=0 tx=0).stop 2023-11-08T19:29:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 shutdown_connections 2023-11-08T19:29:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f00d406aea0 0x7f00d406d360 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00f41f8110 0x7f00f41f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f00f4162d90 0x7f00f41fa5f0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 --2- 172.21.15.119:0/1075267378 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00e0003f20 0x7f00f41487b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 >> 172.21.15.119:0/1075267378 conn(0x7f00f40cb610 msgr2=0x7f00f40c5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 shutdown_connections 2023-11-08T19:29:40.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.327+0000 7f00fa4c9640 1 -- 172.21.15.119:0/1075267378 wait complete. 2023-11-08T19:29:40.346 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:29:40.347 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:29:40.448 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:29:40.452 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:29:40.453 DEBUG:teuthology.orchestra.run.smithi188:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:29:40.554 INFO:teuthology.orchestra.run.smithi188.stdout:1699471763.94965.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:29:40.558 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471763.94965.core'] 2023-11-08T19:29:40.558 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471763.94965.core, deleting it 2023-11-08T19:29:40.559 DEBUG:teuthology.orchestra.run.smithi188:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471763.94965.core 2023-11-08T19:29:40.658 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:29:40.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.876+0000 7fbf9efd7640 1 Processor -- start 2023-11-08T19:29:40.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.876+0000 7fbf9efd7640 1 -- start start 2023-11-08T19:29:40.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9efd7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf98142f80 0x7fbf980cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9efd7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9815d9f0 0x7fbf980c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9efd7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf980c4960 con 0x7fbf9806cb40 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9efd7640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf98143a40 con 0x7fbf98142f80 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9efd7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9815df60 con 0x7fbf9815d9f0 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf98142f80 0x7fbf980cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbf98142f80 0x7fbf980cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33410/0 (socket says 172.21.15.119:33410) 2023-11-08T19:29:40.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 -- 172.21.15.119:0/1689593129 learned_addr learned my addr 172.21.15.119:0/1689593129 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf8ffff640 1 --2- 172.21.15.119:0/1689593129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9815d9f0 0x7fbf980c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 -- 172.21.15.119:0/1689593129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9815d9f0 msgr2=0x7fbf980c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 --2- 172.21.15.119:0/1689593129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9815d9f0 0x7fbf980c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 -- 172.21.15.119:0/1689593129 >> v1:172.21.15.188:6789/0 conn(0x7fbf9806cb40 legacy=0x7fbf980c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.878+0000 7fbf9cd73640 1 -- 172.21.15.119:0/1689593129 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf9815e240 con 0x7fbf98142f80 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.880+0000 7fbf9cd73640 1 --2- 172.21.15.119:0/1689593129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98142f80 0x7fbf980cc6d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009870 tx=0x7fbf9402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a27b14fff36e642 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.880+0000 7fbf8f7fe640 1 -- 172.21.15.119:0/1689593129 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf940317c0 con 0x7fbf98142f80 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.880+0000 7fbf8f7fe640 1 -- 172.21.15.119:0/1689593129 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf94031920 con 0x7fbf98142f80 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.880+0000 7fbf8f7fe640 1 -- 172.21.15.119:0/1689593129 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf94031cb0 con 0x7fbf98142f80 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 -- 172.21.15.119:0/1689593129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98142f80 msgr2=0x7fbf980cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/1689593129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98142f80 0x7fbf980cc6d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009870 tx=0x7fbf9402b860 comp rx=0 tx=0).stop 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 -- 172.21.15.119:0/1689593129 shutdown_connections 2023-11-08T19:29:40.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/1689593129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9815d9f0 0x7fbf980c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/1689593129 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf98142f80 0x7fbf980cc6d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 -- 172.21.15.119:0/1689593129 >> 172.21.15.119:0/1689593129 conn(0x7fbf980cb610 msgr2=0x7fbf980cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:40.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 -- 172.21.15.119:0/1689593129 shutdown_connections 2023-11-08T19:29:40.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.881+0000 7fbf9efd7640 1 -- 172.21.15.119:0/1689593129 wait complete. 2023-11-08T19:29:40.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.883+0000 7fbf9efd7640 1 Processor -- start 2023-11-08T19:29:40.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.883+0000 7fbf9efd7640 1 -- start start 2023-11-08T19:29:40.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9efd7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9806cb40 0x7fbf980affd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9efd7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf98142f80 0x7fbf980b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9efd7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 0x7fbf980b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9efd7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf980c7b80 con 0x7fbf9815d9f0 2023-11-08T19:29:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9efd7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981301c0 con 0x7fbf98142f80 2023-11-08T19:29:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9efd7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf981304a0 con 0x7fbf9806cb40 2023-11-08T19:29:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 0x7fbf980b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 0x7fbf980b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33422/0 (socket says 172.21.15.119:33422) 2023-11-08T19:29:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 -- 172.21.15.119:0/2802253123 learned_addr learned my addr 172.21.15.119:0/2802253123 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:40.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9cd73640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9806cb40 0x7fbf980affd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf8ffff640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf98142f80 0x7fbf980b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 -- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9806cb40 msgr2=0x7fbf980affd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9806cb40 0x7fbf980affd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 -- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf98142f80 msgr2=0x7fbf980b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:40.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.884+0000 7fbf9d574640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf98142f80 0x7fbf980b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf9d574640 1 -- 172.21.15.119:0/2802253123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940094d0 con 0x7fbf9815d9f0 2023-11-08T19:29:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf9d574640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 0x7fbf980b0d00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000b970 tx=0x7fbf9000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9000cb40 con 0x7fbf9815d9f0 2023-11-08T19:29:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9000cca0 con 0x7fbf9815d9f0 2023-11-08T19:29:40.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf90014680 con 0x7fbf9815d9f0 2023-11-08T19:29:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf98130780 con 0x7fbf9815d9f0 2023-11-08T19:29:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.885+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf980b0510 con 0x7fbf9815d9f0 2023-11-08T19:29:40.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.886+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf98130ca0 con 0x7fbf9815d9f0 2023-11-08T19:29:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.887+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf9001d3b0 con 0x7fbf9815d9f0 2023-11-08T19:29:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.888+0000 7fbf8d7fa640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf8406aea0 0x7fbf8406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.888+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 7120+0+0 (secure 0 0 0) 0x7fbf900899c0 con 0x7fbf9815d9f0 2023-11-08T19:29:40.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.888+0000 7fbf9cd73640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf8406aea0 0x7fbf8406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.889+0000 7fbf9cd73640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf8406aea0 0x7fbf8406d360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbf94036000 tx=0x7fbf9403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:40.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:40.893+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbf98130ca0 con 0x7fbf9815d9f0 2023-11-08T19:29:41.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.159+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7fbf98130ca0 con 0x7fbf9815d9f0 2023-11-08T19:29:41.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.164+0000 7fbf8d7fa640 1 -- 172.21.15.119:0/2802253123 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7068 v60) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbf98130ca0 con 0x7fbf9815d9f0 2023-11-08T19:29:41.165 INFO:teuthology.orchestra.run.smithi119.stderr:failed mds gid 7068 2023-11-08T19:29:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.166+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf8406aea0 msgr2=0x7fbf8406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.166+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf8406aea0 0x7fbf8406d360 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fbf94036000 tx=0x7fbf9403d040 comp rx=0 tx=0).stop 2023-11-08T19:29:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.166+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 msgr2=0x7fbf980b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.166+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 0x7fbf980b0d00 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fbf9000b970 tx=0x7fbf9000be30 comp rx=0 tx=0).stop 2023-11-08T19:29:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 shutdown_connections 2023-11-08T19:29:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbf8406aea0 0x7fbf8406d360 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbf9815d9f0 0x7fbf980b0d00 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbf98142f80 0x7fbf980b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 --2- 172.21.15.119:0/2802253123 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbf9806cb40 0x7fbf980affd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 >> 172.21.15.119:0/2802253123 conn(0x7fbf980cb610 msgr2=0x7fbf98142500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:41.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 shutdown_connections 2023-11-08T19:29:41.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.167+0000 7fbf9efd7640 1 -- 172.21.15.119:0/2802253123 wait complete. 2023-11-08T19:29:41.189 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:29:41.189 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:29:41.190 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:29:41.190 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:29:41.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:29:41.191 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi188 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-08T19:29:41.192 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:29:41.192 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:29:41.203 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:29:41.204 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:41.373 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:29:41.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.390+0000 7f1ec8bc9640 1 Processor -- start 2023-11-08T19:29:41.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.390+0000 7f1ec8bc9640 1 -- start start 2023-11-08T19:29:41.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.391+0000 7f1ec8bc9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ec406a390 0x7f1ec40c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.391+0000 7f1ec8bc9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec40ae410 con 0x7f1ec40cc810 2023-11-08T19:29:41.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec8bc9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec406ba30 con 0x7f1ec415d180 2023-11-08T19:29:41.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec8bc9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec406bcf0 con 0x7f1ec406a390 2023-11-08T19:29:41.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ec406a390 0x7f1ec40c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec259c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1ec406a390 0x7f1ec40c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33432/0 (socket says 172.21.15.119:33432) 2023-11-08T19:29:41.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ebbfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1ec40cc810 0x7f1ec415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41234/0 (socket says 172.21.15.119:41234) 2023-11-08T19:29:41.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec259c640 1 -- 172.21.15.119:0/3995002093 learned_addr learned my addr 172.21.15.119:0/3995002093 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:41.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec259c640 1 -- 172.21.15.119:0/3995002093 >> v1:172.21.15.188:6789/0 conn(0x7f1ec415d180 legacy=0x7f1ec4160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec259c640 1 -- 172.21.15.119:0/3995002093 >> v1:172.21.15.119:6789/0 conn(0x7f1ec40cc810 legacy=0x7f1ec415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec1d9b640 1 -- 172.21.15.119:0/3995002093 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145533119 0 0) 0x7f1ec40ae410 con 0x7f1ec40cc810 2023-11-08T19:29:41.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.392+0000 7f1ec259c640 1 -- 172.21.15.119:0/3995002093 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec406bfd0 con 0x7f1ec406a390 2023-11-08T19:29:41.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.394+0000 7f1ec259c640 1 --2- 172.21.15.119:0/3995002093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec406a390 0x7f1ec40c4460 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94fff8600e5409a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:41.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.394+0000 7f1ec1d9b640 1 -- 172.21.15.119:0/3995002093 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb402f840 con 0x7f1ec406a390 2023-11-08T19:29:41.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.394+0000 7f1ec1d9b640 1 -- 172.21.15.119:0/3995002093 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb402f9a0 con 0x7f1ec406a390 2023-11-08T19:29:41.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.394+0000 7f1ec1d9b640 1 -- 172.21.15.119:0/3995002093 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb402fd30 con 0x7f1ec406a390 2023-11-08T19:29:41.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/3995002093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec406a390 msgr2=0x7f1ec40c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/3995002093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec406a390 0x7f1ec40c4460 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009870 tx=0x7f1eb402ba20 comp rx=0 tx=0).stop 2023-11-08T19:29:41.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/3995002093 shutdown_connections 2023-11-08T19:29:41.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/3995002093 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec406a390 0x7f1ec40c4460 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/3995002093 >> 172.21.15.119:0/3995002093 conn(0x7f1ec40cb610 msgr2=0x7f1ec40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:41.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/3995002093 shutdown_connections 2023-11-08T19:29:41.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.395+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/3995002093 wait complete. 2023-11-08T19:29:41.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.397+0000 7f1ec8bc9640 1 Processor -- start 2023-11-08T19:29:41.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.397+0000 7f1ec8bc9640 1 -- start start 2023-11-08T19:29:41.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec8bc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec406a390 0x7f1ec4155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec8bc9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cc810 0x7f1ec40aeff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec8bc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 0x7f1ec40af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec8bc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40b18d0 con 0x7f1ec415d180 2023-11-08T19:29:41.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec8bc9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40ad640 con 0x7f1ec40cc810 2023-11-08T19:29:41.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec8bc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec40ad920 con 0x7f1ec406a390 2023-11-08T19:29:41.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 0x7f1ec40af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 0x7f1ec40af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33444/0 (socket says 172.21.15.119:33444) 2023-11-08T19:29:41.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 -- 172.21.15.119:0/1301921677 learned_addr learned my addr 172.21.15.119:0/1301921677 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:41.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec259c640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec406a390 0x7f1ec4155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ebbfff640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cc810 0x7f1ec40aeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 -- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec406a390 msgr2=0x7f1ec4155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec406a390 0x7f1ec4155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 -- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cc810 msgr2=0x7f1ec40aeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.398+0000 7f1ec2d9d640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cc810 0x7f1ec40aeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ec2d9d640 1 -- 172.21.15.119:0/1301921677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb40094d0 con 0x7f1ec415d180 2023-11-08T19:29:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ec2d9d640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 0x7f1ec40af530 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c6b0 tx=0x7f1ebc00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003b70 con 0x7f1ec415d180 2023-11-08T19:29:41.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc003cf0 con 0x7f1ec415d180 2023-11-08T19:29:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc003ea0 con 0x7f1ec415d180 2023-11-08T19:29:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec40adc00 con 0x7f1ec415d180 2023-11-08T19:29:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.399+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec40ad2d0 con 0x7f1ec415d180 2023-11-08T19:29:41.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.400+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec406af10 con 0x7f1ec415d180 2023-11-08T19:29:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.401+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ebc005310 con 0x7f1ec415d180 2023-11-08T19:29:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.402+0000 7f1ebaffd640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ea406aea0 0x7f1ea406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:41.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.402+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f1ebc089990 con 0x7f1ec415d180 2023-11-08T19:29:41.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.403+0000 7f1ec259c640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ea406aea0 0x7f1ea406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:41.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.403+0000 7f1ec259c640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ea406aea0 0x7f1ea406d360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4036000 tx=0x7f1eb4008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:41.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.407+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1ebc054440 con 0x7f1ec415d180 2023-11-08T19:29:41.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.635+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ec406c2b0 con 0x7f1ec415d180 2023-11-08T19:29:41.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.636+0000 7f1ebaffd640 1 -- 172.21.15.119:0/1301921677 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4148 (secure 0 0 0) 0x7f1ebc057a60 con 0x7f1ec415d180 2023-11-08T19:29:41.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:41.638 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":61,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:41.171207+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:41.639 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 61 2023-11-08T19:29:41.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ea406aea0 msgr2=0x7f1ea406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ea406aea0 0x7f1ea406d360 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4036000 tx=0x7f1eb4008030 comp rx=0 tx=0).stop 2023-11-08T19:29:41.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 msgr2=0x7f1ec40af530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:41.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 0x7f1ec40af530 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc00c6b0 tx=0x7f1ebc00cb70 comp rx=0 tx=0).stop 2023-11-08T19:29:41.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 shutdown_connections 2023-11-08T19:29:41.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1ea406aea0 0x7f1ea406d360 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1ec415d180 0x7f1ec40af530 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ec40cc810 0x7f1ec40aeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 --2- 172.21.15.119:0/1301921677 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ec406a390 0x7f1ec4155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 >> 172.21.15.119:0/1301921677 conn(0x7f1ec40cb610 msgr2=0x7f1ec40c3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:41.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 shutdown_connections 2023-11-08T19:29:41.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:41.639+0000 7f1ec8bc9640 1 -- 172.21.15.119:0/1301921677 wait complete. 2023-11-08T19:29:41.658 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:29:45.753 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-08T19:29:46.420 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:29:46.419+0000 7f692f4b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:46.421 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:29:46.419+0000 7f692f4b8640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:46.659 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:46.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.877+0000 7f71fd6a9640 1 Processor -- start 2023-11-08T19:29:46.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.877+0000 7f71fd6a9640 1 -- start start 2023-11-08T19:29:46.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.879+0000 7f71fd6a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71f8142f80 0x7f71f80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.879+0000 7f71fd6a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f815d9f0 0x7f71f80c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.879+0000 7f71fd6a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71f806cb40 0x7f71f806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.879+0000 7f71fd6a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f8143a40 con 0x7f71f8142f80 2023-11-08T19:29:46.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.879+0000 7f71fd6a9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f806d510 con 0x7f71f806cb40 2023-11-08T19:29:46.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.879+0000 7f71fd6a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f80c4960 con 0x7f71f815d9f0 2023-11-08T19:29:46.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71f8142f80 0x7f71f80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f71f8142f80 0x7f71f80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33460/0 (socket says 172.21.15.119:33460) 2023-11-08T19:29:46.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 -- 172.21.15.119:0/1943135548 learned_addr learned my addr 172.21.15.119:0/1943135548 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 -- 172.21.15.119:0/1943135548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f815d9f0 msgr2=0x7f71f80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:46.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1943135548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f815d9f0 0x7f71f80c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 -- 172.21.15.119:0/1943135548 >> v2:172.21.15.188:3300/0 conn(0x7f71f806cb40 msgr2=0x7f71f806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1943135548 >> v2:172.21.15.188:3300/0 conn(0x7f71f806cb40 0x7f71f806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.880+0000 7f71f6ffd640 1 -- 172.21.15.119:0/1943135548 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f815df60 con 0x7f71f8142f80 2023-11-08T19:29:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.881+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1943135548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8142f80 0x7f71f80cc6d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f71f80cb550 tx=0x7f71e802e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=599b3336521313c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:46.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.881+0000 7f71f67fc640 1 -- 172.21.15.119:0/1943135548 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e802ee00 con 0x7f71f8142f80 2023-11-08T19:29:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71f67fc640 1 -- 172.21.15.119:0/1943135548 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8036440 con 0x7f71f8142f80 2023-11-08T19:29:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71f67fc640 1 -- 172.21.15.119:0/1943135548 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80367d0 con 0x7f71f8142f80 2023-11-08T19:29:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1943135548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8142f80 msgr2=0x7f71f80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1943135548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8142f80 0x7f71f80cc6d0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f71f80cb550 tx=0x7f71e802e050 comp rx=0 tx=0).stop 2023-11-08T19:29:46.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1943135548 shutdown_connections 2023-11-08T19:29:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1943135548 >> v2:172.21.15.188:3300/0 conn(0x7f71f806cb40 0x7f71f806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1943135548 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f815d9f0 0x7f71f80c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1943135548 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f8142f80 0x7f71f80cc6d0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1943135548 >> 172.21.15.119:0/1943135548 conn(0x7f71f80cb610 msgr2=0x7f71f80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:46.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1943135548 shutdown_connections 2023-11-08T19:29:46.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.882+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1943135548 wait complete. 2023-11-08T19:29:46.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.884+0000 7f71fd6a9640 1 Processor -- start 2023-11-08T19:29:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.885+0000 7f71fd6a9640 1 -- start start 2023-11-08T19:29:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71fd6a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f806cb40 0x7f71f8140890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71fd6a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f8142f80 0x7f71f813e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71fd6a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 0x7f71f813ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71fd6a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8141170 con 0x7f71f815d9f0 2023-11-08T19:29:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71fd6a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8141450 con 0x7f71f806cb40 2023-11-08T19:29:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71fd6a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f8141730 con 0x7f71f8142f80 2023-11-08T19:29:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 0x7f71f813ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71f77fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 0x7f71f813ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33472/0 (socket says 172.21.15.119:33472) 2023-11-08T19:29:46.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71f77fe640 1 -- 172.21.15.119:0/1631673263 learned_addr learned my addr 172.21.15.119:0/1631673263 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.886+0000 7f71ee7fc640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f8142f80 0x7f71f813e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f806cb40 0x7f71f8140890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f77fe640 1 -- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f8142f80 msgr2=0x7f71f813e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f77fe640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f8142f80 0x7f71f813e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f77fe640 1 -- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f806cb40 msgr2=0x7f71f8140890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f77fe640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f806cb40 0x7f71f8140890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f77fe640 1 -- 172.21.15.119:0/1631673263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e800b770 con 0x7f71f815d9f0 2023-11-08T19:29:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71ee7fc640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f8142f80 0x7f71f813e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f806cb40 0x7f71f8140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:46.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71f77fe640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 0x7f71f813ed30 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b410 tx=0x7f71f000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011830 con 0x7f71f815d9f0 2023-11-08T19:29:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f00119b0 con 0x7f71f815d9f0 2023-11-08T19:29:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0011d60 con 0x7f71f815d9f0 2023-11-08T19:29:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f8141a10 con 0x7f71f815d9f0 2023-11-08T19:29:46.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.887+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f8140dd0 con 0x7f71f815d9f0 2023-11-08T19:29:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.889+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f80c6f90 con 0x7f71f815d9f0 2023-11-08T19:29:46.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.889+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f71f001a400 con 0x7f71f815d9f0 2023-11-08T19:29:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.890+0000 7f71effff640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71d806ae70 0x7f71d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.890+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f71f0089d20 con 0x7f71f815d9f0 2023-11-08T19:29:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.891+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71d806ae70 0x7f71d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.891+0000 7f71f6ffd640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71d806ae70 0x7f71d806d330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f71e8038000 tx=0x7f71e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:46.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:46.895+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71f80c6f90 con 0x7f71f815d9f0 2023-11-08T19:29:47.120 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.119+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f80c6f90 con 0x7f71f815d9f0 2023-11-08T19:29:47.121 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.120+0000 7f71effff640 1 -- 172.21.15.119:0/1631673263 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f71f80c6f90 con 0x7f71f815d9f0 2023-11-08T19:29:47.121 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:47.122 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:44.218777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:47.122 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 65 2023-11-08T19:29:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71d806ae70 msgr2=0x7f71d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71d806ae70 0x7f71d806d330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f71e8038000 tx=0x7f71e8008040 comp rx=0 tx=0).stop 2023-11-08T19:29:47.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 msgr2=0x7f71f813ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 0x7f71f813ed30 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f71f000b410 tx=0x7f71f000b8d0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 shutdown_connections 2023-11-08T19:29:47.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f71d806ae70 0x7f71d806d330 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f71f815d9f0 0x7f71f813ed30 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71f8142f80 0x7f71f813e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 --2- 172.21.15.119:0/1631673263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71f806cb40 0x7f71f8140890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 >> 172.21.15.119:0/1631673263 conn(0x7f71f80cb610 msgr2=0x7f71f8070080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 shutdown_connections 2023-11-08T19:29:47.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.122+0000 7f71fd6a9640 1 -- 172.21.15.119:0/1631673263 wait complete. 2023-11-08T19:29:47.143 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:29:47.143 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:47.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.348+0000 7fbb590d4640 1 Processor -- start 2023-11-08T19:29:47.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.349+0000 7fbb590d4640 1 -- start start 2023-11-08T19:29:47.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.350+0000 7fbb590d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb540cfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.350+0000 7fbb590d4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5415b6b0 con 0x7fbb540cded0 2023-11-08T19:29:47.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.350+0000 7fbb590d4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb540cc980 con 0x7fbb5415b2d0 2023-11-08T19:29:47.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.350+0000 7fbb590d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5406eb20 con 0x7fbb540caf40 2023-11-08T19:29:47.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.350+0000 7fbb5259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbb540cded0 0x7fbb540cf550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41250/0 (socket says 172.21.15.119:41250) 2023-11-08T19:29:47.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.350+0000 7fbb5259c640 1 -- 172.21.15.119:0/4139760858 learned_addr learned my addr 172.21.15.119:0/4139760858 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:47.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb5359e640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb540cfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285752177 0 0) 0x7fbb5415b6b0 con 0x7fbb540cded0 2023-11-08T19:29:47.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb40003520 con 0x7fbb540cded0 2023-11-08T19:29:47.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3579239683 0 0) 0x7fbb40003520 con 0x7fbb540cded0 2023-11-08T19:29:47.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 msgr2=0x7fbb540cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb540cfc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 >> v1:172.21.15.188:6789/0 conn(0x7fbb5415b2d0 legacy=0x7fbb540cd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540ccc60 con 0x7fbb540cded0 2023-11-08T19:29:47.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fbb3c001670 con 0x7fbb540cded0 2023-11-08T19:29:47.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 >> v1:172.21.15.119:6789/0 conn(0x7fbb540cded0 legacy=0x7fbb540cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.351+0000 7fbb51d9b640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb40004d00 0x7fbb40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5406eb20 con 0x7fbb40003f70 2023-11-08T19:29:47.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4000bb70 con 0x7fbb40004d00 2023-11-08T19:29:47.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4000bd30 con 0x7fbb540caf40 2023-11-08T19:29:47.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb5259c640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb52d9d640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb5259c640 1 -- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 msgr2=0x7fbb400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb5259c640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb400039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb5259c640 1 -- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb40004d00 msgr2=0x7fbb40005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:47.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb5259c640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb40004d00 0x7fbb40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb5259c640 1 -- 172.21.15.119:0/4139760858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40003520 con 0x7fbb40003f70 2023-11-08T19:29:47.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.352+0000 7fbb52d9d640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540caf40 0x7fbb400039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:47.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.353+0000 7fbb5259c640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb400047c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c0e0 tx=0x7fbb400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47b679651adf165a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c005ad0 con 0x7fbb40003f70 2023-11-08T19:29:47.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c005c30 con 0x7fbb40003f70 2023-11-08T19:29:47.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb51d9b640 1 -- 172.21.15.119:0/4139760858 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c02ca30 con 0x7fbb40003f70 2023-11-08T19:29:47.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb590d4640 1 -- 172.21.15.119:0/4139760858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 msgr2=0x7fbb400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb590d4640 1 --2- 172.21.15.119:0/4139760858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb400047c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c0e0 tx=0x7fbb400057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb5359e640 1 -- 172.21.15.119:0/4139760858 reap_dead start 2023-11-08T19:29:47.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb590d4640 1 -- 172.21.15.119:0/4139760858 shutdown_connections 2023-11-08T19:29:47.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb590d4640 1 -- 172.21.15.119:0/4139760858 >> 172.21.15.119:0/4139760858 conn(0x7fbb540d0db0 msgr2=0x7fbb540d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb590d4640 1 -- 172.21.15.119:0/4139760858 shutdown_connections 2023-11-08T19:29:47.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.354+0000 7fbb590d4640 1 -- 172.21.15.119:0/4139760858 wait complete. 2023-11-08T19:29:47.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.356+0000 7fbb590d4640 1 Processor -- start 2023-11-08T19:29:47.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.356+0000 7fbb590d4640 1 -- start start 2023-11-08T19:29:47.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb590d4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb540b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb590d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb541f2640 0x7fbb540c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb590d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540c9f90 0x7fbb540b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb590d4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540ca440 con 0x7fbb40003f70 2023-11-08T19:29:47.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb590d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540cada0 con 0x7fbb541f2640 2023-11-08T19:29:47.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb590d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb540b3100 con 0x7fbb540c9f90 2023-11-08T19:29:47.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb52d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb540b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb52d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb540b2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33476/0 (socket says 172.21.15.119:33476) 2023-11-08T19:29:47.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.357+0000 7fbb52d9d640 1 -- 172.21.15.119:0/1401579327 learned_addr learned my addr 172.21.15.119:0/1401579327 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:47.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb5359e640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540c9f90 0x7fbb540b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb52d9d640 1 -- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540c9f90 msgr2=0x7fbb540b0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb52d9d640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540c9f90 0x7fbb540b0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb52d9d640 1 -- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb541f2640 msgr2=0x7fbb540c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb52d9d640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb541f2640 0x7fbb540c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb52d9d640 1 -- 172.21.15.119:0/1401579327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb540b33e0 con 0x7fbb40003f70 2023-11-08T19:29:47.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb52d9d640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb540b2a40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400dae0 tx=0x7fbb440020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb44003df0 con 0x7fbb40003f70 2023-11-08T19:29:47.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4400b710 con 0x7fbb40003f70 2023-11-08T19:29:47.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400bac0 con 0x7fbb40003f70 2023-11-08T19:29:47.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb540b39f0 con 0x7fbb40003f70 2023-11-08T19:29:47.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.358+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb540b0560 con 0x7fbb40003f70 2023-11-08T19:29:47.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.359+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb540b3f10 con 0x7fbb40003f70 2023-11-08T19:29:47.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.360+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb4400bc20 con 0x7fbb40003f70 2023-11-08T19:29:47.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.362+0000 7fbb4b7fe640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbb3406aea0 0x7fbb3406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:47.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.362+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fbb44089f70 con 0x7fbb40003f70 2023-11-08T19:29:47.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.362+0000 7fbb5259c640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbb3406aea0 0x7fbb3406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:47.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.362+0000 7fbb5259c640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbb3406aea0 0x7fbb3406d360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c034000 tx=0x7fbb3c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:47.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.366+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb44054990 con 0x7fbb40003f70 2023-11-08T19:29:47.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.575+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb540b0720 con 0x7fbb40003f70 2023-11-08T19:29:47.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.576+0000 7fbb4b7fe640 1 -- 172.21.15.119:0/1401579327 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fbb44057fb0 con 0x7fbb40003f70 2023-11-08T19:29:47.578 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:47.578 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:44.218777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:47.579 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 65 2023-11-08T19:29:47.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbb3406aea0 msgr2=0x7fbb3406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbb3406aea0 0x7fbb3406d360 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c034000 tx=0x7fbb3c007000 comp rx=0 tx=0).stop 2023-11-08T19:29:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 msgr2=0x7fbb540b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb540b2a40 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400dae0 tx=0x7fbb440020f0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 shutdown_connections 2023-11-08T19:29:47.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbb3406aea0 0x7fbb3406d360 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb540c9f90 0x7fbb540b0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb541f2640 0x7fbb540c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 --2- 172.21.15.119:0/1401579327 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbb40003f70 0x7fbb540b2a40 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 >> 172.21.15.119:0/1401579327 conn(0x7fbb540d0db0 msgr2=0x7fbb540c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:47.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 shutdown_connections 2023-11-08T19:29:47.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:47.579+0000 7fbb590d4640 1 -- 172.21.15.119:0/1401579327 wait complete. 2023-11-08T19:29:47.999 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:29:44.218777+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 97, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5539, 'mds_1': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5539': {'gid': 5539, 'name': 'a', 'rank': 0, 'incarnation': 61, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.119:6837/3498735251', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 3498735251}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 3498735251}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7065': {'gid': 7065, 'name': 'd', 'rank': 1, 'incarnation': 55, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6834/4246462419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4246462419}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4246462419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:29:48.000 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:29:48.000 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T19:29:48.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.219+0000 7f51f3090640 1 Processor -- start 2023-11-08T19:29:48.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.220+0000 7f51f3090640 1 -- start start 2023-11-08T19:29:48.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f3090640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 0x7f51ec0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f3090640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f51ec06cb40 0x7f51ec06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f3090640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0c4960 con 0x7f51ec142f80 2023-11-08T19:29:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f3090640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec143a40 con 0x7f51ec06cb40 2023-11-08T19:29:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f3090640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ca3e0 con 0x7f51ec15d9f0 2023-11-08T19:29:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f0e2c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f51ec142f80 0x7f51ec0c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41252/0 (socket says 172.21.15.119:41252) 2023-11-08T19:29:48.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.221+0000 7f51f0e2c640 1 -- 172.21.15.119:0/800147559 learned_addr learned my addr 172.21.15.119:0/800147559 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:48.225 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262437207 0 0) 0x7f51ec0c4960 con 0x7f51ec142f80 2023-11-08T19:29:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51dc005730 con 0x7f51ec142f80 2023-11-08T19:29:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51ebfff640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 0x7f51ec0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51f162d640 1 --2- 172.21.15.119:0/800147559 >> v2:172.21.15.188:3300/0 conn(0x7f51ec06cb40 0x7f51ec06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3000020530 0 0) 0x7f51dc005730 con 0x7f51ec142f80 2023-11-08T19:29:48.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 msgr2=0x7f51ec0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 0x7f51ec0cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 >> v2:172.21.15.188:3300/0 conn(0x7f51ec06cb40 msgr2=0x7f51ec06cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 --2- 172.21.15.119:0/800147559 >> v2:172.21.15.188:3300/0 conn(0x7f51ec06cb40 0x7f51ec06cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec15df60 con 0x7f51ec142f80 2023-11-08T19:29:48.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f51e0001670 con 0x7f51ec142f80 2023-11-08T19:29:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 >> v1:172.21.15.119:6789/0 conn(0x7f51ec142f80 legacy=0x7f51ec0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc005b80 0x7f51dc005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 0x7f51dc0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51dc006f10 0x7f51dc007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec143a40 con 0x7f51dc006f10 2023-11-08T19:29:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ca3e0 con 0x7f51dc005b80 2023-11-08T19:29:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.222+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc00dcf0 con 0x7f51ec15d9f0 2023-11-08T19:29:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.223+0000 7f51f162d640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51dc006f10 0x7f51dc007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.223+0000 7f51f162d640 1 -- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 msgr2=0x7f51dc0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.223+0000 7f51f162d640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51ec15d9f0 0x7f51dc0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.223+0000 7f51f162d640 1 -- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc005b80 msgr2=0x7f51dc005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.223+0000 7f51f162d640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51dc005b80 0x7f51dc005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.223+0000 7f51f162d640 1 -- 172.21.15.119:0/800147559 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc005730 con 0x7f51dc006f10 2023-11-08T19:29:48.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.224+0000 7f51f162d640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51dc006f10 0x7f51dc007390 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f51e4007360 tx=0x7f51dc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701f7d9a8662196a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.224+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e402ccf0 con 0x7f51dc006f10 2023-11-08T19:29:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.224+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e402ce50 con 0x7f51dc006f10 2023-11-08T19:29:48.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.224+0000 7f51eb7fe640 1 -- 172.21.15.119:0/800147559 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e4034660 con 0x7f51dc006f10 2023-11-08T19:29:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f3090640 1 -- 172.21.15.119:0/800147559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51dc006f10 msgr2=0x7f51dc007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f3090640 1 --2- 172.21.15.119:0/800147559 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51dc006f10 0x7f51dc007390 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f51e4007360 tx=0x7f51dc007a00 comp rx=0 tx=0).stop 2023-11-08T19:29:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f162d640 1 -- 172.21.15.119:0/800147559 reap_dead start 2023-11-08T19:29:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f3090640 1 -- 172.21.15.119:0/800147559 shutdown_connections 2023-11-08T19:29:48.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f3090640 1 -- 172.21.15.119:0/800147559 >> 172.21.15.119:0/800147559 conn(0x7f51ec0cb610 msgr2=0x7f51ec0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f3090640 1 -- 172.21.15.119:0/800147559 shutdown_connections 2023-11-08T19:29:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.225+0000 7f51f3090640 1 -- 172.21.15.119:0/800147559 wait complete. 2023-11-08T19:29:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.227+0000 7f51f3090640 1 Processor -- start 2023-11-08T19:29:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.227+0000 7f51f3090640 1 -- start start 2023-11-08T19:29:48.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51f3090640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc006f10 0x7f51ec13d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51f3090640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 0x7f51ec0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51f3090640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51ec06bd80 0x7f51ec06c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51f3090640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec06c740 con 0x7f51ec0b3820 2023-11-08T19:29:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51f3090640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0c5c70 con 0x7f51ec06bd80 2023-11-08T19:29:48.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51f3090640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0b42e0 con 0x7f51dc006f10 2023-11-08T19:29:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 0x7f51ec0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 0x7f51ec0b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33504/0 (socket says 172.21.15.119:33504) 2023-11-08T19:29:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.228+0000 7f51ebfff640 1 -- 172.21.15.119:0/3839023300 learned_addr learned my addr 172.21.15.119:0/3839023300 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc006f10 0x7f51ec13d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51f162d640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51ec06bd80 0x7f51ec06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51ebfff640 1 -- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc006f10 msgr2=0x7f51ec13d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51ebfff640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc006f10 0x7f51ec13d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51ebfff640 1 -- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51ec06bd80 msgr2=0x7f51ec06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51ebfff640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51ec06bd80 0x7f51ec06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51ebfff640 1 -- 172.21.15.119:0/3839023300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0b45c0 con 0x7f51ec0b3820 2023-11-08T19:29:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51f162d640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51ec06bd80 0x7f51ec06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51ebfff640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 0x7f51ec0b3c80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f51d800d800 tx=0x7f51d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51e97fa640 1 -- 172.21.15.119:0/3839023300 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800b6c0 con 0x7f51ec0b3820 2023-11-08T19:29:48.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51e97fa640 1 -- 172.21.15.119:0/3839023300 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800b820 con 0x7f51ec0b3820 2023-11-08T19:29:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec0c5f10 con 0x7f51ec0b3820 2023-11-08T19:29:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.229+0000 7f51e97fa640 1 -- 172.21.15.119:0/3839023300 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800bbb0 con 0x7f51ec0b3820 2023-11-08T19:29:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.230+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec13db30 con 0x7f51ec0b3820 2023-11-08T19:29:48.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.232+0000 7f51e97fa640 1 -- 172.21.15.119:0/3839023300 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51d800bd10 con 0x7f51ec0b3820 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.232+0000 7f51e97fa640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f51d006ae70 0x7f51d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.233+0000 7f51e97fa640 1 -- 172.21.15.119:0/3839023300 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f51d8089e40 con 0x7f51ec0b3820 2023-11-08T19:29:48.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.233+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f51d006ae70 0x7f51d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.233+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f51d006ae70 0x7f51d006d330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f51e0000f60 tx=0x7f51e0001690 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.234+0000 7f51f5c29740 1 Processor -- start 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.234+0000 7f51f5c29740 1 -- start start 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a39c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3cc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 0x55599a3cc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f5c29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a30a7e0 con 0x55599a39c710 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a307ed0 con 0x55599a3c92d0 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55599a3037c0 con 0x55599a3c96b0 2023-11-08T19:29:48.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 0x55599a3cc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 0x55599a3cc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33514/0 (socket says 172.21.15.119:33514) 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 -- 172.21.15.119:0/4229935205 learned_addr learned my addr 172.21.15.119:0/4229935205 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a39c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51ebfff640 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 -- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 msgr2=0x55599a3cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 -- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 msgr2=0x55599a39c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a39c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.235+0000 7f51f162d640 1 -- 172.21.15.119:0/4229935205 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55599a2d6b10 con 0x55599a39c710 2023-11-08T19:29:48.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.236+0000 7f51f162d640 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 0x55599a3cc5a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f51e4037040 tx=0x7f51e402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfccb93a73f9f076 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.238+0000 7f51d67fc640 1 -- 172.21.15.119:0/4229935205 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e403e470 con 0x55599a39c710 2023-11-08T19:29:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.238+0000 7f51d67fc640 1 -- 172.21.15.119:0/4229935205 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e403e5d0 con 0x55599a39c710 2023-11-08T19:29:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.238+0000 7f51d67fc640 1 -- 172.21.15.119:0/4229935205 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e403e940 con 0x55599a39c710 2023-11-08T19:29:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 -- 172.21.15.119:0/4229935205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 msgr2=0x55599a3cc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 0x55599a3cc5a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f51e4037040 tx=0x7f51e402e000 comp rx=0 tx=0).stop 2023-11-08T19:29:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 -- 172.21.15.119:0/4229935205 shutdown_connections 2023-11-08T19:29:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a39c710 0x55599a3cc5a0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 --2- 172.21.15.119:0/4229935205 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a39c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 -- 172.21.15.119:0/4229935205 >> 172.21.15.119:0/4229935205 conn(0x55599a39e4e0 msgr2=0x55599a3e7060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 -- 172.21.15.119:0/4229935205 shutdown_connections 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.239+0000 7f51f5c29740 1 -- 172.21.15.119:0/4229935205 wait complete. 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.240+0000 7f51f5c29740 1 -- start start 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.240+0000 7f51f5c29740 1 Processor -- start 2023-11-08T19:29:48.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c92d0 0x55599a3ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a3e09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a2a0c40 con 0x55599a3c96b0 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a2a08e0 con 0x55599a3ceff0 2023-11-08T19:29:48.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55599a066f20 con 0x55599a3c92d0 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51f5c29740 10 client.0 _ll_register_callbacks cb 0x55599a2c2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a3e09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a3e09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33522/0 (socket says 172.21.15.119:33522) 2023-11-08T19:29:48.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.241+0000 7f51ebfff640 1 -- 172.21.15.119:0/2419098204 learned_addr learned my addr 172.21.15.119:0/2419098204 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51f162d640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51ebfff640 1 -- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c92d0 msgr2=0x55599a3ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51ebfff640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c92d0 0x55599a3ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51ebfff640 1 -- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 msgr2=0x55599a3e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51ebfff640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3e0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51ebfff640 1 -- 172.21.15.119:0/2419098204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc005730 con 0x55599a3c96b0 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51f162d640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51ebfff640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a3e09d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f51d800b4b0 tx=0x7f51d8058d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ac3045b67a2d81 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51cb7fe640 1 -- 172.21.15.119:0/2419098204 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8060390 con 0x55599a3c96b0 2023-11-08T19:29:48.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51cb7fe640 1 -- 172.21.15.119:0/2419098204 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f51d80604f0 con 0x55599a3c96b0 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55599a2d7c80 con 0x55599a3c96b0 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51cb7fe640 1 -- 172.21.15.119:0/2419098204 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d805c170 con 0x55599a3c96b0 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51f5c29740 10 client.7434 fetch_fsmap learned FSMap version 65 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.242+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55599a301e40 con 0x55599a3c96b0 2023-11-08T19:29:48.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51cb7fe640 1 -- 172.21.15.119:0/2419098204 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55599a2d7c80 con 0x55599a3c96b0 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51cb7fe640 1 -- 172.21.15.119:0/2419098204 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f51d805c2d0 con 0x55599a3c96b0 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f5c29740 10 client.7434 fetch_fsmap finished waiting for FSMap version 65 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f5c29740 10 client.7434 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f5c29740 20 client.7434 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:48.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f5c29740 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b7ed0 0x55599a3b82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f5c29740 4 client.7434 mds_command: new command op to 5539 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 --> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55599a2fc7a0 con 0x55599a3b7ed0 2023-11-08T19:29:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.243+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b7ed0 0x55599a3b82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.244+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b7ed0 0x55599a3b82b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3987242cc8cca50 server_cookie=28e97c264e06023f in_seq=0 out_seq=0 2023-11-08T19:29:48.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.244+0000 7f51cb7fe640 10 client.7434 ms_handle_connect on v2:172.21.15.119:6835/3498735251 2023-11-08T19:29:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.244+0000 7f51cb7fe640 1 -- 172.21.15.119:0/2419098204 <== mds.0 v2:172.21.15.119:6835/3498735251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55599a2fc7a0 con 0x55599a3b7ed0 2023-11-08T19:29:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.244+0000 7f51cb7fe640 10 client.7434 handle_command_reply: tid=0 2023-11-08T19:29:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 client.7434 shutdown 2023-11-08T19:29:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 msgr2=0x55599a3e09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a3e09d0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f51d800b4b0 tx=0x7f51d8058d60 comp rx=0 tx=0).stop 2023-11-08T19:29:48.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 shutdown_connections 2023-11-08T19:29:48.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b7ed0 0x55599a3b82b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a3e09d0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 --2- 172.21.15.119:0/2419098204 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c92d0 0x55599a3ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51cb7fe640 0 client.7434 ms_handle_reset on v2:172.21.15.119:6835/3498735251 2023-11-08T19:29:48.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 >> 172.21.15.119:0/2419098204 conn(0x55599a39c710 msgr2=0x55599a3a0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 shutdown_connections 2023-11-08T19:29:48.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 1 -- 172.21.15.119:0/2419098204 wait complete. 2023-11-08T19:29:48.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.245+0000 7f51f5c29740 20 client.7434 trim_cache size 0 max 16384 2023-11-08T19:29:48.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.262+0000 7f51f5c29740 1 Processor -- start 2023-11-08T19:29:48.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.262+0000 7f51f5c29740 1 -- start start 2023-11-08T19:29:48.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 0x55599a3dfd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3e0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3cd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f5c29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a3d05e0 con 0x55599a3c92d0 2023-11-08T19:29:48.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a2cf3b0 con 0x55599a3ceff0 2023-11-08T19:29:48.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55599a3c8e00 con 0x55599a3c96b0 2023-11-08T19:29:48.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f0e2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 0x55599a3dfd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f0e2c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 0x55599a3dfd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33526/0 (socket says 172.21.15.119:33526) 2023-11-08T19:29:48.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.263+0000 7f51f0e2c640 1 -- 172.21.15.119:0/641653085 learned_addr learned my addr 172.21.15.119:0/641653085 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51ebfff640 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3e0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f0e2c640 1 -- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 msgr2=0x55599a3e0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f162d640 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3e0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f0e2c640 1 -- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 msgr2=0x55599a3cd050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f0e2c640 1 -- 172.21.15.119:0/641653085 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55599a29fb90 con 0x55599a3c92d0 2023-11-08T19:29:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 0x55599a3dfd00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f51e000ead0 tx=0x7f51e0017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff021679c82427a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51d6ffd640 1 -- 172.21.15.119:0/641653085 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00234d0 con 0x55599a3c92d0 2023-11-08T19:29:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51d6ffd640 1 -- 172.21.15.119:0/641653085 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e0023630 con 0x55599a3c92d0 2023-11-08T19:29:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.264+0000 7f51d6ffd640 1 -- 172.21.15.119:0/641653085 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51e00237a0 con 0x55599a3c92d0 2023-11-08T19:29:48.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 -- 172.21.15.119:0/641653085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 msgr2=0x55599a3dfd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 0x55599a3dfd00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f51e000ead0 tx=0x7f51e0017040 comp rx=0 tx=0).stop 2023-11-08T19:29:48.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 -- 172.21.15.119:0/641653085 shutdown_connections 2023-11-08T19:29:48.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3ceff0 0x55599a3cd050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3c96b0 0x55599a3e0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 --2- 172.21.15.119:0/641653085 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c92d0 0x55599a3dfd00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 -- 172.21.15.119:0/641653085 >> 172.21.15.119:0/641653085 conn(0x55599a39c710 msgr2=0x55599a3e7940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 -- 172.21.15.119:0/641653085 shutdown_connections 2023-11-08T19:29:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.265+0000 7f51f5c29740 1 -- 172.21.15.119:0/641653085 wait complete. 2023-11-08T19:29:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.266+0000 7f51f5c29740 1 -- start start 2023-11-08T19:29:48.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.266+0000 7f51f5c29740 1 Processor -- start 2023-11-08T19:29:48.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a3cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a405500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3ceff0 0x55599a3a1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a405f80 con 0x55599a3c96b0 2023-11-08T19:29:48.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55599a409970 con 0x55599a3c92d0 2023-11-08T19:29:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55599a3e04e0 con 0x55599a3ceff0 2023-11-08T19:29:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51f5c29740 10 client.0 _ll_register_callbacks cb 0x7f51d80548e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a405500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51ebfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a405500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33528/0 (socket says 172.21.15.119:33528) 2023-11-08T19:29:48.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.267+0000 7f51ebfff640 1 -- 172.21.15.119:0/3321022721 learned_addr learned my addr 172.21.15.119:0/3321022721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51ebfff640 1 -- 172.21.15.119:0/3321022721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3ceff0 msgr2=0x55599a3a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51ebfff640 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3ceff0 0x55599a3a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51ebfff640 1 -- 172.21.15.119:0/3321022721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 msgr2=0x55599a3cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51ebfff640 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a3cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51ebfff640 1 -- 172.21.15.119:0/3321022721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e001d040 con 0x55599a3c96b0 2023-11-08T19:29:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51ebfff640 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a405500 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f51d8058460 tx=0x7f51d8016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27506162ff127b3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51cb7fe640 1 -- 172.21.15.119:0/3321022721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d80605a0 con 0x55599a3c96b0 2023-11-08T19:29:48.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51cb7fe640 1 -- 172.21.15.119:0/3321022721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f51d8054510 con 0x55599a3c96b0 2023-11-08T19:29:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51e403e5d0 con 0x55599a3c96b0 2023-11-08T19:29:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51cb7fe640 1 -- 172.21.15.119:0/3321022721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51d8009400 con 0x55599a3c96b0 2023-11-08T19:29:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51f5c29740 10 client.7440 fetch_fsmap learned FSMap version 65 2023-11-08T19:29:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55599a2c5180 con 0x55599a3c96b0 2023-11-08T19:29:48.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.268+0000 7f51cb7fe640 1 -- 172.21.15.119:0/3321022721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51e403e5d0 con 0x55599a3c96b0 2023-11-08T19:29:48.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51cb7fe640 1 -- 172.21.15.119:0/3321022721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f51d805c170 con 0x55599a3c96b0 2023-11-08T19:29:48.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f5c29740 10 client.7440 fetch_fsmap finished waiting for FSMap version 65 2023-11-08T19:29:48.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f5c29740 10 client.7440 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:29:48.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f5c29740 20 client.7440 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:48.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f5c29740 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b85d0 0x55599a3b89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f5c29740 4 client.7440 mds_command: new command op to 5539 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:48.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 --> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55599a2fc4c0 con 0x55599a3b85d0 2023-11-08T19:29:48.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b85d0 0x55599a3b89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51f0e2c640 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b85d0 0x55599a3b89b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d3f50ce7ade581 server_cookie=cc1f43348070ac8b in_seq=0 out_seq=0 2023-11-08T19:29:48.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.269+0000 7f51cb7fe640 10 client.7440 ms_handle_connect on v2:172.21.15.119:6835/3498735251 2023-11-08T19:29:48.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51cb7fe640 1 -- 172.21.15.119:0/3321022721 <== mds.0 v2:172.21.15.119:6835/3498735251 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55599a2fc4c0 con 0x55599a3b85d0 2023-11-08T19:29:48.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51cb7fe640 10 client.7440 handle_command_reply: tid=0 2023-11-08T19:29:48.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 client.7440 shutdown 2023-11-08T19:29:48.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 msgr2=0x55599a405500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a405500 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f51d8058460 tx=0x7f51d8016040 comp rx=0 tx=0).stop 2023-11-08T19:29:48.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 shutdown_connections 2023-11-08T19:29:48.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:6835/3498735251,v1:172.21.15.119:6837/3498735251] conn(0x55599a3b85d0 0x55599a3b89b0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55599a3ceff0 0x55599a3a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55599a3c96b0 0x55599a405500 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 --2- 172.21.15.119:0/3321022721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55599a3c92d0 0x55599a3cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51cb7fe640 0 client.7440 ms_handle_reset on v2:172.21.15.119:6835/3498735251 2023-11-08T19:29:48.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 >> 172.21.15.119:0/3321022721 conn(0x55599a39c710 msgr2=0x55599a39fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 shutdown_connections 2023-11-08T19:29:48.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 1 -- 172.21.15.119:0/3321022721 wait complete. 2023-11-08T19:29:48.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.270+0000 7f51f5c29740 20 client.7440 trim_cache size 0 max 16384 2023-11-08T19:29:48.286 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:29:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:29:48.286 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:29:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5539, 2023-11-08T19:29:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:29:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:29:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:29:48.287 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 7.0889239220000002, 2023-11-08T19:29:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 65, 2023-11-08T19:29:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 98, 2023-11-08T19:29:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 98, 2023-11-08T19:29:48.288 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 62.649443071999997 2023-11-08T19:29:48.288 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:29:48.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f51d006ae70 msgr2=0x7f51d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f51d006ae70 0x7f51d006d330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f51e0000f60 tx=0x7f51e0001690 comp rx=0 tx=0).stop 2023-11-08T19:29:48.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 msgr2=0x7f51ec0b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 0x7f51ec0b3c80 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f51d800d800 tx=0x7f51d800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 shutdown_connections 2023-11-08T19:29:48.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f51d006ae70 0x7f51d006d330 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f51ec06bd80 0x7f51ec06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f51ec0b3820 0x7f51ec0b3c80 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 --2- 172.21.15.119:0/3839023300 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f51dc006f10 0x7f51ec13d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 >> 172.21.15.119:0/3839023300 conn(0x7f51ec0cb610 msgr2=0x7f51ec141990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 shutdown_connections 2023-11-08T19:29:48.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.271+0000 7f51f3090640 1 -- 172.21.15.119:0/3839023300 wait complete. 2023-11-08T19:29:48.292 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:29:48.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.502+0000 7f07244f3640 1 Processor -- start 2023-11-08T19:29:48.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.502+0000 7f07244f3640 1 -- start start 2023-11-08T19:29:48.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.503+0000 7f07244f3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f071c1447a0 0x7f071c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.503+0000 7f07244f3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f071c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.503+0000 7f07244f3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c06aa90 con 0x7f071c0cad30 2023-11-08T19:29:48.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.503+0000 7f07244f3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c070040 con 0x7f071c1447a0 2023-11-08T19:29:48.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.503+0000 7f07244f3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0c3a50 con 0x7f071c06a120 2023-11-08T19:29:48.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.504+0000 7f0721a8e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f071c0cad30 0x7f071c06f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41268/0 (socket says 172.21.15.119:41268) 2023-11-08T19:29:48.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.504+0000 7f0721a8e640 1 -- 172.21.15.119:0/1709635634 learned_addr learned my addr 172.21.15.119:0/1709635634 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:48.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.504+0000 7f0722a90640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f071c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.504+0000 7f072228f640 1 --2- 172.21.15.119:0/1709635634 >> v2:172.21.15.188:3300/0 conn(0x7f071c1447a0 0x7f071c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.504+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078216506 0 0) 0x7f071c06aa90 con 0x7f071c0cad30 2023-11-08T19:29:48.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.504+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0708003520 con 0x7f071c0cad30 2023-11-08T19:29:48.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 19657614 0 0) 0x7f0708003520 con 0x7f071c0cad30 2023-11-08T19:29:48.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 msgr2=0x7f071c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f071c06a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 >> v2:172.21.15.188:3300/0 conn(0x7f071c1447a0 msgr2=0x7f071c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 --2- 172.21.15.119:0/1709635634 >> v2:172.21.15.188:3300/0 conn(0x7f071c1447a0 0x7f071c15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c144b80 con 0x7f071c0cad30 2023-11-08T19:29:48.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f070c001670 con 0x7f071c0cad30 2023-11-08T19:29:48.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 >> v1:172.21.15.119:6789/0 conn(0x7f071c0cad30 legacy=0x7f071c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0708003970 0x7f0708003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f07080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0708004d00 0x7f0708005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c070040 con 0x7f0708003970 2023-11-08T19:29:48.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0c3a50 con 0x7f0708004d00 2023-11-08T19:29:48.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070800b980 con 0x7f071c06a120 2023-11-08T19:29:48.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f072228f640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0708003970 0x7f0708003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.505+0000 7f0721a8e640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f07080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f072228f640 1 -- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 msgr2=0x7f07080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f072228f640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f07080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f072228f640 1 -- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0708004d00 msgr2=0x7f0708005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f0722a90640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0708004d00 0x7f0708005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f072228f640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0708004d00 0x7f0708005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f072228f640 1 -- 172.21.15.119:0/1709635634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0708003520 con 0x7f0708003970 2023-11-08T19:29:48.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.506+0000 7f0721a8e640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f071c06a120 0x7f07080047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.507+0000 7f072228f640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0708003970 0x7f0708003d90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0718009ac0 tx=0x7f07080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf76d74068bd62c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.507+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071802c710 con 0x7f0708003970 2023-11-08T19:29:48.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.507+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071802c870 con 0x7f0708003970 2023-11-08T19:29:48.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.507+0000 7f072128d640 1 -- 172.21.15.119:0/1709635634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071802cc00 con 0x7f0708003970 2023-11-08T19:29:48.517 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.507+0000 7f07244f3640 1 -- 172.21.15.119:0/1709635634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0708003970 msgr2=0x7f0708003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.508+0000 7f07244f3640 1 --2- 172.21.15.119:0/1709635634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0708003970 0x7f0708003d90 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0718009ac0 tx=0x7f07080057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.508+0000 7f0722a90640 1 -- 172.21.15.119:0/1709635634 reap_dead start 2023-11-08T19:29:48.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.508+0000 7f07244f3640 1 -- 172.21.15.119:0/1709635634 shutdown_connections 2023-11-08T19:29:48.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.508+0000 7f07244f3640 1 -- 172.21.15.119:0/1709635634 >> 172.21.15.119:0/1709635634 conn(0x7f071c0cb610 msgr2=0x7f071c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.508+0000 7f07244f3640 1 -- 172.21.15.119:0/1709635634 shutdown_connections 2023-11-08T19:29:48.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.508+0000 7f07244f3640 1 -- 172.21.15.119:0/1709635634 wait complete. 2023-11-08T19:29:48.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.509+0000 7f07244f3640 1 Processor -- start 2023-11-08T19:29:48.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.510+0000 7f07244f3640 1 -- start start 2023-11-08T19:29:48.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.510+0000 7f07244f3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0708003970 0x7f071c141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.510+0000 7f07244f3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f071c0c5bd0 0x7f071c0c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f07244f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 0x7f071c0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f07244f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0b2270 con 0x7f071c0b2700 2023-11-08T19:29:48.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f07244f3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c1f3810 con 0x7f071c0c5bd0 2023-11-08T19:29:48.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f07244f3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0b1940 con 0x7f0708003970 2023-11-08T19:29:48.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 0x7f071c0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 0x7f071c0b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33536/0 (socket says 172.21.15.119:33536) 2023-11-08T19:29:48.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 -- 172.21.15.119:0/1665250407 learned_addr learned my addr 172.21.15.119:0/1665250407 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f072228f640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0708003970 0x7f071c141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.521 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0721a8e640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f071c0c5bd0 0x7f071c0c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 -- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0708003970 msgr2=0x7f071c141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0708003970 0x7f071c141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 -- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f071c0c5bd0 msgr2=0x7f071c0c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f071c0c5bd0 0x7f071c0c6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f0722a90640 1 -- 172.21.15.119:0/1665250407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0b1c20 con 0x7f071c0b2700 2023-11-08T19:29:48.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.511+0000 7f072228f640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0708003970 0x7f071c141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.512+0000 7f0722a90640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 0x7f071c0b2bb0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f071c0ce900 tx=0x7f071400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.512+0000 7f0712ffd640 1 -- 172.21.15.119:0/1665250407 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714017400 con 0x7f071c0b2700 2023-11-08T19:29:48.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.512+0000 7f0712ffd640 1 -- 172.21.15.119:0/1665250407 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714011040 con 0x7f071c0b2700 2023-11-08T19:29:48.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.512+0000 7f0712ffd640 1 -- 172.21.15.119:0/1665250407 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140177c0 con 0x7f071c0b2700 2023-11-08T19:29:48.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.512+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c0b1f00 con 0x7f071c0b2700 2023-11-08T19:29:48.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.512+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c0b32e0 con 0x7f071c0b2700 2023-11-08T19:29:48.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.515+0000 7f0712ffd640 1 -- 172.21.15.119:0/1665250407 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0714017920 con 0x7f071c0b2700 2023-11-08T19:29:48.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.516+0000 7f0712ffd640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f070006aea0 0x7f070006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.516+0000 7f0712ffd640 1 -- 172.21.15.119:0/1665250407 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f071408a0b0 con 0x7f071c0b2700 2023-11-08T19:29:48.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.516+0000 7f072228f640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f070006aea0 0x7f070006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.517+0000 7f0727085740 1 Processor -- start 2023-11-08T19:29:48.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.517+0000 7f0727085740 1 -- start start 2023-11-08T19:29:48.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.517+0000 7f072228f640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f070006aea0 0x7f070006d360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0718036000 tx=0x7f0718030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f0727085740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 0x55651f499dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c72a0 0x55651f4c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f49a300 0x55651f4ca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f0727085740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f40c7e0 con 0x55651f4c6ec0 2023-11-08T19:29:48.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f409ed0 con 0x55651f4c72a0 2023-11-08T19:29:48.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55651f4057c0 con 0x55651f49a300 2023-11-08T19:29:48.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 0x55651f499dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 0x55651f499dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33546/0 (socket says 172.21.15.119:33546) 2023-11-08T19:29:48.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 -- 172.21.15.119:0/3507388148 learned_addr learned my addr 172.21.15.119:0/3507388148 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 -- 172.21.15.119:0/3507388148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f49a300 msgr2=0x55651f4ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f49a300 0x55651f4ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 -- 172.21.15.119:0/3507388148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c72a0 msgr2=0x55651f4c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c72a0 0x55651f4c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 -- 172.21.15.119:0/3507388148 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55651f3d8b10 con 0x55651f4c6ec0 2023-11-08T19:29:48.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.518+0000 7f072228f640 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 0x55651f499dc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0718009bb0 tx=0x7f071802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efbb78ae3b0e944 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f06fffff640 1 -- 172.21.15.119:0/3507388148 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718003610 con 0x55651f4c6ec0 2023-11-08T19:29:48.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f06fffff640 1 -- 172.21.15.119:0/3507388148 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718003770 con 0x55651f4c6ec0 2023-11-08T19:29:48.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f06fffff640 1 -- 172.21.15.119:0/3507388148 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718003ae0 con 0x55651f4c6ec0 2023-11-08T19:29:48.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 -- 172.21.15.119:0/3507388148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 msgr2=0x55651f499dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 0x55651f499dc0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f0718009bb0 tx=0x7f071802e000 comp rx=0 tx=0).stop 2023-11-08T19:29:48.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 -- 172.21.15.119:0/3507388148 shutdown_connections 2023-11-08T19:29:48.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f49a300 0x55651f4ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c72a0 0x55651f4c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 --2- 172.21.15.119:0/3507388148 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c6ec0 0x55651f499dc0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.519+0000 7f0727085740 1 -- 172.21.15.119:0/3507388148 >> 172.21.15.119:0/3507388148 conn(0x55651f49c0d0 msgr2=0x55651f4e4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.520+0000 7f0727085740 1 -- 172.21.15.119:0/3507388148 shutdown_connections 2023-11-08T19:29:48.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.520+0000 7f0727085740 1 -- 172.21.15.119:0/3507388148 wait complete. 2023-11-08T19:29:48.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.520+0000 7f0727085740 1 -- start start 2023-11-08T19:29:48.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.520+0000 7f0727085740 1 Processor -- start 2023-11-08T19:29:48.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.521+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.521+0000 7f0727085740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4b4e00 0x55651f4cc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0727085740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f3a2c40 con 0x55651f4c72a0 2023-11-08T19:29:48.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f3a28e0 con 0x55651f4c6ec0 2023-11-08T19:29:48.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55651f168f20 con 0x55651f4b4e00 2023-11-08T19:29:48.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0727085740 10 client.0 _ll_register_callbacks cb 0x55651f3c4cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:48.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33560/0 (socket says 172.21.15.119:33560) 2023-11-08T19:29:48.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 -- 172.21.15.119:0/2932957831 learned_addr learned my addr 172.21.15.119:0/2932957831 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0722a90640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4b4e00 0x55651f4cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f072228f640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 -- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4b4e00 msgr2=0x55651f4cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4b4e00 0x55651f4cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 -- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 msgr2=0x55651f4b73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4b73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f0721a8e640 1 -- 172.21.15.119:0/2932957831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0708003520 con 0x55651f4c72a0 2023-11-08T19:29:48.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.522+0000 7f072228f640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f0721a8e640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4cc150 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f070c00e2a0 tx=0x7f070c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2beefa2dfd472a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f06fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:48.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f06fd7fa640 1 -- 172.21.15.119:0/2932957831 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070c0074e0 con 0x55651f4c72a0 2023-11-08T19:29:48.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f06fd7fa640 1 -- 172.21.15.119:0/2932957831 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f070c007640 con 0x55651f4c72a0 2023-11-08T19:29:48.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55651f3d9c80 con 0x55651f4c72a0 2023-11-08T19:29:48.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f06fd7fa640 1 -- 172.21.15.119:0/2932957831 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f070c0047e0 con 0x55651f4c72a0 2023-11-08T19:29:48.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f0727085740 10 client.7452 fetch_fsmap learned FSMap version 65 2023-11-08T19:29:48.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.523+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55651f403e40 con 0x55651f4c72a0 2023-11-08T19:29:48.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f06fd7fa640 1 -- 172.21.15.119:0/2932957831 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55651f3d9c80 con 0x55651f4c72a0 2023-11-08T19:29:48.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f06fd7fa640 1 -- 172.21.15.119:0/2932957831 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f070c004940 con 0x55651f4c72a0 2023-11-08T19:29:48.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f0727085740 10 client.7452 fetch_fsmap finished waiting for FSMap version 65 2023-11-08T19:29:48.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f0727085740 10 client.7452 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:29:48.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f0727085740 20 client.7452 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:48.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f0727085740 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f4df2b0 0x55651f4df690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f0727085740 4 client.7452 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:29:48.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55651f3fe7a0 con 0x55651f4df2b0 2023-11-08T19:29:48.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.524+0000 7f072228f640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f4df2b0 0x55651f4df690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.525+0000 7f072228f640 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f4df2b0 0x55651f4df690 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4cb4345862e1c05c server_cookie=6d5ca39a84811295 in_seq=0 out_seq=0 2023-11-08T19:29:48.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.525+0000 7f06fd7fa640 10 client.7452 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:48.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f06fd7fa640 1 -- 172.21.15.119:0/2932957831 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55651f3fe7a0 con 0x55651f4df2b0 2023-11-08T19:29:48.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f06fd7fa640 10 client.7452 handle_command_reply: tid=0 2023-11-08T19:29:48.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 client.7452 shutdown 2023-11-08T19:29:48.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 msgr2=0x55651f4cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4cc150 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f070c00e2a0 tx=0x7f070c007f90 comp rx=0 tx=0).stop 2023-11-08T19:29:48.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 shutdown_connections 2023-11-08T19:29:48.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4b4e00 0x55651f4cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.545 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4cc150 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f4df2b0 0x55651f4df690 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 --2- 172.21.15.119:0/2932957831 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4b73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f06fd7fa640 0 client.7452 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:48.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 >> 172.21.15.119:0/2932957831 conn(0x55651f49a300 msgr2=0x55651f4e8160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 shutdown_connections 2023-11-08T19:29:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.527+0000 7f0727085740 1 -- 172.21.15.119:0/2932957831 wait complete. 2023-11-08T19:29:48.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.528+0000 7f0727085740 20 client.7452 trim_cache size 0 max 16384 2023-11-08T19:29:48.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.544+0000 7f0727085740 1 Processor -- start 2023-11-08T19:29:48.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.544+0000 7f0727085740 1 -- start start 2023-11-08T19:29:48.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.545+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4b4e00 0x55651f4b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.545+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c6ec0 0x55651f4e52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.545+0000 7f0727085740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.545+0000 7f0727085740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f3d1420 con 0x55651f4c72a0 2023-11-08T19:29:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.545+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f49d610 con 0x55651f4b4e00 2023-11-08T19:29:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.545+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55651f4996d0 con 0x55651f4c6ec0 2023-11-08T19:29:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33564/0 (socket says 172.21.15.119:33564) 2023-11-08T19:29:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 -- 172.21.15.119:0/2933441560 learned_addr learned my addr 172.21.15.119:0/2933441560 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0721a8e640 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c6ec0 0x55651f4e52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f072228f640 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4b4e00 0x55651f4b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 -- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c6ec0 msgr2=0x55651f4e52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c6ec0 0x55651f4e52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 -- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4b4e00 msgr2=0x55651f4b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4b4e00 0x55651f4b72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 -- 172.21.15.119:0/2933441560 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55651f3a1b90 con 0x55651f4c72a0 2023-11-08T19:29:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f072228f640 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4b4e00 0x55651f4b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.546+0000 7f0722a90640 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4b2f10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0714054d10 tx=0x7f0714058f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd619a72f0587c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0710ff9640 1 -- 172.21.15.119:0/2933441560 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714070170 con 0x55651f4c72a0 2023-11-08T19:29:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0710ff9640 1 -- 172.21.15.119:0/2933441560 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071405c960 con 0x55651f4c72a0 2023-11-08T19:29:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0710ff9640 1 -- 172.21.15.119:0/2933441560 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714060b10 con 0x55651f4c72a0 2023-11-08T19:29:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 -- 172.21.15.119:0/2933441560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 msgr2=0x55651f4b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4b2f10 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0714054d10 tx=0x7f0714058f10 comp rx=0 tx=0).stop 2023-11-08T19:29:48.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 -- 172.21.15.119:0/2933441560 shutdown_connections 2023-11-08T19:29:48.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4c72a0 0x55651f4b2f10 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c6ec0 0x55651f4e52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 --2- 172.21.15.119:0/2933441560 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4b4e00 0x55651f4b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 -- 172.21.15.119:0/2933441560 >> 172.21.15.119:0/2933441560 conn(0x55651f49a300 msgr2=0x55651f4c9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 -- 172.21.15.119:0/2933441560 shutdown_connections 2023-11-08T19:29:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.547+0000 7f0727085740 1 -- 172.21.15.119:0/2933441560 wait complete. 2023-11-08T19:29:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.548+0000 7f0727085740 1 -- start start 2023-11-08T19:29:48.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.548+0000 7f0727085740 1 Processor -- start 2023-11-08T19:29:48.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 0x55651f546130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c72a0 0x55651f4b4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f4f3f70 con 0x55651f4b4e00 2023-11-08T19:29:48.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55651f5502b0 con 0x55651f4c6ec0 2023-11-08T19:29:48.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55651f4e36e0 con 0x55651f4c72a0 2023-11-08T19:29:48.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.549+0000 7f0727085740 10 client.0 _ll_register_callbacks cb 0x7f070c001b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:29:48.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 0x55651f546130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 0x55651f546130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:33570/0 (socket says 172.21.15.119:33570) 2023-11-08T19:29:48.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 -- 172.21.15.119:0/3806290970 learned_addr learned my addr 172.21.15.119:0/3806290970 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f0721a8e640 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 -- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c72a0 msgr2=0x55651f4b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c72a0 0x55651f4b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 -- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 msgr2=0x55651f4f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4f3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 -- 172.21.15.119:0/3806290970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c0b32e0 con 0x55651f4b4e00 2023-11-08T19:29:48.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f072228f640 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 0x55651f546130 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0718006d00 tx=0x7f071802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a9f3d7e8e96e30 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f06fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:29:48.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f06fd7fa640 1 -- 172.21.15.119:0/3806290970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718005d60 con 0x55651f4b4e00 2023-11-08T19:29:48.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.550+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0718003770 con 0x55651f4b4e00 2023-11-08T19:29:48.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f06fd7fa640 1 -- 172.21.15.119:0/3806290970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f07180038e0 con 0x55651f4b4e00 2023-11-08T19:29:48.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f06fd7fa640 1 -- 172.21.15.119:0/3806290970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0718055b00 con 0x55651f4b4e00 2023-11-08T19:29:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 10 client.7458 fetch_fsmap learned FSMap version 65 2023-11-08T19:29:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55651f3c7180 con 0x55651f4b4e00 2023-11-08T19:29:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f06fd7fa640 1 -- 172.21.15.119:0/3806290970 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718003770 con 0x55651f4b4e00 2023-11-08T19:29:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f06fd7fa640 1 -- 172.21.15.119:0/3806290970 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 65) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f0718055c60 con 0x55651f4b4e00 2023-11-08T19:29:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 10 client.7458 fetch_fsmap finished waiting for FSMap version 65 2023-11-08T19:29:48.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 10 client.7458 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:29:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 20 client.7458 populate_metadata read hostname 'smithi119' 2023-11-08T19:29:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f54f5a0 0x55651f54f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 4 client.7458 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:29:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.551+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55651f3fe4c0 con 0x55651f54f5a0 2023-11-08T19:29:48.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.552+0000 7f0721a8e640 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f54f5a0 0x55651f54f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.553+0000 7f06fd7fa640 10 client.7458 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:48.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.553+0000 7f0721a8e640 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f54f5a0 0x55651f54f980 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=844e0aa32e02cf5f server_cookie=595194e890c966ec in_seq=0 out_seq=0 2023-11-08T19:29:48.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.553+0000 7f06fd7fa640 1 -- 172.21.15.119:0/3806290970 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55651f3fe4c0 con 0x55651f54f5a0 2023-11-08T19:29:48.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.553+0000 7f06fd7fa640 10 client.7458 handle_command_reply: tid=0 2023-11-08T19:29:48.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 client.7458 shutdown 2023-11-08T19:29:48.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 msgr2=0x55651f546130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 0x55651f546130 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f0718006d00 tx=0x7f071802e070 comp rx=0 tx=0).stop 2023-11-08T19:29:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 shutdown_connections 2023-11-08T19:29:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55651f4c72a0 0x55651f4b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55651f54f5a0 0x55651f54f980 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55651f4c6ec0 0x55651f4f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 --2- 172.21.15.119:0/3806290970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55651f4b4e00 0x55651f546130 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f06fd7fa640 0 client.7458 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:29:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 >> 172.21.15.119:0/3806290970 conn(0x55651f49a300 msgr2=0x55651f4e53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 shutdown_connections 2023-11-08T19:29:48.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 1 -- 172.21.15.119:0/3806290970 wait complete. 2023-11-08T19:29:48.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.554+0000 7f0727085740 20 client.7458 trim_cache size 0 max 16384 2023-11-08T19:29:48.569 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:29:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:29:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-11-08T19:29:48.569 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7065, 2023-11-08T19:29:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:29:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:29:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:29:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 39.874343658999997, 2023-11-08T19:29:48.570 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 65, 2023-11-08T19:29:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 98, 2023-11-08T19:29:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 98, 2023-11-08T19:29:48.571 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 62.927797894999998 2023-11-08T19:29:48.571 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:29:48.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f070006aea0 msgr2=0x7f070006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f070006aea0 0x7f070006d360 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0718036000 tx=0x7f0718030000 comp rx=0 tx=0).stop 2023-11-08T19:29:48.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 msgr2=0x7f071c0b2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 0x7f071c0b2bb0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f071c0ce900 tx=0x7f071400f4a0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 shutdown_connections 2023-11-08T19:29:48.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f070006aea0 0x7f070006d360 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f071c0b2700 0x7f071c0b2bb0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f071c0c5bd0 0x7f071c0c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 --2- 172.21.15.119:0/1665250407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0708003970 0x7f071c141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 >> 172.21.15.119:0/1665250407 conn(0x7f071c0cb610 msgr2=0x7f071c15d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 shutdown_connections 2023-11-08T19:29:48.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.555+0000 7f07244f3640 1 -- 172.21.15.119:0/1665250407 wait complete. 2023-11-08T19:29:48.575 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:48.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.772+0000 7faf6b375640 1 Processor -- start 2023-11-08T19:29:48.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.772+0000 7faf6b375640 1 -- start start 2023-11-08T19:29:48.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf6b375640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf640ac8d0 0x7faf640accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf6b375640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf6415d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf6b375640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6406b780 con 0x7faf64160c50 2023-11-08T19:29:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf6b375640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640ad1f0 con 0x7faf640ac8d0 2023-11-08T19:29:48.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf6b375640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6415dd30 con 0x7faf640adee0 2023-11-08T19:29:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf69b72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf6415d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf6a373640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faf640ac8d0 0x7faf640accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf69b72640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf6415d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:55870/0 (socket says 172.21.15.119:55870) 2023-11-08T19:29:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.774+0000 7faf69b72640 1 -- 172.21.15.119:0/3632951512 learned_addr learned my addr 172.21.15.119:0/3632951512 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.775+0000 7faf69b72640 1 -- 172.21.15.119:0/3632951512 >> v2:172.21.15.188:3300/0 conn(0x7faf640ac8d0 msgr2=0x7faf640accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:48.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.775+0000 7faf69b72640 1 --2- 172.21.15.119:0/3632951512 >> v2:172.21.15.188:3300/0 conn(0x7faf640ac8d0 0x7faf640accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.775+0000 7faf69b72640 1 -- 172.21.15.119:0/3632951512 >> v1:172.21.15.188:6789/0 conn(0x7faf64160c50 legacy=0x7faf641610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.775+0000 7faf69b72640 1 -- 172.21.15.119:0/3632951512 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6415e010 con 0x7faf640adee0 2023-11-08T19:29:48.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.775+0000 7faf6a373640 1 --2- 172.21.15.119:0/3632951512 >> v2:172.21.15.188:3300/0 conn(0x7faf640ac8d0 0x7faf640accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:29:48.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.776+0000 7faf69b72640 1 --2- 172.21.15.119:0/3632951512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf6415d410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf54009870 tx=0x7faf5402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37c10f6068ad8fe2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.777+0000 7faf69371640 1 -- 172.21.15.119:0/3632951512 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54030840 con 0x7faf640adee0 2023-11-08T19:29:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.777+0000 7faf69371640 1 -- 172.21.15.119:0/3632951512 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf540309a0 con 0x7faf640adee0 2023-11-08T19:29:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.777+0000 7faf69371640 1 -- 172.21.15.119:0/3632951512 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf54030d30 con 0x7faf640adee0 2023-11-08T19:29:48.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 -- 172.21.15.119:0/3632951512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 msgr2=0x7faf6415d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 --2- 172.21.15.119:0/3632951512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf6415d410 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faf54009870 tx=0x7faf5402b980 comp rx=0 tx=0).stop 2023-11-08T19:29:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 -- 172.21.15.119:0/3632951512 shutdown_connections 2023-11-08T19:29:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 --2- 172.21.15.119:0/3632951512 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf6415d410 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 --2- 172.21.15.119:0/3632951512 >> v2:172.21.15.188:3300/0 conn(0x7faf640ac8d0 0x7faf640accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 -- 172.21.15.119:0/3632951512 >> 172.21.15.119:0/3632951512 conn(0x7faf640b0560 msgr2=0x7faf640b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 -- 172.21.15.119:0/3632951512 shutdown_connections 2023-11-08T19:29:48.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.778+0000 7faf6b375640 1 -- 172.21.15.119:0/3632951512 wait complete. 2023-11-08T19:29:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.780+0000 7faf6b375640 1 Processor -- start 2023-11-08T19:29:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.780+0000 7faf6b375640 1 -- start start 2023-11-08T19:29:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.781+0000 7faf6b375640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 0x7faf6414b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.781+0000 7faf6b375640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf640c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.781+0000 7faf6b375640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf64160c50 0x7faf640c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.781+0000 7faf6b375640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf641f3b40 con 0x7faf640ac8d0 2023-11-08T19:29:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.781+0000 7faf6b375640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640c3700 con 0x7faf64160c50 2023-11-08T19:29:48.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.781+0000 7faf6b375640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf640c39e0 con 0x7faf640adee0 2023-11-08T19:29:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 0x7faf6414b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 0x7faf6414b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58966/0 (socket says 172.21.15.119:58966) 2023-11-08T19:29:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 -- 172.21.15.119:0/378660090 learned_addr learned my addr 172.21.15.119:0/378660090 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf69b72640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf640c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6ab74640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf64160c50 0x7faf640c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 -- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 msgr2=0x7faf640c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf640c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 -- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf64160c50 msgr2=0x7faf640c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:48.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf64160c50 0x7faf640c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 -- 172.21.15.119:0/378660090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf60009400 con 0x7faf640ac8d0 2023-11-08T19:29:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf69b72640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf640c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6ab74640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf64160c50 0x7faf640c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.782+0000 7faf6a373640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 0x7faf6414b6f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e960 tx=0x7faf6000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.783+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6000ca00 con 0x7faf640ac8d0 2023-11-08T19:29:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.783+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6000cb60 con 0x7faf640ac8d0 2023-11-08T19:29:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.783+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60013470 con 0x7faf640ac8d0 2023-11-08T19:29:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.783+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf540094d0 con 0x7faf640ac8d0 2023-11-08T19:29:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.783+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf640c3fa0 con 0x7faf640ac8d0 2023-11-08T19:29:48.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.784+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf6404ece0 con 0x7faf640ac8d0 2023-11-08T19:29:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.785+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf600135d0 con 0x7faf640ac8d0 2023-11-08T19:29:48.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.786+0000 7faf5affd640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faf4806ae70 0x7faf4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.786+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7faf60089f10 con 0x7faf640ac8d0 2023-11-08T19:29:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.786+0000 7faf69b72640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faf4806ae70 0x7faf4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:48.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.787+0000 7faf69b72640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faf4806ae70 0x7faf4806d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faf54036000 tx=0x7faf5403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:48.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:48.792+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faf60054930 con 0x7faf640ac8d0 2023-11-08T19:29:49.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.017+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf640ad6b0 con 0x7faf640ac8d0 2023-11-08T19:29:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.019+0000 7faf5affd640 1 -- 172.21.15.119:0/378660090 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4940 (secure 0 0 0) 0x7faf60017090 con 0x7faf640ac8d0 2023-11-08T19:29:49.020 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:49.020 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:44.218777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 65 2023-11-08T19:29:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.021+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faf4806ae70 msgr2=0x7faf4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.021+0000 7faf6b375640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faf4806ae70 0x7faf4806d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7faf54036000 tx=0x7faf5403b040 comp rx=0 tx=0).stop 2023-11-08T19:29:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.021+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 msgr2=0x7faf6414b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.021+0000 7faf6b375640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 0x7faf6414b6f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7faf6000e960 tx=0x7faf6000ee20 comp rx=0 tx=0).stop 2023-11-08T19:29:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 shutdown_connections 2023-11-08T19:29:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faf4806ae70 0x7faf4806d330 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faf64160c50 0x7faf640c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faf640adee0 0x7faf640c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 --2- 172.21.15.119:0/378660090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faf640ac8d0 0x7faf6414b6f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 >> 172.21.15.119:0/378660090 conn(0x7faf640b0560 msgr2=0x7faf640afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 shutdown_connections 2023-11-08T19:29:49.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.022+0000 7faf6b375640 1 -- 172.21.15.119:0/378660090 wait complete. 2023-11-08T19:29:49.041 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:29:49.266 INFO:teuthology.orchestra.run.smithi119.stdout:{"last_snap":2,"last_created":2,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:29:23.170262+0000","name":"s11","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:29:49.267 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 2, "last_destroyed": 1, "last_snap": 2, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:29:23.170262+0000" } ] } 2023-11-08T19:29:49.268 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:29:49.268 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2 2023-11-08T19:29:49.268 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:49.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.474+0000 7f9d232dc640 1 Processor -- start 2023-11-08T19:29:49.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.475+0000 7f9d232dc640 1 -- start start 2023-11-08T19:29:49.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d232dc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d1c15f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d232dc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9d1c0adf20 0x7f9d1c15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d232dc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d1c15f8c0 con 0x7f9d1c0ad9d0 2023-11-08T19:29:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d232dc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d1c071c70 con 0x7f9d1c0adf20 2023-11-08T19:29:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d232dc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0ca2e0 con 0x7f9d1c0cac30 2023-11-08T19:29:49.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d22adb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9d1c0ad9d0 0x7f9d1c0c90c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40758/0 (socket says 172.21.15.119:40758) 2023-11-08T19:29:49.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.476+0000 7f9d22adb640 1 -- 172.21.15.119:0/1405832638 learned_addr learned my addr 172.21.15.119:0/1405832638 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:49.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/1405832638 >> v2:172.21.15.188:3300/0 conn(0x7f9d1c0adf20 0x7f9d1c15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d222da640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d1c15f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371090179 0 0) 0x7f9d1c15f8c0 con 0x7f9d1c0ad9d0 2023-11-08T19:29:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d08003520 con 0x7f9d1c0ad9d0 2023-11-08T19:29:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 394956236 0 0) 0x7f9d08003520 con 0x7f9d1c0ad9d0 2023-11-08T19:29:49.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 msgr2=0x7f9d1c15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d1c15f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 >> v2:172.21.15.188:3300/0 conn(0x7f9d1c0adf20 msgr2=0x7f9d1c15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 --2- 172.21.15.119:0/1405832638 >> v2:172.21.15.188:3300/0 conn(0x7f9d1c0adf20 0x7f9d1c15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c0ca5c0 con 0x7f9d1c0ad9d0 2023-11-08T19:29:49.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f9d14001670 con 0x7f9d1c0ad9d0 2023-11-08T19:29:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 >> v1:172.21.15.119:6789/0 conn(0x7f9d1c0ad9d0 legacy=0x7f9d1c0c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d08003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d08004d00 0x7f9d08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c071c70 con 0x7f9d08003f40 2023-11-08T19:29:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c0ca2e0 con 0x7f9d08004d00 2023-11-08T19:29:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.477+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d0800b980 con 0x7f9d1c0cac30 2023-11-08T19:29:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d222da640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d08003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d21ad9640 1 -- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 msgr2=0x7f9d08003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d22adb640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d08004d00 0x7f9d08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d08003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d21ad9640 1 -- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d08004d00 msgr2=0x7f9d08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d08004d00 0x7f9d08005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d21ad9640 1 -- 172.21.15.119:0/1405832638 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08003520 con 0x7f9d08003f40 2023-11-08T19:29:49.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.478+0000 7f9d222da640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c0cac30 0x7f9d08003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:49.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.479+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d080047c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009fc0 tx=0x7f9d080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11f3182495d2c29 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.479+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1002cbf0 con 0x7f9d08003f40 2023-11-08T19:29:49.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1000f040 con 0x7f9d08003f40 2023-11-08T19:29:49.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d212d8640 1 -- 172.21.15.119:0/1405832638 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d10034440 con 0x7f9d08003f40 2023-11-08T19:29:49.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d232dc640 1 -- 172.21.15.119:0/1405832638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 msgr2=0x7f9d080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d232dc640 1 --2- 172.21.15.119:0/1405832638 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d080047c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f9d10009fc0 tx=0x7f9d080057f0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d22adb640 1 -- 172.21.15.119:0/1405832638 reap_dead start 2023-11-08T19:29:49.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d232dc640 1 -- 172.21.15.119:0/1405832638 shutdown_connections 2023-11-08T19:29:49.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d232dc640 1 -- 172.21.15.119:0/1405832638 >> 172.21.15.119:0/1405832638 conn(0x7f9d1c0b0560 msgr2=0x7f9d1c0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d232dc640 1 -- 172.21.15.119:0/1405832638 shutdown_connections 2023-11-08T19:29:49.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.480+0000 7f9d232dc640 1 -- 172.21.15.119:0/1405832638 wait complete. 2023-11-08T19:29:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.482+0000 7f9d232dc640 1 Processor -- start 2023-11-08T19:29:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.482+0000 7f9d232dc640 1 -- start start 2023-11-08T19:29:49.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d232dc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d1c0acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d232dc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c14cae0 0x7f9d1c14cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d232dc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d1c157350 0x7f9d1c1577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d232dc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c14d480 con 0x7f9d08003f40 2023-11-08T19:29:49.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d232dc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c157d10 con 0x7f9d1c157350 2023-11-08T19:29:49.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d232dc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1c1f6150 con 0x7f9d1c14cae0 2023-11-08T19:29:49.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d222da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d1c0acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d222da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d1c0acee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58974/0 (socket says 172.21.15.119:58974) 2023-11-08T19:29:49.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.483+0000 7f9d222da640 1 -- 172.21.15.119:0/116401520 learned_addr learned my addr 172.21.15.119:0/116401520 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:49.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c14cae0 0x7f9d1c14cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d22adb640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d1c157350 0x7f9d1c1577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d222da640 1 -- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c14cae0 msgr2=0x7f9d1c14cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d222da640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c14cae0 0x7f9d1c14cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d222da640 1 -- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d1c157350 msgr2=0x7f9d1c1577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d222da640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d1c157350 0x7f9d1c1577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d222da640 1 -- 172.21.15.119:0/116401520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d1c1f6430 con 0x7f9d08003f40 2023-11-08T19:29:49.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d222da640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d1c0acee0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9d180108b0 tx=0x7f9d18010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1800eb60 con 0x7f9d08003f40 2023-11-08T19:29:49.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1800ecc0 con 0x7f9d08003f40 2023-11-08T19:29:49.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1801c400 con 0x7f9d08003f40 2023-11-08T19:29:49.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.484+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c1f6710 con 0x7f9d08003f40 2023-11-08T19:29:49.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.485+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1c1f6cd0 con 0x7f9d08003f40 2023-11-08T19:29:49.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.486+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1c1f6e90 con 0x7f9d08003f40 2023-11-08T19:29:49.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.486+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9d1801c560 con 0x7f9d08003f40 2023-11-08T19:29:49.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.487+0000 7f9d0effd640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d0006ae70 0x7f9d0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.488+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f9d1808a150 con 0x7f9d08003f40 2023-11-08T19:29:49.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.488+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d0006ae70 0x7f9d0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.488+0000 7f9d21ad9640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d0006ae70 0x7f9d0006d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9d10036000 tx=0x7f9d1002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.492+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9d18054ca0 con 0x7f9d08003f40 2023-11-08T19:29:49.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.723+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1c04ece0 con 0x7f9d08003f40 2023-11-08T19:29:49.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.725+0000 7f9d0effd640 1 -- 172.21.15.119:0/116401520 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f9d180582c0 con 0x7f9d08003f40 2023-11-08T19:29:49.727 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:49.727 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:44.218777+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:49.727 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 65 2023-11-08T19:29:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d0006ae70 msgr2=0x7f9d0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d0006ae70 0x7f9d0006d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9d10036000 tx=0x7f9d1002e000 comp rx=0 tx=0).stop 2023-11-08T19:29:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 msgr2=0x7f9d1c0acee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d1c0acee0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f9d180108b0 tx=0x7f9d18010d70 comp rx=0 tx=0).stop 2023-11-08T19:29:49.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 shutdown_connections 2023-11-08T19:29:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9d0006ae70 0x7f9d0006d330 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9d1c157350 0x7f9d1c1577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9d1c14cae0 0x7f9d1c14cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 --2- 172.21.15.119:0/116401520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9d08003f40 0x7f9d1c0acee0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 >> 172.21.15.119:0/116401520 conn(0x7f9d1c0b0560 msgr2=0x7f9d1c0c5b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 shutdown_connections 2023-11-08T19:29:49.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.728+0000 7f9d232dc640 1 -- 172.21.15.119:0/116401520 wait complete. 2023-11-08T19:29:49.749 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:29:49.962 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.960+0000 7fe388ff5640 1 Processor -- start 2023-11-08T19:29:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.960+0000 7fe388ff5640 1 -- start start 2023-11-08T19:29:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.961+0000 7fe388ff5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe38415d4a0 0x7fe3840d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.961+0000 7fe388ff5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3840cfa00 0x7fe3840cfe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.961+0000 7fe388ff5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe38415df50 con 0x7fe384161160 2023-11-08T19:29:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe388ff5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840d03d0 con 0x7fe3840cfa00 2023-11-08T19:29:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe388ff5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840d1e60 con 0x7fe38415d4a0 2023-11-08T19:29:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3840cfa00 0x7fe3840cfe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe3840cfa00 0x7fe3840cfe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58986/0 (socket says 172.21.15.119:58986) 2023-11-08T19:29:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 -- 172.21.15.119:0/1578810715 learned_addr learned my addr 172.21.15.119:0/1578810715 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe38259c640 1 --2- 172.21.15.119:0/1578810715 >> v2:172.21.15.188:3300/0 conn(0x7fe38415d4a0 0x7fe3840d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 -- 172.21.15.119:0/1578810715 >> v2:172.21.15.188:3300/0 conn(0x7fe38415d4a0 msgr2=0x7fe3840d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 --2- 172.21.15.119:0/1578810715 >> v2:172.21.15.188:3300/0 conn(0x7fe38415d4a0 0x7fe3840d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 -- 172.21.15.119:0/1578810715 >> v1:172.21.15.188:6789/0 conn(0x7fe384161160 legacy=0x7fe384161570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.962+0000 7fe381d9b640 1 -- 172.21.15.119:0/1578810715 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840d2830 con 0x7fe3840cfa00 2023-11-08T19:29:49.968 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:29:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.964+0000 7fe381d9b640 1 --2- 172.21.15.119:0/1578810715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3840cfa00 0x7fe3840cfe90 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe384072830 tx=0x7fe36c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1c400208ecdd4c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.964+0000 7fe38159a640 1 -- 172.21.15.119:0/1578810715 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0317e0 con 0x7fe3840cfa00 2023-11-08T19:29:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.964+0000 7fe38159a640 1 -- 172.21.15.119:0/1578810715 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c031940 con 0x7fe3840cfa00 2023-11-08T19:29:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.964+0000 7fe38159a640 1 -- 172.21.15.119:0/1578810715 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c031cd0 con 0x7fe3840cfa00 2023-11-08T19:29:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 -- 172.21.15.119:0/1578810715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3840cfa00 msgr2=0x7fe3840cfe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 --2- 172.21.15.119:0/1578810715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3840cfa00 0x7fe3840cfe90 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fe384072830 tx=0x7fe36c02b880 comp rx=0 tx=0).stop 2023-11-08T19:29:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 -- 172.21.15.119:0/1578810715 shutdown_connections 2023-11-08T19:29:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 --2- 172.21.15.119:0/1578810715 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe3840cfa00 0x7fe3840cfe90 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 --2- 172.21.15.119:0/1578810715 >> v2:172.21.15.188:3300/0 conn(0x7fe38415d4a0 0x7fe3840d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 -- 172.21.15.119:0/1578810715 >> 172.21.15.119:0/1578810715 conn(0x7fe38406a9f0 msgr2=0x7fe38406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 -- 172.21.15.119:0/1578810715 shutdown_connections 2023-11-08T19:29:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.965+0000 7fe388ff5640 1 -- 172.21.15.119:0/1578810715 wait complete. 2023-11-08T19:29:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.967+0000 7fe388ff5640 1 Processor -- start 2023-11-08T19:29:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.967+0000 7fe388ff5640 1 -- start start 2023-11-08T19:29:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe388ff5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3840cfa00 0x7fe3840ca210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe388ff5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 0x7fe3840c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe388ff5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe384161160 0x7fe38414b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe388ff5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe384158290 con 0x7fe38415d4a0 2023-11-08T19:29:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe388ff5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840c9830 con 0x7fe384161160 2023-11-08T19:29:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe388ff5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe384157cd0 con 0x7fe3840cfa00 2023-11-08T19:29:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe381d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 0x7fe3840c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe381d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 0x7fe3840c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58998/0 (socket says 172.21.15.119:58998) 2023-11-08T19:29:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe381d9b640 1 -- 172.21.15.119:0/3561528504 learned_addr learned my addr 172.21.15.119:0/3561528504 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.968+0000 7fe38259c640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3840cfa00 0x7fe3840ca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe382d9d640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe384161160 0x7fe38414b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe381d9b640 1 -- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3840cfa00 msgr2=0x7fe3840ca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe381d9b640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3840cfa00 0x7fe3840ca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe381d9b640 1 -- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe384161160 msgr2=0x7fe38414b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe381d9b640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe384161160 0x7fe38414b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe381d9b640 1 -- 172.21.15.119:0/3561528504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36c0094d0 con 0x7fe38415d4a0 2023-11-08T19:29:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe38259c640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3840cfa00 0x7fe3840ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe381d9b640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 0x7fe3840c92f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000c00 tx=0x7fe36c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c032880 con 0x7fe38415d4a0 2023-11-08T19:29:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe36c0329e0 con 0x7fe38415d4a0 2023-11-08T19:29:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe384157fb0 con 0x7fe38415d4a0 2023-11-08T19:29:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.969+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe36c032d90 con 0x7fe38415d4a0 2023-11-08T19:29:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.970+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3840c6980 con 0x7fe38415d4a0 2023-11-08T19:29:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.971+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3840c6b70 con 0x7fe38415d4a0 2023-11-08T19:29:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.972+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe36c04d3f0 con 0x7fe38415d4a0 2023-11-08T19:29:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.973+0000 7fe37affd640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe36406aea0 0x7fe36406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.973+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fe36c0ab9d0 con 0x7fe38415d4a0 2023-11-08T19:29:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.973+0000 7fe38259c640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe36406aea0 0x7fe36406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.974+0000 7fe38259c640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe36406aea0 0x7fe36406d360 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe374007330 tx=0x7fe37400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:49.977+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe36c076520 con 0x7fe38415d4a0 2023-11-08T19:29:50.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.234+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7fe384074430 con 0x7fe38415d4a0 2023-11-08T19:29:50.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.709+0000 7fe37affd640 1 -- 172.21.15.119:0/3561528504 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5539 v66) v1 ==== 112+0+0 (secure 0 0 0) 0x7fe36c079b40 con 0x7fe38415d4a0 2023-11-08T19:29:50.711 INFO:teuthology.orchestra.run.smithi119.stderr:freezing mds.5539 2023-11-08T19:29:50.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe36406aea0 msgr2=0x7fe36406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe36406aea0 0x7fe36406d360 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe374007330 tx=0x7fe37400b040 comp rx=0 tx=0).stop 2023-11-08T19:29:50.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 msgr2=0x7fe3840c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:50.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 0x7fe3840c92f0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fe36c000c00 tx=0x7fe36c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 shutdown_connections 2023-11-08T19:29:50.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe36406aea0 0x7fe36406d360 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe384161160 0x7fe38414b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe38415d4a0 0x7fe3840c92f0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 --2- 172.21.15.119:0/3561528504 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe3840cfa00 0x7fe3840ca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:50.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 >> 172.21.15.119:0/3561528504 conn(0x7fe38406a9f0 msgr2=0x7fe38406add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:50.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 shutdown_connections 2023-11-08T19:29:50.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:50.713+0000 7fe388ff5640 1 -- 172.21.15.119:0/3561528504 wait complete. 2023-11-08T19:29:50.736 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 2 2023-11-08T19:29:50.967 INFO:teuthology.orchestra.run.smithi119.stdout:{"success":"mds_kill_mdstable_at = '2' (not observed, change may require restart) "} 2023-11-08T19:29:50.967 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '2' (not observed, change may require restart) " } 2023-11-08T19:29:50.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:29:50.968 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s12) 2023-11-08T19:29:50.975 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:29:51.027 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f692bcb1640 time 2023-11-08T19:29:51.027487+0000 2023-11-08T19:29:51.028 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:29:51.029 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.029 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f693776efaf] 2023-11-08T19:29:51.029 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.029 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.029 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.030 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.030 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.030 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.030 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.031 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.031 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:29:51.027+0000 7f692bcb1640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f692bcb1640 time 2023-11-08T19:29:51.027487+0000 2023-11-08T19:29:51.031 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:29:51.031 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.031 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.032 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f693776efaf] 2023-11-08T19:29:51.032 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.032 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.032 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.032 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.033 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.033 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.033 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.033 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.033 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.034 INFO:tasks.ceph.mds.a.smithi119.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:29:51.034 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f692bcb1640 thread_name:MR_Finisher 2023-11-08T19:29:51.034 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.035 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f6936e54db0] 2023-11-08T19:29:51.035 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f6936ea154c] 2023-11-08T19:29:51.035 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:29:51.035 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:29:51.035 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f693776f009] 2023-11-08T19:29:51.036 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.036 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.036 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.036 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.036 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.037 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.037 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.037 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.037 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:29:51.028+0000 7f692bcb1640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:51.038 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f692bcb1640 thread_name:MR_Finisher 2023-11-08T19:29:51.038 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.038 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.038 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f6936e54db0] 2023-11-08T19:29:51.039 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f6936ea154c] 2023-11-08T19:29:51.039 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:29:51.039 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:29:51.039 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f693776f009] 2023-11-08T19:29:51.039 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.040 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.040 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.040 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.040 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.040 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.040 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.041 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.041 INFO:tasks.ceph.mds.a.smithi119.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:51.041 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.042 INFO:tasks.ceph.mds.a.smithi119.stderr: -502> 2023-11-08T19:29:46.419+0000 7f692f4b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:51.042 INFO:tasks.ceph.mds.a.smithi119.stderr: -500> 2023-11-08T19:29:46.419+0000 7f692f4b8640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:51.042 INFO:tasks.ceph.mds.a.smithi119.stderr: -1> 2023-11-08T19:29:51.027+0000 7f692bcb1640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f692bcb1640 time 2023-11-08T19:29:51.027487+0000 2023-11-08T19:29:51.042 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:29:51.042 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.043 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.043 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f693776efaf] 2023-11-08T19:29:51.043 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.043 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.043 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.044 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.044 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.044 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.044 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.044 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.045 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.045 INFO:tasks.ceph.mds.a.smithi119.stderr: 0> 2023-11-08T19:29:51.028+0000 7f692bcb1640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:51.045 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f692bcb1640 thread_name:MR_Finisher 2023-11-08T19:29:51.045 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.045 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.046 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f6936e54db0] 2023-11-08T19:29:51.046 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f6936ea154c] 2023-11-08T19:29:51.046 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:29:51.046 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:29:51.046 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f693776f009] 2023-11-08T19:29:51.047 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.047 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.047 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.047 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.047 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.047 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.048 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.048 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.048 INFO:tasks.ceph.mds.a.smithi119.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:51.048 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.049 INFO:tasks.ceph.mds.a.smithi119.stderr: -502> 2023-11-08T19:29:46.419+0000 7f692f4b8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:29:51.049 INFO:tasks.ceph.mds.a.smithi119.stderr: -500> 2023-11-08T19:29:46.419+0000 7f692f4b8640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:29:51.049 INFO:tasks.ceph.mds.a.smithi119.stderr: -1> 2023-11-08T19:29:51.027+0000 7f692bcb1640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7f692bcb1640 time 2023-11-08T19:29:51.027487+0000 2023-11-08T19:29:51.049 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:29:51.050 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.050 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.050 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f693776efaf] 2023-11-08T19:29:51.050 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.050 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.051 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.052 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.052 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.052 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.052 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.052 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.053 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.053 INFO:tasks.ceph.mds.a.smithi119.stderr: 0> 2023-11-08T19:29:51.028+0000 7f692bcb1640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:29:51.053 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f692bcb1640 thread_name:MR_Finisher 2023-11-08T19:29:51.053 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.053 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:29:51.054 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f6936e54db0] 2023-11-08T19:29:51.054 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f6936ea154c] 2023-11-08T19:29:51.054 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:29:51.054 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:29:51.054 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f693776f009] 2023-11-08T19:29:51.055 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f693776f16d] 2023-11-08T19:29:51.055 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x5614329679ce] 2023-11-08T19:29:51.055 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x561432997bcc] 2023-11-08T19:29:51.055 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x5614329a0b0c] 2023-11-08T19:29:51.055 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x5614329a0c63] 2023-11-08T19:29:51.056 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f69378164a5] 2023-11-08T19:29:51.056 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f6936e9f802] 2023-11-08T19:29:51.056 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f6936e3f450] 2023-11-08T19:29:51.056 INFO:tasks.ceph.mds.a.smithi119.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:29:51.056 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:29:51.225 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:29:51.226 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:29:51.226 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:51.278 INFO:tasks.ceph.mds.a.smithi119.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:29:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.481+0000 7fab012c2640 1 Processor -- start 2023-11-08T19:29:51.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.482+0000 7fab012c2640 1 -- start start 2023-11-08T19:29:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.483+0000 7fab012c2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faafc15d110 0x7faafc0ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:51.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.483+0000 7fab012c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cd160 0x7faafc0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.483+0000 7fab012c2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faafc0ae940 con 0x7faafc0cbf40 2023-11-08T19:29:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.483+0000 7fab012c2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faafc0ce3c0 con 0x7faafc15d110 2023-11-08T19:29:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.483+0000 7fab012c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc0ad4a0 con 0x7faafc0cd160 2023-11-08T19:29:51.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faafc15d110 0x7faafc0ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faafc15d110 0x7faafc0ccc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59002/0 (socket says 172.21.15.119:59002) 2023-11-08T19:29:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 -- 172.21.15.119:0/925889367 learned_addr learned my addr 172.21.15.119:0/925889367 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:51.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faaf27fc640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cd160 0x7faafc0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 -- 172.21.15.119:0/925889367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cd160 msgr2=0x7faafc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cd160 0x7faafc0cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 -- 172.21.15.119:0/925889367 >> v1:172.21.15.188:6789/0 conn(0x7faafc0cbf40 legacy=0x7faafc06c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:51.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faafaffd640 1 -- 172.21.15.119:0/925889367 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faafc0cd5a0 con 0x7faafc15d110 2023-11-08T19:29:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.484+0000 7faaf27fc640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cd160 0x7faafc0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:29:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.486+0000 7faafaffd640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc15d110 0x7faafc0ccc20 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7faaec009870 tx=0x7faaec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7639cc5d404aad0b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.486+0000 7faafa7fc640 1 -- 172.21.15.119:0/925889367 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec030840 con 0x7faafc15d110 2023-11-08T19:29:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.486+0000 7faafa7fc640 1 -- 172.21.15.119:0/925889367 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaec0309a0 con 0x7faafc15d110 2023-11-08T19:29:51.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.486+0000 7faafa7fc640 1 -- 172.21.15.119:0/925889367 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec030d30 con 0x7faafc15d110 2023-11-08T19:29:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 -- 172.21.15.119:0/925889367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc15d110 msgr2=0x7faafc0ccc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc15d110 0x7faafc0ccc20 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7faaec009870 tx=0x7faaec02b980 comp rx=0 tx=0).stop 2023-11-08T19:29:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 -- 172.21.15.119:0/925889367 shutdown_connections 2023-11-08T19:29:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cd160 0x7faafc0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 --2- 172.21.15.119:0/925889367 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc15d110 0x7faafc0ccc20 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 -- 172.21.15.119:0/925889367 >> 172.21.15.119:0/925889367 conn(0x7faafc0b0b60 msgr2=0x7faafc0b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:51.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 -- 172.21.15.119:0/925889367 shutdown_connections 2023-11-08T19:29:51.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.487+0000 7fab012c2640 1 -- 172.21.15.119:0/925889367 wait complete. 2023-11-08T19:29:51.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.489+0000 7fab012c2640 1 Processor -- start 2023-11-08T19:29:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.490+0000 7fab012c2640 1 -- start start 2023-11-08T19:29:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7fab012c2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cbf40 0x7faafc14f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7fab012c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 0x7faafc1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7fab012c2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faafc15d110 0x7faafc15b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:51.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7fab012c2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc15f980 con 0x7faafc0cd160 2023-11-08T19:29:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7fab012c2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc15fc60 con 0x7faafc15d110 2023-11-08T19:29:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7fab012c2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faafc15ff40 con 0x7faafc0cbf40 2023-11-08T19:29:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7faaf27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 0x7faafc1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7faaf27fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 0x7faafc1599a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59010/0 (socket says 172.21.15.119:59010) 2023-11-08T19:29:51.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7faaf27fc640 1 -- 172.21.15.119:0/2167430888 learned_addr learned my addr 172.21.15.119:0/2167430888 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7faafaffd640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cbf40 0x7faafc14f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.491+0000 7faafb7fe640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faafc15d110 0x7faafc15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf27fc640 1 -- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cbf40 msgr2=0x7faafc14f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf27fc640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cbf40 0x7faafc14f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf27fc640 1 -- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faafc15d110 msgr2=0x7faafc15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf27fc640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faafc15d110 0x7faafc15b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf27fc640 1 -- 172.21.15.119:0/2167430888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaec0094d0 con 0x7faafc0cd160 2023-11-08T19:29:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faafaffd640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cbf40 0x7faafc14f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:29:51.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf27fc640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 0x7faafc1599a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7faae4009fc0 tx=0x7faae400d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4003b90 con 0x7faafc0cd160 2023-11-08T19:29:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae400f040 con 0x7faafc0cd160 2023-11-08T19:29:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc160220 con 0x7faafc0cd160 2023-11-08T19:29:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4004030 con 0x7faafc0cd160 2023-11-08T19:29:51.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.492+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc15f4f0 con 0x7faafc0cd160 2023-11-08T19:29:51.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.493+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc15f740 con 0x7faafc0cd160 2023-11-08T19:29:51.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.494+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faae4004190 con 0x7faafc0cd160 2023-11-08T19:29:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.495+0000 7faaf3fff640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faadc06ae70 0x7faadc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.496+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7faae4089fa0 con 0x7faafc0cd160 2023-11-08T19:29:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.496+0000 7faafaffd640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faadc06ae70 0x7faadc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.496+0000 7faafaffd640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faadc06ae70 0x7faadc06d330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faaec036000 tx=0x7faaec03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:51.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.501+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faafc15f740 con 0x7faafc0cd160 2023-11-08T19:29:51.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.728+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faafc15f740 con 0x7faafc0cd160 2023-11-08T19:29:51.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.730+0000 7faaf3fff640 1 -- 172.21.15.119:0/2167430888 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4940 (secure 0 0 0) 0x7faafc15f740 con 0x7faafc0cd160 2023-11-08T19:29:51.731 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:51.732 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:50.235936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:51.732 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 66 2023-11-08T19:29:51.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faadc06ae70 msgr2=0x7faadc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:51.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faadc06ae70 0x7faadc06d330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7faaec036000 tx=0x7faaec03b040 comp rx=0 tx=0).stop 2023-11-08T19:29:51.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 msgr2=0x7faafc1599a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:51.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 0x7faafc1599a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7faae4009fc0 tx=0x7faae400d560 comp rx=0 tx=0).stop 2023-11-08T19:29:51.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 shutdown_connections 2023-11-08T19:29:51.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faadc06ae70 0x7faadc06d330 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faafc15d110 0x7faafc15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faafc0cd160 0x7faafc1599a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 --2- 172.21.15.119:0/2167430888 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faafc0cbf40 0x7faafc14f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:51.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.733+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 >> 172.21.15.119:0/2167430888 conn(0x7faafc0b0b60 msgr2=0x7faafc06c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:51.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.734+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 shutdown_connections 2023-11-08T19:29:51.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:51.734+0000 7fab012c2640 1 -- 172.21.15.119:0/2167430888 wait complete. 2023-11-08T19:29:51.754 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:52.367 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-08T19:29:56.756 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:29:56.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.978+0000 7f7cdc325640 1 Processor -- start 2023-11-08T19:29:56.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.978+0000 7f7cdc325640 1 -- start start 2023-11-08T19:29:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cdc325640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cd406a390 0x7f7cd40c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cdc325640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd40ae410 con 0x7f7cd40cc810 2023-11-08T19:29:56.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cdc325640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cd406ba30 con 0x7f7cd415d180 2023-11-08T19:29:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cdc325640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cd406bcf0 con 0x7f7cd406a390 2023-11-08T19:29:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cda0c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cd406a390 0x7f7cd40c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cd98c0640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f7cd40cc810 0x7f7cd415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:40772/0 (socket says 172.21.15.119:40772) 2023-11-08T19:29:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cda0c1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7cd406a390 0x7f7cd40c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59012/0 (socket says 172.21.15.119:59012) 2023-11-08T19:29:56.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.980+0000 7f7cd98c0640 1 -- 172.21.15.119:0/4088820677 learned_addr learned my addr 172.21.15.119:0/4088820677 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:29:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.981+0000 7f7cda0c1640 1 -- 172.21.15.119:0/4088820677 >> v1:172.21.15.188:6789/0 conn(0x7f7cd415d180 legacy=0x7f7cd4160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:29:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.981+0000 7f7cda0c1640 1 -- 172.21.15.119:0/4088820677 >> v1:172.21.15.119:6789/0 conn(0x7f7cd40cc810 legacy=0x7f7cd415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:56.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.981+0000 7f7cda0c1640 1 -- 172.21.15.119:0/4088820677 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd406bfd0 con 0x7f7cd406a390 2023-11-08T19:29:56.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.982+0000 7f7cda0c1640 1 --2- 172.21.15.119:0/4088820677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd406a390 0x7f7cd40c4460 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00097a0 tx=0x7f7cd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd088c1d4432e50 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.982+0000 7f7cd90bf640 1 -- 172.21.15.119:0/4088820677 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0031830 con 0x7f7cd406a390 2023-11-08T19:29:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.982+0000 7f7cd90bf640 1 -- 172.21.15.119:0/4088820677 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cd0031990 con 0x7f7cd406a390 2023-11-08T19:29:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.982+0000 7f7cd90bf640 1 -- 172.21.15.119:0/4088820677 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cd0031d20 con 0x7f7cd406a390 2023-11-08T19:29:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 -- 172.21.15.119:0/4088820677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd406a390 msgr2=0x7f7cd40c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:56.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 --2- 172.21.15.119:0/4088820677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd406a390 0x7f7cd40c4460 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f7cd00097a0 tx=0x7f7cd002b8d0 comp rx=0 tx=0).stop 2023-11-08T19:29:56.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 -- 172.21.15.119:0/4088820677 shutdown_connections 2023-11-08T19:29:56.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 --2- 172.21.15.119:0/4088820677 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd406a390 0x7f7cd40c4460 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:56.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 -- 172.21.15.119:0/4088820677 >> 172.21.15.119:0/4088820677 conn(0x7f7cd40cb610 msgr2=0x7f7cd40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 -- 172.21.15.119:0/4088820677 shutdown_connections 2023-11-08T19:29:56.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.983+0000 7f7cdc325640 1 -- 172.21.15.119:0/4088820677 wait complete. 2023-11-08T19:29:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.985+0000 7f7cdc325640 1 Processor -- start 2023-11-08T19:29:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.985+0000 7f7cdc325640 1 -- start start 2023-11-08T19:29:56.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cdc325640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd406a390 0x7f7cd4070bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cdc325640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40cc810 0x7f7cd41512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cdc325640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 0x7f7cd4153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cdc325640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd415e610 con 0x7f7cd415d180 2023-11-08T19:29:56.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cdc325640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40ae740 con 0x7f7cd40cc810 2023-11-08T19:29:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cdc325640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd40aea20 con 0x7f7cd406a390 2023-11-08T19:29:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 0x7f7cd4153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 0x7f7cd4153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59016/0 (socket says 172.21.15.119:59016) 2023-11-08T19:29:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 -- 172.21.15.119:0/3764671191 learned_addr learned my addr 172.21.15.119:0/3764671191 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:29:56.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 -- 172.21.15.119:0/3764671191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd406a390 msgr2=0x7f7cd4070bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd406a390 0x7f7cd4070bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 -- 172.21.15.119:0/3764671191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40cc810 msgr2=0x7f7cd41512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40cc810 0x7f7cd41512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.986+0000 7f7cda8c2640 1 -- 172.21.15.119:0/3764671191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd0009400 con 0x7f7cd415d180 2023-11-08T19:29:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.987+0000 7f7cda8c2640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 0x7f7cd4153d50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00b970 tx=0x7f7ccc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:56.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.987+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc00cb40 con 0x7f7cd415d180 2023-11-08T19:29:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.987+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ccc00cca0 con 0x7f7cd415d180 2023-11-08T19:29:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.987+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ccc014680 con 0x7f7cd415d180 2023-11-08T19:29:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.987+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd40aed00 con 0x7f7cd415d180 2023-11-08T19:29:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.987+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd40af2c0 con 0x7f7cd415d180 2023-11-08T19:29:56.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.988+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4074430 con 0x7f7cd415d180 2023-11-08T19:29:56.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.989+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ccc01d3b0 con 0x7f7cd415d180 2023-11-08T19:29:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.990+0000 7f7ccaffd640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7cb806ae70 0x7f7cb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:29:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.990+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f7ccc089ae0 con 0x7f7cd415d180 2023-11-08T19:29:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.990+0000 7f7cda0c1640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7cb806ae70 0x7f7cb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:29:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.991+0000 7f7cda0c1640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7cb806ae70 0x7f7cb806d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0036000 tx=0x7f7cd0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:29:56.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:56.995+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ccc054500 con 0x7f7cd415d180 2023-11-08T19:29:57.226 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.225+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cd406cd50 con 0x7f7cd415d180 2023-11-08T19:29:57.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.226+0000 7f7ccaffd640 1 -- 172.21.15.119:0/3764671191 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f7ccc057b20 con 0x7f7cd415d180 2023-11-08T19:29:57.228 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:29:57.228 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:50.235936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:29:57.229 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 66 2023-11-08T19:29:57.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7cb806ae70 msgr2=0x7f7cb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7cb806ae70 0x7f7cb806d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0036000 tx=0x7f7cd0030000 comp rx=0 tx=0).stop 2023-11-08T19:29:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 msgr2=0x7f7cd4153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:29:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 0x7f7cd4153d50 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f7ccc00b970 tx=0x7f7ccc00be30 comp rx=0 tx=0).stop 2023-11-08T19:29:57.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 shutdown_connections 2023-11-08T19:29:57.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7cb806ae70 0x7f7cb806d330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:57.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7cd415d180 0x7f7cd4153d50 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:57.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7cd40cc810 0x7f7cd41512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:57.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 --2- 172.21.15.119:0/3764671191 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7cd406a390 0x7f7cd4070bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:29:57.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 >> 172.21.15.119:0/3764671191 conn(0x7f7cd40cb610 msgr2=0x7f7cd40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:29:57.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 shutdown_connections 2023-11-08T19:29:57.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:29:57.229+0000 7f7cdc325640 1 -- 172.21.15.119:0/3764671191 wait complete. 2023-11-08T19:29:57.249 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:29:58.981 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-08T19:30:02.250 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:02.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.473+0000 7fd699e4b640 1 Processor -- start 2023-11-08T19:30:02.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.474+0000 7fd699e4b640 1 -- start start 2023-11-08T19:30:02.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.475+0000 7fd699e4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 0x7fd6940cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.475+0000 7fd699e4b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69415d4f0 con 0x7fd6940cdca0 2023-11-08T19:30:02.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.475+0000 7fd699e4b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6940cf1a0 con 0x7fd694071b70 2023-11-08T19:30:02.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.475+0000 7fd699e4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd694071f50 con 0x7fd6940cd780 2023-11-08T19:30:02.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.475+0000 7fd68bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd6940cdca0 0x7fd6940cb6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:51496/0 (socket says 172.21.15.119:51496) 2023-11-08T19:30:02.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.475+0000 7fd68bfff640 1 -- 172.21.15.119:0/1477407413 learned_addr learned my addr 172.21.15.119:0/1477407413 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:02.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588063072 0 0) 0x7fd69415d4f0 con 0x7fd6940cdca0 2023-11-08T19:30:02.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd680003520 con 0x7fd6940cdca0 2023-11-08T19:30:02.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd693fff640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 0x7fd6940cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 417171529 0 0) 0x7fd680003520 con 0x7fd6940cdca0 2023-11-08T19:30:02.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 msgr2=0x7fd6940cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:02.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 0x7fd6940cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 >> v1:172.21.15.188:6789/0 conn(0x7fd694071b70 legacy=0x7fd6940cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69406a1c0 con 0x7fd6940cdca0 2023-11-08T19:30:02.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd67c001670 con 0x7fd6940cdca0 2023-11-08T19:30:02.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 >> v1:172.21.15.119:6789/0 conn(0x7fd6940cdca0 legacy=0x7fd6940cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.476+0000 7fd692ffd640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 0x7fd6800039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd692ffd640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd680004020 0x7fd6800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd692ffd640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd680004d00 0x7fd680005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000bb90 con 0x7fd680004020 2023-11-08T19:30:02.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000bd00 con 0x7fd680004d00 2023-11-08T19:30:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd68000bec0 con 0x7fd6940cd780 2023-11-08T19:30:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd68bfff640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd680004020 0x7fd6800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd6937fe640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 0x7fd6800039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd693fff640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd680004d00 0x7fd680005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd68bfff640 1 -- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 msgr2=0x7fd6800039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd68bfff640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd6940cd780 0x7fd6800039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd68bfff640 1 -- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd680004d00 msgr2=0x7fd680005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:02.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd68bfff640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd680004d00 0x7fd680005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.477+0000 7fd68bfff640 1 -- 172.21.15.119:0/1477407413 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd680003520 con 0x7fd680004020 2023-11-08T19:30:02.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.478+0000 7fd68bfff640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd680004020 0x7fd6800047c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00c0e0 tx=0x7fd6800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925903cd5aafa390 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:02.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c005ad0 con 0x7fd680004020 2023-11-08T19:30:02.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd67c005c50 con 0x7fd680004020 2023-11-08T19:30:02.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd692ffd640 1 -- 172.21.15.119:0/1477407413 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c02c7d0 con 0x7fd680004020 2023-11-08T19:30:02.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd699e4b640 1 -- 172.21.15.119:0/1477407413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd680004020 msgr2=0x7fd6800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd699e4b640 1 --2- 172.21.15.119:0/1477407413 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd680004020 0x7fd6800047c0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00c0e0 tx=0x7fd6800057f0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd693fff640 1 -- 172.21.15.119:0/1477407413 reap_dead start 2023-11-08T19:30:02.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd699e4b640 1 -- 172.21.15.119:0/1477407413 shutdown_connections 2023-11-08T19:30:02.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd699e4b640 1 -- 172.21.15.119:0/1477407413 >> 172.21.15.119:0/1477407413 conn(0x7fd69406c280 msgr2=0x7fd69406c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:02.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd699e4b640 1 -- 172.21.15.119:0/1477407413 shutdown_connections 2023-11-08T19:30:02.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.479+0000 7fd699e4b640 1 -- 172.21.15.119:0/1477407413 wait complete. 2023-11-08T19:30:02.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.482+0000 7fd699e4b640 1 Processor -- start 2023-11-08T19:30:02.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.482+0000 7fd699e4b640 1 -- start start 2023-11-08T19:30:02.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.483+0000 7fd699e4b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd680004020 0x7fd694151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.483+0000 7fd699e4b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6941f2940 0x7fd6940c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.483+0000 7fd699e4b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 0x7fd6940cf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.483+0000 7fd699e4b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c50c0 con 0x7fd6940c4870 2023-11-08T19:30:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.483+0000 7fd699e4b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c6d20 con 0x7fd6941f2940 2023-11-08T19:30:02.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.483+0000 7fd699e4b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6940c6710 con 0x7fd680004020 2023-11-08T19:30:02.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 0x7fd6940cf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 0x7fd6940cf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51492/0 (socket says 172.21.15.119:51492) 2023-11-08T19:30:02.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 -- 172.21.15.119:0/2102634940 learned_addr learned my addr 172.21.15.119:0/2102634940 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:02.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd6937fe640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd680004020 0x7fd694151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd68bfff640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6941f2940 0x7fd6940c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 -- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd680004020 msgr2=0x7fd694151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd680004020 0x7fd694151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 -- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6941f2940 msgr2=0x7fd6940c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6941f2940 0x7fd6940c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 -- 172.21.15.119:0/2102634940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6940c4de0 con 0x7fd6940c4870 2023-11-08T19:30:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd68bfff640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6941f2940 0x7fd6940c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.484+0000 7fd693fff640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 0x7fd6940cf3c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd6940b5610 tx=0x7fd68c00e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:02.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.485+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c007e40 con 0x7fd6940c4870 2023-11-08T19:30:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.485+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68c00f040 con 0x7fd6940c4870 2023-11-08T19:30:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.485+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68c015650 con 0x7fd6940c4870 2023-11-08T19:30:02.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.485+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6940c69b0 con 0x7fd6940c4870 2023-11-08T19:30:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.485+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6940c3ef0 con 0x7fd6940c4870 2023-11-08T19:30:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.486+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6940c6570 con 0x7fd6940c4870 2023-11-08T19:30:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.487+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd68c0157b0 con 0x7fd6940c4870 2023-11-08T19:30:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.488+0000 7fd690ff9640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd67406ae70 0x7fd67406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:02.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.488+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fd68c08a610 con 0x7fd6940c4870 2023-11-08T19:30:02.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.488+0000 7fd6937fe640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd67406ae70 0x7fd67406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:02.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.489+0000 7fd6937fe640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd67406ae70 0x7fd67406d330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd684007fd0 tx=0x7fd684009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:02.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.492+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6940c6570 con 0x7fd6940c4870 2023-11-08T19:30:02.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.721+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6940c6570 con 0x7fd6940c4870 2023-11-08T19:30:02.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.722+0000 7fd690ff9640 1 -- 172.21.15.119:0/2102634940 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fd6940c6570 con 0x7fd6940c4870 2023-11-08T19:30:02.725 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:02.726 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":66,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:29:50.235936+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:02.727 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 66 2023-11-08T19:30:02.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd67406ae70 msgr2=0x7fd67406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd67406ae70 0x7fd67406d330 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd684007fd0 tx=0x7fd684009040 comp rx=0 tx=0).stop 2023-11-08T19:30:02.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 msgr2=0x7fd6940cf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:02.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 0x7fd6940cf3c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd6940b5610 tx=0x7fd68c00e510 comp rx=0 tx=0).stop 2023-11-08T19:30:02.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 shutdown_connections 2023-11-08T19:30:02.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd67406ae70 0x7fd67406d330 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.724+0000 7fd699e4b640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd6940c4870 0x7fd6940cf3c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.725+0000 7fd699e4b640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd6941f2940 0x7fd6940c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.725+0000 7fd699e4b640 1 --2- 172.21.15.119:0/2102634940 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd680004020 0x7fd694151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:02.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.725+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 >> 172.21.15.119:0/2102634940 conn(0x7fd69406c280 msgr2=0x7fd6940cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:02.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.725+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 shutdown_connections 2023-11-08T19:30:02.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:02.725+0000 7fd699e4b640 1 -- 172.21.15.119:0/2102634940 wait complete. 2023-11-08T19:30:02.746 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:05.594 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-08T19:30:07.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:07.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.972+0000 7fbc50733640 1 Processor -- start 2023-11-08T19:30:07.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.972+0000 7fbc50733640 1 -- start start 2023-11-08T19:30:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.973+0000 7fbc50733640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc480c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.973+0000 7fbc50733640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc480cc810 0x7fbc4806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.973+0000 7fbc50733640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbc4806a770 0x7fbc4815ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.973+0000 7fbc50733640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc480ae410 con 0x7fbc480cc810 2023-11-08T19:30:07.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.973+0000 7fbc50733640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc4806ba30 con 0x7fbc4806a770 2023-11-08T19:30:07.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.973+0000 7fbc50733640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc4806bcf0 con 0x7fbc4806a390 2023-11-08T19:30:07.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc480cc810 0x7fbc4806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbc480cc810 0x7fbc4806cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51498/0 (socket says 172.21.15.119:51498) 2023-11-08T19:30:07.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 -- 172.21.15.119:0/3966265882 learned_addr learned my addr 172.21.15.119:0/3966265882 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:07.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 -- 172.21.15.119:0/3966265882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 msgr2=0x7fbc480c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:07.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4e4cf640 1 --2- 172.21.15.119:0/3966265882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc480c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 --2- 172.21.15.119:0/3966265882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc480c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 -- 172.21.15.119:0/3966265882 >> v2:172.21.15.188:3300/0 conn(0x7fbc4806a770 msgr2=0x7fbc4815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4ecd0640 1 --2- 172.21.15.119:0/3966265882 >> v2:172.21.15.188:3300/0 conn(0x7fbc4806a770 0x7fbc4815ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 --2- 172.21.15.119:0/3966265882 >> v2:172.21.15.188:3300/0 conn(0x7fbc4806a770 0x7fbc4815ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4dcce640 1 -- 172.21.15.119:0/3966265882 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc4806bfd0 con 0x7fbc480cc810 2023-11-08T19:30:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.974+0000 7fbc4ecd0640 1 --2- 172.21.15.119:0/3966265882 >> v2:172.21.15.188:3300/0 conn(0x7fbc4806a770 0x7fbc4815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.975+0000 7fbc4dcce640 1 --2- 172.21.15.119:0/3966265882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc4806cf70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbc480ae050 tx=0x7fbc3802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2ae9212db5f5f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc4d4cd640 1 -- 172.21.15.119:0/3966265882 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38031870 con 0x7fbc480cc810 2023-11-08T19:30:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc4d4cd640 1 -- 172.21.15.119:0/3966265882 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc380319d0 con 0x7fbc480cc810 2023-11-08T19:30:07.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc4d4cd640 1 -- 172.21.15.119:0/3966265882 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc38031d60 con 0x7fbc480cc810 2023-11-08T19:30:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 -- 172.21.15.119:0/3966265882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 msgr2=0x7fbc4806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 --2- 172.21.15.119:0/3966265882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc4806cf70 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbc480ae050 tx=0x7fbc3802b910 comp rx=0 tx=0).stop 2023-11-08T19:30:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 -- 172.21.15.119:0/3966265882 shutdown_connections 2023-11-08T19:30:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 --2- 172.21.15.119:0/3966265882 >> v2:172.21.15.188:3300/0 conn(0x7fbc4806a770 0x7fbc4815ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 --2- 172.21.15.119:0/3966265882 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc4806cf70 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 --2- 172.21.15.119:0/3966265882 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc480c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 -- 172.21.15.119:0/3966265882 >> 172.21.15.119:0/3966265882 conn(0x7fbc480cb610 msgr2=0x7fbc480cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.976+0000 7fbc50733640 1 -- 172.21.15.119:0/3966265882 shutdown_connections 2023-11-08T19:30:07.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.977+0000 7fbc50733640 1 -- 172.21.15.119:0/3966265882 wait complete. 2023-11-08T19:30:07.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.978+0000 7fbc50733640 1 Processor -- start 2023-11-08T19:30:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.978+0000 7fbc50733640 1 -- start start 2023-11-08T19:30:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.979+0000 7fbc50733640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc4815fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.979+0000 7fbc50733640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc4806a770 0x7fbc481f5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.979+0000 7fbc50733640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc481556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.979+0000 7fbc50733640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480afed0 con 0x7fbc480cc810 2023-11-08T19:30:07.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.979+0000 7fbc50733640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc481f6dd0 con 0x7fbc4806a770 2023-11-08T19:30:07.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.979+0000 7fbc50733640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc481f6530 con 0x7fbc4806a390 2023-11-08T19:30:07.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc481556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc481556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:51504/0 (socket says 172.21.15.119:51504) 2023-11-08T19:30:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 -- 172.21.15.119:0/1315363316 learned_addr learned my addr 172.21.15.119:0/1315363316 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4e4cf640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc4815fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4dcce640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc4806a770 0x7fbc481f5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 -- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 msgr2=0x7fbc4815fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:07.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc4815fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 -- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc4806a770 msgr2=0x7fbc481f5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:07.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc4806a770 0x7fbc481f5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:07.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4ecd0640 1 -- 172.21.15.119:0/1315363316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380094d0 con 0x7fbc480cc810 2023-11-08T19:30:07.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.980+0000 7fbc4dcce640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc4806a770 0x7fbc481f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:07.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.981+0000 7fbc4ecd0640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc481556b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000cef0 tx=0x7fbc4000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:07.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.981+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000d840 con 0x7fbc480cc810 2023-11-08T19:30:07.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.981+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc4000d9c0 con 0x7fbc480cc810 2023-11-08T19:30:07.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.981+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4000dd70 con 0x7fbc480cc810 2023-11-08T19:30:07.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.981+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc481f6810 con 0x7fbc480cc810 2023-11-08T19:30:07.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.981+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc480afa70 con 0x7fbc480cc810 2023-11-08T19:30:07.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.982+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc48073400 con 0x7fbc480cc810 2023-11-08T19:30:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.984+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc4001e3b0 con 0x7fbc480cc810 2023-11-08T19:30:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.985+0000 7fbc3effd640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc2c06ae70 0x7fbc2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.985+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7fbc4008a5a0 con 0x7fbc480cc810 2023-11-08T19:30:07.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.985+0000 7fbc4e4cf640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc2c06ae70 0x7fbc2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:07.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.986+0000 7fbc4e4cf640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc2c06ae70 0x7fbc2c06d330 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400aac0 tx=0x7fbc44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:07.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:07.990+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc400551c0 con 0x7fbc480cc810 2023-11-08T19:30:08.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.216+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc48074430 con 0x7fbc480cc810 2023-11-08T19:30:08.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.218+0000 7fbc3effd640 1 -- 172.21.15.119:0/1315363316 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+4988 (secure 0 0 0) 0x7fbc400587e0 con 0x7fbc480cc810 2023-11-08T19:30:08.219 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:08.220 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":67,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5144,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:03.696804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":97,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5539,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5539":{"gid":5539,"name":"a","rank":0,"incarnation":61,"state":"up:active","state_seq":18,"addr":"172.21.15.119:6837/3498735251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":3498735251},{"type":"v1","addr":"172.21.15.119:6837","nonce":3498735251}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:30:03.696804+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:08.220 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 67 2023-11-08T19:30:08.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.220+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc2c06ae70 msgr2=0x7fbc2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:08.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.220+0000 7fbc50733640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc2c06ae70 0x7fbc2c06d330 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fbc4400aac0 tx=0x7fbc44008040 comp rx=0 tx=0).stop 2023-11-08T19:30:08.222 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.220+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 msgr2=0x7fbc481556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.220+0000 7fbc50733640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc481556b0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000cef0 tx=0x7fbc4000a710 comp rx=0 tx=0).stop 2023-11-08T19:30:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.220+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 shutdown_connections 2023-11-08T19:30:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.220+0000 7fbc50733640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc2c06ae70 0x7fbc2c06d330 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.221+0000 7fbc50733640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc480cc810 0x7fbc481556b0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.221+0000 7fbc50733640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc4806a770 0x7fbc481f5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.221+0000 7fbc50733640 1 --2- 172.21.15.119:0/1315363316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc4806a390 0x7fbc4815fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.221+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 >> 172.21.15.119:0/1315363316 conn(0x7fbc480cb610 msgr2=0x7fbc4815ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:08.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.221+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 shutdown_connections 2023-11-08T19:30:08.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.221+0000 7fbc50733640 1 -- 172.21.15.119:0/1315363316 wait complete. 2023-11-08T19:30:08.242 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:30:08.242 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:30:08.332 INFO:teuthology.orchestra.run.smithi119.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:30:08.336 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:30:08.336 DEBUG:teuthology.orchestra.run.smithi119:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:30:08.440 INFO:teuthology.orchestra.run.smithi119.stdout:1699471791.112983.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:30:08.444 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471791.112983.core'] 2023-11-08T19:30:08.445 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471791.112983.core, deleting it 2023-11-08T19:30:08.446 DEBUG:teuthology.orchestra.run.smithi119:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471791.112983.core 2023-11-08T19:30:08.539 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:30:08.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.756+0000 7f81e8fcc640 1 Processor -- start 2023-11-08T19:30:08.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.756+0000 7f81e8fcc640 1 -- start start 2023-11-08T19:30:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e8fcc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f81e4142f80 0x7f81e40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:08.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e8fcc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f81e406cb40 0x7f81e406cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e8fcc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e406d4d0 con 0x7f81e415d9f0 2023-11-08T19:30:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e8fcc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e4143a40 con 0x7f81e406cb40 2023-11-08T19:30:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e8fcc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40ca3e0 con 0x7f81e4142f80 2023-11-08T19:30:08.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e1d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f81e415d9f0 0x7f81e40c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48076/0 (socket says 172.21.15.119:48076) 2023-11-08T19:30:08.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e1d9b640 1 -- 172.21.15.119:0/4037613826 learned_addr learned my addr 172.21.15.119:0/4037613826 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:08.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e2d9d640 1 --2- 172.21.15.119:0/4037613826 >> v2:172.21.15.119:3300/0 conn(0x7f81e406cb40 0x7f81e406cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:08.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e259c640 1 --2- 172.21.15.119:0/4037613826 >> v2:172.21.15.188:3300/0 conn(0x7f81e4142f80 0x7f81e40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:08.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.758+0000 7f81e159a640 1 -- 172.21.15.119:0/4037613826 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1478683398 0 0) 0x7f81e406d4d0 con 0x7f81e415d9f0 2023-11-08T19:30:08.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.759+0000 7f81e159a640 1 -- 172.21.15.119:0/4037613826 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81cc003520 con 0x7f81e415d9f0 2023-11-08T19:30:08.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.759+0000 7f81e2d9d640 1 -- 172.21.15.119:0/4037613826 >> v2:172.21.15.188:3300/0 conn(0x7f81e4142f80 msgr2=0x7f81e40cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:08.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.759+0000 7f81e2d9d640 1 --2- 172.21.15.119:0/4037613826 >> v2:172.21.15.188:3300/0 conn(0x7f81e4142f80 0x7f81e40cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.759+0000 7f81e2d9d640 1 -- 172.21.15.119:0/4037613826 >> v1:172.21.15.119:6789/0 conn(0x7f81e415d9f0 legacy=0x7f81e40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.759+0000 7f81e2d9d640 1 -- 172.21.15.119:0/4037613826 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40c4960 con 0x7f81e406cb40 2023-11-08T19:30:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.760+0000 7f81e2d9d640 1 --2- 172.21.15.119:0/4037613826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e406cb40 0x7f81e406cf90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f81e40c3b00 tx=0x7f81dc02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9150c7b857c3d5c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.760+0000 7f81e159a640 1 -- 172.21.15.119:0/4037613826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc02cc70 con 0x7f81e406cb40 2023-11-08T19:30:08.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.760+0000 7f81e159a640 1 -- 172.21.15.119:0/4037613826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc02cdd0 con 0x7f81e406cb40 2023-11-08T19:30:08.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.760+0000 7f81e159a640 1 -- 172.21.15.119:0/4037613826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc034660 con 0x7f81e406cb40 2023-11-08T19:30:08.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 -- 172.21.15.119:0/4037613826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e406cb40 msgr2=0x7f81e406cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:08.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/4037613826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e406cb40 0x7f81e406cf90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f81e40c3b00 tx=0x7f81dc02bec0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 -- 172.21.15.119:0/4037613826 shutdown_connections 2023-11-08T19:30:08.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/4037613826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e406cb40 0x7f81e406cf90 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/4037613826 >> v2:172.21.15.188:3300/0 conn(0x7f81e4142f80 0x7f81e40cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 -- 172.21.15.119:0/4037613826 >> 172.21.15.119:0/4037613826 conn(0x7f81e40cb610 msgr2=0x7f81e40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:08.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 -- 172.21.15.119:0/4037613826 shutdown_connections 2023-11-08T19:30:08.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.761+0000 7f81e8fcc640 1 -- 172.21.15.119:0/4037613826 wait complete. 2023-11-08T19:30:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.763+0000 7f81e8fcc640 1 Processor -- start 2023-11-08T19:30:08.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.763+0000 7f81e8fcc640 1 -- start start 2023-11-08T19:30:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e8fcc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e406cb40 0x7f81e4159830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e8fcc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e4142f80 0x7f81e40ae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e8fcc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 0x7f81e40aec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e8fcc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40b1f20 con 0x7f81e415d9f0 2023-11-08T19:30:08.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e8fcc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e41f3bb0 con 0x7f81e406cb40 2023-11-08T19:30:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e8fcc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e41f3e90 con 0x7f81e4142f80 2023-11-08T19:30:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 0x7f81e40aec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e2d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 0x7f81e40aec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56922/0 (socket says 172.21.15.119:56922) 2023-11-08T19:30:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e2d9d640 1 -- 172.21.15.119:0/184314755 learned_addr learned my addr 172.21.15.119:0/184314755 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:08.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.764+0000 7f81e259c640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e406cb40 0x7f81e4159830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e1d9b640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e4142f80 0x7f81e40ae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e2d9d640 1 -- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e4142f80 msgr2=0x7f81e40ae6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e2d9d640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e4142f80 0x7f81e40ae6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e2d9d640 1 -- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e406cb40 msgr2=0x7f81e4159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:08.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e2d9d640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e406cb40 0x7f81e4159830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e2d9d640 1 -- 172.21.15.119:0/184314755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc009a70 con 0x7f81e415d9f0 2023-11-08T19:30:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e259c640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e406cb40 0x7f81e4159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e2d9d640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 0x7f81e40aec30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009e80 tx=0x7f81dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc040650 con 0x7f81e415d9f0 2023-11-08T19:30:08.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0407b0 con 0x7f81e415d9f0 2023-11-08T19:30:08.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc040b60 con 0x7f81e415d9f0 2023-11-08T19:30:08.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.765+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40b18d0 con 0x7f81e415d9f0 2023-11-08T19:30:08.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.766+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e4159d70 con 0x7f81e415d9f0 2023-11-08T19:30:08.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.767+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e40b20d0 con 0x7f81e415d9f0 2023-11-08T19:30:08.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.767+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81dc040cc0 con 0x7f81e415d9f0 2023-11-08T19:30:08.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.768+0000 7f81d2ffd640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81c406ae70 0x7f81c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:08.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.768+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 7264+0+0 (secure 0 0 0) 0x7f81dc0ac8b0 con 0x7f81e415d9f0 2023-11-08T19:30:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.769+0000 7f81e259c640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81c406ae70 0x7f81c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.769+0000 7f81e259c640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81c406ae70 0x7f81c406d330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f81d4006fd0 tx=0x7f81d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:08.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:08.773+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81e40b20d0 con 0x7f81e415d9f0 2023-11-08T19:30:09.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.033+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f81e4159f30 con 0x7f81e415d9f0 2023-11-08T19:30:09.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.038+0000 7f81d2ffd640 1 -- 172.21.15.119:0/184314755 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5539 v68) v1 ==== 97+0+0 (secure 0 0 0) 0x7f81dc077360 con 0x7f81e415d9f0 2023-11-08T19:30:09.040 INFO:teuthology.orchestra.run.smithi119.stderr:failed mds gid 5539 2023-11-08T19:30:09.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.042+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81c406ae70 msgr2=0x7f81c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.042+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81c406ae70 0x7f81c406d330 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f81d4006fd0 tx=0x7f81d4008040 comp rx=0 tx=0).stop 2023-11-08T19:30:09.044 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.042+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 msgr2=0x7f81e40aec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.042+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 0x7f81e40aec30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009e80 tx=0x7f81dc004a40 comp rx=0 tx=0).stop 2023-11-08T19:30:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.043+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 shutdown_connections 2023-11-08T19:30:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.043+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f81c406ae70 0x7f81c406d330 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.043+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f81e415d9f0 0x7f81e40aec30 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.043+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f81e4142f80 0x7f81e40ae6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.043+0000 7f81e8fcc640 1 --2- 172.21.15.119:0/184314755 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f81e406cb40 0x7f81e4159830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.043+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 >> 172.21.15.119:0/184314755 conn(0x7f81e40cb610 msgr2=0x7f81e40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.044+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 shutdown_connections 2023-11-08T19:30:09.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.044+0000 7f81e8fcc640 1 -- 172.21.15.119:0/184314755 wait complete. 2023-11-08T19:30:09.065 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:30:09.066 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:30:09.066 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:30:09.067 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:30:09.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:30:09.068 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi119 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-08T19:30:09.068 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:30:09.069 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:30:09.075 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:30:09.076 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:09.233 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:30:09.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.259+0000 7f58b5790640 1 Processor -- start 2023-11-08T19:30:09.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.259+0000 7f58b5790640 1 -- start start 2023-11-08T19:30:09.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.260+0000 7f58b5790640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00cffd0 0x7f58b00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.260+0000 7f58b5790640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b00cda70 con 0x7f58b015b080 2023-11-08T19:30:09.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.260+0000 7f58b5790640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b0161fc0 con 0x7f58b0161250 2023-11-08T19:30:09.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.260+0000 7f58b5790640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b0161dd0 con 0x7f58b00cffd0 2023-11-08T19:30:09.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.260+0000 7f58aeffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f58b015b080 0x7f58b00cf8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48096/0 (socket says 172.21.15.119:48096) 2023-11-08T19:30:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.260+0000 7f58aeffd640 1 -- 172.21.15.119:0/4064652070 learned_addr learned my addr 172.21.15.119:0/4064652070 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849172961 0 0) 0x7f58b00cda70 con 0x7f58b015b080 2023-11-08T19:30:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58a67fc640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00cffd0 0x7f58b00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f589c003520 con 0x7f58b015b080 2023-11-08T19:30:09.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296244059 0 0) 0x7f589c003520 con 0x7f58b015b080 2023-11-08T19:30:09.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00cffd0 msgr2=0x7f58b00ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:09.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00cffd0 0x7f58b00ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 >> v1:172.21.15.188:6789/0 conn(0x7f58b0161250 legacy=0x7f58b01616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b00cdcc0 con 0x7f58b015b080 2023-11-08T19:30:09.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f58a0001670 con 0x7f58b015b080 2023-11-08T19:30:09.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 >> v1:172.21.15.119:6789/0 conn(0x7f58b015b080 legacy=0x7f58b00cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f589c0039a0 0x7f589c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f589c0047c0 0x7f589c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f589c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00c0a0 con 0x7f589c0051d0 2023-11-08T19:30:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00c210 con 0x7f589c0039a0 2023-11-08T19:30:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f589c00c3d0 con 0x7f589c0047c0 2023-11-08T19:30:09.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.261+0000 7f58a67fc640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00cffd0 0x7f58b00ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:30:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58af7fe640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f589c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58aeffd640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f589c0039a0 0x7f589c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58af7fe640 1 -- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f589c0047c0 msgr2=0x7f589c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:09.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58af7fe640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f589c0047c0 0x7f589c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58af7fe640 1 -- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f589c0039a0 msgr2=0x7f589c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:09.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58af7fe640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f589c0039a0 0x7f589c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.262+0000 7f58af7fe640 1 -- 172.21.15.119:0/4064652070 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589c003520 con 0x7f589c0051d0 2023-11-08T19:30:09.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.263+0000 7f58af7fe640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f589c00b980 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f58a8001e50 tx=0x7f589c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fcac15331a2be0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:09.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.263+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8003b70 con 0x7f589c0051d0 2023-11-08T19:30:09.271 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.263+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a8003cd0 con 0x7f589c0051d0 2023-11-08T19:30:09.272 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.263+0000 7f58ae7fc640 1 -- 172.21.15.119:0/4064652070 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a8005310 con 0x7f589c0051d0 2023-11-08T19:30:09.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58b5790640 1 -- 172.21.15.119:0/4064652070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 msgr2=0x7f589c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58b5790640 1 --2- 172.21.15.119:0/4064652070 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f589c00b980 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f58a8001e50 tx=0x7f589c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58af7fe640 1 -- 172.21.15.119:0/4064652070 reap_dead start 2023-11-08T19:30:09.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58b5790640 1 -- 172.21.15.119:0/4064652070 shutdown_connections 2023-11-08T19:30:09.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58b5790640 1 -- 172.21.15.119:0/4064652070 >> 172.21.15.119:0/4064652070 conn(0x7f58b006a9f0 msgr2=0x7f58b006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:09.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58b5790640 1 -- 172.21.15.119:0/4064652070 shutdown_connections 2023-11-08T19:30:09.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.264+0000 7f58b5790640 1 -- 172.21.15.119:0/4064652070 wait complete. 2023-11-08T19:30:09.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.266+0000 7f58b5790640 1 Processor -- start 2023-11-08T19:30:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.266+0000 7f58b5790640 1 -- start start 2023-11-08T19:30:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58b5790640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f58b0160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58b5790640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00c5bf0 0x7f58b00c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58b5790640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b015daf0 0x7f58b015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58b5790640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b015f410 con 0x7f589c0051d0 2023-11-08T19:30:09.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58b5790640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b00c6590 con 0x7f58b015daf0 2023-11-08T19:30:09.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58b5790640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b015e790 con 0x7f58b00c5bf0 2023-11-08T19:30:09.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58aeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f58b0160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58aeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f58b0160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56964/0 (socket says 172.21.15.119:56964) 2023-11-08T19:30:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58aeffd640 1 -- 172.21.15.119:0/1495169999 learned_addr learned my addr 172.21.15.119:0/1495169999 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.267+0000 7f58a67fc640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00c5bf0 0x7f58b00c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58af7fe640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b015daf0 0x7f58b015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58aeffd640 1 -- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00c5bf0 msgr2=0x7f58b00c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58aeffd640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00c5bf0 0x7f58b00c6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58aeffd640 1 -- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b015daf0 msgr2=0x7f58b015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58aeffd640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b015daf0 0x7f58b015df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58aeffd640 1 -- 172.21.15.119:0/1495169999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b015ea70 con 0x7f589c0051d0 2023-11-08T19:30:09.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58af7fe640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b015daf0 0x7f58b015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:09.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58aeffd640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f58b0160220 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f58a000d810 tx=0x7f58a000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00074e0 con 0x7f589c0051d0 2023-11-08T19:30:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a0007640 con 0x7f589c0051d0 2023-11-08T19:30:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b015ed50 con 0x7f589c0051d0 2023-11-08T19:30:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a00049e0 con 0x7f589c0051d0 2023-11-08T19:30:09.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.268+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b015e4b0 con 0x7f589c0051d0 2023-11-08T19:30:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.269+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b0074430 con 0x7f589c0051d0 2023-11-08T19:30:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.270+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58a0004b40 con 0x7f589c0051d0 2023-11-08T19:30:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.271+0000 7f58a7fff640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f589006ae70 0x7f589006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.271+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f58a0089a80 con 0x7f589c0051d0 2023-11-08T19:30:09.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.271+0000 7f58a67fc640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f589006ae70 0x7f589006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:09.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.272+0000 7f58a67fc640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f589006ae70 0x7f589006d330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f58b0152470 tx=0x7f5898008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:09.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.275+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58a0054420 con 0x7f589c0051d0 2023-11-08T19:30:09.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.499+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58b00d22a0 con 0x7f589c0051d0 2023-11-08T19:30:09.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.500+0000 7f58a7fff640 1 -- 172.21.15.119:0/1495169999 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4145 (secure 0 0 0) 0x7f58a0057a40 con 0x7f589c0051d0 2023-11-08T19:30:09.502 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:09.502 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":69,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:09.043018+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:replay","state_seq":1,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:09.503 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 69 2023-11-08T19:30:09.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f589006ae70 msgr2=0x7f589006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f589006ae70 0x7f589006d330 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f58b0152470 tx=0x7f5898008040 comp rx=0 tx=0).stop 2023-11-08T19:30:09.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 msgr2=0x7f58b0160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:09.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f58b0160220 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f58a000d810 tx=0x7f58a000dcd0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 shutdown_connections 2023-11-08T19:30:09.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f589006ae70 0x7f589006d330 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f58b015daf0 0x7f58b015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f58b00c5bf0 0x7f58b00c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 --2- 172.21.15.119:0/1495169999 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f589c0051d0 0x7f58b0160220 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:09.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 >> 172.21.15.119:0/1495169999 conn(0x7f58b006a9f0 msgr2=0x7f58b015bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:09.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 shutdown_connections 2023-11-08T19:30:09.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:09.503+0000 7f58b5790640 1 -- 172.21.15.119:0/1495169999 wait complete. 2023-11-08T19:30:09.523 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:30:12.309 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-08T19:30:14.524 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:14.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.742+0000 7f93eaeb4640 1 Processor -- start 2023-11-08T19:30:14.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.742+0000 7f93eaeb4640 1 -- start start 2023-11-08T19:30:14.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.743+0000 7f93eaeb4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f93e4142f80 0x7f93e40cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.743+0000 7f93eaeb4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e40c4960 con 0x7f93e415d9f0 2023-11-08T19:30:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.743+0000 7f93eaeb4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93e415df60 con 0x7f93e406cb40 2023-11-08T19:30:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.743+0000 7f93eaeb4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e4143a40 con 0x7f93e4142f80 2023-11-08T19:30:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e3fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f93e415d9f0 0x7f93e40c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48106/0 (socket says 172.21.15.119:48106) 2023-11-08T19:30:14.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e3fff640 1 -- 172.21.15.119:0/1497231612 learned_addr learned my addr 172.21.15.119:0/1497231612 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e8c50640 1 --2- 172.21.15.119:0/1497231612 >> v2:172.21.15.119:3300/0 conn(0x7f93e4142f80 0x7f93e40cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e37fe640 1 -- 172.21.15.119:0/1497231612 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035711684 0 0) 0x7f93e40c4960 con 0x7f93e415d9f0 2023-11-08T19:30:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e37fe640 1 -- 172.21.15.119:0/1497231612 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93d4005730 con 0x7f93e415d9f0 2023-11-08T19:30:14.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e8c50640 1 -- 172.21.15.119:0/1497231612 >> v1:172.21.15.188:6789/0 conn(0x7f93e406cb40 legacy=0x7f93e40c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e8c50640 1 -- 172.21.15.119:0/1497231612 >> v1:172.21.15.119:6789/0 conn(0x7f93e415d9f0 legacy=0x7f93e40c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:14.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.744+0000 7f93e8c50640 1 -- 172.21.15.119:0/1497231612 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e415e240 con 0x7f93e4142f80 2023-11-08T19:30:14.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.745+0000 7f93e8c50640 1 --2- 172.21.15.119:0/1497231612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e4142f80 0x7f93e40cc6d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f93d80097a0 tx=0x7f93d802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5ce11044054283 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:14.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.745+0000 7f93e37fe640 1 -- 172.21.15.119:0/1497231612 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8031830 con 0x7f93e4142f80 2023-11-08T19:30:14.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93e37fe640 1 -- 172.21.15.119:0/1497231612 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d8031990 con 0x7f93e4142f80 2023-11-08T19:30:14.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93e37fe640 1 -- 172.21.15.119:0/1497231612 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d8031d20 con 0x7f93e4142f80 2023-11-08T19:30:14.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 -- 172.21.15.119:0/1497231612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e4142f80 msgr2=0x7f93e40cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:14.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/1497231612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e4142f80 0x7f93e40cc6d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f93d80097a0 tx=0x7f93d802b8d0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 -- 172.21.15.119:0/1497231612 shutdown_connections 2023-11-08T19:30:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/1497231612 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e4142f80 0x7f93e40cc6d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 -- 172.21.15.119:0/1497231612 >> 172.21.15.119:0/1497231612 conn(0x7f93e40cb610 msgr2=0x7f93e40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 -- 172.21.15.119:0/1497231612 shutdown_connections 2023-11-08T19:30:14.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.746+0000 7f93eaeb4640 1 -- 172.21.15.119:0/1497231612 wait complete. 2023-11-08T19:30:14.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.748+0000 7f93eaeb4640 1 Processor -- start 2023-11-08T19:30:14.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.748+0000 7f93eaeb4640 1 -- start start 2023-11-08T19:30:14.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93eaeb4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 0x7f93e40afd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:14.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93eaeb4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e4142f80 0x7f93e40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:14.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93eaeb4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93e415d9f0 0x7f93e415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:14.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93eaeb4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e415ab20 con 0x7f93e406cb40 2023-11-08T19:30:14.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93eaeb4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e41f3e10 con 0x7f93e4142f80 2023-11-08T19:30:14.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93eaeb4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e40c87d0 con 0x7f93e415d9f0 2023-11-08T19:30:14.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93e8c50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 0x7f93e40afd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:14.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93e8c50640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 0x7f93e40afd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56982/0 (socket says 172.21.15.119:56982) 2023-11-08T19:30:14.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.749+0000 7f93e8c50640 1 -- 172.21.15.119:0/2163320207 learned_addr learned my addr 172.21.15.119:0/2163320207 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:14.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e8c50640 1 -- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93e415d9f0 msgr2=0x7f93e415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:14.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e8c50640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93e415d9f0 0x7f93e415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e8c50640 1 -- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e4142f80 msgr2=0x7f93e40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e3fff640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e4142f80 0x7f93e40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e8c50640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e4142f80 0x7f93e40c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e8c50640 1 -- 172.21.15.119:0/2163320207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d8009400 con 0x7f93e406cb40 2023-11-08T19:30:14.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e8c50640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 0x7f93e40afd70 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f93d802bee0 tx=0x7f93d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:14.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.750+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d803a6f0 con 0x7f93e406cb40 2023-11-08T19:30:14.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.751+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d803a870 con 0x7f93e406cb40 2023-11-08T19:30:14.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.751+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e41f3b30 con 0x7f93e406cb40 2023-11-08T19:30:14.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.751+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d803ac20 con 0x7f93e406cb40 2023-11-08T19:30:14.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.751+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e415a740 con 0x7f93e406cb40 2023-11-08T19:30:14.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.752+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e415ac60 con 0x7f93e406cb40 2023-11-08T19:30:14.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.753+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93d804c3b0 con 0x7f93e406cb40 2023-11-08T19:30:14.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.754+0000 7f93e17fa640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93c806ae70 0x7f93c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:14.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.754+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f93d80ab950 con 0x7f93e406cb40 2023-11-08T19:30:14.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.754+0000 7f93e3fff640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93c806ae70 0x7f93c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:14.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.755+0000 7f93e3fff640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93c806ae70 0x7f93c806d330 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f93e41439b0 tx=0x7f93d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:14.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.758+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93e415ac60 con 0x7f93e406cb40 2023-11-08T19:30:14.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.980+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93e415ac60 con 0x7f93e406cb40 2023-11-08T19:30:14.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.981+0000 7f93e17fa640 1 -- 172.21.15.119:0/2163320207 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f93e415ac60 con 0x7f93e406cb40 2023-11-08T19:30:14.983 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:14.984 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:12.088177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:14.984 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 73 2023-11-08T19:30:14.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93c806ae70 msgr2=0x7f93c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:14.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93c806ae70 0x7f93c806d330 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f93e41439b0 tx=0x7f93d0009040 comp rx=0 tx=0).stop 2023-11-08T19:30:14.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 msgr2=0x7f93e40afd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:14.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 0x7f93e40afd70 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f93d802bee0 tx=0x7f93d80046c0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 shutdown_connections 2023-11-08T19:30:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f93c806ae70 0x7f93c806d330 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93e415d9f0 0x7f93e415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93e4142f80 0x7f93e40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 --2- 172.21.15.119:0/2163320207 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93e406cb40 0x7f93e40afd70 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.984+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 >> 172.21.15.119:0/2163320207 conn(0x7f93e40cb610 msgr2=0x7f93e40ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.985+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 shutdown_connections 2023-11-08T19:30:14.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:14.985+0000 7f93eaeb4640 1 -- 172.21.15.119:0/2163320207 wait complete. 2023-11-08T19:30:15.004 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:30:15.004 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:15.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.201+0000 7f26adb5e640 1 Processor -- start 2023-11-08T19:30:15.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.201+0000 7f26adb5e640 1 -- start start 2023-11-08T19:30:15.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26adb5e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f26a80adf20 0x7f26a815a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:15.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26adb5e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f26a80ad9d0 0x7f26a815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:15.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26adb5e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26a80ca2e0 con 0x7f26a80cac30 2023-11-08T19:30:15.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26adb5e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a8071c70 con 0x7f26a80ad9d0 2023-11-08T19:30:15.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26adb5e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26a815b390 con 0x7f26a80adf20 2023-11-08T19:30:15.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26ad35d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f26a80ad9d0 0x7f26a815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:15.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26ad35d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f26a80ad9d0 0x7f26a815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56994/0 (socket says 172.21.15.119:56994) 2023-11-08T19:30:15.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.203+0000 7f26ad35d640 1 -- 172.21.15.119:0/2090529183 learned_addr learned my addr 172.21.15.119:0/2090529183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:15.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.204+0000 7f26a7fff640 1 --2- 172.21.15.119:0/2090529183 >> v2:172.21.15.188:3300/0 conn(0x7f26a80adf20 0x7f26a815a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:15.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.204+0000 7f26ad35d640 1 -- 172.21.15.119:0/2090529183 >> v2:172.21.15.188:3300/0 conn(0x7f26a80adf20 msgr2=0x7f26a815a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:15.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.204+0000 7f26ad35d640 1 --2- 172.21.15.119:0/2090529183 >> v2:172.21.15.188:3300/0 conn(0x7f26a80adf20 0x7f26a815a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.204+0000 7f26ad35d640 1 -- 172.21.15.119:0/2090529183 >> v1:172.21.15.188:6789/0 conn(0x7f26a80cac30 legacy=0x7f26a815f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:15.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.204+0000 7f26ad35d640 1 -- 172.21.15.119:0/2090529183 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80ca5c0 con 0x7f26a80ad9d0 2023-11-08T19:30:15.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.204+0000 7f26a7fff640 1 --2- 172.21.15.119:0/2090529183 >> v2:172.21.15.188:3300/0 conn(0x7f26a80adf20 0x7f26a815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:15.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.205+0000 7f26ad35d640 1 --2- 172.21.15.119:0/2090529183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80ad9d0 0x7f26a815ae50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f26a80ce940 tx=0x7f26a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc69b9c550a3362b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:15.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.205+0000 7f26a77fe640 1 -- 172.21.15.119:0/2090529183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a002ca80 con 0x7f26a80ad9d0 2023-11-08T19:30:15.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.205+0000 7f26a77fe640 1 -- 172.21.15.119:0/2090529183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a002cbe0 con 0x7f26a80ad9d0 2023-11-08T19:30:15.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.205+0000 7f26a77fe640 1 -- 172.21.15.119:0/2090529183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00343f0 con 0x7f26a80ad9d0 2023-11-08T19:30:15.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 -- 172.21.15.119:0/2090529183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80ad9d0 msgr2=0x7f26a815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:15.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 --2- 172.21.15.119:0/2090529183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80ad9d0 0x7f26a815ae50 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f26a80ce940 tx=0x7f26a002bee0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.210 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 -- 172.21.15.119:0/2090529183 shutdown_connections 2023-11-08T19:30:15.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 --2- 172.21.15.119:0/2090529183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80ad9d0 0x7f26a815ae50 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 --2- 172.21.15.119:0/2090529183 >> v2:172.21.15.188:3300/0 conn(0x7f26a80adf20 0x7f26a815a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 -- 172.21.15.119:0/2090529183 >> 172.21.15.119:0/2090529183 conn(0x7f26a80b0560 msgr2=0x7f26a80b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:15.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 -- 172.21.15.119:0/2090529183 shutdown_connections 2023-11-08T19:30:15.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.206+0000 7f26adb5e640 1 -- 172.21.15.119:0/2090529183 wait complete. 2023-11-08T19:30:15.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.208+0000 7f26adb5e640 1 Processor -- start 2023-11-08T19:30:15.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.208+0000 7f26adb5e640 1 -- start start 2023-11-08T19:30:15.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26adb5e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a80ad9d0 0x7f26a81f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:15.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26adb5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a80adf20 0x7f26a8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:15.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26adb5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 0x7f26a8157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:15.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26adb5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a815cc20 con 0x7f26a80cac30 2023-11-08T19:30:15.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26adb5e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8155c60 con 0x7f26a80ad9d0 2023-11-08T19:30:15.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26adb5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26a8155f40 con 0x7f26a80adf20 2023-11-08T19:30:15.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 0x7f26a8157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:15.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 0x7f26a8157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56996/0 (socket says 172.21.15.119:56996) 2023-11-08T19:30:15.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 -- 172.21.15.119:0/645018341 learned_addr learned my addr 172.21.15.119:0/645018341 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:15.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 -- 172.21.15.119:0/645018341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a80adf20 msgr2=0x7f26a8157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:15.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a80adf20 0x7f26a8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 -- 172.21.15.119:0/645018341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a80ad9d0 msgr2=0x7f26a81f5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:15.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a80ad9d0 0x7f26a81f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.209+0000 7f26ad35d640 1 -- 172.21.15.119:0/645018341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0009b30 con 0x7f26a80cac30 2023-11-08T19:30:15.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.210+0000 7f26ad35d640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 0x7f26a8157e50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f26a0036040 tx=0x7f26a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:15.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.210+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a003e450 con 0x7f26a80cac30 2023-11-08T19:30:15.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.210+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a003e5d0 con 0x7f26a80cac30 2023-11-08T19:30:15.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.210+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a003e980 con 0x7f26a80cac30 2023-11-08T19:30:15.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.210+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26a8156220 con 0x7f26a80cac30 2023-11-08T19:30:15.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.210+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26a81f3700 con 0x7f26a80cac30 2023-11-08T19:30:15.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.211+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26a81f39b0 con 0x7f26a80cac30 2023-11-08T19:30:15.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.212+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26a002cd50 con 0x7f26a80cac30 2023-11-08T19:30:15.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.213+0000 7f26a57fa640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f268c06ae70 0x7f268c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:15.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.213+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f26a00ac940 con 0x7f26a80cac30 2023-11-08T19:30:15.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.213+0000 7f26acb5c640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f268c06ae70 0x7f268c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:15.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.214+0000 7f26acb5c640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f268c06ae70 0x7f268c06d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f26a815aa60 tx=0x7f269c006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:15.221 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.217+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26a81f39b0 con 0x7f26a80cac30 2023-11-08T19:30:15.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.446+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26a81f39b0 con 0x7f26a80cac30 2023-11-08T19:30:15.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.447+0000 7f26a57fa640 1 -- 172.21.15.119:0/645018341 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f26a81f39b0 con 0x7f26a80cac30 2023-11-08T19:30:15.449 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:15.450 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:12.088177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:15.450 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 73 2023-11-08T19:30:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.450+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f268c06ae70 msgr2=0x7f268c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.450+0000 7f26adb5e640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f268c06ae70 0x7f268c06d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f26a815aa60 tx=0x7f269c006ba0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.450+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 msgr2=0x7f26a8157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.450+0000 7f26adb5e640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 0x7f26a8157e50 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f26a0036040 tx=0x7f26a0004a40 comp rx=0 tx=0).stop 2023-11-08T19:30:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 shutdown_connections 2023-11-08T19:30:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f268c06ae70 0x7f268c06d330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f26a80cac30 0x7f26a8157e50 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f26a80adf20 0x7f26a8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 --2- 172.21.15.119:0/645018341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f26a80ad9d0 0x7f26a81f5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:15.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 >> 172.21.15.119:0/645018341 conn(0x7f26a80b0560 msgr2=0x7f26a80cd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:15.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 shutdown_connections 2023-11-08T19:30:15.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:15.451+0000 7f26adb5e640 1 -- 172.21.15.119:0/645018341 wait complete. 2023-11-08T19:30:15.871 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:30:12.088177+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 99, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5144, 'mds_1': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5144': {'gid': 5144, 'name': 'b', 'rank': 0, 'incarnation': 69, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.188:6835/310147451', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 310147451}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 310147451}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7065': {'gid': 7065, 'name': 'd', 'rank': 1, 'incarnation': 55, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6834/4246462419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4246462419}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4246462419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:30:15.872 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:30:15.872 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:30:16.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.094+0000 7ff244c9b640 1 Processor -- start 2023-11-08T19:30:16.096 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.094+0000 7ff244c9b640 1 -- start start 2023-11-08T19:30:16.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff244c9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff240142f80 0x7ff2400cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff244c9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff24006cb40 0x7ff24006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.097 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff244c9b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2400c4960 con 0x7ff24015d9f0 2023-11-08T19:30:16.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff244c9b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff240143a40 con 0x7ff24006cb40 2023-11-08T19:30:16.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff244c9b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400ca3e0 con 0x7ff240142f80 2023-11-08T19:30:16.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff23dd9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff24015d9f0 0x7ff2400c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48108/0 (socket says 172.21.15.119:48108) 2023-11-08T19:30:16.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff23dd9b640 1 -- 172.21.15.119:0/1660396503 learned_addr learned my addr 172.21.15.119:0/1660396503 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:16.098 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/1660396503 >> v2:172.21.15.119:3300/0 conn(0x7ff24006cb40 0x7ff24006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff23ed9d640 1 -- 172.21.15.119:0/1660396503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff240142f80 msgr2=0x7ff2400cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff23e59c640 1 --2- 172.21.15.119:0/1660396503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff240142f80 0x7ff2400cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.097+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/1660396503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff240142f80 0x7ff2400cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.097+0000 7ff23ed9d640 1 -- 172.21.15.119:0/1660396503 >> v1:172.21.15.119:6789/0 conn(0x7ff24015d9f0 legacy=0x7ff2400c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.097+0000 7ff23ed9d640 1 -- 172.21.15.119:0/1660396503 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24015df60 con 0x7ff24006cb40 2023-11-08T19:30:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.096+0000 7ff23d59a640 1 -- 172.21.15.119:0/1660396503 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750070349 0 0) 0x7ff2400c4960 con 0x7ff24015d9f0 2023-11-08T19:30:16.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/1660396503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff24006cfd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff2380097a0 tx=0x7ff23802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41736e85fcedce78 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff23d59a640 1 -- 172.21.15.119:0/1660396503 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238031780 con 0x7ff24006cb40 2023-11-08T19:30:16.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff23d59a640 1 -- 172.21.15.119:0/1660396503 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2380318e0 con 0x7ff24006cb40 2023-11-08T19:30:16.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff23d59a640 1 -- 172.21.15.119:0/1660396503 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238031c70 con 0x7ff24006cb40 2023-11-08T19:30:16.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff244c9b640 1 -- 172.21.15.119:0/1660396503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 msgr2=0x7ff24006cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff244c9b640 1 --2- 172.21.15.119:0/1660396503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff24006cfd0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7ff2380097a0 tx=0x7ff23802b820 comp rx=0 tx=0).stop 2023-11-08T19:30:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff244c9b640 1 -- 172.21.15.119:0/1660396503 shutdown_connections 2023-11-08T19:30:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff244c9b640 1 --2- 172.21.15.119:0/1660396503 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff24006cfd0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff244c9b640 1 --2- 172.21.15.119:0/1660396503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff240142f80 0x7ff2400cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.098+0000 7ff244c9b640 1 -- 172.21.15.119:0/1660396503 >> 172.21.15.119:0/1660396503 conn(0x7ff2400cb610 msgr2=0x7ff2400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.099+0000 7ff244c9b640 1 -- 172.21.15.119:0/1660396503 shutdown_connections 2023-11-08T19:30:16.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.099+0000 7ff244c9b640 1 -- 172.21.15.119:0/1660396503 wait complete. 2023-11-08T19:30:16.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.100+0000 7ff244c9b640 1 Processor -- start 2023-11-08T19:30:16.105 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.100+0000 7ff244c9b640 1 -- start start 2023-11-08T19:30:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff244c9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff2400b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff244c9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff240142f80 0x7ff2401596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff244c9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d9f0 0x7ff2400c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff244c9b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400b30e0 con 0x7ff24006cb40 2023-11-08T19:30:16.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff244c9b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400702c0 con 0x7ff240142f80 2023-11-08T19:30:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff244c9b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400b3310 con 0x7ff24015d9f0 2023-11-08T19:30:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.101+0000 7ff23e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff2400b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff2400b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57026/0 (socket says 172.21.15.119:57026) 2023-11-08T19:30:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 -- 172.21.15.119:0/187238084 learned_addr learned my addr 172.21.15.119:0/187238084 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 -- 172.21.15.119:0/187238084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d9f0 msgr2=0x7ff2400c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d9f0 0x7ff2400c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 -- 172.21.15.119:0/187238084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff240142f80 msgr2=0x7ff2401596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff240142f80 0x7ff2401596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 -- 172.21.15.119:0/187238084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff238009400 con 0x7ff24006cb40 2023-11-08T19:30:16.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff23e59c640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff2400b2ba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c7f0 tx=0x7ff23000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff236ffd640 1 -- 172.21.15.119:0/187238084 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230004220 con 0x7ff24006cb40 2023-11-08T19:30:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff236ffd640 1 -- 172.21.15.119:0/187238084 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230004380 con 0x7ff24006cb40 2023-11-08T19:30:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.102+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff240070560 con 0x7ff24006cb40 2023-11-08T19:30:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.103+0000 7ff236ffd640 1 -- 172.21.15.119:0/187238084 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230007840 con 0x7ff24006cb40 2023-11-08T19:30:16.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.103+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff240070b20 con 0x7ff24006cb40 2023-11-08T19:30:16.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.105+0000 7ff236ffd640 1 -- 172.21.15.119:0/187238084 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff230007bb0 con 0x7ff24006cb40 2023-11-08T19:30:16.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.107+0000 7ff236ffd640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.107+0000 7ff247831740 1 Processor -- start 2023-11-08T19:30:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.107+0000 7ff236ffd640 1 -- 172.21.15.119:0/187238084 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7ff230089b90 con 0x7ff24006cb40 2023-11-08T19:30:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.107+0000 7ff247831740 1 -- start start 2023-11-08T19:30:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.107+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff228007fd0 tx=0x7ff228007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff247831740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 0x55a1a5030dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505e2a0 0x55a1a5060c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5031300 0x55a1a5061190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff247831740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4fa37e0 con 0x55a1a505dec0 2023-11-08T19:30:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4fa0ed0 con 0x55a1a505e2a0 2023-11-08T19:30:16.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4f9c7c0 con 0x55a1a5031300 2023-11-08T19:30:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff23e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 0x55a1a5030dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff23e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 0x55a1a5030dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57028/0 (socket says 172.21.15.119:57028) 2023-11-08T19:30:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff23e59c640 1 -- 172.21.15.119:0/506589391 learned_addr learned my addr 172.21.15.119:0/506589391 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505e2a0 0x55a1a5060c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.108+0000 7ff23e59c640 1 -- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5031300 msgr2=0x55a1a5061190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff23e59c640 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5031300 0x55a1a5061190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff23e59c640 1 -- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505e2a0 msgr2=0x55a1a5060c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff23e59c640 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505e2a0 0x55a1a5060c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff23e59c640 1 -- 172.21.15.119:0/506589391 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a4f6fb10 con 0x55a1a505dec0 2023-11-08T19:30:16.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505e2a0 0x55a1a5060c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff23e59c640 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 0x55a1a5030dc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff230060050 tx=0x7ff230057ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fef410ea9a03af server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff22ffff640 1 -- 172.21.15.119:0/506589391 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230058280 con 0x55a1a505dec0 2023-11-08T19:30:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff22ffff640 1 -- 172.21.15.119:0/506589391 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23005c3d0 con 0x55a1a505dec0 2023-11-08T19:30:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.109+0000 7ff22ffff640 1 -- 172.21.15.119:0/506589391 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230060580 con 0x55a1a505dec0 2023-11-08T19:30:16.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 -- 172.21.15.119:0/506589391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 msgr2=0x55a1a5030dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 0x55a1a5030dc0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7ff230060050 tx=0x7ff230057ef0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 -- 172.21.15.119:0/506589391 shutdown_connections 2023-11-08T19:30:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5031300 0x55a1a5061190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505e2a0 0x55a1a5060c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.118 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 --2- 172.21.15.119:0/506589391 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505dec0 0x55a1a5030dc0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 -- 172.21.15.119:0/506589391 >> 172.21.15.119:0/506589391 conn(0x55a1a50330d0 msgr2=0x55a1a507bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 -- 172.21.15.119:0/506589391 shutdown_connections 2023-11-08T19:30:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.110+0000 7ff247831740 1 -- 172.21.15.119:0/506589391 wait complete. 2023-11-08T19:30:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.111+0000 7ff247831740 1 -- start start 2023-11-08T19:30:16.119 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.111+0000 7ff247831740 1 Processor -- start 2023-11-08T19:30:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505dec0 0x55a1a50522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5062e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.123 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a5075560 0x55a1a50759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4f39c40 con 0x55a1a505e2a0 2023-11-08T19:30:16.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4f398e0 con 0x55a1a5075560 2023-11-08T19:30:16.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4cfff20 con 0x55a1a505dec0 2023-11-08T19:30:16.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff247831740 10 client.0 _ll_register_callbacks cb 0x55a1a4f5bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:16.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff23dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5062e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff23dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5062e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57032/0 (socket says 172.21.15.119:57032) 2023-11-08T19:30:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.112+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4051532634 learned_addr learned my addr 172.21.15.119:0/4051532634 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a5075560 0x55a1a50759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23e59c640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505dec0 0x55a1a50522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505dec0 msgr2=0x55a1a50522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.125 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505dec0 0x55a1a50522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a5075560 msgr2=0x55a1a50759e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a5075560 0x55a1a50759e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4051532634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff240070b20 con 0x55a1a505e2a0 2023-11-08T19:30:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a5075560 0x55a1a50759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:16.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5062e60 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff22800cce0 tx=0x7ff228001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291418aba86f9182 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff22d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff22d7fa640 1 -- 172.21.15.119:0/4051532634 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228012640 con 0x55a1a505e2a0 2023-11-08T19:30:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1a4f70c80 con 0x55a1a505e2a0 2023-11-08T19:30:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.113+0000 7ff22d7fa640 1 -- 172.21.15.119:0/4051532634 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7ff2280127a0 con 0x55a1a505e2a0 2023-11-08T19:30:16.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff22d7fa640 1 -- 172.21.15.119:0/4051532634 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff228012960 con 0x55a1a505e2a0 2023-11-08T19:30:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 10 client.7551 fetch_fsmap learned FSMap version 73 2023-11-08T19:30:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55a1a4f9ae40 con 0x55a1a505e2a0 2023-11-08T19:30:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff22d7fa640 1 -- 172.21.15.119:0/4051532634 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a1a4f70c80 con 0x55a1a505e2a0 2023-11-08T19:30:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff22d7fa640 1 -- 172.21.15.119:0/4051532634 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7ff228012ac0 con 0x55a1a505e2a0 2023-11-08T19:30:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 10 client.7551 fetch_fsmap finished waiting for FSMap version 73 2023-11-08T19:30:16.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 10 client.7551 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:30:16.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 20 client.7551 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:16.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a5073e30 0x55a1a5074210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 4 client.7551 mds_command: new command op to 5144 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:30:16.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.114+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 --> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1a4f957a0 con 0x55a1a5073e30 2023-11-08T19:30:16.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.115+0000 7ff23e59c640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a5073e30 0x55a1a5074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.116+0000 7ff23e59c640 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a5073e30 0x55a1a5074210 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ca3ba6110140d server_cookie=5fd29e9a7a0c510c in_seq=0 out_seq=0 2023-11-08T19:30:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.116+0000 7ff22d7fa640 10 client.7551 ms_handle_connect on v2:172.21.15.188:6833/310147451 2023-11-08T19:30:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff22d7fa640 1 -- 172.21.15.119:0/4051532634 <== mds.0 v2:172.21.15.188:6833/310147451 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a1a4f957a0 con 0x55a1a5073e30 2023-11-08T19:30:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff22d7fa640 10 client.7551 handle_command_reply: tid=0 2023-11-08T19:30:16.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 client.7551 shutdown 2023-11-08T19:30:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 msgr2=0x55a1a5062e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5062e60 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7ff22800cce0 tx=0x7ff228001400 comp rx=0 tx=0).stop 2023-11-08T19:30:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 shutdown_connections 2023-11-08T19:30:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a5073e30 0x55a1a5074210 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a5075560 0x55a1a50759e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5062e60 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff22d7fa640 0 client.7551 ms_handle_reset on v2:172.21.15.188:6833/310147451 2023-11-08T19:30:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 --2- 172.21.15.119:0/4051532634 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505dec0 0x55a1a50522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 >> 172.21.15.119:0/4051532634 conn(0x55a1a5031300 msgr2=0x55a1a5034d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 shutdown_connections 2023-11-08T19:30:16.132 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 1 -- 172.21.15.119:0/4051532634 wait complete. 2023-11-08T19:30:16.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.118+0000 7ff247831740 20 client.7551 trim_cache size 0 max 16384 2023-11-08T19:30:16.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.135+0000 7ff247831740 1 Processor -- start 2023-11-08T19:30:16.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.135+0000 7ff247831740 1 -- start start 2023-11-08T19:30:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 0x55a1a50e06f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff247831740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5034a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5075560 0x55a1a5034fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff247831740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a4fae330 con 0x55a1a505e2a0 2023-11-08T19:30:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a507cbe0 con 0x55a1a505dec0 2023-11-08T19:30:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a5032790 con 0x55a1a5075560 2023-11-08T19:30:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.136+0000 7ff23dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5034a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5034a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57040/0 (socket says 172.21.15.119:57040) 2023-11-08T19:30:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4264050172 learned_addr learned my addr 172.21.15.119:0/4264050172 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4264050172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5075560 msgr2=0x55a1a5034fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5075560 0x55a1a5034fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4264050172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 msgr2=0x55a1a50e06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 0x55a1a50e06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 -- 172.21.15.119:0/4264050172 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a4f38b90 con 0x55a1a505e2a0 2023-11-08T19:30:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5034a60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff22800c660 tx=0x7ff22801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afbfc44beee4636f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.137+0000 7ff234ff9640 1 -- 172.21.15.119:0/4264050172 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228012440 con 0x55a1a505e2a0 2023-11-08T19:30:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff234ff9640 1 -- 172.21.15.119:0/4264050172 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff228018040 con 0x55a1a505e2a0 2023-11-08T19:30:16.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff234ff9640 1 -- 172.21.15.119:0/4264050172 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2280127c0 con 0x55a1a505e2a0 2023-11-08T19:30:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 -- 172.21.15.119:0/4264050172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 msgr2=0x55a1a5034a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5034a60 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7ff22800c660 tx=0x7ff22801c000 comp rx=0 tx=0).stop 2023-11-08T19:30:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 -- 172.21.15.119:0/4264050172 shutdown_connections 2023-11-08T19:30:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a5075560 0x55a1a5034fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a505e2a0 0x55a1a5034a60 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 --2- 172.21.15.119:0/4264050172 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 0x55a1a50e06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 -- 172.21.15.119:0/4264050172 >> 172.21.15.119:0/4264050172 conn(0x55a1a5031300 msgr2=0x55a1a509e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 -- 172.21.15.119:0/4264050172 shutdown_connections 2023-11-08T19:30:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.138+0000 7ff247831740 1 -- 172.21.15.119:0/4264050172 wait complete. 2023-11-08T19:30:16.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.139+0000 7ff247831740 1 -- start start 2023-11-08T19:30:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.139+0000 7ff247831740 1 Processor -- start 2023-11-08T19:30:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 0x55a1a50722d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505e2a0 0x55a1a5060fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 0x55a1a5064810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a50dff30 con 0x55a1a5075560 2023-11-08T19:30:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a507d6b0 con 0x55a1a505dec0 2023-11-08T19:30:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a1a50715e0 con 0x55a1a505e2a0 2023-11-08T19:30:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff247831740 10 client.0 _ll_register_callbacks cb 0x7ff228001980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff23ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 0x55a1a5064810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff23ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 0x55a1a5064810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57046/0 (socket says 172.21.15.119:57046) 2023-11-08T19:30:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.140+0000 7ff23ed9d640 1 -- 172.21.15.119:0/3250847065 learned_addr learned my addr 172.21.15.119:0/3250847065 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23dd9b640 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505e2a0 0x55a1a5060fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23ed9d640 1 -- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505e2a0 msgr2=0x55a1a5060fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505e2a0 0x55a1a5060fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23ed9d640 1 -- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 msgr2=0x55a1a50722d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 0x55a1a50722d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23ed9d640 1 -- 172.21.15.119:0/3250847065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1a4f9ae40 con 0x55a1a5075560 2023-11-08T19:30:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff23ed9d640 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 0x55a1a5064810 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff2380098c0 tx=0x7ff238031eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d2ebae6996382e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff22d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff22d7fa640 1 -- 172.21.15.119:0/3250847065 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff238032620 con 0x55a1a5075560 2023-11-08T19:30:16.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff23005c3d0 con 0x55a1a5075560 2023-11-08T19:30:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.141+0000 7ff22d7fa640 1 -- 172.21.15.119:0/3250847065 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7ff238032780 con 0x55a1a5075560 2023-11-08T19:30:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff22d7fa640 1 -- 172.21.15.119:0/3250847065 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff238032940 con 0x55a1a5075560 2023-11-08T19:30:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 10 client.7557 fetch_fsmap learned FSMap version 73 2023-11-08T19:30:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x55a1a4f5e180 con 0x55a1a5075560 2023-11-08T19:30:16.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff22d7fa640 1 -- 172.21.15.119:0/3250847065 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23005c3d0 con 0x55a1a5075560 2023-11-08T19:30:16.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff22d7fa640 1 -- 172.21.15.119:0/3250847065 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7ff238032aa0 con 0x55a1a5075560 2023-11-08T19:30:16.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 10 client.7557 fetch_fsmap finished waiting for FSMap version 73 2023-11-08T19:30:16.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 10 client.7557 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:30:16.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 20 client.7557 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:16.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a50e6c50 0x55a1a50e7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 4 client.7557 mds_command: new command op to 5144 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:30:16.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.142+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 --> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a1a4f954c0 con 0x55a1a50e6c50 2023-11-08T19:30:16.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.143+0000 7ff23e59c640 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a50e6c50 0x55a1a50e7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.145+0000 7ff22d7fa640 10 client.7557 ms_handle_connect on v2:172.21.15.188:6833/310147451 2023-11-08T19:30:16.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.145+0000 7ff23e59c640 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a50e6c50 0x55a1a50e7030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=567fdff9f0950b97 server_cookie=5660901fe1ab6873 in_seq=0 out_seq=0 2023-11-08T19:30:16.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff22d7fa640 1 -- 172.21.15.119:0/3250847065 <== mds.0 v2:172.21.15.188:6833/310147451 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55a1a4f954c0 con 0x55a1a50e6c50 2023-11-08T19:30:16.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff22d7fa640 10 client.7557 handle_command_reply: tid=0 2023-11-08T19:30:16.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 client.7557 shutdown 2023-11-08T19:30:16.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 msgr2=0x55a1a5064810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 0x55a1a5064810 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7ff2380098c0 tx=0x7ff238031eb0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 shutdown_connections 2023-11-08T19:30:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:6833/310147451,v1:172.21.15.188:6835/310147451] conn(0x55a1a50e6c50 0x55a1a50e7030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55a1a5075560 0x55a1a5064810 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55a1a505e2a0 0x55a1a5060fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff22d7fa640 0 client.7557 ms_handle_reset on v2:172.21.15.188:6833/310147451 2023-11-08T19:30:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 --2- 172.21.15.119:0/3250847065 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55a1a505dec0 0x55a1a50722d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.147+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 >> 172.21.15.119:0/3250847065 conn(0x55a1a5031300 msgr2=0x55a1a5073160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 shutdown_connections 2023-11-08T19:30:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff247831740 1 -- 172.21.15.119:0/3250847065 wait complete. 2023-11-08T19:30:16.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff247831740 20 client.7557 trim_cache size 0 max 16384 2023-11-08T19:30:16.158 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:30:16.158 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:30:16.158 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:30:16.159 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5144, 2023-11-08T19:30:16.159 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:30:16.159 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:30:16.159 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:30:16.159 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 7.0900965119999997, 2023-11-08T19:30:16.159 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 73, 2023-11-08T19:30:16.160 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 99, 2023-11-08T19:30:16.160 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 99, 2023-11-08T19:30:16.160 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 34.770535293999998 2023-11-08T19:30:16.160 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:30:16.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 msgr2=0x7ff22406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ff228007fd0 tx=0x7ff228007ce0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 msgr2=0x7ff2400b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff2400b2ba0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c7f0 tx=0x7ff23000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 shutdown_connections 2023-11-08T19:30:16.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff22406ae70 0x7ff22406d330 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff24015d9f0 0x7ff2400c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff240142f80 0x7ff2401596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 --2- 172.21.15.119:0/187238084 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff24006cb40 0x7ff2400b2ba0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.148+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 >> 172.21.15.119:0/187238084 conn(0x7ff2400cb610 msgr2=0x7ff240072c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.149+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 shutdown_connections 2023-11-08T19:30:16.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.149+0000 7ff244c9b640 1 -- 172.21.15.119:0/187238084 wait complete. 2023-11-08T19:30:16.166 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:30:16.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.371+0000 7fa49d34c640 1 Processor -- start 2023-11-08T19:30:16.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.371+0000 7fa49d34c640 1 -- start start 2023-11-08T19:30:16.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa49d34c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 0x7fa498165cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa49d34c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4980c4a00 con 0x7fa49815af80 2023-11-08T19:30:16.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa49d34c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4980cbd30 con 0x7fa49815b360 2023-11-08T19:30:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa49d34c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980cfe90 con 0x7fa4980c6180 2023-11-08T19:30:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa496ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa49815af80 0x7fa4980cea80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48124/0 (socket says 172.21.15.119:48124) 2023-11-08T19:30:16.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa496ffd640 1 -- 172.21.15.119:0/2904627454 learned_addr learned my addr 172.21.15.119:0/2904627454 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.373+0000 7fa4977fe640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 0x7fa498165cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 886448556 0 0) 0x7fa4980c4a00 con 0x7fa49815af80 2023-11-08T19:30:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa484003520 con 0x7fa49815af80 2023-11-08T19:30:16.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1898630455 0 0) 0x7fa484003520 con 0x7fa49815af80 2023-11-08T19:30:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 msgr2=0x7fa498165cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 0x7fa498165cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 >> v1:172.21.15.188:6789/0 conn(0x7fa49815b360 legacy=0x7fa4980d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4980cf5f0 con 0x7fa49815af80 2023-11-08T19:30:16.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fa4880019f0 con 0x7fa49815af80 2023-11-08T19:30:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 >> v1:172.21.15.119:6789/0 conn(0x7fa49815af80 legacy=0x7fa4980cea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840039a0 0x7fa484003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4840047c0 0x7fa484004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 0x7fa484005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4980cfe90 con 0x7fa4840047c0 2023-11-08T19:30:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48400ba60 con 0x7fa4840039a0 2023-11-08T19:30:16.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.374+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa48400bc20 con 0x7fa4980c6180 2023-11-08T19:30:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4967fc640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4840047c0 0x7fa484004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4977fe640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 0x7fa484005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4967fc640 1 -- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 msgr2=0x7fa484005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4967fc640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4980c6180 0x7fa484005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4967fc640 1 -- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840039a0 msgr2=0x7fa484003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4967fc640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840039a0 0x7fa484003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.375+0000 7fa4967fc640 1 -- 172.21.15.119:0/2904627454 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484003520 con 0x7fa4840047c0 2023-11-08T19:30:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.376+0000 7fa4967fc640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4840047c0 0x7fa484004c40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa498071c50 tx=0x7fa4840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a98c86bbeba44b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.376+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4800052d0 con 0x7fa4840047c0 2023-11-08T19:30:16.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.376+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48002e050 con 0x7fa4840047c0 2023-11-08T19:30:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa495ffb640 1 -- 172.21.15.119:0/2904627454 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4800035f0 con 0x7fa4840047c0 2023-11-08T19:30:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa49d34c640 1 -- 172.21.15.119:0/2904627454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4840047c0 msgr2=0x7fa484004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa49d34c640 1 --2- 172.21.15.119:0/2904627454 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa4840047c0 0x7fa484004c40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7fa498071c50 tx=0x7fa4840057f0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa4977fe640 1 -- 172.21.15.119:0/2904627454 reap_dead start 2023-11-08T19:30:16.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa49d34c640 1 -- 172.21.15.119:0/2904627454 shutdown_connections 2023-11-08T19:30:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa49d34c640 1 -- 172.21.15.119:0/2904627454 >> 172.21.15.119:0/2904627454 conn(0x7fa49808afe0 msgr2=0x7fa49809a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa49d34c640 1 -- 172.21.15.119:0/2904627454 shutdown_connections 2023-11-08T19:30:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.377+0000 7fa49d34c640 1 -- 172.21.15.119:0/2904627454 wait complete. 2023-11-08T19:30:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.379+0000 7fa49d34c640 1 Processor -- start 2023-11-08T19:30:16.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.379+0000 7fa49d34c640 1 -- start start 2023-11-08T19:30:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.380+0000 7fa49d34c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840047c0 0x7fa49814b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.380+0000 7fa49d34c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa49806b230 0x7fa49815c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.380+0000 7fa49d34c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 0x7fa4980b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.380+0000 7fa49d34c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49806c300 con 0x7fa49806c790 2023-11-08T19:30:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.380+0000 7fa49d34c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49806b6f0 con 0x7fa4840047c0 2023-11-08T19:30:16.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.380+0000 7fa49d34c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49806b9d0 con 0x7fa49806b230 2023-11-08T19:30:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 0x7fa4980b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 0x7fa4980b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57066/0 (socket says 172.21.15.119:57066) 2023-11-08T19:30:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 -- 172.21.15.119:0/4181090106 learned_addr learned my addr 172.21.15.119:0/4181090106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 -- 172.21.15.119:0/4181090106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa49806b230 msgr2=0x7fa49815c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa49806b230 0x7fa49815c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 -- 172.21.15.119:0/4181090106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840047c0 msgr2=0x7fa49814b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840047c0 0x7fa49814b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 -- 172.21.15.119:0/4181090106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49806bcb0 con 0x7fa49806c790 2023-11-08T19:30:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.381+0000 7fa4977fe640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 0x7fa4980b2350 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa49815b740 tx=0x7fa490011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.382+0000 7fa48f7fe640 1 -- 172.21.15.119:0/4181090106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490004500 con 0x7fa49806c790 2023-11-08T19:30:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.382+0000 7fa48f7fe640 1 -- 172.21.15.119:0/4181090106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490003610 con 0x7fa49806c790 2023-11-08T19:30:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.382+0000 7fa48f7fe640 1 -- 172.21.15.119:0/4181090106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4900039e0 con 0x7fa49806c790 2023-11-08T19:30:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.382+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49806bf90 con 0x7fa49806c790 2023-11-08T19:30:16.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.382+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49815c9c0 con 0x7fa49806c790 2023-11-08T19:30:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.384+0000 7fa48f7fe640 1 -- 172.21.15.119:0/4181090106 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa490003b40 con 0x7fa49806c790 2023-11-08T19:30:16.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.385+0000 7fa48f7fe640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa47806ae70 0x7fa47806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.385+0000 7fa48f7fe640 1 -- 172.21.15.119:0/4181090106 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fa490089e10 con 0x7fa49806c790 2023-11-08T19:30:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.386+0000 7fa496ffd640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa47806ae70 0x7fa47806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.386+0000 7fa49ffb7740 1 Processor -- start 2023-11-08T19:30:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.386+0000 7fa49ffb7740 1 -- start start 2023-11-08T19:30:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.386+0000 7fa496ffd640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa47806ae70 0x7fa47806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa488007fd0 tx=0x7fa488001810 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861f68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861f98c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f69300 0x555861f99190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861edb7e0 con 0x555861f95ec0 2023-11-08T19:30:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861ed8ed0 con 0x555861f69300 2023-11-08T19:30:16.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555861ed47c0 con 0x555861f962a0 2023-11-08T19:30:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861f68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861f68dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57078/0 (socket says 172.21.15.119:57078) 2023-11-08T19:30:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.387+0000 7fa496ffd640 1 -- 172.21.15.119:0/360023810 learned_addr learned my addr 172.21.15.119:0/360023810 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa496ffd640 1 -- 172.21.15.119:0/360023810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 msgr2=0x555861f98c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa496ffd640 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861f98c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa496ffd640 1 -- 172.21.15.119:0/360023810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f69300 msgr2=0x555861f99190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa496ffd640 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f69300 0x555861f99190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa496ffd640 1 -- 172.21.15.119:0/360023810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555861ea7b10 con 0x555861f95ec0 2023-11-08T19:30:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa496ffd640 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861f68dc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cd80 tx=0x7fa48800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88abc86f693de70 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa48cff9640 1 -- 172.21.15.119:0/360023810 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48801b070 con 0x555861f95ec0 2023-11-08T19:30:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa48cff9640 1 -- 172.21.15.119:0/360023810 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48800ce70 con 0x555861f95ec0 2023-11-08T19:30:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.388+0000 7fa48cff9640 1 -- 172.21.15.119:0/360023810 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488007a00 con 0x555861f95ec0 2023-11-08T19:30:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 -- 172.21.15.119:0/360023810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 msgr2=0x555861f68dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861f68dc0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cd80 tx=0x7fa48800f000 comp rx=0 tx=0).stop 2023-11-08T19:30:16.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 -- 172.21.15.119:0/360023810 shutdown_connections 2023-11-08T19:30:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f69300 0x555861f99190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861f98c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/360023810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861f68dc0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 -- 172.21.15.119:0/360023810 >> 172.21.15.119:0/360023810 conn(0x555861f6b0d0 msgr2=0x555861fb3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 -- 172.21.15.119:0/360023810 shutdown_connections 2023-11-08T19:30:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.389+0000 7fa49ffb7740 1 -- 172.21.15.119:0/360023810 wait complete. 2023-11-08T19:30:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.390+0000 7fa49ffb7740 1 -- start start 2023-11-08T19:30:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.390+0000 7fa49ffb7740 1 Processor -- start 2023-11-08T19:30:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x555861f80210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f972a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861f8ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861e71c40 con 0x555861fc7430 2023-11-08T19:30:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861e718e0 con 0x555861f962a0 2023-11-08T19:30:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555861c37f20 con 0x555861f95ec0 2023-11-08T19:30:16.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa49ffb7740 10 client.0 _ll_register_callbacks cb 0x555861e93cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861f8ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861f8ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57082/0 (socket says 172.21.15.119:57082) 2023-11-08T19:30:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.391+0000 7fa4977fe640 1 -- 172.21.15.119:0/375609530 learned_addr learned my addr 172.21.15.119:0/375609530 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa4977fe640 1 -- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 msgr2=0x555861f80210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa4977fe640 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x555861f80210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa4977fe640 1 -- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 msgr2=0x555861f972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa4977fe640 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa4977fe640 1 -- 172.21.15.119:0/375609530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4980cf5f0 con 0x555861fc7430 2023-11-08T19:30:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa4977fe640 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861f8ede0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa4900602d0 tx=0x7fa49005c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b1d2176dea1068 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:16.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa475ffb640 1 -- 172.21.15.119:0/375609530 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490081be0 con 0x555861fc7430 2023-11-08T19:30:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa475ffb640 1 -- 172.21.15.119:0/375609530 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fa490060370 con 0x555861fc7430 2023-11-08T19:30:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555861ea8c80 con 0x555861fc7430 2023-11-08T19:30:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa475ffb640 1 -- 172.21.15.119:0/375609530 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa490060570 con 0x555861fc7430 2023-11-08T19:30:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa49ffb7740 10 client.7569 fetch_fsmap learned FSMap version 73 2023-11-08T19:30:16.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.392+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x555861ed2e40 con 0x555861fc7430 2023-11-08T19:30:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa475ffb640 1 -- 172.21.15.119:0/375609530 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555861ea8c80 con 0x555861fc7430 2023-11-08T19:30:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa475ffb640 1 -- 172.21.15.119:0/375609530 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fa49001bb70 con 0x555861fc7430 2023-11-08T19:30:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa49ffb7740 10 client.7569 fetch_fsmap finished waiting for FSMap version 73 2023-11-08T19:30:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa49ffb7740 10 client.7569 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:30:16.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa49ffb7740 20 client.7569 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861f9ab50 0x555861f90eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa49ffb7740 4 client.7569 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:30:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.393+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555861ecd7a0 con 0x555861f9ab50 2023-11-08T19:30:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.394+0000 7fa496ffd640 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861f9ab50 0x555861f90eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.395+0000 7fa475ffb640 10 client.7569 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:16.409 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.395+0000 7fa496ffd640 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861f9ab50 0x555861f90eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f95a823164cd4c1 server_cookie=2d53a3b566f3e4c2 in_seq=0 out_seq=0 2023-11-08T19:30:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.396+0000 7fa475ffb640 1 -- 172.21.15.119:0/375609530 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555861ecd7a0 con 0x555861f9ab50 2023-11-08T19:30:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.396+0000 7fa475ffb640 10 client.7569 handle_command_reply: tid=0 2023-11-08T19:30:16.411 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.396+0000 7fa49ffb7740 1 client.7569 shutdown 2023-11-08T19:30:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.396+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 msgr2=0x555861f8ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.396+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861f8ede0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa4900602d0 tx=0x7fa49005c650 comp rx=0 tx=0).stop 2023-11-08T19:30:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 shutdown_connections 2023-11-08T19:30:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861f8ede0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.412 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861f9ab50 0x555861f90eb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/375609530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x555861f80210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 >> 172.21.15.119:0/375609530 conn(0x555861f69300 msgr2=0x555861f6cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa475ffb640 0 client.7569 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:16.413 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 shutdown_connections 2023-11-08T19:30:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 1 -- 172.21.15.119:0/375609530 wait complete. 2023-11-08T19:30:16.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.397+0000 7fa49ffb7740 20 client.7569 trim_cache size 0 max 16384 2023-11-08T19:30:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.414+0000 7fa49ffb7740 1 Processor -- start 2023-11-08T19:30:16.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.414+0000 7fa49ffb7740 1 -- start start 2023-11-08T19:30:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861fcd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861fcdda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861fc7430 0x555861f92d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861f6d9c0 con 0x555861f95ec0 2023-11-08T19:30:16.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861e795d0 con 0x555861fc7430 2023-11-08T19:30:16.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555861f8f870 con 0x555861f962a0 2023-11-08T19:30:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861fcd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa496ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861fcd860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57084/0 (socket says 172.21.15.119:57084) 2023-11-08T19:30:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.415+0000 7fa496ffd640 1 -- 172.21.15.119:0/862307173 learned_addr learned my addr 172.21.15.119:0/862307173 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.419 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa4977fe640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861fc7430 0x555861f92d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa4967fc640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861fcdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa496ffd640 1 -- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 msgr2=0x555861fcdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa496ffd640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861fcdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa496ffd640 1 -- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861fc7430 msgr2=0x555861f92d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa496ffd640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861fc7430 0x555861f92d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa496ffd640 1 -- 172.21.15.119:0/862307173 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555861e70b90 con 0x555861f95ec0 2023-11-08T19:30:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa4977fe640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861fc7430 0x555861f92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa4967fc640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861fcdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa496ffd640 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861fcd860 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cd50 tx=0x7fa488011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345829293e8cdb7d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa48d7fa640 1 -- 172.21.15.119:0/862307173 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488025040 con 0x555861f95ec0 2023-11-08T19:30:16.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa48d7fa640 1 -- 172.21.15.119:0/862307173 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4880079e0 con 0x555861f95ec0 2023-11-08T19:30:16.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.416+0000 7fa48d7fa640 1 -- 172.21.15.119:0/862307173 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48801b040 con 0x555861f95ec0 2023-11-08T19:30:16.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 -- 172.21.15.119:0/862307173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 msgr2=0x555861fcd860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861fcd860 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa48800cd50 tx=0x7fa488011000 comp rx=0 tx=0).stop 2023-11-08T19:30:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 -- 172.21.15.119:0/862307173 shutdown_connections 2023-11-08T19:30:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861fc7430 0x555861f92d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f962a0 0x555861fcdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/862307173 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861f95ec0 0x555861fcd860 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 -- 172.21.15.119:0/862307173 >> 172.21.15.119:0/862307173 conn(0x555861f69300 msgr2=0x555861f64560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 -- 172.21.15.119:0/862307173 shutdown_connections 2023-11-08T19:30:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.417+0000 7fa49ffb7740 1 -- 172.21.15.119:0/862307173 wait complete. 2023-11-08T19:30:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.418+0000 7fa49ffb7740 1 -- start start 2023-11-08T19:30:16.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.418+0000 7fa49ffb7740 1 Processor -- start 2023-11-08T19:30:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x55586201e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f9c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861fc2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861f9b690 con 0x555861fc7430 2023-11-08T19:30:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x555861fc2f60 con 0x555861f962a0 2023-11-08T19:30:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x555861f9c5e0 con 0x555861f95ec0 2023-11-08T19:30:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa49ffb7740 10 client.0 _ll_register_callbacks cb 0x7fa4900548a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861fc2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa4977fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861fc2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57094/0 (socket says 172.21.15.119:57094) 2023-11-08T19:30:16.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa4977fe640 1 -- 172.21.15.119:0/350692791 learned_addr learned my addr 172.21.15.119:0/350692791 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.419+0000 7fa496ffd640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x55586201e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4967fc640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f9c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4977fe640 1 -- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 msgr2=0x55586201e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4977fe640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x55586201e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4977fe640 1 -- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 msgr2=0x555861f9c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4977fe640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f9c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4977fe640 1 -- 172.21.15.119:0/350692791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555861ea7b10 con 0x555861fc7430 2023-11-08T19:30:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa4977fe640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861fc2320 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa49001bfb0 tx=0x7fa490020070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ade98a07c7267c3f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa475ffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:16.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa475ffb640 1 -- 172.21.15.119:0/350692791 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490060570 con 0x555861fc7430 2023-11-08T19:30:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa475ffb640 1 -- 172.21.15.119:0/350692791 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fa490058c00 con 0x555861fc7430 2023-11-08T19:30:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa48800ce70 con 0x555861fc7430 2023-11-08T19:30:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.420+0000 7fa475ffb640 1 -- 172.21.15.119:0/350692791 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa490058040 con 0x555861fc7430 2023-11-08T19:30:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 10 client.7575 fetch_fsmap learned FSMap version 73 2023-11-08T19:30:16.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=2+}) v3 -- 0x555861e96180 con 0x555861fc7430 2023-11-08T19:30:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa475ffb640 1 -- 172.21.15.119:0/350692791 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490064020 con 0x555861fc7430 2023-11-08T19:30:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa475ffb640 1 -- 172.21.15.119:0/350692791 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 73) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fa49007b070 con 0x555861fc7430 2023-11-08T19:30:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 10 client.7575 fetch_fsmap finished waiting for FSMap version 73 2023-11-08T19:30:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 10 client.7575 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:30:16.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 20 client.7575 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861fcd860 0x555861fcdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 4 client.7575 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:30:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.421+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555861ecd4c0 con 0x555861fcd860 2023-11-08T19:30:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.422+0000 7fa496ffd640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861fcd860 0x555861fcdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.424+0000 7fa496ffd640 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861fcd860 0x555861fcdc40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f522e9d9b9dbc0c server_cookie=f799a93d34093464 in_seq=0 out_seq=0 2023-11-08T19:30:16.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.424+0000 7fa475ffb640 10 client.7575 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:16.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa475ffb640 1 -- 172.21.15.119:0/350692791 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x555861ecd4c0 con 0x555861fcd860 2023-11-08T19:30:16.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa475ffb640 10 client.7575 handle_command_reply: tid=0 2023-11-08T19:30:16.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa49ffb7740 1 client.7575 shutdown 2023-11-08T19:30:16.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 msgr2=0x555861fc2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861fc2320 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fa49001bfb0 tx=0x7fa490020070 comp rx=0 tx=0).stop 2023-11-08T19:30:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 shutdown_connections 2023-11-08T19:30:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.425+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x555861fc7430 0x555861fc2320 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x555861fcd860 0x555861fcdc40 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x555861f962a0 0x555861f9c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 1 --2- 172.21.15.119:0/350692791 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x555861f95ec0 0x55586201e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 >> 172.21.15.119:0/350692791 conn(0x555861f69300 msgr2=0x555861f67e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa475ffb640 0 client.7575 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 shutdown_connections 2023-11-08T19:30:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 1 -- 172.21.15.119:0/350692791 wait complete. 2023-11-08T19:30:16.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49ffb7740 20 client.7575 trim_cache size 0 max 16384 2023-11-08T19:30:16.438 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:30:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:30:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-11-08T19:30:16.438 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7065, 2023-11-08T19:30:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:30:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:30:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:30:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 67.745788078999993, 2023-11-08T19:30:16.439 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 73, 2023-11-08T19:30:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 99, 2023-11-08T19:30:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 99, 2023-11-08T19:30:16.440 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 90.799233674999996 2023-11-08T19:30:16.440 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:30:16.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa47806ae70 msgr2=0x7fa47806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa47806ae70 0x7fa47806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa488007fd0 tx=0x7fa488001810 comp rx=0 tx=0).stop 2023-11-08T19:30:16.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 msgr2=0x7fa4980b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 0x7fa4980b2350 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fa49815b740 tx=0x7fa490011920 comp rx=0 tx=0).stop 2023-11-08T19:30:16.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 shutdown_connections 2023-11-08T19:30:16.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa47806ae70 0x7fa47806d330 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa49806c790 0x7fa4980b2350 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa49806b230 0x7fa49815c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 --2- 172.21.15.119:0/4181090106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4840047c0 0x7fa49814b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.426+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 >> 172.21.15.119:0/4181090106 conn(0x7fa49808afe0 msgr2=0x7fa4981597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.427+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 shutdown_connections 2023-11-08T19:30:16.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.427+0000 7fa49d34c640 1 -- 172.21.15.119:0/4181090106 wait complete. 2023-11-08T19:30:16.445 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:16.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.641+0000 7f09993a9640 1 Processor -- start 2023-11-08T19:30:16.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.641+0000 7f09993a9640 1 -- start start 2023-11-08T19:30:16.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.643+0000 7f09993a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 0x7f09940cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.643+0000 7f09993a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09940c4960 con 0x7f099406cb40 2023-11-08T19:30:16.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.643+0000 7f09993a9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f099415df60 con 0x7f0994142f80 2023-11-08T19:30:16.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.643+0000 7f09993a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0994143a40 con 0x7f099415d9f0 2023-11-08T19:30:16.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.644+0000 7f09937fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f099406cb40 0x7f09940c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48136/0 (socket says 172.21.15.119:48136) 2023-11-08T19:30:16.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.644+0000 7f09937fe640 1 -- 172.21.15.119:0/1982798437 learned_addr learned my addr 172.21.15.119:0/1982798437 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:16.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.644+0000 7f098a7fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 0x7f09940cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.644+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1735455570 0 0) 0x7f09940c4960 con 0x7f099406cb40 2023-11-08T19:30:16.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.644+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0980005730 con 0x7f099406cb40 2023-11-08T19:30:16.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.644+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2637209850 0 0) 0x7f0980005730 con 0x7f099406cb40 2023-11-08T19:30:16.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 msgr2=0x7f09940cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 0x7f09940cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 >> v1:172.21.15.188:6789/0 conn(0x7f0994142f80 legacy=0x7f09940c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f099415e240 con 0x7f099406cb40 2023-11-08T19:30:16.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f098c0019f0 con 0x7f099406cb40 2023-11-08T19:30:16.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 >> v1:172.21.15.119:6789/0 conn(0x7f099406cb40 legacy=0x7f09940c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 0x7f0980005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006180 0x7f09800069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0980006f10 0x7f0980007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0994143a40 con 0x7f0980006f10 2023-11-08T19:30:16.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098000dd80 con 0x7f0980006180 2023-11-08T19:30:16.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098000df40 con 0x7f099415d9f0 2023-11-08T19:30:16.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09937fe640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0980006f10 0x7f0980007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f098a7fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006180 0x7f09800069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f0992ffd640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 0x7f0980005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.645+0000 7f09937fe640 1 -- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 msgr2=0x7f0980005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.646+0000 7f09937fe640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f099415d9f0 0x7f0980005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.646+0000 7f09937fe640 1 -- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006180 msgr2=0x7f09800069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:16.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.646+0000 7f09937fe640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006180 0x7f09800069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.646+0000 7f09937fe640 1 -- 172.21.15.119:0/1982798437 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0980005730 con 0x7f0980006f10 2023-11-08T19:30:16.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.646+0000 7f098a7fc640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006180 0x7f09800069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:16.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09937fe640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0980006f10 0x7f0980007390 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a760 tx=0x7f0980007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de5f98533297f5a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c034630 con 0x7f0980006f10 2023-11-08T19:30:16.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c02f070 con 0x7f0980006f10 2023-11-08T19:30:16.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09927fc640 1 -- 172.21.15.119:0/1982798437 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c0349f0 con 0x7f0980006f10 2023-11-08T19:30:16.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09993a9640 1 -- 172.21.15.119:0/1982798437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0980006f10 msgr2=0x7f0980007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09993a9640 1 --2- 172.21.15.119:0/1982798437 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0980006f10 0x7f0980007390 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a760 tx=0x7f0980007a00 comp rx=0 tx=0).stop 2023-11-08T19:30:16.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.647+0000 7f09937fe640 1 -- 172.21.15.119:0/1982798437 reap_dead start 2023-11-08T19:30:16.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.648+0000 7f09993a9640 1 -- 172.21.15.119:0/1982798437 shutdown_connections 2023-11-08T19:30:16.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.648+0000 7f09993a9640 1 -- 172.21.15.119:0/1982798437 >> 172.21.15.119:0/1982798437 conn(0x7f09940cb610 msgr2=0x7f09940cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.648+0000 7f09993a9640 1 -- 172.21.15.119:0/1982798437 shutdown_connections 2023-11-08T19:30:16.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.648+0000 7f09993a9640 1 -- 172.21.15.119:0/1982798437 wait complete. 2023-11-08T19:30:16.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.649+0000 7f09993a9640 1 Processor -- start 2023-11-08T19:30:16.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.650+0000 7f09993a9640 1 -- start start 2023-11-08T19:30:16.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.650+0000 7f09993a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006f10 0x7f09940c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.650+0000 7f09993a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09941f5f20 0x7f09941593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09993a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 0x7f099413ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09993a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940c7860 con 0x7f09940b3750 2023-11-08T19:30:16.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09993a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940cc6d0 con 0x7f0980006f10 2023-11-08T19:30:16.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09993a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09941f6490 con 0x7f09941f5f20 2023-11-08T19:30:16.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 0x7f099413ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 0x7f099413ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57102/0 (socket says 172.21.15.119:57102) 2023-11-08T19:30:16.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 -- 172.21.15.119:0/603743718 learned_addr learned my addr 172.21.15.119:0/603743718 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:16.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f0992ffd640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006f10 0x7f09940c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 -- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09941f5f20 msgr2=0x7f09941593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09941f5f20 0x7f09941593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 -- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006f10 msgr2=0x7f09940c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006f10 0x7f09940c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.651+0000 7f09937fe640 1 -- 172.21.15.119:0/603743718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09941f6770 con 0x7f09940b3750 2023-11-08T19:30:16.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.652+0000 7f09937fe640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 0x7f099413ff10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a760 tx=0x7f098c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.652+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c034430 con 0x7f09940b3750 2023-11-08T19:30:16.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.652+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c034590 con 0x7f09940b3750 2023-11-08T19:30:16.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.652+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09941f6a50 con 0x7f09940b3750 2023-11-08T19:30:16.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.652+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c034900 con 0x7f09940b3750 2023-11-08T19:30:16.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.652+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09941598e0 con 0x7f09940b3750 2023-11-08T19:30:16.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.653+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0994074430 con 0x7f09940b3750 2023-11-08T19:30:16.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.654+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f098c031030 con 0x7f09940b3750 2023-11-08T19:30:16.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.655+0000 7f098bfff640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f097406aea0 0x7f097406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:16.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.655+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f098c0abfe0 con 0x7f09940b3750 2023-11-08T19:30:16.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.655+0000 7f0992ffd640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f097406aea0 0x7f097406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:16.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.656+0000 7f0992ffd640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f097406aea0 0x7f097406d360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0994141060 tx=0x7f0984009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:16.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.660+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f098c076aa0 con 0x7f09940b3750 2023-11-08T19:30:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.889+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0994074430 con 0x7f09940b3750 2023-11-08T19:30:16.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.890+0000 7f098bfff640 1 -- 172.21.15.119:0/603743718 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f098c07a0c0 con 0x7f09940b3750 2023-11-08T19:30:16.892 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:16.892 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:12.088177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:16.892 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 73 2023-11-08T19:30:16.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f097406aea0 msgr2=0x7f097406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f097406aea0 0x7f097406d360 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f0994141060 tx=0x7f0984009040 comp rx=0 tx=0).stop 2023-11-08T19:30:16.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 msgr2=0x7f099413ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:16.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 0x7f099413ff10 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f098c00a760 tx=0x7f098c001cb0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 shutdown_connections 2023-11-08T19:30:16.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f097406aea0 0x7f097406d360 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f09940b3750 0x7f099413ff10 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f09941f5f20 0x7f09941593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 --2- 172.21.15.119:0/603743718 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0980006f10 0x7f09940c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:16.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 >> 172.21.15.119:0/603743718 conn(0x7f09940cb610 msgr2=0x7f099406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:16.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 shutdown_connections 2023-11-08T19:30:16.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:16.893+0000 7f09993a9640 1 -- 172.21.15.119:0/603743718 wait complete. 2023-11-08T19:30:16.914 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-08T19:30:17.156 INFO:teuthology.orchestra.run.smithi188.stdout:{"last_snap":3,"last_created":3,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:29:23.170262+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-08T19:29:51.023531+0000","name":"s12","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:30:17.156 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 3, "last_destroyed": 1, "last_snap": 3, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:29:23.170262+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-08T19:29:51.023531+0000" } ] } 2023-11-08T19:30:17.157 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:30:17.157 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=5 2023-11-08T19:30:17.157 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:17.215 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:30:17.213+0000 7f7ea8e9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:17.215 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:30:17.213+0000 7f7ea8e9c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:17.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.374+0000 7f4d8dad4640 1 Processor -- start 2023-11-08T19:30:17.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.374+0000 7f4d8dad4640 1 -- start start 2023-11-08T19:30:17.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d8dad4640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d8815f4c0 0x7f4d8806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d8dad4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d8806a120 0x7f4d8806a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d8dad4640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8806aa70 con 0x7f4d881447a0 2023-11-08T19:30:17.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d8dad4640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d8815f930 con 0x7f4d8815f4c0 2023-11-08T19:30:17.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d8dad4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d880c3a50 con 0x7f4d8806a120 2023-11-08T19:30:17.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d86ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d8815f4c0 0x7f4d8806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d877fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f4d881447a0 0x7f4d880cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:48148/0 (socket says 172.21.15.119:48148) 2023-11-08T19:30:17.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.376+0000 7f4d877fe640 1 -- 172.21.15.119:0/2328280508 learned_addr learned my addr 172.21.15.119:0/2328280508 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:17.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.377+0000 7f4d87fff640 1 --2- 172.21.15.119:0/2328280508 >> v2:172.21.15.188:3300/0 conn(0x7f4d8806a120 0x7f4d8806a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.377+0000 7f4d86ffd640 1 -- 172.21.15.119:0/2328280508 >> v2:172.21.15.188:3300/0 conn(0x7f4d8806a120 msgr2=0x7f4d8806a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:17.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.377+0000 7f4d86ffd640 1 --2- 172.21.15.119:0/2328280508 >> v2:172.21.15.188:3300/0 conn(0x7f4d8806a120 0x7f4d8806a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.377+0000 7f4d86ffd640 1 -- 172.21.15.119:0/2328280508 >> v1:172.21.15.119:6789/0 conn(0x7f4d881447a0 legacy=0x7f4d880cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.377+0000 7f4d86ffd640 1 -- 172.21.15.119:0/2328280508 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d88144b80 con 0x7f4d8815f4c0 2023-11-08T19:30:17.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.377+0000 7f4d87fff640 1 --2- 172.21.15.119:0/2328280508 >> v2:172.21.15.188:3300/0 conn(0x7f4d8806a120 0x7f4d8806a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:17.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.378+0000 7f4d86ffd640 1 --2- 172.21.15.119:0/2328280508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8815f4c0 0x7f4d8806f930 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4d880b1850 tx=0x7f4d7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a154b640fbc2a477 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:17.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.378+0000 7f4d867fc640 1 -- 172.21.15.119:0/2328280508 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70031830 con 0x7f4d8815f4c0 2023-11-08T19:30:17.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.378+0000 7f4d867fc640 1 -- 172.21.15.119:0/2328280508 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d70031990 con 0x7f4d8815f4c0 2023-11-08T19:30:17.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.378+0000 7f4d867fc640 1 -- 172.21.15.119:0/2328280508 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d70031d20 con 0x7f4d8815f4c0 2023-11-08T19:30:17.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 -- 172.21.15.119:0/2328280508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8815f4c0 msgr2=0x7f4d8806f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/2328280508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8815f4c0 0x7f4d8806f930 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4d880b1850 tx=0x7f4d7002b8d0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 -- 172.21.15.119:0/2328280508 shutdown_connections 2023-11-08T19:30:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/2328280508 >> v2:172.21.15.188:3300/0 conn(0x7f4d8806a120 0x7f4d8806a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/2328280508 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8815f4c0 0x7f4d8806f930 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 -- 172.21.15.119:0/2328280508 >> 172.21.15.119:0/2328280508 conn(0x7f4d880cb610 msgr2=0x7f4d880cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:17.384 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 -- 172.21.15.119:0/2328280508 shutdown_connections 2023-11-08T19:30:17.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.379+0000 7f4d8dad4640 1 -- 172.21.15.119:0/2328280508 wait complete. 2023-11-08T19:30:17.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.381+0000 7f4d8dad4640 1 Processor -- start 2023-11-08T19:30:17.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.381+0000 7f4d8dad4640 1 -- start start 2023-11-08T19:30:17.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d8dad4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 0x7f4d880c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d8dad4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d881447a0 0x7f4d8815dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d8dad4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8815f4c0 0x7f4d880a86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d8dad4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d881f5650 con 0x7f4d8806a120 2023-11-08T19:30:17.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d8dad4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d88140740 con 0x7f4d8815f4c0 2023-11-08T19:30:17.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d8dad4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d881f4520 con 0x7f4d881447a0 2023-11-08T19:30:17.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 0x7f4d880c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 0x7f4d880c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57120/0 (socket says 172.21.15.119:57120) 2023-11-08T19:30:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.382+0000 7f4d877fe640 1 -- 172.21.15.119:0/659969721 learned_addr learned my addr 172.21.15.119:0/659969721 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d877fe640 1 -- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d881447a0 msgr2=0x7f4d8815dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d86ffd640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d881447a0 0x7f4d8815dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d87fff640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8815f4c0 0x7f4d880a86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d877fe640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d881447a0 0x7f4d8815dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d877fe640 1 -- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8815f4c0 msgr2=0x7f4d880a86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d877fe640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8815f4c0 0x7f4d880a86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d877fe640 1 -- 172.21.15.119:0/659969721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d70009400 con 0x7f4d8806a120 2023-11-08T19:30:17.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d877fe640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 0x7f4d880c8010 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c450 tx=0x7f4d7800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78005ce0 con 0x7f4d8806a120 2023-11-08T19:30:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d78005e40 con 0x7f4d8806a120 2023-11-08T19:30:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d881f4730 con 0x7f4d8806a120 2023-11-08T19:30:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d78007310 con 0x7f4d8806a120 2023-11-08T19:30:17.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.383+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d881f4f90 con 0x7f4d8806a120 2023-11-08T19:30:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.385+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d88074430 con 0x7f4d8806a120 2023-11-08T19:30:17.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.385+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d78007470 con 0x7f4d8806a120 2023-11-08T19:30:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.386+0000 7f4d7ffff640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6806ae70 0x7f4d6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.386+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f4d78089250 con 0x7f4d8806a120 2023-11-08T19:30:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.387+0000 7f4d86ffd640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6806ae70 0x7f4d6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.387+0000 7f4d86ffd640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6806ae70 0x7f4d6806d330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4d70036000 tx=0x7f4d70008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:17.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.391+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d78053c80 con 0x7f4d8806a120 2023-11-08T19:30:17.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.618+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d880c5960 con 0x7f4d8806a120 2023-11-08T19:30:17.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.619+0000 7f4d7ffff640 1 -- 172.21.15.119:0/659969721 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4d780572a0 con 0x7f4d8806a120 2023-11-08T19:30:17.621 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:17.621 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":73,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:12.088177+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:17.621 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 73 2023-11-08T19:30:17.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.621+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6806ae70 msgr2=0x7f4d6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.621+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6806ae70 0x7f4d6806d330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4d70036000 tx=0x7f4d70008030 comp rx=0 tx=0).stop 2023-11-08T19:30:17.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.621+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 msgr2=0x7f4d880c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.621+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 0x7f4d880c8010 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800c450 tx=0x7f4d7800c910 comp rx=0 tx=0).stop 2023-11-08T19:30:17.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 shutdown_connections 2023-11-08T19:30:17.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6806ae70 0x7f4d6806d330 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8815f4c0 0x7f4d880a86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d881447a0 0x7f4d8815dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 --2- 172.21.15.119:0/659969721 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d8806a120 0x7f4d880c8010 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 >> 172.21.15.119:0/659969721 conn(0x7f4d880cb610 msgr2=0x7f4d880c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:17.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 shutdown_connections 2023-11-08T19:30:17.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.622+0000 7f4d8dad4640 1 -- 172.21.15.119:0/659969721 wait complete. 2023-11-08T19:30:17.643 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:30:17.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.847+0000 7f848dbc9640 1 Processor -- start 2023-11-08T19:30:17.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.847+0000 7f848dbc9640 1 -- start start 2023-11-08T19:30:17.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.848+0000 7f848dbc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 0x7f848815bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.848+0000 7f848dbc9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8488144960 0x7f84880cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.848+0000 7f848dbc9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848815ea60 con 0x7f848815f1d0 2023-11-08T19:30:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.848+0000 7f848dbc9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84880c9150 con 0x7f8488144960 2023-11-08T19:30:17.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.848+0000 7f848dbc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880cb270 con 0x7f8488144580 2023-11-08T19:30:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.849+0000 7f84877fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 0x7f848815bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.849+0000 7f847ffff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8488144960 0x7f84880cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.849+0000 7f84877fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 0x7f848815bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:48738/0 (socket says 172.21.15.119:48738) 2023-11-08T19:30:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.849+0000 7f84877fe640 1 -- 172.21.15.119:0/232664938 learned_addr learned my addr 172.21.15.119:0/232664938 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.850+0000 7f84877fe640 1 -- 172.21.15.119:0/232664938 >> v2:172.21.15.188:3300/0 conn(0x7f8488144960 msgr2=0x7f84880cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.850+0000 7f84877fe640 1 --2- 172.21.15.119:0/232664938 >> v2:172.21.15.188:3300/0 conn(0x7f8488144960 0x7f84880cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.850+0000 7f84877fe640 1 -- 172.21.15.119:0/232664938 >> v1:172.21.15.188:6789/0 conn(0x7f848815f1d0 legacy=0x7f84880c3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.850+0000 7f84877fe640 1 -- 172.21.15.119:0/232664938 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84880c9360 con 0x7f8488144580 2023-11-08T19:30:17.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.851+0000 7f84877fe640 1 --2- 172.21.15.119:0/232664938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 0x7f848815bff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8478009870 tx=0x7f847802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36664af011eb3479 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:17.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.852+0000 7f8486ffd640 1 -- 172.21.15.119:0/232664938 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478031640 con 0x7f8488144580 2023-11-08T19:30:17.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.852+0000 7f8486ffd640 1 -- 172.21.15.119:0/232664938 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84780317a0 con 0x7f8488144580 2023-11-08T19:30:17.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.852+0000 7f8486ffd640 1 -- 172.21.15.119:0/232664938 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478031b30 con 0x7f8488144580 2023-11-08T19:30:17.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 -- 172.21.15.119:0/232664938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 msgr2=0x7f848815bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 --2- 172.21.15.119:0/232664938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 0x7f848815bff0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8478009870 tx=0x7f847802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 -- 172.21.15.119:0/232664938 shutdown_connections 2023-11-08T19:30:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 --2- 172.21.15.119:0/232664938 >> v2:172.21.15.188:3300/0 conn(0x7f8488144960 0x7f84880cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 --2- 172.21.15.119:0/232664938 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8488144580 0x7f848815bff0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 -- 172.21.15.119:0/232664938 >> 172.21.15.119:0/232664938 conn(0x7f84880cb610 msgr2=0x7f84880cba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:17.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 -- 172.21.15.119:0/232664938 shutdown_connections 2023-11-08T19:30:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.853+0000 7f848dbc9640 1 -- 172.21.15.119:0/232664938 wait complete. 2023-11-08T19:30:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.854+0000 7f848dbc9640 1 Processor -- start 2023-11-08T19:30:17.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.855+0000 7f848dbc9640 1 -- start start 2023-11-08T19:30:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f848dbc9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 0x7f84880c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f848dbc9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8488144960 0x7f84881f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f848dbc9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848815f1d0 0x7f848813d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f848dbc9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84881f35f0 con 0x7f8488144580 2023-11-08T19:30:17.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f848dbc9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c6e10 con 0x7f8488144960 2023-11-08T19:30:17.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f848dbc9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84880c6290 con 0x7f848815f1d0 2023-11-08T19:30:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 0x7f84880c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 0x7f84880c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:57126/0 (socket says 172.21.15.119:57126) 2023-11-08T19:30:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 -- 172.21.15.119:0/3711209087 learned_addr learned my addr 172.21.15.119:0/3711209087 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:17.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f847ffff640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8488144960 0x7f84881f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f8487fff640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848815f1d0 0x7f848813d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 -- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848815f1d0 msgr2=0x7f848813d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848815f1d0 0x7f848813d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 -- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8488144960 msgr2=0x7f84881f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8488144960 0x7f84881f30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f84877fe640 1 -- 172.21.15.119:0/3711209087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780094d0 con 0x7f8488144580 2023-11-08T19:30:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f8487fff640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848815f1d0 0x7f848813d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.856+0000 7f847ffff640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8488144960 0x7f84881f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:17.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.857+0000 7f84877fe640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 0x7f84880c5bd0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8478000c00 tx=0x7f84780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.857+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847803a930 con 0x7f8488144580 2023-11-08T19:30:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.857+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847803aa90 con 0x7f8488144580 2023-11-08T19:30:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.857+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847803ae40 con 0x7f8488144580 2023-11-08T19:30:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.857+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84880c6570 con 0x7f8488144580 2023-11-08T19:30:17.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.857+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84880c6b30 con 0x7f8488144580 2023-11-08T19:30:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.858+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8488074430 con 0x7f8488144580 2023-11-08T19:30:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.859+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f847802f050 con 0x7f8488144580 2023-11-08T19:30:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.860+0000 7f8484ff9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846806ae70 0x7f846806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.860+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f84780abb40 con 0x7f8488144580 2023-11-08T19:30:17.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.860+0000 7f847ffff640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846806ae70 0x7f846806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.861+0000 7f847ffff640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846806ae70 0x7f846806d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8488142070 tx=0x7f8470008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:17.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:17.864+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84780764e0 con 0x7f8488144580 2023-11-08T19:30:18.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:18.125+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f848806cb10 con 0x7f8488144580 2023-11-08T19:30:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.129+0000 7f8484ff9640 1 -- 172.21.15.119:0/3711209087 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5144 v74) v1 ==== 112+0+0 (secure 0 0 0) 0x7f8478079b00 con 0x7f8488144580 2023-11-08T19:30:19.131 INFO:teuthology.orchestra.run.smithi119.stderr:freezing mds.5144 2023-11-08T19:30:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846806ae70 msgr2=0x7f846806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846806ae70 0x7f846806d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8488142070 tx=0x7f8470008040 comp rx=0 tx=0).stop 2023-11-08T19:30:19.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 msgr2=0x7f84880c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 0x7f84880c5bd0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f8478000c00 tx=0x7f84780046c0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 shutdown_connections 2023-11-08T19:30:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f846806ae70 0x7f846806d330 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f848815f1d0 0x7f848813d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.133+0000 7f848dbc9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8488144960 0x7f84881f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.134+0000 7f848dbc9640 1 --2- 172.21.15.119:0/3711209087 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8488144580 0x7f84880c5bd0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.134+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 >> 172.21.15.119:0/3711209087 conn(0x7f84880cb610 msgr2=0x7f848806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.134+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 shutdown_connections 2023-11-08T19:30:19.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.134+0000 7f848dbc9640 1 -- 172.21.15.119:0/3711209087 wait complete. 2023-11-08T19:30:19.156 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 5 2023-11-08T19:30:19.391 INFO:teuthology.orchestra.run.smithi188.stdout:{"success":"mds_kill_mdstable_at = '5' (not observed, change may require restart) "} 2023-11-08T19:30:19.391 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '5' (not observed, change may require restart) " } 2023-11-08T19:30:19.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:30:19.393 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s15) 2023-11-08T19:30:19.402 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:30:19.467 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f7eab6a1640 time 2023-11-08T19:30:19.467305+0000 2023-11-08T19:30:19.468 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:30:19.469 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.469 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7eb116efaf] 2023-11-08T19:30:19.469 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.469 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.469 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.470 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.470 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.470 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.470 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.470 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.471 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.471 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.471 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:30:19.467+0000 7f7eab6a1640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f7eab6a1640 time 2023-11-08T19:30:19.467305+0000 2023-11-08T19:30:19.471 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:30:19.471 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.472 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.472 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7eb116efaf] 2023-11-08T19:30:19.472 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.472 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.472 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.473 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.473 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.473 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.473 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.473 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.474 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.474 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.474 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.474 INFO:tasks.ceph.mds.b.smithi188.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:30:19.474 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f7eab6a1640 thread_name:ms_dispatch 2023-11-08T19:30:19.475 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.475 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f7eb0854db0] 2023-11-08T19:30:19.475 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f7eb08a154c] 2023-11-08T19:30:19.476 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:30:19.476 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:30:19.476 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7eb116f009] 2023-11-08T19:30:19.476 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.476 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.477 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.477 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.477 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.477 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.477 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.478 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.478 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.478 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.478 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:30:19.470+0000 7f7eab6a1640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:19.478 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f7eab6a1640 thread_name:ms_dispatch 2023-11-08T19:30:19.479 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.479 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.479 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f7eb0854db0] 2023-11-08T19:30:19.479 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f7eb08a154c] 2023-11-08T19:30:19.479 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:30:19.479 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:30:19.480 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7eb116f009] 2023-11-08T19:30:19.480 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.480 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.480 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.480 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.481 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.481 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.481 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.481 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.481 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.482 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.482 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:19.482 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.497 INFO:tasks.ceph.mds.b.smithi188.stderr: -217> 2023-11-08T19:30:17.213+0000 7f7ea8e9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:19.497 INFO:tasks.ceph.mds.b.smithi188.stderr: -215> 2023-11-08T19:30:17.213+0000 7f7ea8e9c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:19.498 INFO:tasks.ceph.mds.b.smithi188.stderr: -1> 2023-11-08T19:30:19.467+0000 7f7eab6a1640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f7eab6a1640 time 2023-11-08T19:30:19.467305+0000 2023-11-08T19:30:19.498 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:30:19.499 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.499 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.499 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7eb116efaf] 2023-11-08T19:30:19.499 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.499 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.500 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.500 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.500 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.500 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.500 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.501 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.501 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.501 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.501 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.501 INFO:tasks.ceph.mds.b.smithi188.stderr: 0> 2023-11-08T19:30:19.470+0000 7f7eab6a1640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:19.502 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f7eab6a1640 thread_name:ms_dispatch 2023-11-08T19:30:19.502 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.502 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.502 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f7eb0854db0] 2023-11-08T19:30:19.502 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f7eb08a154c] 2023-11-08T19:30:19.503 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:30:19.503 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:30:19.503 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7eb116f009] 2023-11-08T19:30:19.503 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.503 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.504 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.504 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.504 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.504 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.504 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.505 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.505 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.505 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.505 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:19.505 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.516 INFO:tasks.ceph.mds.b.smithi188.stderr: -217> 2023-11-08T19:30:17.213+0000 7f7ea8e9c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:19.516 INFO:tasks.ceph.mds.b.smithi188.stderr: -215> 2023-11-08T19:30:17.213+0000 7f7ea8e9c640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:19.517 INFO:tasks.ceph.mds.b.smithi188.stderr: -1> 2023-11-08T19:30:19.467+0000 7f7eab6a1640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::handle_commit(ceph::cref_t&)' thread 7f7eab6a1640 time 2023-11-08T19:30:19.467305+0000 2023-11-08T19:30:19.517 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 145: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 5) 2023-11-08T19:30:19.517 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.517 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.518 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f7eb116efaf] 2023-11-08T19:30:19.518 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.518 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.519 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.519 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.519 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.519 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.519 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.520 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.520 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.520 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.520 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.520 INFO:tasks.ceph.mds.b.smithi188.stderr: 0> 2023-11-08T19:30:19.470+0000 7f7eab6a1640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:19.521 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7f7eab6a1640 thread_name:ms_dispatch 2023-11-08T19:30:19.521 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.521 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:19.521 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f7eb0854db0] 2023-11-08T19:30:19.521 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f7eb08a154c] 2023-11-08T19:30:19.521 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:30:19.522 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:30:19.522 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f7eb116f009] 2023-11-08T19:30:19.522 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f7eb116f16d] 2023-11-08T19:30:19.522 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::handle_commit(boost::intrusive_ptr const&)+0x7e7) [0x5588b660b187] 2023-11-08T19:30:19.522 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSRank::handle_message(boost::intrusive_ptr const&)+0xe9) [0x5588b639a0a9] 2023-11-08T19:30:19.523 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSRank::_dispatch(boost::intrusive_ptr const&, bool)+0x4eb) [0x5588b639cb2b] 2023-11-08T19:30:19.523 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSRankDispatcher::ms_dispatch(boost::intrusive_ptr const&)+0x5c) [0x5588b639d01c] 2023-11-08T19:30:19.523 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (MDSDaemon::ms_dispatch2(boost::intrusive_ptr const&)+0x195) [0x5588b6386325] 2023-11-08T19:30:19.523 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: (DispatchQueue::entry()+0x53a) [0x7f7eb135ff9a] 2023-11-08T19:30:19.523 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /usr/lib64/ceph/libceph-common.so.2(+0x3f1741) [0x7f7eb13f1741] 2023-11-08T19:30:19.524 INFO:tasks.ceph.mds.b.smithi188.stderr: 14: /lib64/libc.so.6(+0x9f802) [0x7f7eb089f802] 2023-11-08T19:30:19.524 INFO:tasks.ceph.mds.b.smithi188.stderr: 15: /lib64/libc.so.6(+0x3f450) [0x7f7eb083f450] 2023-11-08T19:30:19.524 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:19.524 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:30:19.649 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:30:19.649 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:30:19.650 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:19.793 INFO:tasks.ceph.mds.b.smithi188.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:30:19.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.861+0000 7f9ebcafc640 1 Processor -- start 2023-11-08T19:30:19.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.861+0000 7f9ebcafc640 1 -- start start 2023-11-08T19:30:19.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.863+0000 7f9ebcafc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb806a390 0x7f9eb80c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.863+0000 7f9ebcafc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9eb815d180 0x7f9eb806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.863+0000 7f9ebcafc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb80ae410 con 0x7f9eb80cc810 2023-11-08T19:30:19.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.863+0000 7f9ebcafc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb806ba30 con 0x7f9eb815d180 2023-11-08T19:30:19.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.863+0000 7f9ebcafc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb806bcf0 con 0x7f9eb806a390 2023-11-08T19:30:19.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9eb815d180 0x7f9eb806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f9eb815d180 0x7f9eb806cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60140/0 (socket says 172.21.15.119:60140) 2023-11-08T19:30:19.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/3160458406 learned_addr learned my addr 172.21.15.119:0/3160458406 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb659c640 1 --2- 172.21.15.119:0/3160458406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb806a390 0x7f9eb80c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/3160458406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb806a390 msgr2=0x7f9eb80c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 --2- 172.21.15.119:0/3160458406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb806a390 0x7f9eb80c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/3160458406 >> v1:172.21.15.188:6789/0 conn(0x7f9eb80cc810 legacy=0x7f9eb815ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:19.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.864+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/3160458406 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb806bfd0 con 0x7f9eb815d180 2023-11-08T19:30:19.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.865+0000 7f9eb6d9d640 1 --2- 172.21.15.119:0/3160458406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb806cf70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009e50 tx=0x7f9eb002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8fd61e2628f413 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.865+0000 7f9eb5d9b640 1 -- 172.21.15.119:0/3160458406 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb002ca80 con 0x7f9eb815d180 2023-11-08T19:30:19.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.865+0000 7f9eb5d9b640 1 -- 172.21.15.119:0/3160458406 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb002cbe0 con 0x7f9eb815d180 2023-11-08T19:30:19.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9eb5d9b640 1 -- 172.21.15.119:0/3160458406 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb00343f0 con 0x7f9eb815d180 2023-11-08T19:30:19.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 -- 172.21.15.119:0/3160458406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 msgr2=0x7f9eb806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/3160458406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb806cf70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009e50 tx=0x7f9eb002bee0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 -- 172.21.15.119:0/3160458406 shutdown_connections 2023-11-08T19:30:19.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/3160458406 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb806cf70 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/3160458406 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb806a390 0x7f9eb80c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 -- 172.21.15.119:0/3160458406 >> 172.21.15.119:0/3160458406 conn(0x7f9eb80cb610 msgr2=0x7f9eb80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:19.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 -- 172.21.15.119:0/3160458406 shutdown_connections 2023-11-08T19:30:19.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.866+0000 7f9ebcafc640 1 -- 172.21.15.119:0/3160458406 wait complete. 2023-11-08T19:30:19.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.868+0000 7f9ebcafc640 1 Processor -- start 2023-11-08T19:30:19.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.868+0000 7f9ebcafc640 1 -- start start 2023-11-08T19:30:19.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9ebcafc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb806a390 0x7f9eb81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9ebcafc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb80cc810 0x7f9eb80b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9ebcafc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb8157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9ebcafc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8156f00 con 0x7f9eb815d180 2023-11-08T19:30:19.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9ebcafc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb815dca0 con 0x7f9eb806a390 2023-11-08T19:30:19.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9ebcafc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb81f7fc0 con 0x7f9eb80cc810 2023-11-08T19:30:19.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9eb6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb8157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9eb6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb8157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60152/0 (socket says 172.21.15.119:60152) 2023-11-08T19:30:19.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.869+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/1799532436 learned_addr learned my addr 172.21.15.119:0/1799532436 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:19.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb659c640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb806a390 0x7f9eb81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eaffff640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb80cc810 0x7f9eb80b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb80cc810 msgr2=0x7f9eb80b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb6d9d640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb80cc810 0x7f9eb80b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb806a390 msgr2=0x7f9eb81569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:19.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb6d9d640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb806a390 0x7f9eb81569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:19.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb6d9d640 1 -- 172.21.15.119:0/1799532436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb0009b30 con 0x7f9eb815d180 2023-11-08T19:30:19.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eaffff640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb80cc810 0x7f9eb80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eb6d9d640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb8157610 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0036040 tx=0x7f9eb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.870+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0040670 con 0x7f9eb815d180 2023-11-08T19:30:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.871+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb00407f0 con 0x7f9eb815d180 2023-11-08T19:30:19.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.871+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0040ba0 con 0x7f9eb815d180 2023-11-08T19:30:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.871+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb81f82a0 con 0x7f9eb815d180 2023-11-08T19:30:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.871+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb815d930 con 0x7f9eb815d180 2023-11-08T19:30:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.872+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb815de30 con 0x7f9eb815d180 2023-11-08T19:30:19.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.873+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9eb002cd50 con 0x7f9eb815d180 2023-11-08T19:30:19.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.874+0000 7f9eaeffd640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e9806ae70 0x7f9e9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:19.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.874+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f9eb00acb50 con 0x7f9eb815d180 2023-11-08T19:30:19.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.874+0000 7f9eb659c640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e9806ae70 0x7f9e9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:19.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.875+0000 7f9eb659c640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e9806ae70 0x7f9e9806d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800a970 tx=0x7f9ea8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:19.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:19.878+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9eb0077580 con 0x7f9eb815d180 2023-11-08T19:30:20.111 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.110+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb8074430 con 0x7f9eb815d180 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.110+0000 7f9eaeffd640 1 -- 172.21.15.119:0/1799532436 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f9eb007aba0 con 0x7f9eb815d180 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:20.112 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:18.127249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:20.113 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2023-11-08T19:30:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e9806ae70 msgr2=0x7f9e9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e9806ae70 0x7f9e9806d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800a970 tx=0x7f9ea8008040 comp rx=0 tx=0).stop 2023-11-08T19:30:20.115 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 msgr2=0x7f9eb8157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb8157610 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0036040 tx=0x7f9eb0004a40 comp rx=0 tx=0).stop 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 shutdown_connections 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f9e9806ae70 0x7f9e9806d330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9eb815d180 0x7f9eb8157610 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.116 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9eb80cc810 0x7f9eb80b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 --2- 172.21.15.119:0/1799532436 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9eb806a390 0x7f9eb81569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.113+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 >> 172.21.15.119:0/1799532436 conn(0x7f9eb80cb610 msgr2=0x7f9eb80b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.114+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 shutdown_connections 2023-11-08T19:30:20.117 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:20.114+0000 7f9ebcafc640 1 -- 172.21.15.119:0/1799532436 wait complete. 2023-11-08T19:30:20.123 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:20.132 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:20.135 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:25.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:25.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.357+0000 7fd2f2b75640 1 Processor -- start 2023-11-08T19:30:25.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.357+0000 7fd2f2b75640 1 -- start start 2023-11-08T19:30:25.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.358+0000 7fd2f2b75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 0x7fd2ec15a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.358+0000 7fd2f2b75640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec0ca2e0 con 0x7fd2ec0ad9d0 2023-11-08T19:30:25.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.359+0000 7fd2f2b75640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ec0ca5c0 con 0x7fd2ec0cac30 2023-11-08T19:30:25.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.359+0000 7fd2f2b75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec071c70 con 0x7fd2ec0adf20 2023-11-08T19:30:25.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.359+0000 7fd2f2374640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fd2ec0ad9d0 0x7fd2ec06b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:56964/0 (socket says 172.21.15.119:56964) 2023-11-08T19:30:25.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.359+0000 7fd2f2374640 1 -- 172.21.15.119:0/2158291663 learned_addr learned my addr 172.21.15.119:0/2158291663 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:25.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.359+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968190150 0 0) 0x7fd2ec0ca2e0 con 0x7fd2ec0ad9d0 2023-11-08T19:30:25.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.359+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2d8003520 con 0x7fd2ec0ad9d0 2023-11-08T19:30:25.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4065991231 0 0) 0x7fd2d8003520 con 0x7fd2ec0ad9d0 2023-11-08T19:30:25.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 msgr2=0x7fd2ec15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:25.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f1372640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 0x7fd2ec15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 0x7fd2ec15a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 >> v1:172.21.15.188:6789/0 conn(0x7fd2ec0cac30 legacy=0x7fd2ec15f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec15eb80 con 0x7fd2ec0ad9d0 2023-11-08T19:30:25.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fd2e40019f0 con 0x7fd2ec0ad9d0 2023-11-08T19:30:25.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 >> v1:172.21.15.119:6789/0 conn(0x7fd2ec0ad9d0 legacy=0x7fd2ec06b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d80039a0 0x7fd2d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 0x7fd2d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec071c70 con 0x7fd2d80047c0 2023-11-08T19:30:25.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d80054e0 con 0x7fd2d80039a0 2023-11-08T19:30:25.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d800bec0 con 0x7fd2ec0adf20 2023-11-08T19:30:25.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.360+0000 7fd2f1372640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f2374640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 0x7fd2d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f1b73640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d80039a0 0x7fd2d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f1372640 1 -- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 msgr2=0x7fd2d800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:25.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f1372640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0adf20 0x7fd2d800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f1372640 1 -- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d80039a0 msgr2=0x7fd2d8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:25.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f1372640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2d80039a0 0x7fd2d8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.361+0000 7fd2f1372640 1 -- 172.21.15.119:0/2158291663 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2d8003520 con 0x7fd2d80047c0 2023-11-08T19:30:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.362+0000 7fd2f1372640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2d8004c40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00a970 tx=0x7fd2d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e11c9ddab88dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.362+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc031680 con 0x7fd2d80047c0 2023-11-08T19:30:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.362+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2dc0317e0 con 0x7fd2d80047c0 2023-11-08T19:30:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.362+0000 7fd2f0b71640 1 -- 172.21.15.119:0/2158291663 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2dc031b90 con 0x7fd2d80047c0 2023-11-08T19:30:25.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2b75640 1 -- 172.21.15.119:0/2158291663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 msgr2=0x7fd2d8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/2158291663 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2d8004c40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc00a970 tx=0x7fd2d80057f0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2374640 1 -- 172.21.15.119:0/2158291663 reap_dead start 2023-11-08T19:30:25.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2b75640 1 -- 172.21.15.119:0/2158291663 shutdown_connections 2023-11-08T19:30:25.371 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2b75640 1 -- 172.21.15.119:0/2158291663 >> 172.21.15.119:0/2158291663 conn(0x7fd2ec0b0560 msgr2=0x7fd2ec0b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:25.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2b75640 1 -- 172.21.15.119:0/2158291663 shutdown_connections 2023-11-08T19:30:25.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.363+0000 7fd2f2b75640 1 -- 172.21.15.119:0/2158291663 wait complete. 2023-11-08T19:30:25.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.365+0000 7fd2f2b75640 1 Processor -- start 2023-11-08T19:30:25.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.365+0000 7fd2f2b75640 1 -- start start 2023-11-08T19:30:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f2b75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2ec0b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f2b75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0cd050 0x7fd2ec0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f2b75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2ec15a3f0 0x7fd2ec0ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f2b75640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec0ce580 con 0x7fd2d80047c0 2023-11-08T19:30:25.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f2b75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec0b4520 con 0x7fd2ec15a3f0 2023-11-08T19:30:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f2b75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec0c7db0 con 0x7fd2ec0cd050 2023-11-08T19:30:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2ec0b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2ec0b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60166/0 (socket says 172.21.15.119:60166) 2023-11-08T19:30:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 -- 172.21.15.119:0/3937399606 learned_addr learned my addr 172.21.15.119:0/3937399606 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:25.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1372640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0cd050 0x7fd2ec0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 -- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0cd050 msgr2=0x7fd2ec0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0cd050 0x7fd2ec0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 -- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2ec15a3f0 msgr2=0x7fd2ec0ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.366+0000 7fd2f1b73640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2ec15a3f0 0x7fd2ec0ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2f1b73640 1 -- 172.21.15.119:0/3937399606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec0cd580 con 0x7fd2d80047c0 2023-11-08T19:30:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2f1b73640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2ec0b3fe0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800e4d0 tx=0x7fd2e800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80053a0 con 0x7fd2d80047c0 2023-11-08T19:30:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e80035f0 con 0x7fd2d80047c0 2023-11-08T19:30:25.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec0c8070 con 0x7fd2d80047c0 2023-11-08T19:30:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80039a0 con 0x7fd2d80047c0 2023-11-08T19:30:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.367+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec1f2a40 con 0x7fd2d80047c0 2023-11-08T19:30:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.368+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec15a810 con 0x7fd2d80047c0 2023-11-08T19:30:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.369+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd2e8003b00 con 0x7fd2d80047c0 2023-11-08T19:30:25.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.370+0000 7fd2e27fc640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2d006ae70 0x7fd2d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.370+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7fd2e8089af0 con 0x7fd2d80047c0 2023-11-08T19:30:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.370+0000 7fd2f1372640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2d006ae70 0x7fd2d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:25.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.371+0000 7fd2f1372640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2d006ae70 0x7fd2d006d330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc036000 tx=0x7fd2dc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:25.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.375+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2e8054490 con 0x7fd2d80047c0 2023-11-08T19:30:25.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.602+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2ec1f2c00 con 0x7fd2d80047c0 2023-11-08T19:30:25.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.603+0000 7fd2e27fc640 1 -- 172.21.15.119:0/3937399606 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fd2e8003db0 con 0x7fd2d80047c0 2023-11-08T19:30:25.605 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:25.605 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:18.127249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:25.606 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2023-11-08T19:30:25.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2d006ae70 msgr2=0x7fd2d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2d006ae70 0x7fd2d006d330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc036000 tx=0x7fd2dc03c040 comp rx=0 tx=0).stop 2023-11-08T19:30:25.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 msgr2=0x7fd2ec0b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:25.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2ec0b3fe0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7fd2e800e4d0 tx=0x7fd2e800e990 comp rx=0 tx=0).stop 2023-11-08T19:30:25.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 shutdown_connections 2023-11-08T19:30:25.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd2d006ae70 0x7fd2d006d330 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd2ec15a3f0 0x7fd2ec0ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd2ec0cd050 0x7fd2ec0ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 --2- 172.21.15.119:0/3937399606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2d80047c0 0x7fd2ec0b3fe0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:25.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.606+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 >> 172.21.15.119:0/3937399606 conn(0x7fd2ec0b0560 msgr2=0x7fd2ec0afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:25.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.607+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 shutdown_connections 2023-11-08T19:30:25.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:25.607+0000 7fd2f2b75640 1 -- 172.21.15.119:0/3937399606 wait complete. 2023-11-08T19:30:25.630 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:25.634 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-08T19:30:30.632 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:30.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.855+0000 7f5e5095e640 1 Processor -- start 2023-11-08T19:30:30.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.856+0000 7f5e5095e640 1 -- start start 2023-11-08T19:30:30.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e5095e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e4c15d380 0x7f5e4c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e5095e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e5095e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0d1380 0x7f5e4c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e5095e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c0d1810 con 0x7f5e4c15d380 2023-11-08T19:30:30.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e5095e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c15eb30 con 0x7f5e4c0d1380 2023-11-08T19:30:30.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e5095e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c15e570 con 0x7f5e4c0cf6c0 2023-11-08T19:30:30.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.857+0000 7f5e4a59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e4c15d380 0x7f5e4c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f5e4c15d380 0x7f5e4c0cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60816/0 (socket says 172.21.15.119:60816) 2023-11-08T19:30:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 -- 172.21.15.119:0/3066699669 learned_addr learned my addr 172.21.15.119:0/3066699669 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4ad9d640 1 --2- 172.21.15.119:0/3066699669 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0d1380 0x7f5e4c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e41d9b640 1 --2- 172.21.15.119:0/3066699669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:30.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 -- 172.21.15.119:0/3066699669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0cf6c0 msgr2=0x7f5e4c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 --2- 172.21.15.119:0/3066699669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 -- 172.21.15.119:0/3066699669 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0d1380 msgr2=0x7f5e4c0b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 --2- 172.21.15.119:0/3066699669 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0d1380 0x7f5e4c0b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4a59c640 1 -- 172.21.15.119:0/3066699669 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c15e850 con 0x7f5e4c15d380 2023-11-08T19:30:30.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.858+0000 7f5e4ad9d640 1 --2- 172.21.15.119:0/3066699669 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0d1380 0x7f5e4c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.859+0000 7f5e4a59c640 1 --2- 172.21.15.119:0/3066699669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c0cf180 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0b0760 tx=0x7f5e3c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22bf3f4be8e4be5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.859+0000 7f5e49d9b640 1 -- 172.21.15.119:0/3066699669 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c02ca70 con 0x7f5e4c15d380 2023-11-08T19:30:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.859+0000 7f5e49d9b640 1 -- 172.21.15.119:0/3066699669 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c02cbd0 con 0x7f5e4c15d380 2023-11-08T19:30:30.864 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.859+0000 7f5e49d9b640 1 -- 172.21.15.119:0/3066699669 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c0343f0 con 0x7f5e4c15d380 2023-11-08T19:30:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 -- 172.21.15.119:0/3066699669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 msgr2=0x7f5e4c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 --2- 172.21.15.119:0/3066699669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c0cf180 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0b0760 tx=0x7f5e3c02bed0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 -- 172.21.15.119:0/3066699669 shutdown_connections 2023-11-08T19:30:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 --2- 172.21.15.119:0/3066699669 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0d1380 0x7f5e4c0b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 --2- 172.21.15.119:0/3066699669 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 --2- 172.21.15.119:0/3066699669 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c0cf180 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 -- 172.21.15.119:0/3066699669 >> 172.21.15.119:0/3066699669 conn(0x7f5e4c08afe0 msgr2=0x7f5e4c09a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:30.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 -- 172.21.15.119:0/3066699669 shutdown_connections 2023-11-08T19:30:30.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.860+0000 7f5e5095e640 1 -- 172.21.15.119:0/3066699669 wait complete. 2023-11-08T19:30:30.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.863+0000 7f5e5095e640 1 Processor -- start 2023-11-08T19:30:30.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.863+0000 7f5e5095e640 1 -- start start 2023-11-08T19:30:30.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e5095e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e5095e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0d1380 0x7f5e4c0c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e5095e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e5095e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c1f3dc0 con 0x7f5e4c15d380 2023-11-08T19:30:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e5095e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c49e0 con 0x7f5e4c0cf6c0 2023-11-08T19:30:30.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e5095e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c4cc0 con 0x7f5e4c0d1380 2023-11-08T19:30:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e4ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e4ad9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c06b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60826/0 (socket says 172.21.15.119:60826) 2023-11-08T19:30:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.864+0000 7f5e4ad9d640 1 -- 172.21.15.119:0/388500186 learned_addr learned my addr 172.21.15.119:0/388500186 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e4ad9d640 1 -- 172.21.15.119:0/388500186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0d1380 msgr2=0x7f5e4c0c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:30.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e4ad9d640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0d1380 0x7f5e4c0c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e4ad9d640 1 -- 172.21.15.119:0/388500186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c0cf6c0 msgr2=0x7f5e4c0c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e4ad9d640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e4ad9d640 1 -- 172.21.15.119:0/388500186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c009b30 con 0x7f5e4c15d380 2023-11-08T19:30:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e4ad9d640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c06b7d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400c7f0 tx=0x7f5e4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44007820 con 0x7f5e4c15d380 2023-11-08T19:30:30.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44007980 con 0x7f5e4c15d380 2023-11-08T19:30:30.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c0c4fa0 con 0x7f5e4c15d380 2023-11-08T19:30:30.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44007d70 con 0x7f5e4c15d380 2023-11-08T19:30:30.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.865+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c0c5560 con 0x7f5e4c15d380 2023-11-08T19:30:30.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.867+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c1f6360 con 0x7f5e4c15d380 2023-11-08T19:30:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.868+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e440113f0 con 0x7f5e4c15d380 2023-11-08T19:30:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.868+0000 7f5e437fe640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e2c06aea0 0x7f5e2c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:30.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.869+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f5e44089ff0 con 0x7f5e4c15d380 2023-11-08T19:30:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.869+0000 7f5e4a59c640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e2c06aea0 0x7f5e2c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.869+0000 7f5e4a59c640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e2c06aea0 0x7f5e2c06d360 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c035000 tx=0x7f5e3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:30.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:30.873+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e4c1f6360 con 0x7f5e4c15d380 2023-11-08T19:30:31.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.104+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e4c1f6360 con 0x7f5e4c15d380 2023-11-08T19:30:31.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.105+0000 7f5e437fe640 1 -- 172.21.15.119:0/388500186 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f5e4c1f6360 con 0x7f5e4c15d380 2023-11-08T19:30:31.107 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:31.107 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":74,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:18.127249+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:31.108 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 74 2023-11-08T19:30:31.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e2c06aea0 msgr2=0x7f5e2c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:31.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e2c06aea0 0x7f5e2c06d360 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c035000 tx=0x7f5e3c02e000 comp rx=0 tx=0).stop 2023-11-08T19:30:31.112 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 msgr2=0x7f5e4c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:31.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c06b7d0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f5e4400c7f0 tx=0x7f5e4400ccb0 comp rx=0 tx=0).stop 2023-11-08T19:30:31.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 shutdown_connections 2023-11-08T19:30:31.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f5e2c06aea0 0x7f5e2c06d360 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:31.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f5e4c15d380 0x7f5e4c06b7d0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:31.113 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0d1380 0x7f5e4c0c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:31.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 --2- 172.21.15.119:0/388500186 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c0cf6c0 0x7f5e4c0c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:31.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 >> 172.21.15.119:0/388500186 conn(0x7f5e4c08afe0 msgr2=0x7f5e4c0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:31.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 shutdown_connections 2023-11-08T19:30:31.114 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:31.110+0000 7f5e5095e640 1 -- 172.21.15.119:0/388500186 wait complete. 2023-11-08T19:30:31.130 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:32.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-08T19:30:36.132 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:36.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.351+0000 7f3412287640 1 Processor -- start 2023-11-08T19:30:36.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.352+0000 7f3412287640 1 -- start start 2023-11-08T19:30:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f3412287640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f340c1447a0 0x7f340c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f3412287640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f340c0cad30 0x7f340c06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f3412287640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06a120 0x7f340c06a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f3412287640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c06aa90 con 0x7f340c0cad30 2023-11-08T19:30:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f3412287640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f340c0c3a50 con 0x7f340c1447a0 2023-11-08T19:30:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f3412287640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c144b80 con 0x7f340c06a120 2023-11-08T19:30:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.353+0000 7f34037fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f340c0cad30 0x7f340c06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:36.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f340c0cad30 0x7f340c06f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60836/0 (socket says 172.21.15.119:60836) 2023-11-08T19:30:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 -- 172.21.15.119:0/2968647937 learned_addr learned my addr 172.21.15.119:0/2968647937 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 -- 172.21.15.119:0/2968647937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06a120 msgr2=0x7f340c06a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 --2- 172.21.15.119:0/2968647937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06a120 0x7f340c06a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 -- 172.21.15.119:0/2968647937 >> v2:172.21.15.188:3300/0 conn(0x7f340c1447a0 msgr2=0x7f340c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:36.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 --2- 172.21.15.119:0/2968647937 >> v2:172.21.15.188:3300/0 conn(0x7f340c1447a0 0x7f340c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.354+0000 7f34037fe640 1 -- 172.21.15.119:0/2968647937 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0c9150 con 0x7f340c0cad30 2023-11-08T19:30:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.355+0000 7f34037fe640 1 --2- 172.21.15.119:0/2968647937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c06f930 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f340c0cfbd0 tx=0x7f33f402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e23c94cc69406a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.355+0000 7f340b7fe640 1 -- 172.21.15.119:0/2968647937 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4031870 con 0x7f340c0cad30 2023-11-08T19:30:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.355+0000 7f340b7fe640 1 -- 172.21.15.119:0/2968647937 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f40319d0 con 0x7f340c0cad30 2023-11-08T19:30:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.355+0000 7f340b7fe640 1 -- 172.21.15.119:0/2968647937 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4031d60 con 0x7f340c0cad30 2023-11-08T19:30:36.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 -- 172.21.15.119:0/2968647937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 msgr2=0x7f340c06f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 --2- 172.21.15.119:0/2968647937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c06f930 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f340c0cfbd0 tx=0x7f33f402b910 comp rx=0 tx=0).stop 2023-11-08T19:30:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 -- 172.21.15.119:0/2968647937 shutdown_connections 2023-11-08T19:30:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 --2- 172.21.15.119:0/2968647937 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c06a120 0x7f340c06a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 --2- 172.21.15.119:0/2968647937 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c06f930 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 --2- 172.21.15.119:0/2968647937 >> v2:172.21.15.188:3300/0 conn(0x7f340c1447a0 0x7f340c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 -- 172.21.15.119:0/2968647937 >> 172.21.15.119:0/2968647937 conn(0x7f340c0cb610 msgr2=0x7f340c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 -- 172.21.15.119:0/2968647937 shutdown_connections 2023-11-08T19:30:36.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.356+0000 7f3412287640 1 -- 172.21.15.119:0/2968647937 wait complete. 2023-11-08T19:30:36.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.358+0000 7f3412287640 1 Processor -- start 2023-11-08T19:30:36.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.359+0000 7f3412287640 1 -- start start 2023-11-08T19:30:36.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f3412287640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c06a120 0x7f340c0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f3412287640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c1425b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f3412287640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c1447a0 0x7f340c142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f3412287640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c7a70 con 0x7f340c0cad30 2023-11-08T19:30:36.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f3412287640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c6df0 con 0x7f340c06a120 2023-11-08T19:30:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f3412287640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0c70d0 con 0x7f340c1447a0 2023-11-08T19:30:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f34037fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c1425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f34037fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c1425b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60850/0 (socket says 172.21.15.119:60850) 2023-11-08T19:30:36.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.360+0000 7f34037fe640 1 -- 172.21.15.119:0/1784173552 learned_addr learned my addr 172.21.15.119:0/1784173552 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34037fe640 1 -- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c1447a0 msgr2=0x7f340c142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f340bfff640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c06a120 0x7f340c0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34037fe640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c1447a0 0x7f340c142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34037fe640 1 -- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c06a120 msgr2=0x7f340c0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:36.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34037fe640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c06a120 0x7f340c0c86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34037fe640 1 -- 172.21.15.119:0/1784173552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f40094d0 con 0x7f340c0cad30 2023-11-08T19:30:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34037fe640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c1425b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000c00 tx=0x7f33f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f403ad10 con 0x7f340c0cad30 2023-11-08T19:30:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f403ae70 con 0x7f340c0cad30 2023-11-08T19:30:36.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f4044610 con 0x7f340c0cad30 2023-11-08T19:30:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.361+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0c73b0 con 0x7f340c0cad30 2023-11-08T19:30:36.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.362+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c13faf0 con 0x7f340c0cad30 2023-11-08T19:30:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.363+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c074430 con 0x7f340c0cad30 2023-11-08T19:30:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.364+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33f404d450 con 0x7f340c0cad30 2023-11-08T19:30:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.365+0000 7f34097fa640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33f006aec0 0x7f33f006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:36.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.365+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f33f402d040 con 0x7f340c0cad30 2023-11-08T19:30:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.365+0000 7f340bfff640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33f006aec0 0x7f33f006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.366+0000 7f340bfff640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33f006aec0 0x7f33f006d380 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00a770 tx=0x7f33fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:36.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.369+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33f4076df0 con 0x7f340c0cad30 2023-11-08T19:30:36.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.598+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f340c070670 con 0x7f340c0cad30 2023-11-08T19:30:36.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.599+0000 7f34097fa640 1 -- 172.21.15.119:0/1784173552 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4988 (secure 0 0 0) 0x7f33f407a410 con 0x7f340c0cad30 2023-11-08T19:30:36.601 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:36.601 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":75,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7515,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:33.709350+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":99,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5144,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5144":{"gid":5144,"name":"b","rank":0,"incarnation":69,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/310147451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":310147451},{"type":"v1","addr":"172.21.15.188:6835","nonce":310147451}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:30:33.709350+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:36.602 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 75 2023-11-08T19:30:36.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33f006aec0 msgr2=0x7f33f006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:36.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33f006aec0 0x7f33f006d380 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00a770 tx=0x7f33fc008040 comp rx=0 tx=0).stop 2023-11-08T19:30:36.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 msgr2=0x7f340c1425b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:36.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c1425b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f33f4000c00 tx=0x7f33f40046c0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 shutdown_connections 2023-11-08T19:30:36.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33f006aec0 0x7f33f006d380 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f340c1447a0 0x7f340c142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f340c0cad30 0x7f340c1425b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 --2- 172.21.15.119:0/1784173552 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f340c06a120 0x7f340c0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:36.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 >> 172.21.15.119:0/1784173552 conn(0x7f340c0cb610 msgr2=0x7f340c143880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:36.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 shutdown_connections 2023-11-08T19:30:36.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:36.602+0000 7f3412287640 1 -- 172.21.15.119:0/1784173552 wait complete. 2023-11-08T19:30:36.621 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:30:36.622 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:30:36.720 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:30:36.724 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:30:36.724 DEBUG:teuthology.orchestra.run.smithi188:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:30:36.823 INFO:teuthology.orchestra.run.smithi188.stdout:1699471819.95258.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:30:36.827 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471819.95258.core'] 2023-11-08T19:30:36.828 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471819.95258.core, deleting it 2023-11-08T19:30:36.828 DEBUG:teuthology.orchestra.run.smithi188:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471819.95258.core 2023-11-08T19:30:36.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:30:37.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.155+0000 7f28f55a5640 1 Processor -- start 2023-11-08T19:30:37.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.156+0000 7f28f55a5640 1 -- start start 2023-11-08T19:30:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.157+0000 7f28f55a5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f28f015b080 0x7f28f00ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.157+0000 7f28f55a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.157+0000 7f28f55a5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00cda70 con 0x7f28f0161250 2023-11-08T19:30:37.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.157+0000 7f28f55a5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f0161dd0 con 0x7f28f015b080 2023-11-08T19:30:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.157+0000 7f28f55a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00d02b0 con 0x7f28f00cf8c0 2023-11-08T19:30:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.157+0000 7f28eeffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f28f015b080 0x7f28f00ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.158+0000 7f28eeffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f28f015b080 0x7f28f00ce7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60854/0 (socket says 172.21.15.119:60854) 2023-11-08T19:30:37.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.158+0000 7f28eeffd640 1 -- 172.21.15.119:0/3985028523 learned_addr learned my addr 172.21.15.119:0/3985028523 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:37.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.158+0000 7f28eeffd640 1 -- 172.21.15.119:0/3985028523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 msgr2=0x7f28f00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:37.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.158+0000 7f28eeffd640 1 --2- 172.21.15.119:0/3985028523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.158+0000 7f28eeffd640 1 -- 172.21.15.119:0/3985028523 >> v1:172.21.15.188:6789/0 conn(0x7f28f0161250 legacy=0x7f28f01616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:37.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.158+0000 7f28eeffd640 1 -- 172.21.15.119:0/3985028523 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f0161fc0 con 0x7f28f015b080 2023-11-08T19:30:37.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.159+0000 7f28eeffd640 1 --2- 172.21.15.119:0/3985028523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f015b080 0x7f28f00ce7a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009e50 tx=0x7f28e002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f2324445bf14bc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:37.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.159+0000 7f28edffb640 1 -- 172.21.15.119:0/3985028523 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e002c9f0 con 0x7f28f015b080 2023-11-08T19:30:37.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.159+0000 7f28edffb640 1 -- 172.21.15.119:0/3985028523 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e002cb50 con 0x7f28f015b080 2023-11-08T19:30:37.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.159+0000 7f28edffb640 1 -- 172.21.15.119:0/3985028523 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00343f0 con 0x7f28f015b080 2023-11-08T19:30:37.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 -- 172.21.15.119:0/3985028523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f015b080 msgr2=0x7f28f00ce7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3985028523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f015b080 0x7f28f00ce7a0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009e50 tx=0x7f28e002be50 comp rx=0 tx=0).stop 2023-11-08T19:30:37.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 -- 172.21.15.119:0/3985028523 shutdown_connections 2023-11-08T19:30:37.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3985028523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3985028523 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f015b080 0x7f28f00ce7a0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 -- 172.21.15.119:0/3985028523 >> 172.21.15.119:0/3985028523 conn(0x7f28f006a9f0 msgr2=0x7f28f006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:37.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 -- 172.21.15.119:0/3985028523 shutdown_connections 2023-11-08T19:30:37.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.160+0000 7f28f55a5640 1 -- 172.21.15.119:0/3985028523 wait complete. 2023-11-08T19:30:37.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.162+0000 7f28f55a5640 1 Processor -- start 2023-11-08T19:30:37.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.162+0000 7f28f55a5640 1 -- start start 2023-11-08T19:30:37.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28f55a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28f55a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f015b080 0x7f28f00cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28f55a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 0x7f28f0158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28f55a5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00cb090 con 0x7f28f0161250 2023-11-08T19:30:37.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28f55a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00cc120 con 0x7f28f015b080 2023-11-08T19:30:37.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28f55a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00c6080 con 0x7f28f00cf8c0 2023-11-08T19:30:37.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28ef7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 0x7f28f0158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28ef7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 0x7f28f0158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60858/0 (socket says 172.21.15.119:60858) 2023-11-08T19:30:37.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.163+0000 7f28ef7fe640 1 -- 172.21.15.119:0/3267952908 learned_addr learned my addr 172.21.15.119:0/3267952908 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:37.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ee7fc640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f015b080 0x7f28f00cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28eeffd640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ef7fe640 1 -- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 msgr2=0x7f28f00c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ef7fe640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ef7fe640 1 -- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f015b080 msgr2=0x7f28f00cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ef7fe640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f015b080 0x7f28f00cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ef7fe640 1 -- 172.21.15.119:0/3267952908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009b30 con 0x7f28f0161250 2023-11-08T19:30:37.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28ef7fe640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 0x7f28f0158dd0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b490 tx=0x7f28e800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:37.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8011830 con 0x7f28f0161250 2023-11-08T19:30:37.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80119b0 con 0x7f28f0161250 2023-11-08T19:30:37.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8011d60 con 0x7f28f0161250 2023-11-08T19:30:37.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.164+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f00cadb0 con 0x7f28f0161250 2023-11-08T19:30:37.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.165+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f00cbdb0 con 0x7f28f0161250 2023-11-08T19:30:37.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.166+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f00cb930 con 0x7f28f0161250 2023-11-08T19:30:37.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.167+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28e801a400 con 0x7f28f0161250 2023-11-08T19:30:37.172 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.168+0000 7f28e77fe640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f28d006ae70 0x7f28d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.168+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 7350+0+0 (secure 0 0 0) 0x7f28e8089da0 con 0x7f28f0161250 2023-11-08T19:30:37.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.168+0000 7f28eeffd640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f28d006ae70 0x7f28d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.169+0000 7f28eeffd640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f28d006ae70 0x7f28d006d330 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f28e0035000 tx=0x7f28e0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:37.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.172+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28f00cb930 con 0x7f28f0161250 2023-11-08T19:30:37.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.433+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f28f00cb930 con 0x7f28f0161250 2023-11-08T19:30:37.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.438+0000 7f28e77fe640 1 -- 172.21.15.119:0/3267952908 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5144 v76) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28f00cb930 con 0x7f28f0161250 2023-11-08T19:30:37.439 INFO:teuthology.orchestra.run.smithi119.stderr:failed mds gid 5144 2023-11-08T19:30:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.440+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f28d006ae70 msgr2=0x7f28d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.440+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f28d006ae70 0x7f28d006d330 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f28e0035000 tx=0x7f28e0030000 comp rx=0 tx=0).stop 2023-11-08T19:30:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.440+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 msgr2=0x7f28f0158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.440+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 0x7f28f0158dd0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b490 tx=0x7f28e800b950 comp rx=0 tx=0).stop 2023-11-08T19:30:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 shutdown_connections 2023-11-08T19:30:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f28d006ae70 0x7f28d006d330 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28f0161250 0x7f28f0158dd0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f28f015b080 0x7f28f00cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 --2- 172.21.15.119:0/3267952908 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28f00cf8c0 0x7f28f00c5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 >> 172.21.15.119:0/3267952908 conn(0x7f28f006a9f0 msgr2=0x7f28f015b460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 shutdown_connections 2023-11-08T19:30:37.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.441+0000 7f28f55a5640 1 -- 172.21.15.119:0/3267952908 wait complete. 2023-11-08T19:30:37.462 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:30:37.463 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:30:37.463 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:30:37.464 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:30:37.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:30:37.465 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi188 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-08T19:30:37.465 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:30:37.466 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:30:37.477 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:30:37.478 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:37.650 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:30:37.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.660+0000 7fbc3e6fc640 1 Processor -- start 2023-11-08T19:30:37.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.660+0000 7fbc3e6fc640 1 -- start start 2023-11-08T19:30:37.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.661+0000 7fbc3e6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.661+0000 7fbc3e6fc640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806cff0 con 0x7fbc3815b900 2023-11-08T19:30:37.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.661+0000 7fbc3e6fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc3806d2d0 con 0x7fbc380c98c0 2023-11-08T19:30:37.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.661+0000 7fbc3e6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3806bd20 con 0x7fbc3815b0d0 2023-11-08T19:30:37.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc3cef9640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbc3815b900 0x7fbc38070b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:49058/0 (socket says 172.21.15.119:49058) 2023-11-08T19:30:37.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc3cef9640 1 -- 172.21.15.119:0/2270168421 learned_addr learned my addr 172.21.15.119:0/2270168421 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:37.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc3d6fa640 1 --2- 172.21.15.119:0/2270168421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1058831975 0 0) 0x7fbc3806cff0 con 0x7fbc3815b900 2023-11-08T19:30:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc24003520 con 0x7fbc3815b900 2023-11-08T19:30:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655991293 0 0) 0x7fbc3806d2d0 con 0x7fbc380c98c0 2023-11-08T19:30:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc3806cff0 con 0x7fbc380c98c0 2023-11-08T19:30:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/2270168421 >> v1:172.21.15.188:6789/0 conn(0x7fbc380c98c0 legacy=0x7fbc380c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/2270168421 >> v1:172.21.15.119:6789/0 conn(0x7fbc3815b900 legacy=0x7fbc38070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.662+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/2270168421 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc3806d760 con 0x7fbc3815b0d0 2023-11-08T19:30:37.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.663+0000 7fbc3d6fa640 1 --2- 172.21.15.119:0/2270168421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380ad7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009dc0 tx=0x7fbc3402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccc0bae7cf5e2194 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:37.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.664+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc3402cc40 con 0x7fbc3815b0d0 2023-11-08T19:30:37.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.664+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc3402cda0 con 0x7fbc3815b0d0 2023-11-08T19:30:37.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.664+0000 7fbc2bfff640 1 -- 172.21.15.119:0/2270168421 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34034660 con 0x7fbc3815b0d0 2023-11-08T19:30:37.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/2270168421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 msgr2=0x7fbc380ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/2270168421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380ad7d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbc34009dc0 tx=0x7fbc3402be90 comp rx=0 tx=0).stop 2023-11-08T19:30:37.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/2270168421 shutdown_connections 2023-11-08T19:30:37.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/2270168421 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380ad7d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/2270168421 >> 172.21.15.119:0/2270168421 conn(0x7fbc380b0560 msgr2=0x7fbc380b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:37.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/2270168421 shutdown_connections 2023-11-08T19:30:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.665+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/2270168421 wait complete. 2023-11-08T19:30:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.666+0000 7fbc3e6fc640 1 Processor -- start 2023-11-08T19:30:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.667+0000 7fbc3e6fc640 1 -- start start 2023-11-08T19:30:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3e6fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 0x7fbc381571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3e6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3e6fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc3815b900 0x7fbc380ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3e6fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380ae2d0 con 0x7fbc380c98c0 2023-11-08T19:30:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3e6fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc381f3890 con 0x7fbc3815b900 2023-11-08T19:30:37.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3e6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc380ae4e0 con 0x7fbc3815b0d0 2023-11-08T19:30:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 0x7fbc381571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 0x7fbc381571b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:60870/0 (socket says 172.21.15.119:60870) 2023-11-08T19:30:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/1450591264 learned_addr learned my addr 172.21.15.119:0/1450591264 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3cef9640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 msgr2=0x7fbc380aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3defb640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc3815b900 0x7fbc380ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc3815b900 msgr2=0x7fbc380ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc3815b900 0x7fbc380ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3d6fa640 1 -- 172.21.15.119:0/1450591264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc34009a70 con 0x7fbc380c98c0 2023-11-08T19:30:37.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3cef9640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:37.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.668+0000 7fbc3defb640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc3815b900 0x7fbc380ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:37.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.669+0000 7fbc3d6fa640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 0x7fbc381571b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fbc34000f80 tx=0x7fbc34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:37.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.669+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34040650 con 0x7fbc380c98c0 2023-11-08T19:30:37.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.669+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc340407b0 con 0x7fbc380c98c0 2023-11-08T19:30:37.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.669+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc34040b60 con 0x7fbc380c98c0 2023-11-08T19:30:37.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.669+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc380ae7c0 con 0x7fbc380c98c0 2023-11-08T19:30:37.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.669+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc380ace10 con 0x7fbc380c98c0 2023-11-08T19:30:37.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.670+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc3804ece0 con 0x7fbc380c98c0 2023-11-08T19:30:37.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.671+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc34040cc0 con 0x7fbc380c98c0 2023-11-08T19:30:37.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.672+0000 7fbc29ffb640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc1c06ae70 0x7fbc1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:37.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.672+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fbc340aca20 con 0x7fbc380c98c0 2023-11-08T19:30:37.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.672+0000 7fbc3cef9640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc1c06ae70 0x7fbc1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:37.679 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.673+0000 7fbc3cef9640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc1c06ae70 0x7fbc1c06d330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbc3806db00 tx=0x7fbc2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:37.680 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.676+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbc34077370 con 0x7fbc380c98c0 2023-11-08T19:30:37.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.903+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc380c7890 con 0x7fbc380c98c0 2023-11-08T19:30:37.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.904+0000 7fbc29ffb640 1 -- 172.21.15.119:0/1450591264 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4149 (secure 0 0 0) 0x7fbc34009450 con 0x7fbc380c98c0 2023-11-08T19:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:37.906 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":77,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:37.444084+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:37.907 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 77 2023-11-08T19:30:37.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc1c06ae70 msgr2=0x7fbc1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc1c06ae70 0x7fbc1c06d330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fbc3806db00 tx=0x7fbc2c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:37.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 msgr2=0x7fbc381571b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:37.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 0x7fbc381571b0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fbc34000f80 tx=0x7fbc34004a40 comp rx=0 tx=0).stop 2023-11-08T19:30:37.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 shutdown_connections 2023-11-08T19:30:37.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbc1c06ae70 0x7fbc1c06d330 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbc3815b900 0x7fbc380ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbc3815b0d0 0x7fbc380aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 --2- 172.21.15.119:0/1450591264 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbc380c98c0 0x7fbc381571b0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:37.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 >> 172.21.15.119:0/1450591264 conn(0x7fbc380b0560 msgr2=0x7fbc3806aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:37.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 shutdown_connections 2023-11-08T19:30:37.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:37.907+0000 7fbc3e6fc640 1 -- 172.21.15.119:0/1450591264 wait complete. 2023-11-08T19:30:37.927 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:30:38.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-08T19:30:42.929 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:43.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.149+0000 7f8a37fff640 1 Processor -- start 2023-11-08T19:30:43.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.150+0000 7f8a37fff640 1 -- start start 2023-11-08T19:30:43.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.151+0000 7f8a37fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a3815b0d0 0x7f8a380ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a380c98c0 0x7f8a380c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a37fff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3806cff0 con 0x7f8a3815b900 2023-11-08T19:30:43.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a37fff640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3806d2d0 con 0x7f8a3815b0d0 2023-11-08T19:30:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a37fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3806bd20 con 0x7f8a380c98c0 2023-11-08T19:30:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a3815b0d0 0x7f8a380ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8a3815b0d0 0x7f8a380ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45656/0 (socket says 172.21.15.119:45656) 2023-11-08T19:30:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 -- 172.21.15.119:0/1212916019 learned_addr learned my addr 172.21.15.119:0/1212916019 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:43.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 -- 172.21.15.119:0/1212916019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a380c98c0 msgr2=0x7f8a380c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:43.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 --2- 172.21.15.119:0/1212916019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a380c98c0 0x7f8a380c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 -- 172.21.15.119:0/1212916019 >> v1:172.21.15.188:6789/0 conn(0x7f8a3815b900 legacy=0x7f8a38070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:43.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.152+0000 7f8a36ffd640 1 -- 172.21.15.119:0/1212916019 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3806d760 con 0x7f8a3815b0d0 2023-11-08T19:30:43.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.153+0000 7f8a36ffd640 1 --2- 172.21.15.119:0/1212916019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3815b0d0 0x7f8a380ad7d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800bb10 tx=0x7f8a2802e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f194a0956326b5a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:43.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a367fc640 1 -- 172.21.15.119:0/1212916019 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a2802ede0 con 0x7f8a3815b0d0 2023-11-08T19:30:43.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a367fc640 1 -- 172.21.15.119:0/1212916019 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a28036440 con 0x7f8a3815b0d0 2023-11-08T19:30:43.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a367fc640 1 -- 172.21.15.119:0/1212916019 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a280367d0 con 0x7f8a3815b0d0 2023-11-08T19:30:43.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 -- 172.21.15.119:0/1212916019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3815b0d0 msgr2=0x7f8a380ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 --2- 172.21.15.119:0/1212916019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3815b0d0 0x7f8a380ad7d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800bb10 tx=0x7f8a2802e030 comp rx=0 tx=0).stop 2023-11-08T19:30:43.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 -- 172.21.15.119:0/1212916019 shutdown_connections 2023-11-08T19:30:43.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 --2- 172.21.15.119:0/1212916019 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a380c98c0 0x7f8a380c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 --2- 172.21.15.119:0/1212916019 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a3815b0d0 0x7f8a380ad7d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 -- 172.21.15.119:0/1212916019 >> 172.21.15.119:0/1212916019 conn(0x7f8a380b0560 msgr2=0x7f8a380b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:43.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.154+0000 7f8a37fff640 1 -- 172.21.15.119:0/1212916019 shutdown_connections 2023-11-08T19:30:43.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.155+0000 7f8a37fff640 1 -- 172.21.15.119:0/1212916019 wait complete. 2023-11-08T19:30:43.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.156+0000 7f8a37fff640 1 Processor -- start 2023-11-08T19:30:43.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.156+0000 7f8a37fff640 1 -- start start 2023-11-08T19:30:43.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.157+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 0x7f8a3814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.157+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3815b0d0 0x7f8a380ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.157+0000 7f8a37fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3815b900 0x7f8a380ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.157+0000 7f8a37fff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3815e060 con 0x7f8a380c98c0 2023-11-08T19:30:43.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.157+0000 7f8a37fff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a38151ff0 con 0x7f8a3815b900 2023-11-08T19:30:43.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.157+0000 7f8a37fff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a381522d0 con 0x7f8a3815b0d0 2023-11-08T19:30:43.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 0x7f8a3814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 0x7f8a3814dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45672/0 (socket says 172.21.15.119:45672) 2023-11-08T19:30:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 -- 172.21.15.119:0/3813842535 learned_addr learned my addr 172.21.15.119:0/3813842535 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 -- 172.21.15.119:0/3813842535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3815b0d0 msgr2=0x7f8a380ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3815b0d0 0x7f8a380ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 -- 172.21.15.119:0/3813842535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3815b900 msgr2=0x7f8a380ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3815b900 0x7f8a380ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 -- 172.21.15.119:0/3813842535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a2800b770 con 0x7f8a380c98c0 2023-11-08T19:30:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.158+0000 7f8a36ffd640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 0x7f8a3814dcd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800b2c0 tx=0x7f8a28004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:43.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.159+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28041680 con 0x7f8a380c98c0 2023-11-08T19:30:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.159+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a280417e0 con 0x7f8a380c98c0 2023-11-08T19:30:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.159+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a381525b0 con 0x7f8a380c98c0 2023-11-08T19:30:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.159+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28041b90 con 0x7f8a380c98c0 2023-11-08T19:30:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.159+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3814e210 con 0x7f8a380c98c0 2023-11-08T19:30:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.160+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a38152ad0 con 0x7f8a380c98c0 2023-11-08T19:30:43.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.161+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8a28041cf0 con 0x7f8a380c98c0 2023-11-08T19:30:43.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.162+0000 7f8a2ffff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a1806ae70 0x7f8a1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.162+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f8a280acc30 con 0x7f8a380c98c0 2023-11-08T19:30:43.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.162+0000 7f8a2e7fc640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a1806ae70 0x7f8a1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.163+0000 7f8a2e7fc640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a1806ae70 0x7f8a1806d330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a380c89c0 tx=0x7f8a20006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:43.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.166+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8a38152ad0 con 0x7f8a380c98c0 2023-11-08T19:30:43.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.391+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a38152ad0 con 0x7f8a380c98c0 2023-11-08T19:30:43.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.392+0000 7f8a2ffff640 1 -- 172.21.15.119:0/3813842535 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f8a38152ad0 con 0x7f8a380c98c0 2023-11-08T19:30:43.395 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:43.395 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:40.469042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:43.396 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 81 2023-11-08T19:30:43.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a1806ae70 msgr2=0x7f8a1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a1806ae70 0x7f8a1806d330 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a380c89c0 tx=0x7f8a20006ba0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 msgr2=0x7f8a3814dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 0x7f8a3814dcd0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800b2c0 tx=0x7f8a28004730 comp rx=0 tx=0).stop 2023-11-08T19:30:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 shutdown_connections 2023-11-08T19:30:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8a1806ae70 0x7f8a1806d330 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8a3815b900 0x7f8a380ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8a3815b0d0 0x7f8a380ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 --2- 172.21.15.119:0/3813842535 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8a380c98c0 0x7f8a3814dcd0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 >> 172.21.15.119:0/3813842535 conn(0x7f8a380b0560 msgr2=0x7f8a38157d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:43.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 shutdown_connections 2023-11-08T19:30:43.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.395+0000 7f8a37fff640 1 -- 172.21.15.119:0/3813842535 wait complete. 2023-11-08T19:30:43.416 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:30:43.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:43.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.608+0000 7fd506bb0640 1 Processor -- start 2023-11-08T19:30:43.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.608+0000 7fd506bb0640 1 -- start start 2023-11-08T19:30:43.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.610+0000 7fd506bb0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50006a970 0x7fd50006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.610+0000 7fd506bb0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5000ccde0 0x7fd5001653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.610+0000 7fd506bb0640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd50006d4b0 con 0x7fd5000c9150 2023-11-08T19:30:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.610+0000 7fd506bb0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5001619b0 con 0x7fd5000ccde0 2023-11-08T19:30:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.610+0000 7fd506bb0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd500147b80 con 0x7fd50006a970 2023-11-08T19:30:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.610+0000 7fd50514d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5000ccde0 0x7fd5001653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50514d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd5000ccde0 0x7fd5001653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45686/0 (socket says 172.21.15.119:45686) 2023-11-08T19:30:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50514d640 1 -- 172.21.15.119:0/595839039 learned_addr learned my addr 172.21.15.119:0/595839039 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50494c640 1 --2- 172.21.15.119:0/595839039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50006a970 0x7fd50006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50514d640 1 -- 172.21.15.119:0/595839039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50006a970 msgr2=0x7fd50006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50514d640 1 --2- 172.21.15.119:0/595839039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50006a970 0x7fd50006cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50514d640 1 -- 172.21.15.119:0/595839039 >> v1:172.21.15.188:6789/0 conn(0x7fd5000c9150 legacy=0x7fd5000cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:43.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.611+0000 7fd50514d640 1 -- 172.21.15.119:0/595839039 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500147000 con 0x7fd5000ccde0 2023-11-08T19:30:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.612+0000 7fd50514d640 1 --2- 172.21.15.119:0/595839039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000ccde0 0x7fd5001653f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009e50 tx=0x7fd4f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0210cbe7340d02 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.612+0000 7fd4ff7fe640 1 -- 172.21.15.119:0/595839039 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f802c9e0 con 0x7fd5000ccde0 2023-11-08T19:30:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd4ff7fe640 1 -- 172.21.15.119:0/595839039 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f802cb40 con 0x7fd5000ccde0 2023-11-08T19:30:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd4ff7fe640 1 -- 172.21.15.119:0/595839039 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f80343f0 con 0x7fd5000ccde0 2023-11-08T19:30:43.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 -- 172.21.15.119:0/595839039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000ccde0 msgr2=0x7fd5001653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 --2- 172.21.15.119:0/595839039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000ccde0 0x7fd5001653f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009e50 tx=0x7fd4f802be40 comp rx=0 tx=0).stop 2023-11-08T19:30:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 -- 172.21.15.119:0/595839039 shutdown_connections 2023-11-08T19:30:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 --2- 172.21.15.119:0/595839039 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd5000ccde0 0x7fd5001653f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 --2- 172.21.15.119:0/595839039 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd50006a970 0x7fd50006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 -- 172.21.15.119:0/595839039 >> 172.21.15.119:0/595839039 conn(0x7fd5000cb610 msgr2=0x7fd5000cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 -- 172.21.15.119:0/595839039 shutdown_connections 2023-11-08T19:30:43.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.613+0000 7fd506bb0640 1 -- 172.21.15.119:0/595839039 wait complete. 2023-11-08T19:30:43.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.615+0000 7fd506bb0640 1 Processor -- start 2023-11-08T19:30:43.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.616+0000 7fd506bb0640 1 -- start start 2023-11-08T19:30:43.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd506bb0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 0x7fd5000b4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd506bb0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5000c9150 0x7fd5000b0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd506bb0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5000ccde0 0x7fd500160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd506bb0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b3cb0 con 0x7fd50006a970 2023-11-08T19:30:43.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd506bb0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b16b0 con 0x7fd5000c9150 2023-11-08T19:30:43.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd506bb0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5000b06a0 con 0x7fd5000ccde0 2023-11-08T19:30:43.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd50494c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 0x7fd5000b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd50494c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 0x7fd5000b4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45690/0 (socket says 172.21.15.119:45690) 2023-11-08T19:30:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd50494c640 1 -- 172.21.15.119:0/1135510594 learned_addr learned my addr 172.21.15.119:0/1135510594 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd50494c640 1 -- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5000ccde0 msgr2=0x7fd500160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd50494c640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5000ccde0 0x7fd500160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd50494c640 1 -- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5000c9150 msgr2=0x7fd5000b0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.617+0000 7fd4fffff640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5000c9150 0x7fd5000b0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd50494c640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5000c9150 0x7fd5000b0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd50494c640 1 -- 172.21.15.119:0/1135510594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8009b30 con 0x7fd50006a970 2023-11-08T19:30:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd4fffff640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5000c9150 0x7fd5000b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:43.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd50494c640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 0x7fd5000b4140 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000d620 tx=0x7fd4f000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000b620 con 0x7fd50006a970 2023-11-08T19:30:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4f000b780 con 0x7fd50006a970 2023-11-08T19:30:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4f000bb30 con 0x7fd50006a970 2023-11-08T19:30:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5000b13d0 con 0x7fd50006a970 2023-11-08T19:30:43.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.618+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5000b3940 con 0x7fd50006a970 2023-11-08T19:30:43.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.619+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5000b3b00 con 0x7fd50006a970 2023-11-08T19:30:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.620+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4f000bc90 con 0x7fd50006a970 2023-11-08T19:30:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.621+0000 7fd4fd7fa640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4e006aea0 0x7fd4e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.621+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fd4f008a1d0 con 0x7fd50006a970 2023-11-08T19:30:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.622+0000 7fd4fffff640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4e006aea0 0x7fd4e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:43.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.622+0000 7fd4fffff640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4e006aea0 0x7fd4e006d360 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd500143e40 tx=0x7fd4e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:43.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.625+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4f0054b20 con 0x7fd50006a970 2023-11-08T19:30:43.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.859+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd500147870 con 0x7fd50006a970 2023-11-08T19:30:43.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.860+0000 7fd4fd7fa640 1 -- 172.21.15.119:0/1135510594 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fd4f0058140 con 0x7fd50006a970 2023-11-08T19:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:43.863 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:40.469042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:43.863 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 81 2023-11-08T19:30:43.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.862+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4e006aea0 msgr2=0x7fd4e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.862+0000 7fd506bb0640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4e006aea0 0x7fd4e006d360 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd500143e40 tx=0x7fd4e8009040 comp rx=0 tx=0).stop 2023-11-08T19:30:43.865 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.862+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 msgr2=0x7fd5000b4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:43.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.862+0000 7fd506bb0640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 0x7fd5000b4140 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd4f000d620 tx=0x7fd4f000dae0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 shutdown_connections 2023-11-08T19:30:43.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd4e006aea0 0x7fd4e006d360 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.866 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5000ccde0 0x7fd500160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd5000c9150 0x7fd5000b0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 --2- 172.21.15.119:0/1135510594 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd50006a970 0x7fd5000b4140 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:43.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 >> 172.21.15.119:0/1135510594 conn(0x7fd5000cb610 msgr2=0x7fd50006fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:43.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 shutdown_connections 2023-11-08T19:30:43.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:43.863+0000 7fd506bb0640 1 -- 172.21.15.119:0/1135510594 wait complete. 2023-11-08T19:30:44.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:30:40.469042+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 100, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 7515, 'mds_1': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7065': {'gid': 7065, 'name': 'd', 'rank': 1, 'incarnation': 55, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6834/4246462419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4246462419}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4246462419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7515': {'gid': 7515, 'name': 'a', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.119:6837/1517894744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.119:6835', 'nonce': 1517894744}, {'type': 'v1', 'addr': '172.21.15.119:6837', 'nonce': 1517894744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:30:44.287 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:30:44.288 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:30:44.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.514+0000 7f73f759e640 1 Processor -- start 2023-11-08T19:30:44.518 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.514+0000 7f73f759e640 1 -- start start 2023-11-08T19:30:44.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.515+0000 7f73f759e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73f80c9c90 0x7f73f806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.515+0000 7f73f759e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f806be20 con 0x7f73f815aea0 2023-11-08T19:30:44.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73f759e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f806c100 con 0x7f73f806d400 2023-11-08T19:30:44.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73f759e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f806c3e0 con 0x7f73f80c9c90 2023-11-08T19:30:44.519 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73f659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f73f815aea0 0x7f73f806b650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41646/0 (socket says 172.21.15.119:41646) 2023-11-08T19:30:44.520 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73f659c640 1 -- 172.21.15.119:0/3397873773 learned_addr learned my addr 172.21.15.119:0/3397873773 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73effff640 1 --2- 172.21.15.119:0/3397873773 >> v2:172.21.15.188:3300/0 conn(0x7f73f80c9c90 0x7f73f806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.522 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602813579 0 0) 0x7f73f806be20 con 0x7f73f815aea0 2023-11-08T19:30:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.516+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73e4003520 con 0x7f73f815aea0 2023-11-08T19:30:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89709759 0 0) 0x7f73e4003520 con 0x7f73f815aea0 2023-11-08T19:30:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 >> v2:172.21.15.188:3300/0 conn(0x7f73f80c9c90 msgr2=0x7f73f806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 --2- 172.21.15.119:0/3397873773 >> v2:172.21.15.188:3300/0 conn(0x7f73f80c9c90 0x7f73f806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.523 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 >> v1:172.21.15.188:6789/0 conn(0x7f73f806d400 legacy=0x7f73f80b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f806c6c0 con 0x7f73f815aea0 2023-11-08T19:30:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f73e8001670 con 0x7f73f815aea0 2023-11-08T19:30:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 >> v1:172.21.15.119:6789/0 conn(0x7f73f815aea0 legacy=0x7f73f806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e4003970 0x7f73e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e40047c0 0x7f73e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.524 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e4005180 0x7f73e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f806c3e0 con 0x7f73e4005180 2023-11-08T19:30:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e400c0d0 con 0x7f73e40047c0 2023-11-08T19:30:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e400c290 con 0x7f73e4003970 2023-11-08T19:30:44.525 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e4005180 0x7f73e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.517+0000 7f73effff640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e40047c0 0x7f73e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.518+0000 7f73f659c640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e4003970 0x7f73e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.518+0000 7f73f6d9d640 1 -- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e4003970 msgr2=0x7f73e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.518+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73e4003970 0x7f73e4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.526 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.518+0000 7f73f6d9d640 1 -- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e40047c0 msgr2=0x7f73e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.518+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e40047c0 0x7f73e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.518+0000 7f73f6d9d640 1 -- 172.21.15.119:0/3397873773 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4003520 con 0x7f73e4005180 2023-11-08T19:30:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.519+0000 7f73f6d9d640 1 -- 172.21.15.119:0/3397873773 reap_dead start 2023-11-08T19:30:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.519+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e4005180 0x7f73e400b980 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001e50 tx=0x7f73e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80752ba99738aace server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.519+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0003b70 con 0x7f73e4005180 2023-11-08T19:30:44.527 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.519+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f0003d10 con 0x7f73e4005180 2023-11-08T19:30:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.519+0000 7f73f5d9b640 1 -- 172.21.15.119:0/3397873773 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0005310 con 0x7f73e4005180 2023-11-08T19:30:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 -- 172.21.15.119:0/3397873773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e4005180 msgr2=0x7f73e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e4005180 0x7f73e400b980 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001e50 tx=0x7f73e40057f0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 -- 172.21.15.119:0/3397873773 shutdown_connections 2023-11-08T19:30:44.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 --2- 172.21.15.119:0/3397873773 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73e4005180 0x7f73e400b980 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 -- 172.21.15.119:0/3397873773 >> 172.21.15.119:0/3397873773 conn(0x7f73f80b0560 msgr2=0x7f73f80b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 -- 172.21.15.119:0/3397873773 shutdown_connections 2023-11-08T19:30:44.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.520+0000 7f73f759e640 1 -- 172.21.15.119:0/3397873773 wait complete. 2023-11-08T19:30:44.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.522+0000 7f73f759e640 1 Processor -- start 2023-11-08T19:30:44.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.522+0000 7f73f759e640 1 -- start start 2023-11-08T19:30:44.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f759e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4005180 0x7f73f80c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f759e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73f8159ab0 0x7f73f80cdc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f759e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 0x7f73f814c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f759e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80ce5b0 con 0x7f73f80ce150 2023-11-08T19:30:44.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f759e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80c89e0 con 0x7f73e4005180 2023-11-08T19:30:44.534 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f759e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80c25a0 con 0x7f73f8159ab0 2023-11-08T19:30:44.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 0x7f73f814c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 0x7f73f814c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45702/0 (socket says 172.21.15.119:45702) 2023-11-08T19:30:44.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 -- 172.21.15.119:0/1104896555 learned_addr learned my addr 172.21.15.119:0/1104896555 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 -- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73f8159ab0 msgr2=0x7f73f80cdc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.535 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f659c640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4005180 0x7f73f80c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73effff640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73f8159ab0 0x7f73f80cdc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73f8159ab0 0x7f73f80cdc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 -- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4005180 msgr2=0x7f73f80c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4005180 0x7f73f80c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.536 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.523+0000 7f73f6d9d640 1 -- 172.21.15.119:0/1104896555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f80c2880 con 0x7f73f80ce150 2023-11-08T19:30:44.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.524+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 0x7f73f814c600 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001f70 tx=0x7f73f00014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.524+0000 7f73eeffd640 1 -- 172.21.15.119:0/1104896555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0006d50 con 0x7f73f80ce150 2023-11-08T19:30:44.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.524+0000 7f73eeffd640 1 -- 172.21.15.119:0/1104896555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f0039480 con 0x7f73f80ce150 2023-11-08T19:30:44.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.524+0000 7f73eeffd640 1 -- 172.21.15.119:0/1104896555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0039850 con 0x7f73f80ce150 2023-11-08T19:30:44.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.524+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f80c2b60 con 0x7f73f80ce150 2023-11-08T19:30:44.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.524+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f80c3120 con 0x7f73f80ce150 2023-11-08T19:30:44.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.526+0000 7f73eeffd640 1 -- 172.21.15.119:0/1104896555 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73f00399b0 con 0x7f73f80ce150 2023-11-08T19:30:44.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.527+0000 7f73eeffd640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73d806ae70 0x7f73d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.527+0000 7f73eeffd640 1 -- 172.21.15.119:0/1104896555 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f73f00abab0 con 0x7f73f80ce150 2023-11-08T19:30:44.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.527+0000 7f73f659c640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73d806ae70 0x7f73d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.528+0000 7f73ffc37740 1 Processor -- start 2023-11-08T19:30:44.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.528+0000 7f73f659c640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73d806ae70 0x7f73d806d330 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f73e8001eb0 tx=0x7f73e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.528+0000 7f73ffc37740 1 -- start start 2023-11-08T19:30:44.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2da6bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2da9be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da6c4c0 0x55dc2da9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73ffc37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2d9de7e0 con 0x55dc2da99460 2023-11-08T19:30:44.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2d9dbed0 con 0x55dc2da99080 2023-11-08T19:30:44.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc2d9d77c0 con 0x55dc2da6c4c0 2023-11-08T19:30:44.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2da9be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2da9be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45710/0 (socket says 172.21.15.119:45710) 2023-11-08T19:30:44.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 -- 172.21.15.119:0/475979009 learned_addr learned my addr 172.21.15.119:0/475979009 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73f659c640 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2da6bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 -- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da6c4c0 msgr2=0x55dc2da9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da6c4c0 0x55dc2da9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 -- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 msgr2=0x55dc2da6bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.529+0000 7f73effff640 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2da6bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.530+0000 7f73effff640 1 -- 172.21.15.119:0/475979009 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc2d9aab10 con 0x55dc2da99460 2023-11-08T19:30:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.530+0000 7f73effff640 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2da9be10 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f73e000e8e0 tx=0x7f73e000eda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfde08c034ac2d35 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.530+0000 7f73d7fff640 1 -- 172.21.15.119:0/475979009 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0010ca0 con 0x55dc2da99460 2023-11-08T19:30:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.530+0000 7f73d7fff640 1 -- 172.21.15.119:0/475979009 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0010e00 con 0x55dc2da99460 2023-11-08T19:30:44.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.530+0000 7f73d7fff640 1 -- 172.21.15.119:0/475979009 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e001c620 con 0x55dc2da99460 2023-11-08T19:30:44.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 -- 172.21.15.119:0/475979009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 msgr2=0x55dc2da9be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2da9be10 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f73e000e8e0 tx=0x7f73e000eda0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 -- 172.21.15.119:0/475979009 shutdown_connections 2023-11-08T19:30:44.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da6c4c0 0x55dc2da9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2da9be10 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 --2- 172.21.15.119:0/475979009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2da6bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 -- 172.21.15.119:0/475979009 >> 172.21.15.119:0/475979009 conn(0x55dc2da6e290 msgr2=0x55dc2dab6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 -- 172.21.15.119:0/475979009 shutdown_connections 2023-11-08T19:30:44.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 -- 172.21.15.119:0/475979009 wait complete. 2023-11-08T19:30:44.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.531+0000 7f73ffc37740 1 -- start start 2023-11-08T19:30:44.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.532+0000 7f73ffc37740 1 Processor -- start 2023-11-08T19:30:44.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2dab1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2daca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 0x55dc2da86990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2d974c40 con 0x55dc2da99460 2023-11-08T19:30:44.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2d9748e0 con 0x55dc2da99080 2023-11-08T19:30:44.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc2d73af20 con 0x55dc2dacabd0 2023-11-08T19:30:44.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73ffc37740 10 client.0 _ll_register_callbacks cb 0x55dc2d996cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:44.549 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2daca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2daca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45720/0 (socket says 172.21.15.119:45720) 2023-11-08T19:30:44.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 -- 172.21.15.119:0/1975439472 learned_addr learned my addr 172.21.15.119:0/1975439472 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 -- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 msgr2=0x55dc2da86990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 0x55dc2da86990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.550 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 -- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 msgr2=0x55dc2dab1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2dab1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.533+0000 7f73effff640 1 -- 172.21.15.119:0/1975439472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e000e510 con 0x55dc2da99460 2023-11-08T19:30:44.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73effff640 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2daca690 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f73e00105c0 tx=0x7f73e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42b5301e00008dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:44.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73d57fa640 1 -- 172.21.15.119:0/1975439472 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0021070 con 0x55dc2da99460 2023-11-08T19:30:44.551 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73d57fa640 1 -- 172.21.15.119:0/1975439472 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f73e0004300 con 0x55dc2da99460 2023-11-08T19:30:44.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dc2d9abc80 con 0x55dc2da99460 2023-11-08T19:30:44.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73d57fa640 1 -- 172.21.15.119:0/1975439472 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73e00044c0 con 0x55dc2da99460 2023-11-08T19:30:44.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 10 client.7656 fetch_fsmap learned FSMap version 81 2023-11-08T19:30:44.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55dc2d9d5e40 con 0x55dc2da99460 2023-11-08T19:30:44.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73d57fa640 1 -- 172.21.15.119:0/1975439472 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55dc2d9abc80 con 0x55dc2da99460 2023-11-08T19:30:44.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73d57fa640 1 -- 172.21.15.119:0/1975439472 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f73e001a020 con 0x55dc2da99460 2023-11-08T19:30:44.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 10 client.7656 fetch_fsmap finished waiting for FSMap version 81 2023-11-08T19:30:44.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 10 client.7656 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:30:44.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 20 client.7656 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:44.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da8a0a0 0x55dc2da87d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 4 client.7656 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:30:44.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.534+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc2d9d07a0 con 0x55dc2da8a0a0 2023-11-08T19:30:44.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.535+0000 7f73f659c640 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da8a0a0 0x55dc2da87d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.536+0000 7f73f659c640 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da8a0a0 0x55dc2da87d70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2719d3ad04c9626a server_cookie=89c3a6b9f0e1b89f in_seq=0 out_seq=0 2023-11-08T19:30:44.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.536+0000 7f73d57fa640 10 client.7656 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:44.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73d57fa640 1 -- 172.21.15.119:0/1975439472 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc2d9d07a0 con 0x55dc2da8a0a0 2023-11-08T19:30:44.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73d57fa640 10 client.7656 handle_command_reply: tid=0 2023-11-08T19:30:44.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 client.7656 shutdown 2023-11-08T19:30:44.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 msgr2=0x55dc2daca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2daca690 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f73e00105c0 tx=0x7f73e00046c0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 shutdown_connections 2023-11-08T19:30:44.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 0x55dc2da86990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99460 0x55dc2daca690 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da8a0a0 0x55dc2da87d70 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 --2- 172.21.15.119:0/1975439472 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99080 0x55dc2dab1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73d57fa640 0 client.7656 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:44.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 >> 172.21.15.119:0/1975439472 conn(0x55dc2da6c4c0 msgr2=0x55dc2da6ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.538+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 shutdown_connections 2023-11-08T19:30:44.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.539+0000 7f73ffc37740 1 -- 172.21.15.119:0/1975439472 wait complete. 2023-11-08T19:30:44.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.539+0000 7f73ffc37740 20 client.7656 trim_cache size 0 max 16384 2023-11-08T19:30:44.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.555+0000 7f73ffc37740 1 Processor -- start 2023-11-08T19:30:44.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.555+0000 7f73ffc37740 1 -- start start 2023-11-08T19:30:44.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da99080 0x55dc2dab14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 0x55dc2da73040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 0x55dc2da6b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73ffc37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2daa02b0 con 0x55dc2dacabd0 2023-11-08T19:30:44.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2daad9c0 con 0x55dc2da99460 2023-11-08T19:30:44.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc2da6bdb0 con 0x55dc2da99080 2023-11-08T19:30:44.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73f6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 0x55dc2da6b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73f6d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 0x55dc2da6b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45730/0 (socket says 172.21.15.119:45730) 2023-11-08T19:30:44.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.556+0000 7f73f6d9d640 1 -- 172.21.15.119:0/442672996 learned_addr learned my addr 172.21.15.119:0/442672996 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73f6d9d640 1 -- 172.21.15.119:0/442672996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da99080 msgr2=0x55dc2dab14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da99080 0x55dc2dab14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73f6d9d640 1 -- 172.21.15.119:0/442672996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 msgr2=0x55dc2da73040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.562 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 0x55dc2da73040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73f6d9d640 1 -- 172.21.15.119:0/442672996 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc2d973b90 con 0x55dc2dacabd0 2023-11-08T19:30:44.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73f6d9d640 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 0x55dc2da6b870 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001f40 tx=0x7f73f0079990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c28f87a7b00de39 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73ecff9640 1 -- 172.21.15.119:0/442672996 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f007a030 con 0x55dc2dacabd0 2023-11-08T19:30:44.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73ecff9640 1 -- 172.21.15.119:0/442672996 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f007e180 con 0x55dc2dacabd0 2023-11-08T19:30:44.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.557+0000 7f73ecff9640 1 -- 172.21.15.119:0/442672996 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f00823c0 con 0x55dc2dacabd0 2023-11-08T19:30:44.563 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 -- 172.21.15.119:0/442672996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 msgr2=0x55dc2da6b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 0x55dc2da6b870 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001f40 tx=0x7f73f0079990 comp rx=0 tx=0).stop 2023-11-08T19:30:44.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 -- 172.21.15.119:0/442672996 shutdown_connections 2023-11-08T19:30:44.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2dacabd0 0x55dc2da6b870 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 0x55dc2da73040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.564 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 --2- 172.21.15.119:0/442672996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2da99080 0x55dc2dab14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 -- 172.21.15.119:0/442672996 >> 172.21.15.119:0/442672996 conn(0x55dc2da6c4c0 msgr2=0x55dc2da6ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 -- 172.21.15.119:0/442672996 shutdown_connections 2023-11-08T19:30:44.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 -- 172.21.15.119:0/442672996 wait complete. 2023-11-08T19:30:44.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.558+0000 7f73ffc37740 1 -- start start 2023-11-08T19:30:44.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 Processor -- start 2023-11-08T19:30:44.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 0x55dc2da984f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 0x55dc2da9b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 0x55dc2da9be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2db11170 con 0x55dc2da99080 2023-11-08T19:30:44.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc2db13880 con 0x55dc2da99460 2023-11-08T19:30:44.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.559+0000 7f73ffc37740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc2db132c0 con 0x55dc2dacabd0 2023-11-08T19:30:44.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73ffc37740 10 client.0 _ll_register_callbacks cb 0x7f73e000e730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:44.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 0x55dc2da984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 0x55dc2da984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45736/0 (socket says 172.21.15.119:45736) 2023-11-08T19:30:44.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 -- 172.21.15.119:0/4276915657 learned_addr learned my addr 172.21.15.119:0/4276915657 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 -- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 msgr2=0x55dc2da9be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 0x55dc2da9be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 -- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 msgr2=0x55dc2da9b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 0x55dc2da9b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 -- 172.21.15.119:0/4276915657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f80c3120 con 0x55dc2da99080 2023-11-08T19:30:44.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73f659c640 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 0x55dc2da984f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f73e80015e0 tx=0x7f73e8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91134a444e96e5a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.560+0000 7f73d57fa640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:44.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73d57fa640 1 -- 172.21.15.119:0/4276915657 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e800c050 con 0x55dc2da99080 2023-11-08T19:30:44.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73d57fa640 1 -- 172.21.15.119:0/4276915657 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f73e8005c80 con 0x55dc2da99080 2023-11-08T19:30:44.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73e0010e00 con 0x55dc2da99080 2023-11-08T19:30:44.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73d57fa640 1 -- 172.21.15.119:0/4276915657 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73e8005e40 con 0x55dc2da99080 2023-11-08T19:30:44.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 10 client.7662 fetch_fsmap learned FSMap version 81 2023-11-08T19:30:44.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55dc2d999180 con 0x55dc2da99080 2023-11-08T19:30:44.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73d57fa640 1 -- 172.21.15.119:0/4276915657 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e0010e00 con 0x55dc2da99080 2023-11-08T19:30:44.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73d57fa640 1 -- 172.21.15.119:0/4276915657 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7f73e8020050 con 0x55dc2da99080 2023-11-08T19:30:44.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 10 client.7662 fetch_fsmap finished waiting for FSMap version 81 2023-11-08T19:30:44.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 10 client.7662 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:30:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 20 client.7662 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da9d630 0x55dc2da9da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 4 client.7662 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:30:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.561+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55dc2d9d04c0 con 0x55dc2da9d630 2023-11-08T19:30:44.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.562+0000 7f73effff640 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da9d630 0x55dc2da9da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.563+0000 7f73effff640 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da9d630 0x55dc2da9da10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3e132e83c259c121 server_cookie=646e956e1e702b0f in_seq=0 out_seq=0 2023-11-08T19:30:44.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.563+0000 7f73d57fa640 10 client.7662 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:44.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.563+0000 7f73d57fa640 1 -- 172.21.15.119:0/4276915657 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55dc2d9d04c0 con 0x55dc2da9d630 2023-11-08T19:30:44.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.563+0000 7f73d57fa640 10 client.7662 handle_command_reply: tid=0 2023-11-08T19:30:44.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.563+0000 7f73ffc37740 1 client.7662 shutdown 2023-11-08T19:30:44.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 msgr2=0x55dc2da984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 0x55dc2da984f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f73e80015e0 tx=0x7f73e8000ed0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 shutdown_connections 2023-11-08T19:30:44.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dc2dacabd0 0x55dc2da9be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x55dc2da9d630 0x55dc2da9da10 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dc2da99460 0x55dc2da9b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 --2- 172.21.15.119:0/4276915657 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55dc2da99080 0x55dc2da984f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 >> 172.21.15.119:0/4276915657 conn(0x55dc2da6c4c0 msgr2=0x55dc2da87ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73d57fa640 0 client.7662 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:30:44.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 shutdown_connections 2023-11-08T19:30:44.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 1 -- 172.21.15.119:0/4276915657 wait complete. 2023-11-08T19:30:44.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73ffc37740 20 client.7662 trim_cache size 0 max 16384 2023-11-08T19:30:44.578 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:30:44.578 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:30:44.579 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-11-08T19:30:44.579 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7065, 2023-11-08T19:30:44.579 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:30:44.579 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:30:44.579 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:30:44.579 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 95.885417326999999, 2023-11-08T19:30:44.580 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 81, 2023-11-08T19:30:44.580 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 100, 2023-11-08T19:30:44.580 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 100, 2023-11-08T19:30:44.580 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 118.938820251 2023-11-08T19:30:44.580 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:30:44.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73d806ae70 msgr2=0x7f73d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73f759e640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73d806ae70 0x7f73d806d330 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f73e8001eb0 tx=0x7f73e8009040 comp rx=0 tx=0).stop 2023-11-08T19:30:44.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 msgr2=0x7f73f814c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.564+0000 7f73f759e640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 0x7f73f814c600 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f73f0001f70 tx=0x7f73f00014a0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 shutdown_connections 2023-11-08T19:30:44.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73d806ae70 0x7f73d806d330 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73f80ce150 0x7f73f814c600 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73f8159ab0 0x7f73f80cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 --2- 172.21.15.119:0/1104896555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73e4005180 0x7f73f80c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 >> 172.21.15.119:0/1104896555 conn(0x7f73f80b0560 msgr2=0x7f73f80ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 shutdown_connections 2023-11-08T19:30:44.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.565+0000 7f73f759e640 1 -- 172.21.15.119:0/1104896555 wait complete. 2023-11-08T19:30:44.584 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T19:30:44.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.777+0000 7fe5652e1640 1 Processor -- start 2023-11-08T19:30:44.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.778+0000 7fe5652e1640 1 -- start start 2023-11-08T19:30:44.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.779+0000 7fe5652e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe560142f80 0x7fe5600cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.779+0000 7fe5652e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015d9f0 0x7fe5600c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.779+0000 7fe5652e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe56006cb40 0x7fe56006cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.779+0000 7fe5652e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe560143a40 con 0x7fe56006cb40 2023-11-08T19:30:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.779+0000 7fe5652e1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe56006d510 con 0x7fe560142f80 2023-11-08T19:30:44.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.779+0000 7fe5652e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5600c4960 con 0x7fe56015d9f0 2023-11-08T19:30:44.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe56006cb40 0x7fe56006cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe56006cb40 0x7fe56006cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45738/0 (socket says 172.21.15.119:45738) 2023-11-08T19:30:44.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 -- 172.21.15.119:0/252622629 learned_addr learned my addr 172.21.15.119:0/252622629 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe5567fc640 1 --2- 172.21.15.119:0/252622629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015d9f0 0x7fe5600c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.785 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55effd640 1 --2- 172.21.15.119:0/252622629 >> v2:172.21.15.188:3300/0 conn(0x7fe560142f80 0x7fe5600cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 -- 172.21.15.119:0/252622629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015d9f0 msgr2=0x7fe5600c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/252622629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015d9f0 0x7fe5600c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 -- 172.21.15.119:0/252622629 >> v2:172.21.15.188:3300/0 conn(0x7fe560142f80 msgr2=0x7fe5600cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/252622629 >> v2:172.21.15.188:3300/0 conn(0x7fe560142f80 0x7fe5600cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.780+0000 7fe55f7fe640 1 -- 172.21.15.119:0/252622629 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56015df60 con 0x7fe56006cb40 2023-11-08T19:30:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.781+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/252622629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe56006cb40 0x7fe56006cfd0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe5600c3b00 tx=0x7fe55802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45d3806b6a03b2a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.781+0000 7fe55e7fc640 1 -- 172.21.15.119:0/252622629 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55802c9f0 con 0x7fe56006cb40 2023-11-08T19:30:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe55e7fc640 1 -- 172.21.15.119:0/252622629 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55802cb50 con 0x7fe56006cb40 2023-11-08T19:30:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe55e7fc640 1 -- 172.21.15.119:0/252622629 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5580343f0 con 0x7fe56006cb40 2023-11-08T19:30:44.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 -- 172.21.15.119:0/252622629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe56006cb40 msgr2=0x7fe56006cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 --2- 172.21.15.119:0/252622629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe56006cb40 0x7fe56006cfd0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fe5600c3b00 tx=0x7fe55802be50 comp rx=0 tx=0).stop 2023-11-08T19:30:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 -- 172.21.15.119:0/252622629 shutdown_connections 2023-11-08T19:30:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 --2- 172.21.15.119:0/252622629 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe56006cb40 0x7fe56006cfd0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 --2- 172.21.15.119:0/252622629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56015d9f0 0x7fe5600c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 --2- 172.21.15.119:0/252622629 >> v2:172.21.15.188:3300/0 conn(0x7fe560142f80 0x7fe5600cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 -- 172.21.15.119:0/252622629 >> 172.21.15.119:0/252622629 conn(0x7fe5600cb610 msgr2=0x7fe5600cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 -- 172.21.15.119:0/252622629 shutdown_connections 2023-11-08T19:30:44.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.782+0000 7fe5652e1640 1 -- 172.21.15.119:0/252622629 wait complete. 2023-11-08T19:30:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.784+0000 7fe5652e1640 1 Processor -- start 2023-11-08T19:30:44.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.784+0000 7fe5652e1640 1 -- start start 2023-11-08T19:30:44.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5652e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56006cb40 0x7fe5601f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5652e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 0x7fe5600b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5652e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56015d9f0 0x7fe560159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5652e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56015a4c0 con 0x7fe560142f80 2023-11-08T19:30:44.792 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5652e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe56015b320 con 0x7fe56015d9f0 2023-11-08T19:30:44.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5652e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56015a710 con 0x7fe56006cb40 2023-11-08T19:30:44.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 0x7fe5600b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 0x7fe5600b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45754/0 (socket says 172.21.15.119:45754) 2023-11-08T19:30:44.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.785+0000 7fe5567fc640 1 -- 172.21.15.119:0/908918432 learned_addr learned my addr 172.21.15.119:0/908918432 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe55effd640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56006cb40 0x7fe5601f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe5567fc640 1 -- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56006cb40 msgr2=0x7fe5601f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe5567fc640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56006cb40 0x7fe5601f3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56015d9f0 0x7fe560159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe5567fc640 1 -- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56015d9f0 msgr2=0x7fe560159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe5567fc640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56015d9f0 0x7fe560159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe5567fc640 1 -- 172.21.15.119:0/908918432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe558009b30 con 0x7fe560142f80 2023-11-08T19:30:44.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe5567fc640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 0x7fe5600b3610 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe54800d620 tx=0x7fe54800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.786+0000 7fe557fff640 1 -- 172.21.15.119:0/908918432 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800b620 con 0x7fe560142f80 2023-11-08T19:30:44.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.787+0000 7fe557fff640 1 -- 172.21.15.119:0/908918432 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54800b780 con 0x7fe560142f80 2023-11-08T19:30:44.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.787+0000 7fe557fff640 1 -- 172.21.15.119:0/908918432 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800bb30 con 0x7fe560142f80 2023-11-08T19:30:44.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.787+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe56015a9f0 con 0x7fe560142f80 2023-11-08T19:30:44.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.787+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56015afb0 con 0x7fe560142f80 2023-11-08T19:30:44.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.789+0000 7fe557fff640 1 -- 172.21.15.119:0/908918432 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe54800bc90 con 0x7fe560142f80 2023-11-08T19:30:44.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.790+0000 7fe557fff640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe54406aea0 0x7fe54406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.790+0000 7fe557fff640 1 -- 172.21.15.119:0/908918432 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fe54808a210 con 0x7fe560142f80 2023-11-08T19:30:44.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.790+0000 7fe55effd640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe54406aea0 0x7fe54406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.790+0000 7fe567e70740 1 Processor -- start 2023-11-08T19:30:44.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.791+0000 7fe55effd640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe54406aea0 0x7fe54406d360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe550006fd0 tx=0x7fe550008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.791+0000 7fe567e70740 1 -- start start 2023-11-08T19:30:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332cec0 0x55b8732ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe567e70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 0x55b87332fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b873300300 0x55b873330190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe567e70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8732727e0 con 0x55b87332d2a0 2023-11-08T19:30:44.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b87326fed0 con 0x55b87332cec0 2023-11-08T19:30:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b87326b7c0 con 0x55b873300300 2023-11-08T19:30:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 0x55b87332fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 0x55b87332fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45770/0 (socket says 172.21.15.119:45770) 2023-11-08T19:30:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 -- 172.21.15.119:0/3049892905 learned_addr learned my addr 172.21.15.119:0/3049892905 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe55effd640 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332cec0 0x55b8732ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b873300300 0x55b873330190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 -- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b873300300 msgr2=0x55b873330190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b873300300 0x55b873330190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 -- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332cec0 msgr2=0x55b8732ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332cec0 0x55b8732ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.792+0000 7fe5567fc640 1 -- 172.21.15.119:0/3049892905 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b87323eb10 con 0x55b87332d2a0 2023-11-08T19:30:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.793+0000 7fe5567fc640 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 0x55b87332fc50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe5480605f0 tx=0x7fe548058f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413a681fbe2b811c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.793+0000 7fe554ff9640 1 -- 172.21.15.119:0/3049892905 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548070180 con 0x55b87332d2a0 2023-11-08T19:30:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.793+0000 7fe554ff9640 1 -- 172.21.15.119:0/3049892905 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54805c970 con 0x55b87332d2a0 2023-11-08T19:30:44.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.793+0000 7fe554ff9640 1 -- 172.21.15.119:0/3049892905 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548060b20 con 0x55b87332d2a0 2023-11-08T19:30:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 -- 172.21.15.119:0/3049892905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 msgr2=0x55b87332fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 0x55b87332fc50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe5480605f0 tx=0x7fe548058f20 comp rx=0 tx=0).stop 2023-11-08T19:30:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 -- 172.21.15.119:0/3049892905 shutdown_connections 2023-11-08T19:30:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b873300300 0x55b873330190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332d2a0 0x55b87332fc50 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 --2- 172.21.15.119:0/3049892905 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332cec0 0x55b8732ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 -- 172.21.15.119:0/3049892905 >> 172.21.15.119:0/3049892905 conn(0x55b8733020d0 msgr2=0x55b87334ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 -- 172.21.15.119:0/3049892905 shutdown_connections 2023-11-08T19:30:44.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.794+0000 7fe567e70740 1 -- 172.21.15.119:0/3049892905 wait complete. 2023-11-08T19:30:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.795+0000 7fe567e70740 1 -- start start 2023-11-08T19:30:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.795+0000 7fe567e70740 1 Processor -- start 2023-11-08T19:30:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b87335b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b873342b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b873345680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b873208c40 con 0x55b87332cec0 2023-11-08T19:30:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8732088e0 con 0x55b87332d2a0 2023-11-08T19:30:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b872fcef20 con 0x55b8733448b0 2023-11-08T19:30:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe567e70740 10 client.0 _ll_register_callbacks cb 0x55b87322acf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:44.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b87335b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b87335b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45772/0 (socket says 172.21.15.119:45772) 2023-11-08T19:30:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 -- 172.21.15.119:0/1785014878 learned_addr learned my addr 172.21.15.119:0/1785014878 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b873345680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe5567fc640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b873342b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 -- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 msgr2=0x55b873345680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b873345680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 -- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 msgr2=0x55b873342b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.796+0000 7fe55effd640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b873342b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe55effd640 1 -- 172.21.15.119:0/1785014878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe56015afb0 con 0x55b87332cec0 2023-11-08T19:30:44.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe55effd640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b87335b850 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe550015d60 tx=0x7fe550004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ce7fd8471e7a0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe54dffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe54dffb640 1 -- 172.21.15.119:0/1785014878 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550015470 con 0x55b87332cec0 2023-11-08T19:30:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe54dffb640 1 -- 172.21.15.119:0/1785014878 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fe55000ec00 con 0x55b87332cec0 2023-11-08T19:30:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b87323fc80 con 0x55b87332cec0 2023-11-08T19:30:44.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe54dffb640 1 -- 172.21.15.119:0/1785014878 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe55000edc0 con 0x55b87332cec0 2023-11-08T19:30:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe567e70740 10 client.7674 fetch_fsmap learned FSMap version 81 2023-11-08T19:30:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.797+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55b873269e40 con 0x55b87332cec0 2023-11-08T19:30:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe54dffb640 1 -- 172.21.15.119:0/1785014878 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b87323fc80 con 0x55b87332cec0 2023-11-08T19:30:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe54dffb640 1 -- 172.21.15.119:0/1785014878 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe550021420 con 0x55b87332cec0 2023-11-08T19:30:44.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe567e70740 10 client.7674 fetch_fsmap finished waiting for FSMap version 81 2023-11-08T19:30:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe567e70740 10 client.7674 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:30:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe567e70740 20 client.7674 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe567e70740 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b87335e410 0x55b87335e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe567e70740 4 client.7674 mds_command: new command op to 7515 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:30:44.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 --> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8732647a0 con 0x55b87335e410 2023-11-08T19:30:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.798+0000 7fe5567fc640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b87335e410 0x55b87335e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.799+0000 7fe5567fc640 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b87335e410 0x55b87335e7f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b4ae11234f0aa5 server_cookie=39380a6fb78b125e in_seq=0 out_seq=0 2023-11-08T19:30:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.799+0000 7fe54dffb640 10 client.7674 ms_handle_connect on v2:172.21.15.119:6835/1517894744 2023-11-08T19:30:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.799+0000 7fe54dffb640 1 -- 172.21.15.119:0/1785014878 <== mds.0 v2:172.21.15.119:6835/1517894744 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b8732647a0 con 0x55b87335e410 2023-11-08T19:30:44.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.799+0000 7fe54dffb640 10 client.7674 handle_command_reply: tid=0 2023-11-08T19:30:44.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.799+0000 7fe567e70740 1 client.7674 shutdown 2023-11-08T19:30:44.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 msgr2=0x55b87335b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b87335b850 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe550015d60 tx=0x7fe550004730 comp rx=0 tx=0).stop 2023-11-08T19:30:44.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 shutdown_connections 2023-11-08T19:30:44.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b87335e410 0x55b87335e7f0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b873345680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b873342b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe54dffb640 0 client.7674 ms_handle_reset on v2:172.21.15.119:6835/1517894744 2023-11-08T19:30:44.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 --2- 172.21.15.119:0/1785014878 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b87335b850 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 >> 172.21.15.119:0/1785014878 conn(0x55b873300300 msgr2=0x55b873303c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 shutdown_connections 2023-11-08T19:30:44.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 1 -- 172.21.15.119:0/1785014878 wait complete. 2023-11-08T19:30:44.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.800+0000 7fe567e70740 20 client.7674 trim_cache size 0 max 16384 2023-11-08T19:30:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.817+0000 7fe567e70740 1 Processor -- start 2023-11-08T19:30:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.817+0000 7fe567e70740 1 -- start start 2023-11-08T19:30:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe567e70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b8732fee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b87332d2a0 0x55b8732ff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8733448b0 0x55b873364820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe567e70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b873332940 con 0x55b87332cec0 2023-11-08T19:30:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b873345a90 con 0x55b8733448b0 2023-11-08T19:30:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b87332f760 con 0x55b87332d2a0 2023-11-08T19:30:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b8732fee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b8732fee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45788/0 (socket says 172.21.15.119:45788) 2023-11-08T19:30:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 -- 172.21.15.119:0/3131170116 learned_addr learned my addr 172.21.15.119:0/3131170116 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8733448b0 0x55b873364820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 -- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b87332d2a0 msgr2=0x55b8732ff380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b87332d2a0 0x55b8732ff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 -- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8733448b0 msgr2=0x55b873364820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8733448b0 0x55b873364820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55effd640 1 -- 172.21.15.119:0/3131170116 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b873207b90 con 0x55b87332cec0 2023-11-08T19:30:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.818+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8733448b0 0x55b873364820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.819+0000 7fe55effd640 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b8732fee40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe550015d30 tx=0x7fe55001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609978da7537f3f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.819+0000 7fe5557fa640 1 -- 172.21.15.119:0/3131170116 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550021420 con 0x55b87332cec0 2023-11-08T19:30:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.819+0000 7fe5557fa640 1 -- 172.21.15.119:0/3131170116 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe550021580 con 0x55b87332cec0 2023-11-08T19:30:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.819+0000 7fe5557fa640 1 -- 172.21.15.119:0/3131170116 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55000ec00 con 0x55b87332cec0 2023-11-08T19:30:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 -- 172.21.15.119:0/3131170116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 msgr2=0x55b8732fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b8732fee40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fe550015d30 tx=0x7fe55001f000 comp rx=0 tx=0).stop 2023-11-08T19:30:44.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 -- 172.21.15.119:0/3131170116 shutdown_connections 2023-11-08T19:30:44.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b8733448b0 0x55b873364820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b87332d2a0 0x55b8732ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 --2- 172.21.15.119:0/3131170116 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b87332cec0 0x55b8732fee40 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 -- 172.21.15.119:0/3131170116 >> 172.21.15.119:0/3131170116 conn(0x55b873300300 msgr2=0x55b8732fb560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 -- 172.21.15.119:0/3131170116 shutdown_connections 2023-11-08T19:30:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.820+0000 7fe567e70740 1 -- 172.21.15.119:0/3131170116 wait complete. 2023-11-08T19:30:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.821+0000 7fe567e70740 1 -- start start 2023-11-08T19:30:44.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.821+0000 7fe567e70740 1 Processor -- start 2023-11-08T19:30:44.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b8733c0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b8733649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 0x55b873364f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8733415d0 con 0x55b8733312d0 2023-11-08T19:30:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8732ff5c0 con 0x55b87332d2a0 2023-11-08T19:30:44.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b87332fd40 con 0x55b8733448b0 2023-11-08T19:30:44.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe567e70740 10 client.0 _ll_register_callbacks cb 0x7fe5500159b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:30:44.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 0x55b873364f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 0x55b873364f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45800/0 (socket says 172.21.15.119:45800) 2023-11-08T19:30:44.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 -- 172.21.15.119:0/2321051017 learned_addr learned my addr 172.21.15.119:0/2321051017 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:44.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55effd640 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b8733c0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 -- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 msgr2=0x55b8733649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b8733649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 -- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 msgr2=0x55b8733c0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:44.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b8733c0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.822+0000 7fe55f7fe640 1 -- 172.21.15.119:0/2321051017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b873269e40 con 0x55b8733312d0 2023-11-08T19:30:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe55f7fe640 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 0x55b873364f10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe55802be90 tx=0x7fe558004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336f362c46df266c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe54dffb640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:30:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe54dffb640 1 -- 172.21.15.119:0/2321051017 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558034ab0 con 0x55b8733312d0 2023-11-08T19:30:44.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe54dffb640 1 -- 172.21.15.119:0/2321051017 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fe558044070 con 0x55b8733312d0 2023-11-08T19:30:44.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8732644c0 con 0x55b8733312d0 2023-11-08T19:30:44.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe54dffb640 1 -- 172.21.15.119:0/2321051017 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe558034c10 con 0x55b8733312d0 2023-11-08T19:30:44.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe567e70740 10 client.7680 fetch_fsmap learned FSMap version 81 2023-11-08T19:30:44.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.823+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) v3 -- 0x55b87322d180 con 0x55b8733312d0 2023-11-08T19:30:44.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe54dffb640 1 -- 172.21.15.119:0/2321051017 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b8732644c0 con 0x55b8733312d0 2023-11-08T19:30:44.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe54dffb640 1 -- 172.21.15.119:0/2321051017 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 81) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe558003990 con 0x55b8733312d0 2023-11-08T19:30:44.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe567e70740 10 client.7680 fetch_fsmap finished waiting for FSMap version 81 2023-11-08T19:30:44.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe567e70740 10 client.7680 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T19:30:44.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe567e70740 20 client.7680 populate_metadata read hostname 'smithi119' 2023-11-08T19:30:44.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe567e70740 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b873340ba0 0x55b87332afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:44.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe567e70740 4 client.7680 mds_command: new command op to 7515 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:30:44.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 --> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b872f6c6c0 con 0x55b873340ba0 2023-11-08T19:30:44.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe55effd640 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b873340ba0 0x55b87332afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:44.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe55effd640 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b873340ba0 0x55b87332afa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb06badee195e4ad server_cookie=a987e8fc0d8507ee in_seq=0 out_seq=0 2023-11-08T19:30:44.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.824+0000 7fe54dffb640 10 client.7680 ms_handle_connect on v2:172.21.15.119:6835/1517894744 2023-11-08T19:30:44.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe54dffb640 1 -- 172.21.15.119:0/2321051017 <== mds.0 v2:172.21.15.119:6835/1517894744 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55b872f6c6c0 con 0x55b873340ba0 2023-11-08T19:30:44.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe54dffb640 10 client.7680 handle_command_reply: tid=0 2023-11-08T19:30:44.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 client.7680 shutdown 2023-11-08T19:30:44.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 msgr2=0x55b873364f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 0x55b873364f10 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fe55802be90 tx=0x7fe558004ab0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 shutdown_connections 2023-11-08T19:30:44.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:6835/1517894744,v1:172.21.15.119:6837/1517894744] conn(0x55b873340ba0 0x55b87332afa0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55b8733312d0 0x55b873364f10 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55b8733448b0 0x55b8733649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.841 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 --2- 172.21.15.119:0/2321051017 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55b87332d2a0 0x55b8733c0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe54dffb640 0 client.7680 ms_handle_reset on v2:172.21.15.119:6835/1517894744 2023-11-08T19:30:44.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 >> 172.21.15.119:0/2321051017 conn(0x55b873300300 msgr2=0x55b87334e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 shutdown_connections 2023-11-08T19:30:44.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 1 -- 172.21.15.119:0/2321051017 wait complete. 2023-11-08T19:30:44.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.825+0000 7fe567e70740 20 client.7680 trim_cache size 0 max 16384 2023-11-08T19:30:44.843 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:30:44.843 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:30:44.843 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:30:44.844 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7515, 2023-11-08T19:30:44.844 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:30:44.844 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:30:44.844 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:30:44.844 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 7.3743349269999996, 2023-11-08T19:30:44.845 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 81, 2023-11-08T19:30:44.845 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 100, 2023-11-08T19:30:44.845 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 100, 2023-11-08T19:30:44.845 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 35.587841513000001 2023-11-08T19:30:44.845 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:30:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe54406aea0 msgr2=0x7fe54406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe54406aea0 0x7fe54406d360 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe550006fd0 tx=0x7fe550008040 comp rx=0 tx=0).stop 2023-11-08T19:30:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 msgr2=0x7fe5600b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 0x7fe5600b3610 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fe54800d620 tx=0x7fe54800dae0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 shutdown_connections 2023-11-08T19:30:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe54406aea0 0x7fe54406d360 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe56015d9f0 0x7fe560159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe560142f80 0x7fe5600b3610 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 --2- 172.21.15.119:0/908918432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe56006cb40 0x7fe5601f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:44.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 >> 172.21.15.119:0/908918432 conn(0x7fe5600cb610 msgr2=0x7fe5600c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:44.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 shutdown_connections 2023-11-08T19:30:44.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:44.826+0000 7fe5652e1640 1 -- 172.21.15.119:0/908918432 wait complete. 2023-11-08T19:30:44.849 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:45.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.055+0000 7fde9cc34640 1 Processor -- start 2023-11-08T19:30:45.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.055+0000 7fde9cc34640 1 -- start start 2023-11-08T19:30:45.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.057+0000 7fde9cc34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde9806a970 0x7fde9806cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.057+0000 7fde9cc34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980ccde0 0x7fde981653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.057+0000 7fde9cc34640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9806d4b0 con 0x7fde980c9150 2023-11-08T19:30:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.057+0000 7fde9cc34640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde981619b0 con 0x7fde9806a970 2023-11-08T19:30:45.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.057+0000 7fde9cc34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde98147b80 con 0x7fde980ccde0 2023-11-08T19:30:45.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde9659c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fde9806a970 0x7fde9806cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde8dd9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fde980c9150 0x7fde980cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41648/0 (socket says 172.21.15.119:41648) 2023-11-08T19:30:45.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde8dd9b640 1 -- 172.21.15.119:0/1662802722 learned_addr learned my addr 172.21.15.119:0/1662802722 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:30:45.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde96d9d640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980ccde0 0x7fde981653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde9659c640 1 -- 172.21.15.119:0/1662802722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980ccde0 msgr2=0x7fde981653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:45.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde9659c640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980ccde0 0x7fde981653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde9659c640 1 -- 172.21.15.119:0/1662802722 >> v1:172.21.15.119:6789/0 conn(0x7fde980c9150 legacy=0x7fde980cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde9659c640 1 -- 172.21.15.119:0/1662802722 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98147000 con 0x7fde9806a970 2023-11-08T19:30:45.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde96d9d640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980ccde0 0x7fde981653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:45.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.058+0000 7fde95d9b640 1 -- 172.21.15.119:0/1662802722 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569527057 0 0) 0x7fde80001110 con 0x7fde980c9150 2023-11-08T19:30:45.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.060+0000 7fde9659c640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde9806cf70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fde880097d0 tx=0x7fde8802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b8a0b1a19b9c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:45.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.060+0000 7fde95d9b640 1 -- 172.21.15.119:0/1662802722 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88031780 con 0x7fde9806a970 2023-11-08T19:30:45.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.060+0000 7fde95d9b640 1 -- 172.21.15.119:0/1662802722 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde880318e0 con 0x7fde9806a970 2023-11-08T19:30:45.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.060+0000 7fde95d9b640 1 -- 172.21.15.119:0/1662802722 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88031c70 con 0x7fde9806a970 2023-11-08T19:30:45.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 -- 172.21.15.119:0/1662802722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 msgr2=0x7fde9806cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde9806cf70 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fde880097d0 tx=0x7fde8802b820 comp rx=0 tx=0).stop 2023-11-08T19:30:45.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 -- 172.21.15.119:0/1662802722 shutdown_connections 2023-11-08T19:30:45.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980ccde0 0x7fde981653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 --2- 172.21.15.119:0/1662802722 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde9806cf70 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 -- 172.21.15.119:0/1662802722 >> 172.21.15.119:0/1662802722 conn(0x7fde980cb610 msgr2=0x7fde980cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:45.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 -- 172.21.15.119:0/1662802722 shutdown_connections 2023-11-08T19:30:45.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.061+0000 7fde9cc34640 1 -- 172.21.15.119:0/1662802722 wait complete. 2023-11-08T19:30:45.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.063+0000 7fde9cc34640 1 Processor -- start 2023-11-08T19:30:45.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.063+0000 7fde9cc34640 1 -- start start 2023-11-08T19:30:45.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.064+0000 7fde9cc34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde98160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.064+0000 7fde9cc34640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980c9150 0x7fde980b4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.064+0000 7fde9cc34640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980ccde0 0x7fde980c7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.064+0000 7fde9cc34640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98149a60 con 0x7fde9806a970 2023-11-08T19:30:45.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.064+0000 7fde9cc34640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98149d40 con 0x7fde980ccde0 2023-11-08T19:30:45.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.064+0000 7fde9cc34640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9814a020 con 0x7fde980c9150 2023-11-08T19:30:45.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde98160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde98160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45822/0 (socket says 172.21.15.119:45822) 2023-11-08T19:30:45.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 -- 172.21.15.119:0/4134150317 learned_addr learned my addr 172.21.15.119:0/4134150317 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:45.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde8dd9b640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980c9150 0x7fde980b4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde96d9d640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980ccde0 0x7fde980c7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 -- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980c9150 msgr2=0x7fde980b4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980c9150 0x7fde980b4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 -- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980ccde0 msgr2=0x7fde980c7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980ccde0 0x7fde980c7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde9659c640 1 -- 172.21.15.119:0/4134150317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde88009400 con 0x7fde9806a970 2023-11-08T19:30:45.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.065+0000 7fde96d9d640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980ccde0 0x7fde980c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:45.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.066+0000 7fde9659c640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde98160500 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fde88034040 tx=0x7fde880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:45.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.066+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde880328e0 con 0x7fde9806a970 2023-11-08T19:30:45.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.066+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde88032a60 con 0x7fde9806a970 2023-11-08T19:30:45.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.066+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde88032e10 con 0x7fde9806a970 2023-11-08T19:30:45.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.066+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9814a300 con 0x7fde9806a970 2023-11-08T19:30:45.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.066+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde98160a40 con 0x7fde9806a970 2023-11-08T19:30:45.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.067+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde98161b00 con 0x7fde9806a970 2023-11-08T19:30:45.076 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.068+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde8804d3f0 con 0x7fde9806a970 2023-11-08T19:30:45.077 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.069+0000 7fde8f7fe640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde7806aea0 0x7fde7806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.070+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fde880abaf0 con 0x7fde9806a970 2023-11-08T19:30:45.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.070+0000 7fde8dd9b640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde7806aea0 0x7fde7806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.070+0000 7fde8dd9b640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde7806aea0 0x7fde7806d360 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde9814ab10 tx=0x7fde800079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:45.079 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:30:45.072+0000 7f8b3f18b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:45.079 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:30:45.072+0000 7f8b3f18b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:45.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.074+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde88076440 con 0x7fde9806a970 2023-11-08T19:30:45.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.299+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde98074430 con 0x7fde9806a970 2023-11-08T19:30:45.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.301+0000 7fde8f7fe640 1 -- 172.21.15.119:0/4134150317 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fde88079a60 con 0x7fde9806a970 2023-11-08T19:30:45.303 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:45.304 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:40.469042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:45.304 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 81 2023-11-08T19:30:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde7806aea0 msgr2=0x7fde7806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde7806aea0 0x7fde7806d360 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fde9814ab10 tx=0x7fde800079b0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 msgr2=0x7fde98160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde98160500 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fde88034040 tx=0x7fde880046c0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 shutdown_connections 2023-11-08T19:30:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fde7806aea0 0x7fde7806d360 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fde980ccde0 0x7fde980c7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fde980c9150 0x7fde980b4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 --2- 172.21.15.119:0/4134150317 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fde9806a970 0x7fde98160500 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 >> 172.21.15.119:0/4134150317 conn(0x7fde980cb610 msgr2=0x7fde9806ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.303+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 shutdown_connections 2023-11-08T19:30:45.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.304+0000 7fde9cc34640 1 -- 172.21.15.119:0/4134150317 wait complete. 2023-11-08T19:30:45.324 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:30:45.545 INFO:teuthology.orchestra.run.smithi119.stdout:{"last_snap":4,"last_created":4,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:29:23.170262+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-08T19:29:51.023531+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2023-11-08T19:30:19.457611+0000","name":"s15","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:30:45.546 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 4, "last_destroyed": 1, "last_snap": 4, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:29:23.170262+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-08T19:29:51.023531+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2023-11-08T19:30:19.457611+0000" } ] } 2023-11-08T19:30:45.546 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:30:45.547 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=6 2023-11-08T19:30:45.547 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:45.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.760+0000 7fef7ec90640 1 Processor -- start 2023-11-08T19:30:45.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.760+0000 7fef7ec90640 1 -- start start 2023-11-08T19:30:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.762+0000 7fef7ec90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.762+0000 7fef7ec90640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef7806cb40 0x7fef7806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.762+0000 7fef7ec90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef780c4960 con 0x7fef78142f80 2023-11-08T19:30:45.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.762+0000 7fef7ec90640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef78143a40 con 0x7fef7806cb40 2023-11-08T19:30:45.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.762+0000 7fef7ec90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780ca3e0 con 0x7fef7815d9f0 2023-11-08T19:30:45.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.762+0000 7fef7d22d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef7806cb40 0x7fef7806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef7d22d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef7806cb40 0x7fef7806cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45824/0 (socket says 172.21.15.119:45824) 2023-11-08T19:30:45.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef7d22d640 1 -- 172.21.15.119:0/4282701917 learned_addr learned my addr 172.21.15.119:0/4282701917 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef77fff640 1 --2- 172.21.15.119:0/4282701917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef7d22d640 1 -- 172.21.15.119:0/4282701917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 msgr2=0x7fef780cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef7d22d640 1 --2- 172.21.15.119:0/4282701917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef7d22d640 1 -- 172.21.15.119:0/4282701917 >> v1:172.21.15.188:6789/0 conn(0x7fef78142f80 legacy=0x7fef780c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.763+0000 7fef7d22d640 1 -- 172.21.15.119:0/4282701917 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7815df60 con 0x7fef7806cb40 2023-11-08T19:30:45.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.764+0000 7fef7d22d640 1 --2- 172.21.15.119:0/4282701917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef7806cb40 0x7fef7806cfd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fef70009e50 tx=0x7fef7002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdfcca1bc6a7c5d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.764+0000 7fef777fe640 1 -- 172.21.15.119:0/4282701917 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7002ca70 con 0x7fef7806cb40 2023-11-08T19:30:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.764+0000 7fef777fe640 1 -- 172.21.15.119:0/4282701917 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7002cbd0 con 0x7fef7806cb40 2023-11-08T19:30:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.764+0000 7fef777fe640 1 -- 172.21.15.119:0/4282701917 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef700343f0 con 0x7fef7806cb40 2023-11-08T19:30:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 -- 172.21.15.119:0/4282701917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef7806cb40 msgr2=0x7fef7806cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 --2- 172.21.15.119:0/4282701917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef7806cb40 0x7fef7806cfd0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fef70009e50 tx=0x7fef7002bed0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 -- 172.21.15.119:0/4282701917 shutdown_connections 2023-11-08T19:30:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 --2- 172.21.15.119:0/4282701917 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef7806cb40 0x7fef7806cfd0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 --2- 172.21.15.119:0/4282701917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 -- 172.21.15.119:0/4282701917 >> 172.21.15.119:0/4282701917 conn(0x7fef780cb610 msgr2=0x7fef780cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:45.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 -- 172.21.15.119:0/4282701917 shutdown_connections 2023-11-08T19:30:45.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.765+0000 7fef7ec90640 1 -- 172.21.15.119:0/4282701917 wait complete. 2023-11-08T19:30:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.767+0000 7fef7ec90640 1 Processor -- start 2023-11-08T19:30:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.767+0000 7fef7ec90640 1 -- start start 2023-11-08T19:30:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.768+0000 7fef7ec90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef7806cb40 0x7fef780b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.768+0000 7fef7ec90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 0x7fef780b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.768+0000 7fef7ec90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.768+0000 7fef7ec90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7815b0c0 con 0x7fef78142f80 2023-11-08T19:30:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.768+0000 7fef7ec90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780b4210 con 0x7fef7806cb40 2023-11-08T19:30:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.768+0000 7fef7ec90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef780b44f0 con 0x7fef7815d9f0 2023-11-08T19:30:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 0x7fef780b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 0x7fef780b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45840/0 (socket says 172.21.15.119:45840) 2023-11-08T19:30:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 -- 172.21.15.119:0/3421490106 learned_addr learned my addr 172.21.15.119:0/3421490106 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef7ca2c640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef7806cb40 0x7fef780b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef7d22d640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 -- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 msgr2=0x7fef780c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 -- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef7806cb40 msgr2=0x7fef780b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef7806cb40 0x7fef780b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef77fff640 1 -- 172.21.15.119:0/3421490106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70009b30 con 0x7fef78142f80 2023-11-08T19:30:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.769+0000 7fef7d22d640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.770+0000 7fef77fff640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 0x7fef780b39e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fef6400d5f0 tx=0x7fef6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:45.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.770+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef640125f0 con 0x7fef78142f80 2023-11-08T19:30:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.770+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef64012750 con 0x7fef78142f80 2023-11-08T19:30:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.770+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64012b00 con 0x7fef78142f80 2023-11-08T19:30:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.770+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef780b47d0 con 0x7fef78142f80 2023-11-08T19:30:45.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.770+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef780b3f20 con 0x7fef78142f80 2023-11-08T19:30:45.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.771+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef7815b200 con 0x7fef78142f80 2023-11-08T19:30:45.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.772+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef64012c60 con 0x7fef78142f80 2023-11-08T19:30:45.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.773+0000 7fef757fa640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:45.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.773+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fef6408a1b0 con 0x7fef78142f80 2023-11-08T19:30:45.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.773+0000 7fef7ca2c640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:45.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.774+0000 7fef7ca2c640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef5c06ae70 0x7fef5c06d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef7813e960 tx=0x7fef6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:45.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:45.777+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fef7815b200 con 0x7fef78142f80 2023-11-08T19:30:46.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.006+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef7815b200 con 0x7fef78142f80 2023-11-08T19:30:46.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.008+0000 7fef757fa640 1 -- 172.21.15.119:0/3421490106 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fef7815b200 con 0x7fef78142f80 2023-11-08T19:30:46.010 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:46.011 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":81,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:40.469042+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:46.011 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 81 2023-11-08T19:30:46.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef5c06ae70 msgr2=0x7fef5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef5c06ae70 0x7fef5c06d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fef7813e960 tx=0x7fef6c009040 comp rx=0 tx=0).stop 2023-11-08T19:30:46.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 msgr2=0x7fef780b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 0x7fef780b39e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fef6400d5f0 tx=0x7fef6400dab0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 shutdown_connections 2023-11-08T19:30:46.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef5c06ae70 0x7fef5c06d330 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef7815d9f0 0x7fef780c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef78142f80 0x7fef780b39e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 --2- 172.21.15.119:0/3421490106 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef7806cb40 0x7fef780b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 >> 172.21.15.119:0/3421490106 conn(0x7fef780cb610 msgr2=0x7fef780ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.010+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 shutdown_connections 2023-11-08T19:30:46.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.011+0000 7fef7ec90640 1 -- 172.21.15.119:0/3421490106 wait complete. 2023-11-08T19:30:46.031 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:30:46.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.242+0000 7f4d84f21640 1 Processor -- start 2023-11-08T19:30:46.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.242+0000 7f4d84f21640 1 -- start start 2023-11-08T19:30:46.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d84f21640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4d8015e110 0x7f4d800aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d84f21640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d800b2900 0x7f4d8006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d84f21640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d800cf610 con 0x7f4d8015e4f0 2023-11-08T19:30:46.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d84f21640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800cef90 con 0x7f4d800b2900 2023-11-08T19:30:46.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d84f21640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800ce9d0 con 0x7f4d8015e110 2023-11-08T19:30:46.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d7effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d800b2900 0x7f4d8006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d7effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4d800b2900 0x7f4d8006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45842/0 (socket says 172.21.15.119:45842) 2023-11-08T19:30:46.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d7effd640 1 -- 172.21.15.119:0/215515107 learned_addr learned my addr 172.21.15.119:0/215515107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:46.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.244+0000 7f4d7f7fe640 1 --2- 172.21.15.119:0/215515107 >> v2:172.21.15.188:3300/0 conn(0x7f4d8015e110 0x7f4d800aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.245+0000 7f4d7effd640 1 -- 172.21.15.119:0/215515107 >> v2:172.21.15.188:3300/0 conn(0x7f4d8015e110 msgr2=0x7f4d800aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:46.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.245+0000 7f4d7effd640 1 --2- 172.21.15.119:0/215515107 >> v2:172.21.15.188:3300/0 conn(0x7f4d8015e110 0x7f4d800aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.245+0000 7f4d7effd640 1 -- 172.21.15.119:0/215515107 >> v1:172.21.15.188:6789/0 conn(0x7f4d8015e4f0 legacy=0x7f4d800709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:46.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.245+0000 7f4d7effd640 1 -- 172.21.15.119:0/215515107 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800cecb0 con 0x7f4d800b2900 2023-11-08T19:30:46.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.246+0000 7f4d7effd640 1 --2- 172.21.15.119:0/215515107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d8006a120 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4d800cd560 tx=0x7f4d7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3aabaa38cea0e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.246+0000 7f4d7e7fc640 1 -- 172.21.15.119:0/215515107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74031870 con 0x7f4d800b2900 2023-11-08T19:30:46.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.246+0000 7f4d7e7fc640 1 -- 172.21.15.119:0/215515107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d740319d0 con 0x7f4d800b2900 2023-11-08T19:30:46.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.246+0000 7f4d7e7fc640 1 -- 172.21.15.119:0/215515107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74031d60 con 0x7f4d800b2900 2023-11-08T19:30:46.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 -- 172.21.15.119:0/215515107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 msgr2=0x7f4d8006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 --2- 172.21.15.119:0/215515107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d8006a120 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f4d800cd560 tx=0x7f4d7402b910 comp rx=0 tx=0).stop 2023-11-08T19:30:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 -- 172.21.15.119:0/215515107 shutdown_connections 2023-11-08T19:30:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 --2- 172.21.15.119:0/215515107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d8006a120 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 --2- 172.21.15.119:0/215515107 >> v2:172.21.15.188:3300/0 conn(0x7f4d8015e110 0x7f4d800aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 -- 172.21.15.119:0/215515107 >> 172.21.15.119:0/215515107 conn(0x7f4d800cfab0 msgr2=0x7f4d800cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 -- 172.21.15.119:0/215515107 shutdown_connections 2023-11-08T19:30:46.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.247+0000 7f4d84f21640 1 -- 172.21.15.119:0/215515107 wait complete. 2023-11-08T19:30:46.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.249+0000 7f4d84f21640 1 Processor -- start 2023-11-08T19:30:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.249+0000 7f4d84f21640 1 -- start start 2023-11-08T19:30:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d84f21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d801f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d84f21640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d8015e110 0x7f4d800c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d84f21640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8015e4f0 0x7f4d800afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d84f21640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800c8440 con 0x7f4d800b2900 2023-11-08T19:30:46.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d84f21640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b3eb0 con 0x7f4d8015e4f0 2023-11-08T19:30:46.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d84f21640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800b4190 con 0x7f4d8015e110 2023-11-08T19:30:46.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d801f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d801f3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45844/0 (socket says 172.21.15.119:45844) 2023-11-08T19:30:46.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 -- 172.21.15.119:0/2805856698 learned_addr learned my addr 172.21.15.119:0/2805856698 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 -- 172.21.15.119:0/2805856698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d8015e110 msgr2=0x7f4d800c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d8015e110 0x7f4d800c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 -- 172.21.15.119:0/2805856698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8015e4f0 msgr2=0x7f4d800afdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8015e4f0 0x7f4d800afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.257 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.250+0000 7f4d7f7fe640 1 -- 172.21.15.119:0/2805856698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740094d0 con 0x7f4d800b2900 2023-11-08T19:30:46.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.251+0000 7f4d7f7fe640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d801f3a50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000d800 tx=0x7f4d7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.251+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000b6c0 con 0x7f4d800b2900 2023-11-08T19:30:46.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.251+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7000b820 con 0x7f4d800b2900 2023-11-08T19:30:46.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.251+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d800b4470 con 0x7f4d800b2900 2023-11-08T19:30:46.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.251+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000bbb0 con 0x7f4d800b2900 2023-11-08T19:30:46.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.251+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d801f36d0 con 0x7f4d800b2900 2023-11-08T19:30:46.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.252+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d8004ece0 con 0x7f4d800b2900 2023-11-08T19:30:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.253+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4d7000bd10 con 0x7f4d800b2900 2023-11-08T19:30:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.254+0000 7f4d67fff640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6c06ae70 0x7f4d6c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.254+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f4d7008a280 con 0x7f4d800b2900 2023-11-08T19:30:46.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.254+0000 7f4d7effd640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6c06ae70 0x7f4d6c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.255+0000 7f4d7effd640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6c06ae70 0x7f4d6c06d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d74036000 tx=0x7f4d74030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:46.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.258+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4d70054c60 con 0x7f4d800b2900 2023-11-08T19:30:46.529 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.527+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f4d8015a760 con 0x7f4d800b2900 2023-11-08T19:30:46.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.916+0000 7f4d67fff640 1 -- 172.21.15.119:0/2805856698 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.7515 v82) v1 ==== 112+0+0 (secure 0 0 0) 0x7f4d70058280 con 0x7f4d800b2900 2023-11-08T19:30:46.919 INFO:teuthology.orchestra.run.smithi119.stderr:freezing mds.7515 2023-11-08T19:30:46.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.919+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6c06ae70 msgr2=0x7f4d6c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.919+0000 7f4d84f21640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6c06ae70 0x7f4d6c06d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4d74036000 tx=0x7f4d74030000 comp rx=0 tx=0).stop 2023-11-08T19:30:46.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.919+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 msgr2=0x7f4d801f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:46.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.919+0000 7f4d84f21640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d801f3a50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f4d7000d800 tx=0x7f4d7000dcc0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 shutdown_connections 2023-11-08T19:30:46.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4d6c06ae70 0x7f4d6c06d330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4d8015e4f0 0x7f4d800afdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4d8015e110 0x7f4d800c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 --2- 172.21.15.119:0/2805856698 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4d800b2900 0x7f4d801f3a50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:46.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 >> 172.21.15.119:0/2805856698 conn(0x7f4d800cfab0 msgr2=0x7f4d800708b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:46.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 shutdown_connections 2023-11-08T19:30:46.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:46.920+0000 7f4d84f21640 1 -- 172.21.15.119:0/2805856698 wait complete. 2023-11-08T19:30:46.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json config set mds_kill_mdstable_at 6 2023-11-08T19:30:47.170 INFO:teuthology.orchestra.run.smithi119.stdout:{"success":"mds_kill_mdstable_at = '6' (not observed, change may require restart) "} 2023-11-08T19:30:47.171 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '6' (not observed, change may require restart) " } 2023-11-08T19:30:47.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:30:47.172 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s16) 2023-11-08T19:30:47.178 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:30:47.422 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:30:47.423 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:30:47.423 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:47.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.637+0000 7faec9b8c640 1 Processor -- start 2023-11-08T19:30:47.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.637+0000 7faec9b8c640 1 -- start start 2023-11-08T19:30:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.638+0000 7faec9b8c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faec415b0d0 0x7faec40ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.638+0000 7faec9b8c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faec40c98c0 0x7faec40c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.638+0000 7faec9b8c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec406cff0 con 0x7faec415b900 2023-11-08T19:30:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.638+0000 7faec9b8c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec406d2d0 con 0x7faec415b0d0 2023-11-08T19:30:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.638+0000 7faec9b8c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec406bd20 con 0x7faec40c98c0 2023-11-08T19:30:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faec415b0d0 0x7faec40ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7faec415b0d0 0x7faec40ad7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45846/0 (socket says 172.21.15.119:45846) 2023-11-08T19:30:47.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 -- 172.21.15.119:0/2208514409 learned_addr learned my addr 172.21.15.119:0/2208514409 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:47.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec938b640 1 --2- 172.21.15.119:0/2208514409 >> v2:172.21.15.188:3300/0 conn(0x7faec40c98c0 0x7faec40c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 -- 172.21.15.119:0/2208514409 >> v2:172.21.15.188:3300/0 conn(0x7faec40c98c0 msgr2=0x7faec40c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:47.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 --2- 172.21.15.119:0/2208514409 >> v2:172.21.15.188:3300/0 conn(0x7faec40c98c0 0x7faec40c9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 -- 172.21.15.119:0/2208514409 >> v1:172.21.15.188:6789/0 conn(0x7faec415b900 legacy=0x7faec4070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:47.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec8b8a640 1 -- 172.21.15.119:0/2208514409 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec406d760 con 0x7faec415b0d0 2023-11-08T19:30:47.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.639+0000 7faec938b640 1 --2- 172.21.15.119:0/2208514409 >> v2:172.21.15.188:3300/0 conn(0x7faec40c98c0 0x7faec40c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:30:47.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.640+0000 7faec8b8a640 1 --2- 172.21.15.119:0/2208514409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40ad7d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7faec415eaa0 tx=0x7faeb402e060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94360d0c804f8eaf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.641+0000 7faec37fe640 1 -- 172.21.15.119:0/2208514409 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb402ea00 con 0x7faec415b0d0 2023-11-08T19:30:47.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.641+0000 7faec37fe640 1 -- 172.21.15.119:0/2208514409 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb402eb60 con 0x7faec415b0d0 2023-11-08T19:30:47.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.641+0000 7faec37fe640 1 -- 172.21.15.119:0/2208514409 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb4009600 con 0x7faec415b0d0 2023-11-08T19:30:47.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.641+0000 7faec9b8c640 1 -- 172.21.15.119:0/2208514409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 msgr2=0x7faec40ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.641+0000 7faec9b8c640 1 --2- 172.21.15.119:0/2208514409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40ad7d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7faec415eaa0 tx=0x7faeb402e060 comp rx=0 tx=0).stop 2023-11-08T19:30:47.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.642+0000 7faec9b8c640 1 -- 172.21.15.119:0/2208514409 shutdown_connections 2023-11-08T19:30:47.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.642+0000 7faec9b8c640 1 --2- 172.21.15.119:0/2208514409 >> v2:172.21.15.188:3300/0 conn(0x7faec40c98c0 0x7faec40c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.642+0000 7faec9b8c640 1 --2- 172.21.15.119:0/2208514409 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40ad7d0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.642+0000 7faec9b8c640 1 -- 172.21.15.119:0/2208514409 >> 172.21.15.119:0/2208514409 conn(0x7faec40b0560 msgr2=0x7faec40b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:47.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.642+0000 7faec9b8c640 1 -- 172.21.15.119:0/2208514409 shutdown_connections 2023-11-08T19:30:47.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.642+0000 7faec9b8c640 1 -- 172.21.15.119:0/2208514409 wait complete. 2023-11-08T19:30:47.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.643+0000 7faec9b8c640 1 Processor -- start 2023-11-08T19:30:47.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.644+0000 7faec9b8c640 1 -- start start 2023-11-08T19:30:47.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec9b8c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec40c98c0 0x7faec40c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec9b8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec9b8c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec415b900 0x7faec414aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec9b8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40c7470 con 0x7faec415b0d0 2023-11-08T19:30:47.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec9b8c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec40b23c0 con 0x7faec415b900 2023-11-08T19:30:47.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec9b8c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec40c76c0 con 0x7faec40c98c0 2023-11-08T19:30:47.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40acb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:45862/0 (socket says 172.21.15.119:45862) 2023-11-08T19:30:47.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 -- 172.21.15.119:0/13623388 learned_addr learned my addr 172.21.15.119:0/13623388 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:47.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec938b640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec415b900 0x7faec414aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.651 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec8b8a640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec40c98c0 0x7faec40c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 -- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec40c98c0 msgr2=0x7faec40c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec40c98c0 0x7faec40c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 -- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec415b900 msgr2=0x7faec414aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec415b900 0x7faec414aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.652 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.645+0000 7faec3fff640 1 -- 172.21.15.119:0/13623388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb400bd40 con 0x7faec415b0d0 2023-11-08T19:30:47.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.646+0000 7faec3fff640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40acb40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c410 tx=0x7faeac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.646+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0035f0 con 0x7faec415b0d0 2023-11-08T19:30:47.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.646+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac003750 con 0x7faec415b0d0 2023-11-08T19:30:47.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.646+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec40c79a0 con 0x7faec415b0d0 2023-11-08T19:30:47.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.646+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac003b40 con 0x7faec415b0d0 2023-11-08T19:30:47.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.646+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec40c86e0 con 0x7faec415b0d0 2023-11-08T19:30:47.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.647+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec40c7ec0 con 0x7faec415b0d0 2023-11-08T19:30:47.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.648+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faeac003ca0 con 0x7faec415b0d0 2023-11-08T19:30:47.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.649+0000 7faec17fa640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faea406ae70 0x7faea406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:47.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.650+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7faeac089380 con 0x7faec415b0d0 2023-11-08T19:30:47.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.650+0000 7faec8b8a640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faea406ae70 0x7faea406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:47.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.650+0000 7faec8b8a640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faea406ae70 0x7faea406d330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faeb4037000 tx=0x7faeb4032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:47.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.654+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faec40c7ec0 con 0x7faec415b0d0 2023-11-08T19:30:47.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.883+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faec40c7ec0 con 0x7faec415b0d0 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.884+0000 7faec17fa640 1 -- 172.21.15.119:0/13623388 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4944 (secure 0 0 0) 0x7faec40c7ec0 con 0x7faec415b0d0 2023-11-08T19:30:47.886 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:47.887 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:46.530020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:47.887 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faea406ae70 msgr2=0x7faea406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faea406ae70 0x7faea406d330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7faeb4037000 tx=0x7faeb4032000 comp rx=0 tx=0).stop 2023-11-08T19:30:47.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 msgr2=0x7faec40acb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40acb40 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c410 tx=0x7faeac00c8d0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 shutdown_connections 2023-11-08T19:30:47.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7faea406ae70 0x7faea406d330 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faec415b900 0x7faec414aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7faec415b0d0 0x7faec40acb40 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 --2- 172.21.15.119:0/13623388 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faec40c98c0 0x7faec40c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:47.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 >> 172.21.15.119:0/13623388 conn(0x7faec40b0560 msgr2=0x7faec40cdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:47.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 shutdown_connections 2023-11-08T19:30:47.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:47.887+0000 7faec9b8c640 1 -- 172.21.15.119:0/13623388 wait complete. 2023-11-08T19:30:47.908 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:49.247 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f8b3b984640 time 2023-11-08T19:30:49.247199+0000 2023-11-08T19:30:49.248 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:30:49.248 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.249 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8b4756efaf] 2023-11-08T19:30:49.249 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.249 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.249 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.250 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.250 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.250 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.250 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.251 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.251 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:30:49.247+0000 7f8b3b984640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f8b3b984640 time 2023-11-08T19:30:49.247199+0000 2023-11-08T19:30:49.251 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:30:49.251 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.252 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.252 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8b4756efaf] 2023-11-08T19:30:49.252 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.252 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.252 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.253 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.253 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.253 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.253 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.254 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.254 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.254 INFO:tasks.ceph.mds.a.smithi119.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:30:49.254 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f8b3b984640 thread_name:MR_Finisher 2023-11-08T19:30:49.255 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.255 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f8b46c54db0] 2023-11-08T19:30:49.255 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f8b46ca154c] 2023-11-08T19:30:49.256 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:30:49.256 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:30:49.256 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8b4756f009] 2023-11-08T19:30:49.256 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.256 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.257 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.257 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.257 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.257 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.257 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.258 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.258 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:30:49.250+0000 7f8b3b984640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:49.258 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f8b3b984640 thread_name:MR_Finisher 2023-11-08T19:30:49.258 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.259 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.259 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f8b46c54db0] 2023-11-08T19:30:49.259 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f8b46ca154c] 2023-11-08T19:30:49.259 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:30:49.259 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:30:49.260 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8b4756f009] 2023-11-08T19:30:49.260 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.260 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.260 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.261 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.261 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.261 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.261 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.261 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.262 INFO:tasks.ceph.mds.a.smithi119.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:49.262 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.276 INFO:tasks.ceph.mds.a.smithi119.stderr: -324> 2023-11-08T19:30:45.072+0000 7f8b3f18b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:49.276 INFO:tasks.ceph.mds.a.smithi119.stderr: -322> 2023-11-08T19:30:45.072+0000 7f8b3f18b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:49.278 INFO:tasks.ceph.mds.a.smithi119.stderr: -1> 2023-11-08T19:30:49.247+0000 7f8b3b984640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f8b3b984640 time 2023-11-08T19:30:49.247199+0000 2023-11-08T19:30:49.278 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:30:49.278 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.278 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.279 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8b4756efaf] 2023-11-08T19:30:49.279 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.279 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.279 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.280 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.280 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.280 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.280 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.281 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.281 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.281 INFO:tasks.ceph.mds.a.smithi119.stderr: 0> 2023-11-08T19:30:49.250+0000 7f8b3b984640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:49.281 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f8b3b984640 thread_name:MR_Finisher 2023-11-08T19:30:49.282 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.282 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.282 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f8b46c54db0] 2023-11-08T19:30:49.282 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f8b46ca154c] 2023-11-08T19:30:49.282 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:30:49.283 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:30:49.283 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8b4756f009] 2023-11-08T19:30:49.283 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.283 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.283 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.284 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.284 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.284 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.284 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.284 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.285 INFO:tasks.ceph.mds.a.smithi119.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:49.285 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.294 INFO:tasks.ceph.mds.a.smithi119.stderr: -324> 2023-11-08T19:30:45.072+0000 7f8b3f18b640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:30:49.295 INFO:tasks.ceph.mds.a.smithi119.stderr: -322> 2023-11-08T19:30:45.072+0000 7f8b3f18b640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:30:49.296 INFO:tasks.ceph.mds.a.smithi119.stderr: -1> 2023-11-08T19:30:49.247+0000 7f8b3b984640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_commit_logged(ceph::cref_t&)' thread 7f8b3b984640 time 2023-11-08T19:30:49.247199+0000 2023-11-08T19:30:49.296 INFO:tasks.ceph.mds.a.smithi119.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 171: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 6) 2023-11-08T19:30:49.296 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.297 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.297 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7f8b4756efaf] 2023-11-08T19:30:49.297 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.297 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.297 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.298 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.298 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.298 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.298 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.299 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.299 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.299 INFO:tasks.ceph.mds.a.smithi119.stderr: 0> 2023-11-08T19:30:49.250+0000 7f8b3b984640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:30:49.299 INFO:tasks.ceph.mds.a.smithi119.stderr: in thread 7f8b3b984640 thread_name:MR_Finisher 2023-11-08T19:30:49.299 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.300 INFO:tasks.ceph.mds.a.smithi119.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:30:49.300 INFO:tasks.ceph.mds.a.smithi119.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7f8b46c54db0] 2023-11-08T19:30:49.300 INFO:tasks.ceph.mds.a.smithi119.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7f8b46ca154c] 2023-11-08T19:30:49.300 INFO:tasks.ceph.mds.a.smithi119.stderr: 3: raise() 2023-11-08T19:30:49.301 INFO:tasks.ceph.mds.a.smithi119.stderr: 4: abort() 2023-11-08T19:30:49.301 INFO:tasks.ceph.mds.a.smithi119.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7f8b4756f009] 2023-11-08T19:30:49.301 INFO:tasks.ceph.mds.a.smithi119.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7f8b4756f16d] 2023-11-08T19:30:49.301 INFO:tasks.ceph.mds.a.smithi119.stderr: 7: (MDSTableServer::_commit_logged(boost::intrusive_ptr const&)+0x41b) [0x557c2745821b] 2023-11-08T19:30:49.301 INFO:tasks.ceph.mds.a.smithi119.stderr: 8: (MDSContext::complete(int)+0x5c) [0x557c27487bcc] 2023-11-08T19:30:49.302 INFO:tasks.ceph.mds.a.smithi119.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x557c27490b0c] 2023-11-08T19:30:49.302 INFO:tasks.ceph.mds.a.smithi119.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x557c27490c63] 2023-11-08T19:30:49.302 INFO:tasks.ceph.mds.a.smithi119.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7f8b476164a5] 2023-11-08T19:30:49.302 INFO:tasks.ceph.mds.a.smithi119.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7f8b46c9f802] 2023-11-08T19:30:49.303 INFO:tasks.ceph.mds.a.smithi119.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7f8b46c3f450] 2023-11-08T19:30:49.303 INFO:tasks.ceph.mds.a.smithi119.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:30:49.303 INFO:tasks.ceph.mds.a.smithi119.stderr: 2023-11-08T19:30:49.453 INFO:tasks.ceph.mds.a.smithi119.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:30:50.269 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:50.279 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:30:52.296 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-08T19:30:52.909 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:53.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.133+0000 7f220e89b640 1 Processor -- start 2023-11-08T19:30:53.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.133+0000 7f220e89b640 1 -- start start 2023-11-08T19:30:53.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220e89b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f220815f4c0 0x7f220806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220e89b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f220806a120 0x7f220806a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220e89b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f220806aa70 con 0x7f22081447a0 2023-11-08T19:30:53.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220e89b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f220815f930 con 0x7f220806a120 2023-11-08T19:30:53.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220e89b640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22080c3a50 con 0x7f220815f4c0 2023-11-08T19:30:53.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220ce38640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f220806a120 0x7f220806a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220ce38640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f220806a120 0x7f220806a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38714/0 (socket says 172.21.15.119:38714) 2023-11-08T19:30:53.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220ce38640 1 -- 172.21.15.119:0/16214762 learned_addr learned my addr 172.21.15.119:0/16214762 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:53.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f2207fff640 1 --1- 172.21.15.119:0/16214762 >> v1:172.21.15.119:6789/0 conn(0x7f22081447a0 0x7f22080cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34330/0 (socket says 172.21.15.119:34330) 2023-11-08T19:30:53.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f220ce38640 1 -- 172.21.15.119:0/16214762 >> v2:172.21.15.188:3300/0 conn(0x7f220815f4c0 msgr2=0x7f220806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:53.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.135+0000 7f22077fe640 1 --2- 172.21.15.119:0/16214762 >> v2:172.21.15.188:3300/0 conn(0x7f220815f4c0 0x7f220806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.136+0000 7f220ce38640 1 --2- 172.21.15.119:0/16214762 >> v2:172.21.15.188:3300/0 conn(0x7f220815f4c0 0x7f220806f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.136+0000 7f220ce38640 1 -- 172.21.15.119:0/16214762 >> v1:172.21.15.119:6789/0 conn(0x7f22081447a0 legacy=0x7f22080cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.136+0000 7f220ce38640 1 -- 172.21.15.119:0/16214762 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2208144b80 con 0x7f220806a120 2023-11-08T19:30:53.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.137+0000 7f220ce38640 1 --2- 172.21.15.119:0/16214762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220806a120 0x7f220806a530 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f220806df20 tx=0x7f220002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f304f2fbc7ee2b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.137+0000 7f2206ffd640 1 -- 172.21.15.119:0/16214762 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200030860 con 0x7f220806a120 2023-11-08T19:30:53.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.137+0000 7f2206ffd640 1 -- 172.21.15.119:0/16214762 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22000309c0 con 0x7f220806a120 2023-11-08T19:30:53.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.137+0000 7f2206ffd640 1 -- 172.21.15.119:0/16214762 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200030d50 con 0x7f220806a120 2023-11-08T19:30:53.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 -- 172.21.15.119:0/16214762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220806a120 msgr2=0x7f220806a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 --2- 172.21.15.119:0/16214762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220806a120 0x7f220806a530 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f220806df20 tx=0x7f220002b9a0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 -- 172.21.15.119:0/16214762 shutdown_connections 2023-11-08T19:30:53.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 --2- 172.21.15.119:0/16214762 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f220806a120 0x7f220806a530 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 --2- 172.21.15.119:0/16214762 >> v2:172.21.15.188:3300/0 conn(0x7f220815f4c0 0x7f220806f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 -- 172.21.15.119:0/16214762 >> 172.21.15.119:0/16214762 conn(0x7f22080cb610 msgr2=0x7f22080cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:53.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 -- 172.21.15.119:0/16214762 shutdown_connections 2023-11-08T19:30:53.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.138+0000 7f220e89b640 1 -- 172.21.15.119:0/16214762 wait complete. 2023-11-08T19:30:53.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.140+0000 7f220e89b640 1 Processor -- start 2023-11-08T19:30:53.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.140+0000 7f220e89b640 1 -- start start 2023-11-08T19:30:53.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.141+0000 7f220e89b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220806a120 0x7f22081f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.141+0000 7f220e89b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 0x7f22080cdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.141+0000 7f220e89b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220815f4c0 0x7f22081fa340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.141+0000 7f220e89b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080ce2e0 con 0x7f22081447a0 2023-11-08T19:30:53.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.141+0000 7f220e89b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22080c7500 con 0x7f220815f4c0 2023-11-08T19:30:53.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.141+0000 7f220e89b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f220815bed0 con 0x7f220806a120 2023-11-08T19:30:53.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 0x7f22080cdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 0x7f22080cdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:38722/0 (socket says 172.21.15.119:38722) 2023-11-08T19:30:53.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 -- 172.21.15.119:0/1524113091 learned_addr learned my addr 172.21.15.119:0/1524113091 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:53.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f2207fff640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220806a120 0x7f22081f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 -- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220806a120 msgr2=0x7f22081f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220806a120 0x7f22081f74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 -- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220815f4c0 msgr2=0x7f22081fa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220815f4c0 0x7f22081fa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 -- 172.21.15.119:0/1524113091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000094d0 con 0x7f22081447a0 2023-11-08T19:30:53.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.142+0000 7f22077fe640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 0x7f22080cdda0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f21f800ce10 tx=0x7f21f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.143+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8003b90 con 0x7f22081447a0 2023-11-08T19:30:53.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.143+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f8003cf0 con 0x7f22081447a0 2023-11-08T19:30:53.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.143+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22080c6ed0 con 0x7f22081447a0 2023-11-08T19:30:53.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.143+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f8003ea0 con 0x7f22081447a0 2023-11-08T19:30:53.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.143+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22080ce4e0 con 0x7f22081447a0 2023-11-08T19:30:53.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.144+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2208074430 con 0x7f22081447a0 2023-11-08T19:30:53.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.145+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21f8004240 con 0x7f22081447a0 2023-11-08T19:30:53.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.146+0000 7f2204ff9640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21e806ae70 0x7f21e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:53.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.146+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f21f8012070 con 0x7f22081447a0 2023-11-08T19:30:53.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.146+0000 7f2207fff640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21e806ae70 0x7f21e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:53.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.147+0000 7f2207fff640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21e806ae70 0x7f21e806d330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000f60 tx=0x7f21f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:53.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.152+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21f80550c0 con 0x7f22081447a0 2023-11-08T19:30:53.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.381+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f220813fc50 con 0x7f22081447a0 2023-11-08T19:30:53.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.382+0000 7f2204ff9640 1 -- 172.21.15.119:0/1524113091 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f21f80586e0 con 0x7f22081447a0 2023-11-08T19:30:53.383 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:53.384 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:46.530020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:53.384 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2023-11-08T19:30:53.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21e806ae70 msgr2=0x7f21e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21e806ae70 0x7f21e806d330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f21f0000f60 tx=0x7f21f0009040 comp rx=0 tx=0).stop 2023-11-08T19:30:53.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 msgr2=0x7f22080cdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:53.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 0x7f22080cdda0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f21f800ce10 tx=0x7f21f800a660 comp rx=0 tx=0).stop 2023-11-08T19:30:53.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 shutdown_connections 2023-11-08T19:30:53.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f21e806ae70 0x7f21e806d330 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f220815f4c0 0x7f22081fa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f22081447a0 0x7f22080cdda0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 --2- 172.21.15.119:0/1524113091 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f220806a120 0x7f22081f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:53.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 >> 172.21.15.119:0/1524113091 conn(0x7f22080cb610 msgr2=0x7f2208143620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:53.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 shutdown_connections 2023-11-08T19:30:53.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:53.384+0000 7f220e89b640 1 -- 172.21.15.119:0/1524113091 wait complete. 2023-11-08T19:30:53.405 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:30:58.407 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:30:58.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.629+0000 7fd28e41a640 1 Processor -- start 2023-11-08T19:30:58.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.629+0000 7fd28e41a640 1 -- start start 2023-11-08T19:30:58.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.630+0000 7fd28e41a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd28806d520 0x7fd28806cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.630+0000 7fd28e41a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2880cc640 0x7fd2880ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.630+0000 7fd28e41a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd28815a870 con 0x7fd28806bee0 2023-11-08T19:30:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.630+0000 7fd28e41a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28806cbc0 con 0x7fd2880cc640 2023-11-08T19:30:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.630+0000 7fd28e41a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28815cfa0 con 0x7fd28806d520 2023-11-08T19:30:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2880cc640 0x7fd2880ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd2880cc640 0x7fd2880ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37300/0 (socket says 172.21.15.119:37300) 2023-11-08T19:30:58.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 -- 172.21.15.119:0/3998487533 learned_addr learned my addr 172.21.15.119:0/3998487533 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:58.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28cc17640 1 --2- 172.21.15.119:0/3998487533 >> v2:172.21.15.188:3300/0 conn(0x7fd28806d520 0x7fd28806cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 -- 172.21.15.119:0/3998487533 >> v2:172.21.15.188:3300/0 conn(0x7fd28806d520 msgr2=0x7fd28806cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:30:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 --2- 172.21.15.119:0/3998487533 >> v2:172.21.15.188:3300/0 conn(0x7fd28806d520 0x7fd28806cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 -- 172.21.15.119:0/3998487533 >> v1:172.21.15.119:6789/0 conn(0x7fd28806bee0 legacy=0x7fd28806c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd28dc19640 1 -- 172.21.15.119:0/3998487533 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2880c9ad0 con 0x7fd2880cc640 2023-11-08T19:30:58.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.631+0000 7fd27bfff640 1 -- 172.21.15.119:0/3998487533 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779890200 0 0) 0x7fd284001110 con 0x7fd28806bee0 2023-11-08T19:30:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28dc19640 1 --2- 172.21.15.119:0/3998487533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880ccaf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd28806f240 tx=0x7fd28002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cada7b7c7071943 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd27bfff640 1 -- 172.21.15.119:0/3998487533 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28002cb10 con 0x7fd2880cc640 2023-11-08T19:30:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd27bfff640 1 -- 172.21.15.119:0/3998487533 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28002cc70 con 0x7fd2880cc640 2023-11-08T19:30:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd27bfff640 1 -- 172.21.15.119:0/3998487533 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800343f0 con 0x7fd2880cc640 2023-11-08T19:30:58.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28e41a640 1 -- 172.21.15.119:0/3998487533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 msgr2=0x7fd2880ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28e41a640 1 --2- 172.21.15.119:0/3998487533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880ccaf0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fd28806f240 tx=0x7fd28002bf70 comp rx=0 tx=0).stop 2023-11-08T19:30:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28e41a640 1 -- 172.21.15.119:0/3998487533 shutdown_connections 2023-11-08T19:30:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28e41a640 1 --2- 172.21.15.119:0/3998487533 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880ccaf0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28e41a640 1 --2- 172.21.15.119:0/3998487533 >> v2:172.21.15.188:3300/0 conn(0x7fd28806d520 0x7fd28806cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.633+0000 7fd28e41a640 1 -- 172.21.15.119:0/3998487533 >> 172.21.15.119:0/3998487533 conn(0x7fd2880b0560 msgr2=0x7fd2880b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.634+0000 7fd28e41a640 1 -- 172.21.15.119:0/3998487533 shutdown_connections 2023-11-08T19:30:58.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.634+0000 7fd28e41a640 1 -- 172.21.15.119:0/3998487533 wait complete. 2023-11-08T19:30:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.635+0000 7fd28e41a640 1 Processor -- start 2023-11-08T19:30:58.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.635+0000 7fd28e41a640 1 -- start start 2023-11-08T19:30:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.636+0000 7fd28e41a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28806bee0 0x7fd2880c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.636+0000 7fd28e41a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd28806d520 0x7fd2880c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.636+0000 7fd28e41a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.636+0000 7fd28e41a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880c3d60 con 0x7fd2880cc640 2023-11-08T19:30:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.636+0000 7fd28e41a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880c4040 con 0x7fd28806d520 2023-11-08T19:30:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.636+0000 7fd28e41a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2880c4320 con 0x7fd28806bee0 2023-11-08T19:30:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37302/0 (socket says 172.21.15.119:37302) 2023-11-08T19:30:58.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 -- 172.21.15.119:0/711265320 learned_addr learned my addr 172.21.15.119:0/711265320 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:30:58.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 -- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28806bee0 msgr2=0x7fd2880c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28806bee0 0x7fd2880c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 -- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd28806d520 msgr2=0x7fd2880c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28cc17640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd28806d520 0x7fd2880c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd28806d520 0x7fd2880c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 -- 172.21.15.119:0/711265320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd280009b30 con 0x7fd2880cc640 2023-11-08T19:30:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.637+0000 7fd28dc19640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880c8a50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd280036040 tx=0x7fd280004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.638+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280009510 con 0x7fd2880cc640 2023-11-08T19:30:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.638+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28003e450 con 0x7fd2880cc640 2023-11-08T19:30:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.638+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28003e800 con 0x7fd2880cc640 2023-11-08T19:30:58.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.638+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2880c4600 con 0x7fd2880cc640 2023-11-08T19:30:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.638+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2880c5d00 con 0x7fd2880cc640 2023-11-08T19:30:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.639+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2880c4b20 con 0x7fd2880cc640 2023-11-08T19:30:58.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.640+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd28003e960 con 0x7fd2880cc640 2023-11-08T19:30:58.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.641+0000 7fd279ffb640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd26c06aea0 0x7fd26c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:30:58.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.641+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fd2800acb90 con 0x7fd2880cc640 2023-11-08T19:30:58.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.641+0000 7fd28d418640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd26c06aea0 0x7fd26c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:30:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.642+0000 7fd28d418640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd26c06aea0 0x7fd26c06d360 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd284007fd0 tx=0x7fd2840077c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:30:58.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.645+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd2880c4b20 con 0x7fd2880cc640 2023-11-08T19:30:58.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.883+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2880c4b20 con 0x7fd2880cc640 2023-11-08T19:30:58.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.885+0000 7fd279ffb640 1 -- 172.21.15.119:0/711265320 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fd280077590 con 0x7fd2880cc640 2023-11-08T19:30:58.886 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:30:58.887 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":82,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:30:46.530020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:30:58.887 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 82 2023-11-08T19:30:58.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.887+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd26c06aea0 msgr2=0x7fd26c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.887+0000 7fd28e41a640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd26c06aea0 0x7fd26c06d360 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd284007fd0 tx=0x7fd2840077c0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.887+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 msgr2=0x7fd2880c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:30:58.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.887+0000 7fd28e41a640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880c8a50 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd280036040 tx=0x7fd280004a40 comp rx=0 tx=0).stop 2023-11-08T19:30:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 shutdown_connections 2023-11-08T19:30:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd26c06aea0 0x7fd26c06d360 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd2880cc640 0x7fd2880c8a50 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd28806d520 0x7fd2880c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 --2- 172.21.15.119:0/711265320 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd28806bee0 0x7fd2880c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:30:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 >> 172.21.15.119:0/711265320 conn(0x7fd2880b0560 msgr2=0x7fd28815c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:30:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 shutdown_connections 2023-11-08T19:30:58.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:30:58.888+0000 7fd28e41a640 1 -- 172.21.15.119:0/711265320 wait complete. 2023-11-08T19:30:58.908 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~7s 2023-11-08T19:30:58.909 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:31:03.910 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.131+0000 7f4a53034640 1 Processor -- start 2023-11-08T19:31:04.133 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.131+0000 7f4a53034640 1 -- start start 2023-11-08T19:31:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a53034640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c15d9f0 0x7f4a4c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a53034640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a4c06cb40 0x7f4a4c06cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a53034640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a4c0c4960 con 0x7f4a4c142f80 2023-11-08T19:31:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a53034640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c143a40 con 0x7f4a4c06cb40 2023-11-08T19:31:04.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a53034640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0ca3e0 con 0x7f4a4c15d9f0 2023-11-08T19:31:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a515d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a4c06cb40 0x7f4a4c06cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a515d1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4a4c06cb40 0x7f4a4c06cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37306/0 (socket says 172.21.15.119:37306) 2023-11-08T19:31:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.133+0000 7f4a515d1640 1 -- 172.21.15.119:0/3787430524 learned_addr learned my addr 172.21.15.119:0/3787430524 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.134+0000 7f4a515d1640 1 -- 172.21.15.119:0/3787430524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c15d9f0 msgr2=0x7f4a4c0cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.134+0000 7f4a515d1640 1 --2- 172.21.15.119:0/3787430524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c15d9f0 0x7f4a4c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.134+0000 7f4a515d1640 1 -- 172.21.15.119:0/3787430524 >> v1:172.21.15.188:6789/0 conn(0x7f4a4c142f80 legacy=0x7f4a4c0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.134+0000 7f4a515d1640 1 -- 172.21.15.119:0/3787430524 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c15df60 con 0x7f4a4c06cb40 2023-11-08T19:31:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a515d1640 1 --2- 172.21.15.119:0/3787430524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c06cfd0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400bb10 tx=0x7f4a4402e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bb5a3b1d54ff71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a4b7fe640 1 -- 172.21.15.119:0/3787430524 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a4402ede0 con 0x7f4a4c06cb40 2023-11-08T19:31:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a4b7fe640 1 -- 172.21.15.119:0/3787430524 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a44036440 con 0x7f4a4c06cb40 2023-11-08T19:31:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a4b7fe640 1 -- 172.21.15.119:0/3787430524 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a440367d0 con 0x7f4a4c06cb40 2023-11-08T19:31:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a53034640 1 -- 172.21.15.119:0/3787430524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 msgr2=0x7f4a4c06cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a53034640 1 --2- 172.21.15.119:0/3787430524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c06cfd0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400bb10 tx=0x7f4a4402e030 comp rx=0 tx=0).stop 2023-11-08T19:31:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a53034640 1 -- 172.21.15.119:0/3787430524 shutdown_connections 2023-11-08T19:31:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a53034640 1 --2- 172.21.15.119:0/3787430524 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c06cfd0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.135+0000 7f4a53034640 1 --2- 172.21.15.119:0/3787430524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c15d9f0 0x7f4a4c0cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.136+0000 7f4a53034640 1 -- 172.21.15.119:0/3787430524 >> 172.21.15.119:0/3787430524 conn(0x7f4a4c0cb610 msgr2=0x7f4a4c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.136+0000 7f4a53034640 1 -- 172.21.15.119:0/3787430524 shutdown_connections 2023-11-08T19:31:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.136+0000 7f4a53034640 1 -- 172.21.15.119:0/3787430524 wait complete. 2023-11-08T19:31:04.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.137+0000 7f4a53034640 1 Processor -- start 2023-11-08T19:31:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.137+0000 7f4a53034640 1 -- start start 2023-11-08T19:31:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.138+0000 7f4a53034640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c1f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a53034640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c142f80 0x7f4a4c1f5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a53034640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c15d9f0 0x7f4a4c0cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a53034640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0c6c60 con 0x7f4a4c06cb40 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a53034640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0c6f40 con 0x7f4a4c15d9f0 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a53034640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0c7220 con 0x7f4a4c142f80 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c1f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c1f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37322/0 (socket says 172.21.15.119:37322) 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 -- 172.21.15.119:0/3865146342 learned_addr learned my addr 172.21.15.119:0/3865146342 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 -- 172.21.15.119:0/3865146342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c142f80 msgr2=0x7f4a4c1f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c142f80 0x7f4a4c1f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 -- 172.21.15.119:0/3865146342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c15d9f0 msgr2=0x7f4a4c0cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c15d9f0 0x7f4a4c0cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 -- 172.21.15.119:0/3865146342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4400b770 con 0x7f4a4c06cb40 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.139+0000 7f4a50dd0640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c1f3790 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00ba50 tx=0x7f4a3c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.140+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00cc10 con 0x7f4a4c06cb40 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.140+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00cd70 con 0x7f4a4c06cb40 2023-11-08T19:31:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.140+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c014680 con 0x7f4a4c06cb40 2023-11-08T19:31:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.140+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c0c7500 con 0x7f4a4c06cb40 2023-11-08T19:31:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.140+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1f5870 con 0x7f4a4c06cb40 2023-11-08T19:31:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.141+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c0c7740 con 0x7f4a4c06cb40 2023-11-08T19:31:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.143+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a3c01d3b0 con 0x7f4a4c06cb40 2023-11-08T19:31:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.143+0000 7f4a497fa640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2c06ae70 0x7f4a2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.144+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7f4a3c089c80 con 0x7f4a4c06cb40 2023-11-08T19:31:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.144+0000 7f4a4bfff640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2c06ae70 0x7f4a2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.144+0000 7f4a4bfff640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2c06ae70 0x7f4a2c06d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0717e0 tx=0x7f4a34006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.149+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4a4c0c7740 con 0x7f4a4c06cb40 2023-11-08T19:31:04.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.374+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4a4c0c7740 con 0x7f4a4c06cb40 2023-11-08T19:31:04.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.375+0000 7f4a497fa640 1 -- 172.21.15.119:0/3865146342 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+4992 (secure 0 0 0) 0x7f4a4c0c7740 con 0x7f4a4c06cb40 2023-11-08T19:31:04.377 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:04.377 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":83,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5210,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":83,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:03.723927+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":100,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7515,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7515":{"gid":7515,"name":"a","rank":0,"incarnation":77,"state":"up:active","state_seq":12,"addr":"172.21.15.119:6837/1517894744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1517894744},{"type":"v1","addr":"172.21.15.119:6837","nonce":1517894744}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:31:03.723926+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:04.377 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 83 2023-11-08T19:31:04.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2c06ae70 msgr2=0x7f4a2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2c06ae70 0x7f4a2c06d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c0717e0 tx=0x7f4a34006c90 comp rx=0 tx=0).stop 2023-11-08T19:31:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 msgr2=0x7f4a4c1f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c1f3790 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00ba50 tx=0x7f4a3c00bf10 comp rx=0 tx=0).stop 2023-11-08T19:31:04.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 shutdown_connections 2023-11-08T19:31:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4a2c06ae70 0x7f4a2c06d330 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c15d9f0 0x7f4a4c0cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c142f80 0x7f4a4c1f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 --2- 172.21.15.119:0/3865146342 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4a4c06cb40 0x7f4a4c1f3790 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 >> 172.21.15.119:0/3865146342 conn(0x7f4a4c0cb610 msgr2=0x7f4a4c06c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 shutdown_connections 2023-11-08T19:31:04.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.378+0000 7f4a53034640 1 -- 172.21.15.119:0/3865146342 wait complete. 2023-11-08T19:31:04.399 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:31:04.400 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:31:04.488 INFO:teuthology.orchestra.run.smithi119.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:31:04.492 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:31:04.493 DEBUG:teuthology.orchestra.run.smithi119:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:31:04.593 INFO:teuthology.orchestra.run.smithi119.stdout:1699471849.116079.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i a', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:31:04.597 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471849.116079.core'] 2023-11-08T19:31:04.598 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471849.116079.core, deleting it 2023-11-08T19:31:04.598 DEBUG:teuthology.orchestra.run.smithi119:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471849.116079.core 2023-11-08T19:31:04.699 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:31:04.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.915+0000 7fa5c8a1a640 1 Processor -- start 2023-11-08T19:31:04.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.916+0000 7fa5c8a1a640 1 -- start start 2023-11-08T19:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.917+0000 7fa5c8a1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c40cc810 0x7fa5c40c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.917+0000 7fa5c8a1a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa5c415d180 0x7fa5c406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.917+0000 7fa5c8a1a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c40ae410 con 0x7fa5c406a390 2023-11-08T19:31:04.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.917+0000 7fa5c8a1a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c406ba30 con 0x7fa5c415d180 2023-11-08T19:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.917+0000 7fa5c8a1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c406bcf0 con 0x7fa5c40cc810 2023-11-08T19:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c2d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa5c415d180 0x7fa5c406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c259c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa5c406a390 0x7fa5c415ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42660/0 (socket says 172.21.15.119:42660) 2023-11-08T19:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c259c640 1 -- 172.21.15.119:0/3109292596 learned_addr learned my addr 172.21.15.119:0/3109292596 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:04.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c2d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa5c415d180 0x7fa5c406cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37330/0 (socket says 172.21.15.119:37330) 2023-11-08T19:31:04.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c1d9b640 1 --2- 172.21.15.119:0/3109292596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c40cc810 0x7fa5c40c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c2d9d640 1 -- 172.21.15.119:0/3109292596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c40cc810 msgr2=0x7fa5c40c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:04.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c2d9d640 1 --2- 172.21.15.119:0/3109292596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c40cc810 0x7fa5c40c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c2d9d640 1 -- 172.21.15.119:0/3109292596 >> v1:172.21.15.119:6789/0 conn(0x7fa5c406a390 legacy=0x7fa5c415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.918+0000 7fa5c2d9d640 1 -- 172.21.15.119:0/3109292596 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c406bfd0 con 0x7fa5c415d180 2023-11-08T19:31:04.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.919+0000 7fa5c2d9d640 1 --2- 172.21.15.119:0/3109292596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c415d180 0x7fa5c406cf70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009d90 tx=0x7fa5bc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7da062fb66d9b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c159a640 1 -- 172.21.15.119:0/3109292596 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc02cc50 con 0x7fa5c415d180 2023-11-08T19:31:04.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c159a640 1 -- 172.21.15.119:0/3109292596 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc02cdb0 con 0x7fa5c415d180 2023-11-08T19:31:04.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c159a640 1 -- 172.21.15.119:0/3109292596 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc034660 con 0x7fa5c415d180 2023-11-08T19:31:04.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/3109292596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c415d180 msgr2=0x7fa5c406cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/3109292596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c415d180 0x7fa5c406cf70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009d90 tx=0x7fa5bc02bea0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/3109292596 shutdown_connections 2023-11-08T19:31:04.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/3109292596 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c415d180 0x7fa5c406cf70 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/3109292596 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c40cc810 0x7fa5c40c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/3109292596 >> 172.21.15.119:0/3109292596 conn(0x7fa5c40cb610 msgr2=0x7fa5c40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/3109292596 shutdown_connections 2023-11-08T19:31:04.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.920+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/3109292596 wait complete. 2023-11-08T19:31:04.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.922+0000 7fa5c8a1a640 1 Processor -- start 2023-11-08T19:31:04.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.922+0000 7fa5c8a1a640 1 -- start start 2023-11-08T19:31:04.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.923+0000 7fa5c8a1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c406a390 0x7fa5c414d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.923+0000 7fa5c8a1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 0x7fa5c41f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.923+0000 7fa5c8a1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c415d180 0x7fa5c415e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.923+0000 7fa5c8a1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c41f3310 con 0x7fa5c40cc810 2023-11-08T19:31:04.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.923+0000 7fa5c8a1a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c41f35f0 con 0x7fa5c415d180 2023-11-08T19:31:04.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.923+0000 7fa5c8a1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c41f38d0 con 0x7fa5c406a390 2023-11-08T19:31:04.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 0x7fa5c41f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 0x7fa5c41f2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37346/0 (socket says 172.21.15.119:37346) 2023-11-08T19:31:04.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 -- 172.21.15.119:0/2621512263 learned_addr learned my addr 172.21.15.119:0/2621512263 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:04.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c259c640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c406a390 0x7fa5c414d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 -- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c406a390 msgr2=0x7fa5c414d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c2d9d640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c415d180 0x7fa5c415e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c406a390 0x7fa5c414d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 -- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c415d180 msgr2=0x7fa5c415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:04.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c415d180 0x7fa5c415e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:04.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 -- 172.21.15.119:0/2621512263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc009a70 con 0x7fa5c40cc810 2023-11-08T19:31:04.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c259c640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c406a390 0x7fa5c414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:04.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.924+0000 7fa5c1d9b640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 0x7fa5c41f2c30 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d800 tx=0x7fa5b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.925+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800b6c0 con 0x7fa5c40cc810 2023-11-08T19:31:04.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.925+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b800b820 con 0x7fa5c40cc810 2023-11-08T19:31:04.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.925+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b800bbb0 con 0x7fa5c40cc810 2023-11-08T19:31:04.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.925+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c40b2a30 con 0x7fa5c40cc810 2023-11-08T19:31:04.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.925+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c415d880 con 0x7fa5c40cc810 2023-11-08T19:31:04.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.926+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c4074430 con 0x7fa5c40cc810 2023-11-08T19:31:04.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.927+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa5b800bd10 con 0x7fa5c40cc810 2023-11-08T19:31:04.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.928+0000 7fa5b2ffd640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa5a406aec0 0x7fa5a406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:04.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.928+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 7436+0+0 (secure 0 0 0) 0x7fa5b808a200 con 0x7fa5c40cc810 2023-11-08T19:31:04.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.928+0000 7fa5c259c640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa5a406aec0 0x7fa5a406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:04.934 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.929+0000 7fa5c259c640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa5a406aec0 0x7fa5a406d380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007fd0 tx=0x7fa5b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:04.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:04.932+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5b8054c60 con 0x7fa5c40cc810 2023-11-08T19:31:05.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.196+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7fa5c406cd50 con 0x7fa5c40cc810 2023-11-08T19:31:05.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.202+0000 7fa5b2ffd640 1 -- 172.21.15.119:0/2621512263 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 7515 v84) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa5b8058280 con 0x7fa5c40cc810 2023-11-08T19:31:05.203 INFO:teuthology.orchestra.run.smithi119.stderr:failed mds gid 7515 2023-11-08T19:31:05.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.204+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa5a406aec0 msgr2=0x7fa5a406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.204+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa5a406aec0 0x7fa5a406d380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4007fd0 tx=0x7fa5b4009040 comp rx=0 tx=0).stop 2023-11-08T19:31:05.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 msgr2=0x7fa5c41f2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 0x7fa5c41f2c30 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa5b800d800 tx=0x7fa5b800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 shutdown_connections 2023-11-08T19:31:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa5a406aec0 0x7fa5a406d380 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5c415d180 0x7fa5c415e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa5c40cc810 0x7fa5c41f2c30 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 --2- 172.21.15.119:0/2621512263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5c406a390 0x7fa5c414d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 >> 172.21.15.119:0/2621512263 conn(0x7fa5c40cb610 msgr2=0x7fa5c40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 shutdown_connections 2023-11-08T19:31:05.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.205+0000 7fa5c8a1a640 1 -- 172.21.15.119:0/2621512263 wait complete. 2023-11-08T19:31:05.225 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T19:31:05.226 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T19:31:05.226 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:31:05.227 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:31:05.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:31:05.227 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi119 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a' 2023-11-08T19:31:05.228 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:31:05.228 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T19:31:05.235 INFO:tasks.ceph.mds.a:Started 2023-11-08T19:31:05.235 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:05.390 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:31:05.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.415+0000 7ff02784b640 1 Processor -- start 2023-11-08T19:31:05.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.415+0000 7ff02784b640 1 -- start start 2023-11-08T19:31:05.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff02784b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015d410 0x7ff0200ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff02784b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff02015df00 0x7ff02006dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.417 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff02784b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff02006b570 con 0x7ff02015e5f0 2023-11-08T19:31:05.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff02784b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02006c1f0 con 0x7ff02015df00 2023-11-08T19:31:05.418 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff02784b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02006b850 con 0x7ff02015d410 2023-11-08T19:31:05.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff025de8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff02015df00 0x7ff02006dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff025de8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff02015df00 0x7ff02006dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37366/0 (socket says 172.21.15.119:37366) 2023-11-08T19:31:05.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.416+0000 7ff025de8640 1 -- 172.21.15.119:0/3327266833 learned_addr learned my addr 172.21.15.119:0/3327266833 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:05.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.417+0000 7ff0255e7640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015d410 0x7ff0200ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.417+0000 7ff025de8640 1 -- 172.21.15.119:0/3327266833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015d410 msgr2=0x7ff0200ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.417+0000 7ff025de8640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015d410 0x7ff0200ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.417+0000 7ff025de8640 1 -- 172.21.15.119:0/3327266833 >> v1:172.21.15.188:6789/0 conn(0x7ff02015e5f0 legacy=0x7ff02015d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.417+0000 7ff025de8640 1 -- 172.21.15.119:0/3327266833 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02006bb30 con 0x7ff02015df00 2023-11-08T19:31:05.421 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.417+0000 7ff0255e7640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015d410 0x7ff0200ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.418+0000 7ff025de8640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015df00 0x7ff02006dce0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff018009e50 tx=0x7ff01802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b2e6a0ffbdbf91 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.418+0000 7ff017fff640 1 -- 172.21.15.119:0/3327266833 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01802c9e0 con 0x7ff02015df00 2023-11-08T19:31:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.418+0000 7ff017fff640 1 -- 172.21.15.119:0/3327266833 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01802cb40 con 0x7ff02015df00 2023-11-08T19:31:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.418+0000 7ff017fff640 1 -- 172.21.15.119:0/3327266833 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0180343f0 con 0x7ff02015df00 2023-11-08T19:31:05.422 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.418+0000 7ff02784b640 1 -- 172.21.15.119:0/3327266833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015df00 msgr2=0x7ff02006dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.418+0000 7ff02784b640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015df00 0x7ff02006dce0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7ff018009e50 tx=0x7ff01802be40 comp rx=0 tx=0).stop 2023-11-08T19:31:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.419+0000 7ff02784b640 1 -- 172.21.15.119:0/3327266833 shutdown_connections 2023-11-08T19:31:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.419+0000 7ff02784b640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015df00 0x7ff02006dce0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.419+0000 7ff02784b640 1 --2- 172.21.15.119:0/3327266833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015d410 0x7ff0200ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.419+0000 7ff02784b640 1 -- 172.21.15.119:0/3327266833 >> 172.21.15.119:0/3327266833 conn(0x7ff0200d0db0 msgr2=0x7ff0200d11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.419+0000 7ff02784b640 1 -- 172.21.15.119:0/3327266833 shutdown_connections 2023-11-08T19:31:05.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.419+0000 7ff02784b640 1 -- 172.21.15.119:0/3327266833 wait complete. 2023-11-08T19:31:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.420+0000 7ff02784b640 1 Processor -- start 2023-11-08T19:31:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.420+0000 7ff02784b640 1 -- start start 2023-11-08T19:31:05.426 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.421+0000 7ff02784b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 0x7ff0201601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.421+0000 7ff02784b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015df00 0x7ff02006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.421+0000 7ff02784b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02015e5f0 0x7ff02006e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.421+0000 7ff02784b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff020158cf0 con 0x7ff02015d410 2023-11-08T19:31:05.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.421+0000 7ff02784b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0201f6dc0 con 0x7ff02015e5f0 2023-11-08T19:31:05.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.421+0000 7ff02784b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0201606e0 con 0x7ff02015df00 2023-11-08T19:31:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 0x7ff0201601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 0x7ff0201601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:37374/0 (socket says 172.21.15.119:37374) 2023-11-08T19:31:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 -- 172.21.15.119:0/4058975567 learned_addr learned my addr 172.21.15.119:0/4058975567 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:05.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 -- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015df00 msgr2=0x7ff02006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff024de6640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015df00 0x7ff02006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff025de8640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02015e5f0 0x7ff02006e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015df00 0x7ff02006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 -- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02015e5f0 msgr2=0x7ff02006e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02015e5f0 0x7ff02006e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 -- 172.21.15.119:0/4058975567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff018009b30 con 0x7ff02015d410 2023-11-08T19:31:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.422+0000 7ff0255e7640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 0x7ff0201601a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00c7f0 tx=0x7ff01c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.423+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c005b90 con 0x7ff02015d410 2023-11-08T19:31:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.423+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c005cf0 con 0x7ff02015d410 2023-11-08T19:31:05.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.423+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff020158350 con 0x7ff02015d410 2023-11-08T19:31:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.423+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c007a30 con 0x7ff02015d410 2023-11-08T19:31:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.423+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff020158910 con 0x7ff02015d410 2023-11-08T19:31:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.424+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff020074430 con 0x7ff02015d410 2023-11-08T19:31:05.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.425+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff01c007b90 con 0x7ff02015d410 2023-11-08T19:31:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.426+0000 7ff015ffb640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff00406ae70 0x7ff00406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.426+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7ff01c089cf0 con 0x7ff02015d410 2023-11-08T19:31:05.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.426+0000 7ff024de6640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff00406ae70 0x7ff00406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.427+0000 7ff024de6640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff00406ae70 0x7ff00406d330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff0200ae1a0 tx=0x7ff010008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:05.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.430+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff01c054670 con 0x7ff02015d410 2023-11-08T19:31:05.519 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~13s 2023-11-08T19:31:05.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.656+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff02006d710 con 0x7ff02015d410 2023-11-08T19:31:05.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.657+0000 7ff015ffb640 1 -- 172.21.15.119:0/4058975567 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4149 (secure 0 0 0) 0x7ff01c01e090 con 0x7ff02015d410 2023-11-08T19:31:05.659 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:05.659 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":85,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:05.209678+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:replay","state_seq":1,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:05.659 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 85 2023-11-08T19:31:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff00406ae70 msgr2=0x7ff00406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff00406ae70 0x7ff00406d330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff0200ae1a0 tx=0x7ff010008040 comp rx=0 tx=0).stop 2023-11-08T19:31:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 msgr2=0x7ff0201601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 0x7ff0201601a0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00c7f0 tx=0x7ff01c00cbf0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 shutdown_connections 2023-11-08T19:31:05.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff00406ae70 0x7ff00406d330 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff02015e5f0 0x7ff02006e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff02015df00 0x7ff02006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 --2- 172.21.15.119:0/4058975567 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff02015d410 0x7ff0201601a0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 >> 172.21.15.119:0/4058975567 conn(0x7ff0200d0db0 msgr2=0x7ff0200ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 shutdown_connections 2023-11-08T19:31:05.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:05.660+0000 7ff02784b640 1 -- 172.21.15.119:0/4058975567 wait complete. 2023-11-08T19:31:05.679 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:31:10.680 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:10.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.906+0000 7f1d594f2640 1 Processor -- start 2023-11-08T19:31:10.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.906+0000 7f1d594f2640 1 -- start start 2023-11-08T19:31:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.908+0000 7f1d594f2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1d541447a0 0x7f1d5415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.908+0000 7f1d594f2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d540c3a50 con 0x7f1d540cad30 2023-11-08T19:31:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.908+0000 7f1d594f2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d54144b80 con 0x7f1d5406a120 2023-11-08T19:31:10.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.908+0000 7f1d594f2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d54070040 con 0x7f1d541447a0 2023-11-08T19:31:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.908+0000 7f1d4a7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f1d540cad30 0x7f1d5406f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57072/0 (socket says 172.21.15.119:57072) 2023-11-08T19:31:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.908+0000 7f1d4a7fc640 1 -- 172.21.15.119:0/1874140894 learned_addr learned my addr 172.21.15.119:0/1874140894 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:10.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.909+0000 7f1d52ffd640 1 --2- 172.21.15.119:0/1874140894 >> v2:172.21.15.119:3300/0 conn(0x7f1d541447a0 0x7f1d5415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.909+0000 7f1d527fc640 1 -- 172.21.15.119:0/1874140894 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26680264 0 0) 0x7f1d540c3a50 con 0x7f1d540cad30 2023-11-08T19:31:10.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.909+0000 7f1d527fc640 1 -- 172.21.15.119:0/1874140894 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d40003520 con 0x7f1d540cad30 2023-11-08T19:31:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.909+0000 7f1d52ffd640 1 -- 172.21.15.119:0/1874140894 >> v1:172.21.15.188:6789/0 conn(0x7f1d5406a120 legacy=0x7f1d5406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.909+0000 7f1d52ffd640 1 -- 172.21.15.119:0/1874140894 >> v1:172.21.15.119:6789/0 conn(0x7f1d540cad30 legacy=0x7f1d5406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.909+0000 7f1d52ffd640 1 -- 172.21.15.119:0/1874140894 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540c9150 con 0x7f1d541447a0 2023-11-08T19:31:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.911+0000 7f1d52ffd640 1 --2- 172.21.15.119:0/1874140894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d541447a0 0x7f1d5415f4c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009d90 tx=0x7f1d4402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357880751e91baf7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.911+0000 7f1d527fc640 1 -- 172.21.15.119:0/1874140894 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4402cd00 con 0x7f1d541447a0 2023-11-08T19:31:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.911+0000 7f1d527fc640 1 -- 172.21.15.119:0/1874140894 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4402ce60 con 0x7f1d541447a0 2023-11-08T19:31:10.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.911+0000 7f1d527fc640 1 -- 172.21.15.119:0/1874140894 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d44034660 con 0x7f1d541447a0 2023-11-08T19:31:10.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.911+0000 7f1d594f2640 1 -- 172.21.15.119:0/1874140894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d541447a0 msgr2=0x7f1d5415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.911+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1874140894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d541447a0 0x7f1d5415f4c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009d90 tx=0x7f1d4402bf50 comp rx=0 tx=0).stop 2023-11-08T19:31:10.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.912+0000 7f1d594f2640 1 -- 172.21.15.119:0/1874140894 shutdown_connections 2023-11-08T19:31:10.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.912+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1874140894 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d541447a0 0x7f1d5415f4c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.912+0000 7f1d594f2640 1 -- 172.21.15.119:0/1874140894 >> 172.21.15.119:0/1874140894 conn(0x7f1d540cb610 msgr2=0x7f1d540cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:10.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.912+0000 7f1d594f2640 1 -- 172.21.15.119:0/1874140894 shutdown_connections 2023-11-08T19:31:10.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.912+0000 7f1d594f2640 1 -- 172.21.15.119:0/1874140894 wait complete. 2023-11-08T19:31:10.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.913+0000 7f1d594f2640 1 Processor -- start 2023-11-08T19:31:10.917 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.914+0000 7f1d594f2640 1 -- start start 2023-11-08T19:31:10.918 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.914+0000 7f1d594f2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5406a120 0x7f1d540b3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.914+0000 7f1d594f2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 0x7f1d541f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.914+0000 7f1d594f2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d541447a0 0x7f1d540b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.914+0000 7f1d594f2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d540ccf50 con 0x7f1d540cad30 2023-11-08T19:31:10.919 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d594f2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d541f6570 con 0x7f1d5406a120 2023-11-08T19:31:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d594f2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d541f67d0 con 0x7f1d541447a0 2023-11-08T19:31:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 0x7f1d541f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 0x7f1d541f6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49808/0 (socket says 172.21.15.119:49808) 2023-11-08T19:31:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 -- 172.21.15.119:0/1536906347 learned_addr learned my addr 172.21.15.119:0/1536906347 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:10.920 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 -- 172.21.15.119:0/1536906347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d541447a0 msgr2=0x7f1d540b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d541447a0 0x7f1d540b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 -- 172.21.15.119:0/1536906347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5406a120 msgr2=0x7f1d540b3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5406a120 0x7f1d540b3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 -- 172.21.15.119:0/1536906347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d44009a70 con 0x7f1d540cad30 2023-11-08T19:31:10.921 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.915+0000 7f1d4a7fc640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 0x7f1d541f6030 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c400 tx=0x7f1d3c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.916+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005ce0 con 0x7f1d540cad30 2023-11-08T19:31:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.916+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c005e40 con 0x7f1d540cad30 2023-11-08T19:31:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.916+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d541f6ab0 con 0x7f1d540cad30 2023-11-08T19:31:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.916+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0072d0 con 0x7f1d540cad30 2023-11-08T19:31:10.922 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.916+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d540c5bd0 con 0x7f1d540cad30 2023-11-08T19:31:10.923 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.917+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54074430 con 0x7f1d540cad30 2023-11-08T19:31:10.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.918+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d3c0074b0 con 0x7f1d540cad30 2023-11-08T19:31:10.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.919+0000 7f1d4bfff640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1d3406ae70 0x7f1d3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:10.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.919+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f1d3c089270 con 0x7f1d540cad30 2023-11-08T19:31:10.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.919+0000 7f1d52ffd640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1d3406ae70 0x7f1d3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:10.924 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.920+0000 7f1d52ffd640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1d3406ae70 0x7f1d3406d330 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1d44036000 tx=0x7f1d4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:10.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:10.924+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d3c053c10 con 0x7f1d540cad30 2023-11-08T19:31:11.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.153+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1d5406caa0 con 0x7f1d540cad30 2023-11-08T19:31:11.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.154+0000 7f1d4bfff640 1 -- 172.21.15.119:0/1536906347 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f1d3c057230 con 0x7f1d540cad30 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:11.157 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:08.250393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:11.158 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 89 2023-11-08T19:31:11.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1d3406ae70 msgr2=0x7f1d3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1d3406ae70 0x7f1d3406d330 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1d44036000 tx=0x7f1d4402e000 comp rx=0 tx=0).stop 2023-11-08T19:31:11.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 msgr2=0x7f1d541f6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 0x7f1d541f6030 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00c400 tx=0x7f1d3c00c8c0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 shutdown_connections 2023-11-08T19:31:11.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1d3406ae70 0x7f1d3406d330 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1d541447a0 0x7f1d540b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1d540cad30 0x7f1d541f6030 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 --2- 172.21.15.119:0/1536906347 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1d5406a120 0x7f1d540b3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.157+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 >> 172.21.15.119:0/1536906347 conn(0x7f1d540cb610 msgr2=0x7f1d5415bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:11.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.158+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 shutdown_connections 2023-11-08T19:31:11.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.158+0000 7f1d594f2640 1 -- 172.21.15.119:0/1536906347 wait complete. 2023-11-08T19:31:11.178 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:31:11.179 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:11.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.388+0000 7f1a1754c640 1 Processor -- start 2023-11-08T19:31:11.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.388+0000 7f1a1754c640 1 -- start start 2023-11-08T19:31:11.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.389+0000 7f1a1754c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a1006a970 0x7f1a1006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.389+0000 7f1a1754c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.390+0000 7f1a1754c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a1006d4b0 con 0x7f1a100c9150 2023-11-08T19:31:11.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.390+0000 7f1a1754c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a101619b0 con 0x7f1a1006a970 2023-11-08T19:31:11.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.390+0000 7f1a1754c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10147b80 con 0x7f1a100ccde0 2023-11-08T19:31:11.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.390+0000 7f1a152e8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a1006a970 0x7f1a1006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.390+0000 7f1a15ae9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a152e8640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1a1006a970 0x7f1a1006cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:47630/0 (socket says 172.21.15.119:47630) 2023-11-08T19:31:11.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a14ae7640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f1a100c9150 0x7f1a100cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:50108/0 (socket says 172.21.15.119:50108) 2023-11-08T19:31:11.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a15ae9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:43326/0 (socket says 172.21.15.119:43326) 2023-11-08T19:31:11.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a152e8640 1 -- 172.21.15.119:0/1942599311 learned_addr learned my addr 172.21.15.119:0/1942599311 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a152e8640 1 -- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 msgr2=0x7f1a101653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a152e8640 1 --2- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101653f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a152e8640 1 -- 172.21.15.119:0/1942599311 >> v1:172.21.15.188:6789/0 conn(0x7f1a100c9150 legacy=0x7f1a100cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.391+0000 7f1a152e8640 1 -- 172.21.15.119:0/1942599311 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10147000 con 0x7f1a1006a970 2023-11-08T19:31:11.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.393+0000 7f1a152e8640 1 --2- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a1006cf70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0098a0 tx=0x7f1a0c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32186a76f9b67da server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.393+0000 7f1a03fff640 1 -- 172.21.15.119:0/1942599311 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c031640 con 0x7f1a1006a970 2023-11-08T19:31:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.393+0000 7f1a03fff640 1 -- 172.21.15.119:0/1942599311 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0c0317a0 con 0x7f1a1006a970 2023-11-08T19:31:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.393+0000 7f1a03fff640 1 -- 172.21.15.119:0/1942599311 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c031b30 con 0x7f1a1006a970 2023-11-08T19:31:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 -- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 msgr2=0x7f1a1006cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 --2- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a1006cf70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c0098a0 tx=0x7f1a0c02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 -- 172.21.15.119:0/1942599311 shutdown_connections 2023-11-08T19:31:11.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 --2- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 --2- 172.21.15.119:0/1942599311 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a1006cf70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 -- 172.21.15.119:0/1942599311 >> 172.21.15.119:0/1942599311 conn(0x7f1a100cb610 msgr2=0x7f1a100cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:11.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 -- 172.21.15.119:0/1942599311 shutdown_connections 2023-11-08T19:31:11.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.394+0000 7f1a1754c640 1 -- 172.21.15.119:0/1942599311 wait complete. 2023-11-08T19:31:11.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.396+0000 7f1a1754c640 1 Processor -- start 2023-11-08T19:31:11.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.396+0000 7f1a1754c640 1 -- start start 2023-11-08T19:31:11.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a1754c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a101f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a1754c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 0x7f1a100b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a1754c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101f81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a1754c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1014a730 con 0x7f1a100c9150 2023-11-08T19:31:11.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a1754c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10148d80 con 0x7f1a1006a970 2023-11-08T19:31:11.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a1754c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100b4020 con 0x7f1a100ccde0 2023-11-08T19:31:11.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.397+0000 7f1a14ae7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 0x7f1a100b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 0x7f1a100b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49816/0 (socket says 172.21.15.119:49816) 2023-11-08T19:31:11.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 -- 172.21.15.119:0/3416394562 learned_addr learned my addr 172.21.15.119:0/3416394562 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:11.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a15ae9640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101f81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a152e8640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a101f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 -- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 msgr2=0x7f1a101f81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.402 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101f81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 -- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 msgr2=0x7f1a101f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a101f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 -- 172.21.15.119:0/3416394562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0c0094d0 con 0x7f1a100c9150 2023-11-08T19:31:11.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.398+0000 7f1a14ae7640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 0x7f1a100b3ae0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c4d0 tx=0x7f1a0400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:11.403 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.399+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a04007480 con 0x7f1a100c9150 2023-11-08T19:31:11.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.399+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a040075e0 con 0x7f1a100c9150 2023-11-08T19:31:11.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.399+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a040049e0 con 0x7f1a100c9150 2023-11-08T19:31:11.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.399+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a101f75a0 con 0x7f1a100c9150 2023-11-08T19:31:11.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.399+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a101f7b60 con 0x7f1a100c9150 2023-11-08T19:31:11.404 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.400+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a10074430 con 0x7f1a100c9150 2023-11-08T19:31:11.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.401+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1a04004b40 con 0x7f1a100c9150 2023-11-08T19:31:11.405 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.402+0000 7f1a01ffb640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19f406ae70 0x7f19f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:11.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.402+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f1a04088d00 con 0x7f1a100c9150 2023-11-08T19:31:11.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.402+0000 7f1a152e8640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19f406ae70 0x7f19f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:11.406 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.403+0000 7f1a152e8640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19f406ae70 0x7f19f406d330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c036000 tx=0x7f1a0c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:11.407 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.406+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1a040536a0 con 0x7f1a100c9150 2023-11-08T19:31:11.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.638+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a1006d4b0 con 0x7f1a100c9150 2023-11-08T19:31:11.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.639+0000 7f1a01ffb640 1 -- 172.21.15.119:0/3416394562 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f1a1006d4b0 con 0x7f1a100c9150 2023-11-08T19:31:11.640 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:11.641 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:08.250393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:11.641 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 89 2023-11-08T19:31:11.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19f406ae70 msgr2=0x7f19f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19f406ae70 0x7f19f406d330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c036000 tx=0x7f1a0c03c040 comp rx=0 tx=0).stop 2023-11-08T19:31:11.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 msgr2=0x7f1a100b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:11.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 0x7f1a100b3ae0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f1a0400c4d0 tx=0x7f1a0400c990 comp rx=0 tx=0).stop 2023-11-08T19:31:11.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 shutdown_connections 2023-11-08T19:31:11.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f19f406ae70 0x7f19f406d330 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1a100ccde0 0x7f1a101f81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1a100c9150 0x7f1a100b3ae0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 --2- 172.21.15.119:0/3416394562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1a1006a970 0x7f1a101f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:11.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 >> 172.21.15.119:0/3416394562 conn(0x7f1a100cb610 msgr2=0x7f1a10071060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:11.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 shutdown_connections 2023-11-08T19:31:11.645 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:11.642+0000 7f1a1754c640 1 -- 172.21.15.119:0/3416394562 wait complete. 2023-11-08T19:31:12.066 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 2, 'explicitly_allowed_features': 2, 'created': '2023-11-08T19:28:52.484445+0000', 'modified': '2023-11-08T19:31:08.250393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 101, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 2, 'in': [0, 1], 'up': {'mds_0': 5210, 'mds_1': 7065}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5210': {'gid': 5210, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.188:6835/2339927924', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6833', 'nonce': 2339927924}, {'type': 'v1', 'addr': '172.21.15.188:6835', 'nonce': 2339927924}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_7065': {'gid': 7065, 'name': 'd', 'rank': 1, 'incarnation': 55, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.188:6834/4246462419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 4246462419}, {'type': 'v1', 'addr': '172.21.15.188:6834', 'nonce': 4246462419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T19:31:12.066 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 2/2 2023-11-08T19:31:12.067 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T19:31:12.231 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-08T19:31:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.290+0000 7fefa214c640 1 Processor -- start 2023-11-08T19:31:12.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.291+0000 7fefa214c640 1 -- start start 2023-11-08T19:31:12.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fefa214c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c15b080 0x7fef9c0ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fefa214c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef9c0cf8c0 0x7fef9c0cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fefa214c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fef9c161250 0x7fef9c161690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fefa214c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0cda70 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fefa214c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9c0d0280 con 0x7fef9c161250 2023-11-08T19:31:12.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fefa214c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c161fc0 con 0x7fef9c15b080 2023-11-08T19:31:12.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fef9affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef9c0cf8c0 0x7fef9c0cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.292+0000 7fef9affd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fef9c0cf8c0 0x7fef9c0cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49832/0 (socket says 172.21.15.119:49832) 2023-11-08T19:31:12.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9affd640 1 -- 172.21.15.119:0/1806302678 learned_addr learned my addr 172.21.15.119:0/1806302678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/1806302678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c15b080 0x7fef9c0ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9bfff640 1 --2- 172.21.15.119:0/1806302678 >> v2:172.21.15.188:3300/0 conn(0x7fef9c161250 0x7fef9c161690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9affd640 1 -- 172.21.15.119:0/1806302678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c15b080 msgr2=0x7fef9c0ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9affd640 1 --2- 172.21.15.119:0/1806302678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c15b080 0x7fef9c0ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9affd640 1 -- 172.21.15.119:0/1806302678 >> v2:172.21.15.188:3300/0 conn(0x7fef9c161250 msgr2=0x7fef9c161690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9affd640 1 --2- 172.21.15.119:0/1806302678 >> v2:172.21.15.188:3300/0 conn(0x7fef9c161250 0x7fef9c161690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.293+0000 7fef9affd640 1 -- 172.21.15.119:0/1806302678 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9c0cdcc0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.294+0000 7fef9affd640 1 --2- 172.21.15.119:0/1806302678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0cfd40 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0cd590 tx=0x7fef8402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5025f753988a9172 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.294+0000 7fef9a7fc640 1 -- 172.21.15.119:0/1806302678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840317d0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.294+0000 7fef9a7fc640 1 -- 172.21.15.119:0/1806302678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84031930 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.294+0000 7fef9a7fc640 1 -- 172.21.15.119:0/1806302678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84031cc0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 -- 172.21.15.119:0/1806302678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 msgr2=0x7fef9c0cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 --2- 172.21.15.119:0/1806302678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0cfd40 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fef9c0cd590 tx=0x7fef8402b870 comp rx=0 tx=0).stop 2023-11-08T19:31:12.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 -- 172.21.15.119:0/1806302678 shutdown_connections 2023-11-08T19:31:12.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 --2- 172.21.15.119:0/1806302678 >> v2:172.21.15.188:3300/0 conn(0x7fef9c161250 0x7fef9c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 --2- 172.21.15.119:0/1806302678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0cfd40 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 --2- 172.21.15.119:0/1806302678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c15b080 0x7fef9c0ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 -- 172.21.15.119:0/1806302678 >> 172.21.15.119:0/1806302678 conn(0x7fef9c06a9f0 msgr2=0x7fef9c06add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 -- 172.21.15.119:0/1806302678 shutdown_connections 2023-11-08T19:31:12.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.295+0000 7fefa214c640 1 -- 172.21.15.119:0/1806302678 wait complete. 2023-11-08T19:31:12.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.297+0000 7fefa214c640 1 Processor -- start 2023-11-08T19:31:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.297+0000 7fefa214c640 1 -- start start 2023-11-08T19:31:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fefa214c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fefa214c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c15b080 0x7fef9c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fefa214c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c161250 0x7fef9c0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fefa214c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0cd2f0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fefa214c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0ccd30 con 0x7fef9c15b080 2023-11-08T19:31:12.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fefa214c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9c0cd010 con 0x7fef9c161250 2023-11-08T19:31:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49840/0 (socket says 172.21.15.119:49840) 2023-11-08T19:31:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.298+0000 7fef9b7fe640 1 -- 172.21.15.119:0/4253216329 learned_addr learned my addr 172.21.15.119:0/4253216329 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9affd640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c15b080 0x7fef9c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9b7fe640 1 -- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c161250 msgr2=0x7fef9c0c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c161250 0x7fef9c0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9b7fe640 1 -- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c15b080 msgr2=0x7fef9c0ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c15b080 0x7fef9c0ca0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9b7fe640 1 -- 172.21.15.119:0/4253216329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef840094d0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0c5f00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fef8c007e60 tx=0x7fef8c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef93fff640 1 -- 172.21.15.119:0/4253216329 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c014620 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef93fff640 1 -- 172.21.15.119:0/4253216329 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00ee90 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fef93fff640 1 -- 172.21.15.119:0/4253216329 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c0149c0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.321 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.299+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef9c15e0a0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.300+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c0c6440 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.302+0000 7fef93fff640 1 -- 172.21.15.119:0/4253216329 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef8c014b20 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.303+0000 7fef93fff640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef7c06ae70 0x7fef7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.303+0000 7fef93fff640 1 -- 172.21.15.119:0/4253216329 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fef8c08a6b0 con 0x7fef9c0cf8c0 2023-11-08T19:31:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.303+0000 7fef9affd640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef7c06ae70 0x7fef7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.304+0000 7fefa4dbb740 1 Processor -- start 2023-11-08T19:31:12.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.304+0000 7fef9affd640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef7c06ae70 0x7fef7c06d330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fef84036000 tx=0x7fef8403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.304+0000 7fefa4dbb740 1 -- start start 2023-11-08T19:31:12.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec73bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec76bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec73c300 0x55faec76c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec6ae7e0 con 0x55faec7692a0 2023-11-08T19:31:12.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec6abed0 con 0x55faec768ec0 2023-11-08T19:31:12.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55faec6a77c0 con 0x55faec73c300 2023-11-08T19:31:12.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fef9affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec76bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.326 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fef9affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec76bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49842/0 (socket says 172.21.15.119:49842) 2023-11-08T19:31:12.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.305+0000 7fef9affd640 1 -- 172.21.15.119:0/273480371 learned_addr learned my addr 172.21.15.119:0/273480371 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec73bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.327 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9bfff640 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec73c300 0x55faec76c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9affd640 1 -- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec73c300 msgr2=0x55faec76c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9affd640 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec73c300 0x55faec76c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9affd640 1 -- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 msgr2=0x55faec73bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.328 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9affd640 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec73bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9affd640 1 -- 172.21.15.119:0/273480371 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faec67ab10 con 0x55faec7692a0 2023-11-08T19:31:12.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef9affd640 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec76bc50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef8402be80 tx=0x7fef84030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d5a2a8e9ff34a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef917fa640 1 -- 172.21.15.119:0/273480371 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840329b0 con 0x55faec7692a0 2023-11-08T19:31:12.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef917fa640 1 -- 172.21.15.119:0/273480371 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84032b10 con 0x55faec7692a0 2023-11-08T19:31:12.329 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.306+0000 7fef917fa640 1 -- 172.21.15.119:0/273480371 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84032c80 con 0x55faec7692a0 2023-11-08T19:31:12.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 -- 172.21.15.119:0/273480371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 msgr2=0x55faec76bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec76bc50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fef8402be80 tx=0x7fef84030000 comp rx=0 tx=0).stop 2023-11-08T19:31:12.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 -- 172.21.15.119:0/273480371 shutdown_connections 2023-11-08T19:31:12.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec73c300 0x55faec76c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.330 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec76bc50 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/273480371 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec73bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 -- 172.21.15.119:0/273480371 >> 172.21.15.119:0/273480371 conn(0x55faec73e0d0 msgr2=0x55faec786c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 -- 172.21.15.119:0/273480371 shutdown_connections 2023-11-08T19:31:12.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.307+0000 7fefa4dbb740 1 -- 172.21.15.119:0/273480371 wait complete. 2023-11-08T19:31:12.331 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.308+0000 7fefa4dbb740 1 -- start start 2023-11-08T19:31:12.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.308+0000 7fefa4dbb740 1 Processor -- start 2023-11-08T19:31:12.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec781280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec75cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec76dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.332 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec644c40 con 0x55faec7692a0 2023-11-08T19:31:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec6448e0 con 0x55faec768ec0 2023-11-08T19:31:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55faec40af20 con 0x55faec780be0 2023-11-08T19:31:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fefa4dbb740 10 client.0 _ll_register_callbacks cb 0x55faec666cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:31:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fef9affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec75cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fef9affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec75cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49856/0 (socket says 172.21.15.119:49856) 2023-11-08T19:31:12.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.309+0000 7fef9affd640 1 -- 172.21.15.119:0/617227606 learned_addr learned my addr 172.21.15.119:0/617227606 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9bfff640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec76dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec781280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9affd640 1 -- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 msgr2=0x55faec76dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9affd640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec76dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9affd640 1 -- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 msgr2=0x55faec781280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9affd640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec781280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9affd640 1 -- 172.21.15.119:0/617227606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c0cdcc0 con 0x55faec7692a0 2023-11-08T19:31:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef9affd640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec75cf20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fef84050040 tx=0x7fef84004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=849e8775f526ddb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:31:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef7a7fc640 1 -- 172.21.15.119:0/617227606 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840329b0 con 0x55faec7692a0 2023-11-08T19:31:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fef7a7fc640 1 -- 172.21.15.119:0/617227606 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fef84003bd0 con 0x55faec7692a0 2023-11-08T19:31:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.310+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55faec67bc80 con 0x55faec7692a0 2023-11-08T19:31:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fef7a7fc640 1 -- 172.21.15.119:0/617227606 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef840044f0 con 0x55faec7692a0 2023-11-08T19:31:12.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 10 client.7776 fetch_fsmap learned FSMap version 89 2023-11-08T19:31:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55faec6a5e40 con 0x55faec7692a0 2023-11-08T19:31:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fef7a7fc640 1 -- 172.21.15.119:0/617227606 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55faec67bc80 con 0x55faec7692a0 2023-11-08T19:31:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fef7a7fc640 1 -- 172.21.15.119:0/617227606 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 89) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fef8404e400 con 0x55faec7692a0 2023-11-08T19:31:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 10 client.7776 fetch_fsmap finished waiting for FSMap version 89 2023-11-08T19:31:12.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 10 client.7776 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:31:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 20 client.7776 populate_metadata read hostname 'smithi119' 2023-11-08T19:31:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec77f810 0x55faec780040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 4 client.7776 mds_command: new command op to 5210 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:31:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.311+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 --> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55faec6a07a0 con 0x55faec77f810 2023-11-08T19:31:12.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.312+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec77f810 0x55faec780040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.313+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec77f810 0x55faec780040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8004d43afa019744 server_cookie=b8f04ad96bc5ba43 in_seq=0 out_seq=0 2023-11-08T19:31:12.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.313+0000 7fef7a7fc640 10 client.7776 ms_handle_connect on v2:172.21.15.188:6833/2339927924 2023-11-08T19:31:12.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.314+0000 7fef7a7fc640 1 -- 172.21.15.119:0/617227606 <== mds.0 v2:172.21.15.188:6833/2339927924 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55faec6a07a0 con 0x55faec77f810 2023-11-08T19:31:12.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.314+0000 7fef7a7fc640 10 client.7776 handle_command_reply: tid=0 2023-11-08T19:31:12.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 client.7776 shutdown 2023-11-08T19:31:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 msgr2=0x55faec75cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec75cf20 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fef84050040 tx=0x7fef84004730 comp rx=0 tx=0).stop 2023-11-08T19:31:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 shutdown_connections 2023-11-08T19:31:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec77f810 0x55faec780040 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec76dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec7692a0 0x55faec75cf20 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/617227606 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec768ec0 0x55faec781280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fef7a7fc640 0 client.7776 ms_handle_reset on v2:172.21.15.188:6833/2339927924 2023-11-08T19:31:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 >> 172.21.15.119:0/617227606 conn(0x55faec73c300 msgr2=0x55faec78a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 shutdown_connections 2023-11-08T19:31:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 1 -- 172.21.15.119:0/617227606 wait complete. 2023-11-08T19:31:12.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.315+0000 7fefa4dbb740 20 client.7776 trim_cache size 0 max 16384 2023-11-08T19:31:12.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.332+0000 7fefa4dbb740 1 Processor -- start 2023-11-08T19:31:12.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.332+0000 7fefa4dbb740 1 -- start start 2023-11-08T19:31:12.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec76eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec7692a0 0x55faec78a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec780be0 0x55faec780010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec76bd40 con 0x55faec768ec0 2023-11-08T19:31:12.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec796a40 con 0x55faec780be0 2023-11-08T19:31:12.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55faec65cd00 con 0x55faec7692a0 2023-11-08T19:31:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec76eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec76eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49872/0 (socket says 172.21.15.119:49872) 2023-11-08T19:31:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.333+0000 7fef9b7fe640 1 -- 172.21.15.119:0/689927255 learned_addr learned my addr 172.21.15.119:0/689927255 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef9b7fe640 1 -- 172.21.15.119:0/689927255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec7692a0 msgr2=0x55faec78a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec7692a0 0x55faec78a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef9b7fe640 1 -- 172.21.15.119:0/689927255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec780be0 msgr2=0x55faec780010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec780be0 0x55faec780010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef9b7fe640 1 -- 172.21.15.119:0/689927255 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55faec643b90 con 0x55faec768ec0 2023-11-08T19:31:12.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec76eff0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9b0 tx=0x7fef8c088380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769f14bcb19a778e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef91ffb640 1 -- 172.21.15.119:0/689927255 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c05ce40 con 0x55faec768ec0 2023-11-08T19:31:12.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef91ffb640 1 -- 172.21.15.119:0/689927255 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c060ff0 con 0x55faec768ec0 2023-11-08T19:31:12.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.334+0000 7fef91ffb640 1 -- 172.21.15.119:0/689927255 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c057090 con 0x55faec768ec0 2023-11-08T19:31:12.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 -- 172.21.15.119:0/689927255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 msgr2=0x55faec76eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec76eff0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00e9b0 tx=0x7fef8c088380 comp rx=0 tx=0).stop 2023-11-08T19:31:12.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 -- 172.21.15.119:0/689927255 shutdown_connections 2023-11-08T19:31:12.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec780be0 0x55faec780010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec7692a0 0x55faec78a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/689927255 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec76eff0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 -- 172.21.15.119:0/689927255 >> 172.21.15.119:0/689927255 conn(0x55faec73c300 msgr2=0x55faec7a85d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 -- 172.21.15.119:0/689927255 shutdown_connections 2023-11-08T19:31:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.335+0000 7fefa4dbb740 1 -- 172.21.15.119:0/689927255 wait complete. 2023-11-08T19:31:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.336+0000 7fefa4dbb740 1 -- start start 2023-11-08T19:31:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.336+0000 7fefa4dbb740 1 Processor -- start 2023-11-08T19:31:12.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec786b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec7692a0 0x55faec798d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec7992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec783830 con 0x55faec768ec0 2023-11-08T19:31:12.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x55faec741650 con 0x55faec7692a0 2023-11-08T19:31:12.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x55faec741000 con 0x55faec780be0 2023-11-08T19:31:12.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fefa4dbb740 10 client.0 _ll_register_callbacks cb 0x55faec769b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:31:12.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec786b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec786b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49878/0 (socket says 172.21.15.119:49878) 2023-11-08T19:31:12.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fef9b7fe640 1 -- 172.21.15.119:0/2779271630 learned_addr learned my addr 172.21.15.119:0/2779271630 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.337+0000 7fef9bfff640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec7992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9affd640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec7692a0 0x55faec798d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9b7fe640 1 -- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 msgr2=0x55faec7992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec7992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9b7fe640 1 -- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec7692a0 msgr2=0x55faec798d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec7692a0 0x55faec798d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9b7fe640 1 -- 172.21.15.119:0/2779271630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9c0c6440 con 0x55faec768ec0 2023-11-08T19:31:12.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9bfff640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec7992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:12.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9affd640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec7692a0 0x55faec798d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:12.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef9b7fe640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec786b50 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fef8c007e30 tx=0x7fef8c05f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c696f5a4a75eede server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef7a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:31:12.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef7a7fc640 1 -- 172.21.15.119:0/2779271630 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c055400 con 0x55faec768ec0 2023-11-08T19:31:12.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef7a7fc640 1 -- 172.21.15.119:0/2779271630 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fef8c055560 con 0x55faec768ec0 2023-11-08T19:31:12.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef84032b10 con 0x55faec768ec0 2023-11-08T19:31:12.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef7a7fc640 1 -- 172.21.15.119:0/2779271630 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef8c0581e0 con 0x55faec768ec0 2023-11-08T19:31:12.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fefa4dbb740 10 client.7782 fetch_fsmap learned FSMap version 89 2023-11-08T19:31:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x55faec669180 con 0x55faec768ec0 2023-11-08T19:31:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.338+0000 7fef7a7fc640 1 -- 172.21.15.119:0/2779271630 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84032b10 con 0x55faec768ec0 2023-11-08T19:31:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fef7a7fc640 1 -- 172.21.15.119:0/2779271630 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 89) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fef8c060b60 con 0x55faec768ec0 2023-11-08T19:31:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fefa4dbb740 10 client.7782 fetch_fsmap finished waiting for FSMap version 89 2023-11-08T19:31:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fefa4dbb740 10 client.7782 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T19:31:12.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fefa4dbb740 20 client.7782 populate_metadata read hostname 'smithi119' 2023-11-08T19:31:12.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec7854c0 0x55faec7858a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fefa4dbb740 4 client.7782 mds_command: new command op to 5210 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:31:12.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.339+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 --> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55faec6a04c0 con 0x55faec7854c0 2023-11-08T19:31:12.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.340+0000 7fef9affd640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec7854c0 0x55faec7858a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fef9affd640 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec7854c0 0x55faec7858a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9762b4d9454feec server_cookie=5610612d4277be88 in_seq=0 out_seq=0 2023-11-08T19:31:12.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fef7a7fc640 10 client.7782 ms_handle_connect on v2:172.21.15.188:6833/2339927924 2023-11-08T19:31:12.362 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fef7a7fc640 1 -- 172.21.15.119:0/2779271630 <== mds.0 v2:172.21.15.188:6833/2339927924 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55faec6a04c0 con 0x55faec7854c0 2023-11-08T19:31:12.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fef7a7fc640 10 client.7782 handle_command_reply: tid=0 2023-11-08T19:31:12.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 client.7782 shutdown 2023-11-08T19:31:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 msgr2=0x55faec786b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec786b50 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fef8c007e30 tx=0x7fef8c05f040 comp rx=0 tx=0).stop 2023-11-08T19:31:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 shutdown_connections 2023-11-08T19:31:12.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:6833/2339927924,v1:172.21.15.188:6835/2339927924] conn(0x55faec7854c0 0x55faec7858a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55faec780be0 0x55faec7992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55faec7692a0 0x55faec798d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 --2- 172.21.15.119:0/2779271630 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x55faec768ec0 0x55faec786b50 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fef7a7fc640 0 client.7782 ms_handle_reset on v2:172.21.15.188:6833/2339927924 2023-11-08T19:31:12.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.341+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 >> 172.21.15.119:0/2779271630 conn(0x55faec73c300 msgr2=0x55faec7803d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 shutdown_connections 2023-11-08T19:31:12.366 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa4dbb740 1 -- 172.21.15.119:0/2779271630 wait complete. 2023-11-08T19:31:12.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa4dbb740 20 client.7782 trim_cache size 0 max 16384 2023-11-08T19:31:12.367 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:31:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:31:12.367 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 0, 2023-11-08T19:31:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 5210, 2023-11-08T19:31:12.368 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:31:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:31:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:31:12.369 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 7.1231111360000003, 2023-11-08T19:31:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 89, 2023-11-08T19:31:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 101, 2023-11-08T19:31:12.370 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 101, 2023-11-08T19:31:12.371 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 34.688359329000001 2023-11-08T19:31:12.371 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:31:12.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef7c06ae70 msgr2=0x7fef7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef7c06ae70 0x7fef7c06d330 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fef84036000 tx=0x7fef8403d040 comp rx=0 tx=0).stop 2023-11-08T19:31:12.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 msgr2=0x7fef9c0c5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0c5f00 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fef8c007e60 tx=0x7fef8c00e4b0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 shutdown_connections 2023-11-08T19:31:12.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fef7c06ae70 0x7fef7c06d330 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fef9c161250 0x7fef9c0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fef9c15b080 0x7fef9c0ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 --2- 172.21.15.119:0/4253216329 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fef9c0cf8c0 0x7fef9c0c5f00 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 >> 172.21.15.119:0/4253216329 conn(0x7fef9c06a9f0 msgr2=0x7fef9c0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.342+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 shutdown_connections 2023-11-08T19:31:12.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.343+0000 7fefa214c640 1 -- 172.21.15.119:0/4253216329 wait complete. 2023-11-08T19:31:12.375 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T19:31:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.593+0000 7fe4bf21a640 1 Processor -- start 2023-11-08T19:31:12.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.594+0000 7fe4bf21a640 1 -- start start 2023-11-08T19:31:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.595+0000 7fe4bf21a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4b806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.595+0000 7fe4bf21a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4b806a120 0x7fe4b806a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.595+0000 7fe4bf21a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b815f900 con 0x7fe4b81447a0 2023-11-08T19:31:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.595+0000 7fe4bf21a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b806aab0 con 0x7fe4b806a120 2023-11-08T19:31:12.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.595+0000 7fe4bf21a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80c3a50 con 0x7fe4b815f4c0 2023-11-08T19:31:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.596+0000 7fe4bcfb6640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe4b81447a0 0x7fe4b80cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57082/0 (socket says 172.21.15.119:57082) 2023-11-08T19:31:12.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.596+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/2314350340 learned_addr learned my addr 172.21.15.119:0/2314350340 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.596+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4b806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.596+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/2314350340 >> v2:172.21.15.188:3300/0 conn(0x7fe4b806a120 0x7fe4b806a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.596+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744859340 0 0) 0x7fe4ac001110 con 0x7fe4b81447a0 2023-11-08T19:31:12.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.596+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4a8003520 con 0x7fe4b81447a0 2023-11-08T19:31:12.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1198496030 0 0) 0x7fe4a8003520 con 0x7fe4b81447a0 2023-11-08T19:31:12.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 msgr2=0x7fe4b806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4b806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 >> v2:172.21.15.188:3300/0 conn(0x7fe4b806a120 msgr2=0x7fe4b806a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 --2- 172.21.15.119:0/2314350340 >> v2:172.21.15.188:3300/0 conn(0x7fe4b806a120 0x7fe4b806a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b8144b80 con 0x7fe4b81447a0 2023-11-08T19:31:12.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fe4ac001800 con 0x7fe4b81447a0 2023-11-08T19:31:12.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 >> v1:172.21.15.119:6789/0 conn(0x7fe4b81447a0 legacy=0x7fe4b80cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a8003970 0x7fe4a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8004d00 0x7fe4a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b806aab0 con 0x7fe4a8003970 2023-11-08T19:31:12.604 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80c3a50 con 0x7fe4a8004d00 2023-11-08T19:31:12.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a800b980 con 0x7fe4b815f4c0 2023-11-08T19:31:12.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.597+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a8003970 0x7fe4a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 msgr2=0x7fe4a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.605 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8004d00 0x7fe4a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4a80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8004d00 msgr2=0x7fe4a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8004d00 0x7fe4a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/2314350340 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a8003520 con 0x7fe4a8003970 2023-11-08T19:31:12.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b815f4c0 0x7fe4a80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:12.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.598+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8004d00 0x7fe4a8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:12.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.599+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a8003970 0x7fe4a8003d90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00c240 tx=0x7fe4a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a45e707fd6e07925 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.599+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac005ad0 con 0x7fe4a8003970 2023-11-08T19:31:12.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.599+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac005c50 con 0x7fe4a8003970 2023-11-08T19:31:12.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.599+0000 7fe4b77fe640 1 -- 172.21.15.119:0/2314350340 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac02c8d0 con 0x7fe4a8003970 2023-11-08T19:31:12.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bf21a640 1 -- 172.21.15.119:0/2314350340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a8003970 msgr2=0x7fe4a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/2314350340 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4a8003970 0x7fe4a8003d90 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00c240 tx=0x7fe4a80057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/2314350340 reap_dead start 2023-11-08T19:31:12.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bf21a640 1 -- 172.21.15.119:0/2314350340 shutdown_connections 2023-11-08T19:31:12.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bf21a640 1 -- 172.21.15.119:0/2314350340 >> 172.21.15.119:0/2314350340 conn(0x7fe4b80cb610 msgr2=0x7fe4b80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bf21a640 1 -- 172.21.15.119:0/2314350340 shutdown_connections 2023-11-08T19:31:12.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.600+0000 7fe4bf21a640 1 -- 172.21.15.119:0/2314350340 wait complete. 2023-11-08T19:31:12.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.602+0000 7fe4bf21a640 1 Processor -- start 2023-11-08T19:31:12.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.602+0000 7fe4bf21a640 1 -- start start 2023-11-08T19:31:12.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bf21a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8003970 0x7fe4b815a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bf21a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b80c7630 0x7fe4b80c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bf21a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 0x7fe4b80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bf21a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80c71a0 con 0x7fe4b80c6110 2023-11-08T19:31:12.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bf21a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80c6590 con 0x7fe4a8003970 2023-11-08T19:31:12.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bf21a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b80c6870 con 0x7fe4b80c7630 2023-11-08T19:31:12.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bd7b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 0x7fe4b80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bd7b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 0x7fe4b80cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49894/0 (socket says 172.21.15.119:49894) 2023-11-08T19:31:12.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.603+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/3901277821 learned_addr learned my addr 172.21.15.119:0/3901277821 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8003970 0x7fe4b815a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b80c7630 msgr2=0x7fe4b80c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b80c7630 0x7fe4b80c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8003970 msgr2=0x7fe4b815a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8003970 0x7fe4b815a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/3901277821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b80c6b50 con 0x7fe4b80c6110 2023-11-08T19:31:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 0x7fe4b80cdca0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80ce900 tx=0x7fe4b000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4b57fa640 1 -- 172.21.15.119:0/3901277821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0017400 con 0x7fe4b80c6110 2023-11-08T19:31:12.617 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4b57fa640 1 -- 172.21.15.119:0/3901277821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b0011040 con 0x7fe4b80c6110 2023-11-08T19:31:12.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4b57fa640 1 -- 172.21.15.119:0/3901277821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0017780 con 0x7fe4b80c6110 2023-11-08T19:31:12.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.604+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b80c6e30 con 0x7fe4b80c6110 2023-11-08T19:31:12.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.605+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b8142870 con 0x7fe4b80c6110 2023-11-08T19:31:12.618 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.607+0000 7fe4b57fa640 1 -- 172.21.15.119:0/3901277821 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4b00203b0 con 0x7fe4b80c6110 2023-11-08T19:31:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.608+0000 7fe4b57fa640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe49c06ae70 0x7fe49c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.608+0000 7fe4b57fa640 1 -- 172.21.15.119:0/3901277821 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fe4b0089ff0 con 0x7fe4b80c6110 2023-11-08T19:31:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.608+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe49c06ae70 0x7fe49c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.608+0000 7fe4c1dab740 1 Processor -- start 2023-11-08T19:31:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.609+0000 7fe4c1dab740 1 -- start start 2023-11-08T19:31:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.609+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe49c06ae70 0x7fe49c06d330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac034000 tx=0x7fe4ac007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.609+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abdec0 0x561803a90dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.609+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 0x561803ac0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.609+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803a91300 0x561803ac1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803a037e0 con 0x561803abe2a0 2023-11-08T19:31:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803a00ed0 con 0x561803a91300 2023-11-08T19:31:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618039fc7c0 con 0x561803abdec0 2023-11-08T19:31:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 0x561803ac0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 0x561803ac0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49902/0 (socket says 172.21.15.119:49902) 2023-11-08T19:31:12.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 -- 172.21.15.119:0/2719763053 learned_addr learned my addr 172.21.15.119:0/2719763053 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 -- 172.21.15.119:0/2719763053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abdec0 msgr2=0x561803a90dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abdec0 0x561803a90dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 -- 172.21.15.119:0/2719763053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803a91300 msgr2=0x561803ac1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803a91300 0x561803ac1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 -- 172.21.15.119:0/2719763053 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618039cfb10 con 0x561803abe2a0 2023-11-08T19:31:12.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.610+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 0x561803ac0c50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009a80 tx=0x7fe4a4009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f023a6f528c16a47 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4a27fc640 1 -- 172.21.15.119:0/2719763053 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4010dc0 con 0x561803abe2a0 2023-11-08T19:31:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4a27fc640 1 -- 172.21.15.119:0/2719763053 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a40143f0 con 0x561803abe2a0 2023-11-08T19:31:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4a27fc640 1 -- 172.21.15.119:0/2719763053 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4014780 con 0x561803abe2a0 2023-11-08T19:31:12.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 -- 172.21.15.119:0/2719763053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 msgr2=0x561803ac0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 0x561803ac0c50 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009a80 tx=0x7fe4a4009f40 comp rx=0 tx=0).stop 2023-11-08T19:31:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 -- 172.21.15.119:0/2719763053 shutdown_connections 2023-11-08T19:31:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803a91300 0x561803ac1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abe2a0 0x561803ac0c50 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/2719763053 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abdec0 0x561803a90dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.611+0000 7fe4c1dab740 1 -- 172.21.15.119:0/2719763053 >> 172.21.15.119:0/2719763053 conn(0x561803a930d0 msgr2=0x561803adbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.612+0000 7fe4c1dab740 1 -- 172.21.15.119:0/2719763053 shutdown_connections 2023-11-08T19:31:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.612+0000 7fe4c1dab740 1 -- 172.21.15.119:0/2719763053 wait complete. 2023-11-08T19:31:12.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.612+0000 7fe4c1dab740 1 -- start start 2023-11-08T19:31:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.612+0000 7fe4c1dab740 1 Processor -- start 2023-11-08T19:31:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.613+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803abd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.613+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 0x561803ad5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.613+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 0x561803ac36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.613+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803999c40 con 0x561803abdec0 2023-11-08T19:31:12.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.613+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618039998e0 con 0x561803abe2a0 2023-11-08T19:31:12.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.613+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x56180375ff20 con 0x561803ad5ec0 2023-11-08T19:31:12.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4c1dab740 10 client.0 _ll_register_callbacks cb 0x5618039bbcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:31:12.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803abd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803abd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49906/0 (socket says 172.21.15.119:49906) 2023-11-08T19:31:12.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/648663695 learned_addr learned my addr 172.21.15.119:0/648663695 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 msgr2=0x561803ac36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 0x561803ac36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 msgr2=0x561803ad5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 0x561803ad5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/648663695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a4009760 con 0x561803abdec0 2023-11-08T19:31:12.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803abd4f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac001390 tx=0x7fe4ac001b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df5356e2365796 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.614+0000 7fe4977fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:31:12.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4977fe640 1 -- 172.21.15.119:0/648663695 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac04a070 con 0x561803abdec0 2023-11-08T19:31:12.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4977fe640 1 -- 172.21.15.119:0/648663695 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fe4ac004e40 con 0x561803abdec0 2023-11-08T19:31:12.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618039d0c80 con 0x561803abdec0 2023-11-08T19:31:12.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4977fe640 1 -- 172.21.15.119:0/648663695 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4ac03c040 con 0x561803abdec0 2023-11-08T19:31:12.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 10 client.7794 fetch_fsmap learned FSMap version 89 2023-11-08T19:31:12.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5618039fae40 con 0x561803abdec0 2023-11-08T19:31:12.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4977fe640 1 -- 172.21.15.119:0/648663695 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618039d0c80 con 0x561803abdec0 2023-11-08T19:31:12.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4977fe640 1 -- 172.21.15.119:0/648663695 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 89) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe4ac007070 con 0x561803abdec0 2023-11-08T19:31:12.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 10 client.7794 fetch_fsmap finished waiting for FSMap version 89 2023-11-08T19:31:12.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 10 client.7794 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:31:12.638 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 20 client.7794 populate_metadata read hostname 'smithi119' 2023-11-08T19:31:12.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aee150 0x561803aebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 4 client.7794 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T19:31:12.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.615+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618039f57a0 con 0x561803aee150 2023-11-08T19:31:12.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.616+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aee150 0x561803aebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.617+0000 7fe4b7fff640 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aee150 0x561803aebde0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4edacd30c2434b3 server_cookie=bd72d02483b5a791 in_seq=0 out_seq=0 2023-11-08T19:31:12.639 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.617+0000 7fe4977fe640 10 client.7794 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:31:12.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4977fe640 1 -- 172.21.15.119:0/648663695 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5618039f57a0 con 0x561803aee150 2023-11-08T19:31:12.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4977fe640 10 client.7794 handle_command_reply: tid=0 2023-11-08T19:31:12.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 client.7794 shutdown 2023-11-08T19:31:12.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 msgr2=0x561803abd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803abd4f0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac001390 tx=0x7fe4ac001b90 comp rx=0 tx=0).stop 2023-11-08T19:31:12.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 shutdown_connections 2023-11-08T19:31:12.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 0x561803ac36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aee150 0x561803aebde0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 0x561803ad5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/648663695 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803abd4f0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4977fe640 0 client.7794 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:31:12.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 >> 172.21.15.119:0/648663695 conn(0x561803a91300 msgr2=0x561803ac0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 shutdown_connections 2023-11-08T19:31:12.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 1 -- 172.21.15.119:0/648663695 wait complete. 2023-11-08T19:31:12.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.619+0000 7fe4c1dab740 20 client.7794 trim_cache size 0 max 16384 2023-11-08T19:31:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.637+0000 7fe4c1dab740 1 Processor -- start 2023-11-08T19:31:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.637+0000 7fe4c1dab740 1 -- start start 2023-11-08T19:31:12.654 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803ac1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 0x561803ad6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 0x561803ad4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.655 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803a959c0 con 0x561803abdec0 2023-11-08T19:31:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803aff020 con 0x561803abe2a0 2023-11-08T19:31:12.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561803adf910 con 0x561803ad5ec0 2023-11-08T19:31:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803ac1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803ac1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49912/0 (socket says 172.21.15.119:49912) 2023-11-08T19:31:12.657 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/979784829 learned_addr learned my addr 172.21.15.119:0/979784829 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/979784829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 msgr2=0x561803ad4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.658 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 0x561803ad4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/979784829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 msgr2=0x561803ad6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 0x561803ad6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.659 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.638+0000 7fe4bcfb6640 1 -- 172.21.15.119:0/979784829 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561803998b90 con 0x561803abdec0 2023-11-08T19:31:12.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.639+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803ac1120 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac007940 tx=0x7fe4ac04a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78cc9b179eadadd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.639+0000 7fe4a2ffd640 1 -- 172.21.15.119:0/979784829 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac007530 con 0x561803abdec0 2023-11-08T19:31:12.660 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.639+0000 7fe4a2ffd640 1 -- 172.21.15.119:0/979784829 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac007690 con 0x561803abdec0 2023-11-08T19:31:12.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.639+0000 7fe4a2ffd640 1 -- 172.21.15.119:0/979784829 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac035670 con 0x561803abdec0 2023-11-08T19:31:12.661 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.639+0000 7fe4c1dab740 1 -- 172.21.15.119:0/979784829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 msgr2=0x561803ac1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.639+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803ac1120 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac007940 tx=0x7fe4ac04a000 comp rx=0 tx=0).stop 2023-11-08T19:31:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 -- 172.21.15.119:0/979784829 shutdown_connections 2023-11-08T19:31:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803ad5ec0 0x561803ad4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abe2a0 0x561803ad6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/979784829 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803abdec0 0x561803ac1120 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 -- 172.21.15.119:0/979784829 >> 172.21.15.119:0/979784829 conn(0x561803a91300 msgr2=0x561803adf160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 -- 172.21.15.119:0/979784829 shutdown_connections 2023-11-08T19:31:12.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 -- 172.21.15.119:0/979784829 wait complete. 2023-11-08T19:31:12.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.640+0000 7fe4c1dab740 1 -- start start 2023-11-08T19:31:12.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.641+0000 7fe4c1dab740 1 Processor -- start 2023-11-08T19:31:12.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.641+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abdec0 0x561803b46940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abe2a0 0x561803ad15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4c1dab740 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 0x561803ad1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803a91820 con 0x561803ad5ec0 2023-11-08T19:31:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x561803adcf40 con 0x561803abdec0 2023-11-08T19:31:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4c1dab740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x561803ad6850 con 0x561803abe2a0 2023-11-08T19:31:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4c1dab740 10 client.0 _ll_register_callbacks cb 0x7fe4a80036b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T19:31:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 0x561803ad1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 0x561803ad1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49926/0 (socket says 172.21.15.119:49926) 2023-11-08T19:31:12.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/1905118410 learned_addr learned my addr 172.21.15.119:0/1905118410 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.669 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abe2a0 msgr2=0x561803ad15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abe2a0 0x561803ad15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abdec0 msgr2=0x561803b46940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.670 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abdec0 0x561803b46940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 -- 172.21.15.119:0/1905118410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618039fae40 con 0x561803ad5ec0 2023-11-08T19:31:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4bd7b7640 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 0x561803ad1b20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0054b30 tx=0x7fe4b0058d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a813e43bc38d254b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.642+0000 7fe4977fe640 10 client.0 ms_handle_connect on v2:172.21.15.119:3300/0 2023-11-08T19:31:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4977fe640 1 -- 172.21.15.119:0/1905118410 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0058a90 con 0x561803ad5ec0 2023-11-08T19:31:12.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4977fe640 1 -- 172.21.15.119:0/1905118410 <== mon.0 v2:172.21.15.119:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fe4b0060930 con 0x561803ad5ec0 2023-11-08T19:31:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4a40143f0 con 0x561803ad5ec0 2023-11-08T19:31:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4977fe640 1 -- 172.21.15.119:0/1905118410 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe4b0057b20 con 0x561803ad5ec0 2023-11-08T19:31:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 10 client.7800 fetch_fsmap learned FSMap version 89 2023-11-08T19:31:12.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) v3 -- 0x5618039be180 con 0x561803ad5ec0 2023-11-08T19:31:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4977fe640 1 -- 172.21.15.119:0/1905118410 <== mon.0 v2:172.21.15.119:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a40143f0 con 0x561803ad5ec0 2023-11-08T19:31:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4977fe640 1 -- 172.21.15.119:0/1905118410 <== mon.0 v2:172.21.15.119:3300/0 5 ==== fsmap(e 89) v1 ==== 3306+0+0 (secure 0 0 0) 0x7fe4b0014070 con 0x561803ad5ec0 2023-11-08T19:31:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 10 client.7800 fetch_fsmap finished waiting for FSMap version 89 2023-11-08T19:31:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 10 client.7800 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T19:31:12.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 20 client.7800 populate_metadata read hostname 'smithi119' 2023-11-08T19:31:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aec6d0 0x561803aecab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 4 client.7800 mds_command: new command op to 7065 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T19:31:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.643+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 --> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5618039f54c0 con 0x561803aec6d0 2023-11-08T19:31:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.644+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aec6d0 0x561803aecab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.645+0000 7fe4bcfb6640 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aec6d0 0x561803aecab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd3571a54a9dfdbc server_cookie=c5c9bc2f679fcdf4 in_seq=0 out_seq=0 2023-11-08T19:31:12.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.645+0000 7fe4977fe640 10 client.7800 ms_handle_connect on v2:172.21.15.188:6832/4246462419 2023-11-08T19:31:12.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.645+0000 7fe4977fe640 1 -- 172.21.15.119:0/1905118410 <== mds.1 v2:172.21.15.188:6832/4246462419 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x5618039f54c0 con 0x561803aec6d0 2023-11-08T19:31:12.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.645+0000 7fe4977fe640 10 client.7800 handle_command_reply: tid=0 2023-11-08T19:31:12.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 client.7800 shutdown 2023-11-08T19:31:12.675 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 msgr2=0x561803ad1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 0x561803ad1b20 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0054b30 tx=0x7fe4b0058d30 comp rx=0 tx=0).stop 2023-11-08T19:31:12.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 shutdown_connections 2023-11-08T19:31:12.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x561803ad5ec0 0x561803ad1b20 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x561803abe2a0 0x561803ad15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.676 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:6832/4246462419,v1:172.21.15.188:6834/4246462419] conn(0x561803aec6d0 0x561803aecab0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 --2- 172.21.15.119:0/1905118410 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x561803abdec0 0x561803b46940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 >> 172.21.15.119:0/1905118410 conn(0x561803a91300 msgr2=0x561803ad4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4977fe640 0 client.7800 ms_handle_reset on v2:172.21.15.188:6832/4246462419 2023-11-08T19:31:12.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 shutdown_connections 2023-11-08T19:31:12.677 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 1 -- 172.21.15.119:0/1905118410 wait complete. 2023-11-08T19:31:12.678 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.646+0000 7fe4c1dab740 20 client.7800 trim_cache size 0 max 16384 2023-11-08T19:31:12.678 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:31:12.678 INFO:teuthology.orchestra.run.smithi119.stdout: "cluster_fsid": "ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d", 2023-11-08T19:31:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "whoami": 1, 2023-11-08T19:31:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "id": 7065, 2023-11-08T19:31:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "want_state": "up:active", 2023-11-08T19:31:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "state": "up:active", 2023-11-08T19:31:12.679 INFO:teuthology.orchestra.run.smithi119.stdout: "fs_name": "cephfs", 2023-11-08T19:31:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "rank_uptime": 123.966331333, 2023-11-08T19:31:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "mdsmap_epoch": 89, 2023-11-08T19:31:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch": 101, 2023-11-08T19:31:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "osdmap_epoch_barrier": 101, 2023-11-08T19:31:12.680 INFO:teuthology.orchestra.run.smithi119.stdout: "uptime": 147.019738184 2023-11-08T19:31:12.681 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:31:12.682 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe49c06ae70 msgr2=0x7fe49c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe49c06ae70 0x7fe49c06d330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac034000 tx=0x7fe4ac007000 comp rx=0 tx=0).stop 2023-11-08T19:31:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 msgr2=0x7fe4b80cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 0x7fe4b80cdca0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe4b80ce900 tx=0x7fe4b000f4a0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.683 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 shutdown_connections 2023-11-08T19:31:12.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe49c06ae70 0x7fe49c06d330 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe4b80c6110 0x7fe4b80cdca0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4b80c7630 0x7fe4b80c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 --2- 172.21.15.119:0/3901277821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4a8003970 0x7fe4b815a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.684 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 >> 172.21.15.119:0/3901277821 conn(0x7fe4b80cb610 msgr2=0x7fe4b815df90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 shutdown_connections 2023-11-08T19:31:12.685 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.647+0000 7fe4bf21a640 1 -- 172.21.15.119:0/3901277821 wait complete. 2023-11-08T19:31:12.685 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:12.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.894+0000 7fcd35ae1640 1 Processor -- start 2023-11-08T19:31:12.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.894+0000 7fcd35ae1640 1 -- start start 2023-11-08T19:31:12.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.895+0000 7fcd35ae1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcd30142f80 0x7fcd300cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.895+0000 7fcd35ae1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd3015d9f0 0x7fcd300c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.895+0000 7fcd35ae1640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd300c4960 con 0x7fcd3006cb40 2023-11-08T19:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd35ae1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd30143a40 con 0x7fcd3015d9f0 2023-11-08T19:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd35ae1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3015df60 con 0x7fcd30142f80 2023-11-08T19:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd2ffff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fcd3006cb40 0x7fcd300c4e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57084/0 (socket says 172.21.15.119:57084) 2023-11-08T19:31:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd2ffff640 1 -- 172.21.15.119:0/3385306122 learned_addr learned my addr 172.21.15.119:0/3385306122 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd27fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fcd3015d9f0 0x7fcd300c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd2f7fe640 1 --2- 172.21.15.119:0/3385306122 >> v2:172.21.15.188:3300/0 conn(0x7fcd30142f80 0x7fcd300cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd27fff640 1 -- 172.21.15.119:0/3385306122 >> v2:172.21.15.188:3300/0 conn(0x7fcd30142f80 msgr2=0x7fcd300cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:12.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd27fff640 1 --2- 172.21.15.119:0/3385306122 >> v2:172.21.15.188:3300/0 conn(0x7fcd30142f80 0x7fcd300cc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd27fff640 1 -- 172.21.15.119:0/3385306122 >> v1:172.21.15.119:6789/0 conn(0x7fcd3006cb40 legacy=0x7fcd300c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd27fff640 1 -- 172.21.15.119:0/3385306122 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3015e240 con 0x7fcd3015d9f0 2023-11-08T19:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.896+0000 7fcd2effd640 1 -- 172.21.15.119:0/3385306122 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008334674 0 0) 0x7fcd300c4960 con 0x7fcd3006cb40 2023-11-08T19:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.898+0000 7fcd27fff640 1 --2- 172.21.15.119:0/3385306122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3015d9f0 0x7fcd300c9cd0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fcd300c3b00 tx=0x7fcd1802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1105a852d44e6e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.898+0000 7fcd2effd640 1 -- 172.21.15.119:0/3385306122 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18031830 con 0x7fcd3015d9f0 2023-11-08T19:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.898+0000 7fcd2effd640 1 -- 172.21.15.119:0/3385306122 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18031990 con 0x7fcd3015d9f0 2023-11-08T19:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.898+0000 7fcd2effd640 1 -- 172.21.15.119:0/3385306122 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18031d20 con 0x7fcd3015d9f0 2023-11-08T19:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.898+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3385306122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3015d9f0 msgr2=0x7fcd300c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.898+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3385306122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3015d9f0 0x7fcd300c9cd0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fcd300c3b00 tx=0x7fcd1802b8d0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.899+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3385306122 shutdown_connections 2023-11-08T19:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.899+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3385306122 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3015d9f0 0x7fcd300c9cd0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.899+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3385306122 >> v2:172.21.15.188:3300/0 conn(0x7fcd30142f80 0x7fcd300cc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.899+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3385306122 >> 172.21.15.119:0/3385306122 conn(0x7fcd300cb610 msgr2=0x7fcd300cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.899+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3385306122 shutdown_connections 2023-11-08T19:31:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.899+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3385306122 wait complete. 2023-11-08T19:31:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.900+0000 7fcd35ae1640 1 Processor -- start 2023-11-08T19:31:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.900+0000 7fcd35ae1640 1 -- start start 2023-11-08T19:31:12.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.901+0000 7fcd35ae1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 0x7fcd3013d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.901+0000 7fcd35ae1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd30142f80 0x7fcd300b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.901+0000 7fcd35ae1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3015d9f0 0x7fcd3006bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.901+0000 7fcd35ae1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b3c40 con 0x7fcd3006cb40 2023-11-08T19:31:12.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.901+0000 7fcd35ae1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b3f20 con 0x7fcd30142f80 2023-11-08T19:31:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.901+0000 7fcd35ae1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd300b4200 con 0x7fcd3015d9f0 2023-11-08T19:31:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 0x7fcd3013d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 0x7fcd3013d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49944/0 (socket says 172.21.15.119:49944) 2023-11-08T19:31:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 -- 172.21.15.119:0/3765378959 learned_addr learned my addr 172.21.15.119:0/3765378959 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 -- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3015d9f0 msgr2=0x7fcd3006bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd27fff640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd30142f80 0x7fcd300b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3015d9f0 0x7fcd3006bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 -- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd30142f80 msgr2=0x7fcd300b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd30142f80 0x7fcd300b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 -- 172.21.15.119:0/3765378959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18009400 con 0x7fcd3006cb40 2023-11-08T19:31:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.902+0000 7fcd2f7fe640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 0x7fcd3013d730 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000d5f0 tx=0x7fcd2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.903+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd200125f0 con 0x7fcd3006cb40 2023-11-08T19:31:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.903+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd20012750 con 0x7fcd3006cb40 2023-11-08T19:31:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.903+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd20012b00 con 0x7fcd3006cb40 2023-11-08T19:31:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.903+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd300b44e0 con 0x7fcd3006cb40 2023-11-08T19:31:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.903+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3013dc70 con 0x7fcd3006cb40 2023-11-08T19:31:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.904+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3015d4d0 con 0x7fcd3006cb40 2023-11-08T19:31:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.905+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd20012c60 con 0x7fcd3006cb40 2023-11-08T19:31:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.906+0000 7fcd2cff9640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1006ae70 0x7fcd1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.906+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7fcd2008a2a0 con 0x7fcd3006cb40 2023-11-08T19:31:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.906+0000 7fcd27fff640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1006ae70 0x7fcd1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.907+0000 7fcd27fff640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1006ae70 0x7fcd1006d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcd18036000 tx=0x7fcd18008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:12.910+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd20054b90 con 0x7fcd3006cb40 2023-11-08T19:31:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.137+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd30074430 con 0x7fcd3006cb40 2023-11-08T19:31:13.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.138+0000 7fcd2cff9640 1 -- 172.21.15.119:0/3765378959 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fcd200581b0 con 0x7fcd3006cb40 2023-11-08T19:31:13.140 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.140 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:08.250393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:13.141 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 89 2023-11-08T19:31:13.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1006ae70 msgr2=0x7fcd1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1006ae70 0x7fcd1006d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fcd18036000 tx=0x7fcd18008030 comp rx=0 tx=0).stop 2023-11-08T19:31:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 msgr2=0x7fcd3013d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 0x7fcd3013d730 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fcd2000d5f0 tx=0x7fcd2000dab0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 shutdown_connections 2023-11-08T19:31:13.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fcd1006ae70 0x7fcd1006d330 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd3015d9f0 0x7fcd3006bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd30142f80 0x7fcd300b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 --2- 172.21.15.119:0/3765378959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fcd3006cb40 0x7fcd3013d730 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 >> 172.21.15.119:0/3765378959 conn(0x7fcd300cb610 msgr2=0x7fcd300cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:13.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 shutdown_connections 2023-11-08T19:31:13.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.141+0000 7fcd35ae1640 1 -- 172.21.15.119:0/3765378959 wait complete. 2023-11-08T19:31:13.161 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-08T19:31:13.402 INFO:teuthology.orchestra.run.smithi188.stdout:{"last_snap":5,"last_created":5,"last_destroyed":1,"pending_noop":[],"snaps":[{"snapid":2,"ino":1099511627777,"stamp":"2023-11-08T19:29:23.170262+0000","name":"s11","metadata":{}},{"snapid":3,"ino":1099511627777,"stamp":"2023-11-08T19:29:51.023531+0000","name":"s12","metadata":{}},{"snapid":4,"ino":1099511627777,"stamp":"2023-11-08T19:30:19.457611+0000","name":"s15","metadata":{}},{"snapid":5,"ino":1099511627777,"stamp":"2023-11-08T19:30:47.226840+0000","name":"s16","metadata":{}}],"need_to_purge":{},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:31:13.402 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 1, "last_snap": 5, "need_to_purge": {}, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s11", "snapid": 2, "stamp": "2023-11-08T19:29:23.170262+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s12", "snapid": 3, "stamp": "2023-11-08T19:29:51.023531+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s15", "snapid": 4, "stamp": "2023-11-08T19:30:19.457611+0000" }, { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2023-11-08T19:30:47.226840+0000" } ] } 2023-11-08T19:31:13.403 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:31:13.404 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:31:13.413 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:31:13.473 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:31:13.471+0000 7fb766e4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:31:13.474 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:31:13.471+0000 7fb766e4f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:31:13.582 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2023-11-08T19:31:13.583 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2023-11-08T19:31:13.583 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:31:13.583 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2023-11-08T19:31:13.583 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.584 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2023-11-08T19:31:13.584 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.584 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2023-11-08T19:31:13.584 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.584 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2023-11-08T19:31:13.585 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2023-11-08T19:31:13.585 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.585 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2023-11-08T19:31:13.585 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2023-11-08T19:31:13.585 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.585 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:31:13.586 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2023-11-08T19:31:13.586 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.586 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:31:13.586 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:31:13.587 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.587 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2023-11-08T19:31:13.587 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2023-11-08T19:31:13.587 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2023-11-08T19:31:13.588 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:31:13.588 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:31:13.588 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2023-11-08T19:31:13.588 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:31:13.588 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2023-11-08T19:31:13.588 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:31:13.589 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:31:13.589 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2023-11-08T19:31:13.589 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.589 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:31:13.589 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.590 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2023-11-08T19:31:13.590 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.590 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2023-11-08T19:31:13.590 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.590 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2023-11-08T19:31:13.591 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2023-11-08T19:31:13.591 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2023-11-08T19:31:13.591 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-08T19:31:13.591 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2023-11-08T19:31:13.591 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:31:13.592 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.592 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2023-11-08T19:31:13.592 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2023-11-08T19:31:13.593 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.593 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2023-11-08T19:31:13.593 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2023-11-08T19:31:13.593 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:31:13.593 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.593 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2023-11-08T19:31:13.594 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2023-11-08T19:31:13.594 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.594 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2023-11-08T19:31:13.594 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2023-11-08T19:31:13.594 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2023-11-08T19:31:13.595 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2023-11-08T19:31:13.595 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.595 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2023-11-08T19:31:13.595 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2023-11-08T19:31:13.595 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.596 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:31:13.596 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.596 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2023-11-08T19:31:13.596 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.596 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2023-11-08T19:31:13.596 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.597 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:31:13.597 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2023-11-08T19:31:13.597 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.597 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2023-11-08T19:31:13.597 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2023-11-08T19:31:13.598 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2023-11-08T19:31:13.598 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2023-11-08T19:31:13.598 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2023-11-08T19:31:13.598 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2023-11-08T19:31:13.599 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:31:13.599 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2023-11-08T19:31:13.599 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2023-11-08T19:31:13.599 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:31:13.599 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2023-11-08T19:31:13.600 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:31:13.600 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:31:13.600 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2023-11-08T19:31:13.600 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-08T19:31:13.600 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.601 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2023-11-08T19:31:13.601 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:31:13.601 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2023-11-08T19:31:13.601 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-08T19:31:13.601 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-08T19:31:13.601 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:13.602 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:13.602 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2023-11-08T19:31:13.602 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.603 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2023-11-08T19:31:13.603 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:31:13.603 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-08T19:31:13.603 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2023-11-08T19:31:13.603 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-08T19:31:13.604 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2023-11-08T19:31:13.604 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2023-11-08T19:31:13.604 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:13.604 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2023-11-08T19:31:13.604 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2023-11-08T19:31:13.605 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:13.605 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.605 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:31:13.605 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:31:13.605 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:31:13.606 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.606 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:31:13.606 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:31:13.606 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:31:13.606 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.607 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2023-11-08T19:31:13.607 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:31:13.607 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.607 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:31:13.607 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2023-11-08T19:31:13.607 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:31:13.608 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2023-11-08T19:31:13.608 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2023-11-08T19:31:13.608 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2023-11-08T19:31:13.608 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2023-11-08T19:31:13.608 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:31:13.609 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:31:13.609 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2023-11-08T19:31:13.609 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2023-11-08T19:31:13.609 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:31:13.609 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2023-11-08T19:31:13.610 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-08T19:31:13.610 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-08T19:31:13.610 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2023-11-08T19:31:13.611 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2023-11-08T19:31:13.611 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2023-11-08T19:31:13.611 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-08T19:31:13.611 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-08T19:31:13.611 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-08T19:31:13.612 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-08T19:31:13.612 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2023-11-08T19:31:13.612 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2023-11-08T19:31:13.612 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2023-11-08T19:31:13.613 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:31:13.613 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2023-11-08T19:31:13.613 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2023-11-08T19:31:13.613 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2023-11-08T19:31:13.613 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2023-11-08T19:31:13.614 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-08T19:31:13.614 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-08T19:31:13.614 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-08T19:31:13.614 INFO:teuthology.orchestra.run.smithi188.stdout: mds_reconnect_timeout = 5 2023-11-08T19:31:13.615 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-11-08T19:31:13.615 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2023-11-08T19:31:13.615 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:31:13.615 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2023-11-08T19:31:13.616 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.616 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2023-11-08T19:31:13.616 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.616 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2023-11-08T19:31:13.616 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.617 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2023-11-08T19:31:13.617 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2023-11-08T19:31:13.617 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.617 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2023-11-08T19:31:13.617 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2023-11-08T19:31:13.618 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.618 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:31:13.618 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2023-11-08T19:31:13.618 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.618 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:31:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:31:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2023-11-08T19:31:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.619 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2023-11-08T19:31:13.620 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2023-11-08T19:31:13.620 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:31:13.620 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:31:13.620 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2023-11-08T19:31:13.620 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:31:13.621 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2023-11-08T19:31:13.621 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:31:13.621 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:31:13.621 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2023-11-08T19:31:13.621 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.622 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:31:13.622 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.622 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2023-11-08T19:31:13.622 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.622 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2023-11-08T19:31:13.623 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.623 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2023-11-08T19:31:13.623 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2023-11-08T19:31:13.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2023-11-08T19:31:13.624 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-08T19:31:13.624 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2023-11-08T19:31:13.624 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:31:13.624 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.625 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2023-11-08T19:31:13.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2023-11-08T19:31:13.625 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2023-11-08T19:31:13.625 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2023-11-08T19:31:13.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:31:13.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2023-11-08T19:31:13.626 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2023-11-08T19:31:13.626 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2023-11-08T19:31:13.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2023-11-08T19:31:13.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2023-11-08T19:31:13.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2023-11-08T19:31:13.627 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.627 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2023-11-08T19:31:13.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2023-11-08T19:31:13.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:31:13.628 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.628 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2023-11-08T19:31:13.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.629 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2023-11-08T19:31:13.629 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.629 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:31:13.629 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2023-11-08T19:31:13.630 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.630 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2023-11-08T19:31:13.630 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2023-11-08T19:31:13.630 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2023-11-08T19:31:13.630 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2023-11-08T19:31:13.631 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2023-11-08T19:31:13.631 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2023-11-08T19:31:13.631 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:31:13.631 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2023-11-08T19:31:13.631 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2023-11-08T19:31:13.631 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:31:13.632 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2023-11-08T19:31:13.632 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:31:13.632 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:31:13.632 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2023-11-08T19:31:13.632 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-11-08T19:31:13.633 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.633 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2023-11-08T19:31:13.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:31:13.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2023-11-08T19:31:13.633 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-11-08T19:31:13.634 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-11-08T19:31:13.634 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:13.634 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:13.635 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2023-11-08T19:31:13.635 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.635 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2023-11-08T19:31:13.635 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:31:13.635 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-11-08T19:31:13.636 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2023-11-08T19:31:13.636 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-11-08T19:31:13.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2023-11-08T19:31:13.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2023-11-08T19:31:13.636 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:31:13.637 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2023-11-08T19:31:13.637 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2023-11-08T19:31:13.637 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:31:13.637 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.637 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:31:13.637 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:31:13.638 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:31:13.638 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.638 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:31:13.638 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:31:13.638 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:31:13.639 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.639 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2023-11-08T19:31:13.639 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:31:13.639 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.639 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:31:13.640 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2023-11-08T19:31:13.640 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:13.640 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2023-11-08T19:31:13.640 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2023-11-08T19:31:13.640 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2023-11-08T19:31:13.641 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2023-11-08T19:31:13.641 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:31:13.641 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:31:13.641 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2023-11-08T19:31:13.641 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-11-08T19:31:13.641 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:31:13.642 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2023-11-08T19:31:13.642 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-11-08T19:31:13.642 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-11-08T19:31:13.642 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2023-11-08T19:31:13.642 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2023-11-08T19:31:13.643 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2023-11-08T19:31:13.643 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-11-08T19:31:13.643 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-11-08T19:31:13.643 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-11-08T19:31:13.643 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-11-08T19:31:13.644 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2023-11-08T19:31:13.644 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2023-11-08T19:31:13.644 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2023-11-08T19:31:13.644 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:31:13.644 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2023-11-08T19:31:13.644 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2023-11-08T19:31:13.645 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2023-11-08T19:31:13.645 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2023-11-08T19:31:13.645 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-11-08T19:31:13.645 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-11-08T19:31:13.645 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-11-08T19:31:13.646 INFO:teuthology.orchestra.run.smithi119.stdout: mds_reconnect_timeout = 5 2023-11-08T19:31:13.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:13.648 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir d1/dir/.snap/*) 2023-11-08T19:31:13.772 INFO:tasks.cephfs.test_snapshots:testing snapserver mds_kill_mdstable_at=2, also kill snapclient 2023-11-08T19:31:13.773 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:13.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.985+0000 7feec7e1a640 1 Processor -- start 2023-11-08T19:31:13.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.985+0000 7feec7e1a640 1 -- start start 2023-11-08T19:31:13.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec7e1a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feec0159d80 0x7feec00ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec7e1a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feec00703a0 0x7feec015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec7e1a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec015dd60 con 0x7feec00cb2e0 2023-11-08T19:31:13.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec7e1a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec015da30 con 0x7feec0159d80 2023-11-08T19:31:13.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec7e1a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec00ca8b0 con 0x7feec00703a0 2023-11-08T19:31:13.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feec0159d80 0x7feec00ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7feec0159d80 0x7feec00ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49958/0 (socket says 172.21.15.119:49958) 2023-11-08T19:31:13.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 -- 172.21.15.119:0/3001239180 learned_addr learned my addr 172.21.15.119:0/3001239180 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:13.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 -- 172.21.15.119:0/3001239180 >> v2:172.21.15.188:3300/0 conn(0x7feec00703a0 msgr2=0x7feec015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:13.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 --2- 172.21.15.119:0/3001239180 >> v2:172.21.15.188:3300/0 conn(0x7feec00703a0 0x7feec015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 -- 172.21.15.119:0/3001239180 >> v1:172.21.15.188:6789/0 conn(0x7feec00cb2e0 legacy=0x7feec006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:13.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.987+0000 7feec5bb6640 1 -- 172.21.15.119:0/3001239180 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00cab90 con 0x7feec0159d80 2023-11-08T19:31:13.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.988+0000 7feec5bb6640 1 --2- 172.21.15.119:0/3001239180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec0159d80 0x7feec00ca080 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feec006dc40 tx=0x7feebc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86947b2f61f6fe2c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:13.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.989+0000 7feec4bb4640 1 -- 172.21.15.119:0/3001239180 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc031870 con 0x7feec0159d80 2023-11-08T19:31:13.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.989+0000 7feec4bb4640 1 -- 172.21.15.119:0/3001239180 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feebc0319d0 con 0x7feec0159d80 2023-11-08T19:31:13.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.989+0000 7feec4bb4640 1 -- 172.21.15.119:0/3001239180 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feebc031d60 con 0x7feec0159d80 2023-11-08T19:31:13.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.989+0000 7feec7e1a640 1 -- 172.21.15.119:0/3001239180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec0159d80 msgr2=0x7feec00ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:13.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.989+0000 7feec7e1a640 1 --2- 172.21.15.119:0/3001239180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec0159d80 0x7feec00ca080 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7feec006dc40 tx=0x7feebc02b910 comp rx=0 tx=0).stop 2023-11-08T19:31:13.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.990+0000 7feec7e1a640 1 -- 172.21.15.119:0/3001239180 shutdown_connections 2023-11-08T19:31:13.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.990+0000 7feec7e1a640 1 --2- 172.21.15.119:0/3001239180 >> v2:172.21.15.188:3300/0 conn(0x7feec00703a0 0x7feec015e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.990+0000 7feec7e1a640 1 --2- 172.21.15.119:0/3001239180 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec0159d80 0x7feec00ca080 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:13.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.990+0000 7feec7e1a640 1 -- 172.21.15.119:0/3001239180 >> 172.21.15.119:0/3001239180 conn(0x7feec00b06e0 msgr2=0x7feec00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:13.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.990+0000 7feec7e1a640 1 -- 172.21.15.119:0/3001239180 shutdown_connections 2023-11-08T19:31:13.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.990+0000 7feec7e1a640 1 -- 172.21.15.119:0/3001239180 wait complete. 2023-11-08T19:31:13.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.991+0000 7feec7e1a640 1 Processor -- start 2023-11-08T19:31:13.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 -- start start 2023-11-08T19:31:13.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feec00703a0 0x7feec014cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 0x7feec014dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feec0159d80 0x7feec00cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:13.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec015b2e0 con 0x7feec00cb2e0 2023-11-08T19:31:13.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec014d2f0 con 0x7feec0159d80 2023-11-08T19:31:13.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.992+0000 7feec7e1a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c7330 con 0x7feec00703a0 2023-11-08T19:31:13.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 0x7feec014dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 0x7feec014dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49962/0 (socket says 172.21.15.119:49962) 2023-11-08T19:31:13.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 -- 172.21.15.119:0/467052529 learned_addr learned my addr 172.21.15.119:0/467052529 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:13.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec63b7640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feec0159d80 0x7feec00cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec5bb6640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feec00703a0 0x7feec014cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:13.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 -- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feec00703a0 msgr2=0x7feec014cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feec00703a0 0x7feec014cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 -- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feec0159d80 msgr2=0x7feec00cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feec0159d80 0x7feec00cebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.993+0000 7feec53b5640 1 -- 172.21.15.119:0/467052529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0094d0 con 0x7feec00cb2e0 2023-11-08T19:31:14.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.994+0000 7feec53b5640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 0x7feec014dee0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feeb000b4a0 tx=0x7feeb000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.994+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0010610 con 0x7feec00cb2e0 2023-11-08T19:31:14.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.994+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb0010770 con 0x7feec00cb2e0 2023-11-08T19:31:14.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.994+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0010b20 con 0x7feec00cb2e0 2023-11-08T19:31:14.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.994+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec015b4f0 con 0x7feec00cb2e0 2023-11-08T19:31:14.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.994+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec015bab0 con 0x7feec00cb2e0 2023-11-08T19:31:14.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.995+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec01595e0 con 0x7feec00cb2e0 2023-11-08T19:31:14.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.996+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feeb0010c80 con 0x7feec00cb2e0 2023-11-08T19:31:14.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.997+0000 7feeb67fc640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feea406ae70 0x7feea406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.998+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7feeb0089a30 con 0x7feec00cb2e0 2023-11-08T19:31:14.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.998+0000 7feec5bb6640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feea406ae70 0x7feea406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:13.998+0000 7feec5bb6640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feea406ae70 0x7feea406d330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7feebc036000 tx=0x7feebc03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.002+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feeb0054320 con 0x7feec00cb2e0 2023-11-08T19:31:14.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.231+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feec004e490 con 0x7feec00cb2e0 2023-11-08T19:31:14.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.232+0000 7feeb67fc640 1 -- 172.21.15.119:0/467052529 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+4944 (secure 0 0 0) 0x7feeb0057940 con 0x7feec00cb2e0 2023-11-08T19:31:14.234 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:14.234 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":89,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:08.250393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:14.234 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 89 2023-11-08T19:31:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feea406ae70 msgr2=0x7feea406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feea406ae70 0x7feea406d330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7feebc036000 tx=0x7feebc03c040 comp rx=0 tx=0).stop 2023-11-08T19:31:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 msgr2=0x7feec014dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 0x7feec014dee0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7feeb000b4a0 tx=0x7feeb000b960 comp rx=0 tx=0).stop 2023-11-08T19:31:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 shutdown_connections 2023-11-08T19:31:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feea406ae70 0x7feea406d330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feec0159d80 0x7feec00cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feec00cb2e0 0x7feec014dee0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 --2- 172.21.15.119:0/467052529 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feec00703a0 0x7feec014cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 >> 172.21.15.119:0/467052529 conn(0x7feec00b06e0 msgr2=0x7feec00ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 shutdown_connections 2023-11-08T19:31:14.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.235+0000 7feec7e1a640 1 -- 172.21.15.119:0/467052529 wait complete. 2023-11-08T19:31:14.256 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json dump snaps --server 2023-11-08T19:31:14.495 INFO:teuthology.orchestra.run.smithi188.stdout:{"last_snap":9,"last_created":5,"last_destroyed":8,"pending_noop":[],"snaps":[{"snapid":5,"ino":1099511627777,"stamp":"2023-11-08T19:30:47.226840+0000","name":"s16","metadata":{}}],"need_to_purge":{"17":[2,3,4,6,7,8]},"pending_update":[],"pending_destroy":[{"version":16,"removed_snap":5,"seq":9}]} 2023-11-08T19:31:14.496 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 8, "last_snap": 9, "need_to_purge": { "17": [ 2, 3, 4, 6, 7, 8 ] }, "pending_destroy": [ { "removed_snap": 5, "seq": 9, "version": 16 } ], "pending_noop": [], "pending_update": [], "snaps": [ { "ino": 1099511627777, "metadata": {}, "name": "s16", "snapid": 5, "stamp": "2023-11-08T19:30:47.226840+0000" } ] } 2023-11-08T19:31:14.497 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:0 true 2023-11-08T19:31:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.720+0000 7f37749c9640 1 Processor -- start 2023-11-08T19:31:14.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.721+0000 7f37749c9640 1 -- start start 2023-11-08T19:31:14.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.722+0000 7f37749c9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3770140180 0x7f377015e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.722+0000 7f37749c9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377015e700 con 0x7f37700b43a0 2023-11-08T19:31:14.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.722+0000 7f37749c9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f377015dd30 con 0x7f3770142940 2023-11-08T19:31:14.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.722+0000 7f37749c9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377006de50 con 0x7f3770140180 2023-11-08T19:31:14.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.722+0000 7f376e59c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3770140180 0x7f377015e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.723+0000 7f376ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f37700b43a0 0x7f37701408b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57088/0 (socket says 172.21.15.119:57088) 2023-11-08T19:31:14.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.723+0000 7f376ed9d640 1 -- 172.21.15.119:0/3953455488 learned_addr learned my addr 172.21.15.119:0/3953455488 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:14.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.723+0000 7f376e59c640 1 -- 172.21.15.119:0/3953455488 >> v1:172.21.15.188:6789/0 conn(0x7f3770142940 legacy=0x7f37700b3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:14.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.723+0000 7f376e59c640 1 -- 172.21.15.119:0/3953455488 >> v1:172.21.15.119:6789/0 conn(0x7f37700b43a0 legacy=0x7f37701408b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.723+0000 7f376e59c640 1 -- 172.21.15.119:0/3953455488 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377015d700 con 0x7f3770140180 2023-11-08T19:31:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.723+0000 7f376d59a640 1 -- 172.21.15.119:0/3953455488 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086229483 0 0) 0x7f3768001160 con 0x7f37700b43a0 2023-11-08T19:31:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.724+0000 7f376e59c640 1 --2- 172.21.15.119:0/3953455488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770140180 0x7f377015e1c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f3760009d90 tx=0x7f376002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d2631ae388d362 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.724+0000 7f376d59a640 1 -- 172.21.15.119:0/3953455488 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376002cc60 con 0x7f3770140180 2023-11-08T19:31:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f376d59a640 1 -- 172.21.15.119:0/3953455488 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376002cdc0 con 0x7f3770140180 2023-11-08T19:31:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f376d59a640 1 -- 172.21.15.119:0/3953455488 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760034660 con 0x7f3770140180 2023-11-08T19:31:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 -- 172.21.15.119:0/3953455488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770140180 msgr2=0x7f377015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 --2- 172.21.15.119:0/3953455488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770140180 0x7f377015e1c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f3760009d90 tx=0x7f376002beb0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 -- 172.21.15.119:0/3953455488 shutdown_connections 2023-11-08T19:31:14.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 --2- 172.21.15.119:0/3953455488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770140180 0x7f377015e1c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 -- 172.21.15.119:0/3953455488 >> 172.21.15.119:0/3953455488 conn(0x7f37700b2990 msgr2=0x7f37700b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:14.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 -- 172.21.15.119:0/3953455488 shutdown_connections 2023-11-08T19:31:14.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.725+0000 7f37749c9640 1 -- 172.21.15.119:0/3953455488 wait complete. 2023-11-08T19:31:14.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.727+0000 7f37749c9640 1 Processor -- start 2023-11-08T19:31:14.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.727+0000 7f37749c9640 1 -- start start 2023-11-08T19:31:14.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f37749c9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37700b43a0 0x7f3770146190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f37749c9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3770140180 0x7f377006c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f37749c9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 0x7f37701587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f37749c9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3770145d70 con 0x7f3770142940 2023-11-08T19:31:14.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f37749c9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37701f3a90 con 0x7f37700b43a0 2023-11-08T19:31:14.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f37749c9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377013e2b0 con 0x7f3770140180 2023-11-08T19:31:14.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f376ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 0x7f37701587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f376ed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 0x7f37701587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:49978/0 (socket says 172.21.15.119:49978) 2023-11-08T19:31:14.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.728+0000 7f376ed9d640 1 -- 172.21.15.119:0/3110296610 learned_addr learned my addr 172.21.15.119:0/3110296610 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:14.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376e59c640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37700b43a0 0x7f3770146190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376dd9b640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3770140180 0x7f377006c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376ed9d640 1 -- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3770140180 msgr2=0x7f377006c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376ed9d640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3770140180 0x7f377006c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376ed9d640 1 -- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37700b43a0 msgr2=0x7f3770146190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:14.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376ed9d640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37700b43a0 0x7f3770146190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:14.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376ed9d640 1 -- 172.21.15.119:0/3110296610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3760009a70 con 0x7f3770142940 2023-11-08T19:31:14.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f376ed9d640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 0x7f37701587d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f376800c410 tx=0x7f3768001ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.729+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37680074e0 con 0x7f3770142940 2023-11-08T19:31:14.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.730+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3768007640 con 0x7f3770142940 2023-11-08T19:31:14.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.730+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37701450f0 con 0x7f3770142940 2023-11-08T19:31:14.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.730+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3768004a70 con 0x7f3770142940 2023-11-08T19:31:14.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.730+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37701456b0 con 0x7f3770142940 2023-11-08T19:31:14.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.731+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3770145eb0 con 0x7f3770142940 2023-11-08T19:31:14.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.732+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3768004bd0 con 0x7f3770142940 2023-11-08T19:31:14.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.733+0000 7f3766ffd640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375006ae70 0x7f375006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:14.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.733+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f3768089950 con 0x7f3770142940 2023-11-08T19:31:14.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.733+0000 7f376e59c640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375006ae70 0x7f375006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:14.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.734+0000 7f376e59c640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375006ae70 0x7f375006d330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3760036000 tx=0x7f3760030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:14.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:14.737+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3770145eb0 con 0x7f3770142940 2023-11-08T19:31:15.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.002+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"} v 0) v1 -- 0x7f377015e700 con 0x7f3770142940 2023-11-08T19:31:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.741+0000 7f3766ffd640 1 -- 172.21.15.119:0/3110296610 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:0", "val": "true"}]=0 freezing mds.5210 v90) v1 ==== 112+0+0 (secure 0 0 0) 0x7f377015e700 con 0x7f3770142940 2023-11-08T19:31:15.743 INFO:teuthology.orchestra.run.smithi119.stderr:freezing mds.5210 2023-11-08T19:31:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.746+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375006ae70 msgr2=0x7f375006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375006ae70 0x7f375006d330 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f3760036000 tx=0x7f3760030000 comp rx=0 tx=0).stop 2023-11-08T19:31:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 msgr2=0x7f37701587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:15.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 0x7f37701587d0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f376800c410 tx=0x7f3768001ec0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 shutdown_connections 2023-11-08T19:31:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f375006ae70 0x7f375006d330 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3770142940 0x7f37701587d0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3770140180 0x7f377006c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 --2- 172.21.15.119:0/3110296610 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f37700b43a0 0x7f3770146190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 >> 172.21.15.119:0/3110296610 conn(0x7f37700b2990 msgr2=0x7f377015f330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 shutdown_connections 2023-11-08T19:31:15.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.747+0000 7f37749c9640 1 -- 172.21.15.119:0/3110296610 wait complete. 2023-11-08T19:31:15.772 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds freeze 6:1 true 2023-11-08T19:31:15.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.988+0000 7f278976c640 1 Processor -- start 2023-11-08T19:31:15.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.989+0000 7f278976c640 1 -- start start 2023-11-08T19:31:15.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.990+0000 7f278976c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 0x7f27840c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:15.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.990+0000 7f278976c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27840ca090 con 0x7f27841616b0 2023-11-08T19:31:15.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.990+0000 7f278976c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278406f9b0 con 0x7f27840cbaa0 2023-11-08T19:31:15.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.990+0000 7f278976c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2784070250 con 0x7f27841612d0 2023-11-08T19:31:15.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.990+0000 7f27827fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f27841616b0 0x7f27840cb390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57098/0 (socket says 172.21.15.119:57098) 2023-11-08T19:31:15.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.990+0000 7f27827fc640 1 -- 172.21.15.119:0/1902688393 learned_addr learned my addr 172.21.15.119:0/1902688393 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2782ffd640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 0x7f27840c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:15.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762483416 0 0) 0x7f27840ca090 con 0x7f27841616b0 2023-11-08T19:31:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f276c003520 con 0x7f27841616b0 2023-11-08T19:31:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 980242791 0 0) 0x7f276c003520 con 0x7f27841616b0 2023-11-08T19:31:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 msgr2=0x7f27840c9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 0x7f27840c9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 >> v1:172.21.15.188:6789/0 conn(0x7f27840cbaa0 legacy=0x7f27840cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278406fc90 con 0x7f27841616b0 2023-11-08T19:31:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2778001670 con 0x7f27841616b0 2023-11-08T19:31:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 >> v1:172.21.15.119:6789/0 conn(0x7f27841616b0 legacy=0x7f27840cb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0039a0 0x7f276c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:15.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f276c0047c0 0x7f276c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 0x7f276c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c00ba80 con 0x7f276c0047c0 2023-11-08T19:31:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c00bbf0 con 0x7f276c0039a0 2023-11-08T19:31:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.991+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f276c00bdb0 con 0x7f27841612d0 2023-11-08T19:31:15.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27827fc640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f276c0047c0 0x7f276c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f2782ffd640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0039a0 0x7f276c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27837fe640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 0x7f276c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27827fc640 1 -- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 msgr2=0x7f276c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27827fc640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27841612d0 0x7f276c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27827fc640 1 -- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0039a0 msgr2=0x7f276c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:15.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27827fc640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0039a0 0x7f276c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.992+0000 7f27827fc640 1 -- 172.21.15.119:0/1902688393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f276c003520 con 0x7f276c0047c0 2023-11-08T19:31:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.993+0000 7f27827fc640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f276c0047c0 0x7f276c004c40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f277800c210 tx=0x7f276c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40789eb55c15c80c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.993+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2778005ad0 con 0x7f276c0047c0 2023-11-08T19:31:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.993+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2778005c30 con 0x7f276c0047c0 2023-11-08T19:31:16.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.993+0000 7f2781ffb640 1 -- 172.21.15.119:0/1902688393 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277802c890 con 0x7f276c0047c0 2023-11-08T19:31:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f278976c640 1 -- 172.21.15.119:0/1902688393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f276c0047c0 msgr2=0x7f276c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f278976c640 1 --2- 172.21.15.119:0/1902688393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f276c0047c0 0x7f276c004c40 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f277800c210 tx=0x7f276c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f27837fe640 1 -- 172.21.15.119:0/1902688393 reap_dead start 2023-11-08T19:31:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f278976c640 1 -- 172.21.15.119:0/1902688393 shutdown_connections 2023-11-08T19:31:16.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f278976c640 1 -- 172.21.15.119:0/1902688393 >> 172.21.15.119:0/1902688393 conn(0x7f27840b06e0 msgr2=0x7f27840b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f278976c640 1 -- 172.21.15.119:0/1902688393 shutdown_connections 2023-11-08T19:31:16.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.994+0000 7f278976c640 1 -- 172.21.15.119:0/1902688393 wait complete. 2023-11-08T19:31:16.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.996+0000 7f278976c640 1 Processor -- start 2023-11-08T19:31:16.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.996+0000 7f278976c640 1 -- start start 2023-11-08T19:31:16.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f278976c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0047c0 0x7f27840c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f278976c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27840b35e0 0x7f27840c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f278976c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 0x7f278414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f278976c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840c7dc0 con 0x7f27840b3a40 2023-11-08T19:31:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f278976c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27840c5280 con 0x7f276c0047c0 2023-11-08T19:31:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f278976c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27840c7140 con 0x7f27840b35e0 2023-11-08T19:31:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f27837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 0x7f278414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f27837fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 0x7f278414ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50004/0 (socket says 172.21.15.119:50004) 2023-11-08T19:31:16.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f27837fe640 1 -- 172.21.15.119:0/2569481415 learned_addr learned my addr 172.21.15.119:0/2569481415 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:16.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f2782ffd640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0047c0 0x7f27840c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.997+0000 7f27827fc640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27840b35e0 0x7f27840c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f27837fe640 1 -- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27840b35e0 msgr2=0x7f27840c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f27837fe640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27840b35e0 0x7f27840c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f27837fe640 1 -- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0047c0 msgr2=0x7f27840c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f27837fe640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0047c0 0x7f27840c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f27837fe640 1 -- 172.21.15.119:0/2569481415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27840c7420 con 0x7f27840b3a40 2023-11-08T19:31:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f27837fe640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 0x7f278414ab20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2784070e80 tx=0x7f277c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c003b70 con 0x7f27840b3a40 2023-11-08T19:31:16.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c003cd0 con 0x7f27840b3a40 2023-11-08T19:31:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c0052b0 con 0x7f27840b3a40 2023-11-08T19:31:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27840c7700 con 0x7f27840b3a40 2023-11-08T19:31:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:15.998+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27841581f0 con 0x7f27840b3a40 2023-11-08T19:31:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.000+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27840c7f00 con 0x7f27840b3a40 2023-11-08T19:31:16.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.001+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f277c005410 con 0x7f27840b3a40 2023-11-08T19:31:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.002+0000 7f276b7fe640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f277006aea0 0x7f277006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.002+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f277c0897e0 con 0x7f27840b3a40 2023-11-08T19:31:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.002+0000 7f2782ffd640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f277006aea0 0x7f277006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:16.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.003+0000 7f2782ffd640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f277006aea0 0x7f277006d360 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2774006fd0 tx=0x7f2774008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:16.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.006+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27840c7f00 con 0x7f27840b3a40 2023-11-08T19:31:16.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.257+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"} v 0) v1 -- 0x7f27840c7f00 con 0x7f27840b3a40 2023-11-08T19:31:16.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.762+0000 7f276b7fe640 1 -- 172.21.15.119:0/2569481415 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds freeze", "role_or_gid": "6:1", "val": "true"}]=0 freezing mds.7065 v91) v1 ==== 112+0+0 (secure 0 0 0) 0x7f27840c7f00 con 0x7f27840b3a40 2023-11-08T19:31:16.764 INFO:teuthology.orchestra.run.smithi119.stderr:freezing mds.7065 2023-11-08T19:31:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f277006aea0 msgr2=0x7f277006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f277006aea0 0x7f277006d360 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f2774006fd0 tx=0x7f2774008040 comp rx=0 tx=0).stop 2023-11-08T19:31:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 msgr2=0x7f278414ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 0x7f278414ab20 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f2784070e80 tx=0x7f277c00dc00 comp rx=0 tx=0).stop 2023-11-08T19:31:16.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 shutdown_connections 2023-11-08T19:31:16.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f277006aea0 0x7f277006d360 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27840b3a40 0x7f278414ab20 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27840b35e0 0x7f27840c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 --2- 172.21.15.119:0/2569481415 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f276c0047c0 0x7f27840c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:16.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 >> 172.21.15.119:0/2569481415 conn(0x7f27840b06e0 msgr2=0x7f27840b17a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:16.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 shutdown_connections 2023-11-08T19:31:16.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:16.767+0000 7f278976c640 1 -- 172.21.15.119:0/2569481415 wait complete. 2023-11-08T19:31:16.790 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json config set mds_kill_mdstable_at 2 2023-11-08T19:31:17.018 INFO:teuthology.orchestra.run.smithi188.stdout:{"success":"mds_kill_mdstable_at = '2' (not observed, change may require restart) "} 2023-11-08T19:31:17.019 DEBUG:tasks.cephfs.filesystem:_json_asok output { "success": "mds_kill_mdstable_at = '2' (not observed, change may require restart) " } 2023-11-08T19:31:17.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:17.020 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22) 2023-11-08T19:31:17.029 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2023-11-08T19:31:17.084 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb763648640 time 2023-11-08T19:31:17.084236+0000 2023-11-08T19:31:17.084 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:31:17.085 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.086 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fb76f16efaf] 2023-11-08T19:31:17.086 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.086 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.086 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.087 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.087 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.087 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.087 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.087 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.088 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:31:17.084+0000 7fb763648640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb763648640 time 2023-11-08T19:31:17.084236+0000 2023-11-08T19:31:17.088 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:31:17.088 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.088 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.088 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fb76f16efaf] 2023-11-08T19:31:17.088 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.089 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.089 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.089 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.089 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.089 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.090 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.090 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.090 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.090 INFO:tasks.ceph.mds.b.smithi188.stderr:*** Caught signal (Aborted) ** 2023-11-08T19:31:17.090 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7fb763648640 thread_name:MR_Finisher 2023-11-08T19:31:17.091 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.091 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fb76e854db0] 2023-11-08T19:31:17.091 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fb76e8a154c] 2023-11-08T19:31:17.092 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:31:17.092 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:31:17.092 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fb76f16f009] 2023-11-08T19:31:17.092 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.092 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.092 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.093 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.093 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.093 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.093 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.093 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.094 INFO:tasks.ceph.mds.b.smithi188.stderr:2023-11-08T19:31:17.086+0000 7fb763648640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:31:17.094 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7fb763648640 thread_name:MR_Finisher 2023-11-08T19:31:17.094 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.094 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.094 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fb76e854db0] 2023-11-08T19:31:17.095 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fb76e8a154c] 2023-11-08T19:31:17.095 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:31:17.095 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:31:17.095 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fb76f16f009] 2023-11-08T19:31:17.095 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.095 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.096 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.096 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.096 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.096 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.096 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.097 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.097 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:31:17.097 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.115 INFO:tasks.ceph.mds.b.smithi188.stderr: -410> 2023-11-08T19:31:13.471+0000 7fb766e4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:31:17.115 INFO:tasks.ceph.mds.b.smithi188.stderr: -408> 2023-11-08T19:31:13.471+0000 7fb766e4f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:31:17.118 INFO:tasks.ceph.mds.b.smithi188.stderr: -1> 2023-11-08T19:31:17.084+0000 7fb763648640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb763648640 time 2023-11-08T19:31:17.084236+0000 2023-11-08T19:31:17.118 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:31:17.118 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.118 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.119 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fb76f16efaf] 2023-11-08T19:31:17.119 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.119 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.119 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.119 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.120 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.120 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.120 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.120 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.120 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.120 INFO:tasks.ceph.mds.b.smithi188.stderr: 0> 2023-11-08T19:31:17.086+0000 7fb763648640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:31:17.121 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7fb763648640 thread_name:MR_Finisher 2023-11-08T19:31:17.121 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.121 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.121 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fb76e854db0] 2023-11-08T19:31:17.121 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fb76e8a154c] 2023-11-08T19:31:17.122 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:31:17.122 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:31:17.122 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fb76f16f009] 2023-11-08T19:31:17.122 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.122 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.123 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.123 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.123 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.123 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.123 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.123 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.124 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:31:17.124 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.134 INFO:tasks.ceph.mds.b.smithi188.stderr: -410> 2023-11-08T19:31:13.471+0000 7fb766e4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:31:17.135 INFO:tasks.ceph.mds.b.smithi188.stderr: -408> 2023-11-08T19:31:13.471+0000 7fb766e4f640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:31:17.136 INFO:tasks.ceph.mds.b.smithi188.stderr: -1> 2023-11-08T19:31:17.084+0000 7fb763648640 -1 /home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: In function 'void MDSTableServer::_prepare_logged(ceph::cref_t&, version_t)' thread 7fb763648640 time 2023-11-08T19:31:17.084236+0000 2023-11-08T19:31:17.136 INFO:tasks.ceph.mds.b.smithi188.stderr:/home/jenkins-build/build/workspace/ceph-dev-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-7035-g7532011a/rpm/el9/BUILD/ceph-18.0.0-7035-g7532011a/src/mds/MDSTableServer.cc: 77: FAILED ceph_assert(g_conf()->mds_kill_mdstable_at != 2) 2023-11-08T19:31:17.137 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.137 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.137 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x12e) [0x7fb76f16efaf] 2023-11-08T19:31:17.137 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.137 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.138 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.138 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.138 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.138 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.138 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.139 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.139 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.139 INFO:tasks.ceph.mds.b.smithi188.stderr: 0> 2023-11-08T19:31:17.086+0000 7fb763648640 -1 *** Caught signal (Aborted) ** 2023-11-08T19:31:17.139 INFO:tasks.ceph.mds.b.smithi188.stderr: in thread 7fb763648640 thread_name:MR_Finisher 2023-11-08T19:31:17.139 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.140 INFO:tasks.ceph.mds.b.smithi188.stderr: ceph version 18.0.0-7035-g7532011a (7532011aba9c81bdc6f987da6b4c05fb8ea5f4d4) reef (dev) 2023-11-08T19:31:17.140 INFO:tasks.ceph.mds.b.smithi188.stderr: 1: /lib64/libc.so.6(+0x54db0) [0x7fb76e854db0] 2023-11-08T19:31:17.140 INFO:tasks.ceph.mds.b.smithi188.stderr: 2: /lib64/libc.so.6(+0xa154c) [0x7fb76e8a154c] 2023-11-08T19:31:17.140 INFO:tasks.ceph.mds.b.smithi188.stderr: 3: raise() 2023-11-08T19:31:17.140 INFO:tasks.ceph.mds.b.smithi188.stderr: 4: abort() 2023-11-08T19:31:17.141 INFO:tasks.ceph.mds.b.smithi188.stderr: 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x188) [0x7fb76f16f009] 2023-11-08T19:31:17.141 INFO:tasks.ceph.mds.b.smithi188.stderr: 6: /usr/lib64/ceph/libceph-common.so.2(+0x16f16d) [0x7fb76f16f16d] 2023-11-08T19:31:17.141 INFO:tasks.ceph.mds.b.smithi188.stderr: 7: (MDSTableServer::_prepare_logged(boost::intrusive_ptr const&, unsigned long)+0x73e) [0x55a07286c9ce] 2023-11-08T19:31:17.141 INFO:tasks.ceph.mds.b.smithi188.stderr: 8: (MDSContext::complete(int)+0x5c) [0x55a07289cbcc] 2023-11-08T19:31:17.141 INFO:tasks.ceph.mds.b.smithi188.stderr: 9: (MDSIOContextBase::complete(int)+0x31c) [0x55a0728a5b0c] 2023-11-08T19:31:17.141 INFO:tasks.ceph.mds.b.smithi188.stderr: 10: (MDSLogContextBase::complete(int)+0x53) [0x55a0728a5c63] 2023-11-08T19:31:17.142 INFO:tasks.ceph.mds.b.smithi188.stderr: 11: (Finisher::finisher_thread_entry()+0x175) [0x7fb76f2164a5] 2023-11-08T19:31:17.142 INFO:tasks.ceph.mds.b.smithi188.stderr: 12: /lib64/libc.so.6(+0x9f802) [0x7fb76e89f802] 2023-11-08T19:31:17.142 INFO:tasks.ceph.mds.b.smithi188.stderr: 13: /lib64/libc.so.6(+0x3f450) [0x7fb76e83f450] 2023-11-08T19:31:17.142 INFO:tasks.ceph.mds.b.smithi188.stderr: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-11-08T19:31:17.142 INFO:tasks.ceph.mds.b.smithi188.stderr: 2023-11-08T19:31:17.279 INFO:teuthology.orchestra.run.smithi119.stdout:{"mds_beacon_grace":"15.000000"} 2023-11-08T19:31:17.279 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2023-11-08T19:31:17.280 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:17.472 INFO:tasks.ceph.mds.b.smithi188.stderr:daemon-helper: command crashed with signal 6 2023-11-08T19:31:17.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.490+0000 7f9369d00640 1 Processor -- start 2023-11-08T19:31:17.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.491+0000 7f9369d00640 1 -- start start 2023-11-08T19:31:17.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f9369d00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 0x7f93640cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f9369d00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f93640cb4e0 0x7f93640cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f9369d00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93640cc6c0 con 0x7f93641555b0 2023-11-08T19:31:17.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f9369d00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93640cbeb0 con 0x7f93640cb4e0 2023-11-08T19:31:17.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f9369d00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9364071770 con 0x7f93640ce040 2023-11-08T19:31:17.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f93637fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f93641555b0 0x7f9364070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57108/0 (socket says 172.21.15.119:57108) 2023-11-08T19:31:17.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.492+0000 7f93637fe640 1 -- 172.21.15.119:0/1480104399 learned_addr learned my addr 172.21.15.119:0/1480104399 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:17.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f9362ffd640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 0x7f93640cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f9363fff640 1 --2- 172.21.15.119:0/1480104399 >> v2:172.21.15.188:3300/0 conn(0x7f93640cb4e0 0x7f93640cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784551799 0 0) 0x7f93640cc6c0 con 0x7f93641555b0 2023-11-08T19:31:17.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9350003520 con 0x7f93641555b0 2023-11-08T19:31:17.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2872386609 0 0) 0x7f9350003520 con 0x7f93641555b0 2023-11-08T19:31:17.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 msgr2=0x7f93640cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:17.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 0x7f93640cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 >> v2:172.21.15.188:3300/0 conn(0x7f93640cb4e0 msgr2=0x7f93640cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:17.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 --2- 172.21.15.119:0/1480104399 >> v2:172.21.15.188:3300/0 conn(0x7f93640cb4e0 0x7f93640cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9364071a50 con 0x7f93641555b0 2023-11-08T19:31:17.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.493+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f93540019f0 con 0x7f93641555b0 2023-11-08T19:31:17.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 >> v1:172.21.15.119:6789/0 conn(0x7f93641555b0 legacy=0x7f9364070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9350003970 0x7f9350003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93500047c0 0x7f9350004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 0x7f9350005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640cbeb0 con 0x7f93500047c0 2023-11-08T19:31:17.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9364071770 con 0x7f9350003970 2023-11-08T19:31:17.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f935000b9d0 con 0x7f93640ce040 2023-11-08T19:31:17.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f9362ffd640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93500047c0 0x7f9350004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f9362ffd640 1 -- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 msgr2=0x7f9350005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:17.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f9362ffd640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f93640ce040 0x7f9350005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f9362ffd640 1 -- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9350003970 msgr2=0x7f9350003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:17.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93637fe640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9350003970 0x7f9350003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f9362ffd640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9350003970 0x7f9350003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f9362ffd640 1 -- 172.21.15.119:0/1480104399 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9350003520 con 0x7f93500047c0 2023-11-08T19:31:17.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.494+0000 7f93637fe640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9350003970 0x7f9350003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:17.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.495+0000 7f9362ffd640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93500047c0 0x7f9350004c40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f934c009af0 tx=0x7f93500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5004197b3a728ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c02c710 con 0x7f93500047c0 2023-11-08T19:31:17.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f934c02c870 con 0x7f93500047c0 2023-11-08T19:31:17.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f93627fc640 1 -- 172.21.15.119:0/1480104399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f934c02cc00 con 0x7f93500047c0 2023-11-08T19:31:17.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9369d00640 1 -- 172.21.15.119:0/1480104399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93500047c0 msgr2=0x7f9350004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9369d00640 1 --2- 172.21.15.119:0/1480104399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93500047c0 0x7f9350004c40 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f934c009af0 tx=0x7f93500057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9363fff640 1 -- 172.21.15.119:0/1480104399 reap_dead start 2023-11-08T19:31:17.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9369d00640 1 -- 172.21.15.119:0/1480104399 shutdown_connections 2023-11-08T19:31:17.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9369d00640 1 -- 172.21.15.119:0/1480104399 >> 172.21.15.119:0/1480104399 conn(0x7f93640b06e0 msgr2=0x7f93640b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:17.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9369d00640 1 -- 172.21.15.119:0/1480104399 shutdown_connections 2023-11-08T19:31:17.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.496+0000 7f9369d00640 1 -- 172.21.15.119:0/1480104399 wait complete. 2023-11-08T19:31:17.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.498+0000 7f9369d00640 1 Processor -- start 2023-11-08T19:31:17.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.498+0000 7f9369d00640 1 -- start start 2023-11-08T19:31:17.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9369d00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93500047c0 0x7f93640c3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9369d00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9364145de0 0x7f9364154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9369d00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 0x7f93640c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9369d00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640c8a90 con 0x7f93641546d0 2023-11-08T19:31:17.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9369d00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9364154b50 con 0x7f93500047c0 2023-11-08T19:31:17.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9369d00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936414f260 con 0x7f9364145de0 2023-11-08T19:31:17.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.499+0000 7f9363fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 0x7f93640c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 0x7f93640c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:50028/0 (socket says 172.21.15.119:50028) 2023-11-08T19:31:17.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 -- 172.21.15.119:0/1000323288 learned_addr learned my addr 172.21.15.119:0/1000323288 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:17.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9362ffd640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9364145de0 0x7f9364154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 -- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9364145de0 msgr2=0x7f9364154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9364145de0 0x7f9364154190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 -- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93500047c0 msgr2=0x7f93640c3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f93637fe640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93500047c0 0x7f93640c3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93500047c0 0x7f93640c3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 -- 172.21.15.119:0/1000323288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93640c87b0 con 0x7f93641546d0 2023-11-08T19:31:17.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f93637fe640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93500047c0 0x7f93640c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:17.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.500+0000 7f9363fff640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 0x7f93640c7ba0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f936406de40 tx=0x7f935c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.501+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00c680 con 0x7f93641546d0 2023-11-08T19:31:17.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.501+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c00c7e0 con 0x7f93641546d0 2023-11-08T19:31:17.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.501+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00cb90 con 0x7f93641546d0 2023-11-08T19:31:17.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.501+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93640c26a0 con 0x7f93641546d0 2023-11-08T19:31:17.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.501+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93640c2c60 con 0x7f93641546d0 2023-11-08T19:31:17.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.502+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93640c2500 con 0x7f93641546d0 2023-11-08T19:31:17.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.503+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f935c00ccf0 con 0x7f93641546d0 2023-11-08T19:31:17.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.504+0000 7f935bfff640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f934406ae70 0x7f934406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:17.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.504+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f935c089530 con 0x7f93641546d0 2023-11-08T19:31:17.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.504+0000 7f93637fe640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f934406ae70 0x7f934406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:17.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.505+0000 7f93637fe640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f934406ae70 0x7f934406d330 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9354007c40 tx=0x7f9354009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:17.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.508+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93640c2500 con 0x7f93641546d0 2023-11-08T19:31:17.738 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.737+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93640c2500 con 0x7f93641546d0 2023-11-08T19:31:17.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.738+0000 7f935bfff640 1 -- 172.21.15.119:0/1000323288 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f93640c2500 con 0x7f93641546d0 2023-11-08T19:31:17.740 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:17.740 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:16.258526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:17.741 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 91 2023-11-08T19:31:17.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f934406ae70 msgr2=0x7f934406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f934406ae70 0x7f934406d330 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f9354007c40 tx=0x7f9354009040 comp rx=0 tx=0).stop 2023-11-08T19:31:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 msgr2=0x7f93640c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 0x7f93640c7ba0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f936406de40 tx=0x7f935c00ec00 comp rx=0 tx=0).stop 2023-11-08T19:31:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 shutdown_connections 2023-11-08T19:31:17.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f934406ae70 0x7f934406d330 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f93641546d0 0x7f93640c7ba0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9364145de0 0x7f9364154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 --2- 172.21.15.119:0/1000323288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f93500047c0 0x7f93640c3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 >> 172.21.15.119:0/1000323288 conn(0x7f93640b06e0 msgr2=0x7f93640ae630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:17.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 shutdown_connections 2023-11-08T19:31:17.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:17.741+0000 7f9369d00640 1 -- 172.21.15.119:0/1000323288 wait complete. 2023-11-08T19:31:17.763 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:31:18.844 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-08T19:31:20.422 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:20.432 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:22.765 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:22.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.990+0000 7ff4a3a6f640 1 Processor -- start 2023-11-08T19:31:22.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.991+0000 7ff4a3a6f640 1 -- start start 2023-11-08T19:31:22.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a3a6f640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff49c159d80 0x7ff49c0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a3a6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:22.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a3a6f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff49c15dd60 con 0x7ff49c0cb2e0 2023-11-08T19:31:22.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a3a6f640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff49c15da30 con 0x7ff49c159d80 2023-11-08T19:31:22.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a3a6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c0ca8b0 con 0x7ff49c0703a0 2023-11-08T19:31:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a200c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff49c0cb2e0 0x7ff49c06c200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:36962/0 (socket says 172.21.15.119:36962) 2023-11-08T19:31:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a180b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff49c159d80 0x7ff49c0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.992+0000 7ff4a200c640 1 -- 172.21.15.119:0/2255359549 learned_addr learned my addr 172.21.15.119:0/2255359549 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a100a640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:22.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a180b640 1 -- 172.21.15.119:0/2255359549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 msgr2=0x7ff49c15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a180b640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a180b640 1 -- 172.21.15.119:0/2255359549 >> v1:172.21.15.119:6789/0 conn(0x7ff49c0cb2e0 legacy=0x7ff49c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a180b640 1 -- 172.21.15.119:0/2255359549 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff49c0cab90 con 0x7ff49c159d80 2023-11-08T19:31:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a0809640 1 -- 172.21.15.119:0/2255359549 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632101731 0 0) 0x7ff49c15dd60 con 0x7ff49c0cb2e0 2023-11-08T19:31:22.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.993+0000 7ff4a100a640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.994+0000 7ff4a180b640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0ca080 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff4980097a0 tx=0x7ff49802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5f683f71ecfd8f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.994+0000 7ff4a0809640 1 -- 172.21.15.119:0/2255359549 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49802d040 con 0x7ff49c159d80 2023-11-08T19:31:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.994+0000 7ff4a0809640 1 -- 172.21.15.119:0/2255359549 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff498031810 con 0x7ff49c159d80 2023-11-08T19:31:22.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.994+0000 7ff4a0809640 1 -- 172.21.15.119:0/2255359549 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498031ba0 con 0x7ff49c159d80 2023-11-08T19:31:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/2255359549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 msgr2=0x7ff49c0ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0ca080 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7ff4980097a0 tx=0x7ff49802b8b0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/2255359549 shutdown_connections 2023-11-08T19:31:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/2255359549 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0ca080 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/2255359549 >> 172.21.15.119:0/2255359549 conn(0x7ff49c0b06e0 msgr2=0x7ff49c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:23.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/2255359549 shutdown_connections 2023-11-08T19:31:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.995+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/2255359549 wait complete. 2023-11-08T19:31:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.997+0000 7ff4a3a6f640 1 Processor -- start 2023-11-08T19:31:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.997+0000 7ff4a3a6f640 1 -- start start 2023-11-08T19:31:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a3a6f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:23.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a3a6f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c0cb2e0 0x7ff49c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a3a6f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0af7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a3a6f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c15bea0 con 0x7ff49c159d80 2023-11-08T19:31:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a3a6f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c15c180 con 0x7ff49c0cb2e0 2023-11-08T19:31:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a3a6f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c15c460 con 0x7ff49c0703a0 2023-11-08T19:31:23.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a200c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0af7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a200c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0af7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35324/0 (socket says 172.21.15.119:35324) 2023-11-08T19:31:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.998+0000 7ff4a200c640 1 -- 172.21.15.119:0/904034488 learned_addr learned my addr 172.21.15.119:0/904034488 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a180b640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a100a640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c0cb2e0 0x7ff49c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:23.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a200c640 1 -- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 msgr2=0x7ff49c0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a200c640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a200c640 1 -- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c0cb2e0 msgr2=0x7ff49c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a200c640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c0cb2e0 0x7ff49c15cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a200c640 1 -- 172.21.15.119:0/904034488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff498009400 con 0x7ff49c159d80 2023-11-08T19:31:23.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4a200c640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0af7d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c7a0 tx=0x7ff49400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:23.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:22.999+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400a640 con 0x7ff49c159d80 2023-11-08T19:31:23.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.000+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff49400a7a0 con 0x7ff49c159d80 2023-11-08T19:31:23.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.000+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49400abd0 con 0x7ff49c159d80 2023-11-08T19:31:23.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.000+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c15c740 con 0x7ff49c159d80 2023-11-08T19:31:23.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.000+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c0c55a0 con 0x7ff49c159d80 2023-11-08T19:31:23.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.001+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c04e490 con 0x7ff49c159d80 2023-11-08T19:31:23.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.002+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff49400ad30 con 0x7ff49c159d80 2023-11-08T19:31:23.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.003+0000 7ff4927fc640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff48006ae70 0x7ff48006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:23.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.003+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7ff4940892d0 con 0x7ff49c159d80 2023-11-08T19:31:23.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.003+0000 7ff4a180b640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff48006ae70 0x7ff48006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:23.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.004+0000 7ff4a180b640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff48006ae70 0x7ff48006d330 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff498036000 tx=0x7ff498008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:23.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.008+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff494053c50 con 0x7ff49c159d80 2023-11-08T19:31:23.234 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.232+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff49c0ad7b0 con 0x7ff49c159d80 2023-11-08T19:31:23.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.233+0000 7ff4927fc640 1 -- 172.21.15.119:0/904034488 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff494018090 con 0x7ff49c159d80 2023-11-08T19:31:23.236 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:23.236 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:16.258526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:23.236 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 91 2023-11-08T19:31:23.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.235+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff48006ae70 msgr2=0x7ff48006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:23.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.235+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff48006ae70 0x7ff48006d330 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff498036000 tx=0x7ff498008030 comp rx=0 tx=0).stop 2023-11-08T19:31:23.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.235+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 msgr2=0x7ff49c0af7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:23.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0af7d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7ff49400c7a0 tx=0x7ff49400cc60 comp rx=0 tx=0).stop 2023-11-08T19:31:23.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 shutdown_connections 2023-11-08T19:31:23.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff48006ae70 0x7ff48006d330 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff49c159d80 0x7ff49c0af7d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff49c0cb2e0 0x7ff49c15cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 --2- 172.21.15.119:0/904034488 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff49c0703a0 0x7ff49c0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:23.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 >> 172.21.15.119:0/904034488 conn(0x7ff49c0b06e0 msgr2=0x7ff49c0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:23.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 shutdown_connections 2023-11-08T19:31:23.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:23.236+0000 7ff4a3a6f640 1 -- 172.21.15.119:0/904034488 wait complete. 2023-11-08T19:31:23.258 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:31:25.456 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~7s 2023-11-08T19:31:28.260 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:28.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.473+0000 7f1c5c3a5640 1 Processor -- start 2023-11-08T19:31:28.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.473+0000 7f1c5c3a5640 1 -- start start 2023-11-08T19:31:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5c3a5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c541555b0 0x7f1c540ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5c3a5640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c540cd110 0x7f1c54070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5c3a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cb4e0 0x7f1c540cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5c3a5640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c540cd590 con 0x7f1c540cd110 2023-11-08T19:31:28.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5c3a5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c540cbe70 con 0x7f1c541555b0 2023-11-08T19:31:28.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5c3a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540cc6c0 con 0x7f1c540cb4e0 2023-11-08T19:31:28.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c540cd110 0x7f1c54070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1c540cd110 0x7f1c54070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35338/0 (socket says 172.21.15.119:35338) 2023-11-08T19:31:28.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 -- 172.21.15.119:0/3289104737 learned_addr learned my addr 172.21.15.119:0/3289104737 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 -- 172.21.15.119:0/3289104737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cb4e0 msgr2=0x7f1c540cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:28.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5a942640 1 --2- 172.21.15.119:0/3289104737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cb4e0 0x7f1c540cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5a141640 1 --2- 172.21.15.119:0/3289104737 >> v2:172.21.15.188:3300/0 conn(0x7f1c541555b0 0x7f1c540ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 --2- 172.21.15.119:0/3289104737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cb4e0 0x7f1c540cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 -- 172.21.15.119:0/3289104737 >> v2:172.21.15.188:3300/0 conn(0x7f1c541555b0 msgr2=0x7f1c540ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 --2- 172.21.15.119:0/3289104737 >> v2:172.21.15.188:3300/0 conn(0x7f1c541555b0 0x7f1c540ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c59940640 1 -- 172.21.15.119:0/3289104737 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c540714d0 con 0x7f1c540cd110 2023-11-08T19:31:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.475+0000 7f1c5a141640 1 --2- 172.21.15.119:0/3289104737 >> v2:172.21.15.188:3300/0 conn(0x7f1c541555b0 0x7f1c540ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.476+0000 7f1c59940640 1 --2- 172.21.15.119:0/3289104737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c540cd110 0x7f1c54070f90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1c540cb4b0 tx=0x7f1c4402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb9a3bcc276dffa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.476+0000 7f1c5913f640 1 -- 172.21.15.119:0/3289104737 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c440317e0 con 0x7f1c540cd110 2023-11-08T19:31:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.476+0000 7f1c5913f640 1 -- 172.21.15.119:0/3289104737 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c44031940 con 0x7f1c540cd110 2023-11-08T19:31:28.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.476+0000 7f1c5913f640 1 -- 172.21.15.119:0/3289104737 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44031cd0 con 0x7f1c540cd110 2023-11-08T19:31:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/3289104737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c540cd110 msgr2=0x7f1c54070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/3289104737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c540cd110 0x7f1c54070f90 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f1c540cb4b0 tx=0x7f1c4402b880 comp rx=0 tx=0).stop 2023-11-08T19:31:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/3289104737 shutdown_connections 2023-11-08T19:31:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/3289104737 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cb4e0 0x7f1c540cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/3289104737 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c540cd110 0x7f1c54070f90 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/3289104737 >> v2:172.21.15.188:3300/0 conn(0x7f1c541555b0 0x7f1c540ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/3289104737 >> 172.21.15.119:0/3289104737 conn(0x7f1c540b06e0 msgr2=0x7f1c540b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/3289104737 shutdown_connections 2023-11-08T19:31:28.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.477+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/3289104737 wait complete. 2023-11-08T19:31:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.479+0000 7f1c5c3a5640 1 Processor -- start 2023-11-08T19:31:28.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.479+0000 7f1c5c3a5640 1 -- start start 2023-11-08T19:31:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5c3a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c540cb4e0 0x7f1c540b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5c3a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cd110 0x7f1c5414ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5c3a5640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 0x7f1c540af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5c3a5640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c54150470 con 0x7f1c541555b0 2023-11-08T19:31:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5c3a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540b3400 con 0x7f1c540cb4e0 2023-11-08T19:31:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5c3a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c540b2860 con 0x7f1c540cd110 2023-11-08T19:31:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 0x7f1c540af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 0x7f1c540af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:35346/0 (socket says 172.21.15.119:35346) 2023-11-08T19:31:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 -- 172.21.15.119:0/477678711 learned_addr learned my addr 172.21.15.119:0/477678711 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:28.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c59940640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cd110 0x7f1c5414ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a141640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c540cb4e0 0x7f1c540b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 -- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cd110 msgr2=0x7f1c5414ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cd110 0x7f1c5414ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 -- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c540cb4e0 msgr2=0x7f1c540b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:28.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c540cb4e0 0x7f1c540b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.480+0000 7f1c5a942640 1 -- 172.21.15.119:0/477678711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440094d0 con 0x7f1c541555b0 2023-11-08T19:31:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.481+0000 7f1c5a942640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 0x7f1c540af700 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00cbb0 tx=0x7f1c4c00a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.481+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c00cea0 con 0x7f1c541555b0 2023-11-08T19:31:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.481+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4c00abd0 con 0x7f1c541555b0 2023-11-08T19:31:28.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.481+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4c011410 con 0x7f1c541555b0 2023-11-08T19:31:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.481+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c540b2ad0 con 0x7f1c541555b0 2023-11-08T19:31:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.481+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c540b3090 con 0x7f1c541555b0 2023-11-08T19:31:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.482+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c540b3590 con 0x7f1c541555b0 2023-11-08T19:31:28.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.483+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c4c011570 con 0x7f1c541555b0 2023-11-08T19:31:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.484+0000 7f1c4affd640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1c3806ae70 0x7f1c3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:28.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.484+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f1c4c089c00 con 0x7f1c541555b0 2023-11-08T19:31:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.484+0000 7f1c5a141640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1c3806ae70 0x7f1c3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.485+0000 7f1c5a141640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1c3806ae70 0x7f1c3806d330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c540b03f0 tx=0x7f1c50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:28.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.488+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c4c0544f0 con 0x7f1c541555b0 2023-11-08T19:31:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.719+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c540ce5b0 con 0x7f1c541555b0 2023-11-08T19:31:28.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.721+0000 7f1c4affd640 1 -- 172.21.15.119:0/477678711 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f1c4c057b10 con 0x7f1c541555b0 2023-11-08T19:31:28.722 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:28.723 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":91,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:16.258526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:28.723 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 91 2023-11-08T19:31:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.722+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1c3806ae70 msgr2=0x7f1c3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1c3806ae70 0x7f1c3806d330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c540b03f0 tx=0x7f1c50008040 comp rx=0 tx=0).stop 2023-11-08T19:31:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 msgr2=0x7f1c540af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 0x7f1c540af700 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c00cbb0 tx=0x7f1c4c00a4c0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 shutdown_connections 2023-11-08T19:31:28.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1c3806ae70 0x7f1c3806d330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1c541555b0 0x7f1c540af700 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c540cd110 0x7f1c5414ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 --2- 172.21.15.119:0/477678711 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c540cb4e0 0x7f1c540b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 >> 172.21.15.119:0/477678711 conn(0x7f1c540b06e0 msgr2=0x7f1c54070300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 shutdown_connections 2023-11-08T19:31:28.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:28.723+0000 7f1c5c3a5640 1 -- 172.21.15.119:0/477678711 wait complete. 2023-11-08T19:31:28.746 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2023-11-08T19:31:32.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~13s 2023-11-08T19:31:33.748 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:33.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.969+0000 7f467359e640 1 Processor -- start 2023-11-08T19:31:33.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.969+0000 7f467359e640 1 -- start start 2023-11-08T19:31:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f467359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f466c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f467359e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0ae3c0 con 0x7f466c0c9010 2023-11-08T19:31:33.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f467359e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0adb20 con 0x7f466c0c9470 2023-11-08T19:31:33.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f467359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0c98e0 con 0x7f466c155190 2023-11-08T19:31:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f4671d9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f466c0c9010 0x7f466c15b910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60578/0 (socket says 172.21.15.119:60578) 2023-11-08T19:31:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f4671d9b640 1 -- 172.21.15.119:0/1639168003 learned_addr learned my addr 172.21.15.119:0/1639168003 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.971+0000 7f467259c640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f466c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 380366563 0 0) 0x7f466c0ae3c0 con 0x7f466c0c9010 2023-11-08T19:31:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4658003520 con 0x7f466c0c9010 2023-11-08T19:31:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1615099940 0 0) 0x7f4658003520 con 0x7f466c0c9010 2023-11-08T19:31:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 msgr2=0x7f466c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:33.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f466c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 >> v1:172.21.15.188:6789/0 conn(0x7f466c0c9470 legacy=0x7f466c06a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0ade00 con 0x7f466c0c9010 2023-11-08T19:31:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f4664001670 con 0x7f466c0c9010 2023-11-08T19:31:33.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 >> v1:172.21.15.119:6789/0 conn(0x7f466c0c9010 legacy=0x7f466c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:33.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f46580039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4658003f20 0x7f46580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f4658005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465800bb10 con 0x7f4658004d00 2023-11-08T19:31:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f465800bc50 con 0x7f4658003f20 2023-11-08T19:31:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.972+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f465800be10 con 0x7f466c155190 2023-11-08T19:31:33.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4672d9d640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f4658005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4671d9b640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4658003f20 0x7f46580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f467259c640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f46580039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4672d9d640 1 -- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 msgr2=0x7f46580039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:33.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4672d9d640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f46580039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4672d9d640 1 -- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4658003f20 msgr2=0x7f46580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4672d9d640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4658003f20 0x7f46580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f4672d9d640 1 -- 172.21.15.119:0/1639168003 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4658003520 con 0x7f4658004d00 2023-11-08T19:31:33.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.973+0000 7f467259c640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c155190 0x7f46580039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.974+0000 7f4672d9d640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f4658005180 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4668001e50 tx=0x7f46580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af243a6a2a7151a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.974+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668003b70 con 0x7f4658004d00 2023-11-08T19:31:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.974+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4668003cd0 con 0x7f4658004d00 2023-11-08T19:31:33.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.974+0000 7f467159a640 1 -- 172.21.15.119:0/1639168003 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668005310 con 0x7f4658004d00 2023-11-08T19:31:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f467359e640 1 -- 172.21.15.119:0/1639168003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 msgr2=0x7f4658005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f467359e640 1 --2- 172.21.15.119:0/1639168003 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f4658005180 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4668001e50 tx=0x7f46580057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:33.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f4672d9d640 1 -- 172.21.15.119:0/1639168003 reap_dead start 2023-11-08T19:31:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f467359e640 1 -- 172.21.15.119:0/1639168003 shutdown_connections 2023-11-08T19:31:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f467359e640 1 -- 172.21.15.119:0/1639168003 >> 172.21.15.119:0/1639168003 conn(0x7f466c0b1020 msgr2=0x7f466c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:33.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f467359e640 1 -- 172.21.15.119:0/1639168003 shutdown_connections 2023-11-08T19:31:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.975+0000 7f467359e640 1 -- 172.21.15.119:0/1639168003 wait complete. 2023-11-08T19:31:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.977+0000 7f467359e640 1 Processor -- start 2023-11-08T19:31:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.977+0000 7f467359e640 1 -- start start 2023-11-08T19:31:33.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467359e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f466c0afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f466c146380 0x7f466c146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c15c410 0x7f466c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467359e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c146d40 con 0x7f4658004d00 2023-11-08T19:31:33.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0cc6c0 con 0x7f466c146380 2023-11-08T19:31:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0cc960 con 0x7f466c15c410 2023-11-08T19:31:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f466c0afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467259c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f466c0afc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40360/0 (socket says 172.21.15.119:40360) 2023-11-08T19:31:33.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.978+0000 7f467259c640 1 -- 172.21.15.119:0/3280052676 learned_addr learned my addr 172.21.15.119:0/3280052676 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:33.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f4672d9d640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c15c410 0x7f466c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f4671d9b640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f466c146380 0x7f466c146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467259c640 1 -- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c15c410 msgr2=0x7f466c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:33.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467259c640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c15c410 0x7f466c15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467259c640 1 -- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f466c146380 msgr2=0x7f466c146800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467259c640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f466c146380 0x7f466c146800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:33.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467259c640 1 -- 172.21.15.119:0/3280052676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0ccc40 con 0x7f4658004d00 2023-11-08T19:31:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f4672d9d640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c15c410 0x7f466c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f4671d9b640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f466c146380 0x7f466c146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:33.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467259c640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f466c0afc00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f46600108b0 tx=0x7f4660010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:33.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466000eb60 con 0x7f4658004d00 2023-11-08T19:31:33.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466000ecc0 con 0x7f4658004d00 2023-11-08T19:31:33.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466001c400 con 0x7f4658004d00 2023-11-08T19:31:33.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.979+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c0ccf20 con 0x7f4658004d00 2023-11-08T19:31:33.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.980+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c15cdd0 con 0x7f4658004d00 2023-11-08T19:31:33.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.981+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c04e490 con 0x7f4658004d00 2023-11-08T19:31:33.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.982+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f466001c560 con 0x7f4658004d00 2023-11-08T19:31:33.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.982+0000 7f465effd640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f465006ae70 0x7f465006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:33.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.983+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f466008a310 con 0x7f4658004d00 2023-11-08T19:31:33.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.983+0000 7f4671d9b640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f465006ae70 0x7f465006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:33.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.983+0000 7f4671d9b640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f465006ae70 0x7f465006d330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4664001050 tx=0x7f4664001c20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:33.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:33.987+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4660054d40 con 0x7f4658004d00 2023-11-08T19:31:34.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.213+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f466c06c920 con 0x7f4658004d00 2023-11-08T19:31:34.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.214+0000 7f465effd640 1 -- 172.21.15.119:0/3280052676 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4992 (secure 0 0 0) 0x7f4660058360 con 0x7f4658004d00 2023-11-08T19:31:34.216 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:34.216 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:33.736859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:31:33.736859+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:34.216 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 92 2023-11-08T19:31:34.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f465006ae70 msgr2=0x7f465006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f465006ae70 0x7f465006d330 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4664001050 tx=0x7f4664001c20 comp rx=0 tx=0).stop 2023-11-08T19:31:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 msgr2=0x7f466c0afc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f466c0afc00 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f46600108b0 tx=0x7f4660010d70 comp rx=0 tx=0).stop 2023-11-08T19:31:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 shutdown_connections 2023-11-08T19:31:34.219 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f465006ae70 0x7f465006d330 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f466c15c410 0x7f466c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f466c146380 0x7f466c146800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 --2- 172.21.15.119:0/3280052676 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4658004d00 0x7f466c0afc00 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 >> 172.21.15.119:0/3280052676 conn(0x7f466c0b1020 msgr2=0x7f466c15b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 shutdown_connections 2023-11-08T19:31:34.220 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.217+0000 7f467359e640 1 -- 172.21.15.119:0/3280052676 wait complete. 2023-11-08T19:31:34.237 DEBUG:tasks.ceph_test_case:wait_until_true: success in 15s and 0 retries 2023-11-08T19:31:34.237 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -n kernel.core_pattern 2023-11-08T19:31:34.337 INFO:teuthology.orchestra.run.smithi188.stdout:/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T19:31:34.341 INFO:tasks.cephfs.cephfs_test_case:Clearing core from directory: /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:31:34.342 DEBUG:teuthology.orchestra.run.smithi188:> cd /home/ubuntu/cephtest/archive/coredump && sudo ls | sudo xargs file 2023-11-08T19:31:34.447 INFO:teuthology.orchestra.run.smithi188.stdout:1699471877.95448.core: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mds -f --cluster ceph -i b', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mds', platform: 'x86_64' 2023-11-08T19:31:34.451 INFO:tasks.cephfs.cephfs_test_case:Enumerated cores: ['1699471877.95448.core'] 2023-11-08T19:31:34.451 INFO:tasks.cephfs.cephfs_test_case:Found core file 1699471877.95448.core, deleting it 2023-11-08T19:31:34.452 DEBUG:teuthology.orchestra.run.smithi188:> cd /home/ubuntu/cephtest/archive/coredump && sudo rm -f 1699471877.95448.core 2023-11-08T19:31:34.549 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.767+0000 7f199f59e640 1 Processor -- start 2023-11-08T19:31:34.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.767+0000 7f199f59e640 1 -- start start 2023-11-08T19:31:34.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.769+0000 7f199f59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f19a0155190 0x7f19a00caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.769+0000 7f199f59e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a00c98a0 con 0x7f19a00c9470 2023-11-08T19:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.769+0000 7f199f59e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a00ae3c0 con 0x7f19a00c9010 2023-11-08T19:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.769+0000 7f199f59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a00adb20 con 0x7f19a0155190 2023-11-08T19:31:34.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.769+0000 7f199ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f19a00c9470 0x7f19a006a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60586/0 (socket says 172.21.15.119:60586) 2023-11-08T19:31:34.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.769+0000 7f199ed9d640 1 -- 172.21.15.119:0/2223603979 learned_addr learned my addr 172.21.15.119:0/2223603979 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1546048004 0 0) 0x7f19a00c98a0 con 0x7f19a00c9470 2023-11-08T19:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199e59c640 1 --2- 172.21.15.119:0/2223603979 >> v2:172.21.15.188:3300/0 conn(0x7f19a0155190 0x7f19a00caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:34.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f198c003520 con 0x7f19a00c9470 2023-11-08T19:31:34.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2181297537 0 0) 0x7f198c003520 con 0x7f19a00c9470 2023-11-08T19:31:34.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 >> v2:172.21.15.188:3300/0 conn(0x7f19a0155190 msgr2=0x7f19a00caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:34.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 --2- 172.21.15.119:0/2223603979 >> v2:172.21.15.188:3300/0 conn(0x7f19a0155190 0x7f19a00caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 >> v1:172.21.15.188:6789/0 conn(0x7f19a00c9010 legacy=0x7f19a015b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:34.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00ade00 con 0x7f19a00c9470 2023-11-08T19:31:34.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f19980019f0 con 0x7f19a00c9470 2023-11-08T19:31:34.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 >> v1:172.21.15.119:6789/0 conn(0x7f19a00c9470 legacy=0x7f19a006a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:34.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198c0039c0 0x7f198c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198c0047c0 0x7f198c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f198c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a00adb20 con 0x7f198c005250 2023-11-08T19:31:34.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00c1a0 con 0x7f198c0039c0 2023-11-08T19:31:34.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.770+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198c00c360 con 0x7f198c0047c0 2023-11-08T19:31:34.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199ed9d640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f198c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199e59c640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198c0039c0 0x7f198c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f1997fff640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198c0047c0 0x7f198c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199ed9d640 1 -- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198c0047c0 msgr2=0x7f198c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199ed9d640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f198c0047c0 0x7f198c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199ed9d640 1 -- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198c0039c0 msgr2=0x7f198c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:34.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199ed9d640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f198c0039c0 0x7f198c003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.771+0000 7f199ed9d640 1 -- 172.21.15.119:0/2223603979 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198c003520 con 0x7f198c005250 2023-11-08T19:31:34.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.772+0000 7f199ed9d640 1 -- 172.21.15.119:0/2223603979 reap_dead start 2023-11-08T19:31:34.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.772+0000 7f199ed9d640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f198c00b980 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f199800a810 tx=0x7f198c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1069636eafd16e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:34.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.772+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998034630 con 0x7f198c005250 2023-11-08T19:31:34.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.772+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199802f070 con 0x7f198c005250 2023-11-08T19:31:34.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.772+0000 7f199dd9b640 1 -- 172.21.15.119:0/2223603979 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1998034a10 con 0x7f198c005250 2023-11-08T19:31:34.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 -- 172.21.15.119:0/2223603979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 msgr2=0x7f198c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:34.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f198c00b980 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f199800a810 tx=0x7f198c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 -- 172.21.15.119:0/2223603979 shutdown_connections 2023-11-08T19:31:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 --2- 172.21.15.119:0/2223603979 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f198c00b980 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 -- 172.21.15.119:0/2223603979 >> 172.21.15.119:0/2223603979 conn(0x7f19a00b1020 msgr2=0x7f19a00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 -- 172.21.15.119:0/2223603979 shutdown_connections 2023-11-08T19:31:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.773+0000 7f199f59e640 1 -- 172.21.15.119:0/2223603979 wait complete. 2023-11-08T19:31:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.775+0000 7f199f59e640 1 Processor -- start 2023-11-08T19:31:34.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.775+0000 7f199f59e640 1 -- start start 2023-11-08T19:31:34.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199f59e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f19a01edc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199f59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19a015e750 0x7f19a00af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199f59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19a015ebd0 0x7f19a00afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.786 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199f59e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a013db80 con 0x7f198c005250 2023-11-08T19:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199f59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a015f480 con 0x7f19a015e750 2023-11-08T19:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199f59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a015f1a0 con 0x7f19a015ebd0 2023-11-08T19:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f19a01edc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f19a01edc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40372/0 (socket says 172.21.15.119:40372) 2023-11-08T19:31:34.787 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 -- 172.21.15.119:0/1449109850 learned_addr learned my addr 172.21.15.119:0/1449109850 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 -- 172.21.15.119:0/1449109850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19a015ebd0 msgr2=0x7f19a00afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19a015ebd0 0x7f19a00afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 -- 172.21.15.119:0/1449109850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19a015e750 msgr2=0x7f19a00af6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19a015e750 0x7f19a00af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:34.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.776+0000 7f199e59c640 1 -- 172.21.15.119:0/1449109850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00b17e0 con 0x7f198c005250 2023-11-08T19:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.777+0000 7f199e59c640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f19a01edc20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f199000e7e0 tx=0x7f199000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.777+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000c6d0 con 0x7f198c005250 2023-11-08T19:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.777+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199000c830 con 0x7f198c005250 2023-11-08T19:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.777+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199000cbe0 con 0x7f198c005250 2023-11-08T19:31:34.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.777+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a00b1ac0 con 0x7f198c005250 2023-11-08T19:31:34.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.777+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a01ee160 con 0x7f198c005250 2023-11-08T19:31:34.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.778+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a004e490 con 0x7f198c005250 2023-11-08T19:31:34.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.779+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f199000cd40 con 0x7f198c005250 2023-11-08T19:31:34.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.780+0000 7f1996ffd640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f198006ae70 0x7f198006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:34.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.780+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f1990089b20 con 0x7f198c005250 2023-11-08T19:31:34.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.780+0000 7f1997fff640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f198006ae70 0x7f198006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:34.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.781+0000 7f1997fff640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f198006ae70 0x7f198006d330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f19880015b0 tx=0x7f1988009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:34.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:34.784+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f19900544a0 con 0x7f198c005250 2023-11-08T19:31:35.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.010+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19a006cf00 con 0x7f198c005250 2023-11-08T19:31:35.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.011+0000 7f1996ffd640 1 -- 172.21.15.119:0/1449109850 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+4992 (secure 0 0 0) 0x7f1990009380 con 0x7f198c005250 2023-11-08T19:31:35.013 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:35.013 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":92,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":7746,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":86}],"filesystems":[{"mdsmap":{"epoch":92,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:33.736859+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":101,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":5210,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5210":{"gid":5210,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":11,"addr":"172.21.15.188:6835/2339927924","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6833","nonce":2339927924},{"type":"v1","addr":"172.21.15.188:6835","nonce":2339927924}]},"join_fscid":-1,"laggy_since":"2023-11-08T19:31:33.736859+0000","export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:35.014 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 92 2023-11-08T19:31:35.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f198006ae70 msgr2=0x7f198006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f198006ae70 0x7f198006d330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f19880015b0 tx=0x7f1988009040 comp rx=0 tx=0).stop 2023-11-08T19:31:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 msgr2=0x7f19a01edc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f19a01edc20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f199000e7e0 tx=0x7f199000eca0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 shutdown_connections 2023-11-08T19:31:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f198006ae70 0x7f198006d330 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f19a015ebd0 0x7f19a00afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f19a015e750 0x7f19a00af6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 --2- 172.21.15.119:0/1449109850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f198c005250 0x7f19a01edc20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 >> 172.21.15.119:0/1449109850 conn(0x7f19a00b1020 msgr2=0x7f19a01535d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:35.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 shutdown_connections 2023-11-08T19:31:35.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.014+0000 7f199f59e640 1 -- 172.21.15.119:0/1449109850 wait complete. 2023-11-08T19:31:35.035 INFO:tasks.ceph.mds.d:Sent signal 9 2023-11-08T19:31:35.036 INFO:tasks.cephfs.mount:Suspending the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:31:35.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:35.037 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 down 2023-11-08T19:31:35.136 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:0 2023-11-08T19:31:35.238 INFO:tasks.ceph.mds.d.smithi188.stderr:daemon-helper: command crashed with signal 9 2023-11-08T19:31:35.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.343+0000 7f5798abb640 1 Processor -- start 2023-11-08T19:31:35.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.343+0000 7f5798abb640 1 -- start start 2023-11-08T19:31:35.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.344+0000 7f5798abb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940cdfb0 0x7f57940b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.344+0000 7f5798abb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57940b6ba0 0x7f57940cf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.344+0000 7f5798abb640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57940d1900 con 0x7f57940b2130 2023-11-08T19:31:35.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.344+0000 7f5798abb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57940b2510 con 0x7f57940b6ba0 2023-11-08T19:31:35.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.344+0000 7f5798abb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57940ce590 con 0x7f57940cdfb0 2023-11-08T19:31:35.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.344+0000 7f5792d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57940b6ba0 0x7f57940cf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5792d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f57940b6ba0 0x7f57940cf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40386/0 (socket says 172.21.15.119:40386) 2023-11-08T19:31:35.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5792d9d640 1 -- 172.21.15.119:0/3111214008 learned_addr learned my addr 172.21.15.119:0/3111214008 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:35.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5791d9b640 1 --2- 172.21.15.119:0/3111214008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940cdfb0 0x7f57940b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5792d9d640 1 -- 172.21.15.119:0/3111214008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940cdfb0 msgr2=0x7f57940b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:35.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5792d9d640 1 --2- 172.21.15.119:0/3111214008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940cdfb0 0x7f57940b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5792d9d640 1 -- 172.21.15.119:0/3111214008 >> v1:172.21.15.188:6789/0 conn(0x7f57940b2130 legacy=0x7f5794161bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:35.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.345+0000 7f5792d9d640 1 -- 172.21.15.119:0/3111214008 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57940ce870 con 0x7f57940b6ba0 2023-11-08T19:31:35.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.346+0000 7f5792d9d640 1 --2- 172.21.15.119:0/3111214008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940b6ba0 0x7f57940cf980 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2664bcbb25b2e9bc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:35.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f579159a640 1 -- 172.21.15.119:0/3111214008 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0317c0 con 0x7f57940b6ba0 2023-11-08T19:31:35.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f579159a640 1 -- 172.21.15.119:0/3111214008 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c031920 con 0x7f57940b6ba0 2023-11-08T19:31:35.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f579159a640 1 -- 172.21.15.119:0/3111214008 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c031cb0 con 0x7f57940b6ba0 2023-11-08T19:31:35.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 -- 172.21.15.119:0/3111214008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940b6ba0 msgr2=0x7f57940cf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 --2- 172.21.15.119:0/3111214008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940b6ba0 0x7f57940cf980 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c02b860 comp rx=0 tx=0).stop 2023-11-08T19:31:35.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 -- 172.21.15.119:0/3111214008 shutdown_connections 2023-11-08T19:31:35.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 --2- 172.21.15.119:0/3111214008 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940b6ba0 0x7f57940cf980 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 --2- 172.21.15.119:0/3111214008 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940cdfb0 0x7f57940b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 -- 172.21.15.119:0/3111214008 >> 172.21.15.119:0/3111214008 conn(0x7f579406a920 msgr2=0x7f579406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:35.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 -- 172.21.15.119:0/3111214008 shutdown_connections 2023-11-08T19:31:35.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.347+0000 7f5798abb640 1 -- 172.21.15.119:0/3111214008 wait complete. 2023-11-08T19:31:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.349+0000 7f5798abb640 1 Processor -- start 2023-11-08T19:31:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.349+0000 7f5798abb640 1 -- start start 2023-11-08T19:31:35.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5798abb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940b2130 0x7f579415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5798abb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57940b6ba0 0x7f57940cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5798abb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 0x7f579415ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5798abb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57940ca830 con 0x7f57940cdfb0 2023-11-08T19:31:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5798abb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794154a00 con 0x7f57940b6ba0 2023-11-08T19:31:35.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5798abb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579415aa60 con 0x7f57940b2130 2023-11-08T19:31:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.350+0000 7f5792d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 0x7f579415ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 0x7f579415ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40400/0 (socket says 172.21.15.119:40400) 2023-11-08T19:31:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 -- 172.21.15.119:0/3922076774 learned_addr learned my addr 172.21.15.119:0/3922076774 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5791d9b640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57940b6ba0 0x7f57940cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f579259c640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940b2130 0x7f579415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 -- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940b2130 msgr2=0x7f579415a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940b2130 0x7f579415a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 -- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57940b6ba0 msgr2=0x7f57940cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57940b6ba0 0x7f57940cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 -- 172.21.15.119:0/3922076774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c0094d0 con 0x7f57940cdfb0 2023-11-08T19:31:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f579259c640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940b2130 0x7f579415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5791d9b640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57940b6ba0 0x7f57940cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.351+0000 7f5792d9d640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 0x7f579415ed50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.352+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c032920 con 0x7f57940cdfb0 2023-11-08T19:31:35.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.352+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c032ac0 con 0x7f57940cdfb0 2023-11-08T19:31:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.352+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c032e70 con 0x7f57940cdfb0 2023-11-08T19:31:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.352+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57940ca1e0 con 0x7f57940cdfb0 2023-11-08T19:31:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.352+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57940ccee0 con 0x7f57940cdfb0 2023-11-08T19:31:35.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.353+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57940ca040 con 0x7f57940cdfb0 2023-11-08T19:31:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.354+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f578c03b3f0 con 0x7f57940cdfb0 2023-11-08T19:31:35.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.355+0000 7f578affd640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f577406aea0 0x7f577406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.355+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 7522+0+0 (secure 0 0 0) 0x7f578c0ac370 con 0x7f57940cdfb0 2023-11-08T19:31:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.356+0000 7f579259c640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f577406aea0 0x7f577406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.356+0000 7f579259c640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f577406aea0 0x7f577406d360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5784004520 tx=0x7f5784009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:35.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.360+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57940ca040 con 0x7f57940cdfb0 2023-11-08T19:31:35.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.618+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:0"} v 0) v1 -- 0x7f57940ca040 con 0x7f57940cdfb0 2023-11-08T19:31:35.624 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.623+0000 7f578affd640 1 -- 172.21.15.119:0/3922076774 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:0"}]=0 failed mds gid 5210 v93) v1 ==== 97+0+0 (secure 0 0 0) 0x7f57940ca040 con 0x7f57940cdfb0 2023-11-08T19:31:35.624 INFO:teuthology.orchestra.run.smithi119.stderr:failed mds gid 5210 2023-11-08T19:31:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.625+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f577406aea0 msgr2=0x7f577406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.625+0000 7f5798abb640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f577406aea0 0x7f577406d360 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5784004520 tx=0x7f5784009040 comp rx=0 tx=0).stop 2023-11-08T19:31:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.625+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 msgr2=0x7f579415ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.625+0000 7f5798abb640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 0x7f579415ed50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f578c009870 tx=0x7f578c0046c0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.625+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 shutdown_connections 2023-11-08T19:31:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f577406aea0 0x7f577406d360 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f57940cdfb0 0x7f579415ed50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f57940b6ba0 0x7f57940cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 --2- 172.21.15.119:0/3922076774 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f57940b2130 0x7f579415a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 >> 172.21.15.119:0/3922076774 conn(0x7f579406a920 msgr2=0x7f57940b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:35.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 shutdown_connections 2023-11-08T19:31:35.629 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.626+0000 7f5798abb640 1 -- 172.21.15.119:0/3922076774 wait complete. 2023-11-08T19:31:35.647 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T19:31:35.647 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T19:31:35.647 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:31:35.648 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:31:35.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:31:35.649 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi188 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b' 2023-11-08T19:31:35.649 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:31:35.650 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T19:31:35.662 INFO:tasks.ceph.mds.b:Started 2023-11-08T19:31:35.662 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:35.840 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:31:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.844+0000 7f0601047640 1 Processor -- start 2023-11-08T19:31:35.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.844+0000 7f0601047640 1 -- start start 2023-11-08T19:31:35.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.845+0000 7f0601047640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f05fc0c9010 0x7f05fc0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.845+0000 7f0601047640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0cb1a0 con 0x7f05fc155190 2023-11-08T19:31:35.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.845+0000 7f0601047640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0ae3c0 con 0x7f05fc15b910 2023-11-08T19:31:35.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.845+0000 7f0601047640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc0adb20 con 0x7f05fc0c9010 2023-11-08T19:31:35.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fbfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f05fc155190 0x7f05fc0caa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60596/0 (socket says 172.21.15.119:60596) 2023-11-08T19:31:35.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fbfff640 1 -- 172.21.15.119:0/3096050476 learned_addr learned my addr 172.21.15.119:0/3096050476 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:35.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05f3fff640 1 --2- 172.21.15.119:0/3096050476 >> v2:172.21.15.119:3300/0 conn(0x7f05fc0c9010 0x7f05fc0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483504533 0 0) 0x7f05fc0cb1a0 con 0x7f05fc155190 2023-11-08T19:31:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e8003520 con 0x7f05fc155190 2023-11-08T19:31:35.850 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215210512 0 0) 0x7f05fc0ae3c0 con 0x7f05fc15b910 2023-11-08T19:31:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc0cb1a0 con 0x7f05fc15b910 2023-11-08T19:31:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05f3fff640 1 -- 172.21.15.119:0/3096050476 >> v1:172.21.15.188:6789/0 conn(0x7f05fc15b910 legacy=0x7f05fc06a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05f3fff640 1 -- 172.21.15.119:0/3096050476 >> v1:172.21.15.119:6789/0 conn(0x7f05fc155190 legacy=0x7f05fc0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652920376 0 0) 0x7f05e8003520 con 0x7f05fc155190 2023-11-08T19:31:35.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.846+0000 7f05f3fff640 1 -- 172.21.15.119:0/3096050476 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0ade00 con 0x7f05fc0c9010 2023-11-08T19:31:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.848+0000 7f05f3fff640 1 --2- 172.21.15.119:0/3096050476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc0c9470 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097a0 tx=0x7f05e402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83673c5ccfbd1a91 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.848+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec001670 con 0x7f05fc0c9010 2023-11-08T19:31:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.848+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e402e050 con 0x7f05fc0c9010 2023-11-08T19:31:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.848+0000 7f05fb7fe640 1 -- 172.21.15.119:0/3096050476 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4030730 con 0x7f05fc0c9010 2023-11-08T19:31:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 -- 172.21.15.119:0/3096050476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 msgr2=0x7f05fc0c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 --2- 172.21.15.119:0/3096050476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc0c9470 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f05e40097a0 tx=0x7f05e402b790 comp rx=0 tx=0).stop 2023-11-08T19:31:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 -- 172.21.15.119:0/3096050476 shutdown_connections 2023-11-08T19:31:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 --2- 172.21.15.119:0/3096050476 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc0c9470 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 -- 172.21.15.119:0/3096050476 >> 172.21.15.119:0/3096050476 conn(0x7f05fc0b1020 msgr2=0x7f05fc0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 -- 172.21.15.119:0/3096050476 shutdown_connections 2023-11-08T19:31:35.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.849+0000 7f0601047640 1 -- 172.21.15.119:0/3096050476 wait complete. 2023-11-08T19:31:35.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.850+0000 7f0601047640 1 Processor -- start 2023-11-08T19:31:35.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 -- start start 2023-11-08T19:31:35.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc1590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05fc155190 0x7f05fc15d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05fc15b910 0x7f05fc15de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc159920 con 0x7f05fc0c9010 2023-11-08T19:31:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc159c00 con 0x7f05fc15b910 2023-11-08T19:31:35.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.851+0000 7f0601047640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc159ee0 con 0x7f05fc155190 2023-11-08T19:31:35.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc1590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc1590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40422/0 (socket says 172.21.15.119:40422) 2023-11-08T19:31:35.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 -- 172.21.15.119:0/1968027153 learned_addr learned my addr 172.21.15.119:0/1968027153 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:35.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05f3fff640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05fc155190 0x7f05fc15d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f0600846640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05fc15b910 0x7f05fc15de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 -- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05fc155190 msgr2=0x7f05fc15d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05fc155190 0x7f05fc15d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 -- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05fc15b910 msgr2=0x7f05fc15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05fc15b910 0x7f05fc15de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:35.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 -- 172.21.15.119:0/1968027153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e4009400 con 0x7f05fc0c9010 2023-11-08T19:31:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f0600846640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05fc15b910 0x7f05fc15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.852+0000 7f05fbfff640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc1590a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c750 tx=0x7f05ec00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.853+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec005ad0 con 0x7f05fc0c9010 2023-11-08T19:31:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.853+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ec005c30 con 0x7f05fc0c9010 2023-11-08T19:31:35.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.853+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ec007070 con 0x7f05fc0c9010 2023-11-08T19:31:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.853+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc15a1c0 con 0x7f05fc0c9010 2023-11-08T19:31:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.853+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc1595e0 con 0x7f05fc0c9010 2023-11-08T19:31:35.861 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.854+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc04e490 con 0x7f05fc0c9010 2023-11-08T19:31:35.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.855+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05ec0071d0 con 0x7f05fc0c9010 2023-11-08T19:31:35.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.856+0000 7f05f97fa640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05dc06ae70 0x7f05dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:35.862 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.856+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f05ec089b50 con 0x7f05fc0c9010 2023-11-08T19:31:35.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.856+0000 7f05f3fff640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05dc06ae70 0x7f05dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:35.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.857+0000 7f05f3fff640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05dc06ae70 0x7f05dc06d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f05e4034000 tx=0x7f05e402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:35.863 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:35.860+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05ec0543f0 con 0x7f05fc0c9010 2023-11-08T19:31:36.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.088+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05fc15bf90 con 0x7f05fc0c9010 2023-11-08T19:31:36.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.089+0000 7f05f97fa640 1 -- 172.21.15.119:0/1968027153 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4149 (secure 0 0 0) 0x7f05ec057a10 con 0x7f05fc0c9010 2023-11-08T19:31:36.090 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:36.090 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":94,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:35.630060+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:replay","state_seq":1,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:36.091 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 94 2023-11-08T19:31:36.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.091+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05dc06ae70 msgr2=0x7f05dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:36.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.091+0000 7f0601047640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05dc06ae70 0x7f05dc06d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f05e4034000 tx=0x7f05e402f000 comp rx=0 tx=0).stop 2023-11-08T19:31:36.093 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.091+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 msgr2=0x7f05fc1590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.091+0000 7f0601047640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc1590a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00c750 tx=0x7f05ec00cc10 comp rx=0 tx=0).stop 2023-11-08T19:31:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 shutdown_connections 2023-11-08T19:31:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f05dc06ae70 0x7f05dc06d330 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05fc15b910 0x7f05fc15de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:36.094 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05fc155190 0x7f05fc15d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:36.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 --2- 172.21.15.119:0/1968027153 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f05fc0c9010 0x7f05fc1590a0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:36.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 >> 172.21.15.119:0/1968027153 conn(0x7f05fc0b1020 msgr2=0x7f05fc0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:36.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 shutdown_connections 2023-11-08T19:31:36.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:36.092+0000 7f0601047640 1 -- 172.21.15.119:0/1968027153 wait complete. 2023-11-08T19:31:36.112 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:31:38.680 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-08T19:31:38.681 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-08T19:31:41.113 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:41.338 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.337+0000 7f2fb4fb6640 1 Processor -- start 2023-11-08T19:31:41.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.337+0000 7f2fb4fb6640 1 -- start start 2023-11-08T19:31:41.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.339+0000 7f2fb4fb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 0x7f2fb015e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.339+0000 7f2fb4fb6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb015dd60 con 0x7f2fb0159d80 2023-11-08T19:31:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.339+0000 7f2fb4fb6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb015da30 con 0x7f2fb00cb2e0 2023-11-08T19:31:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.339+0000 7f2fb4fb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00ca8b0 con 0x7f2fb00703a0 2023-11-08T19:31:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.339+0000 7f2fae59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2fb0159d80 0x7f2fb00ca080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:34592/0 (socket says 172.21.15.119:34592) 2023-11-08T19:31:41.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.339+0000 7f2fae59c640 1 -- 172.21.15.119:0/4005830100 learned_addr learned my addr 172.21.15.119:0/4005830100 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fa5d9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 0x7f2fb015e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488332512 0 0) 0x7f2fb015dd60 con 0x7f2fb0159d80 2023-11-08T19:31:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f9c003520 con 0x7f2fb0159d80 2023-11-08T19:31:41.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961899227 0 0) 0x7f2f9c003520 con 0x7f2fb0159d80 2023-11-08T19:31:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 msgr2=0x7f2fb015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 0x7f2fb015e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 >> v1:172.21.15.188:6789/0 conn(0x7f2fb00cb2e0 legacy=0x7f2fb006c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb00cab90 con 0x7f2fb0159d80 2023-11-08T19:31:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2fa0001670 con 0x7f2fb0159d80 2023-11-08T19:31:41.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 >> v1:172.21.15.119:6789/0 conn(0x7f2fb0159d80 legacy=0x7f2fb00ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c0039a0 0x7f2f9c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0047c0 0x7f2f9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 0x7f2f9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00ca8b0 con 0x7f2f9c0039a0 2023-11-08T19:31:41.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c00ba60 con 0x7f2f9c0047c0 2023-11-08T19:31:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.340+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f9c00bc20 con 0x7f2fb00703a0 2023-11-08T19:31:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fae59c640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c0039a0 0x7f2f9c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fa5d9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0047c0 0x7f2f9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2faed9d640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 0x7f2f9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fae59c640 1 -- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 msgr2=0x7f2f9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fae59c640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb00703a0 0x7f2f9c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fae59c640 1 -- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0047c0 msgr2=0x7f2f9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fae59c640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0047c0 0x7f2f9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fae59c640 1 -- 172.21.15.119:0/4005830100 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c003520 con 0x7f2f9c0039a0 2023-11-08T19:31:41.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.341+0000 7f2fa5d9b640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0047c0 0x7f2f9c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.342+0000 7f2fae59c640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c0039a0 0x7f2f9c003dc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0001eb0 tx=0x7f2f9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600c53aa9262198d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0005ad0 con 0x7f2f9c0039a0 2023-11-08T19:31:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa0005c50 con 0x7f2f9c0039a0 2023-11-08T19:31:41.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fadd9b640 1 -- 172.21.15.119:0/4005830100 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa002c7a0 con 0x7f2f9c0039a0 2023-11-08T19:31:41.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/4005830100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c0039a0 msgr2=0x7f2f9c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/4005830100 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2f9c0039a0 0x7f2f9c003dc0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0001eb0 tx=0x7f2f9c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2faed9d640 1 -- 172.21.15.119:0/4005830100 reap_dead start 2023-11-08T19:31:41.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/4005830100 shutdown_connections 2023-11-08T19:31:41.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/4005830100 >> 172.21.15.119:0/4005830100 conn(0x7f2fb00b06e0 msgr2=0x7f2fb00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:41.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/4005830100 shutdown_connections 2023-11-08T19:31:41.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.343+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/4005830100 wait complete. 2023-11-08T19:31:41.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.345+0000 7f2fb4fb6640 1 Processor -- start 2023-11-08T19:31:41.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.345+0000 7f2fb4fb6640 1 -- start start 2023-11-08T19:31:41.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2fb4fb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0039a0 0x7f2fb00c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2fb4fb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb01f3790 0x7f2fb0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2fb4fb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 0x7f2fb0153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2fb4fb6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb00c59c0 con 0x7f2fb0155bd0 2023-11-08T19:31:41.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2fb4fb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0158f50 con 0x7f2f9c0039a0 2023-11-08T19:31:41.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2fb4fb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb0157970 con 0x7f2fb01f3790 2023-11-08T19:31:41.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.346+0000 7f2faed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 0x7f2fb0153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 0x7f2fb0153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43542/0 (socket says 172.21.15.119:43542) 2023-11-08T19:31:41.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 -- 172.21.15.119:0/2795590589 learned_addr learned my addr 172.21.15.119:0/2795590589 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:41.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2fa5d9b640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb01f3790 0x7f2fb0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2fae59c640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0039a0 0x7f2fb00c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 -- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb01f3790 msgr2=0x7f2fb0158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb01f3790 0x7f2fb0158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 -- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0039a0 msgr2=0x7f2fb00c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0039a0 0x7f2fb00c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 -- 172.21.15.119:0/2795590589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb0157c50 con 0x7f2fb0155bd0 2023-11-08T19:31:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2faed9d640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 0x7f2fb0153900 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00ce870 tx=0x7f2fa800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.347+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8003b70 con 0x7f2fb0155bd0 2023-11-08T19:31:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.348+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8003cf0 con 0x7f2fb0155bd0 2023-11-08T19:31:41.357 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.348+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8003ea0 con 0x7f2fb0155bd0 2023-11-08T19:31:41.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.348+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb0157f30 con 0x7f2fb0155bd0 2023-11-08T19:31:41.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.348+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb01584f0 con 0x7f2fb0155bd0 2023-11-08T19:31:41.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.349+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb0158710 con 0x7f2fb0155bd0 2023-11-08T19:31:41.358 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.350+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fa80052b0 con 0x7f2fb0155bd0 2023-11-08T19:31:41.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.351+0000 7f2fa77fe640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2f9006aea0 0x7f2f9006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.351+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f2fa8089760 con 0x7f2fb0155bd0 2023-11-08T19:31:41.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.351+0000 7f2fae59c640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2f9006aea0 0x7f2f9006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.352+0000 7f2fae59c640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2f9006aea0 0x7f2f9006d360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0034000 tx=0x7f2fa0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:41.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.355+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fa8054000 con 0x7f2fb0155bd0 2023-11-08T19:31:41.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.580+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fb006dcc0 con 0x7f2fb0155bd0 2023-11-08T19:31:41.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.582+0000 7f2fa77fe640 1 -- 172.21.15.119:0/2795590589 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f2fa8057620 con 0x7f2fb0155bd0 2023-11-08T19:31:41.583 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:41.584 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:35.652638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:resolve","state_seq":9,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:41.584 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-11-08T19:31:41.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2f9006aea0 msgr2=0x7f2f9006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2f9006aea0 0x7f2f9006d360 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0034000 tx=0x7f2fa0009030 comp rx=0 tx=0).stop 2023-11-08T19:31:41.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 msgr2=0x7f2fb0153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 0x7f2fb0153900 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2fb00ce870 tx=0x7f2fa800dc00 comp rx=0 tx=0).stop 2023-11-08T19:31:41.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 shutdown_connections 2023-11-08T19:31:41.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2f9006aea0 0x7f2f9006d360 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2fb0155bd0 0x7f2fb0153900 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2fb01f3790 0x7f2fb0158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 --2- 172.21.15.119:0/2795590589 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2f9c0039a0 0x7f2fb00c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.584+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 >> 172.21.15.119:0/2795590589 conn(0x7f2fb00b06e0 msgr2=0x7f2fb0070860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:41.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.585+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 shutdown_connections 2023-11-08T19:31:41.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.585+0000 7f2fb4fb6640 1 -- 172.21.15.119:0/2795590589 wait complete. 2023-11-08T19:31:41.605 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:31:41.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:41.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.817+0000 7f8f5e94c640 1 Processor -- start 2023-11-08T19:31:41.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.817+0000 7f8f5e94c640 1 -- start start 2023-11-08T19:31:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.818+0000 7f8f5e94c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8f58140180 0x7f8f5815e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.818+0000 7f8f5e94c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f58142940 0x7f8f58142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.818+0000 7f8f5e94c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f5815e700 con 0x7f8f580b3c90 2023-11-08T19:31:41.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f5e94c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5815dd30 con 0x7f8f58142940 2023-11-08T19:31:41.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f5e94c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5806de50 con 0x7f8f58140180 2023-11-08T19:31:41.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f58142940 0x7f8f58142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f58142940 0x7f8f58142dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43544/0 (socket says 172.21.15.119:43544) 2023-11-08T19:31:41.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 -- 172.21.15.119:0/2053917959 learned_addr learned my addr 172.21.15.119:0/2053917959 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:41.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 -- 172.21.15.119:0/2053917959 >> v2:172.21.15.188:3300/0 conn(0x7f8f58140180 msgr2=0x7f8f5815e1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:31:41.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 --2- 172.21.15.119:0/2053917959 >> v2:172.21.15.188:3300/0 conn(0x7f8f58140180 0x7f8f5815e1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 -- 172.21.15.119:0/2053917959 >> v1:172.21.15.188:6789/0 conn(0x7f8f580b3c90 legacy=0x7f8f580b40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:41.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.819+0000 7f8f4ffff640 1 -- 172.21.15.119:0/2053917959 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5815d700 con 0x7f8f58142940 2023-11-08T19:31:41.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.820+0000 7f8f4ffff640 1 --2- 172.21.15.119:0/2053917959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f58142940 0x7f8f58142dd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8f580736e0 tx=0x7f8f4002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d5911f830418d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:41.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f577fe640 1 -- 172.21.15.119:0/2053917959 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40031890 con 0x7f8f58142940 2023-11-08T19:31:41.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f577fe640 1 -- 172.21.15.119:0/2053917959 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f400319f0 con 0x7f8f58142940 2023-11-08T19:31:41.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f577fe640 1 -- 172.21.15.119:0/2053917959 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40031d80 con 0x7f8f58142940 2023-11-08T19:31:41.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 -- 172.21.15.119:0/2053917959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f58142940 msgr2=0x7f8f58142dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/2053917959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f58142940 0x7f8f58142dd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f8f580736e0 tx=0x7f8f4002b930 comp rx=0 tx=0).stop 2023-11-08T19:31:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 -- 172.21.15.119:0/2053917959 shutdown_connections 2023-11-08T19:31:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/2053917959 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f58142940 0x7f8f58142dd0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/2053917959 >> v2:172.21.15.188:3300/0 conn(0x7f8f58140180 0x7f8f5815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 -- 172.21.15.119:0/2053917959 >> 172.21.15.119:0/2053917959 conn(0x7f8f580b2990 msgr2=0x7f8f580b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:41.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 -- 172.21.15.119:0/2053917959 shutdown_connections 2023-11-08T19:31:41.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.821+0000 7f8f5e94c640 1 -- 172.21.15.119:0/2053917959 wait complete. 2023-11-08T19:31:41.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.823+0000 7f8f5e94c640 1 Processor -- start 2023-11-08T19:31:41.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.823+0000 7f8f5e94c640 1 -- start start 2023-11-08T19:31:41.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f5e94c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 0x7f8f58156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f5e94c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f58140180 0x7f8f58144150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f5e94c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f58142940 0x7f8f58144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f5e94c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f581f2ae0 con 0x7f8f580b3c90 2023-11-08T19:31:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f5e94c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58144cc0 con 0x7f8f58140180 2023-11-08T19:31:41.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f5e94c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f58144fa0 con 0x7f8f58142940 2023-11-08T19:31:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 0x7f8f58156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f57fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 0x7f8f58156a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43548/0 (socket says 172.21.15.119:43548) 2023-11-08T19:31:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.824+0000 7f8f57fff640 1 -- 172.21.15.119:0/339947499 learned_addr learned my addr 172.21.15.119:0/339947499 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f5cee9640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f58142940 0x7f8f58144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f4ffff640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f58140180 0x7f8f58144150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f57fff640 1 -- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f58142940 msgr2=0x7f8f58144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f57fff640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f58142940 0x7f8f58144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f57fff640 1 -- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f58140180 msgr2=0x7f8f58144150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f57fff640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f58140180 0x7f8f58144150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:41.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f57fff640 1 -- 172.21.15.119:0/339947499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f400094d0 con 0x7f8f580b3c90 2023-11-08T19:31:41.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f5cee9640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f58142940 0x7f8f58144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:41.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.825+0000 7f8f57fff640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 0x7f8f58156a10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800b3e0 tx=0x7f8f4800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:41.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.826+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011830 con 0x7f8f580b3c90 2023-11-08T19:31:41.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.826+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f480119b0 con 0x7f8f580b3c90 2023-11-08T19:31:41.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.826+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48011d60 con 0x7f8f580b3c90 2023-11-08T19:31:41.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.826+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f58145280 con 0x7f8f580b3c90 2023-11-08T19:31:41.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.826+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f581596a0 con 0x7f8f580b3c90 2023-11-08T19:31:41.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.827+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f58159890 con 0x7f8f580b3c90 2023-11-08T19:31:41.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.828+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f4801a400 con 0x7f8f580b3c90 2023-11-08T19:31:41.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.829+0000 7f8f557fa640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f3806ae70 0x7f8f3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:41.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.829+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f8f48089f80 con 0x7f8f580b3c90 2023-11-08T19:31:41.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.829+0000 7f8f4ffff640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f3806ae70 0x7f8f3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:41.834 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.830+0000 7f8f4ffff640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f3806ae70 0x7f8f3806d330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8f40036000 tx=0x7f8f4003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:41.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:41.833+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f48054820 con 0x7f8f580b3c90 2023-11-08T19:31:42.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.060+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f5804e490 con 0x7f8f580b3c90 2023-11-08T19:31:42.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.061+0000 7f8f557fa640 1 -- 172.21.15.119:0/339947499 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4941 (secure 0 0 0) 0x7f8f48015020 con 0x7f8f580b3c90 2023-11-08T19:31:42.065 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:42.066 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:35.652638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:resolve","state_seq":9,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:42.066 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-11-08T19:31:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f3806ae70 msgr2=0x7f8f3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f3806ae70 0x7f8f3806d330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8f40036000 tx=0x7f8f4003c040 comp rx=0 tx=0).stop 2023-11-08T19:31:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 msgr2=0x7f8f58156a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 0x7f8f58156a10 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f8f4800b3e0 tx=0x7f8f4800b8a0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 shutdown_connections 2023-11-08T19:31:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f3806ae70 0x7f8f3806d330 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f58142940 0x7f8f58144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f58140180 0x7f8f58144150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 --2- 172.21.15.119:0/339947499 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f580b3c90 0x7f8f58156a10 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 >> 172.21.15.119:0/339947499 conn(0x7f8f580b2990 msgr2=0x7f8f5815ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 shutdown_connections 2023-11-08T19:31:42.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.064+0000 7f8f5e94c640 1 -- 172.21.15.119:0/339947499 wait complete. 2023-11-08T19:31:42.084 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:resolve 2023-11-08T19:31:42.085 DEBUG:tasks.cephfs.filesystem:reached state 'up:resolve' in 0.47937965393066406s 2023-11-08T19:31:42.085 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:42.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.298+0000 7fdfd13a9640 1 Processor -- start 2023-11-08T19:31:42.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.298+0000 7fdfd13a9640 1 -- start start 2023-11-08T19:31:42.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfd13a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfd13a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdfcc0703a0 0x7fdfcc15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfd13a9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfcc0cb2e0 0x7fdfcc0cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfd13a9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc0cbc80 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfd13a9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc15dd60 con 0x7fdfcc0703a0 2023-11-08T19:31:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfd13a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc15da30 con 0x7fdfcc159d80 2023-11-08T19:31:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfcb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfcc0cb2e0 0x7fdfcc0cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:42.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfcb7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdfcc0cb2e0 0x7fdfcc0cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43554/0 (socket says 172.21.15.119:43554) 2023-11-08T19:31:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.300+0000 7fdfcb7fe640 1 -- 172.21.15.119:0/2911997673 learned_addr learned my addr 172.21.15.119:0/2911997673 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfcaffd640 1 --2- 172.21.15.119:0/2911997673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfc3fff640 1 --2- 172.21.15.119:0/2911997673 >> v2:172.21.15.188:3300/0 conn(0x7fdfcc0703a0 0x7fdfcc15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfcb7fe640 1 -- 172.21.15.119:0/2911997673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 msgr2=0x7fdfcc0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfcb7fe640 1 --2- 172.21.15.119:0/2911997673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc0ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfcb7fe640 1 -- 172.21.15.119:0/2911997673 >> v2:172.21.15.188:3300/0 conn(0x7fdfcc0703a0 msgr2=0x7fdfcc15e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:42.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfcb7fe640 1 --2- 172.21.15.119:0/2911997673 >> v2:172.21.15.188:3300/0 conn(0x7fdfcc0703a0 0x7fdfcc15e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.301+0000 7fdfcb7fe640 1 -- 172.21.15.119:0/2911997673 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc0ca8b0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.302+0000 7fdfcb7fe640 1 --2- 172.21.15.119:0/2911997673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc0cb740 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0ce8f0 tx=0x7fdfc402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82aad0a92e291d6d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:42.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.302+0000 7fdfca7fc640 1 -- 172.21.15.119:0/2911997673 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc402ca70 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.302+0000 7fdfca7fc640 1 -- 172.21.15.119:0/2911997673 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc402cbd0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.302+0000 7fdfca7fc640 1 -- 172.21.15.119:0/2911997673 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc40343f0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 -- 172.21.15.119:0/2911997673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 msgr2=0x7fdfcc0cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/2911997673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc0cb740 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc0ce8f0 tx=0x7fdfc402bed0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 -- 172.21.15.119:0/2911997673 shutdown_connections 2023-11-08T19:31:42.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/2911997673 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc0cb740 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/2911997673 >> v2:172.21.15.188:3300/0 conn(0x7fdfcc0703a0 0x7fdfcc15e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/2911997673 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 -- 172.21.15.119:0/2911997673 >> 172.21.15.119:0/2911997673 conn(0x7fdfcc0b06e0 msgr2=0x7fdfcc0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 -- 172.21.15.119:0/2911997673 shutdown_connections 2023-11-08T19:31:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.303+0000 7fdfd13a9640 1 -- 172.21.15.119:0/2911997673 wait complete. 2023-11-08T19:31:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.305+0000 7fdfd13a9640 1 Processor -- start 2023-11-08T19:31:42.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.305+0000 7fdfd13a9640 1 -- start start 2023-11-08T19:31:42.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfd13a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfcc0703a0 0x7fdfcc15cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfd13a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfd13a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfd13a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc15a6a0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfd13a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1f2620 con 0x7fdfcc0703a0 2023-11-08T19:31:42.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfd13a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc1f2ab0 con 0x7fdfcc159d80 2023-11-08T19:31:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfc3fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc15a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43566/0 (socket says 172.21.15.119:43566) 2023-11-08T19:31:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.306+0000 7fdfc3fff640 1 -- 172.21.15.119:0/1966946258 learned_addr learned my addr 172.21.15.119:0/1966946258 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfcb7fe640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:42.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc3fff640 1 -- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 msgr2=0x7fdfcc1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc3fff640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc3fff640 1 -- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfcc0703a0 msgr2=0x7fdfcc15cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc3fff640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfcc0703a0 0x7fdfcc15cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc3fff640 1 -- 172.21.15.119:0/1966946258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4009b30 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc3fff640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc15a160 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400d800 tx=0x7fdfb400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb40096a0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb4009800 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc15d690 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb4009990 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.307+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc1f1fd0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.308+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc04e490 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.309+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdfb4009af0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.310+0000 7fdfc37fe640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfac06aea0 0x7fdfac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:42.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.310+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7608+0+0 (secure 0 0 0) 0x7fdfb408a290 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.311+0000 7fdfcaffd640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfac06aea0 0x7fdfac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.311+0000 7fdfcaffd640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfac06aea0 0x7fdfac06d360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0098f0 tx=0x7fdfbc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:42.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.315+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdfb4054bf0 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.540+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfcc071090 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.541+0000 7fdfc37fe640 1 -- 172.21.15.119:0/1966946258 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) v1 ==== 94+0+4941 (secure 0 0 0) 0x7fdfb4058210 con 0x7fdfcc0cb2e0 2023-11-08T19:31:42.543 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:42.544 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":95,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":95,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:35.652638+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":7065},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7065":{"gid":7065,"name":"d","rank":1,"incarnation":55,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6834/4246462419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4246462419},{"type":"v1","addr":"172.21.15.188:6834","nonce":4246462419}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:resolve","state_seq":9,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:42.544 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 95 2023-11-08T19:31:42.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfac06aea0 msgr2=0x7fdfac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfac06aea0 0x7fdfac06d360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0098f0 tx=0x7fdfbc008040 comp rx=0 tx=0).stop 2023-11-08T19:31:42.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 msgr2=0x7fdfcc15a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:42.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc15a160 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400d800 tx=0x7fdfb400dcc0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 shutdown_connections 2023-11-08T19:31:42.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfac06aea0 0x7fdfac06d360 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdfcc159d80 0x7fdfcc1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfcc0cb2e0 0x7fdfcc15a160 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 --2- 172.21.15.119:0/1966946258 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfcc0703a0 0x7fdfcc15cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:42.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.544+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 >> 172.21.15.119:0/1966946258 conn(0x7fdfcc0b06e0 msgr2=0x7fdfcc06d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:42.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.545+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 shutdown_connections 2023-11-08T19:31:42.548 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:42.545+0000 7fdfd13a9640 1 -- 172.21.15.119:0/1966946258 wait complete. 2023-11-08T19:31:42.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:31:42.791 INFO:teuthology.orchestra.run.smithi119.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{"17":[2,3,4,5,6,7,8,9]},"pending_update":[{"version":18,"snapinfo":{"snapid":10,"ino":1099511627777,"stamp":"2023-11-08T19:31:17.080798+0000","name":"s22","metadata":{}}}],"pending_destroy":[]} 2023-11-08T19:31:42.791 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": { "17": [ 2, 3, 4, 5, 6, 7, 8, 9 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [ { "snapInfinityo": { "ino": 1099511627777, "metadata": {}, "name": "s22", "snapid": 10, "stamp": "2023-11-08T19:31:17.080798+0000" }, "version": 18 } ], "snaps": [] } 2023-11-08T19:31:42.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 6:1 2023-11-08T19:31:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.012+0000 7f280d439640 1 Processor -- start 2023-11-08T19:31:43.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.013+0000 7f280d439640 1 -- start start 2023-11-08T19:31:43.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.014+0000 7f280d439640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f280815adb0 0x7f2808071680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.014+0000 7f280d439640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f28080cbab0 0x7f280815e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.014+0000 7f280d439640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28080cd150 con 0x7f2808070ef0 2023-11-08T19:31:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.014+0000 7f280d439640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2808071d10 con 0x7f28080cbab0 2023-11-08T19:31:43.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.014+0000 7f280d439640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28080cc860 con 0x7f280815adb0 2023-11-08T19:31:43.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f28077fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f28080cbab0 0x7f280815e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f28077fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f28080cbab0 0x7f280815e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43574/0 (socket says 172.21.15.119:43574) 2023-11-08T19:31:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f28077fe640 1 -- 172.21.15.119:0/2946908735 learned_addr learned my addr 172.21.15.119:0/2946908735 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f28077fe640 1 -- 172.21.15.119:0/2946908735 >> v2:172.21.15.188:3300/0 conn(0x7f280815adb0 msgr2=0x7f2808071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f2806ffd640 1 --2- 172.21.15.119:0/2946908735 >> v2:172.21.15.188:3300/0 conn(0x7f280815adb0 0x7f2808071680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f28077fe640 1 --2- 172.21.15.119:0/2946908735 >> v2:172.21.15.188:3300/0 conn(0x7f280815adb0 0x7f2808071680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.015+0000 7f28077fe640 1 -- 172.21.15.119:0/2946908735 >> v1:172.21.15.188:6789/0 conn(0x7f2808070ef0 legacy=0x7f28080cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:43.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.016+0000 7f28077fe640 1 -- 172.21.15.119:0/2946908735 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808071350 con 0x7f28080cbab0 2023-11-08T19:31:43.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.017+0000 7f28077fe640 1 --2- 172.21.15.119:0/2946908735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28080cbab0 0x7f280815e980 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f280806bfc0 tx=0x7f280002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23416ec52c997b68 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.017+0000 7f2805ffb640 1 -- 172.21.15.119:0/2946908735 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800031870 con 0x7f28080cbab0 2023-11-08T19:31:43.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.017+0000 7f2805ffb640 1 -- 172.21.15.119:0/2946908735 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28000319d0 con 0x7f28080cbab0 2023-11-08T19:31:43.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.017+0000 7f2805ffb640 1 -- 172.21.15.119:0/2946908735 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2800031d60 con 0x7f28080cbab0 2023-11-08T19:31:43.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.017+0000 7f280d439640 1 -- 172.21.15.119:0/2946908735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28080cbab0 msgr2=0x7f280815e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.017+0000 7f280d439640 1 --2- 172.21.15.119:0/2946908735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28080cbab0 0x7f280815e980 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f280806bfc0 tx=0x7f280002b910 comp rx=0 tx=0).stop 2023-11-08T19:31:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.018+0000 7f280d439640 1 -- 172.21.15.119:0/2946908735 shutdown_connections 2023-11-08T19:31:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.018+0000 7f280d439640 1 --2- 172.21.15.119:0/2946908735 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f28080cbab0 0x7f280815e980 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.018+0000 7f280d439640 1 --2- 172.21.15.119:0/2946908735 >> v2:172.21.15.188:3300/0 conn(0x7f280815adb0 0x7f2808071680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.018+0000 7f280d439640 1 -- 172.21.15.119:0/2946908735 >> 172.21.15.119:0/2946908735 conn(0x7f28080b06e0 msgr2=0x7f28080b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.018+0000 7f280d439640 1 -- 172.21.15.119:0/2946908735 shutdown_connections 2023-11-08T19:31:43.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.018+0000 7f280d439640 1 -- 172.21.15.119:0/2946908735 wait complete. 2023-11-08T19:31:43.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.019+0000 7f280d439640 1 Processor -- start 2023-11-08T19:31:43.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.020+0000 7f280d439640 1 -- start start 2023-11-08T19:31:43.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.020+0000 7f280d439640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 0x7f2808150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.020+0000 7f280d439640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28080cbab0 0x7f280815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.020+0000 7f280d439640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280815adb0 0x7f280815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f280d439640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28081f2780 con 0x7f2808070ef0 2023-11-08T19:31:43.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f280d439640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c84c0 con 0x7f280815adb0 2023-11-08T19:31:43.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f280d439640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28080c87a0 con 0x7f28080cbab0 2023-11-08T19:31:43.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 0x7f2808150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 0x7f2808150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43584/0 (socket says 172.21.15.119:43584) 2023-11-08T19:31:43.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 -- 172.21.15.119:0/3983481146 learned_addr learned my addr 172.21.15.119:0/3983481146 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 -- 172.21.15.119:0/3983481146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28080cbab0 msgr2=0x7f280815d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28080cbab0 0x7f280815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 -- 172.21.15.119:0/3983481146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280815adb0 msgr2=0x7f280815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280815adb0 0x7f280815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 -- 172.21.15.119:0/3983481146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28000094d0 con 0x7f2808070ef0 2023-11-08T19:31:43.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.021+0000 7f2806ffd640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 0x7f2808150480 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f27f800ce10 tx=0x7f27f800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.022+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003b90 con 0x7f2808070ef0 2023-11-08T19:31:43.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.022+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8003cf0 con 0x7f2808070ef0 2023-11-08T19:31:43.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.022+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8003ea0 con 0x7f2808070ef0 2023-11-08T19:31:43.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.022+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28080c8a80 con 0x7f2808070ef0 2023-11-08T19:31:43.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.022+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28080acb90 con 0x7f2808070ef0 2023-11-08T19:31:43.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.023+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f280804e490 con 0x7f2808070ef0 2023-11-08T19:31:43.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.024+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27f8004240 con 0x7f2808070ef0 2023-11-08T19:31:43.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.025+0000 7f27ff7fe640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.025+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 7608+0+0 (secure 0 0 0) 0x7f27f8012070 con 0x7f2808070ef0 2023-11-08T19:31:43.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.025+0000 7f28067fc640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.026+0000 7f28067fc640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006fd0 tx=0x7f27f0006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.029+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f27f8055120 con 0x7f2808070ef0 2023-11-08T19:31:43.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.289+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "6:1"} v 0) v1 -- 0x7f28080ce020 con 0x7f2808070ef0 2023-11-08T19:31:43.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.295+0000 7f27ff7fe640 1 -- 172.21.15.119:0/3983481146 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "6:1"}]=0 failed mds gid 7065 v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7f27f8058740 con 0x7f2808070ef0 2023-11-08T19:31:43.296 INFO:teuthology.orchestra.run.smithi119.stderr:failed mds gid 7065 2023-11-08T19:31:43.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.297+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 msgr2=0x7f27e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.297+0000 7f280d439640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f27f0006fd0 tx=0x7f27f0006bc0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 msgr2=0x7f2808150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 0x7f2808150480 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f27f800ce10 tx=0x7f27f800a660 comp rx=0 tx=0).stop 2023-11-08T19:31:43.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 shutdown_connections 2023-11-08T19:31:43.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f27e806ae70 0x7f27e806d330 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f280815adb0 0x7f280815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f28080cbab0 0x7f280815d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 --2- 172.21.15.119:0/3983481146 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2808070ef0 0x7f2808150480 secure :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f27f800ce10 tx=0x7f27f800a660 comp rx=0 tx=0).stop 2023-11-08T19:31:43.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 >> 172.21.15.119:0/3983481146 conn(0x7f28080b06e0 msgr2=0x7f280815a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:43.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 shutdown_connections 2023-11-08T19:31:43.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.298+0000 7f280d439640 1 -- 172.21.15.119:0/3983481146 wait complete. 2023-11-08T19:31:43.321 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T19:31:43.322 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T19:31:43.322 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:31:43.323 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:31:43.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:31:43.323 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi188 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d' 2023-11-08T19:31:43.324 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:31:43.324 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T19:31:43.331 INFO:tasks.ceph.mds.d:Started 2023-11-08T19:31:43.331 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:43.494 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2023-11-08T19:31:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.526+0000 7f70b584b640 1 Processor -- start 2023-11-08T19:31:43.528 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.526+0000 7f70b584b640 1 -- start start 2023-11-08T19:31:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70b584b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b00ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.530 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70b584b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70b00cb2e0 0x7f70b00cb740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70b584b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70b00cbc80 con 0x7f70b00703a0 2023-11-08T19:31:43.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70b584b640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b015dd60 con 0x7f70b00cb2e0 2023-11-08T19:31:43.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70b584b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b015e900 con 0x7f70b0159d80 2023-11-08T19:31:43.531 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70af7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70b00cb2e0 0x7f70b00cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70af7fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f70b00cb2e0 0x7f70b00cb740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43598/0 (socket says 172.21.15.119:43598) 2023-11-08T19:31:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.528+0000 7f70af7fe640 1 -- 172.21.15.119:0/1325693783 learned_addr learned my addr 172.21.15.119:0/1325693783 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.529+0000 7f70aeffd640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b00ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.529+0000 7f70af7fe640 1 -- 172.21.15.119:0/1325693783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 msgr2=0x7f70b00ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:43.532 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.529+0000 7f70af7fe640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b00ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.529+0000 7f70af7fe640 1 -- 172.21.15.119:0/1325693783 >> v1:172.21.15.188:6789/0 conn(0x7f70b00703a0 legacy=0x7f70b015e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.529+0000 7f70af7fe640 1 -- 172.21.15.119:0/1325693783 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b015da30 con 0x7f70b00cb2e0 2023-11-08T19:31:43.533 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.529+0000 7f70aeffd640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:31:43.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.530+0000 7f70af7fe640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00cb740 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f70a800d150 tx=0x7f70a802e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef600177529c75b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.530+0000 7f70adffb640 1 -- 172.21.15.119:0/1325693783 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a800b0d0 con 0x7f70b00cb2e0 2023-11-08T19:31:43.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.530+0000 7f70adffb640 1 -- 172.21.15.119:0/1325693783 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a800b230 con 0x7f70b00cb2e0 2023-11-08T19:31:43.537 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.530+0000 7f70adffb640 1 -- 172.21.15.119:0/1325693783 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a8009600 con 0x7f70b00cb2e0 2023-11-08T19:31:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 -- 172.21.15.119:0/1325693783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 msgr2=0x7f70b00cb740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00cb740 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f70a800d150 tx=0x7f70a802e160 comp rx=0 tx=0).stop 2023-11-08T19:31:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 -- 172.21.15.119:0/1325693783 shutdown_connections 2023-11-08T19:31:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00cb740 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.538 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 --2- 172.21.15.119:0/1325693783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b00ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 -- 172.21.15.119:0/1325693783 >> 172.21.15.119:0/1325693783 conn(0x7f70b00b06e0 msgr2=0x7f70b00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:43.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 -- 172.21.15.119:0/1325693783 shutdown_connections 2023-11-08T19:31:43.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.531+0000 7f70b584b640 1 -- 172.21.15.119:0/1325693783 wait complete. 2023-11-08T19:31:43.539 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.533+0000 7f70b584b640 1 Processor -- start 2023-11-08T19:31:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.533+0000 7f70b584b640 1 -- start start 2023-11-08T19:31:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70b584b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70b00703a0 0x7f70b00ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70b584b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70b584b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b0158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.540 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70b584b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00ad190 con 0x7f70b00cb2e0 2023-11-08T19:31:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70b584b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00ad470 con 0x7f70b00703a0 2023-11-08T19:31:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70b584b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b00ad750 con 0x7f70b0159d80 2023-11-08T19:31:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.541 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:43604/0 (socket says 172.21.15.119:43604) 2023-11-08T19:31:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 -- 172.21.15.119:0/1602946678 learned_addr learned my addr 172.21.15.119:0/1602946678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70aeffd640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70b00703a0 0x7f70b00ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 -- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 msgr2=0x7f70b0158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70af7fe640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b0158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b0158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 -- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70b00703a0 msgr2=0x7f70b00ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70b00703a0 0x7f70b00ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.534+0000 7f70ae7fc640 1 -- 172.21.15.119:0/1602946678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a800bd40 con 0x7f70b00cb2e0 2023-11-08T19:31:43.543 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.535+0000 7f70ae7fc640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00ac8b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f709800c450 tx=0x7f709800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.535+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70980052d0 con 0x7f70b00cb2e0 2023-11-08T19:31:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.535+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7098005430 con 0x7f70b00cb2e0 2023-11-08T19:31:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.535+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098012870 con 0x7f70b00cb2e0 2023-11-08T19:31:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.535+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b00ada30 con 0x7f70b00cb2e0 2023-11-08T19:31:43.544 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.535+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b00acdf0 con 0x7f70b00cb2e0 2023-11-08T19:31:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.536+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b00adf50 con 0x7f70b00cb2e0 2023-11-08T19:31:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.537+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f70980129d0 con 0x7f70b00cb2e0 2023-11-08T19:31:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.538+0000 7f70a7fff640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f709006ae70 0x7f709006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.538+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7694+0+0 (secure 0 0 0) 0x7f7098089a40 con 0x7f70b00cb2e0 2023-11-08T19:31:43.546 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.538+0000 7f70aeffd640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f709006ae70 0x7f709006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.539+0000 7f70aeffd640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f709006ae70 0x7f709006d330 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f70a0006fd0 tx=0x7f70a0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:43.547 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.542+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70b00adf50 con 0x7f70b00cb2e0 2023-11-08T19:31:43.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.771+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70b00adf50 con 0x7f70b00cb2e0 2023-11-08T19:31:43.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.772+0000 7f70a7fff640 1 -- 172.21.15.119:0/1602946678 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4147 (secure 0 0 0) 0x7f70b00adf50 con 0x7f70b00cb2e0 2023-11-08T19:31:43.774 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:43.774 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":97,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:43.299802+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:replay","state_seq":1,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:resolve","state_seq":9,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:43.774 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 97 2023-11-08T19:31:43.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f709006ae70 msgr2=0x7f709006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f709006ae70 0x7f709006d330 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f70a0006fd0 tx=0x7f70a0008040 comp rx=0 tx=0).stop 2023-11-08T19:31:43.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 msgr2=0x7f70b00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:43.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00ac8b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f709800c450 tx=0x7f709800c910 comp rx=0 tx=0).stop 2023-11-08T19:31:43.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 shutdown_connections 2023-11-08T19:31:43.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f709006ae70 0x7f709006d330 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f70b0159d80 0x7f70b0158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f70b00cb2e0 0x7f70b00ac8b0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 --2- 172.21.15.119:0/1602946678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f70b00703a0 0x7f70b00ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 >> 172.21.15.119:0/1602946678 conn(0x7f70b00b06e0 msgr2=0x7f70b006f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 shutdown_connections 2023-11-08T19:31:43.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:43.775+0000 7f70b584b640 1 -- 172.21.15.119:0/1602946678 wait complete. 2023-11-08T19:31:43.795 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=30 retry_count=0)... 2023-11-08T19:31:45.393 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-08T19:31:48.797 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:49.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.012+0000 7fb271767640 1 Processor -- start 2023-11-08T19:31:49.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.012+0000 7fb271767640 1 -- start start 2023-11-08T19:31:49.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb271767640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 0x7fb26c0caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb271767640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb26c0c9010 0x7fb26c0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb271767640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26c0ae3c0 con 0x7fb26c15b910 2023-11-08T19:31:49.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb271767640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb26c0adb20 con 0x7fb26c0c9010 2023-11-08T19:31:49.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb271767640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0ade00 con 0x7fb26c155190 2023-11-08T19:31:49.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb270f66640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fb26c15b910 0x7fb26c06a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52510/0 (socket says 172.21.15.119:52510) 2023-11-08T19:31:49.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.014+0000 7fb270f66640 1 -- 172.21.15.119:0/4125649618 learned_addr learned my addr 172.21.15.119:0/4125649618 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26bfff640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 0x7fb26c0caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26b7fe640 1 --2- 172.21.15.119:0/4125649618 >> v2:172.21.15.188:3300/0 conn(0x7fb26c0c9010 0x7fb26c0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286642849 0 0) 0x7fb26c0ae3c0 con 0x7fb26c15b910 2023-11-08T19:31:49.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb254003520 con 0x7fb26c15b910 2023-11-08T19:31:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1789070537 0 0) 0x7fb254003520 con 0x7fb26c15b910 2023-11-08T19:31:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 msgr2=0x7fb26c0caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 0x7fb26c0caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 >> v2:172.21.15.188:3300/0 conn(0x7fb26c0c9010 msgr2=0x7fb26c0c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 --2- 172.21.15.119:0/4125649618 >> v2:172.21.15.188:3300/0 conn(0x7fb26c0c9010 0x7fb26c0c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c0ae0e0 con 0x7fb26c15b910 2023-11-08T19:31:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fb2640019f0 con 0x7fb26c15b910 2023-11-08T19:31:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 >> v1:172.21.15.119:6789/0 conn(0x7fb26c15b910 legacy=0x7fb26c06a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 0x7fb254003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb254003ef0 0x7fb2540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb254005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0adb20 con 0x7fb254004d00 2023-11-08T19:31:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c0ade00 con 0x7fb254003ef0 2023-11-08T19:31:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.015+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb25400b9d0 con 0x7fb26c155190 2023-11-08T19:31:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb270f66640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb254005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb26bfff640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 0x7fb254003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb270f66640 1 -- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 msgr2=0x7fb254003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb270f66640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c155190 0x7fb254003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb26b7fe640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb254003ef0 0x7fb2540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb270f66640 1 -- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb254003ef0 msgr2=0x7fb2540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:49.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb270f66640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb254003ef0 0x7fb2540047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.016+0000 7fb270f66640 1 -- 172.21.15.119:0/4125649618 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254003520 con 0x7fb254004d00 2023-11-08T19:31:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.017+0000 7fb270f66640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb254005180 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb26400a900 tx=0x7fb2540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317f4ae6f22091d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.017+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb264034630 con 0x7fb254004d00 2023-11-08T19:31:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.017+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb26402f070 con 0x7fb254004d00 2023-11-08T19:31:49.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.017+0000 7fb26affd640 1 -- 172.21.15.119:0/4125649618 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2640349d0 con 0x7fb254004d00 2023-11-08T19:31:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb271767640 1 -- 172.21.15.119:0/4125649618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 msgr2=0x7fb254005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb271767640 1 --2- 172.21.15.119:0/4125649618 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb254005180 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fb26400a900 tx=0x7fb2540057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb270f66640 1 -- 172.21.15.119:0/4125649618 reap_dead start 2023-11-08T19:31:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb271767640 1 -- 172.21.15.119:0/4125649618 shutdown_connections 2023-11-08T19:31:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb271767640 1 -- 172.21.15.119:0/4125649618 >> 172.21.15.119:0/4125649618 conn(0x7fb26c0b1020 msgr2=0x7fb26c0ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb271767640 1 -- 172.21.15.119:0/4125649618 shutdown_connections 2023-11-08T19:31:49.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.018+0000 7fb271767640 1 -- 172.21.15.119:0/4125649618 wait complete. 2023-11-08T19:31:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.020+0000 7fb271767640 1 Processor -- start 2023-11-08T19:31:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.020+0000 7fb271767640 1 -- start start 2023-11-08T19:31:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb271767640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb26c150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb271767640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb26c0b16b0 0x7fb26c1540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb271767640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c0b1b30 0x7fb26c1545e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb271767640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c1f2360 con 0x7fb254004d00 2023-11-08T19:31:49.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb271767640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c1f16e0 con 0x7fb26c0b16b0 2023-11-08T19:31:49.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb271767640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb26c1f19c0 con 0x7fb26c0b1b30 2023-11-08T19:31:49.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb26bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb26c150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb26bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb26c150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52478/0 (socket says 172.21.15.119:52478) 2023-11-08T19:31:49.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.021+0000 7fb26bfff640 1 -- 172.21.15.119:0/1729673291 learned_addr learned my addr 172.21.15.119:0/1729673291 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:49.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26b7fe640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb26c0b16b0 0x7fb26c1540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26bfff640 1 -- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c0b1b30 msgr2=0x7fb26c1545e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb270f66640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c0b1b30 0x7fb26c1545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26bfff640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c0b1b30 0x7fb26c1545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26bfff640 1 -- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb26c0b16b0 msgr2=0x7fb26c1540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26bfff640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb26c0b16b0 0x7fb26c1540a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26bfff640 1 -- 172.21.15.119:0/1729673291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb26c1f1ca0 con 0x7fb254004d00 2023-11-08T19:31:49.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb26bfff640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb26c150340 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb25c010860 tx=0x7fb25c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:49.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.022+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c009610 con 0x7fb254004d00 2023-11-08T19:31:49.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.023+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25c00eb60 con 0x7fb254004d00 2023-11-08T19:31:49.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.023+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb26c1f1f80 con 0x7fb254004d00 2023-11-08T19:31:49.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.023+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25c01c400 con 0x7fb254004d00 2023-11-08T19:31:49.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.023+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb26c150b90 con 0x7fb254004d00 2023-11-08T19:31:49.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.024+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb26c0b3670 con 0x7fb254004d00 2023-11-08T19:31:49.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.025+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb25c01c560 con 0x7fb254004d00 2023-11-08T19:31:49.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.026+0000 7fb268ff9640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb25806ae70 0x7fb25806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.026+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7694+0+0 (secure 0 0 0) 0x7fb25c08a380 con 0x7fb254004d00 2023-11-08T19:31:49.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.026+0000 7fb26b7fe640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb25806ae70 0x7fb25806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.027+0000 7fb26b7fe640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb25806ae70 0x7fb25806d330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb26c15eee0 tx=0x7fb260008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:49.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.030+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb25c054d20 con 0x7fb254004d00 2023-11-08T19:31:49.262 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.261+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb26c04e490 con 0x7fb254004d00 2023-11-08T19:31:49.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.262+0000 7fb268ff9640 1 -- 172.21.15.119:0/1729673291 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb25c058340 con 0x7fb254004d00 2023-11-08T19:31:49.264 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:49.264 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:44.316944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:49.264 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 99 2023-11-08T19:31:49.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb25806ae70 msgr2=0x7fb25806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb25806ae70 0x7fb25806d330 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb26c15eee0 tx=0x7fb260008040 comp rx=0 tx=0).stop 2023-11-08T19:31:49.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 msgr2=0x7fb26c150340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb26c150340 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fb25c010860 tx=0x7fb25c010d20 comp rx=0 tx=0).stop 2023-11-08T19:31:49.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 shutdown_connections 2023-11-08T19:31:49.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fb25806ae70 0x7fb25806d330 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb26c0b1b30 0x7fb26c1545e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb26c0b16b0 0x7fb26c1540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 --2- 172.21.15.119:0/1729673291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fb254004d00 0x7fb26c150340 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 >> 172.21.15.119:0/1729673291 conn(0x7fb26c0b1020 msgr2=0x7fb26c0ce9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:49.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 shutdown_connections 2023-11-08T19:31:49.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.265+0000 7fb271767640 1 -- 172.21.15.119:0/1729673291 wait complete. 2023-11-08T19:31:49.285 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-11-08T19:31:49.286 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:49.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.496+0000 7f35b6d73640 1 Processor -- start 2023-11-08T19:31:49.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.497+0000 7f35b6d73640 1 -- start start 2023-11-08T19:31:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.498+0000 7f35b6d73640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35b0165a50 0x7f35b0167e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.498+0000 7f35b6d73640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b006d060 con 0x7f35b015d470 2023-11-08T19:31:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.498+0000 7f35b6d73640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b015d980 con 0x7f35b006a030 2023-11-08T19:31:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.498+0000 7f35b6d73640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b006bc00 con 0x7f35b0165a50 2023-11-08T19:31:49.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b6572640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35b0165a50 0x7f35b0167e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b5570640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f35b015d470 0x7f35b00ade20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52518/0 (socket says 172.21.15.119:52518) 2023-11-08T19:31:49.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b6572640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f35b0165a50 0x7f35b0167e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52494/0 (socket says 172.21.15.119:52494) 2023-11-08T19:31:49.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b5570640 1 -- 172.21.15.119:0/1318323923 learned_addr learned my addr 172.21.15.119:0/1318323923 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:49.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b6572640 1 -- 172.21.15.119:0/1318323923 >> v1:172.21.15.188:6789/0 conn(0x7f35b006a030 legacy=0x7f35b006a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b6572640 1 -- 172.21.15.119:0/1318323923 >> v1:172.21.15.119:6789/0 conn(0x7f35b015d470 legacy=0x7f35b00ade20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b6572640 1 -- 172.21.15.119:0/1318323923 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b015dc60 con 0x7f35b0165a50 2023-11-08T19:31:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.499+0000 7f35b4d6f640 1 -- 172.21.15.119:0/1318323923 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2581247777 0 0) 0x7f35b006d060 con 0x7f35b015d470 2023-11-08T19:31:49.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.500+0000 7f35b6572640 1 --2- 172.21.15.119:0/1318323923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b0165a50 0x7f35b0167e40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009d90 tx=0x7f35a802bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db2b80071d1a49f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.500+0000 7f35b4d6f640 1 -- 172.21.15.119:0/1318323923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a802ccf0 con 0x7f35b0165a50 2023-11-08T19:31:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.500+0000 7f35b4d6f640 1 -- 172.21.15.119:0/1318323923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a802ce50 con 0x7f35b0165a50 2023-11-08T19:31:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.500+0000 7f35b4d6f640 1 -- 172.21.15.119:0/1318323923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8034660 con 0x7f35b0165a50 2023-11-08T19:31:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 -- 172.21.15.119:0/1318323923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b0165a50 msgr2=0x7f35b0167e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 --2- 172.21.15.119:0/1318323923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b0165a50 0x7f35b0167e40 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f35a8009d90 tx=0x7f35a802bf40 comp rx=0 tx=0).stop 2023-11-08T19:31:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 -- 172.21.15.119:0/1318323923 shutdown_connections 2023-11-08T19:31:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 --2- 172.21.15.119:0/1318323923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b0165a50 0x7f35b0167e40 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 -- 172.21.15.119:0/1318323923 >> 172.21.15.119:0/1318323923 conn(0x7f35b00b1020 msgr2=0x7f35b00ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 -- 172.21.15.119:0/1318323923 shutdown_connections 2023-11-08T19:31:49.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.501+0000 7f35b6d73640 1 -- 172.21.15.119:0/1318323923 wait complete. 2023-11-08T19:31:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.503+0000 7f35b6d73640 1 Processor -- start 2023-11-08T19:31:49.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.503+0000 7f35b6d73640 1 -- start start 2023-11-08T19:31:49.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b6d73640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 0x7f35b0145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b6d73640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b015d470 0x7f35b015ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b6d73640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35b0165a50 0x7f35b0154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b6d73640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0145670 con 0x7f35b006a030 2023-11-08T19:31:49.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b6d73640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0146560 con 0x7f35b015d470 2023-11-08T19:31:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b6d73640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01469f0 con 0x7f35b0165a50 2023-11-08T19:31:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b5d71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 0x7f35b0145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b5d71640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 0x7f35b0145130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52510/0 (socket says 172.21.15.119:52510) 2023-11-08T19:31:49.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.504+0000 7f35b5d71640 1 -- 172.21.15.119:0/4162502043 learned_addr learned my addr 172.21.15.119:0/4162502043 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5570640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b015d470 0x7f35b015ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b6572640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35b0165a50 0x7f35b0154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5d71640 1 -- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35b0165a50 msgr2=0x7f35b0154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5d71640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35b0165a50 0x7f35b0154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5d71640 1 -- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b015d470 msgr2=0x7f35b015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5d71640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b015d470 0x7f35b015ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5d71640 1 -- 172.21.15.119:0/4162502043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8009a70 con 0x7f35b006a030 2023-11-08T19:31:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5570640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b015d470 0x7f35b015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:49.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b5d71640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 0x7f35b0145130 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00c450 tx=0x7f35ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac0052d0 con 0x7f35b006a030 2023-11-08T19:31:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35ac005430 con 0x7f35b006a030 2023-11-08T19:31:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b0145950 con 0x7f35b006a030 2023-11-08T19:31:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35ac012870 con 0x7f35b006a030 2023-11-08T19:31:49.513 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.505+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b0145f10 con 0x7f35b006a030 2023-11-08T19:31:49.514 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.507+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b00ce760 con 0x7f35b006a030 2023-11-08T19:31:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.507+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f35ac0129d0 con 0x7f35b006a030 2023-11-08T19:31:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.508+0000 7f35a67fc640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f359406aef0 0x7f359406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.508+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 7694+0+0 (secure 0 0 0) 0x7f35ac0897a0 con 0x7f35b006a030 2023-11-08T19:31:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.509+0000 7f35b5570640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f359406aef0 0x7f359406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:49.515 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.509+0000 7f35b5570640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f359406aef0 0x7f359406d3b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f35b01f5a90 tx=0x7f35a0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:49.516 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.513+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f35ac054200 con 0x7f35b006a030 2023-11-08T19:31:49.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.741+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35b004e490 con 0x7f35b006a030 2023-11-08T19:31:49.743 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.742+0000 7f35a67fc640 1 -- 172.21.15.119:0/4162502043 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f35ac057820 con 0x7f35b006a030 2023-11-08T19:31:49.743 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:49.743 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:44.316944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:49.744 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 99 2023-11-08T19:31:49.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.744+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f359406aef0 msgr2=0x7f359406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.744+0000 7f35b6d73640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f359406aef0 0x7f359406d3b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f35b01f5a90 tx=0x7f35a0009040 comp rx=0 tx=0).stop 2023-11-08T19:31:49.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.744+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 msgr2=0x7f35b0145130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.744+0000 7f35b6d73640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 0x7f35b0145130 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f35ac00c450 tx=0x7f35ac00c910 comp rx=0 tx=0).stop 2023-11-08T19:31:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 shutdown_connections 2023-11-08T19:31:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f359406aef0 0x7f359406d3b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f35b0165a50 0x7f35b0154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f35b015d470 0x7f35b015ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 --2- 172.21.15.119:0/4162502043 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f35b006a030 0x7f35b0145130 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:49.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 >> 172.21.15.119:0/4162502043 conn(0x7f35b00b1020 msgr2=0x7f35b0153170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:49.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 shutdown_connections 2023-11-08T19:31:49.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:49.745+0000 7f35b6d73640 1 -- 172.21.15.119:0/4162502043 wait complete. 2023-11-08T19:31:49.766 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-08T19:31:50.576 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:50.585 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:31:50.767 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:50.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.996+0000 7f4f69d1d640 1 Processor -- start 2023-11-08T19:31:50.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.997+0000 7f4f69d1d640 1 -- start start 2023-11-08T19:31:51.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f69d1d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f64159d80 0x7f4f640ca080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:51.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f69d1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640703a0 0x7f4f6415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:51.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f69d1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f6415dd60 con 0x7f4f640cb2e0 2023-11-08T19:31:51.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f69d1d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f6415da30 con 0x7f4f64159d80 2023-11-08T19:31:51.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f69d1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640ca8b0 con 0x7f4f640703a0 2023-11-08T19:31:51.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f637fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f64159d80 0x7f4f640ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:51.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f637fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f4f64159d80 0x7f4f640ca080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52514/0 (socket says 172.21.15.119:52514) 2023-11-08T19:31:51.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.998+0000 7f4f637fe640 1 -- 172.21.15.119:0/3922162179 learned_addr learned my addr 172.21.15.119:0/3922162179 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:51.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.999+0000 7f4f5affd640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640703a0 0x7f4f6415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:51.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.999+0000 7f4f637fe640 1 -- 172.21.15.119:0/3922162179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640703a0 msgr2=0x7f4f6415e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:51.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.999+0000 7f4f637fe640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640703a0 0x7f4f6415e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.999+0000 7f4f637fe640 1 -- 172.21.15.119:0/3922162179 >> v1:172.21.15.188:6789/0 conn(0x7f4f640cb2e0 legacy=0x7f4f6406c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.999+0000 7f4f637fe640 1 -- 172.21.15.119:0/3922162179 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f640cab90 con 0x7f4f64159d80 2023-11-08T19:31:51.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:50.999+0000 7f4f5affd640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640703a0 0x7f4f6415e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.000+0000 7f4f637fe640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f64159d80 0x7f4f640ca080 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009870 tx=0x7f4f5402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e59a1cac3b416e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:51.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.000+0000 7f4f62ffd640 1 -- 172.21.15.119:0/3922162179 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54031640 con 0x7f4f64159d80 2023-11-08T19:31:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.000+0000 7f4f62ffd640 1 -- 172.21.15.119:0/3922162179 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f540317a0 con 0x7f4f64159d80 2023-11-08T19:31:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.000+0000 7f4f62ffd640 1 -- 172.21.15.119:0/3922162179 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54031b30 con 0x7f4f64159d80 2023-11-08T19:31:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 -- 172.21.15.119:0/3922162179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f64159d80 msgr2=0x7f4f640ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f64159d80 0x7f4f640ca080 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009870 tx=0x7f4f5402b8f0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 -- 172.21.15.119:0/3922162179 shutdown_connections 2023-11-08T19:31:51.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640703a0 0x7f4f6415e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/3922162179 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f64159d80 0x7f4f640ca080 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 -- 172.21.15.119:0/3922162179 >> 172.21.15.119:0/3922162179 conn(0x7f4f640b06e0 msgr2=0x7f4f640b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:51.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 -- 172.21.15.119:0/3922162179 shutdown_connections 2023-11-08T19:31:51.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.001+0000 7f4f69d1d640 1 -- 172.21.15.119:0/3922162179 wait complete. 2023-11-08T19:31:51.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.003+0000 7f4f69d1d640 1 Processor -- start 2023-11-08T19:31:51.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.003+0000 7f4f69d1d640 1 -- start start 2023-11-08T19:31:51.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f69d1d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 0x7f4f640c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:51.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f69d1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640cb2e0 0x7f4f640c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:51.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f69d1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f64159d80 0x7f4f6415a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:51.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f69d1d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c74b0 con 0x7f4f640703a0 2023-11-08T19:31:51.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f69d1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f64151760 con 0x7f4f64159d80 2023-11-08T19:31:51.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f69d1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f640c5a10 con 0x7f4f640cb2e0 2023-11-08T19:31:51.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f637fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 0x7f4f640c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:51.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f637fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 0x7f4f640c61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52520/0 (socket says 172.21.15.119:52520) 2023-11-08T19:31:51.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.004+0000 7f4f637fe640 1 -- 172.21.15.119:0/828915964 learned_addr learned my addr 172.21.15.119:0/828915964 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:51.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f5affd640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640cb2e0 0x7f4f640c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:51.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f637fe640 1 -- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640cb2e0 msgr2=0x7f4f640c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:51.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f63fff640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f64159d80 0x7f4f6415a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:51.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f637fe640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640cb2e0 0x7f4f640c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f637fe640 1 -- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f64159d80 msgr2=0x7f4f6415a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:51.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f637fe640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f64159d80 0x7f4f6415a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f637fe640 1 -- 172.21.15.119:0/828915964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f540094d0 con 0x7f4f640703a0 2023-11-08T19:31:51.013 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f637fe640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 0x7f4f640c61c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009870 tx=0x7f4f540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:51.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5403ac60 con 0x7f4f640703a0 2023-11-08T19:31:51.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5403ade0 con 0x7f4f640703a0 2023-11-08T19:31:51.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f54044630 con 0x7f4f640703a0 2023-11-08T19:31:51.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.005+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6415a970 con 0x7f4f640703a0 2023-11-08T19:31:51.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.006+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f640c6b10 con 0x7f4f640703a0 2023-11-08T19:31:51.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.007+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6404e490 con 0x7f4f640703a0 2023-11-08T19:31:51.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.008+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f5402f050 con 0x7f4f640703a0 2023-11-08T19:31:51.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.008+0000 7f4f60ff9640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4f4406ae70 0x7f4f4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:51.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.009+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f4f540abeb0 con 0x7f4f640703a0 2023-11-08T19:31:51.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.009+0000 7f4f5affd640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4f4406ae70 0x7f4f4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:51.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.009+0000 7f4f5affd640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4f4406ae70 0x7f4f4406d330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4f64156e20 tx=0x7f4f4c0084d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:51.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.013+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f54076800 con 0x7f4f640703a0 2023-11-08T19:31:51.242 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.241+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f6415d810 con 0x7f4f640703a0 2023-11-08T19:31:51.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.242+0000 7f4f60ff9640 1 -- 172.21.15.119:0/828915964 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f4f54035020 con 0x7f4f640703a0 2023-11-08T19:31:51.243 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:51.244 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":99,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:44.316944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:reconnect","state_seq":4,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:reconnect","state_seq":12,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:51.244 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 99 2023-11-08T19:31:51.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.244+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4f4406ae70 msgr2=0x7f4f4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:51.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4f4406ae70 0x7f4f4406d330 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4f64156e20 tx=0x7f4f4c0084d0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 msgr2=0x7f4f640c61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 0x7f4f640c61c0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009870 tx=0x7f4f540046c0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 shutdown_connections 2023-11-08T19:31:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f4f4406ae70 0x7f4f4406d330 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4f64159d80 0x7f4f6415a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4f640cb2e0 0x7f4f640c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 --2- 172.21.15.119:0/828915964 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f4f640703a0 0x7f4f640c61c0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 >> 172.21.15.119:0/828915964 conn(0x7f4f640b06e0 msgr2=0x7f4f640cc1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 shutdown_connections 2023-11-08T19:31:51.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:51.245+0000 7f4f69d1d640 1 -- 172.21.15.119:0/828915964 wait complete. 2023-11-08T19:31:51.263 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:reconnect 2023-11-08T19:31:52.265 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:52.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.485+0000 7f0564887640 1 Processor -- start 2023-11-08T19:31:52.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.486+0000 7f0564887640 1 -- start start 2023-11-08T19:31:52.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.487+0000 7f0564887640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f05600cbc10 0x7f056006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.487+0000 7f0564887640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0560070710 con 0x7f056006fef0 2023-11-08T19:31:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.487+0000 7f0564887640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05600709f0 con 0x7f056015d100 2023-11-08T19:31:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.487+0000 7f0564887640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0560070fb0 con 0x7f05600cbc10 2023-11-08T19:31:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.487+0000 7f055ed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f056006fef0 0x7f056006c090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:52532/0 (socket says 172.21.15.119:52532) 2023-11-08T19:31:52.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.487+0000 7f055ed9d640 1 -- 172.21.15.119:0/1978008970 learned_addr learned my addr 172.21.15.119:0/1978008970 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:31:52.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f0557fff640 1 --2- 172.21.15.119:0/1978008970 >> v2:172.21.15.188:3300/0 conn(0x7f05600cbc10 0x7f056006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220034108 0 0) 0x7f0560070710 con 0x7f056006fef0 2023-11-08T19:31:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054c003520 con 0x7f056006fef0 2023-11-08T19:31:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3053719656 0 0) 0x7f054c003520 con 0x7f056006fef0 2023-11-08T19:31:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 >> v2:172.21.15.188:3300/0 conn(0x7f05600cbc10 msgr2=0x7f056006f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:52.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 --2- 172.21.15.119:0/1978008970 >> v2:172.21.15.188:3300/0 conn(0x7f05600cbc10 0x7f056006f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 >> v1:172.21.15.188:6789/0 conn(0x7f056015d100 legacy=0x7f05600cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560070cd0 con 0x7f056006fef0 2023-11-08T19:31:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f05580019f0 con 0x7f056006fef0 2023-11-08T19:31:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 >> v1:172.21.15.119:6789/0 conn(0x7f056006fef0 legacy=0x7f056006c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054c0039c0 0x7f054c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560070fb0 con 0x7f054c0047c0 2023-11-08T19:31:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00c120 con 0x7f054c0051d0 2023-11-08T19:31:52.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.488+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00c2e0 con 0x7f054c0039c0 2023-11-08T19:31:52.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f0557fff640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f0557fff640 1 -- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054c0039c0 msgr2=0x7f054c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:52.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f055e59c640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054c0039c0 0x7f054c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f055ed9d640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f0557fff640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f054c0039c0 0x7f054c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f0557fff640 1 -- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054c0051d0 msgr2=0x7f054c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:52.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f0557fff640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f0557fff640 1 -- 172.21.15.119:0/1978008970 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c003520 con 0x7f054c0047c0 2023-11-08T19:31:52.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f055ed9d640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f054c0051d0 0x7f054c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:52.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.489+0000 7f055ed9d640 1 -- 172.21.15.119:0/1978008970 reap_dead start 2023-11-08T19:31:52.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.490+0000 7f0557fff640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0548009a90 tx=0x7f054c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aa010eb9aee052f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.490+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054802c680 con 0x7f054c0047c0 2023-11-08T19:31:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.490+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054802c7e0 con 0x7f054c0047c0 2023-11-08T19:31:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.490+0000 7f055dd9b640 1 -- 172.21.15.119:0/1978008970 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054802cb70 con 0x7f054c0047c0 2023-11-08T19:31:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 -- 172.21.15.119:0/1978008970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 msgr2=0x7f054c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f0548009a90 tx=0x7f054c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 -- 172.21.15.119:0/1978008970 shutdown_connections 2023-11-08T19:31:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 --2- 172.21.15.119:0/1978008970 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f054c004c40 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 -- 172.21.15.119:0/1978008970 >> 172.21.15.119:0/1978008970 conn(0x7f05600b06e0 msgr2=0x7f05600b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 -- 172.21.15.119:0/1978008970 shutdown_connections 2023-11-08T19:31:52.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.491+0000 7f0564887640 1 -- 172.21.15.119:0/1978008970 wait complete. 2023-11-08T19:31:52.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.493+0000 7f0564887640 1 Processor -- start 2023-11-08T19:31:52.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.493+0000 7f0564887640 1 -- start start 2023-11-08T19:31:52.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f0564887640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f05601f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f0564887640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056014f000 0x7f05601e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f0564887640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05601e7e60 0x7f05601f3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f0564887640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056015c430 con 0x7f054c0047c0 2023-11-08T19:31:52.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f0564887640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601f47c0 con 0x7f05601e7e60 2023-11-08T19:31:52.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f0564887640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05601e82e0 con 0x7f056014f000 2023-11-08T19:31:52.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f055e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f05601f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f055e59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f05601f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52536/0 (socket says 172.21.15.119:52536) 2023-11-08T19:31:52.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.494+0000 7f055e59c640 1 -- 172.21.15.119:0/2849253141 learned_addr learned my addr 172.21.15.119:0/2849253141 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:52.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f0557fff640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056014f000 0x7f05601e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055ed9d640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05601e7e60 0x7f05601f3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055e59c640 1 -- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056014f000 msgr2=0x7f05601e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055e59c640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056014f000 0x7f05601e7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055e59c640 1 -- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05601e7e60 msgr2=0x7f05601f3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055e59c640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05601e7e60 0x7f05601f3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055e59c640 1 -- 172.21.15.119:0/2849253141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05601e8570 con 0x7f054c0047c0 2023-11-08T19:31:52.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f055e59c640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f05601f4280 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f055000d540 tx=0x7f055000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:52.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550005480 con 0x7f054c0047c0 2023-11-08T19:31:52.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500035f0 con 0x7f054c0047c0 2023-11-08T19:31:52.506 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550003980 con 0x7f054c0047c0 2023-11-08T19:31:52.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05600c5c60 con 0x7f054c0047c0 2023-11-08T19:31:52.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.495+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05600c6220 con 0x7f054c0047c0 2023-11-08T19:31:52.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.496+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056004e490 con 0x7f054c0047c0 2023-11-08T19:31:52.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.497+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0550003ae0 con 0x7f054c0047c0 2023-11-08T19:31:52.507 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.498+0000 7f0556ffd640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f054006ae70 0x7f054006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:52.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.498+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f0550089d30 con 0x7f054c0047c0 2023-11-08T19:31:52.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.498+0000 7f0557fff640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f054006ae70 0x7f054006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:52.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.499+0000 7f0557fff640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f054006ae70 0x7f054006d330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0548036000 tx=0x7f0548030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:52.508 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.502+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05500545f0 con 0x7f054c0047c0 2023-11-08T19:31:52.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.728+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05601e6cf0 con 0x7f054c0047c0 2023-11-08T19:31:52.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.729+0000 7f0556ffd640 1 -- 172.21.15.119:0/2849253141 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f0550057c10 con 0x7f054c0047c0 2023-11-08T19:31:52.731 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:52.731 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":100,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:50.923502+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:rejoin","state_seq":14,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:52.732 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 100 2023-11-08T19:31:52.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f054006ae70 msgr2=0x7f054006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f054006ae70 0x7f054006d330 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0548036000 tx=0x7f0548030000 comp rx=0 tx=0).stop 2023-11-08T19:31:52.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 msgr2=0x7f05601f4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:52.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f05601f4280 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f055000d540 tx=0x7f055000da00 comp rx=0 tx=0).stop 2023-11-08T19:31:52.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 shutdown_connections 2023-11-08T19:31:52.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f054006ae70 0x7f054006d330 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f05601e7e60 0x7f05601f3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f056014f000 0x7f05601e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 --2- 172.21.15.119:0/2849253141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f054c0047c0 0x7f05601f4280 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:52.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 >> 172.21.15.119:0/2849253141 conn(0x7f05600b06e0 msgr2=0x7f05600c1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:52.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 shutdown_connections 2023-11-08T19:31:52.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:52.732+0000 7f0564887640 1 -- 172.21.15.119:0/2849253141 wait complete. 2023-11-08T19:31:52.751 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:rejoin 2023-11-08T19:31:53.753 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:53.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.978+0000 7fd7704f3640 1 Processor -- start 2023-11-08T19:31:53.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.978+0000 7fd7704f3640 1 -- start start 2023-11-08T19:31:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.979+0000 7fd7704f3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76815d100 0x7fd7680cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:53.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd7704f3640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7680cba40 0x7fd76806f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd7704f3640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7680cbea0 con 0x7fd76806fef0 2023-11-08T19:31:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd7704f3640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd768070fb0 con 0x7fd7680cba40 2023-11-08T19:31:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd7704f3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd768070710 con 0x7fd76815d100 2023-11-08T19:31:53.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd76da8e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7680cba40 0x7fd76806f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd76da8e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fd7680cba40 0x7fd76806f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52552/0 (socket says 172.21.15.119:52552) 2023-11-08T19:31:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd76da8e640 1 -- 172.21.15.119:0/3174556659 learned_addr learned my addr 172.21.15.119:0/3174556659 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:53.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd76e28f640 1 --2- 172.21.15.119:0/3174556659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76815d100 0x7fd7680cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd76da8e640 1 -- 172.21.15.119:0/3174556659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76815d100 msgr2=0x7fd7680cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.980+0000 7fd76da8e640 1 --2- 172.21.15.119:0/3174556659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76815d100 0x7fd7680cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.981+0000 7fd76da8e640 1 -- 172.21.15.119:0/3174556659 >> v1:172.21.15.119:6789/0 conn(0x7fd76806fef0 legacy=0x7fd7680cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.981+0000 7fd76da8e640 1 -- 172.21.15.119:0/3174556659 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7680709f0 con 0x7fd7680cba40 2023-11-08T19:31:53.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.982+0000 7fd76da8e640 1 --2- 172.21.15.119:0/3174556659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7680cba40 0x7fd76806f9b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fd758009870 tx=0x7fd75802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5881a494ecb33374 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.982+0000 7fd76d28d640 1 -- 172.21.15.119:0/3174556659 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758031870 con 0x7fd7680cba40 2023-11-08T19:31:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.982+0000 7fd76d28d640 1 -- 172.21.15.119:0/3174556659 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7580319d0 con 0x7fd7680cba40 2023-11-08T19:31:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.982+0000 7fd76d28d640 1 -- 172.21.15.119:0/3174556659 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758031d60 con 0x7fd7680cba40 2023-11-08T19:31:53.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 -- 172.21.15.119:0/3174556659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7680cba40 msgr2=0x7fd76806f9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 --2- 172.21.15.119:0/3174556659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7680cba40 0x7fd76806f9b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fd758009870 tx=0x7fd75802b910 comp rx=0 tx=0).stop 2023-11-08T19:31:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 -- 172.21.15.119:0/3174556659 shutdown_connections 2023-11-08T19:31:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 --2- 172.21.15.119:0/3174556659 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7680cba40 0x7fd76806f9b0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 --2- 172.21.15.119:0/3174556659 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76815d100 0x7fd7680cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 -- 172.21.15.119:0/3174556659 >> 172.21.15.119:0/3174556659 conn(0x7fd7680b06e0 msgr2=0x7fd7680b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:53.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 -- 172.21.15.119:0/3174556659 shutdown_connections 2023-11-08T19:31:53.988 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.983+0000 7fd7704f3640 1 -- 172.21.15.119:0/3174556659 wait complete. 2023-11-08T19:31:53.989 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.984+0000 7fd7704f3640 1 Processor -- start 2023-11-08T19:31:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.985+0000 7fd7704f3640 1 -- start start 2023-11-08T19:31:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.985+0000 7fd7704f3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76806fef0 0x7fd76815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:53.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd7704f3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7680cba40 0x7fd7681e3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd7704f3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 0x7fd7680c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd7704f3640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680c3080 con 0x7fd76815d100 2023-11-08T19:31:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd7704f3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7681e4050 con 0x7fd7680cba40 2023-11-08T19:31:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd7704f3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680c2400 con 0x7fd76806fef0 2023-11-08T19:31:53.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 0x7fd7680c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 0x7fd7680c34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52554/0 (socket says 172.21.15.119:52554) 2023-11-08T19:31:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 -- 172.21.15.119:0/1502293520 learned_addr learned my addr 172.21.15.119:0/1502293520 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 -- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76806fef0 msgr2=0x7fd76815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:53.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76806fef0 0x7fd76815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76da8e640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7680cba40 0x7fd7681e3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 -- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7680cba40 msgr2=0x7fd7681e3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7680cba40 0x7fd7681e3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:53.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.986+0000 7fd76ea90640 1 -- 172.21.15.119:0/1502293520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580094d0 con 0x7fd76815d100 2023-11-08T19:31:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.987+0000 7fd76ea90640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 0x7fd7680c34a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c390 tx=0x7fd76000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.987+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7600052d0 con 0x7fd76815d100 2023-11-08T19:31:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.987+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd760005430 con 0x7fd76815d100 2023-11-08T19:31:53.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.987+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd760003810 con 0x7fd76815d100 2023-11-08T19:31:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.987+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7680c26e0 con 0x7fd76815d100 2023-11-08T19:31:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.987+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7680c2ca0 con 0x7fd76815d100 2023-11-08T19:31:53.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.988+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd76804e490 con 0x7fd76815d100 2023-11-08T19:31:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.989+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd760003970 con 0x7fd76815d100 2023-11-08T19:31:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.990+0000 7fd75effd640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd74c06ae70 0x7fd74c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:53.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.990+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7737+0+0 (secure 0 0 0) 0x7fd760089200 con 0x7fd76815d100 2023-11-08T19:31:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.990+0000 7fd76e28f640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd74c06ae70 0x7fd74c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.991+0000 7fd76e28f640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd74c06ae70 0x7fd74c06d330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7640070b0 tx=0x7fd764007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:53.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:53.995+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd760053b50 con 0x7fd76815d100 2023-11-08T19:31:54.223 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.221+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7680c31c0 con 0x7fd76815d100 2023-11-08T19:31:54.224 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.222+0000 7fd75effd640 1 -- 172.21.15.119:0/1502293520 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7fd760057170 con 0x7fd76815d100 2023-11-08T19:31:54.224 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:54.224 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:54.225 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd74c06ae70 msgr2=0x7fd74c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd74c06ae70 0x7fd74c06d330 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd7640070b0 tx=0x7fd764007040 comp rx=0 tx=0).stop 2023-11-08T19:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 msgr2=0x7fd7680c34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 0x7fd7680c34a0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fd76000c390 tx=0x7fd76000c850 comp rx=0 tx=0).stop 2023-11-08T19:31:54.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 shutdown_connections 2023-11-08T19:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd74c06ae70 0x7fd74c06d330 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd76815d100 0x7fd7680c34a0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7680cba40 0x7fd7681e3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 --2- 172.21.15.119:0/1502293520 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd76806fef0 0x7fd76815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 >> 172.21.15.119:0/1502293520 conn(0x7fd7680b06e0 msgr2=0x7fd7680710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:54.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 shutdown_connections 2023-11-08T19:31:54.229 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.225+0000 7fd7704f3640 1 -- 172.21.15.119:0/1502293520 wait complete. 2023-11-08T19:31:54.246 DEBUG:tasks.cephfs.filesystem:Looked up MDS state for mds.0: up:active 2023-11-08T19:31:54.247 DEBUG:tasks.cephfs.filesystem:reached state 'up:active' in 4.960813522338867s 2023-11-08T19:31:54.247 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.461+0000 7f560d1b6640 1 Processor -- start 2023-11-08T19:31:54.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.462+0000 7f560d1b6640 1 -- start start 2023-11-08T19:31:54.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f560d1b6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56081555b0 0x7f56080ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:54.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f560d1b6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f56080cb4e0 0x7f56080cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:54.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f560d1b6640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56080cbe70 con 0x7f56080cd110 2023-11-08T19:31:54.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f560d1b6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080cd590 con 0x7f56081555b0 2023-11-08T19:31:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f560d1b6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56080cc6c0 con 0x7f56080cb4e0 2023-11-08T19:31:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f5606d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56081555b0 0x7f56080ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.463+0000 7f5606d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f56081555b0 0x7f56080ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52560/0 (socket says 172.21.15.119:52560) 2023-11-08T19:31:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.464+0000 7f5606d9d640 1 -- 172.21.15.119:0/467491890 learned_addr learned my addr 172.21.15.119:0/467491890 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:54.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.464+0000 7f560759e640 1 --2- 172.21.15.119:0/467491890 >> v2:172.21.15.188:3300/0 conn(0x7f56080cb4e0 0x7f56080cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.464+0000 7f5606d9d640 1 -- 172.21.15.119:0/467491890 >> v2:172.21.15.188:3300/0 conn(0x7f56080cb4e0 msgr2=0x7f56080cb930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.464+0000 7f5606d9d640 1 --2- 172.21.15.119:0/467491890 >> v2:172.21.15.188:3300/0 conn(0x7f56080cb4e0 0x7f56080cb930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.464+0000 7f5606d9d640 1 -- 172.21.15.119:0/467491890 >> v1:172.21.15.119:6789/0 conn(0x7f56080cd110 legacy=0x7f5608070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.464+0000 7f5606d9d640 1 -- 172.21.15.119:0/467491890 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5608071770 con 0x7f56081555b0 2023-11-08T19:31:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.465+0000 7f5606d9d640 1 --2- 172.21.15.119:0/467491890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56081555b0 0x7f56080ce040 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f56080ae600 tx=0x7f55f802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696c0edf44da1dd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.465+0000 7f5605d9b640 1 -- 172.21.15.119:0/467491890 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f802ca80 con 0x7f56081555b0 2023-11-08T19:31:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.465+0000 7f5605d9b640 1 -- 172.21.15.119:0/467491890 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f802cbe0 con 0x7f56081555b0 2023-11-08T19:31:54.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.465+0000 7f5605d9b640 1 -- 172.21.15.119:0/467491890 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f80343f0 con 0x7f56081555b0 2023-11-08T19:31:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 -- 172.21.15.119:0/467491890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56081555b0 msgr2=0x7f56080ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 --2- 172.21.15.119:0/467491890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56081555b0 0x7f56080ce040 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f56080ae600 tx=0x7f55f802bee0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 -- 172.21.15.119:0/467491890 shutdown_connections 2023-11-08T19:31:54.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 --2- 172.21.15.119:0/467491890 >> v2:172.21.15.188:3300/0 conn(0x7f56080cb4e0 0x7f56080cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 --2- 172.21.15.119:0/467491890 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56081555b0 0x7f56080ce040 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 -- 172.21.15.119:0/467491890 >> 172.21.15.119:0/467491890 conn(0x7f56080b06e0 msgr2=0x7f56080b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 -- 172.21.15.119:0/467491890 shutdown_connections 2023-11-08T19:31:54.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.466+0000 7f560d1b6640 1 -- 172.21.15.119:0/467491890 wait complete. 2023-11-08T19:31:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.468+0000 7f560d1b6640 1 Processor -- start 2023-11-08T19:31:54.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.468+0000 7f560d1b6640 1 -- start start 2023-11-08T19:31:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f560d1b6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 0x7f56080ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f560d1b6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56080cd110 0x7f560814f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f560d1b6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56081555b0 0x7f56081e34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f560d1b6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c9f60 con 0x7f56080cb4e0 2023-11-08T19:31:54.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f560d1b6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c92e0 con 0x7f56081555b0 2023-11-08T19:31:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f560d1b6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56080c95c0 con 0x7f56080cd110 2023-11-08T19:31:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.469+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 0x7f56080ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 0x7f56080ca380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52568/0 (socket says 172.21.15.119:52568) 2023-11-08T19:31:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 -- 172.21.15.119:0/2982829923 learned_addr learned my addr 172.21.15.119:0/2982829923 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:54.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 -- 172.21.15.119:0/2982829923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56080cd110 msgr2=0x7f560814f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56080cd110 0x7f560814f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 -- 172.21.15.119:0/2982829923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56081555b0 msgr2=0x7f56081e34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56081555b0 0x7f56081e34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 -- 172.21.15.119:0/2982829923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f8009b30 con 0x7f56080cb4e0 2023-11-08T19:31:54.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f5606d9d640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 0x7f56080ca380 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f55f8000f80 tx=0x7f55f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:54.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f8009510 con 0x7f56080cb4e0 2023-11-08T19:31:54.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.470+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f803e450 con 0x7f56080cb4e0 2023-11-08T19:31:54.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.471+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56080c98a0 con 0x7f56080cb4e0 2023-11-08T19:31:54.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.471+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f803e800 con 0x7f56080cb4e0 2023-11-08T19:31:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.471+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560814fa90 con 0x7f56080cb4e0 2023-11-08T19:31:54.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.472+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56080ca1d0 con 0x7f56080cb4e0 2023-11-08T19:31:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.473+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55f802cd50 con 0x7f56080cb4e0 2023-11-08T19:31:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.474+0000 7f55ef7fe640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f55f406aea0 0x7f55f406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.474+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7737+0+0 (secure 0 0 0) 0x7f55f80ac9d0 con 0x7f56080cb4e0 2023-11-08T19:31:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.474+0000 7f560659c640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f55f406aea0 0x7f55f406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:54.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.475+0000 7f560659c640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f55f406aea0 0x7f55f406d360 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f55fc000f60 tx=0x7f55fc0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:54.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.478+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55f8077200 con 0x7f56080cb4e0 2023-11-08T19:31:54.705 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.703+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f560804e490 con 0x7f56080cb4e0 2023-11-08T19:31:54.706 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.704+0000 7f55ef7fe640 1 -- 172.21.15.119:0/2982829923 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f55f807a820 con 0x7f56080cb4e0 2023-11-08T19:31:54.707 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:54.707 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:54.707 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:31:54.709 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f55f406aea0 msgr2=0x7f55f406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f55f406aea0 0x7f55f406d360 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f55fc000f60 tx=0x7f55fc0079b0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 msgr2=0x7f56080ca380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:54.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 0x7f56080ca380 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f55f8000f80 tx=0x7f55f8004a40 comp rx=0 tx=0).stop 2023-11-08T19:31:54.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 shutdown_connections 2023-11-08T19:31:54.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f55f406aea0 0x7f55f406d360 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f56081555b0 0x7f56081e34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f56080cd110 0x7f560814f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 --2- 172.21.15.119:0/2982829923 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f56080cb4e0 0x7f56080ca380 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:54.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 >> 172.21.15.119:0/2982829923 conn(0x7f56080b06e0 msgr2=0x7f56080b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:54.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 shutdown_connections 2023-11-08T19:31:54.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:54.708+0000 7f560d1b6640 1 -- 172.21.15.119:0/2982829923 wait complete. 2023-11-08T19:31:54.729 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:31:54.948 INFO:teuthology.orchestra.run.smithi119.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{"17":[2,3,4,5,6,7,8,9]},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:31:54.949 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": { "17": [ 2, 3, 4, 5, 6, 7, 8, 9 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-08T19:31:54.949 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T19:31:54.950 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:31:55.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.156+0000 7ff5e0183640 1 Processor -- start 2023-11-08T19:31:55.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.156+0000 7ff5e0183640 1 -- start start 2023-11-08T19:31:55.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5e0183640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff5d80ce040 0x7ff5d80cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:55.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5e0183640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d80cb4e0 0x7ff5d80cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:55.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5e0183640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5d80cc6c0 con 0x7ff5d81555b0 2023-11-08T19:31:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5e0183640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d80cbe50 con 0x7ff5d80cb4e0 2023-11-08T19:31:55.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5e0183640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5d8071770 con 0x7ff5d80ce040 2023-11-08T19:31:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5de720640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d80cb4e0 0x7ff5d80cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:55.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5de720640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff5d80cb4e0 0x7ff5d80cb910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52572/0 (socket says 172.21.15.119:52572) 2023-11-08T19:31:55.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5de720640 1 -- 172.21.15.119:0/3967031407 learned_addr learned my addr 172.21.15.119:0/3967031407 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:55.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.158+0000 7ff5dd71e640 1 --2- 172.21.15.119:0/3967031407 >> v2:172.21.15.188:3300/0 conn(0x7ff5d80ce040 0x7ff5d80cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:55.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.159+0000 7ff5de720640 1 -- 172.21.15.119:0/3967031407 >> v2:172.21.15.188:3300/0 conn(0x7ff5d80ce040 msgr2=0x7ff5d80cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:55.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.159+0000 7ff5de720640 1 --2- 172.21.15.119:0/3967031407 >> v2:172.21.15.188:3300/0 conn(0x7ff5d80ce040 0x7ff5d80cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.159+0000 7ff5de720640 1 -- 172.21.15.119:0/3967031407 >> v1:172.21.15.188:6789/0 conn(0x7ff5d81555b0 legacy=0x7ff5d8070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:31:55.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.159+0000 7ff5de720640 1 -- 172.21.15.119:0/3967031407 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d8071a50 con 0x7ff5d80cb4e0 2023-11-08T19:31:55.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.160+0000 7ff5de720640 1 --2- 172.21.15.119:0/3967031407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80cb4e0 0x7ff5d80cb910 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80cc560 tx=0x7ff5d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c84e1d37df94303 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:55.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.160+0000 7ff5dcf1d640 1 -- 172.21.15.119:0/3967031407 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d002c9f0 con 0x7ff5d80cb4e0 2023-11-08T19:31:55.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.160+0000 7ff5dcf1d640 1 -- 172.21.15.119:0/3967031407 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d002cb50 con 0x7ff5d80cb4e0 2023-11-08T19:31:55.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.160+0000 7ff5dcf1d640 1 -- 172.21.15.119:0/3967031407 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d00343f0 con 0x7ff5d80cb4e0 2023-11-08T19:31:55.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 -- 172.21.15.119:0/3967031407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80cb4e0 msgr2=0x7ff5d80cb910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:55.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 --2- 172.21.15.119:0/3967031407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80cb4e0 0x7ff5d80cb910 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7ff5d80cc560 tx=0x7ff5d002be50 comp rx=0 tx=0).stop 2023-11-08T19:31:55.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 -- 172.21.15.119:0/3967031407 shutdown_connections 2023-11-08T19:31:55.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 --2- 172.21.15.119:0/3967031407 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80cb4e0 0x7ff5d80cb910 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 --2- 172.21.15.119:0/3967031407 >> v2:172.21.15.188:3300/0 conn(0x7ff5d80ce040 0x7ff5d80cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 -- 172.21.15.119:0/3967031407 >> 172.21.15.119:0/3967031407 conn(0x7ff5d80b06e0 msgr2=0x7ff5d80b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:55.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 -- 172.21.15.119:0/3967031407 shutdown_connections 2023-11-08T19:31:55.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.161+0000 7ff5e0183640 1 -- 172.21.15.119:0/3967031407 wait complete. 2023-11-08T19:31:55.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.162+0000 7ff5e0183640 1 Processor -- start 2023-11-08T19:31:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.163+0000 7ff5e0183640 1 -- start start 2023-11-08T19:31:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5e0183640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5d80cb4e0 0x7ff5d81447f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5e0183640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 0x7ff5d80c3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5e0183640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5d81555b0 0x7ff5d81f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:55.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5e0183640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d8144d30 con 0x7ff5d80ce040 2023-11-08T19:31:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5e0183640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80c4330 con 0x7ff5d81555b0 2023-11-08T19:31:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5e0183640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5d80c45b0 con 0x7ff5d80cb4e0 2023-11-08T19:31:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 0x7ff5d80c3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:55.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 0x7ff5d80c3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52578/0 (socket says 172.21.15.119:52578) 2023-11-08T19:31:55.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 -- 172.21.15.119:0/332459973 learned_addr learned my addr 172.21.15.119:0/332459973 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:31:55.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5de720640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5d81555b0 0x7ff5d81f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:55.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5ddf1f640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5d80cb4e0 0x7ff5d81447f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:55.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 -- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5d80cb4e0 msgr2=0x7ff5d81447f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:55.170 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5d80cb4e0 0x7ff5d81447f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 -- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5d81555b0 msgr2=0x7ff5d81f22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5d81555b0 0x7ff5d81f22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5dd71e640 1 -- 172.21.15.119:0/332459973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009b30 con 0x7ff5d80ce040 2023-11-08T19:31:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.164+0000 7ff5de720640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5d81555b0 0x7ff5d81f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:31:55.171 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.165+0000 7ff5dd71e640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 0x7ff5d80c3df0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800d800 tx=0x7ff5c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:55.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.165+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800b6c0 con 0x7ff5d80ce040 2023-11-08T19:31:55.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.165+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c800b820 con 0x7ff5d80ce040 2023-11-08T19:31:55.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.165+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c800bbb0 con 0x7ff5d80ce040 2023-11-08T19:31:55.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.165+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5d80c4890 con 0x7ff5d80ce040 2023-11-08T19:31:55.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.165+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5d80c4e50 con 0x7ff5d80ce040 2023-11-08T19:31:55.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.166+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5d804e490 con 0x7ff5d80ce040 2023-11-08T19:31:55.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.167+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5c800bd10 con 0x7ff5d80ce040 2023-11-08T19:31:55.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.168+0000 7ff5ce7fc640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5bc06aec0 0x7ff5bc06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:31:55.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.168+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 7737+0+0 (secure 0 0 0) 0x7ff5c808a1f0 con 0x7ff5d80ce040 2023-11-08T19:31:55.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.168+0000 7ff5ddf1f640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5bc06aec0 0x7ff5bc06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:31:55.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.169+0000 7ff5ddf1f640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5bc06aec0 0x7ff5bc06d380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400a750 tx=0x7ff5d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:31:55.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.172+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5c8054c20 con 0x7ff5d80ce040 2023-11-08T19:31:55.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.392+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5d815dd70 con 0x7ff5d80ce040 2023-11-08T19:31:55.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.393+0000 7ff5ce7fc640 1 -- 172.21.15.119:0/332459973 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7ff5c8058240 con 0x7ff5d80ce040 2023-11-08T19:31:55.394 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:31:55.395 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:31:55.395 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:31:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.395+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5bc06aec0 msgr2=0x7ff5bc06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:55.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.395+0000 7ff5e0183640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5bc06aec0 0x7ff5bc06d380 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff5d400a750 tx=0x7ff5d4008040 comp rx=0 tx=0).stop 2023-11-08T19:31:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.395+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 msgr2=0x7ff5d80c3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:31:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.395+0000 7ff5e0183640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 0x7ff5d80c3df0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7ff5c800d800 tx=0x7ff5c800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 shutdown_connections 2023-11-08T19:31:55.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5bc06aec0 0x7ff5bc06d380 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5d81555b0 0x7ff5d81f22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5d80ce040 0x7ff5d80c3df0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 --2- 172.21.15.119:0/332459973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5d80cb4e0 0x7ff5d81447f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:31:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 >> 172.21.15.119:0/332459973 conn(0x7ff5d80b06e0 msgr2=0x7ff5d806f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:31:55.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 shutdown_connections 2023-11-08T19:31:55.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:31:55.396+0000 7ff5e0183640 1 -- 172.21.15.119:0/332459973 wait complete. 2023-11-08T19:31:55.416 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json dump snaps --server 2023-11-08T19:31:55.639 INFO:teuthology.orchestra.run.smithi119.stdout:{"last_snap":10,"last_created":5,"last_destroyed":9,"pending_noop":[],"snaps":[],"need_to_purge":{"17":[2,3,4,5,6,7,8,9]},"pending_update":[],"pending_destroy":[]} 2023-11-08T19:31:55.640 DEBUG:tasks.cephfs.filesystem:_json_asok output { "last_created": 5, "last_destroyed": 9, "last_snap": 10, "need_to_purge": { "17": [ 2, 3, 4, 5, 6, 7, 8, 9 ] }, "pending_destroy": [], "pending_noop": [], "pending_update": [], "snaps": [] } 2023-11-08T19:31:55.640 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T19:31:55.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:31:55.641 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link set brx.0 up 2023-11-08T19:31:56.239 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:31:56.238+0000 7f89f3cd5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:31:56.239 INFO:tasks.ceph.mds.a.smithi119.stderr:2023-11-08T19:31:56.238+0000 7f89f3cd5640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:32:03.560 INFO:teuthology.orchestra.run.smithi119.stderr:mkdir: cannot create directory ‘d1/dir/.snap/s22’: Permission denied 2023-11-08T19:32:03.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:03.563 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T19:32:03.564 DEBUG:teuthology.orchestra.run.smithi119:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:32:03.683 DEBUG:teuthology.orchestra.run.smithi188:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T19:32:03.808 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T19:32:03.808 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:32:03.809 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:03.871 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:32:03.871 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:03.926 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:32:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.141+0000 7fd7e9f93640 1 Processor -- start 2023-11-08T19:32:04.143 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.142+0000 7fd7e9f93640 1 -- start start 2023-11-08T19:32:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.143+0000 7fd7e9f93640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 0x7fd7e40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.143+0000 7fd7e9f93640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 0x7fd7e40c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.143+0000 7fd7e9f93640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7e40ae3c0 con 0x7fd7e415b910 2023-11-08T19:32:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.143+0000 7fd7e9f93640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7e40adb20 con 0x7fd7e4155190 2023-11-08T19:32:04.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.143+0000 7fd7e9f93640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e40ade00 con 0x7fd7e40c9010 2023-11-08T19:32:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e8f91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 0x7fd7e40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 0x7fd7e40c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 0x7fd7e40c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:51906/0 (socket says 172.21.15.119:51906) 2023-11-08T19:32:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e8f91640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 0x7fd7e40caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:38094/0 (socket says 172.21.15.119:38094) 2023-11-08T19:32:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 -- 172.21.15.119:0/2039676186 learned_addr learned my addr 172.21.15.119:0/2039676186 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:04.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 -- 172.21.15.119:0/2039676186 >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 msgr2=0x7fd7e40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 --2- 172.21.15.119:0/2039676186 >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 0x7fd7e40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 -- 172.21.15.119:0/2039676186 >> v1:172.21.15.188:6789/0 conn(0x7fd7e415b910 legacy=0x7fd7e406a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e3fff640 1 -- 172.21.15.119:0/2039676186 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e40ae0e0 con 0x7fd7e40c9010 2023-11-08T19:32:04.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.144+0000 7fd7e8f91640 1 --2- 172.21.15.119:0/2039676186 >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 0x7fd7e40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:32:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.146+0000 7fd7e3fff640 1 --2- 172.21.15.119:0/2039676186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 0x7fd7e40c9470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009870 tx=0x7fd7d002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4905496a5019c823 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.146+0000 7fd7e37fe640 1 -- 172.21.15.119:0/2039676186 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0031640 con 0x7fd7e40c9010 2023-11-08T19:32:04.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e37fe640 1 -- 172.21.15.119:0/2039676186 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d00317a0 con 0x7fd7e40c9010 2023-11-08T19:32:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e37fe640 1 -- 172.21.15.119:0/2039676186 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d0031b30 con 0x7fd7e40c9010 2023-11-08T19:32:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 -- 172.21.15.119:0/2039676186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 msgr2=0x7fd7e40c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/2039676186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 0x7fd7e40c9470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0009870 tx=0x7fd7d002b8f0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 -- 172.21.15.119:0/2039676186 shutdown_connections 2023-11-08T19:32:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/2039676186 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e40c9010 0x7fd7e40c9470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/2039676186 >> v2:172.21.15.188:3300/0 conn(0x7fd7e4155190 0x7fd7e40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 -- 172.21.15.119:0/2039676186 >> 172.21.15.119:0/2039676186 conn(0x7fd7e40b1020 msgr2=0x7fd7e40ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:04.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 -- 172.21.15.119:0/2039676186 shutdown_connections 2023-11-08T19:32:04.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.147+0000 7fd7e9f93640 1 -- 172.21.15.119:0/2039676186 wait complete. 2023-11-08T19:32:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.149+0000 7fd7e9f93640 1 Processor -- start 2023-11-08T19:32:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.149+0000 7fd7e9f93640 1 -- start start 2023-11-08T19:32:04.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e9f93640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 0x7fd7e41473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e9f93640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e4155190 0x7fd7e4159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e9f93640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7e415b910 0x7fd7e415a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e9f93640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415de20 con 0x7fd7e40c9010 2023-11-08T19:32:04.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e9f93640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415e090 con 0x7fd7e415b910 2023-11-08T19:32:04.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e9f93640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7e415e370 con 0x7fd7e4155190 2023-11-08T19:32:04.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.150+0000 7fd7e8f91640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 0x7fd7e41473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 0x7fd7e41473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46134/0 (socket says 172.21.15.119:46134) 2023-11-08T19:32:04.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 -- 172.21.15.119:0/1361518393 learned_addr learned my addr 172.21.15.119:0/1361518393 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:04.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e9792640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7e415b910 0x7fd7e415a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e3fff640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e4155190 0x7fd7e4159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 -- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e4155190 msgr2=0x7fd7e4159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e4155190 0x7fd7e4159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 -- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7e415b910 msgr2=0x7fd7e415a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7e415b910 0x7fd7e415a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 -- 172.21.15.119:0/1361518393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d8009400 con 0x7fd7e40c9010 2023-11-08T19:32:04.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e3fff640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e4155190 0x7fd7e4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:04.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.151+0000 7fd7e8f91640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 0x7fd7e41473e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800e960 tx=0x7fd7d800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:04.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.152+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d800ca00 con 0x7fd7e40c9010 2023-11-08T19:32:04.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.152+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d800cb60 con 0x7fd7e40c9010 2023-11-08T19:32:04.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.152+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8013470 con 0x7fd7e40c9010 2023-11-08T19:32:04.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.152+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d00094d0 con 0x7fd7e40c9010 2023-11-08T19:32:04.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.152+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7e4147920 con 0x7fd7e40c9010 2023-11-08T19:32:04.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.153+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7e404e490 con 0x7fd7e40c9010 2023-11-08T19:32:04.163 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.154+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd7d80135d0 con 0x7fd7e40c9010 2023-11-08T19:32:04.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.155+0000 7fd7e17fa640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd7c806ae70 0x7fd7c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.155+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7fd7d808a0d0 con 0x7fd7e40c9010 2023-11-08T19:32:04.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.156+0000 7fd7e3fff640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd7c806ae70 0x7fd7c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.156+0000 7fd7e3fff640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd7c806ae70 0x7fd7c806d330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0036000 tx=0x7fd7d003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:04.165 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.160+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd7d8054960 con 0x7fd7e40c9010 2023-11-08T19:32:04.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.350+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7e406be20 con 0x7fd7e40c9010 2023-11-08T19:32:04.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.354+0000 7fd7e17fa640 1 -- 172.21.15.119:0/1361518393 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23077 (secure 0 0 0) 0x7fd7d8017090 con 0x7fd7e40c9010 2023-11-08T19:32:04.356 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:04.356 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":106,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:31:58.747478+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1419527548":"2023-11-08T20:31:50.849149+0000","172.21.15.188:6834/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.188:6833/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.119:6837/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.119:6835/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.188:6835/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.188:6833/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.119:6837/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6832/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.119:6835/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6835/168553637":"2023-11-09T19:29:41.161236+0000","172.21.15.188:6835/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.188:6833/168553637":"2023-11-09T19:29:41.161236+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":8}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:32:04.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd7c806ae70 msgr2=0x7fd7c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd7c806ae70 0x7fd7c806d330 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0036000 tx=0x7fd7d003c040 comp rx=0 tx=0).stop 2023-11-08T19:32:04.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 msgr2=0x7fd7e41473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 0x7fd7e41473e0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fd7d800e960 tx=0x7fd7d800ee20 comp rx=0 tx=0).stop 2023-11-08T19:32:04.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 shutdown_connections 2023-11-08T19:32:04.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fd7c806ae70 0x7fd7c806d330 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd7e415b910 0x7fd7e415a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd7e4155190 0x7fd7e4159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 --2- 172.21.15.119:0/1361518393 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fd7e40c9010 0x7fd7e41473e0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 >> 172.21.15.119:0/1361518393 conn(0x7fd7e40b1020 msgr2=0x7fd7e4154580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:04.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 shutdown_connections 2023-11-08T19:32:04.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.357+0000 7fd7e9f93640 1 -- 172.21.15.119:0/1361518393 wait complete. 2023-11-08T19:32:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:28:50.669529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:28:51.498158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:32:04.378 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:32:04.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.587+0000 7ff60860e640 1 Processor -- start 2023-11-08T19:32:04.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.587+0000 7ff60860e640 1 -- start start 2023-11-08T19:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.589+0000 7ff60860e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff6001555b0 0x7ff6000ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.589+0000 7ff60860e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6000cc6c0 con 0x7ff6000cb4e0 2023-11-08T19:32:04.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.589+0000 7ff60860e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff600071770 con 0x7ff6000cd110 2023-11-08T19:32:04.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.589+0000 7ff60860e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6000cd590 con 0x7ff6001555b0 2023-11-08T19:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.589+0000 7ff606bab640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7ff6000cb4e0 0x7ff60015b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45952/0 (socket says 172.21.15.119:45952) 2023-11-08T19:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.589+0000 7ff606bab640 1 -- 172.21.15.119:0/2453568821 learned_addr learned my addr 172.21.15.119:0/2453568821 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:04.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6063aa640 1 --2- 172.21.15.119:0/2453568821 >> v2:172.21.15.188:3300/0 conn(0x7ff6001555b0 0x7ff6000ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4001378659 0 0) 0x7ff6000cc6c0 con 0x7ff6000cb4e0 2023-11-08T19:32:04.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5ec003520 con 0x7ff6000cb4e0 2023-11-08T19:32:04.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830712751 0 0) 0x7ff5ec003520 con 0x7ff6000cb4e0 2023-11-08T19:32:04.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 >> v2:172.21.15.188:3300/0 conn(0x7ff6001555b0 msgr2=0x7ff6000ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:04.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 --2- 172.21.15.119:0/2453568821 >> v2:172.21.15.188:3300/0 conn(0x7ff6001555b0 0x7ff6000ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 >> v1:172.21.15.188:6789/0 conn(0x7ff6000cd110 legacy=0x7ff600070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff600071a50 con 0x7ff6000cb4e0 2023-11-08T19:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7ff5f80019f0 con 0x7ff6000cb4e0 2023-11-08T19:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 >> v1:172.21.15.119:6789/0 conn(0x7ff6000cb4e0 legacy=0x7ff60015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0047c0 0x7ff5ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ec005250 0x7ff5ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000cd590 con 0x7ff5ec0039c0 2023-11-08T19:32:04.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec00c1a0 con 0x7ff5ec0047c0 2023-11-08T19:32:04.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.590+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec00c360 con 0x7ff5ec005250 2023-11-08T19:32:04.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff6063aa640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff606bab640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ec005250 0x7ff5ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff6063aa640 1 -- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ec005250 msgr2=0x7ff5ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff6063aa640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff5ec005250 0x7ff5ec00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff6063aa640 1 -- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0047c0 msgr2=0x7ff5ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff6063aa640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0047c0 0x7ff5ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff6063aa640 1 -- 172.21.15.119:0/2453568821 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec003520 con 0x7ff5ec0039c0 2023-11-08T19:32:04.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.591+0000 7ff606bab640 1 -- 172.21.15.119:0/2453568821 reap_dead start 2023-11-08T19:32:04.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.592+0000 7ff6063aa640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003dc0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00ba30 tx=0x7ff5ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8ef491d27b20d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc032660 con 0x7ff5ec0039c0 2023-11-08T19:32:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc0327c0 con 0x7ff5ec0039c0 2023-11-08T19:32:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff6053a8640 1 -- 172.21.15.119:0/2453568821 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc032b50 con 0x7ff5ec0039c0 2023-11-08T19:32:04.608 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 -- 172.21.15.119:0/2453568821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0039c0 msgr2=0x7ff5ec003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003dc0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00ba30 tx=0x7ff5ec0057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 -- 172.21.15.119:0/2453568821 shutdown_connections 2023-11-08T19:32:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 --2- 172.21.15.119:0/2453568821 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003dc0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 -- 172.21.15.119:0/2453568821 >> 172.21.15.119:0/2453568821 conn(0x7ff6000b06e0 msgr2=0x7ff6000b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:04.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 -- 172.21.15.119:0/2453568821 shutdown_connections 2023-11-08T19:32:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.593+0000 7ff60860e640 1 -- 172.21.15.119:0/2453568821 wait complete. 2023-11-08T19:32:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.595+0000 7ff60860e640 1 Processor -- start 2023-11-08T19:32:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.595+0000 7ff60860e640 1 -- start start 2023-11-08T19:32:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff60860e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0039c0 0x7ff60014d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff60860e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 0x7ff60014ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff60860e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000aeb10 0x7ff6000aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff60860e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6000af500 con 0x7ff600154950 2023-11-08T19:32:04.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff60860e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60014ddf0 con 0x7ff5ec0039c0 2023-11-08T19:32:04.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff60860e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6001550a0 con 0x7ff6000aeb10 2023-11-08T19:32:04.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff605ba9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 0x7ff60014ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff605ba9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 0x7ff60014ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46148/0 (socket says 172.21.15.119:46148) 2023-11-08T19:32:04.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.596+0000 7ff605ba9640 1 -- 172.21.15.119:0/17571063 learned_addr learned my addr 172.21.15.119:0/17571063 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:04.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff6063aa640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0039c0 0x7ff60014d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff606bab640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000aeb10 0x7ff6000aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff605ba9640 1 -- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000aeb10 msgr2=0x7ff6000aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff605ba9640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000aeb10 0x7ff6000aefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff605ba9640 1 -- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0039c0 msgr2=0x7ff60014d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff605ba9640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0039c0 0x7ff60014d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff605ba9640 1 -- 172.21.15.119:0/17571063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6000b1950 con 0x7ff600154950 2023-11-08T19:32:04.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff6063aa640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0039c0 0x7ff60014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:04.613 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff605ba9640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 0x7ff60014ccb0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c3f0 tx=0x7ff5f000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0011790 con 0x7ff600154950 2023-11-08T19:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.597+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f00118f0 con 0x7ff600154950 2023-11-08T19:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.598+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f0011ca0 con 0x7ff600154950 2023-11-08T19:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.598+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff600154dd0 con 0x7ff600154950 2023-11-08T19:32:04.614 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.598+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6000c7060 con 0x7ff600154950 2023-11-08T19:32:04.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.599+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6000c7280 con 0x7ff600154950 2023-11-08T19:32:04.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.600+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5f001c460 con 0x7ff600154950 2023-11-08T19:32:04.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.601+0000 7ff5f6ffd640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5e406ae70 0x7ff5e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:04.615 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.601+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7ff5f0089a20 con 0x7ff600154950 2023-11-08T19:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.601+0000 7ff6063aa640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5e406ae70 0x7ff5e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.602+0000 7ff6063aa640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5e406ae70 0x7ff5e406d330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00ba00 tx=0x7ff5fc03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:04.616 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.606+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5f0054220 con 0x7ff600154950 2023-11-08T19:32:04.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.796+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff60004e490 con 0x7ff600154950 2023-11-08T19:32:04.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.797+0000 7ff5f6ffd640 1 -- 172.21.15.119:0/17571063 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5f0057840 con 0x7ff600154950 2023-11-08T19:32:04.799 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:32:04.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.799+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5e406ae70 msgr2=0x7ff5e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.799+0000 7ff60860e640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5e406ae70 0x7ff5e406d330 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00ba00 tx=0x7ff5fc03d040 comp rx=0 tx=0).stop 2023-11-08T19:32:04.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.799+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 msgr2=0x7ff60014ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:04.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.799+0000 7ff60860e640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 0x7ff60014ccb0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c3f0 tx=0x7ff5f000c8b0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 shutdown_connections 2023-11-08T19:32:04.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff5e406ae70 0x7ff5e406d330 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff6000aeb10 0x7ff6000aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff600154950 0x7ff60014ccb0 secure :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7ff5f000c3f0 tx=0x7ff5f000c8b0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 --2- 172.21.15.119:0/17571063 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff5ec0039c0 0x7ff60014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:04.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 >> 172.21.15.119:0/17571063 conn(0x7ff6000b06e0 msgr2=0x7ff60006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:04.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 shutdown_connections 2023-11-08T19:32:04.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:04.800+0000 7ff60860e640 1 -- 172.21.15.119:0/17571063 wait complete. 2023-11-08T19:32:04.820 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:32:05.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.020+0000 7f1fc6e90640 1 Processor -- start 2023-11-08T19:32:05.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.021+0000 7f1fc6e90640 1 -- start start 2023-11-08T19:32:05.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.022+0000 7f1fc6e90640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1fc015d100 0x7f1fc00cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.022+0000 7f1fc6e90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc00cba40 0x7f1fc006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.022+0000 7f1fc6e90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1fc006fef0 0x7f1fc00cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.022+0000 7f1fc6e90640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc0070fb0 con 0x7f1fc015d100 2023-11-08T19:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.022+0000 7f1fc6e90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc00cbea0 con 0x7f1fc006fef0 2023-11-08T19:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.022+0000 7f1fc6e90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc0070710 con 0x7f1fc00cba40 2023-11-08T19:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1fc015d100 0x7f1fc00cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f1fc015d100 0x7f1fc00cb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46164/0 (socket says 172.21.15.119:46164) 2023-11-08T19:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 -- 172.21.15.119:0/150098303 learned_addr learned my addr 172.21.15.119:0/150098303 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fbffff640 1 --2- 172.21.15.119:0/150098303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc00cba40 0x7f1fc006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc542d640 1 --2- 172.21.15.119:0/150098303 >> v2:172.21.15.188:3300/0 conn(0x7f1fc006fef0 0x7f1fc00cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 -- 172.21.15.119:0/150098303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc00cba40 msgr2=0x7f1fc006f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:05.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 --2- 172.21.15.119:0/150098303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc00cba40 0x7f1fc006f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 -- 172.21.15.119:0/150098303 >> v2:172.21.15.188:3300/0 conn(0x7f1fc006fef0 msgr2=0x7f1fc00cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:05.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 --2- 172.21.15.119:0/150098303 >> v2:172.21.15.188:3300/0 conn(0x7f1fc006fef0 0x7f1fc00cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.023+0000 7f1fc4c2c640 1 -- 172.21.15.119:0/150098303 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc00709f0 con 0x7f1fc015d100 2023-11-08T19:32:05.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.024+0000 7f1fc4c2c640 1 --2- 172.21.15.119:0/150098303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc00cb500 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00adcd0 tx=0x7f1fb402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed09173e2bb388 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fbf7fe640 1 -- 172.21.15.119:0/150098303 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb40317e0 con 0x7f1fc015d100 2023-11-08T19:32:05.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fbf7fe640 1 -- 172.21.15.119:0/150098303 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb4031940 con 0x7f1fc015d100 2023-11-08T19:32:05.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fbf7fe640 1 -- 172.21.15.119:0/150098303 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4031cd0 con 0x7f1fc015d100 2023-11-08T19:32:05.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 -- 172.21.15.119:0/150098303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 msgr2=0x7f1fc00cb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/150098303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc00cb500 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f1fc00adcd0 tx=0x7f1fb402b880 comp rx=0 tx=0).stop 2023-11-08T19:32:05.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 -- 172.21.15.119:0/150098303 shutdown_connections 2023-11-08T19:32:05.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/150098303 >> v2:172.21.15.188:3300/0 conn(0x7f1fc006fef0 0x7f1fc00cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/150098303 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc00cba40 0x7f1fc006f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/150098303 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc00cb500 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 -- 172.21.15.119:0/150098303 >> 172.21.15.119:0/150098303 conn(0x7f1fc00b06e0 msgr2=0x7f1fc00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:05.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 -- 172.21.15.119:0/150098303 shutdown_connections 2023-11-08T19:32:05.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.025+0000 7f1fc6e90640 1 -- 172.21.15.119:0/150098303 wait complete. 2023-11-08T19:32:05.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.027+0000 7f1fc6e90640 1 Processor -- start 2023-11-08T19:32:05.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.027+0000 7f1fc6e90640 1 -- start start 2023-11-08T19:32:05.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.028+0000 7f1fc6e90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc006fef0 0x7f1fc00c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.028+0000 7f1fc6e90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc00cba40 0x7f1fc01e3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.028+0000 7f1fc6e90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.028+0000 7f1fc6e90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01e3da0 con 0x7f1fc015d100 2023-11-08T19:32:05.035 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.028+0000 7f1fc6e90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01e3fd0 con 0x7f1fc00cba40 2023-11-08T19:32:05.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.028+0000 7f1fc6e90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc01e42b0 con 0x7f1fc006fef0 2023-11-08T19:32:05.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc014e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46170/0 (socket says 172.21.15.119:46170) 2023-11-08T19:32:05.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 -- 172.21.15.119:0/3911031832 learned_addr learned my addr 172.21.15.119:0/3911031832 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:05.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc4c2c640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc006fef0 0x7f1fc00c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fbffff640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc00cba40 0x7f1fc01e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 -- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc006fef0 msgr2=0x7f1fc00c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc006fef0 0x7f1fc00c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 -- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc00cba40 msgr2=0x7f1fc01e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc00cba40 0x7f1fc01e3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 -- 172.21.15.119:0/3911031832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40094d0 con 0x7f1fc015d100 2023-11-08T19:32:05.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.029+0000 7f1fc542d640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc014e080 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8007d20 tx=0x7f1fb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.030+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014420 con 0x7f1fc015d100 2023-11-08T19:32:05.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.030+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb80145a0 con 0x7f1fc015d100 2023-11-08T19:32:05.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.030+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc01e4590 con 0x7f1fc015d100 2023-11-08T19:32:05.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.030+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb8014950 con 0x7f1fc015d100 2023-11-08T19:32:05.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.030+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc01e4b50 con 0x7f1fc015d100 2023-11-08T19:32:05.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.031+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc004e490 con 0x7f1fc015d100 2023-11-08T19:32:05.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.032+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1fb8014ab0 con 0x7f1fc015d100 2023-11-08T19:32:05.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.033+0000 7f1fbd7fa640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1fa406aea0 0x7f1fa406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.040 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.033+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f1fb808a720 con 0x7f1fc015d100 2023-11-08T19:32:05.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.033+0000 7f1fc4c2c640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1fa406aea0 0x7f1fa406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.034+0000 7f1fc4c2c640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1fa406aea0 0x7f1fa406d360 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4036000 tx=0x7f1fb403d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.041 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.037+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1fb8054f20 con 0x7f1fc015d100 2023-11-08T19:32:05.227 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.225+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7f1fc00b0d80 con 0x7f1fc015d100 2023-11-08T19:32:05.228 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.225+0000 7f1fbd7fa640 1 -- 172.21.15.119:0/3911031832 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7f1fb8058540 con 0x7f1fc015d100 2023-11-08T19:32:05.228 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:05.230 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1fa406aea0 msgr2=0x7f1fa406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1fa406aea0 0x7f1fa406d360 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4036000 tx=0x7f1fb403d040 comp rx=0 tx=0).stop 2023-11-08T19:32:05.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 msgr2=0x7f1fc014e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc014e080 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8007d20 tx=0x7f1fb800e4b0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 shutdown_connections 2023-11-08T19:32:05.231 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f1fa406aea0 0x7f1fa406d360 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1fc015d100 0x7f1fc014e080 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fc00cba40 0x7f1fc01e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 --2- 172.21.15.119:0/3911031832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fc006fef0 0x7f1fc00c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 >> 172.21.15.119:0/3911031832 conn(0x7f1fc00b06e0 msgr2=0x7f1fc006bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:05.232 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 shutdown_connections 2023-11-08T19:32:05.233 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.228+0000 7f1fc6e90640 1 -- 172.21.15.119:0/3911031832 wait complete. 2023-11-08T19:32:05.248 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:32:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.434+0000 7fbda18da640 1 Processor -- start 2023-11-08T19:32:05.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.435+0000 7fbda18da640 1 -- start start 2023-11-08T19:32:05.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.436+0000 7fbda18da640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd9c06ab60 0x7fbd9c164b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.436+0000 7fbda18da640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c0ae1b0 con 0x7fbd9c159f00 2023-11-08T19:32:05.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.436+0000 7fbda18da640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd9c15a460 con 0x7fbd9c159b20 2023-11-08T19:32:05.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.436+0000 7fbda18da640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd9c15a740 con 0x7fbd9c06ab60 2023-11-08T19:32:05.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.436+0000 7fbd9bfff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbd9c159f00 0x7fbd9c06a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45964/0 (socket says 172.21.15.119:45964) 2023-11-08T19:32:05.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9bfff640 1 -- 172.21.15.119:0/743978339 learned_addr learned my addr 172.21.15.119:0/743978339 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbda10d9640 1 --2- 172.21.15.119:0/743978339 >> v2:172.21.15.188:3300/0 conn(0x7fbd9c06ab60 0x7fbd9c164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3345088899 0 0) 0x7fbd9c0ae1b0 con 0x7fbd9c159f00 2023-11-08T19:32:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd8c003520 con 0x7fbd9c159f00 2023-11-08T19:32:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2127799179 0 0) 0x7fbd8c003520 con 0x7fbd9c159f00 2023-11-08T19:32:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 >> v2:172.21.15.188:3300/0 conn(0x7fbd9c06ab60 msgr2=0x7fbd9c164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 --2- 172.21.15.119:0/743978339 >> v2:172.21.15.188:3300/0 conn(0x7fbd9c06ab60 0x7fbd9c164b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 >> v1:172.21.15.188:6789/0 conn(0x7fbd9c159b20 legacy=0x7fbd9c0ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c15aa20 con 0x7fbd9c159f00 2023-11-08T19:32:05.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fbd88001670 con 0x7fbd9c159f00 2023-11-08T19:32:05.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 >> v1:172.21.15.119:6789/0 conn(0x7fbd9c159f00 legacy=0x7fbd9c06a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c003970 0x7fbd8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c005180 0x7fbd8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c15a740 con 0x7fbd8c0047c0 2023-11-08T19:32:05.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c00c120 con 0x7fbd8c003970 2023-11-08T19:32:05.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.437+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8c00c2e0 con 0x7fbd8c005180 2023-11-08T19:32:05.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbda08d8640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c003970 0x7fbd8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbd9bfff640 1 -- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c005180 msgr2=0x7fbd8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:05.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd8c005180 0x7fbd8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbd9bfff640 1 -- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c003970 msgr2=0x7fbd8c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:05.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd8c003970 0x7fbd8c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbd9bfff640 1 -- 172.21.15.119:0/743978339 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8c003520 con 0x7fbd8c0047c0 2023-11-08T19:32:05.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.438+0000 7fbda10d9640 1 -- 172.21.15.119:0/743978339 reap_dead start 2023-11-08T19:32:05.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.439+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd8c004c40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c210 tx=0x7fbd8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198444cc0f70a2cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.439+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88005ad0 con 0x7fbd8c0047c0 2023-11-08T19:32:05.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.439+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd88005c30 con 0x7fbd8c0047c0 2023-11-08T19:32:05.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.439+0000 7fbd9b7fe640 1 -- 172.21.15.119:0/743978339 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8802ca90 con 0x7fbd8c0047c0 2023-11-08T19:32:05.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 -- 172.21.15.119:0/743978339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 msgr2=0x7fbd8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd8c004c40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fbd8800c210 tx=0x7fbd8c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 -- 172.21.15.119:0/743978339 shutdown_connections 2023-11-08T19:32:05.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 --2- 172.21.15.119:0/743978339 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd8c004c40 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 -- 172.21.15.119:0/743978339 >> 172.21.15.119:0/743978339 conn(0x7fbd9c0b0a20 msgr2=0x7fbd9c0b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:05.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 -- 172.21.15.119:0/743978339 shutdown_connections 2023-11-08T19:32:05.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.440+0000 7fbda18da640 1 -- 172.21.15.119:0/743978339 wait complete. 2023-11-08T19:32:05.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.442+0000 7fbda18da640 1 Processor -- start 2023-11-08T19:32:05.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.442+0000 7fbda18da640 1 -- start start 2023-11-08T19:32:05.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda18da640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd9c14bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda18da640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd9c168ba0 0x7fbd9c167060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda18da640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd9c0acb10 0x7fbd9c0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda18da640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c168480 con 0x7fbd8c0047c0 2023-11-08T19:32:05.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda18da640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c14cfc0 con 0x7fbd9c168ba0 2023-11-08T19:32:05.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda18da640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd9c14d210 con 0x7fbd9c0acb10 2023-11-08T19:32:05.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda08d8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd9c14bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda08d8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd9c14bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46194/0 (socket says 172.21.15.119:46194) 2023-11-08T19:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.443+0000 7fbda08d8640 1 -- 172.21.15.119:0/1080366183 learned_addr learned my addr 172.21.15.119:0/1080366183 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda10d9640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd9c0acb10 0x7fbd9c0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda08d8640 1 -- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd9c0acb10 msgr2=0x7fbd9c0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd9c168ba0 0x7fbd9c167060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda08d8640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd9c0acb10 0x7fbd9c0cbf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda08d8640 1 -- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd9c168ba0 msgr2=0x7fbd9c167060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda08d8640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd9c168ba0 0x7fbd9c167060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda08d8640 1 -- 172.21.15.119:0/1080366183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd9c14ebe0 con 0x7fbd8c0047c0 2023-11-08T19:32:05.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda08d8640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd9c14bc10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d430 tx=0x7fbd9000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9000b070 con 0x7fbd8c0047c0 2023-11-08T19:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd900054e0 con 0x7fbd8c0047c0 2023-11-08T19:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd90003810 con 0x7fbd8c0047c0 2023-11-08T19:32:05.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c14eec0 con 0x7fbd8c0047c0 2023-11-08T19:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.444+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9c14d4f0 con 0x7fbd8c0047c0 2023-11-08T19:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.445+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9c14ea40 con 0x7fbd8c0047c0 2023-11-08T19:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.446+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd90003970 con 0x7fbd8c0047c0 2023-11-08T19:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.447+0000 7fbd997fa640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd8006aea0 0x7fbd8006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.447+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7fbd90089700 con 0x7fbd8c0047c0 2023-11-08T19:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.447+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd8006aea0 0x7fbd8006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.448+0000 7fbd9bfff640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd8006aea0 0x7fbd8006d360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbd88034000 tx=0x7fbd88009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.451+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd9c14ea40 con 0x7fbd8c0047c0 2023-11-08T19:32:05.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.641+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fbd9c14ea40 con 0x7fbd8c0047c0 2023-11-08T19:32:05.644 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.642+0000 7fbd997fa640 1 -- 172.21.15.119:0/1080366183 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbd9c14ea40 con 0x7fbd8c0047c0 2023-11-08T19:32:05.645 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:05.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd8006aea0 msgr2=0x7fbd8006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd8006aea0 0x7fbd8006d360 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbd88034000 tx=0x7fbd88009030 comp rx=0 tx=0).stop 2023-11-08T19:32:05.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 msgr2=0x7fbd9c14bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd9c14bc10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fbd9000d430 tx=0x7fbd9000d8f0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 shutdown_connections 2023-11-08T19:32:05.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbd8006aea0 0x7fbd8006d360 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd9c0acb10 0x7fbd9c0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd9c168ba0 0x7fbd9c167060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 --2- 172.21.15.119:0/1080366183 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbd8c0047c0 0x7fbd9c14bc10 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.644+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 >> 172.21.15.119:0/1080366183 conn(0x7fbd9c0b0a20 msgr2=0x7fbd9c0b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.645+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 shutdown_connections 2023-11-08T19:32:05.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.645+0000 7fbda18da640 1 -- 172.21.15.119:0/1080366183 wait complete. 2023-11-08T19:32:05.666 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:32:05.867 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.865+0000 7f6660cd4640 1 Processor -- start 2023-11-08T19:32:05.868 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.865+0000 7f6660cd4640 1 -- start start 2023-11-08T19:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f6660cd4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f665c1555b0 0x7f665c0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f6660cd4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 0x7f665c0cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.869 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f6660cd4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665c0cc6c0 con 0x7f665c0cd110 2023-11-08T19:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f6660cd4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f665c0cd590 con 0x7f665c1555b0 2023-11-08T19:32:05.870 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f6660cd4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0cbeb0 con 0x7f665c0cb4e0 2023-11-08T19:32:05.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f665ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 0x7f665c0cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.871 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.867+0000 7f665a59c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f665c1555b0 0x7f665c0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.868+0000 7f665ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 0x7f665c0cb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.119:51956/0 (socket says 172.21.15.119:51956) 2023-11-08T19:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.868+0000 7f665ad9d640 1 -- 172.21.15.119:0/1342602055 learned_addr learned my addr 172.21.15.119:0/1342602055 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.868+0000 7f665ad9d640 1 -- 172.21.15.119:0/1342602055 >> v2:172.21.15.188:3300/0 conn(0x7f665c1555b0 msgr2=0x7f665c0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:05.872 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.868+0000 7f665ad9d640 1 --2- 172.21.15.119:0/1342602055 >> v2:172.21.15.188:3300/0 conn(0x7f665c1555b0 0x7f665c0ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.868+0000 7f665ad9d640 1 -- 172.21.15.119:0/1342602055 >> v1:172.21.15.188:6789/0 conn(0x7f665c0cd110 legacy=0x7f665c070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.873 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.868+0000 7f665ad9d640 1 -- 172.21.15.119:0/1342602055 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665c071770 con 0x7f665c0cb4e0 2023-11-08T19:32:05.874 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.870+0000 7f665ad9d640 1 --2- 172.21.15.119:0/1342602055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 0x7f665c0cb970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6654009e50 tx=0x7f665402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dc39c023362cd98 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.870+0000 7f665959a640 1 -- 172.21.15.119:0/1342602055 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665402ca70 con 0x7f665c0cb4e0 2023-11-08T19:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.870+0000 7f665959a640 1 -- 172.21.15.119:0/1342602055 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665402cbd0 con 0x7f665c0cb4e0 2023-11-08T19:32:05.875 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.870+0000 7f665959a640 1 -- 172.21.15.119:0/1342602055 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66540343f0 con 0x7f665c0cb4e0 2023-11-08T19:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 -- 172.21.15.119:0/1342602055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 msgr2=0x7f665c0cb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 --2- 172.21.15.119:0/1342602055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 0x7f665c0cb970 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6654009e50 tx=0x7f665402bed0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 -- 172.21.15.119:0/1342602055 shutdown_connections 2023-11-08T19:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 --2- 172.21.15.119:0/1342602055 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cb4e0 0x7f665c0cb970 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.876 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 --2- 172.21.15.119:0/1342602055 >> v2:172.21.15.188:3300/0 conn(0x7f665c1555b0 0x7f665c0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 -- 172.21.15.119:0/1342602055 >> 172.21.15.119:0/1342602055 conn(0x7f665c0b06e0 msgr2=0x7f665c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:05.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 -- 172.21.15.119:0/1342602055 shutdown_connections 2023-11-08T19:32:05.877 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.871+0000 7f6660cd4640 1 -- 172.21.15.119:0/1342602055 wait complete. 2023-11-08T19:32:05.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.873+0000 7f6660cd4640 1 Processor -- start 2023-11-08T19:32:05.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.873+0000 7f6660cd4640 1 -- start start 2023-11-08T19:32:05.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f6660cd4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 0x7f665c14d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f6660cd4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cd110 0x7f665c1f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f6660cd4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c1555b0 0x7f665c0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f6660cd4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0c6600 con 0x7f665c0cb4e0 2023-11-08T19:32:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f6660cd4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0b2860 con 0x7f665c1555b0 2023-11-08T19:32:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f6660cd4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665c0c5c60 con 0x7f665c0cd110 2023-11-08T19:32:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 0x7f665c14d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 0x7f665c14d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46200/0 (socket says 172.21.15.119:46200) 2023-11-08T19:32:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 -- 172.21.15.119:0/2908919227 learned_addr learned my addr 172.21.15.119:0/2908919227 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 -- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cd110 msgr2=0x7f665c1f35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cd110 0x7f665c1f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 -- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c1555b0 msgr2=0x7f665c0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:05.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665ad9d640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c1555b0 0x7f665c0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.874+0000 7f665a59c640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c1555b0 0x7f665c0c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f665a59c640 1 -- 172.21.15.119:0/2908919227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009b30 con 0x7f665c0cb4e0 2023-11-08T19:32:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f665a59c640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 0x7f665c14d310 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f664c00e9b0 tx=0x7f664c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c009670 con 0x7f665c0cb4e0 2023-11-08T19:32:05.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c00c9a0 con 0x7f665c0cb4e0 2023-11-08T19:32:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c00cd50 con 0x7f665c0cb4e0 2023-11-08T19:32:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665c0c5f40 con 0x7f665c0cb4e0 2023-11-08T19:32:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.875+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665c14d850 con 0x7f665c0cb4e0 2023-11-08T19:32:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.876+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665c04e490 con 0x7f665c0cb4e0 2023-11-08T19:32:05.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.877+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f664c013470 con 0x7f665c0cb4e0 2023-11-08T19:32:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.878+0000 7f664affd640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f663c06ae70 0x7f663c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.878+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f664c089fe0 con 0x7f665c0cb4e0 2023-11-08T19:32:05.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.878+0000 7f6659d9b640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f663c06ae70 0x7f663c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:05.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.879+0000 7f6659d9b640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f663c06ae70 0x7f663c06d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6650000f60 tx=0x7f66500079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:05.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:05.882+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f664c0548d0 con 0x7f665c0cb4e0 2023-11-08T19:32:06.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.057+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f665c0cf8c0 con 0x7f665c0cb4e0 2023-11-08T19:32:06.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.058+0000 7f664affd640 1 -- 172.21.15.119:0/2908919227 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v106) v1 ==== 107+0+11 (secure 0 0 0) 0x7f664c0170b0 con 0x7f665c0cb4e0 2023-11-08T19:32:06.060 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f663c06ae70 msgr2=0x7f663c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f663c06ae70 0x7f663c06d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6650000f60 tx=0x7f66500079b0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 msgr2=0x7f665c14d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 0x7f665c14d310 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f664c00e9b0 tx=0x7f664c00ee70 comp rx=0 tx=0).stop 2023-11-08T19:32:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 shutdown_connections 2023-11-08T19:32:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f663c06ae70 0x7f663c06d330 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f665c1555b0 0x7f665c0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f665c0cd110 0x7f665c1f35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 --2- 172.21.15.119:0/2908919227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f665c0cb4e0 0x7f665c14d310 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.060+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 >> 172.21.15.119:0/2908919227 conn(0x7f665c0b06e0 msgr2=0x7f665c153bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:06.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.061+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 shutdown_connections 2023-11-08T19:32:06.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.061+0000 7f6660cd4640 1 -- 172.21.15.119:0/2908919227 wait complete. 2023-11-08T19:32:06.079 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:32:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.291+0000 7f27a8075640 1 Processor -- start 2023-11-08T19:32:06.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.291+0000 7f27a8075640 1 -- start start 2023-11-08T19:32:06.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a8075640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f27a00ce040 0x7f27a00cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a8075640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f27a00cb4e0 0x7f27a00cb910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a8075640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a00cc6c0 con 0x7f27a01555b0 2023-11-08T19:32:06.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a8075640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a00cbe50 con 0x7f27a00ce040 2023-11-08T19:32:06.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a8075640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a0071770 con 0x7f27a00cb4e0 2023-11-08T19:32:06.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a5e11640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f27a01555b0 0x7f27a0070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45968/0 (socket says 172.21.15.119:45968) 2023-11-08T19:32:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a5e11640 1 -- 172.21.15.119:0/397085150 learned_addr learned my addr 172.21.15.119:0/397085150 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:06.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.293+0000 7f27a5610640 1 --2- 172.21.15.119:0/397085150 >> v2:172.21.15.119:3300/0 conn(0x7f27a00ce040 0x7f27a00cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a6612640 1 --2- 172.21.15.119:0/397085150 >> v2:172.21.15.188:3300/0 conn(0x7f27a00cb4e0 0x7f27a00cb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a4e0f640 1 -- 172.21.15.119:0/397085150 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151096719 0 0) 0x7f27a00cc6c0 con 0x7f27a01555b0 2023-11-08T19:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a4e0f640 1 -- 172.21.15.119:0/397085150 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f278c003520 con 0x7f27a01555b0 2023-11-08T19:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a5610640 1 -- 172.21.15.119:0/397085150 >> v2:172.21.15.188:3300/0 conn(0x7f27a00cb4e0 msgr2=0x7f27a00cb910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a5610640 1 --2- 172.21.15.119:0/397085150 >> v2:172.21.15.188:3300/0 conn(0x7f27a00cb4e0 0x7f27a00cb910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a5610640 1 -- 172.21.15.119:0/397085150 >> v1:172.21.15.119:6789/0 conn(0x7f27a01555b0 legacy=0x7f27a0070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.294+0000 7f27a5610640 1 -- 172.21.15.119:0/397085150 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a0071a50 con 0x7f27a00ce040 2023-11-08T19:32:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.295+0000 7f27a5610640 1 --2- 172.21.15.119:0/397085150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a00cd110 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f27a00ae600 tx=0x7f279002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9684dd3d7fc7498 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.295+0000 7f27a4e0f640 1 -- 172.21.15.119:0/397085150 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279002e070 con 0x7f27a00ce040 2023-11-08T19:32:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.295+0000 7f27a4e0f640 1 -- 172.21.15.119:0/397085150 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27900315a0 con 0x7f27a00ce040 2023-11-08T19:32:06.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.295+0000 7f27a4e0f640 1 -- 172.21.15.119:0/397085150 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790031930 con 0x7f27a00ce040 2023-11-08T19:32:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 -- 172.21.15.119:0/397085150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 msgr2=0x7f27a00cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 --2- 172.21.15.119:0/397085150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a00cd110 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f27a00ae600 tx=0x7f279002b840 comp rx=0 tx=0).stop 2023-11-08T19:32:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 -- 172.21.15.119:0/397085150 shutdown_connections 2023-11-08T19:32:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 --2- 172.21.15.119:0/397085150 >> v2:172.21.15.188:3300/0 conn(0x7f27a00cb4e0 0x7f27a00cb910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 --2- 172.21.15.119:0/397085150 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a00cd110 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 -- 172.21.15.119:0/397085150 >> 172.21.15.119:0/397085150 conn(0x7f27a00b06e0 msgr2=0x7f27a00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:06.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 -- 172.21.15.119:0/397085150 shutdown_connections 2023-11-08T19:32:06.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.296+0000 7f27a8075640 1 -- 172.21.15.119:0/397085150 wait complete. 2023-11-08T19:32:06.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.298+0000 7f27a8075640 1 Processor -- start 2023-11-08T19:32:06.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.298+0000 7f27a8075640 1 -- start start 2023-11-08T19:32:06.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a8075640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27a00cb4e0 0x7f27a00c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a8075640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a014d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a8075640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27a01555b0 0x7f27a006fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a8075640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00c7000 con 0x7f27a00ce040 2023-11-08T19:32:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a8075640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00c6100 con 0x7f27a01555b0 2023-11-08T19:32:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a8075640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00c6380 con 0x7f27a00cb4e0 2023-11-08T19:32:06.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a5610640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a014d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a5610640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a014d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46226/0 (socket says 172.21.15.119:46226) 2023-11-08T19:32:06.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.299+0000 7f27a5610640 1 -- 172.21.15.119:0/1684905192 learned_addr learned my addr 172.21.15.119:0/1684905192 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:06.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5610640 1 -- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27a00cb4e0 msgr2=0x7f27a00c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5e11640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27a00cb4e0 0x7f27a00c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a6612640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27a01555b0 0x7f27a006fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5610640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27a00cb4e0 0x7f27a00c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5610640 1 -- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27a01555b0 msgr2=0x7f27a006fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5610640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27a01555b0 0x7f27a006fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5610640 1 -- 172.21.15.119:0/1684905192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009400 con 0x7f27a00ce040 2023-11-08T19:32:06.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a5610640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a014d390 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f279002be50 tx=0x7f27900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:06.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790032840 con 0x7f27a00ce040 2023-11-08T19:32:06.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790031f00 con 0x7f27a00ce040 2023-11-08T19:32:06.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790032be0 con 0x7f27a00ce040 2023-11-08T19:32:06.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a00c6660 con 0x7f27a00ce040 2023-11-08T19:32:06.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.300+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a00c6c20 con 0x7f27a00ce040 2023-11-08T19:32:06.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.302+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a004e490 con 0x7f27a00ce040 2023-11-08T19:32:06.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.302+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2790032d40 con 0x7f27a00ce040 2023-11-08T19:32:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.303+0000 7f27967fc640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f278406ae70 0x7f278406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.303+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f27900abda0 con 0x7f27a00ce040 2023-11-08T19:32:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.303+0000 7f27a5e11640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f278406ae70 0x7f278406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.304+0000 7f27a5e11640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f278406ae70 0x7f278406d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f279c0011e0 tx=0x7f279c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:06.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.307+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2790076800 con 0x7f27a00ce040 2023-11-08T19:32:06.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.496+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27a0151a90 con 0x7f27a00ce040 2023-11-08T19:32:06.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.497+0000 7f27967fc640 1 -- 172.21.15.119:0/1684905192 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2790079e20 con 0x7f27a00ce040 2023-11-08T19:32:06.500 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.499+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f278406ae70 msgr2=0x7f278406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.499+0000 7f27a8075640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f278406ae70 0x7f278406d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f279c0011e0 tx=0x7f279c009040 comp rx=0 tx=0).stop 2023-11-08T19:32:06.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 msgr2=0x7f27a014d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a014d390 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f279002be50 tx=0x7f27900046c0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 shutdown_connections 2023-11-08T19:32:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f278406ae70 0x7f278406d330 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f27a01555b0 0x7f27a006fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f27a00ce040 0x7f27a014d390 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 --2- 172.21.15.119:0/1684905192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27a00cb4e0 0x7f27a00c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 >> 172.21.15.119:0/1684905192 conn(0x7f27a00b06e0 msgr2=0x7f27a0070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 shutdown_connections 2023-11-08T19:32:06.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.500+0000 7f27a8075640 1 -- 172.21.15.119:0/1684905192 wait complete. 2023-11-08T19:32:06.519 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.736+0000 7f025c17c640 1 Processor -- start 2023-11-08T19:32:06.739 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.736+0000 7f025c17c640 1 -- start start 2023-11-08T19:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f025c17c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f02540703a0 0x7f025415e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f025c17c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 0x7f02540cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f025c17c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025415dd60 con 0x7f0254159d80 2023-11-08T19:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f025c17c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02540cbcb0 con 0x7f02540703a0 2023-11-08T19:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f025c17c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025415da30 con 0x7f02540cb2e0 2023-11-08T19:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f0259f18640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f0254159d80 0x7f02540ca080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45984/0 (socket says 172.21.15.119:45984) 2023-11-08T19:32:06.741 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f0259f18640 1 -- 172.21.15.119:0/104290583 learned_addr learned my addr 172.21.15.119:0/104290583 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:06.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f0259717640 1 --2- 172.21.15.119:0/104290583 >> v2:172.21.15.188:3300/0 conn(0x7f02540703a0 0x7f025415e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.738+0000 7f025a719640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 0x7f02540cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670931318 0 0) 0x7f025415dd60 con 0x7f0254159d80 2023-11-08T19:32:06.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0240003520 con 0x7f0254159d80 2023-11-08T19:32:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1444948038 0 0) 0x7f0240003520 con 0x7f0254159d80 2023-11-08T19:32:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 msgr2=0x7f02540cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 0x7f02540cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 >> v2:172.21.15.188:3300/0 conn(0x7f02540703a0 msgr2=0x7f025415e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:06.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 --2- 172.21.15.119:0/104290583 >> v2:172.21.15.188:3300/0 conn(0x7f02540703a0 0x7f025415e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540ca8b0 con 0x7f0254159d80 2023-11-08T19:32:06.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f0250001670 con 0x7f0254159d80 2023-11-08T19:32:06.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 >> v1:172.21.15.119:6789/0 conn(0x7f0254159d80 legacy=0x7f02540ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0240003970 0x7f0240003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02400047c0 0x7f0240004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 0x7f0240005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540cbcb0 con 0x7f0240003970 2023-11-08T19:32:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025415da30 con 0x7f02400047c0 2023-11-08T19:32:06.748 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024000b980 con 0x7f02540cb2e0 2023-11-08T19:32:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.739+0000 7f0259f18640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0240003970 0x7f0240003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259717640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02400047c0 0x7f0240004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259f18640 1 -- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 msgr2=0x7f0240005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259f18640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02540cb2e0 0x7f0240005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.749 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259f18640 1 -- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02400047c0 msgr2=0x7f0240004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259f18640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02400047c0 0x7f0240004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259f18640 1 -- 172.21.15.119:0/104290583 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0240003520 con 0x7f0240003970 2023-11-08T19:32:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.740+0000 7f0259717640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02400047c0 0x7f0240004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:06.750 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.741+0000 7f0259f18640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0240003970 0x7f0240003d90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0250007c00 tx=0x7f02400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1300c7b6e46cf6c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:06.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.741+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f025002ec60 con 0x7f0240003970 2023-11-08T19:32:06.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.741+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f025002ede0 con 0x7f0240003970 2023-11-08T19:32:06.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.741+0000 7f0258f16640 1 -- 172.21.15.119:0/104290583 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0250005b10 con 0x7f0240003970 2023-11-08T19:32:06.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025c17c640 1 -- 172.21.15.119:0/104290583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0240003970 msgr2=0x7f0240003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025c17c640 1 --2- 172.21.15.119:0/104290583 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0240003970 0x7f0240003d90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f0250007c00 tx=0x7f02400057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025a719640 1 -- 172.21.15.119:0/104290583 reap_dead start 2023-11-08T19:32:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025c17c640 1 -- 172.21.15.119:0/104290583 shutdown_connections 2023-11-08T19:32:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025c17c640 1 -- 172.21.15.119:0/104290583 >> 172.21.15.119:0/104290583 conn(0x7f02540b06e0 msgr2=0x7f02540b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025c17c640 1 -- 172.21.15.119:0/104290583 shutdown_connections 2023-11-08T19:32:06.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.742+0000 7f025c17c640 1 -- 172.21.15.119:0/104290583 wait complete. 2023-11-08T19:32:06.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.744+0000 7f025c17c640 1 Processor -- start 2023-11-08T19:32:06.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.744+0000 7f025c17c640 1 -- start start 2023-11-08T19:32:06.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025c17c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0240003970 0x7f02540c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025c17c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02541585d0 0x7f02540c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025c17c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 0x7f02540cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025c17c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c59a0 con 0x7f02540c65d0 2023-11-08T19:32:06.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025c17c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c8520 con 0x7f0240003970 2023-11-08T19:32:06.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025c17c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02540c6f40 con 0x7f02541585d0 2023-11-08T19:32:06.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 0x7f02540cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 0x7f02540cd2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46236/0 (socket says 172.21.15.119:46236) 2023-11-08T19:32:06.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 -- 172.21.15.119:0/3799274288 learned_addr learned my addr 172.21.15.119:0/3799274288 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:06.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f0259f18640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0240003970 0x7f02540c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 -- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02541585d0 msgr2=0x7f02540c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02541585d0 0x7f02540c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 -- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0240003970 msgr2=0x7f02540c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0240003970 0x7f02540c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.745+0000 7f025a719640 1 -- 172.21.15.119:0/3799274288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540c7220 con 0x7f02540c65d0 2023-11-08T19:32:06.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.746+0000 7f025a719640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 0x7f02540cd2b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f02540ae620 tx=0x7f024c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:06.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.746+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c010040 con 0x7f02540c65d0 2023-11-08T19:32:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.746+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c00bce0 con 0x7f02540c65d0 2023-11-08T19:32:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.746+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02540c7500 con 0x7f02540c65d0 2023-11-08T19:32:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.746+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c014640 con 0x7f02540c65d0 2023-11-08T19:32:06.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.746+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02540c7ac0 con 0x7f02540c65d0 2023-11-08T19:32:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.747+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025404e490 con 0x7f02540c65d0 2023-11-08T19:32:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.748+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f024c0147a0 con 0x7f02540c65d0 2023-11-08T19:32:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.749+0000 7f024a7fc640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f023806aea0 0x7f023806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.749+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f024c08a0f0 con 0x7f02540c65d0 2023-11-08T19:32:06.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.749+0000 7f0259f18640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f023806aea0 0x7f023806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:06.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.750+0000 7f0259f18640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f023806aea0 0x7f023806d360 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0250036000 tx=0x7f0250007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:06.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.753+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f024c054990 con 0x7f02540c65d0 2023-11-08T19:32:06.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.947+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02540c9950 con 0x7f02540c65d0 2023-11-08T19:32:06.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.948+0000 7f024a7fc640 1 -- 172.21.15.119:0/3799274288 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f024c057fb0 con 0x7f02540c65d0 2023-11-08T19:32:06.951 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:32:06.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f023806aea0 msgr2=0x7f023806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f023806aea0 0x7f023806d360 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f0250036000 tx=0x7f0250007000 comp rx=0 tx=0).stop 2023-11-08T19:32:06.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 msgr2=0x7f02540cd2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:06.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 0x7f02540cd2b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f02540ae620 tx=0x7f024c00b480 comp rx=0 tx=0).stop 2023-11-08T19:32:06.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 shutdown_connections 2023-11-08T19:32:06.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f023806aea0 0x7f023806d360 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f02540c65d0 0x7f02540cd2b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f02541585d0 0x7f02540c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 --2- 172.21.15.119:0/3799274288 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0240003970 0x7f02540c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:06.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 >> 172.21.15.119:0/3799274288 conn(0x7f02540b06e0 msgr2=0x7f02540ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:06.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.951+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 shutdown_connections 2023-11-08T19:32:06.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:06.952+0000 7f025c17c640 1 -- 172.21.15.119:0/3799274288 wait complete. 2023-11-08T19:32:06.973 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T19:32:07.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.185+0000 7f3eb2935640 1 Processor -- start 2023-11-08T19:32:07.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.185+0000 7f3eb2935640 1 -- start start 2023-11-08T19:32:07.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb2935640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3eac06c010 0x7f3eac071790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:07.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb2935640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3eac06ffd0 0x7f3eac0ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:07.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb2935640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eac0ca680 con 0x7f3eac1434d0 2023-11-08T19:32:07.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb2935640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac071300 con 0x7f3eac06c010 2023-11-08T19:32:07.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb2935640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eac070440 con 0x7f3eac06ffd0 2023-11-08T19:32:07.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb1132640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f3eac06c010 0x7f3eac071790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:07.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb1933640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f3eac1434d0 0x7f3eac06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45992/0 (socket says 172.21.15.119:45992) 2023-11-08T19:32:07.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.187+0000 7f3eb1933640 1 -- 172.21.15.119:0/3614097325 learned_addr learned my addr 172.21.15.119:0/3614097325 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:07.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.188+0000 7f3eb2134640 1 --2- 172.21.15.119:0/3614097325 >> v2:172.21.15.188:3300/0 conn(0x7f3eac06ffd0 0x7f3eac0ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:07.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.188+0000 7f3eb1132640 1 -- 172.21.15.119:0/3614097325 >> v2:172.21.15.188:3300/0 conn(0x7f3eac06ffd0 msgr2=0x7f3eac0ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:07.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.188+0000 7f3eb1132640 1 --2- 172.21.15.119:0/3614097325 >> v2:172.21.15.188:3300/0 conn(0x7f3eac06ffd0 0x7f3eac0ca140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.188+0000 7f3eb1132640 1 -- 172.21.15.119:0/3614097325 >> v1:172.21.15.119:6789/0 conn(0x7f3eac1434d0 legacy=0x7f3eac06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:07.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.188+0000 7f3eb1132640 1 -- 172.21.15.119:0/3614097325 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0706f0 con 0x7f3eac06c010 2023-11-08T19:32:07.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.188+0000 7f3eb0931640 1 -- 172.21.15.119:0/3614097325 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233114390 0 0) 0x7f3ea8001110 con 0x7f3eac1434d0 2023-11-08T19:32:07.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.189+0000 7f3eb1132640 1 --2- 172.21.15.119:0/3614097325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac071790 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3eac06f5f0 tx=0x7f3e9c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770ecd0efd3b0b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:07.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.189+0000 7f3eb0931640 1 -- 172.21.15.119:0/3614097325 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c031830 con 0x7f3eac06c010 2023-11-08T19:32:07.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.189+0000 7f3eb0931640 1 -- 172.21.15.119:0/3614097325 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c031990 con 0x7f3eac06c010 2023-11-08T19:32:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.189+0000 7f3eb0931640 1 -- 172.21.15.119:0/3614097325 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c031d20 con 0x7f3eac06c010 2023-11-08T19:32:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 -- 172.21.15.119:0/3614097325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 msgr2=0x7f3eac071790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 --2- 172.21.15.119:0/3614097325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac071790 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f3eac06f5f0 tx=0x7f3e9c02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 -- 172.21.15.119:0/3614097325 shutdown_connections 2023-11-08T19:32:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 --2- 172.21.15.119:0/3614097325 >> v2:172.21.15.188:3300/0 conn(0x7f3eac06ffd0 0x7f3eac0ca140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 --2- 172.21.15.119:0/3614097325 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac071790 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 -- 172.21.15.119:0/3614097325 >> 172.21.15.119:0/3614097325 conn(0x7f3eac0b0ed0 msgr2=0x7f3eac0ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:07.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 -- 172.21.15.119:0/3614097325 shutdown_connections 2023-11-08T19:32:07.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.190+0000 7f3eb2935640 1 -- 172.21.15.119:0/3614097325 wait complete. 2023-11-08T19:32:07.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.192+0000 7f3eb2935640 1 Processor -- start 2023-11-08T19:32:07.200 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.192+0000 7f3eb2935640 1 -- start start 2023-11-08T19:32:07.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb2935640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac139b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:07.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb2935640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eac06ffd0 0x7f3eac158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:07.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb2935640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eac1434d0 0x7f3eac0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:07.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb2935640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac1396f0 con 0x7f3eac06c010 2023-11-08T19:32:07.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb2935640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0b3700 con 0x7f3eac1434d0 2023-11-08T19:32:07.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb2935640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eac0b39e0 con 0x7f3eac06ffd0 2023-11-08T19:32:07.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb1933640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac139b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:07.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb1933640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac139b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46256/0 (socket says 172.21.15.119:46256) 2023-11-08T19:32:07.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.193+0000 7f3eb1933640 1 -- 172.21.15.119:0/2506544599 learned_addr learned my addr 172.21.15.119:0/2506544599 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:07.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb2134640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eac1434d0 0x7f3eac0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:07.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1933640 1 -- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eac06ffd0 msgr2=0x7f3eac158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:07.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1132640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eac06ffd0 0x7f3eac158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:07.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1933640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eac06ffd0 0x7f3eac158c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1933640 1 -- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eac1434d0 msgr2=0x7f3eac0b2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:07.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1933640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eac1434d0 0x7f3eac0b2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1933640 1 -- 172.21.15.119:0/2506544599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c009400 con 0x7f3eac06c010 2023-11-08T19:32:07.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb2134640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eac1434d0 0x7f3eac0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:07.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb1933640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac139b80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800c400 tx=0x7f3ea800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:07.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8005ce0 con 0x7f3eac06c010 2023-11-08T19:32:07.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea8005e40 con 0x7f3eac06c010 2023-11-08T19:32:07.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8007310 con 0x7f3eac06c010 2023-11-08T19:32:07.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.194+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eac1390a0 con 0x7f3eac06c010 2023-11-08T19:32:07.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.195+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eac1591b0 con 0x7f3eac06c010 2023-11-08T19:32:07.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.196+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eac04e490 con 0x7f3eac06c010 2023-11-08T19:32:07.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.197+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ea8007470 con 0x7f3eac06c010 2023-11-08T19:32:07.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.198+0000 7f3ea27fc640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3e9006aea0 0x7f3e9006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:07.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.198+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f3ea8089270 con 0x7f3eac06c010 2023-11-08T19:32:07.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.198+0000 7f3eb1132640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3e9006aea0 0x7f3e9006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:07.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.199+0000 7f3eb1132640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3e9006aea0 0x7f3e9006d360 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c036000 tx=0x7f3e9c03c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:07.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.202+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ea8053b10 con 0x7f3eac06c010 2023-11-08T19:32:07.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.405+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3eac0ca680 con 0x7f3eac06c010 2023-11-08T19:32:07.408 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.406+0000 7f3ea27fc640 1 -- 172.21.15.119:0/2506544599 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 13 entries v106) v1 ==== 81+0+799 (secure 0 0 0) 0x7f3eac0ca680 con 0x7f3eac06c010 2023-11-08T19:32:07.409 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:0/1419527548 2023-11-08T20:31:50.849149+0000 2023-11-08T19:32:07.409 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6834/4246462419 2023-11-09T19:31:43.291623+0000 2023-11-08T19:32:07.409 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6833/2339927924 2023-11-09T19:31:35.619931+0000 2023-11-08T19:32:07.409 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/1517894744 2023-11-09T19:31:05.198475+0000 2023-11-08T19:32:07.410 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/1517894744 2023-11-09T19:31:05.198475+0000 2023-11-08T19:32:07.411 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6835/310147451 2023-11-09T19:30:37.434843+0000 2023-11-08T19:32:07.411 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6833/310147451 2023-11-09T19:30:37.434843+0000 2023-11-08T19:32:07.411 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6837/3498735251 2023-11-09T19:30:09.035506+0000 2023-11-08T19:32:07.412 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6832/4246462419 2023-11-09T19:31:43.291623+0000 2023-11-08T19:32:07.412 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.119:6835/3498735251 2023-11-09T19:30:09.035506+0000 2023-11-08T19:32:07.412 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6835/168553637 2023-11-09T19:29:41.161236+0000 2023-11-08T19:32:07.412 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6835/2339927924 2023-11-09T19:31:35.619931+0000 2023-11-08T19:32:07.412 INFO:teuthology.orchestra.run.smithi119.stdout:172.21.15.188:6833/168553637 2023-11-09T19:29:41.161236+0000 2023-11-08T19:32:07.413 INFO:teuthology.orchestra.run.smithi119.stderr:listed 13 entries 2023-11-08T19:32:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3e9006aea0 msgr2=0x7f3e9006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3e9006aea0 0x7f3e9006d360 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c036000 tx=0x7f3e9c03c040 comp rx=0 tx=0).stop 2023-11-08T19:32:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 msgr2=0x7f3eac139b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac139b80 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f3ea800c400 tx=0x7f3ea800c8c0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.414 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 shutdown_connections 2023-11-08T19:32:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f3e9006aea0 0x7f3e9006d360 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3eac1434d0 0x7f3eac0b2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3eac06ffd0 0x7f3eac158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 --2- 172.21.15.119:0/2506544599 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f3eac06c010 0x7f3eac139b80 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:07.415 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 >> 172.21.15.119:0/2506544599 conn(0x7f3eac0b0ed0 msgr2=0x7f3eac13e020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:07.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 shutdown_connections 2023-11-08T19:32:07.416 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.409+0000 7f3eb2935640 1 -- 172.21.15.119:0/2506544599 wait complete. 2023-11-08T19:32:07.431 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-08T19:32:07.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:07.432 DEBUG:teuthology.orchestra.run.smithi119:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-08T19:32:07.533 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:32:07.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:07.534 DEBUG:teuthology.orchestra.run.smithi119:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T19:32:07.591 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:32:07.592 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:07.653 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:32:07.653 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:32:07.654 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:07.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:32:07.714 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:32:07.715 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:07.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:32:07.778 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:32:07.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.995+0000 7f8cdc9e7640 1 Processor -- start 2023-11-08T19:32:07.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.996+0000 7f8cdc9e7640 1 -- start start 2023-11-08T19:32:07.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.997+0000 7f8cdc9e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8cd80cf140 0x7f8cd8165f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.997+0000 7f8cdc9e7640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80ccda0 con 0x7f8cd80b3030 2023-11-08T19:32:08.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.997+0000 7f8cdc9e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd80d0300 con 0x7f8cd80b6200 2023-11-08T19:32:08.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.997+0000 7f8cdc9e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd80cd060 con 0x7f8cd80cf140 2023-11-08T19:32:08.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.997+0000 7f8ccdd9b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8cd80b3030 0x7f8cd80cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45994/0 (socket says 172.21.15.119:45994) 2023-11-08T19:32:08.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.997+0000 7f8ccdd9b640 1 -- 172.21.15.119:0/636376554 learned_addr learned my addr 172.21.15.119:0/636376554 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:08.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd6d9d640 1 --2- 172.21.15.119:0/636376554 >> v2:172.21.15.188:3300/0 conn(0x7f8cd80cf140 0x7f8cd8165f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1432154137 0 0) 0x7f8cd80ccda0 con 0x7f8cd80b3030 2023-11-08T19:32:08.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc4003520 con 0x7f8cd80b3030 2023-11-08T19:32:08.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 648172570 0 0) 0x7f8cc4003520 con 0x7f8cd80b3030 2023-11-08T19:32:08.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 >> v2:172.21.15.188:3300/0 conn(0x7f8cd80cf140 msgr2=0x7f8cd8165f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:08.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 --2- 172.21.15.119:0/636376554 >> v2:172.21.15.188:3300/0 conn(0x7f8cd80cf140 0x7f8cd8165f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 >> v1:172.21.15.188:6789/0 conn(0x7f8cd80b6200 legacy=0x7f8cd815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80cfd80 con 0x7f8cd80b3030 2023-11-08T19:32:08.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f8cc0001670 con 0x7f8cd80b3030 2023-11-08T19:32:08.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.998+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 >> v1:172.21.15.119:6789/0 conn(0x7f8cd80b3030 legacy=0x7f8cd80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd5d9b640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cc4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd5d9b640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40047c0 0x7f8cc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd5d9b640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc4005180 0x7f8cc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80cd060 con 0x7f8cc4003970 2023-11-08T19:32:08.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400c120 con 0x7f8cc40047c0 2023-11-08T19:32:08.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc400c2e0 con 0x7f8cc4005180 2023-11-08T19:32:08.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd659c640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cc4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8ccdd9b640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40047c0 0x7f8cc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd659c640 1 -- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc4005180 msgr2=0x7f8cc400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:08.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd659c640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cc4005180 0x7f8cc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd659c640 1 -- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40047c0 msgr2=0x7f8cc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:08.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd659c640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40047c0 0x7f8cc4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd659c640 1 -- 172.21.15.119:0/636376554 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4003520 con 0x7f8cc4003970 2023-11-08T19:32:08.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8ccdd9b640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cc40047c0 0x7f8cc4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:08.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:07.999+0000 7f8cd6d9d640 1 -- 172.21.15.119:0/636376554 reap_dead start 2023-11-08T19:32:08.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.000+0000 7f8cd659c640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cc4003d90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8001750 tx=0x7f8cc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7e6c29e7e4ecb44 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80052d0 con 0x7f8cc4003970 2023-11-08T19:32:08.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc802e050 con 0x7f8cc4003970 2023-11-08T19:32:08.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cd5d9b640 1 -- 172.21.15.119:0/636376554 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8003830 con 0x7f8cc4003970 2023-11-08T19:32:08.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/636376554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 msgr2=0x7f8cc4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cc4003d90 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8001750 tx=0x7f8cc40057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/636376554 shutdown_connections 2023-11-08T19:32:08.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/636376554 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cc4003d90 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/636376554 >> 172.21.15.119:0/636376554 conn(0x7f8cd806dc60 msgr2=0x7f8cd806e070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:08.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/636376554 shutdown_connections 2023-11-08T19:32:08.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.001+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/636376554 wait complete. 2023-11-08T19:32:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.003+0000 7f8cdc9e7640 1 Processor -- start 2023-11-08T19:32:08.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.003+0000 7f8cdc9e7640 1 -- start start 2023-11-08T19:32:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cdc9e7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cd8145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cdc9e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cd81f01c0 0x7f8cd813fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cdc9e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd8140420 0x7f8cd80c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cdc9e7640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80c6f60 con 0x7f8cc4003970 2023-11-08T19:32:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cdc9e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80c4cc0 con 0x7f8cd8140420 2023-11-08T19:32:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cdc9e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd80c65c0 con 0x7f8cd81f01c0 2023-11-08T19:32:08.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.004+0000 7f8cd659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cd8145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cd8145f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46274/0 (socket says 172.21.15.119:46274) 2023-11-08T19:32:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 -- 172.21.15.119:0/1168360935 learned_addr learned my addr 172.21.15.119:0/1168360935 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8ccdd9b640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cd81f01c0 0x7f8cd813fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 -- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cd81f01c0 msgr2=0x7f8cd813fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cd81f01c0 0x7f8cd813fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd6d9d640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd8140420 0x7f8cd80c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 -- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd8140420 msgr2=0x7f8cd80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd8140420 0x7f8cd80c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 -- 172.21.15.119:0/1168360935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80c68a0 con 0x7f8cc4003970 2023-11-08T19:32:08.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8ccdd9b640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cd81f01c0 0x7f8cd813fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:08.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.005+0000 7f8cd659c640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cd8145f80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8034040 tx=0x7f8cc8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.006+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc80076c0 con 0x7f8cc4003970 2023-11-08T19:32:08.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.006+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc8007a30 con 0x7f8cc4003970 2023-11-08T19:32:08.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.006+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc8006be0 con 0x7f8cc4003970 2023-11-08T19:32:08.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.006+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd80c6b80 con 0x7f8cc4003970 2023-11-08T19:32:08.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.006+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd80c61e0 con 0x7f8cc4003970 2023-11-08T19:32:08.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.007+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd8067520 con 0x7f8cc4003970 2023-11-08T19:32:08.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.008+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8cc8006d40 con 0x7f8cc4003970 2023-11-08T19:32:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.009+0000 7f8ccf7fe640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8cb806ae70 0x7f8cb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.009+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f8cc80abe80 con 0x7f8cc4003970 2023-11-08T19:32:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.009+0000 7f8ccdd9b640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8cb806ae70 0x7f8cb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.010+0000 7f8ccdd9b640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8cb806ae70 0x7f8cb806d330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80d17f0 tx=0x7f8cc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.013+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8cc8076830 con 0x7f8cc4003970 2023-11-08T19:32:08.243 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.241+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cd80b2e10 con 0x7f8cc4003970 2023-11-08T19:32:08.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.242+0000 7f8ccf7fe640 1 -- 172.21.15.119:0/1168360935 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f8cc804b020 con 0x7f8cc4003970 2023-11-08T19:32:08.245 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:08.245 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:32:08.246 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:32:08.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8cb806ae70 msgr2=0x7f8cb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8cb806ae70 0x7f8cb806d330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8cd80d17f0 tx=0x7f8cc0009040 comp rx=0 tx=0).stop 2023-11-08T19:32:08.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 msgr2=0x7f8cd8145f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cd8145f80 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8034040 tx=0x7f8cc8001120 comp rx=0 tx=0).stop 2023-11-08T19:32:08.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 shutdown_connections 2023-11-08T19:32:08.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8cb806ae70 0x7f8cb806d330 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8cd8140420 0x7f8cd80c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8cd81f01c0 0x7f8cd813fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 --2- 172.21.15.119:0/1168360935 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8cc4003970 0x7f8cd8145f80 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.245+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 >> 172.21.15.119:0/1168360935 conn(0x7f8cd806dc60 msgr2=0x7f8cd81551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.246+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 shutdown_connections 2023-11-08T19:32:08.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.246+0000 7f8cdc9e7640 1 -- 172.21.15.119:0/1168360935 wait complete. 2023-11-08T19:32:08.267 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:32:08.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.479+0000 7f38620b0640 1 Processor -- start 2023-11-08T19:32:08.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.479+0000 7f38620b0640 1 -- start start 2023-11-08T19:32:08.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.480+0000 7f38620b0640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f385c0703a0 0x7f385c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.480+0000 7f38620b0640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f385c0cb2e0 0x7f385c0cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.480+0000 7f38620b0640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f385c15dd60 con 0x7f385c159d80 2023-11-08T19:32:08.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.480+0000 7f38620b0640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c0cbcb0 con 0x7f385c0703a0 2023-11-08T19:32:08.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.480+0000 7f38620b0640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385c15da30 con 0x7f385c0cb2e0 2023-11-08T19:32:08.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385b7fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f385c159d80 0x7f385c0ca080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:45998/0 (socket says 172.21.15.119:45998) 2023-11-08T19:32:08.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385b7fe640 1 -- 172.21.15.119:0/471748145 learned_addr learned my addr 172.21.15.119:0/471748145 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:08.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385affd640 1 --2- 172.21.15.119:0/471748145 >> v2:172.21.15.119:3300/0 conn(0x7f385c0703a0 0x7f385c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385a7fc640 1 -- 172.21.15.119:0/471748145 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769709559 0 0) 0x7f385c15dd60 con 0x7f385c159d80 2023-11-08T19:32:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385affd640 1 -- 172.21.15.119:0/471748145 >> v2:172.21.15.188:3300/0 conn(0x7f385c0cb2e0 msgr2=0x7f385c0cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385bfff640 1 --2- 172.21.15.119:0/471748145 >> v2:172.21.15.188:3300/0 conn(0x7f385c0cb2e0 0x7f385c0cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385affd640 1 --2- 172.21.15.119:0/471748145 >> v2:172.21.15.188:3300/0 conn(0x7f385c0cb2e0 0x7f385c0cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385affd640 1 -- 172.21.15.119:0/471748145 >> v1:172.21.15.119:6789/0 conn(0x7f385c159d80 legacy=0x7f385c0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.481+0000 7f385affd640 1 -- 172.21.15.119:0/471748145 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385c0ca8b0 con 0x7f385c0703a0 2023-11-08T19:32:08.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.482+0000 7f385affd640 1 --2- 172.21.15.119:0/471748145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0703a0 0x7f385c15e1f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f385c0cc040 tx=0x7f384402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d041a2adcbe307 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f385a7fc640 1 -- 172.21.15.119:0/471748145 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844031830 con 0x7f385c0703a0 2023-11-08T19:32:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f385a7fc640 1 -- 172.21.15.119:0/471748145 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3844031990 con 0x7f385c0703a0 2023-11-08T19:32:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f385a7fc640 1 -- 172.21.15.119:0/471748145 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3844031d20 con 0x7f385c0703a0 2023-11-08T19:32:08.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 -- 172.21.15.119:0/471748145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0703a0 msgr2=0x7f385c15e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 --2- 172.21.15.119:0/471748145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0703a0 0x7f385c15e1f0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f385c0cc040 tx=0x7f384402b8d0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 -- 172.21.15.119:0/471748145 shutdown_connections 2023-11-08T19:32:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 --2- 172.21.15.119:0/471748145 >> v2:172.21.15.188:3300/0 conn(0x7f385c0cb2e0 0x7f385c0cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 --2- 172.21.15.119:0/471748145 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0703a0 0x7f385c15e1f0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 -- 172.21.15.119:0/471748145 >> 172.21.15.119:0/471748145 conn(0x7f385c0b06e0 msgr2=0x7f385c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 -- 172.21.15.119:0/471748145 shutdown_connections 2023-11-08T19:32:08.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.483+0000 7f38620b0640 1 -- 172.21.15.119:0/471748145 wait complete. 2023-11-08T19:32:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.485+0000 7f38620b0640 1 Processor -- start 2023-11-08T19:32:08.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.485+0000 7f38620b0640 1 -- start start 2023-11-08T19:32:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f38620b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c0703a0 0x7f385c0a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f38620b0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 0x7f385c1f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f38620b0640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c159d80 0x7f385c0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f38620b0640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c0c5fd0 con 0x7f385c0cb2e0 2023-11-08T19:32:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f38620b0640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1f3950 con 0x7f385c159d80 2023-11-08T19:32:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f38620b0640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f385c1f3c30 con 0x7f385c0703a0 2023-11-08T19:32:08.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.486+0000 7f385affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 0x7f385c1f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 0x7f385c1f2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:46284/0 (socket says 172.21.15.119:46284) 2023-11-08T19:32:08.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 -- 172.21.15.119:0/2833835846 learned_addr learned my addr 172.21.15.119:0/2833835846 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:08.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385b7fe640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c0703a0 0x7f385c0a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385bfff640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c159d80 0x7f385c0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 -- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c0703a0 msgr2=0x7f385c0a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c0703a0 0x7f385c0a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 -- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c159d80 msgr2=0x7f385c0c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c159d80 0x7f385c0c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 -- 172.21.15.119:0/2833835846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3844009400 con 0x7f385c0cb2e0 2023-11-08T19:32:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f385affd640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 0x7f385c1f2fe0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3844000c00 tx=0x7f38440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.487+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384403aa60 con 0x7f385c0cb2e0 2023-11-08T19:32:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.488+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384403abe0 con 0x7f385c0cb2e0 2023-11-08T19:32:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.488+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38440443f0 con 0x7f385c0cb2e0 2023-11-08T19:32:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.488+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f385c1f3f10 con 0x7f385c0cb2e0 2023-11-08T19:32:08.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.488+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f385c143990 con 0x7f385c0cb2e0 2023-11-08T19:32:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.489+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385c04e490 con 0x7f385c0cb2e0 2023-11-08T19:32:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.490+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f384404d450 con 0x7f385c0cb2e0 2023-11-08T19:32:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.490+0000 7f3853fff640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f383c06ae70 0x7f383c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.491+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f38440ac110 con 0x7f385c0cb2e0 2023-11-08T19:32:08.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.491+0000 7f385b7fe640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f383c06ae70 0x7f383c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.491+0000 7f385b7fe640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f383c06ae70 0x7f383c06d330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f384c0012e0 tx=0x7f384c007ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.495+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3844076910 con 0x7f385c0cb2e0 2023-11-08T19:32:08.689 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.686+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f385c0c6110 con 0x7f385c0cb2e0 2023-11-08T19:32:08.691 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.688+0000 7f3853fff640 1 -- 172.21.15.119:0/2833835846 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23077 (secure 0 0 0) 0x7f3844035070 con 0x7f385c0cb2e0 2023-11-08T19:32:08.691 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:08.692 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":106,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:31:58.747478+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1419527548":"2023-11-08T20:31:50.849149+0000","172.21.15.188:6834/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.188:6833/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.119:6837/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.119:6835/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.188:6835/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.188:6833/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.119:6837/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6832/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.119:6835/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6835/168553637":"2023-11-09T19:29:41.161236+0000","172.21.15.188:6835/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.188:6833/168553637":"2023-11-09T19:29:41.161236+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":8}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:32:08.693 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f383c06ae70 msgr2=0x7f383c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f383c06ae70 0x7f383c06d330 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f384c0012e0 tx=0x7f384c007ef0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 msgr2=0x7f385c1f2fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 0x7f385c1f2fe0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f3844000c00 tx=0x7f38440046c0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.694 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 shutdown_connections 2023-11-08T19:32:08.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f383c06ae70 0x7f383c06d330 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c159d80 0x7f385c0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f385c0cb2e0 0x7f385c1f2fe0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.695 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 --2- 172.21.15.119:0/2833835846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c0703a0 0x7f385c0a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 >> 172.21.15.119:0/2833835846 conn(0x7f385c0b06e0 msgr2=0x7f385c0ca080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:08.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 shutdown_connections 2023-11-08T19:32:08.696 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.691+0000 7f38620b0640 1 -- 172.21.15.119:0/2833835846 wait complete. 2023-11-08T19:32:08.712 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T19:24:46.723787+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'snap-test-pool', 'create_time': '2023-11-08T19:26:39.314051+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'test_data_pool', 'create_time': '2023-11-08T19:27:45.435414+0000', 'flags': 16385, 'flags_names': 'hashpspool,pool_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'pool', 'snap_seq': 2, 'snap_epoch': 71, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'test_metadata_pool', 'create_time': '2023-11-08T19:27:46.245481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T19:28:50.669529+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T19:28:51.498158+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 105, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T19:32:08.714 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T19:32:08.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.923+0000 7f73dc832640 1 Processor -- start 2023-11-08T19:32:08.925 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.923+0000 7f73dc832640 1 -- start start 2023-11-08T19:32:08.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.924+0000 7f73dc832640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f73d81555b0 0x7f73d80ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73dc832640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73d80cb4e0 0x7f73d80cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73dc832640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d80cbe70 con 0x7f73d80cd110 2023-11-08T19:32:08.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73dc832640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80cd590 con 0x7f73d80cb4e0 2023-11-08T19:32:08.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73dc832640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80cc6c0 con 0x7f73d81555b0 2023-11-08T19:32:08.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73d80cb4e0 0x7f73d80cb930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.928 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f73d80cb4e0 0x7f73d80cb930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56650/0 (socket says 172.21.15.119:56650) 2023-11-08T19:32:08.929 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 -- 172.21.15.119:0/3504274224 learned_addr learned my addr 172.21.15.119:0/3504274224 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:08.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 -- 172.21.15.119:0/3504274224 >> v2:172.21.15.188:3300/0 conn(0x7f73d81555b0 msgr2=0x7f73d80ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:08.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d659c640 1 --2- 172.21.15.119:0/3504274224 >> v2:172.21.15.188:3300/0 conn(0x7f73d81555b0 0x7f73d80ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 --2- 172.21.15.119:0/3504274224 >> v2:172.21.15.188:3300/0 conn(0x7f73d81555b0 0x7f73d80ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 -- 172.21.15.119:0/3504274224 >> v1:172.21.15.119:6789/0 conn(0x7f73d80cd110 legacy=0x7f73d8070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.925+0000 7f73d6d9d640 1 -- 172.21.15.119:0/3504274224 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8071770 con 0x7f73d80cb4e0 2023-11-08T19:32:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73d6d9d640 1 --2- 172.21.15.119:0/3504274224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cb4e0 0x7f73d80cb930 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f73d80cb4b0 tx=0x7f73d002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604e5029e9aa1386 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73d559a640 1 -- 172.21.15.119:0/3504274224 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d002ca80 con 0x7f73d80cb4e0 2023-11-08T19:32:08.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73d559a640 1 -- 172.21.15.119:0/3504274224 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d002cbe0 con 0x7f73d80cb4e0 2023-11-08T19:32:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73d559a640 1 -- 172.21.15.119:0/3504274224 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d00343f0 con 0x7f73d80cb4e0 2023-11-08T19:32:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73dc832640 1 -- 172.21.15.119:0/3504274224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cb4e0 msgr2=0x7f73d80cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73dc832640 1 --2- 172.21.15.119:0/3504274224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cb4e0 0x7f73d80cb930 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f73d80cb4b0 tx=0x7f73d002bee0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73dc832640 1 -- 172.21.15.119:0/3504274224 shutdown_connections 2023-11-08T19:32:08.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.927+0000 7f73dc832640 1 --2- 172.21.15.119:0/3504274224 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cb4e0 0x7f73d80cb930 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.928+0000 7f73dc832640 1 --2- 172.21.15.119:0/3504274224 >> v2:172.21.15.188:3300/0 conn(0x7f73d81555b0 0x7f73d80ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.928+0000 7f73dc832640 1 -- 172.21.15.119:0/3504274224 >> 172.21.15.119:0/3504274224 conn(0x7f73d80b06e0 msgr2=0x7f73d80b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.928+0000 7f73dc832640 1 -- 172.21.15.119:0/3504274224 shutdown_connections 2023-11-08T19:32:08.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.928+0000 7f73dc832640 1 -- 172.21.15.119:0/3504274224 wait complete. 2023-11-08T19:32:08.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.929+0000 7f73dc832640 1 Processor -- start 2023-11-08T19:32:08.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.929+0000 7f73dc832640 1 -- start start 2023-11-08T19:32:08.935 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.930+0000 7f73dc832640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73d80cb4e0 0x7f73d80c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.930+0000 7f73dc832640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 0x7f73d814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.930+0000 7f73dc832640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73d81555b0 0x7f73d80c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.930+0000 7f73dc832640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815b550 con 0x7f73d80cd110 2023-11-08T19:32:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.930+0000 7f73dc832640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d815b9e0 con 0x7f73d81555b0 2023-11-08T19:32:08.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.930+0000 7f73dc832640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d80c6d90 con 0x7f73d80cb4e0 2023-11-08T19:32:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 0x7f73d814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 0x7f73d814f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56664/0 (socket says 172.21.15.119:56664) 2023-11-08T19:32:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 -- 172.21.15.119:0/3283968110 learned_addr learned my addr 172.21.15.119:0/3283968110 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 -- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73d80cb4e0 msgr2=0x7f73d80c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73d80cb4e0 0x7f73d80c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d6d9d640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73d81555b0 0x7f73d80c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 -- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73d81555b0 msgr2=0x7f73d80c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73d81555b0 0x7f73d80c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 -- 172.21.15.119:0/3283968110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0009b30 con 0x7f73d80cd110 2023-11-08T19:32:08.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.931+0000 7f73d5d9b640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 0x7f73d814f220 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c400 tx=0x7f73cc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.932+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc0052d0 con 0x7f73d80cd110 2023-11-08T19:32:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.932+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc005430 con 0x7f73d80cd110 2023-11-08T19:32:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.932+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc003830 con 0x7f73d80cd110 2023-11-08T19:32:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.932+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d815ac20 con 0x7f73d80cd110 2023-11-08T19:32:08.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.932+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d815b1e0 con 0x7f73d80cd110 2023-11-08T19:32:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.933+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d815b370 con 0x7f73d80cd110 2023-11-08T19:32:08.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.934+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73cc003990 con 0x7f73d80cd110 2023-11-08T19:32:08.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.935+0000 7f73c6ffd640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73b806ae70 0x7f73b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:08.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.935+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f73cc089280 con 0x7f73d80cd110 2023-11-08T19:32:08.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.935+0000 7f73d659c640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73b806ae70 0x7f73b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:08.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.936+0000 7f73d659c640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73b806ae70 0x7f73b806d330 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006fd0 tx=0x7f73c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:08.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:08.940+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73cc053b20 con 0x7f73d80cd110 2023-11-08T19:32:09.130 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.128+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f73d80cbe70 con 0x7f73d80cd110 2023-11-08T19:32:09.131 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.129+0000 7f73c6ffd640 1 -- 172.21.15.119:0/3283968110 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f73d80cbe70 con 0x7f73d80cd110 2023-11-08T19:32:09.131 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 1 2023-11-08T19:32:09.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73b806ae70 msgr2=0x7f73b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.134 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73b806ae70 0x7f73b806d330 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f73c8006fd0 tx=0x7f73c8008040 comp rx=0 tx=0).stop 2023-11-08T19:32:09.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 msgr2=0x7f73d814f220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 0x7f73d814f220 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c400 tx=0x7f73cc00c8c0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 shutdown_connections 2023-11-08T19:32:09.135 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73b806ae70 0x7f73b806d330 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73d81555b0 0x7f73d80c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f73d80cd110 0x7f73d814f220 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 --2- 172.21.15.119:0/3283968110 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73d80cb4e0 0x7f73d80c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.131+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 >> 172.21.15.119:0/3283968110 conn(0x7f73d80b06e0 msgr2=0x7f73d80ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:09.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.132+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 shutdown_connections 2023-11-08T19:32:09.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.132+0000 7f73dc832640 1 -- 172.21.15.119:0/3283968110 wait complete. 2023-11-08T19:32:09.151 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get snap-test-pool pg_num 2023-11-08T19:32:09.359 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.357+0000 7efec9fe2640 1 Processor -- start 2023-11-08T19:32:09.360 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.357+0000 7efec9fe2640 1 -- start start 2023-11-08T19:32:09.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.358+0000 7efec9fe2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 0x7efec406a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.358+0000 7efec9fe2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec406a990 0x7efec40cd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.358+0000 7efec9fe2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efec40ccdc0 con 0x7efec415ac10 2023-11-08T19:32:09.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.358+0000 7efec9fe2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec406ae20 con 0x7efec406f770 2023-11-08T19:32:09.361 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.358+0000 7efec9fe2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec406fd50 con 0x7efec406a990 2023-11-08T19:32:09.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.359+0000 7efec3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 0x7efec406a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.359+0000 7efec97e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec406a990 0x7efec40cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.363 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.359+0000 7efec8fe0640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7efec415ac10 0x7efec406d470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.119:53564/0 (socket says 172.21.15.119:53564) 2023-11-08T19:32:09.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.360+0000 7efec8fe0640 1 -- 172.21.15.119:0/3417907819 learned_addr learned my addr 172.21.15.119:0/3417907819 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:09.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.360+0000 7efec3fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 0x7efec406a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:57596/0 (socket says 172.21.15.119:57596) 2023-11-08T19:32:09.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.360+0000 7efec97e1640 1 -- 172.21.15.119:0/3417907819 >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 msgr2=0x7efec406a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:09.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.360+0000 7efec97e1640 1 --2- 172.21.15.119:0/3417907819 >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 0x7efec406a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.364 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.360+0000 7efec97e1640 1 -- 172.21.15.119:0/3417907819 >> v1:172.21.15.188:6789/0 conn(0x7efec415ac10 legacy=0x7efec406d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.361+0000 7efec97e1640 1 -- 172.21.15.119:0/3417907819 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec4070030 con 0x7efec406a990 2023-11-08T19:32:09.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.360+0000 7efec37fe640 1 -- 172.21.15.119:0/3417907819 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901023179 0 0) 0x7efec40ccdc0 con 0x7efec415ac10 2023-11-08T19:32:09.365 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.361+0000 7efec3fff640 1 --2- 172.21.15.119:0/3417907819 >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 0x7efec406a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T19:32:09.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.362+0000 7efec97e1640 1 --2- 172.21.15.119:0/3417907819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec406a990 0x7efec40cd230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efebc009d90 tx=0x7efebc02beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8816ecf591e4769a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:09.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec37fe640 1 -- 172.21.15.119:0/3417907819 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc02cc60 con 0x7efec406a990 2023-11-08T19:32:09.367 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec37fe640 1 -- 172.21.15.119:0/3417907819 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc02cdc0 con 0x7efec406a990 2023-11-08T19:32:09.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec37fe640 1 -- 172.21.15.119:0/3417907819 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc034660 con 0x7efec406a990 2023-11-08T19:32:09.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 -- 172.21.15.119:0/3417907819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec406a990 msgr2=0x7efec40cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 --2- 172.21.15.119:0/3417907819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec406a990 0x7efec40cd230 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efebc009d90 tx=0x7efebc02beb0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.368 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 -- 172.21.15.119:0/3417907819 shutdown_connections 2023-11-08T19:32:09.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 --2- 172.21.15.119:0/3417907819 >> v2:172.21.15.188:3300/0 conn(0x7efec406f770 0x7efec406a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 --2- 172.21.15.119:0/3417907819 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec406a990 0x7efec40cd230 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 -- 172.21.15.119:0/3417907819 >> 172.21.15.119:0/3417907819 conn(0x7efec40b0a20 msgr2=0x7efec40b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:09.369 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 -- 172.21.15.119:0/3417907819 shutdown_connections 2023-11-08T19:32:09.370 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.363+0000 7efec9fe2640 1 -- 172.21.15.119:0/3417907819 wait complete. 2023-11-08T19:32:09.372 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.365+0000 7efec9fe2640 1 Processor -- start 2023-11-08T19:32:09.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.365+0000 7efec9fe2640 1 -- start start 2023-11-08T19:32:09.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec9fe2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 0x7efec414f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec9fe2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efec406f770 0x7efec41f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec9fe2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec415ac10 0x7efec41485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.373 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec9fe2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41f49a0 con 0x7efec406a990 2023-11-08T19:32:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec9fe2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41f4510 con 0x7efec406f770 2023-11-08T19:32:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec9fe2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec4148b00 con 0x7efec415ac10 2023-11-08T19:32:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.366+0000 7efec8fe0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 0x7efec414f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 0x7efec414f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56666/0 (socket says 172.21.15.119:56666) 2023-11-08T19:32:09.374 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 -- 172.21.15.119:0/2005334642 learned_addr learned my addr 172.21.15.119:0/2005334642 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec3fff640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efec406f770 0x7efec41f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec97e1640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec415ac10 0x7efec41485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 -- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec415ac10 msgr2=0x7efec41485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.375 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec415ac10 0x7efec41485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 -- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efec406f770 msgr2=0x7efec41f0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efec406f770 0x7efec41f0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 -- 172.21.15.119:0/2005334642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb0009400 con 0x7efec406a990 2023-11-08T19:32:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec3fff640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efec406f770 0x7efec41f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec8fe0640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 0x7efec414f390 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efeb800c500 tx=0x7efeb800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.367+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8007560 con 0x7efec406a990 2023-11-08T19:32:09.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.368+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb80047e0 con 0x7efec406a990 2023-11-08T19:32:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.368+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efebc009a70 con 0x7efec406a990 2023-11-08T19:32:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.368+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb8004b90 con 0x7efec406a990 2023-11-08T19:32:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.368+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efec41f3be0 con 0x7efec406a990 2023-11-08T19:32:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.369+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec404ece0 con 0x7efec406a990 2023-11-08T19:32:09.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.370+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efeb8004cf0 con 0x7efec406a990 2023-11-08T19:32:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.371+0000 7efec17fa640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efea806ae70 0x7efea806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.379 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.371+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7efeb8089250 con 0x7efec406a990 2023-11-08T19:32:09.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.371+0000 7efec3fff640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efea806ae70 0x7efea806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.372+0000 7efec3fff640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efea806ae70 0x7efea806d330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efec414af30 tx=0x7efeb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:09.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.375+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efeb8053b80 con 0x7efec406a990 2023-11-08T19:32:09.552 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.550+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"} v 0) v1 -- 0x7efec406c110 con 0x7efec406a990 2023-11-08T19:32:09.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.550+0000 7efec17fa640 1 -- 172.21.15.119:0/2005334642 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "snap-test-pool", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7efeb8018020 con 0x7efec406a990 2023-11-08T19:32:09.553 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:09.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efea806ae70 msgr2=0x7efea806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efea806ae70 0x7efea806d330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efec414af30 tx=0x7efeb0008040 comp rx=0 tx=0).stop 2023-11-08T19:32:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 msgr2=0x7efec414f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 0x7efec414f390 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efeb800c500 tx=0x7efeb800c9c0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 shutdown_connections 2023-11-08T19:32:09.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efea806ae70 0x7efea806d330 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efec415ac10 0x7efec41485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efec406f770 0x7efec41f0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 --2- 172.21.15.119:0/2005334642 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efec406a990 0x7efec414f390 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 >> 172.21.15.119:0/2005334642 conn(0x7efec40b0a20 msgr2=0x7efec41553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:09.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 shutdown_connections 2023-11-08T19:32:09.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.553+0000 7efec9fe2640 1 -- 172.21.15.119:0/2005334642 wait complete. 2023-11-08T19:32:09.574 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_data_pool pg_num 2023-11-08T19:32:09.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.785+0000 7fe7f0064640 1 Processor -- start 2023-11-08T19:32:09.788 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.786+0000 7fe7f0064640 1 -- start start 2023-11-08T19:32:09.789 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7f0064640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7e80ce240 0x7fe7e80b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7f0064640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7f0064640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7e8161b60 con 0x7fe7e815d070 2023-11-08T19:32:09.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7f0064640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7e80d1500 con 0x7fe7e80ce240 2023-11-08T19:32:09.790 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7f0064640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e8161050 con 0x7fe7e8160bc0 2023-11-08T19:32:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7ed5ff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7e80ce240 0x7fe7e80b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7ed5ff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe7e80ce240 0x7fe7e80b89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56682/0 (socket says 172.21.15.119:56682) 2023-11-08T19:32:09.791 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.787+0000 7fe7ed5ff640 1 -- 172.21.15.119:0/1771444803 learned_addr learned my addr 172.21.15.119:0/1771444803 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:09.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.788+0000 7fe7ed5ff640 1 -- 172.21.15.119:0/1771444803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 msgr2=0x7fe7e80d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:09.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.788+0000 7fe7ee601640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.793 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.788+0000 7fe7ed5ff640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80d1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.788+0000 7fe7ed5ff640 1 -- 172.21.15.119:0/1771444803 >> v1:172.21.15.188:6789/0 conn(0x7fe7e815d070 legacy=0x7fe7e80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.788+0000 7fe7ed5ff640 1 -- 172.21.15.119:0/1771444803 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e81612c0 con 0x7fe7e80ce240 2023-11-08T19:32:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.788+0000 7fe7ee601640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:09.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.789+0000 7fe7ed5ff640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e80b89c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8576839239ca977 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:09.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.789+0000 7fe7ecdfe640 1 -- 172.21.15.119:0/1771444803 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8031640 con 0x7fe7e80ce240 2023-11-08T19:32:09.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.789+0000 7fe7ecdfe640 1 -- 172.21.15.119:0/1771444803 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7d80317a0 con 0x7fe7e80ce240 2023-11-08T19:32:09.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.789+0000 7fe7ecdfe640 1 -- 172.21.15.119:0/1771444803 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8031b30 con 0x7fe7e80ce240 2023-11-08T19:32:09.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 -- 172.21.15.119:0/1771444803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 msgr2=0x7fe7e80b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.795 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e80b89c0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009870 tx=0x7fe7d802b8f0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 -- 172.21.15.119:0/1771444803 shutdown_connections 2023-11-08T19:32:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e80b89c0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 --2- 172.21.15.119:0/1771444803 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 -- 172.21.15.119:0/1771444803 >> 172.21.15.119:0/1771444803 conn(0x7fe7e80cf970 msgr2=0x7fe7e80cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 -- 172.21.15.119:0/1771444803 shutdown_connections 2023-11-08T19:32:09.797 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.790+0000 7fe7f0064640 1 -- 172.21.15.119:0/1771444803 wait complete. 2023-11-08T19:32:09.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.792+0000 7fe7f0064640 1 Processor -- start 2023-11-08T19:32:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.792+0000 7fe7f0064640 1 -- start start 2023-11-08T19:32:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7f0064640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e81502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7f0064640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7e815d070 0x7fe7e80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7f0064640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7f0064640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e815e520 con 0x7fe7e80ce240 2023-11-08T19:32:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7f0064640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e80ccd40 con 0x7fe7e815d070 2023-11-08T19:32:09.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7f0064640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7e815d760 con 0x7fe7e8160bc0 2023-11-08T19:32:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.793+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e81502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e81502f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56686/0 (socket says 172.21.15.119:56686) 2023-11-08T19:32:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 -- 172.21.15.119:0/3838524658 learned_addr learned my addr 172.21.15.119:0/3838524658 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 -- 172.21.15.119:0/3838524658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 msgr2=0x7fe7e80cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.803 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 -- 172.21.15.119:0/3838524658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7e815d070 msgr2=0x7fe7e80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7e815d070 0x7fe7e80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 -- 172.21.15.119:0/3838524658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d80094d0 con 0x7fe7e80ce240 2023-11-08T19:32:09.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7ede00640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e81502f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4004640 tx=0x7fe7e400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:09.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.794+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4011790 con 0x7fe7e80ce240 2023-11-08T19:32:09.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.795+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e40118f0 con 0x7fe7e80ce240 2023-11-08T19:32:09.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.795+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4011ce0 con 0x7fe7e80ce240 2023-11-08T19:32:09.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.795+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7e815da40 con 0x7fe7e80ce240 2023-11-08T19:32:09.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.795+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7e815e000 con 0x7fe7e80ce240 2023-11-08T19:32:09.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.796+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7e815e1f0 con 0x7fe7e80ce240 2023-11-08T19:32:09.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.797+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7e4012420 con 0x7fe7e80ce240 2023-11-08T19:32:09.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.798+0000 7fe7de7fc640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe7cc06aef0 0x7fe7cc06d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:09.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.798+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7fe7e40898b0 con 0x7fe7e80ce240 2023-11-08T19:32:09.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.798+0000 7fe7ed5ff640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe7cc06aef0 0x7fe7cc06d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.799+0000 7fe7ed5ff640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe7cc06aef0 0x7fe7cc06d3b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8036000 tx=0x7fe7d8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:09.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.802+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7e40542c0 con 0x7fe7e80ce240 2023-11-08T19:32:09.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.974+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"} v 0) v1 -- 0x7fe7e804e490 con 0x7fe7e80ce240 2023-11-08T19:32:09.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.975+0000 7fe7de7fc640 1 -- 172.21.15.119:0/3838524658 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_data_pool", "var": "pg_num"}]=0 v106) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe7e40578e0 con 0x7fe7e80ce240 2023-11-08T19:32:09.978 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:09.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe7cc06aef0 msgr2=0x7fe7cc06d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe7cc06aef0 0x7fe7cc06d3b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8036000 tx=0x7fe7d8030000 comp rx=0 tx=0).stop 2023-11-08T19:32:09.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 msgr2=0x7fe7e81502f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e81502f0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fe7e4004640 tx=0x7fe7e400c8e0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 shutdown_connections 2023-11-08T19:32:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe7cc06aef0 0x7fe7cc06d3b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7e8160bc0 0x7fe7e80cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe7e815d070 0x7fe7e80cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 --2- 172.21.15.119:0/3838524658 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe7e80ce240 0x7fe7e81502f0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.977+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 >> 172.21.15.119:0/3838524658 conn(0x7fe7e80cf970 msgr2=0x7fe7e815ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.978+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 shutdown_connections 2023-11-08T19:32:09.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:09.978+0000 7fe7f0064640 1 -- 172.21.15.119:0/3838524658 wait complete. 2023-11-08T19:32:09.997 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get test_metadata_pool pg_num 2023-11-08T19:32:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.199+0000 7f43b62fc640 1 Processor -- start 2023-11-08T19:32:10.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.199+0000 7f43b62fc640 1 -- start start 2023-11-08T19:32:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43b62fc640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43b015d070 0x7f43b00b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43b62fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00ce240 0x7f43b00d1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43b62fc640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b0161b60 con 0x7f43b0160bc0 2023-11-08T19:32:10.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43b62fc640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b00d1500 con 0x7f43b015d070 2023-11-08T19:32:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43b62fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0161040 con 0x7f43b00ce240 2023-11-08T19:32:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43affff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43b015d070 0x7f43b00b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43affff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f43b015d070 0x7f43b00b89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56694/0 (socket says 172.21.15.119:56694) 2023-11-08T19:32:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.200+0000 7f43affff640 1 -- 172.21.15.119:0/1158892797 learned_addr learned my addr 172.21.15.119:0/1158892797 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:10.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.201+0000 7f43affff640 1 -- 172.21.15.119:0/1158892797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00ce240 msgr2=0x7f43b00d1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.201+0000 7f43affff640 1 --2- 172.21.15.119:0/1158892797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00ce240 0x7f43b00d1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.201+0000 7f43affff640 1 -- 172.21.15.119:0/1158892797 >> v1:172.21.15.188:6789/0 conn(0x7f43b0160bc0 legacy=0x7f43b00cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:10.205 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.201+0000 7f43affff640 1 -- 172.21.15.119:0/1158892797 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b01612c0 con 0x7f43b015d070 2023-11-08T19:32:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.202+0000 7f43affff640 1 --2- 172.21.15.119:0/1158892797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b015d070 0x7f43b00b89c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009ea0 tx=0x7f43a002bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7b19ed45db7af9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:10.206 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.202+0000 7f43aeffd640 1 -- 172.21.15.119:0/1158892797 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a002cae0 con 0x7f43b015d070 2023-11-08T19:32:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.202+0000 7f43aeffd640 1 -- 172.21.15.119:0/1158892797 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a002cc40 con 0x7f43b015d070 2023-11-08T19:32:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.202+0000 7f43aeffd640 1 -- 172.21.15.119:0/1158892797 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a00343f0 con 0x7f43b015d070 2023-11-08T19:32:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.202+0000 7f43b62fc640 1 -- 172.21.15.119:0/1158892797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b015d070 msgr2=0x7f43b00b89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.202+0000 7f43b62fc640 1 --2- 172.21.15.119:0/1158892797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b015d070 0x7f43b00b89c0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f43a0009ea0 tx=0x7f43a002bf40 comp rx=0 tx=0).stop 2023-11-08T19:32:10.207 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.203+0000 7f43b62fc640 1 -- 172.21.15.119:0/1158892797 shutdown_connections 2023-11-08T19:32:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.203+0000 7f43b62fc640 1 --2- 172.21.15.119:0/1158892797 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b00ce240 0x7f43b00d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.203+0000 7f43b62fc640 1 --2- 172.21.15.119:0/1158892797 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b015d070 0x7f43b00b89c0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.203+0000 7f43b62fc640 1 -- 172.21.15.119:0/1158892797 >> 172.21.15.119:0/1158892797 conn(0x7f43b00cf970 msgr2=0x7f43b00cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:10.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.203+0000 7f43b62fc640 1 -- 172.21.15.119:0/1158892797 shutdown_connections 2023-11-08T19:32:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.203+0000 7f43b62fc640 1 -- 172.21.15.119:0/1158892797 wait complete. 2023-11-08T19:32:10.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.204+0000 7f43b62fc640 1 Processor -- start 2023-11-08T19:32:10.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.204+0000 7f43b62fc640 1 -- start start 2023-11-08T19:32:10.211 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.205+0000 7f43b62fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00ce240 0x7f43b015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.205+0000 7f43b62fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b015d070 0x7f43b015fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.205+0000 7f43b62fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 0x7f43b0149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.205+0000 7f43b62fc640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0160360 con 0x7f43b0160bc0 2023-11-08T19:32:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.205+0000 7f43b62fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c8080 con 0x7f43b00ce240 2023-11-08T19:32:10.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.205+0000 7f43b62fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b00c8360 con 0x7f43b015d070 2023-11-08T19:32:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 0x7f43b0149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 0x7f43b0149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56710/0 (socket says 172.21.15.119:56710) 2023-11-08T19:32:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 -- 172.21.15.119:0/3245025850 learned_addr learned my addr 172.21.15.119:0/3245025850 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43affff640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00ce240 0x7f43b015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43af7fe640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b015d070 0x7f43b015fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 -- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b015d070 msgr2=0x7f43b015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b015d070 0x7f43b015fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 -- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00ce240 msgr2=0x7f43b015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00ce240 0x7f43b015f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 -- 172.21.15.119:0/3245025850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0009b30 con 0x7f43b0160bc0 2023-11-08T19:32:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.206+0000 7f43b4899640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 0x7f43b0149f10 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b3c0 tx=0x7f43a800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.207+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011830 con 0x7f43b0160bc0 2023-11-08T19:32:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.207+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8011990 con 0x7f43b0160bc0 2023-11-08T19:32:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.207+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b00c8640 con 0x7f43b0160bc0 2023-11-08T19:32:10.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.207+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8011d40 con 0x7f43b0160bc0 2023-11-08T19:32:10.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.207+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b00c8c00 con 0x7f43b0160bc0 2023-11-08T19:32:10.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.208+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00c8e20 con 0x7f43b0160bc0 2023-11-08T19:32:10.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.209+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43a801b460 con 0x7f43b0160bc0 2023-11-08T19:32:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.210+0000 7f43acff9640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f439c06ae70 0x7f439c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.210+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f43a8089c50 con 0x7f43b0160bc0 2023-11-08T19:32:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.210+0000 7f43affff640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f439c06ae70 0x7f439c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.210+0000 7f43affff640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f439c06ae70 0x7f439c06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f43a0035000 tx=0x7f43a0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:10.218 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.214+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f43a808f020 con 0x7f43b0160bc0 2023-11-08T19:32:10.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.391+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"} v 0) v1 -- 0x7f43b004e490 con 0x7f43b0160bc0 2023-11-08T19:32:10.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.391+0000 7f43acff9640 1 -- 172.21.15.119:0/3245025850 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "test_metadata_pool", "var": "pg_num"}]=0 v106) v1 ==== 107+0+11 (secure 0 0 0) 0x7f43a8054460 con 0x7f43b0160bc0 2023-11-08T19:32:10.394 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:10.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f439c06ae70 msgr2=0x7f439c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f439c06ae70 0x7f439c06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f43a0035000 tx=0x7f43a0030000 comp rx=0 tx=0).stop 2023-11-08T19:32:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 msgr2=0x7f43b0149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 0x7f43b0149f10 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f43a800b3c0 tx=0x7f43a800b880 comp rx=0 tx=0).stop 2023-11-08T19:32:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 shutdown_connections 2023-11-08T19:32:10.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f439c06ae70 0x7f439c06d330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f43b0160bc0 0x7f43b0149f10 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f43b015d070 0x7f43b015fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 --2- 172.21.15.119:0/3245025850 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f43b00ce240 0x7f43b015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.398 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 >> 172.21.15.119:0/3245025850 conn(0x7f43b00cf970 msgr2=0x7f43b00b5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:10.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 shutdown_connections 2023-11-08T19:32:10.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.394+0000 7f43b62fc640 1 -- 172.21.15.119:0/3245025850 wait complete. 2023-11-08T19:32:10.414 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T19:32:10.619 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.618+0000 7f759007c640 1 Processor -- start 2023-11-08T19:32:10.620 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.618+0000 7f759007c640 1 -- start start 2023-11-08T19:32:10.621 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f759007c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f758806d0a0 0x7f75880700c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f759007c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7588070710 con 0x7f75880cbc10 2023-11-08T19:32:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f759007c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75880709f0 con 0x7f758815d100 2023-11-08T19:32:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f759007c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7588070fb0 con 0x7f758806d0a0 2023-11-08T19:32:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f758e619640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f758806d0a0 0x7f75880700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.622 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f758e619640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f758806d0a0 0x7f75880700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56714/0 (socket says 172.21.15.119:56714) 2023-11-08T19:32:10.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f758e619640 1 -- 172.21.15.119:0/1429507090 learned_addr learned my addr 172.21.15.119:0/1429507090 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:10.623 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.620+0000 7f758d617640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f75880cbc10 0x7f758806f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42450/0 (socket says 172.21.15.119:42450) 2023-11-08T19:32:10.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.621+0000 7f758e619640 1 -- 172.21.15.119:0/1429507090 >> v1:172.21.15.188:6789/0 conn(0x7f758815d100 legacy=0x7f75880cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:10.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.621+0000 7f758e619640 1 -- 172.21.15.119:0/1429507090 >> v1:172.21.15.119:6789/0 conn(0x7f75880cbc10 legacy=0x7f758806f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.625 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.621+0000 7f758e619640 1 -- 172.21.15.119:0/1429507090 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588070cd0 con 0x7f758806d0a0 2023-11-08T19:32:10.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.621+0000 7f758ce16640 1 -- 172.21.15.119:0/1429507090 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476804251 0 0) 0x7f7588070710 con 0x7f75880cbc10 2023-11-08T19:32:10.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.622+0000 7f758e619640 1 --2- 172.21.15.119:0/1429507090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f75880700c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7580009e50 tx=0x7f758002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ab08149296401c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:10.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.622+0000 7f758ce16640 1 -- 172.21.15.119:0/1429507090 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758002ca80 con 0x7f758806d0a0 2023-11-08T19:32:10.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.622+0000 7f758ce16640 1 -- 172.21.15.119:0/1429507090 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758002cbe0 con 0x7f758806d0a0 2023-11-08T19:32:10.626 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.622+0000 7f758ce16640 1 -- 172.21.15.119:0/1429507090 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75800343f0 con 0x7f758806d0a0 2023-11-08T19:32:10.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 -- 172.21.15.119:0/1429507090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 msgr2=0x7f75880700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 --2- 172.21.15.119:0/1429507090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f75880700c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f7580009e50 tx=0x7f758002bee0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 -- 172.21.15.119:0/1429507090 shutdown_connections 2023-11-08T19:32:10.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 --2- 172.21.15.119:0/1429507090 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f75880700c0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.627 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 -- 172.21.15.119:0/1429507090 >> 172.21.15.119:0/1429507090 conn(0x7f75880b06e0 msgr2=0x7f75880b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:10.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 -- 172.21.15.119:0/1429507090 shutdown_connections 2023-11-08T19:32:10.628 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.623+0000 7f759007c640 1 -- 172.21.15.119:0/1429507090 wait complete. 2023-11-08T19:32:10.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.624+0000 7f759007c640 1 Processor -- start 2023-11-08T19:32:10.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 -- start start 2023-11-08T19:32:10.630 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f7588158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75880cbc10 0x7f758814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f758815d100 0x7f7588154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758814eb10 con 0x7f758806d0a0 2023-11-08T19:32:10.631 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75880cf6c0 con 0x7f758815d100 2023-11-08T19:32:10.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.625+0000 7f759007c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7588158880 con 0x7f75880cbc10 2023-11-08T19:32:10.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f7588158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f7588158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56722/0 (socket says 172.21.15.119:56722) 2023-11-08T19:32:10.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 -- 172.21.15.119:0/113211900 learned_addr learned my addr 172.21.15.119:0/113211900 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:10.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 -- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75880cbc10 msgr2=0x7f758814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.632 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758d617640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75880cbc10 0x7f758814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758e619640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f758815d100 0x7f7588154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75880cbc10 0x7f758814e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 -- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f758815d100 msgr2=0x7f7588154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f758815d100 0x7f7588154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.626+0000 7f758de18640 1 -- 172.21.15.119:0/113211900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7580009b30 con 0x7f758806d0a0 2023-11-08T19:32:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.627+0000 7f758de18640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f7588158210 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f758400b3f0 tx=0x7f758400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.627+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584011830 con 0x7f758806d0a0 2023-11-08T19:32:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.627+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7584011990 con 0x7f758806d0a0 2023-11-08T19:32:10.634 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.627+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584011d40 con 0x7f758806d0a0 2023-11-08T19:32:10.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.627+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758814daf0 con 0x7f758806d0a0 2023-11-08T19:32:10.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.627+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758814e0b0 con 0x7f758806d0a0 2023-11-08T19:32:10.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.628+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758814e2a0 con 0x7f758806d0a0 2023-11-08T19:32:10.635 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.629+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f758401a400 con 0x7f758806d0a0 2023-11-08T19:32:10.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.630+0000 7f757a7fc640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756c06ae70 0x7f756c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:10.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.630+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f7584089c70 con 0x7f758806d0a0 2023-11-08T19:32:10.636 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.630+0000 7f758d617640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756c06ae70 0x7f756c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:10.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.631+0000 7f758d617640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756c06ae70 0x7f756c06d330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f757c007fd0 tx=0x7f757c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:10.637 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.634+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7584054500 con 0x7f758806d0a0 2023-11-08T19:32:10.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.815+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f758804e490 con 0x7f758806d0a0 2023-11-08T19:32:10.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.816+0000 7f757a7fc640 1 -- 172.21.15.119:0/113211900 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75840150a0 con 0x7f758806d0a0 2023-11-08T19:32:10.818 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 32 2023-11-08T19:32:10.820 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.818+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756c06ae70 msgr2=0x7f756c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.818+0000 7f759007c640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756c06ae70 0x7f756c06d330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f757c007fd0 tx=0x7f757c009040 comp rx=0 tx=0).stop 2023-11-08T19:32:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.818+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 msgr2=0x7f7588158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.818+0000 7f759007c640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f7588158210 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f758400b3f0 tx=0x7f758400b8b0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 shutdown_connections 2023-11-08T19:32:10.821 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f756c06ae70 0x7f756c06d330 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f758815d100 0x7f7588154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75880cbc10 0x7f758814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 --2- 172.21.15.119:0/113211900 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f758806d0a0 0x7f7588158210 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:10.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 >> 172.21.15.119:0/113211900 conn(0x7f75880b06e0 msgr2=0x7f75880aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 shutdown_connections 2023-11-08T19:32:10.823 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:10.819+0000 7f759007c640 1 -- 172.21.15.119:0/113211900 wait complete. 2023-11-08T19:32:10.838 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T19:32:11.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.045+0000 7feacec44640 1 Processor -- start 2023-11-08T19:32:11.047 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.045+0000 7feacec44640 1 -- start start 2023-11-08T19:32:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.046+0000 7feacec44640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7feac815d070 0x7feac80b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.046+0000 7feacec44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 0x7feac80cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.046+0000 7feacec44640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac8161b60 con 0x7feac80ce070 2023-11-08T19:32:11.048 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.046+0000 7feacec44640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac80ce500 con 0x7feac815d070 2023-11-08T19:32:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.046+0000 7feacec44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac80d1500 con 0x7feac8160bc0 2023-11-08T19:32:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.047+0000 7feacd1e1640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7feac80ce070 0x7feac806c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42458/0 (socket says 172.21.15.119:42458) 2023-11-08T19:32:11.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.047+0000 7feacd1e1640 1 -- 172.21.15.119:0/590901806 learned_addr learned my addr 172.21.15.119:0/590901806 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:11.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.047+0000 7feacc9e0640 1 --2- 172.21.15.119:0/590901806 >> v2:172.21.15.188:3300/0 conn(0x7feac815d070 0x7feac80b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.047+0000 7feabffff640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 0x7feac80cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.047+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081728838 0 0) 0x7feac8161b60 con 0x7feac80ce070 2023-11-08T19:32:11.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.047+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feab0003520 con 0x7feac80ce070 2023-11-08T19:32:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 177987168 0 0) 0x7feab0003520 con 0x7feac80ce070 2023-11-08T19:32:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 msgr2=0x7feac80cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 0x7feac80cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 >> v2:172.21.15.188:3300/0 conn(0x7feac815d070 msgr2=0x7feac80b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.053 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 --2- 172.21.15.119:0/590901806 >> v2:172.21.15.188:3300/0 conn(0x7feac815d070 0x7feac80b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac8161040 con 0x7feac80ce070 2023-11-08T19:32:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7feac00019f0 con 0x7feac80ce070 2023-11-08T19:32:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 >> v1:172.21.15.119:6789/0 conn(0x7feac80ce070 legacy=0x7feac806c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab0003970 0x7feab0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feab00047c0 0x7feab0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 0x7feab0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80ce500 con 0x7feab00047c0 2023-11-08T19:32:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac80d1500 con 0x7feab0003970 2023-11-08T19:32:11.055 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feab000b980 con 0x7feac8160bc0 2023-11-08T19:32:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.048+0000 7feabffff640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feab00047c0 0x7feab0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feacc9e0640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab0003970 0x7feab0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feabffff640 1 -- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 msgr2=0x7feab0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feabffff640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac8160bc0 0x7feab0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feabffff640 1 -- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab0003970 msgr2=0x7feab0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feabffff640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab0003970 0x7feab0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feabffff640 1 -- 172.21.15.119:0/590901806 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab0003520 con 0x7feab00047c0 2023-11-08T19:32:11.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.049+0000 7feacc9e0640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab0003970 0x7feab0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:11.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feabffff640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feab00047c0 0x7feab0004c40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7feab8009ac0 tx=0x7feab00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2783e26668fa90f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab802c6d0 con 0x7feab00047c0 2023-11-08T19:32:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab802c830 con 0x7feab00047c0 2023-11-08T19:32:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feabf7fe640 1 -- 172.21.15.119:0/590901806 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab802cbc0 con 0x7feab00047c0 2023-11-08T19:32:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feacec44640 1 -- 172.21.15.119:0/590901806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feab00047c0 msgr2=0x7feab0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feacec44640 1 --2- 172.21.15.119:0/590901806 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feab00047c0 0x7feab0004c40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7feab8009ac0 tx=0x7feab00057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.058 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.050+0000 7feacd1e1640 1 -- 172.21.15.119:0/590901806 reap_dead start 2023-11-08T19:32:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.051+0000 7feacec44640 1 -- 172.21.15.119:0/590901806 shutdown_connections 2023-11-08T19:32:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.051+0000 7feacec44640 1 -- 172.21.15.119:0/590901806 >> 172.21.15.119:0/590901806 conn(0x7feac80cf970 msgr2=0x7feac80cfd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.051+0000 7feacec44640 1 -- 172.21.15.119:0/590901806 shutdown_connections 2023-11-08T19:32:11.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.051+0000 7feacec44640 1 -- 172.21.15.119:0/590901806 wait complete. 2023-11-08T19:32:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.052+0000 7feacec44640 1 Processor -- start 2023-11-08T19:32:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.053+0000 7feacec44640 1 -- start start 2023-11-08T19:32:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.053+0000 7feacec44640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00047c0 0x7feac80c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.053+0000 7feacec44640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 0x7feac80b7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.053+0000 7feacec44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac81500d0 0x7feac8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feacec44640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac8150a90 con 0x7feac80cccd0 2023-11-08T19:32:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feacec44640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac81f64e0 con 0x7feab00047c0 2023-11-08T19:32:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feacec44640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac81f67c0 con 0x7feac81500d0 2023-11-08T19:32:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 0x7feac80b7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 0x7feac80b7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56750/0 (socket says 172.21.15.119:56750) 2023-11-08T19:32:11.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 -- 172.21.15.119:0/4240475094 learned_addr learned my addr 172.21.15.119:0/4240475094 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:11.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 -- 172.21.15.119:0/4240475094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac81500d0 msgr2=0x7feac8150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac81500d0 0x7feac8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 -- 172.21.15.119:0/4240475094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00047c0 msgr2=0x7feac80c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00047c0 0x7feac80c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 -- 172.21.15.119:0/4240475094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac81f6aa0 con 0x7feac80cccd0 2023-11-08T19:32:11.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.054+0000 7feabffff640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 0x7feac80b7900 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feab8000c00 tx=0x7feab8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.055+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8003810 con 0x7feac80cccd0 2023-11-08T19:32:11.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.055+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab8003990 con 0x7feac80cccd0 2023-11-08T19:32:11.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.055+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac81f6d80 con 0x7feac80cccd0 2023-11-08T19:32:11.065 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.055+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8003d40 con 0x7feac80cccd0 2023-11-08T19:32:11.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.055+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac81f7340 con 0x7feac80cccd0 2023-11-08T19:32:11.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.056+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac804e490 con 0x7feac80cccd0 2023-11-08T19:32:11.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.057+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feab8040450 con 0x7feac80cccd0 2023-11-08T19:32:11.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.058+0000 7feabd7fa640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feab406aea0 0x7feab406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.066 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.058+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7feab80abc20 con 0x7feac80cccd0 2023-11-08T19:32:11.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.059+0000 7feacc9e0640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feab406aea0 0x7feab406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.059+0000 7feacc9e0640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feab406aea0 0x7feab406d360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7feac80b8370 tx=0x7feac4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.063+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feab80764c0 con 0x7feac80cccd0 2023-11-08T19:32:11.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.253+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feac80d0ba0 con 0x7feac80cccd0 2023-11-08T19:32:11.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.254+0000 7feabd7fa640 1 -- 172.21.15.119:0/4240475094 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7feab8079ae0 con 0x7feac80cccd0 2023-11-08T19:32:11.255 INFO:teuthology.orchestra.run.smithi119.stdout:pg_num: 64 2023-11-08T19:32:11.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feab406aea0 msgr2=0x7feab406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feab406aea0 0x7feab406d360 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7feac80b8370 tx=0x7feac4008040 comp rx=0 tx=0).stop 2023-11-08T19:32:11.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 msgr2=0x7feac80b7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.258 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 0x7feac80b7900 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7feab8000c00 tx=0x7feab8004730 comp rx=0 tx=0).stop 2023-11-08T19:32:11.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 shutdown_connections 2023-11-08T19:32:11.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7feab406aea0 0x7feab406d360 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feac81500d0 0x7feac8150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7feac80cccd0 0x7feac80b7900 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.259 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 --2- 172.21.15.119:0/4240475094 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00047c0 0x7feac80c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 >> 172.21.15.119:0/4240475094 conn(0x7feac80cf970 msgr2=0x7feac80d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:11.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 shutdown_connections 2023-11-08T19:32:11.260 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.256+0000 7feacec44640 1 -- 172.21.15.119:0/4240475094 wait complete. 2023-11-08T19:32:11.277 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:32:11.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.482+0000 7f8fc7cc2640 1 Processor -- start 2023-11-08T19:32:11.484 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.483+0000 7f8fc7cc2640 1 -- start start 2023-11-08T19:32:11.485 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.484+0000 7f8fc7cc2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 0x7f8fc00cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.484+0000 7f8fc7cc2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00cc6c0 con 0x7f8fc01555b0 2023-11-08T19:32:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.484+0000 7f8fc7cc2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc0071770 con 0x7f8fc00cb4e0 2023-11-08T19:32:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.484+0000 7f8fc7cc2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc0071a50 con 0x7f8fc00ce040 2023-11-08T19:32:11.486 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.484+0000 7f8fc5a5e640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8fc01555b0 0x7f8fc0070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42460/0 (socket says 172.21.15.119:42460) 2023-11-08T19:32:11.487 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.484+0000 7f8fc5a5e640 1 -- 172.21.15.119:0/2921014555 learned_addr learned my addr 172.21.15.119:0/2921014555 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:11.488 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201931359 0 0) 0x7f8fc00cc6c0 con 0x7f8fc01555b0 2023-11-08T19:32:11.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc525d640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 0x7f8fc00cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa8003520 con 0x7f8fc01555b0 2023-11-08T19:32:11.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 434760885 0 0) 0x7f8fa8003520 con 0x7f8fc01555b0 2023-11-08T19:32:11.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 msgr2=0x7f8fc00cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 0x7f8fc00cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 >> v1:172.21.15.188:6789/0 conn(0x7f8fc00cb4e0 legacy=0x7f8fc015b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00cc910 con 0x7f8fc01555b0 2023-11-08T19:32:11.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f8fb00019f0 con 0x7f8fc01555b0 2023-11-08T19:32:11.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.485+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 >> v1:172.21.15.119:6789/0 conn(0x7f8fc01555b0 legacy=0x7f8fc0070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.490 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc4a5c640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 0x7f8fa80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc4a5c640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8003fd0 0x7f8fa80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc4a5c640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa8004d00 0x7f8fa8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc0071a50 con 0x7f8fa8003fd0 2023-11-08T19:32:11.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa800bbc0 con 0x7f8fa8004d00 2023-11-08T19:32:11.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa800bd80 con 0x7f8fc00ce040 2023-11-08T19:32:11.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc525d640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8003fd0 0x7f8fa80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc525d640 1 -- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 msgr2=0x7f8fa80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc525d640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fc00ce040 0x7f8fa80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.492 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc525d640 1 -- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa8004d00 msgr2=0x7f8fa8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc525d640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fa8004d00 0x7f8fa8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.486+0000 7f8fc525d640 1 -- 172.21.15.119:0/2921014555 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa8003520 con 0x7f8fa8003fd0 2023-11-08T19:32:11.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.487+0000 7f8fc525d640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8003fd0 0x7f8fa80047c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400ba30 tx=0x7f8fa80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=683e2e9cf3d5838e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4032680 con 0x7f8fa8003fd0 2023-11-08T19:32:11.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb40327e0 con 0x7f8fa8003fd0 2023-11-08T19:32:11.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc4a5c640 1 -- 172.21.15.119:0/2921014555 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4032b70 con 0x7f8fa8003fd0 2023-11-08T19:32:11.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/2921014555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8003fd0 msgr2=0x7f8fa80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/2921014555 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fa8003fd0 0x7f8fa80047c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400ba30 tx=0x7f8fa80057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc625f640 1 -- 172.21.15.119:0/2921014555 reap_dead start 2023-11-08T19:32:11.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/2921014555 shutdown_connections 2023-11-08T19:32:11.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/2921014555 >> 172.21.15.119:0/2921014555 conn(0x7f8fc00b06e0 msgr2=0x7f8fc00b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:11.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/2921014555 shutdown_connections 2023-11-08T19:32:11.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.488+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/2921014555 wait complete. 2023-11-08T19:32:11.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.490+0000 7f8fc7cc2640 1 Processor -- start 2023-11-08T19:32:11.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.490+0000 7f8fc7cc2640 1 -- start start 2023-11-08T19:32:11.498 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc7cc2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa8003fd0 0x7f8fc00c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc7cc2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 0x7f8fc0150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc7cc2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00b2250 0x7f8fc00ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc7cc2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b1ff0 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc7cc2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc0151130 con 0x7f8fc00b2250 2023-11-08T19:32:11.499 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc7cc2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00b43f0 con 0x7f8fa8003fd0 2023-11-08T19:32:11.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc525d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 0x7f8fc0150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc525d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 0x7f8fc0150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56764/0 (socket says 172.21.15.119:56764) 2023-11-08T19:32:11.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.491+0000 7f8fc525d640 1 -- 172.21.15.119:0/1503970076 learned_addr learned my addr 172.21.15.119:0/1503970076 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:11.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc525d640 1 -- 172.21.15.119:0/1503970076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa8003fd0 msgr2=0x7f8fc00c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.500 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc525d640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa8003fd0 0x7f8fc00c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc525d640 1 -- 172.21.15.119:0/1503970076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00b2250 msgr2=0x7f8fc00ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc525d640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00b2250 0x7f8fc00ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc525d640 1 -- 172.21.15.119:0/1503970076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00b1930 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc525d640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 0x7f8fc0150bf0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8fb402bf00 tx=0x7f8fb4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.501 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4030aa0 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4030c20 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb403b420 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.492+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc00b1c10 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.493+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00b28b0 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.494+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc00b1dd0 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.503 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.495+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fb403b580 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.495+0000 7f8fbe7fc640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8fac06aea0 0x7f8fac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.496+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f8fb40ac950 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.496+0000 7f8fc5a5e640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8fac06aea0 0x7f8fac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.504 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.496+0000 7f8fc5a5e640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8fac06aea0 0x7f8fac06d360 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0001350 tx=0x7f8fb0001910 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.505 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.500+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fb407ad00 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.727+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc004e490 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.728+0000 7f8fbe7fc640 1 -- 172.21.15.119:0/1503970076 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f8fb4036070 con 0x7f8fc00b3dc0 2023-11-08T19:32:11.730 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:11.730 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:32:11.731 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:32:11.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8fac06aea0 msgr2=0x7f8fac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8fac06aea0 0x7f8fac06d360 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0001350 tx=0x7f8fb0001910 comp rx=0 tx=0).stop 2023-11-08T19:32:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 msgr2=0x7f8fc0150bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 0x7f8fc0150bf0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f8fb402bf00 tx=0x7f8fb4004730 comp rx=0 tx=0).stop 2023-11-08T19:32:11.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 shutdown_connections 2023-11-08T19:32:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8fac06aea0 0x7f8fac06d360 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8fc00b2250 0x7f8fc00ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8fc00b3dc0 0x7f8fc0150bf0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 --2- 172.21.15.119:0/1503970076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8fa8003fd0 0x7f8fc00c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 >> 172.21.15.119:0/1503970076 conn(0x7f8fc00b06e0 msgr2=0x7f8fc006fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 shutdown_connections 2023-11-08T19:32:11.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.731+0000 7f8fc7cc2640 1 -- 172.21.15.119:0/1503970076 wait complete. 2023-11-08T19:32:11.752 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:32:11.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.964+0000 7fab3807c640 1 Processor -- start 2023-11-08T19:32:11.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.964+0000 7fab3807c640 1 -- start start 2023-11-08T19:32:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab3807c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab301555b0 0x7fab300ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab3807c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab300cb4e0 0x7fab300cb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab3807c640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab300cc6c0 con 0x7fab300cd110 2023-11-08T19:32:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab3807c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab300cd590 con 0x7fab301555b0 2023-11-08T19:32:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab3807c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300cbeb0 con 0x7fab300cb4e0 2023-11-08T19:32:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab35e18640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fab301555b0 0x7fab300ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.968 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab35617640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fab300cd110 0x7fab30070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42464/0 (socket says 172.21.15.119:42464) 2023-11-08T19:32:11.969 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab35617640 1 -- 172.21.15.119:0/753456074 learned_addr learned my addr 172.21.15.119:0/753456074 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.966+0000 7fab36619640 1 --2- 172.21.15.119:0/753456074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab300cb4e0 0x7fab300cb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.967+0000 7fab35e18640 1 -- 172.21.15.119:0/753456074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab300cb4e0 msgr2=0x7fab300cb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.967+0000 7fab35e18640 1 --2- 172.21.15.119:0/753456074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab300cb4e0 0x7fab300cb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.967+0000 7fab35e18640 1 -- 172.21.15.119:0/753456074 >> v1:172.21.15.119:6789/0 conn(0x7fab300cd110 legacy=0x7fab30070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.967+0000 7fab35e18640 1 -- 172.21.15.119:0/753456074 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab30071770 con 0x7fab301555b0 2023-11-08T19:32:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.967+0000 7fab34e16640 1 -- 172.21.15.119:0/753456074 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184058355 0 0) 0x7fab300cc6c0 con 0x7fab300cd110 2023-11-08T19:32:11.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.968+0000 7fab35e18640 1 --2- 172.21.15.119:0/753456074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab301555b0 0x7fab300ce040 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0097a0 tx=0x7fab2c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e7e47bbcf5d7e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.968+0000 7fab34e16640 1 -- 172.21.15.119:0/753456074 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c031780 con 0x7fab301555b0 2023-11-08T19:32:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.968+0000 7fab34e16640 1 -- 172.21.15.119:0/753456074 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab2c0318e0 con 0x7fab301555b0 2023-11-08T19:32:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.968+0000 7fab34e16640 1 -- 172.21.15.119:0/753456074 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2c031c70 con 0x7fab301555b0 2023-11-08T19:32:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 -- 172.21.15.119:0/753456074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab301555b0 msgr2=0x7fab300ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 --2- 172.21.15.119:0/753456074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab301555b0 0x7fab300ce040 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fab2c0097a0 tx=0x7fab2c02b820 comp rx=0 tx=0).stop 2023-11-08T19:32:11.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 -- 172.21.15.119:0/753456074 shutdown_connections 2023-11-08T19:32:11.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 --2- 172.21.15.119:0/753456074 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab300cb4e0 0x7fab300cb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 --2- 172.21.15.119:0/753456074 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab301555b0 0x7fab300ce040 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 -- 172.21.15.119:0/753456074 >> 172.21.15.119:0/753456074 conn(0x7fab300b06e0 msgr2=0x7fab300b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:11.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 -- 172.21.15.119:0/753456074 shutdown_connections 2023-11-08T19:32:11.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.969+0000 7fab3807c640 1 -- 172.21.15.119:0/753456074 wait complete. 2023-11-08T19:32:11.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.971+0000 7fab3807c640 1 Processor -- start 2023-11-08T19:32:11.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.971+0000 7fab3807c640 1 -- start start 2023-11-08T19:32:11.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab3807c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab300cb4e0 0x7fab300c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab3807c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 0x7fab300c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab3807c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab301555b0 0x7fab301f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab3807c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab30151730 con 0x7fab300cd110 2023-11-08T19:32:11.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab3807c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300ca340 con 0x7fab300cb4e0 2023-11-08T19:32:11.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab3807c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab30156100 con 0x7fab301555b0 2023-11-08T19:32:11.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab35617640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 0x7fab300c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab35617640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 0x7fab300c7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56786/0 (socket says 172.21.15.119:56786) 2023-11-08T19:32:11.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab35617640 1 -- 172.21.15.119:0/3712412048 learned_addr learned my addr 172.21.15.119:0/3712412048 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:11.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab35617640 1 -- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab301555b0 msgr2=0x7fab301f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab35e18640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab300cb4e0 0x7fab300c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.972+0000 7fab36619640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab301555b0 0x7fab301f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab35617640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab301555b0 0x7fab301f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab35617640 1 -- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab300cb4e0 msgr2=0x7fab300c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:11.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab35617640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab300cb4e0 0x7fab300c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:11.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab35617640 1 -- 172.21.15.119:0/3712412048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c009400 con 0x7fab300cd110 2023-11-08T19:32:11.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab35e18640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab300cb4e0 0x7fab300c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:11.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab36619640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab301555b0 0x7fab301f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:11.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab35617640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 0x7fab300c7600 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c4b0 tx=0x7fab2000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab200074e0 con 0x7fab300cd110 2023-11-08T19:32:11.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab20007660 con 0x7fab300cd110 2023-11-08T19:32:11.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab30155e20 con 0x7fab300cd110 2023-11-08T19:32:11.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20004a20 con 0x7fab300cd110 2023-11-08T19:32:11.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.973+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab300c9fd0 con 0x7fab300cd110 2023-11-08T19:32:11.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.974+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab300c9b50 con 0x7fab300cd110 2023-11-08T19:32:11.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.976+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab20004b80 con 0x7fab300cd110 2023-11-08T19:32:11.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.977+0000 7fab267fc640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fab1406aea0 0x7fab1406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:11.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.977+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7fab200893b0 con 0x7fab300cd110 2023-11-08T19:32:11.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.977+0000 7fab35e18640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fab1406aea0 0x7fab1406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:11.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.978+0000 7fab35e18640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fab1406aea0 0x7fab1406d360 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fab2c036000 tx=0x7fab2c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:11.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:11.981+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab300c9b50 con 0x7fab300cd110 2023-11-08T19:32:12.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.178+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab300c9b50 con 0x7fab300cd110 2023-11-08T19:32:12.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.182+0000 7fab267fc640 1 -- 172.21.15.119:0/3712412048 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23077 (secure 0 0 0) 0x7fab300c9b50 con 0x7fab300cd110 2023-11-08T19:32:12.184 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:12.185 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":106,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:31:58.747478+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1419527548":"2023-11-08T20:31:50.849149+0000","172.21.15.188:6834/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.188:6833/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.119:6837/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.119:6835/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.188:6835/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.188:6833/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.119:6837/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6832/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.119:6835/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6835/168553637":"2023-11-09T19:29:41.161236+0000","172.21.15.188:6835/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.188:6833/168553637":"2023-11-09T19:29:41.161236+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":8}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:32:12.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fab1406aea0 msgr2=0x7fab1406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fab1406aea0 0x7fab1406d360 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fab2c036000 tx=0x7fab2c03d040 comp rx=0 tx=0).stop 2023-11-08T19:32:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 msgr2=0x7fab300c7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 0x7fab300c7600 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fab2000c4b0 tx=0x7fab2000c970 comp rx=0 tx=0).stop 2023-11-08T19:32:12.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 shutdown_connections 2023-11-08T19:32:12.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fab1406aea0 0x7fab1406d360 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fab301555b0 0x7fab301f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fab300cd110 0x7fab300c7600 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 --2- 172.21.15.119:0/3712412048 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fab300cb4e0 0x7fab300c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.185+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 >> 172.21.15.119:0/3712412048 conn(0x7fab300b06e0 msgr2=0x7fab3015d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.186+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 shutdown_connections 2023-11-08T19:32:12.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.186+0000 7fab3807c640 1 -- 172.21.15.119:0/3712412048 wait complete. 2023-11-08T19:32:12.206 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:32:12.207 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:32:12.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.419+0000 7fe706f75640 1 Processor -- start 2023-11-08T19:32:12.420 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.419+0000 7fe706f75640 1 -- start start 2023-11-08T19:32:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.420+0000 7fe706f75640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe700155190 0x7fe7000caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe706f75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe7000c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe706f75640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000ae3c0 con 0x7fe70015b910 2023-11-08T19:32:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe706f75640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7000adb20 con 0x7fe700155190 2023-11-08T19:32:12.423 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe706f75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000ade00 con 0x7fe7000c9010 2023-11-08T19:32:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe705f73640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe700155190 0x7fe7000caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe705f73640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe700155190 0x7fe7000caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56792/0 (socket says 172.21.15.119:56792) 2023-11-08T19:32:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe706774640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe70015b910 0x7fe70006a030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42476/0 (socket says 172.21.15.119:42476) 2023-11-08T19:32:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe705f73640 1 -- 172.21.15.119:0/208605326 learned_addr learned my addr 172.21.15.119:0/208605326 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:12.424 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.421+0000 7fe705772640 1 --2- 172.21.15.119:0/208605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe7000c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.422+0000 7fe705f73640 1 -- 172.21.15.119:0/208605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 msgr2=0x7fe7000c9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:12.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.422+0000 7fe705f73640 1 --2- 172.21.15.119:0/208605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe7000c9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.422+0000 7fe705f73640 1 -- 172.21.15.119:0/208605326 >> v1:172.21.15.119:6789/0 conn(0x7fe70015b910 legacy=0x7fe70006a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.425 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.422+0000 7fe705f73640 1 -- 172.21.15.119:0/208605326 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000ae0e0 con 0x7fe700155190 2023-11-08T19:32:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.423+0000 7fe705f73640 1 --2- 172.21.15.119:0/208605326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700155190 0x7fe7000caa90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009870 tx=0x7fe6fc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30216b60839c2010 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.423+0000 7fe704f71640 1 -- 172.21.15.119:0/208605326 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc031640 con 0x7fe700155190 2023-11-08T19:32:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.423+0000 7fe704f71640 1 -- 172.21.15.119:0/208605326 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc0317a0 con 0x7fe700155190 2023-11-08T19:32:12.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.423+0000 7fe704f71640 1 -- 172.21.15.119:0/208605326 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc031b30 con 0x7fe700155190 2023-11-08T19:32:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.423+0000 7fe706f75640 1 -- 172.21.15.119:0/208605326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700155190 msgr2=0x7fe7000caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 --2- 172.21.15.119:0/208605326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700155190 0x7fe7000caa90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009870 tx=0x7fe6fc02b8f0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 -- 172.21.15.119:0/208605326 shutdown_connections 2023-11-08T19:32:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 --2- 172.21.15.119:0/208605326 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe7000c9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 --2- 172.21.15.119:0/208605326 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe700155190 0x7fe7000caa90 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.428 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 -- 172.21.15.119:0/208605326 >> 172.21.15.119:0/208605326 conn(0x7fe7000b1020 msgr2=0x7fe7000ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 -- 172.21.15.119:0/208605326 shutdown_connections 2023-11-08T19:32:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.424+0000 7fe706f75640 1 -- 172.21.15.119:0/208605326 wait complete. 2023-11-08T19:32:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.425+0000 7fe706f75640 1 Processor -- start 2023-11-08T19:32:12.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.426+0000 7fe706f75640 1 -- start start 2023-11-08T19:32:12.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.426+0000 7fe706f75640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe70014b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.426+0000 7fe706f75640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe700155190 0x7fe7000b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.426+0000 7fe706f75640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 0x7fe700157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706f75640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000b22f0 con 0x7fe70015b910 2023-11-08T19:32:12.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706f75640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7001575b0 con 0x7fe700155190 2023-11-08T19:32:12.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706f75640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe700156a30 con 0x7fe7000c9010 2023-11-08T19:32:12.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 0x7fe700157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 0x7fe700157ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56796/0 (socket says 172.21.15.119:56796) 2023-11-08T19:32:12.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 -- 172.21.15.119:0/3671590858 learned_addr learned my addr 172.21.15.119:0/3671590858 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:12.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe705772640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe700155190 0x7fe7000b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 -- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 msgr2=0x7fe70014b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe70014b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 -- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe700155190 msgr2=0x7fe7000b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe700155190 0x7fe7000b1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.427+0000 7fe706774640 1 -- 172.21.15.119:0/3671590858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc0094d0 con 0x7fe70015b910 2023-11-08T19:32:12.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.428+0000 7fe706774640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 0x7fe700157ec0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c780 tx=0x7fe6f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:12.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.428+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800ce90 con 0x7fe70015b910 2023-11-08T19:32:12.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.428+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f800a640 con 0x7fe70015b910 2023-11-08T19:32:12.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.428+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800a9f0 con 0x7fe70015b910 2023-11-08T19:32:12.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.428+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe700156d10 con 0x7fe70015b910 2023-11-08T19:32:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.428+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7001572d0 con 0x7fe70015b910 2023-11-08T19:32:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.429+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe70015bdf0 con 0x7fe70015b910 2023-11-08T19:32:12.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.430+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6f800ab50 con 0x7fe70015b910 2023-11-08T19:32:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.431+0000 7fe6f67fc640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6e406aea0 0x7fe6e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.431+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7fe6f80894a0 con 0x7fe70015b910 2023-11-08T19:32:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.431+0000 7fe705f73640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6e406aea0 0x7fe6e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.432+0000 7fe705f73640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6e406aea0 0x7fe6e406d360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc036000 tx=0x7fe6fc030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:12.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.435+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe70015bdf0 con 0x7fe70015b910 2023-11-08T19:32:12.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.661+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe70015bdf0 con 0x7fe70015b910 2023-11-08T19:32:12.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.662+0000 7fe6f67fc640 1 -- 172.21.15.119:0/3671590858 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7fe70015bdf0 con 0x7fe70015b910 2023-11-08T19:32:12.664 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:12.664 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:32:12.665 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:32:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6e406aea0 msgr2=0x7fe6e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6e406aea0 0x7fe6e406d360 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc036000 tx=0x7fe6fc030000 comp rx=0 tx=0).stop 2023-11-08T19:32:12.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 msgr2=0x7fe700157ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 0x7fe700157ec0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c780 tx=0x7fe6f800cc40 comp rx=0 tx=0).stop 2023-11-08T19:32:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 shutdown_connections 2023-11-08T19:32:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe6e406aea0 0x7fe6e406d360 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe70015b910 0x7fe700157ec0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe700155190 0x7fe7000b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.667 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 --2- 172.21.15.119:0/3671590858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe7000c9010 0x7fe70014b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 >> 172.21.15.119:0/3671590858 conn(0x7fe7000b1020 msgr2=0x7fe7000ae850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 shutdown_connections 2023-11-08T19:32:12.668 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.665+0000 7fe706f75640 1 -- 172.21.15.119:0/3671590858 wait complete. 2023-11-08T19:32:12.684 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:32:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.895+0000 7f0ee2dfb640 1 Processor -- start 2023-11-08T19:32:12.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.895+0000 7f0ee2dfb640 1 -- start start 2023-11-08T19:32:12.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0ee2dfb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0edc1555b0 0x7f0edc0ce040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0ee2dfb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0edc0cd110 0x7f0edc070f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0ee2dfb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0cb930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0ee2dfb640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0edc0cd590 con 0x7f0edc0cd110 2023-11-08T19:32:12.898 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0ee2dfb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0edc0cbe70 con 0x7f0edc1555b0 2023-11-08T19:32:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0ee2dfb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0cc6c0 con 0x7f0edc0cb4e0 2023-11-08T19:32:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.896+0000 7f0edbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0edc0cd110 0x7f0edc070f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f0edc0cd110 0x7f0edc070f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56808/0 (socket says 172.21.15.119:56808) 2023-11-08T19:32:12.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 -- 172.21.15.119:0/2034253867 learned_addr learned my addr 172.21.15.119:0/2034253867 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0ee0b97640 1 --2- 172.21.15.119:0/2034253867 >> v2:172.21.15.188:3300/0 conn(0x7f0edc1555b0 0x7f0edc0ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 -- 172.21.15.119:0/2034253867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 msgr2=0x7f0edc0cb930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T19:32:12.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 --2- 172.21.15.119:0/2034253867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0cb930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 -- 172.21.15.119:0/2034253867 >> v2:172.21.15.188:3300/0 conn(0x7f0edc1555b0 msgr2=0x7f0edc0ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 --2- 172.21.15.119:0/2034253867 >> v2:172.21.15.188:3300/0 conn(0x7f0edc1555b0 0x7f0edc0ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0edbfff640 1 -- 172.21.15.119:0/2034253867 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0edc0714d0 con 0x7f0edc0cd110 2023-11-08T19:32:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.897+0000 7f0ee0b97640 1 --2- 172.21.15.119:0/2034253867 >> v2:172.21.15.188:3300/0 conn(0x7f0edc1555b0 0x7f0edc0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:12.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.898+0000 7f0edbfff640 1 --2- 172.21.15.119:0/2034253867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc070f90 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0cb4b0 tx=0x7f0ec802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b81066c399727cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.898+0000 7f0edb7fe640 1 -- 172.21.15.119:0/2034253867 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8031800 con 0x7f0edc0cd110 2023-11-08T19:32:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.898+0000 7f0edb7fe640 1 -- 172.21.15.119:0/2034253867 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec8031960 con 0x7f0edc0cd110 2023-11-08T19:32:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.898+0000 7f0edb7fe640 1 -- 172.21.15.119:0/2034253867 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8031cf0 con 0x7f0edc0cd110 2023-11-08T19:32:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2034253867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 msgr2=0x7f0edc070f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2034253867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc070f90 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0cb4b0 tx=0x7f0ec802b8a0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2034253867 shutdown_connections 2023-11-08T19:32:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2034253867 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2034253867 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc070f90 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2034253867 >> v2:172.21.15.188:3300/0 conn(0x7f0edc1555b0 0x7f0edc0ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2034253867 >> 172.21.15.119:0/2034253867 conn(0x7f0edc0b06e0 msgr2=0x7f0edc0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2034253867 shutdown_connections 2023-11-08T19:32:12.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.899+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2034253867 wait complete. 2023-11-08T19:32:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.901+0000 7f0ee2dfb640 1 Processor -- start 2023-11-08T19:32:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.901+0000 7f0ee2dfb640 1 -- start start 2023-11-08T19:32:12.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0ee2dfb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0ee2dfb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc1f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0ee2dfb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0edc1555b0 0x7f0edc1f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0ee2dfb640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0b4170 con 0x7f0edc0cd110 2023-11-08T19:32:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0ee2dfb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc1f6850 con 0x7f0edc1555b0 2023-11-08T19:32:12.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0ee2dfb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0edc0b3840 con 0x7f0edc0cb4e0 2023-11-08T19:32:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc1f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0edbfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc1f5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56822/0 (socket says 172.21.15.119:56822) 2023-11-08T19:32:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.902+0000 7f0edbfff640 1 -- 172.21.15.119:0/2768898262 learned_addr learned my addr 172.21.15.119:0/2768898262 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0ee0b97640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0edbfff640 1 -- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 msgr2=0x7f0edc0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0edbfff640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0edbfff640 1 -- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0edc1555b0 msgr2=0x7f0edc1f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0edbfff640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0edc1555b0 0x7f0edc1f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0edbfff640 1 -- 172.21.15.119:0/2768898262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec80094d0 con 0x7f0edc0cd110 2023-11-08T19:32:12.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0edbfff640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc1f5ae0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec803ad30 con 0x7f0edc0cd110 2023-11-08T19:32:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ec803ae90 con 0x7f0edc0cd110 2023-11-08T19:32:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ec8044610 con 0x7f0edc0cd110 2023-11-08T19:32:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.903+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0edc0b3b20 con 0x7f0edc0cd110 2023-11-08T19:32:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.904+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0edc1f6560 con 0x7f0edc0cd110 2023-11-08T19:32:12.911 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.905+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0edc04e490 con 0x7f0edc0cd110 2023-11-08T19:32:12.912 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.905+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ec804d450 con 0x7f0edc0cd110 2023-11-08T19:32:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.906+0000 7f0ed97fa640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0ec006ae70 0x7f0ec006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.907+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f0ec802e070 con 0x7f0edc0cd110 2023-11-08T19:32:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.907+0000 7f0ee0b97640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0ec006ae70 0x7f0ec006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.907+0000 7f0ee0b97640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0ec006ae70 0x7f0ec006d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000a990 tx=0x7f0ed0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:12.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:12.911+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ec8076ba0 con 0x7f0edc0cd110 2023-11-08T19:32:13.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.144+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0edc0738e0 con 0x7f0edc0cd110 2023-11-08T19:32:13.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.145+0000 7f0ed97fa640 1 -- 172.21.15.119:0/2768898262 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4943 (secure 0 0 0) 0x7f0ec8035020 con 0x7f0edc0cd110 2023-11-08T19:32:13.147 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:13.147 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":101,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2023-11-08T19:28:52.484445+0000","modified":"2023-11-08T19:31:51.913004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":7746,"mds_1":5821},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5821":{"gid":5821,"name":"b","rank":1,"incarnation":97,"state":"up:active","state_seq":7,"addr":"172.21.15.188:6833/865908207","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":865908207},{"type":"v1","addr":"172.21.15.188:6833","nonce":865908207}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_7746":{"gid":7746,"name":"a","rank":0,"incarnation":94,"state":"up:active","state_seq":15,"addr":"172.21.15.119:6837/2823709152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":2823709152},{"type":"v1","addr":"172.21.15.119:6837","nonce":2823709152}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T19:32:13.147 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 101 2023-11-08T19:32:13.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0ec006ae70 msgr2=0x7f0ec006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0ec006ae70 0x7f0ec006d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0ed000a990 tx=0x7f0ed0008040 comp rx=0 tx=0).stop 2023-11-08T19:32:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 msgr2=0x7f0edc1f5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc1f5ae0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f0ec8000c00 tx=0x7f0ec80046c0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 shutdown_connections 2023-11-08T19:32:13.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f0ec006ae70 0x7f0ec006d330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0edc1555b0 0x7f0edc1f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0edc0cd110 0x7f0edc1f5ae0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 --2- 172.21.15.119:0/2768898262 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0edc0cb4e0 0x7f0edc0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 >> 172.21.15.119:0/2768898262 conn(0x7f0edc0b06e0 msgr2=0x7f0edc15d750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:13.151 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 shutdown_connections 2023-11-08T19:32:13.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.148+0000 7f0ee2dfb640 1 -- 172.21.15.119:0/2768898262 wait complete. 2023-11-08T19:32:13.167 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:32:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.374+0000 7f8f70834640 1 Processor -- start 2023-11-08T19:32:13.376 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.375+0000 7f8f70834640 1 -- start start 2023-11-08T19:32:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f70834640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f6c0703a0 0x7f8f6c15e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.377 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f70834640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c15dd60 con 0x7f8f6c159d80 2023-11-08T19:32:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f70834640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f6c15da30 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f70834640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6c0ca8b0 con 0x7f8f6c0703a0 2023-11-08T19:32:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f69d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f8f6c0703a0 0x7f8f6c15e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f6a59c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f8f6c159d80 0x7f8f6c0ca080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42478/0 (socket says 172.21.15.119:42478) 2023-11-08T19:32:13.378 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.376+0000 7f8f6a59c640 1 -- 172.21.15.119:0/3534937920 learned_addr learned my addr 172.21.15.119:0/3534937920 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.377+0000 7f8f69d9b640 1 -- 172.21.15.119:0/3534937920 >> v1:172.21.15.188:6789/0 conn(0x7f8f6c0cb2e0 legacy=0x7f8f6c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.377+0000 7f8f69d9b640 1 -- 172.21.15.119:0/3534937920 >> v1:172.21.15.119:6789/0 conn(0x7f8f6c159d80 legacy=0x7f8f6c0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.377+0000 7f8f69d9b640 1 -- 172.21.15.119:0/3534937920 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f6c0cab90 con 0x7f8f6c0703a0 2023-11-08T19:32:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.377+0000 7f8f6959a640 1 -- 172.21.15.119:0/3534937920 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021408551 0 0) 0x7f8f6c15dd60 con 0x7f8f6c159d80 2023-11-08T19:32:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.378+0000 7f8f69d9b640 1 --2- 172.21.15.119:0/3534937920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0703a0 0x7f8f6c15e1f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009d90 tx=0x7f8f5c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6990661a67fec7b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.378+0000 7f8f6959a640 1 -- 172.21.15.119:0/3534937920 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c02cc60 con 0x7f8f6c0703a0 2023-11-08T19:32:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.378+0000 7f8f6959a640 1 -- 172.21.15.119:0/3534937920 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c02cdc0 con 0x7f8f6c0703a0 2023-11-08T19:32:13.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.378+0000 7f8f6959a640 1 -- 172.21.15.119:0/3534937920 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c034660 con 0x7f8f6c0703a0 2023-11-08T19:32:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 -- 172.21.15.119:0/3534937920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0703a0 msgr2=0x7f8f6c15e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 --2- 172.21.15.119:0/3534937920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0703a0 0x7f8f6c15e1f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c009d90 tx=0x7f8f5c02beb0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 -- 172.21.15.119:0/3534937920 shutdown_connections 2023-11-08T19:32:13.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 --2- 172.21.15.119:0/3534937920 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0703a0 0x7f8f6c15e1f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 -- 172.21.15.119:0/3534937920 >> 172.21.15.119:0/3534937920 conn(0x7f8f6c0b06e0 msgr2=0x7f8f6c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:13.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 -- 172.21.15.119:0/3534937920 shutdown_connections 2023-11-08T19:32:13.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.379+0000 7f8f70834640 1 -- 172.21.15.119:0/3534937920 wait complete. 2023-11-08T19:32:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.380+0000 7f8f70834640 1 Processor -- start 2023-11-08T19:32:13.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 -- start start 2023-11-08T19:32:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c0703a0 0x7f8f6c0ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 0x7f8f6c0c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c159d80 0x7f8f6c0ac960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0adc70 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c153310 con 0x7f8f6c159d80 2023-11-08T19:32:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.381+0000 7f8f70834640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f6c0acff0 con 0x7f8f6c0703a0 2023-11-08T19:32:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 0x7f8f6c0c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 0x7f8f6c0c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56846/0 (socket says 172.21.15.119:56846) 2023-11-08T19:32:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 -- 172.21.15.119:0/1008952127 learned_addr learned my addr 172.21.15.119:0/1008952127 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:13.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f6a59c640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c0703a0 0x7f8f6c0ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 -- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c0703a0 msgr2=0x7f8f6c0ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c0703a0 0x7f8f6c0ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 -- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c159d80 msgr2=0x7f8f6c0ac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f6ad9d640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c159d80 0x7f8f6c0ac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c159d80 0x7f8f6c0ac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f69d9b640 1 -- 172.21.15.119:0/1008952127 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c009a70 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.382+0000 7f8f6ad9d640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c159d80 0x7f8f6c0ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.383+0000 7f8f69d9b640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 0x7f8f6c0c6fd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c037040 tx=0x7f8f5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:13.389 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.383+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c03e610 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.383+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c03e770 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.383+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c03eb20 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.383+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6c0ad2d0 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.383+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6c0ad890 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.390 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.384+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6c04e490 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.385+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f5c03b050 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.386+0000 7f8f62ffd640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f4c06ae70 0x7f8f4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.387+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f8f5c0ac8e0 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.387+0000 7f8f6a59c640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f4c06ae70 0x7f8f4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.387+0000 7f8f6a59c640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f4c06ae70 0x7f8f4c06d330 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8f54007fd0 tx=0x7f8f54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:13.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.391+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f5c077170 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.580+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f6c0c9950 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.582+0000 7f8f62ffd640 1 -- 172.21.15.119:0/1008952127 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+23077 (secure 0 0 0) 0x7f8f5c04e090 con 0x7f8f6c0cb2e0 2023-11-08T19:32:13.583 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:13.584 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":106,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:31:58.747478+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T19:28:50.669529+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T19:28:51.498158+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":9,"snap_epoch":105,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":2}]},{"pgid":"17.36","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:0/1419527548":"2023-11-08T20:31:50.849149+0000","172.21.15.188:6834/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.188:6833/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.119:6837/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.119:6835/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.188:6835/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.188:6833/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.119:6837/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6832/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.119:6835/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6835/168553637":"2023-11-09T19:29:41.161236+0000","172.21.15.188:6835/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.188:6833/168553637":"2023-11-09T19:29:41.161236+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":8}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:32:13.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.584+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f4c06ae70 msgr2=0x7f8f4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.584+0000 7f8f70834640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f4c06ae70 0x7f8f4c06d330 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8f54007fd0 tx=0x7f8f54009040 comp rx=0 tx=0).stop 2023-11-08T19:32:13.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.584+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 msgr2=0x7f8f6c0c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.584+0000 7f8f70834640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 0x7f8f6c0c6fd0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c037040 tx=0x7f8f5c004a40 comp rx=0 tx=0).stop 2023-11-08T19:32:13.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 shutdown_connections 2023-11-08T19:32:13.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f8f4c06ae70 0x7f8f4c06d330 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f6c159d80 0x7f8f6c0ac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f8f6c0cb2e0 0x7f8f6c0c6fd0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 --2- 172.21.15.119:0/1008952127 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f6c0703a0 0x7f8f6c0ae090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 >> 172.21.15.119:0/1008952127 conn(0x7f8f6c0b06e0 msgr2=0x7f8f6c0709d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:13.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 shutdown_connections 2023-11-08T19:32:13.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.585+0000 7f8f70834640 1 -- 172.21.15.119:0/1008952127 wait complete. 2023-11-08T19:32:13.606 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T19:32:13.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.820+0000 7f0826fc4640 1 Processor -- start 2023-11-08T19:32:13.822 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.821+0000 7f0826fc4640 1 -- start start 2023-11-08T19:32:13.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.822+0000 7f0826fc4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f082015d000 0x7f08200ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0826fc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0820160b90 0x7f08200c6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0826fc4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0820161100 con 0x7f082006a030 2023-11-08T19:32:13.824 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0826fc4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08201613e0 con 0x7f082015d000 2023-11-08T19:32:13.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0826fc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08201616c0 con 0x7f0820160b90 2023-11-08T19:32:13.825 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0825fc2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f082015d000 0x7f08200ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0825fc2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f082015d000 0x7f08200ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.119:57700/0 (socket says 172.21.15.119:57700) 2023-11-08T19:32:13.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f0825fc2640 1 -- 172.21.15.119:0/3198413891 learned_addr learned my addr 172.21.15.119:0/3198413891 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:13.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.823+0000 7f08257c1640 1 --2- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0820160b90 0x7f08200c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.826 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.824+0000 7f0825fc2640 1 -- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0820160b90 msgr2=0x7f08200c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:13.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.824+0000 7f0825fc2640 1 --2- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0820160b90 0x7f08200c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.824+0000 7f0825fc2640 1 -- 172.21.15.119:0/3198413891 >> v1:172.21.15.188:6789/0 conn(0x7f082006a030 legacy=0x7f0820070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.827 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.824+0000 7f0825fc2640 1 -- 172.21.15.119:0/3198413891 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08201619a0 con 0x7f082015d000 2023-11-08T19:32:13.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.826+0000 7f0825fc2640 1 --2- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082015d000 0x7f08200ac8a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f081c009850 tx=0x7f081c02b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c80840200ccf0b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:13.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.826+0000 7f0824fc0640 1 -- 172.21.15.119:0/3198413891 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c004230 con 0x7f082015d000 2023-11-08T19:32:13.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.826+0000 7f0824fc0640 1 -- 172.21.15.119:0/3198413891 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c004390 con 0x7f082015d000 2023-11-08T19:32:13.828 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.826+0000 7f0824fc0640 1 -- 172.21.15.119:0/3198413891 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c031830 con 0x7f082015d000 2023-11-08T19:32:13.829 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.827+0000 7f0826fc4640 1 -- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082015d000 msgr2=0x7f08200ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.827+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082015d000 0x7f08200ac8a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f081c009850 tx=0x7f081c02b8d0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.827+0000 7f0826fc4640 1 -- 172.21.15.119:0/3198413891 shutdown_connections 2023-11-08T19:32:13.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.827+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0820160b90 0x7f08200c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.827+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3198413891 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082015d000 0x7f08200ac8a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.830 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.827+0000 7f0826fc4640 1 -- 172.21.15.119:0/3198413891 >> 172.21.15.119:0/3198413891 conn(0x7f08200cc540 msgr2=0x7f08200cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:13.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.828+0000 7f0826fc4640 1 -- 172.21.15.119:0/3198413891 shutdown_connections 2023-11-08T19:32:13.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.828+0000 7f0826fc4640 1 -- 172.21.15.119:0/3198413891 wait complete. 2023-11-08T19:32:13.831 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.830+0000 7f0826fc4640 1 Processor -- start 2023-11-08T19:32:13.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.830+0000 7f0826fc4640 1 -- start start 2023-11-08T19:32:13.832 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.831+0000 7f0826fc4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082006a030 0x7f0820154950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.831+0000 7f0826fc4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f082015d000 0x7f08200c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.831+0000 7f0826fc4640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 0x7f082006b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.831+0000 7f0826fc4640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082006d110 con 0x7f0820160b90 2023-11-08T19:32:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.831+0000 7f0826fc4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08201f2db0 con 0x7f082006a030 2023-11-08T19:32:13.833 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.831+0000 7f0826fc4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08201f3090 con 0x7f082015d000 2023-11-08T19:32:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 0x7f082006b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 0x7f082006b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56850/0 (socket says 172.21.15.119:56850) 2023-11-08T19:32:13.835 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 -- 172.21.15.119:0/3902554107 learned_addr learned my addr 172.21.15.119:0/3902554107 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08257c1640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f082015d000 0x7f08200c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f0825fc2640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082006a030 0x7f0820154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 -- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f082015d000 msgr2=0x7f08200c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f082015d000 0x7f08200c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 -- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082006a030 msgr2=0x7f0820154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082006a030 0x7f0820154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 -- 172.21.15.119:0/3902554107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c009460 con 0x7f0820160b90 2023-11-08T19:32:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f0825fc2640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082006a030 0x7f0820154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.832+0000 7f08267c3640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 0x7f082006b140 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f081800c6b0 tx=0x7f081800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:13.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.833+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818003b70 con 0x7f0820160b90 2023-11-08T19:32:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.833+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0818003cf0 con 0x7f0820160b90 2023-11-08T19:32:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.833+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08201f3370 con 0x7f0820160b90 2023-11-08T19:32:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.833+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0818003ea0 con 0x7f0820160b90 2023-11-08T19:32:13.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.833+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0820154e90 con 0x7f0820160b90 2023-11-08T19:32:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.834+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082006d280 con 0x7f0820160b90 2023-11-08T19:32:13.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.835+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0818005310 con 0x7f0820160b90 2023-11-08T19:32:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.836+0000 7f08167fc640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f080406aea0 0x7f080406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.837+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 7765+0+0 (secure 0 0 0) 0x7f0818089c60 con 0x7f0820160b90 2023-11-08T19:32:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.837+0000 7f0825fc2640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f080406aea0 0x7f080406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:13.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.837+0000 7f0825fc2640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f080406aea0 0x7f080406d360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f081c035000 tx=0x7f081c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:13.842 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:13.841+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0818054460 con 0x7f0820160b90 2023-11-08T19:32:14.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.069+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f0820067520 con 0x7f0820160b90 2023-11-08T19:32:14.071 INFO:tasks.ceph.mon.a.smithi119.stderr:2023-11-08T19:32:14.070+0000 7f8fd7f7c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T19:32:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.074+0000 7f08167fc640 1 -- 172.21.15.119:0/3902554107 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v102) v1 ==== 161+0+0 (secure 0 0 0) 0x7f0818057a80 con 0x7f0820160b90 2023-11-08T19:32:14.075 INFO:teuthology.orchestra.run.smithi119.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T19:32:14.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f080406aea0 msgr2=0x7f080406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:14.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f080406aea0 0x7f080406d360 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f081c035000 tx=0x7f081c008030 comp rx=0 tx=0).stop 2023-11-08T19:32:14.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 msgr2=0x7f082006b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:14.078 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 0x7f082006b140 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f081800c6b0 tx=0x7f081800cb70 comp rx=0 tx=0).stop 2023-11-08T19:32:14.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 shutdown_connections 2023-11-08T19:32:14.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f080406aea0 0x7f080406d360 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f0820160b90 0x7f082006b140 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f082015d000 0x7f08200c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 --2- 172.21.15.119:0/3902554107 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f082006a030 0x7f0820154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 >> 172.21.15.119:0/3902554107 conn(0x7f08200cc540 msgr2=0x7f0820072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:14.080 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 shutdown_connections 2023-11-08T19:32:14.081 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.077+0000 7f0826fc4640 1 -- 172.21.15.119:0/3902554107 wait complete. 2023-11-08T19:32:14.098 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T19:32:14.110 INFO:tasks.ceph.mds.a.smithi119.stderr: -517> 2023-11-08T19:31:56.238+0000 7f89f3cd5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T19:32:14.110 INFO:tasks.ceph.mds.a.smithi119.stderr: -515> 2023-11-08T19:31:56.238+0000 7f89f3cd5640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-11-08T19:32:14.162 INFO:tasks.ceph.mds.a.smithi119.stdout:starting mds.a at 2023-11-08T19:32:14.169 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2023-11-08T19:32:14.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.297+0000 7f2d98487640 1 Processor -- start 2023-11-08T19:32:14.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.297+0000 7f2d98487640 1 -- start start 2023-11-08T19:32:14.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d98487640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d90143aa0 0x7f2d900b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:14.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d98487640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d900b2000 0x7f2d9015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:14.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d98487640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d900b2510 con 0x7f2d9015ef10 2023-11-08T19:32:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d98487640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9015f3c0 con 0x7f2d900b2000 2023-11-08T19:32:14.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d98487640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900b27f0 con 0x7f2d90143aa0 2023-11-08T19:32:14.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d95a22640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f2d900b2000 0x7f2d9015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:14.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d96a24640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2d9015ef10 0x7f2d9006ba10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42494/0 (socket says 172.21.15.119:42494) 2023-11-08T19:32:14.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d96a24640 1 -- 172.21.15.119:0/1657958781 learned_addr learned my addr 172.21.15.119:0/1657958781 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:14.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.299+0000 7f2d96223640 1 --2- 172.21.15.119:0/1657958781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d90143aa0 0x7f2d900b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:14.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.300+0000 7f2d95a22640 1 -- 172.21.15.119:0/1657958781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d90143aa0 msgr2=0x7f2d900b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:14.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.300+0000 7f2d95a22640 1 --2- 172.21.15.119:0/1657958781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d90143aa0 0x7f2d900b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.300+0000 7f2d95a22640 1 -- 172.21.15.119:0/1657958781 >> v1:172.21.15.119:6789/0 conn(0x7f2d9015ef10 legacy=0x7f2d9006ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.300+0000 7f2d95a22640 1 -- 172.21.15.119:0/1657958781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900b2ad0 con 0x7f2d900b2000 2023-11-08T19:32:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.300+0000 7f2d95221640 1 -- 172.21.15.119:0/1657958781 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474756284 0 0) 0x7f2d900b2510 con 0x7f2d9015ef10 2023-11-08T19:32:14.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.301+0000 7f2d95a22640 1 --2- 172.21.15.119:0/1657958781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d900b2000 0x7f2d9015e9d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2d840097a0 tx=0x7f2d8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886b08c4b46f6dc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.301+0000 7f2d95221640 1 -- 172.21.15.119:0/1657958781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8402d040 con 0x7f2d900b2000 2023-11-08T19:32:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.301+0000 7f2d95221640 1 -- 172.21.15.119:0/1657958781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84031810 con 0x7f2d900b2000 2023-11-08T19:32:14.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.301+0000 7f2d95221640 1 -- 172.21.15.119:0/1657958781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84031ba0 con 0x7f2d900b2000 2023-11-08T19:32:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 -- 172.21.15.119:0/1657958781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d900b2000 msgr2=0x7f2d9015e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 --2- 172.21.15.119:0/1657958781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d900b2000 0x7f2d9015e9d0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f2d840097a0 tx=0x7f2d8402b8b0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 -- 172.21.15.119:0/1657958781 shutdown_connections 2023-11-08T19:32:14.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 --2- 172.21.15.119:0/1657958781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d900b2000 0x7f2d9015e9d0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 --2- 172.21.15.119:0/1657958781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d90143aa0 0x7f2d900b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 -- 172.21.15.119:0/1657958781 >> 172.21.15.119:0/1657958781 conn(0x7f2d90066530 msgr2=0x7f2d9006a060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:14.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 -- 172.21.15.119:0/1657958781 shutdown_connections 2023-11-08T19:32:14.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.302+0000 7f2d98487640 1 -- 172.21.15.119:0/1657958781 wait complete. 2023-11-08T19:32:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.304+0000 7f2d98487640 1 Processor -- start 2023-11-08T19:32:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.304+0000 7f2d98487640 1 -- start start 2023-11-08T19:32:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d98487640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d90143aa0 0x7f2d9015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d98487640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d9015ef10 0x7f2d90131ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:14.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d98487640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 0x7f2d900aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d98487640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9013f300 con 0x7f2d90132430 2023-11-08T19:32:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d98487640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900ab0c0 con 0x7f2d90143aa0 2023-11-08T19:32:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d98487640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900ab3a0 con 0x7f2d9015ef10 2023-11-08T19:32:14.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d96a24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 0x7f2d900aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d96a24640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 0x7f2d900aa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56892/0 (socket says 172.21.15.119:56892) 2023-11-08T19:32:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d96a24640 1 -- 172.21.15.119:0/2465277335 learned_addr learned my addr 172.21.15.119:0/2465277335 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.305+0000 7f2d95a22640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d9015ef10 0x7f2d90131ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96a24640 1 -- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d9015ef10 msgr2=0x7f2d90131ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:14.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96a24640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d9015ef10 0x7f2d90131ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96a24640 1 -- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d90143aa0 msgr2=0x7f2d9015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96223640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d90143aa0 0x7f2d9015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96a24640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d90143aa0 0x7f2d9015b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:14.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96a24640 1 -- 172.21.15.119:0/2465277335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009400 con 0x7f2d90132430 2023-11-08T19:32:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d96a24640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 0x7f2d900aa550 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c780 tx=0x7f2d8800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800ce90 con 0x7f2d90132430 2023-11-08T19:32:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8800a640 con 0x7f2d90132430 2023-11-08T19:32:14.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8800a9d0 con 0x7f2d90132430 2023-11-08T19:32:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d900ab680 con 0x7f2d90132430 2023-11-08T19:32:14.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.306+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d900abc40 con 0x7f2d90132430 2023-11-08T19:32:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.307+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9004e490 con 0x7f2d90132430 2023-11-08T19:32:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.308+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2d8800ab30 con 0x7f2d90132430 2023-11-08T19:32:14.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.309+0000 7f2d82ffd640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2d7406ae70 0x7f2d7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.309+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7909+0+0 (secure 0 0 0) 0x7f2d88089470 con 0x7f2d90132430 2023-11-08T19:32:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.310+0000 7f2d96223640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2d7406ae70 0x7f2d7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.310+0000 7f2d96223640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2d7406ae70 0x7f2d7406d330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006fd0 tx=0x7f2d8c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:14.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.314+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2d88053be0 con 0x7f2d90132430 2023-11-08T19:32:14.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:14.540+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2d900abe30 con 0x7f2d90132430 2023-11-08T19:32:15.079 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.077+0000 7f2d82ffd640 1 -- 172.21.15.119:0/2465277335 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v103) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2d88018020 con 0x7f2d90132430 2023-11-08T19:32:15.085 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2d7406ae70 msgr2=0x7f2d7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2d7406ae70 0x7f2d7406d330 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c006fd0 tx=0x7f2d8c008040 comp rx=0 tx=0).stop 2023-11-08T19:32:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 msgr2=0x7f2d900aa550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 0x7f2d900aa550 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800c780 tx=0x7f2d8800cc40 comp rx=0 tx=0).stop 2023-11-08T19:32:15.086 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 shutdown_connections 2023-11-08T19:32:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2d7406ae70 0x7f2d7406d330 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2d90132430 0x7f2d900aa550 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.081+0000 7f2d98487640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d9015ef10 0x7f2d90131ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.082+0000 7f2d98487640 1 --2- 172.21.15.119:0/2465277335 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d90143aa0 0x7f2d9015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.082+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 >> 172.21.15.119:0/2465277335 conn(0x7f2d900b2000 msgr2=0x7f2d9015d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.082+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 shutdown_connections 2023-11-08T19:32:15.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.082+0000 7f2d98487640 1 -- 172.21.15.119:0/2465277335 wait complete. 2023-11-08T19:32:15.108 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T19:32:15.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.378+0000 7fdc38ea9640 1 Processor -- start 2023-11-08T19:32:15.380 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.379+0000 7fdc38ea9640 1 -- start start 2023-11-08T19:32:15.381 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc38ea9640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc340ce040 0x7fdc340cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc38ea9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340cb4e0 0x7fdc340cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc38ea9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc340cc6c0 con 0x7fdc341555b0 2023-11-08T19:32:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc38ea9640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc340cbe90 con 0x7fdc340ce040 2023-11-08T19:32:15.382 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc38ea9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc34071770 con 0x7fdc340cb4e0 2023-11-08T19:32:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc31d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc340ce040 0x7fdc340cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc31d9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fdc340ce040 0x7fdc340cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56908/0 (socket says 172.21.15.119:56908) 2023-11-08T19:32:15.383 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.380+0000 7fdc31d9b640 1 -- 172.21.15.119:0/3560179315 learned_addr learned my addr 172.21.15.119:0/3560179315 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:15.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.381+0000 7fdc32d9d640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340cb4e0 0x7fdc340cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:15.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.381+0000 7fdc31d9b640 1 -- 172.21.15.119:0/3560179315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340cb4e0 msgr2=0x7fdc340cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:15.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.381+0000 7fdc31d9b640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340cb4e0 0x7fdc340cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.381+0000 7fdc31d9b640 1 -- 172.21.15.119:0/3560179315 >> v1:172.21.15.188:6789/0 conn(0x7fdc341555b0 legacy=0x7fdc34070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:15.385 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.381+0000 7fdc31d9b640 1 -- 172.21.15.119:0/3560179315 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc34071a50 con 0x7fdc340ce040 2023-11-08T19:32:15.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.381+0000 7fdc32d9d640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340cb4e0 0x7fdc340cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:15.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.382+0000 7fdc31d9b640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc340ce040 0x7fdc340cd110 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009870 tx=0x7fdc1c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d669e50e2ad85d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:15.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.382+0000 7fdc3159a640 1 -- 172.21.15.119:0/3560179315 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c0317c0 con 0x7fdc340ce040 2023-11-08T19:32:15.386 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.382+0000 7fdc3159a640 1 -- 172.21.15.119:0/3560179315 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1c031920 con 0x7fdc340ce040 2023-11-08T19:32:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.382+0000 7fdc3159a640 1 -- 172.21.15.119:0/3560179315 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c031cb0 con 0x7fdc340ce040 2023-11-08T19:32:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3560179315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc340ce040 msgr2=0x7fdc340cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc340ce040 0x7fdc340cd110 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009870 tx=0x7fdc1c02b860 comp rx=0 tx=0).stop 2023-11-08T19:32:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3560179315 shutdown_connections 2023-11-08T19:32:15.387 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc340ce040 0x7fdc340cd110 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3560179315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340cb4e0 0x7fdc340cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3560179315 >> 172.21.15.119:0/3560179315 conn(0x7fdc340b06e0 msgr2=0x7fdc340b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3560179315 shutdown_connections 2023-11-08T19:32:15.388 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.383+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3560179315 wait complete. 2023-11-08T19:32:15.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.385+0000 7fdc38ea9640 1 Processor -- start 2023-11-08T19:32:15.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.385+0000 7fdc38ea9640 1 -- start start 2023-11-08T19:32:15.391 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc38ea9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340cb4e0 0x7fdc340c6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc38ea9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340ce040 0x7fdc341eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc38ea9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 0x7fdc340af830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc38ea9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c5bc0 con 0x7fdc341555b0 2023-11-08T19:32:15.392 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc38ea9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c5e80 con 0x7fdc340cb4e0 2023-11-08T19:32:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc38ea9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc340c6160 con 0x7fdc340ce040 2023-11-08T19:32:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc32d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 0x7fdc340af830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc32d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 0x7fdc340af830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56914/0 (socket says 172.21.15.119:56914) 2023-11-08T19:32:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc32d9d640 1 -- 172.21.15.119:0/3749930098 learned_addr learned my addr 172.21.15.119:0/3749930098 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:15.393 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.386+0000 7fdc3259c640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340cb4e0 0x7fdc340c6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc31d9b640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340ce040 0x7fdc341eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc32d9d640 1 -- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340ce040 msgr2=0x7fdc341eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc32d9d640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340ce040 0x7fdc341eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.394 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc32d9d640 1 -- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340cb4e0 msgr2=0x7fdc340c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc32d9d640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340cb4e0 0x7fdc340c6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc32d9d640 1 -- 172.21.15.119:0/3749930098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0094d0 con 0x7fdc341555b0 2023-11-08T19:32:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc32d9d640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 0x7fdc340af830 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c7f0 tx=0x7fdc2c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:15.395 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c007820 con 0x7fdc341555b0 2023-11-08T19:32:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2c007980 con 0x7fdc341555b0 2023-11-08T19:32:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc340c6440 con 0x7fdc341555b0 2023-11-08T19:32:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.387+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c007d70 con 0x7fdc341555b0 2023-11-08T19:32:15.396 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.388+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc341f0440 con 0x7fdc341555b0 2023-11-08T19:32:15.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.388+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc340c6960 con 0x7fdc341555b0 2023-11-08T19:32:15.397 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.389+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc2c0113f0 con 0x7fdc341555b0 2023-11-08T19:32:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.390+0000 7fdc2affd640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc1406ae70 0x7fdc1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:15.399 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.390+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 7909+0+0 (secure 0 0 0) 0x7fdc2c08a330 con 0x7fdc341555b0 2023-11-08T19:32:15.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.391+0000 7fdc3259c640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc1406ae70 0x7fdc1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:15.400 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.391+0000 7fdc3259c640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc1406ae70 0x7fdc1406d330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009f20 tx=0x7fdc240094d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:15.401 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.395+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc340c6960 con 0x7fdc341555b0 2023-11-08T19:32:15.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:15.583+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdc340c6960 con 0x7fdc341555b0 2023-11-08T19:32:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.081+0000 7fdc2affd640 1 -- 172.21.15.119:0/3749930098 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7fdc340c6960 con 0x7fdc341555b0 2023-11-08T19:32:16.083 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_metadata' removed 2023-11-08T19:32:16.087 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc1406ae70 msgr2=0x7fdc1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc1406ae70 0x7fdc1406d330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdc24009f20 tx=0x7fdc240094d0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 msgr2=0x7fdc340af830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 0x7fdc340af830 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00c7f0 tx=0x7fdc2c00ccb0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.088 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 shutdown_connections 2023-11-08T19:32:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdc1406ae70 0x7fdc1406d330 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdc341555b0 0x7fdc340af830 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc340ce040 0x7fdc341eff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 --2- 172.21.15.119:0/3749930098 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc340cb4e0 0x7fdc340c6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.089 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.086+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 >> 172.21.15.119:0/3749930098 conn(0x7fdc340b06e0 msgr2=0x7fdc34152300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:16.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.087+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 shutdown_connections 2023-11-08T19:32:16.090 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.087+0000 7fdc38ea9640 1 -- 172.21.15.119:0/3749930098 wait complete. 2023-11-08T19:32:16.111 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T19:32:16.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.332+0000 7fadfffff640 1 Processor -- start 2023-11-08T19:32:16.333 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.332+0000 7fadfffff640 1 -- start start 2023-11-08T19:32:16.334 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.333+0000 7fadfffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 0x7fae0006fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.333+0000 7fadfffff640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0006d480 con 0x7fae0006ce80 2023-11-08T19:32:16.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.333+0000 7fadfffff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0006d760 con 0x7fae0015aba0 2023-11-08T19:32:16.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.333+0000 7fadfffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae000ce7a0 con 0x7fae0006f7e0 2023-11-08T19:32:16.335 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.333+0000 7fadf67fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fae0006ce80 0x7fae000cccd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42506/0 (socket says 172.21.15.119:42506) 2023-11-08T19:32:16.336 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.333+0000 7fadf67fc640 1 -- 172.21.15.119:0/1891505177 learned_addr learned my addr 172.21.15.119:0/1891505177 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:16.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 212160717 0 0) 0x7fae0006d480 con 0x7fae0006ce80 2023-11-08T19:32:16.339 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadec003520 con 0x7fae0006ce80 2023-11-08T19:32:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadff7fe640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 0x7fae0006fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858943193 0 0) 0x7fadec003520 con 0x7fae0006ce80 2023-11-08T19:32:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 msgr2=0x7fae0006fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 0x7fae0006fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.340 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 >> v1:172.21.15.188:6789/0 conn(0x7fae0015aba0 legacy=0x7fae000ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0006da40 con 0x7fae0006ce80 2023-11-08T19:32:16.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fade8001670 con 0x7fae0006ce80 2023-11-08T19:32:16.341 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 >> v1:172.21.15.119:6789/0 conn(0x7fae0006ce80 legacy=0x7fae000cccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 0x7fadec0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec003fd0 0x7fadec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.334+0000 7fadfe7fc640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fadec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000ce7a0 con 0x7fadec004d00 2023-11-08T19:32:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec00bb40 con 0x7fadec003fd0 2023-11-08T19:32:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec00bd00 con 0x7fae0006f7e0 2023-11-08T19:32:16.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadff7fe640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fadec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadf67fc640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec003fd0 0x7fadec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadfeffd640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 0x7fadec0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadff7fe640 1 -- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 msgr2=0x7fadec0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadff7fe640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae0006f7e0 0x7fadec0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.344 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadff7fe640 1 -- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec003fd0 msgr2=0x7fadec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadff7fe640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec003fd0 0x7fadec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadff7fe640 1 -- 172.21.15.119:0/1891505177 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec003520 con 0x7fadec004d00 2023-11-08T19:32:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.335+0000 7fadf67fc640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fadec003fd0 0x7fadec0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:16.345 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.336+0000 7fadff7fe640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fadec005180 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fadf8007360 tx=0x7fadec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c194481ad868f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:16.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf802ccd0 con 0x7fadec004d00 2023-11-08T19:32:16.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf802ce30 con 0x7fadec004d00 2023-11-08T19:32:16.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfe7fc640 1 -- 172.21.15.119:0/1891505177 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8034660 con 0x7fadec004d00 2023-11-08T19:32:16.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfffff640 1 -- 172.21.15.119:0/1891505177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 msgr2=0x7fadec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.346 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfffff640 1 --2- 172.21.15.119:0/1891505177 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fadec005180 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fadf8007360 tx=0x7fadec0057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadff7fe640 1 -- 172.21.15.119:0/1891505177 reap_dead start 2023-11-08T19:32:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfffff640 1 -- 172.21.15.119:0/1891505177 shutdown_connections 2023-11-08T19:32:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfffff640 1 -- 172.21.15.119:0/1891505177 >> 172.21.15.119:0/1891505177 conn(0x7fae000ce900 msgr2=0x7fae000cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfffff640 1 -- 172.21.15.119:0/1891505177 shutdown_connections 2023-11-08T19:32:16.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.337+0000 7fadfffff640 1 -- 172.21.15.119:0/1891505177 wait complete. 2023-11-08T19:32:16.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.339+0000 7fadfffff640 1 Processor -- start 2023-11-08T19:32:16.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.339+0000 7fadfffff640 1 -- start start 2023-11-08T19:32:16.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfffff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fae0015cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae00155120 0x7fae000c3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae000ae360 0x7fae000ae7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfffff640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000aed20 con 0x7fadec004d00 2023-11-08T19:32:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015d3c0 con 0x7fae000ae360 2023-11-08T19:32:16.351 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015d6a0 con 0x7fae00155120 2023-11-08T19:32:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fae0015cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfeffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fae0015cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56934/0 (socket says 172.21.15.119:56934) 2023-11-08T19:32:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.340+0000 7fadfeffd640 1 -- 172.21.15.119:0/2458020808 learned_addr learned my addr 172.21.15.119:0/2458020808 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadf67fc640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae00155120 0x7fae000c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.352 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadff7fe640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae000ae360 0x7fae000ae7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfeffd640 1 -- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae00155120 msgr2=0x7fae000c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfeffd640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae00155120 0x7fae000c3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfeffd640 1 -- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae000ae360 msgr2=0x7fae000ae7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfeffd640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae000ae360 0x7fae000ae7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:16.353 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfeffd640 1 -- 172.21.15.119:0/2458020808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0015d980 con 0x7fadec004d00 2023-11-08T19:32:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfeffd640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fae0015cd90 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d570 tx=0x7fadf000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0005480 con 0x7fadec004d00 2023-11-08T19:32:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0003610 con 0x7fadec004d00 2023-11-08T19:32:16.354 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0003980 con 0x7fadec004d00 2023-11-08T19:32:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae0015dc60 con 0x7fadec004d00 2023-11-08T19:32:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.341+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae000c3c40 con 0x7fadec004d00 2023-11-08T19:32:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.343+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0004ece0 con 0x7fadec004d00 2023-11-08T19:32:16.355 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.344+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadf0003ae0 con 0x7fadec004d00 2023-11-08T19:32:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.345+0000 7fadfcff9640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fade006ae70 0x7fade006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.345+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 7471+0+0 (secure 0 0 0) 0x7fadf0089c20 con 0x7fadec004d00 2023-11-08T19:32:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.345+0000 7fadf67fc640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fade006ae70 0x7fade006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.346+0000 7fadf67fc640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fade006ae70 0x7fade006d330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fade8001eb0 tx=0x7fade8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:16.356 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.349+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fadf00545f0 con 0x7fadec004d00 2023-11-08T19:32:16.542 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:16.541+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fae000cdb60 con 0x7fadec004d00 2023-11-08T19:32:17.095 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.093+0000 7fadfcff9640 1 -- 172.21.15.119:0/2458020808 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v109) v1 ==== 169+0+0 (secure 0 0 0) 0x7fadf0057c10 con 0x7fadec004d00 2023-11-08T19:32:17.095 INFO:teuthology.orchestra.run.smithi119.stderr:pool 'cephfs_data' removed 2023-11-08T19:32:17.100 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.098+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fade006ae70 msgr2=0x7fade006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.098+0000 7fadfffff640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fade006ae70 0x7fade006d330 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fade8001eb0 tx=0x7fade8009040 comp rx=0 tx=0).stop 2023-11-08T19:32:17.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.098+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 msgr2=0x7fae0015cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.098+0000 7fadfffff640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fae0015cd90 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d570 tx=0x7fadf000da30 comp rx=0 tx=0).stop 2023-11-08T19:32:17.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.099+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 shutdown_connections 2023-11-08T19:32:17.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.099+0000 7fadfffff640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fade006ae70 0x7fade006d330 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.099+0000 7fadfffff640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fae000ae360 0x7fae000ae7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.102 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.099+0000 7fadfffff640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fae00155120 0x7fae000c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.099+0000 7fadfffff640 1 --2- 172.21.15.119:0/2458020808 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fadec004d00 0x7fae0015cd90 secure :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fadf000d570 tx=0x7fadf000da30 comp rx=0 tx=0).stop 2023-11-08T19:32:17.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.099+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 >> 172.21.15.119:0/2458020808 conn(0x7fae000ce900 msgr2=0x7fae000c59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:17.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.100+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 shutdown_connections 2023-11-08T19:32:17.103 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.100+0000 7fadfffff640 1 -- 172.21.15.119:0/2458020808 wait complete. 2023-11-08T19:32:17.124 DEBUG:teuthology.orchestra.run.smithi119:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:32:17.134 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T19:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout:[global] 2023-11-08T19:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: chdir = "" 2023-11-08T19:32:17.286 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: auth supported = cephx 2023-11-08T19:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: filestore xattr use omap = true 2023-11-08T19:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: mon clock drift allowed = 1.000 2023-11-08T19:32:17.287 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: osd crush chooseleaf type = 0 2023-11-08T19:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: auth debug = true 2023-11-08T19:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on old message = true 2023-11-08T19:32:17.288 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on bug = true 2023-11-08T19:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: mon pg warn max object skew = 0 2023-11-08T19:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.289 INFO:teuthology.orchestra.run.smithi119.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default size = 2 2023-11-08T19:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.290 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow primary affinity = true 2023-11-08T19:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd allow pg remap = true 2023-11-08T19:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on no sortbitwise = false 2023-11-08T19:32:17.291 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: mon warn on too few osds = false 2023-11-08T19:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: mon_allow_pool_size_one = true 2023-11-08T19:32:17.292 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: osd default data pool replay window = 5 2023-11-08T19:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.293 INFO:teuthology.orchestra.run.smithi119.stdout: mon allow pool delete = true 2023-11-08T19:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: mon cluster log file level = debug 2023-11-08T19:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: debug asserts on shutdown = true 2023-11-08T19:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: mon health detail to clog = false 2023-11-08T19:32:17.294 INFO:teuthology.orchestra.run.smithi119.stdout: mon host = "172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-08T19:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: ms die on skipped message = False 2023-11-08T19:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: fsid = ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout:[osd] 2023-11-08T19:32:17.295 INFO:teuthology.orchestra.run.smithi119.stdout: osd journal size = 100 2023-11-08T19:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub load threshold = 5.0 2023-11-08T19:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: osd scrub max interval = 600 2023-11-08T19:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:32:17.296 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: osd recover clone overlap = true 2023-11-08T19:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: osd recovery max chunk = 1048576 2023-11-08T19:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug shutdown = true 2023-11-08T19:32:17.297 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug op order = true 2023-11-08T19:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug verify stray on activate = true 2023-11-08T19:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug trim objects = true 2023-11-08T19:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: osd open classes on start = true 2023-11-08T19:32:17.298 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug pg log writeout = true 2023-11-08T19:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: osd map max advance = 10 2023-11-08T19:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.299 INFO:teuthology.orchestra.run.smithi119.stdout: journal zero on create = true 2023-11-08T19:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: filestore apply finisher threads = 3 2023-11-08T19:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.300 INFO:teuthology.orchestra.run.smithi119.stdout: bdev debug aio = true 2023-11-08T19:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: osd debug misdirected ops = true 2023-11-08T19:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore block size = 96636764160 2023-11-08T19:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: bluestore fsck on mount = True 2023-11-08T19:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluefs = 20 2023-11-08T19:32:17.301 INFO:teuthology.orchestra.run.smithi119.stdout: debug bluestore = 20 2023-11-08T19:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: debug osd = 20 2023-11-08T19:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: debug rocksdb = 10 2023-11-08T19:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:32:17.302 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd full ratio = 0.9 2023-11-08T19:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd objectstore = bluestore 2023-11-08T19:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-11-08T19:32:17.303 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout:[mgr] 2023-11-08T19:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: debug mgr = 20 2023-11-08T19:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-11-08T19:32:17.304 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-11-08T19:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: mgr/telemetry/nag = false 2023-11-08T19:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.305 INFO:teuthology.orchestra.run.smithi119.stdout:[mon] 2023-11-08T19:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: debug mon = 20 2023-11-08T19:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: debug paxos = 20 2023-11-08T19:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: debug auth = 20 2023-11-08T19:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: mon data avail warn = 5 2023-11-08T19:32:17.306 INFO:teuthology.orchestra.run.smithi119.stdout: mon mgr mkfs grace = 240 2023-11-08T19:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd reporter subtree level = osd 2023-11-08T19:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: mon osd prime pg temp = true 2023-11-08T19:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:32:17.307 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.308 INFO:teuthology.orchestra.run.smithi119.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: # 1m isn't quite enough 2023-11-08T19:32:17.309 INFO:teuthology.orchestra.run.smithi119.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: mon op complaint time = 120 2023-11-08T19:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:17.310 INFO:teuthology.orchestra.run.smithi119.stdout:[client] 2023-11-08T19:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: rgw cache enabled = true 2023-11-08T19:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable ops log = true 2023-11-08T19:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: rgw enable usage log = true 2023-11-08T19:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:32:17.311 INFO:teuthology.orchestra.run.smithi119.stdout: client mount timeout = 600 2023-11-08T19:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: debug client = 20 2023-11-08T19:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: pid file = "" 2023-11-08T19:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-11-08T19:32:17.312 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-11-08T19:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.a] 2023-11-08T19:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.b] 2023-11-08T19:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout:[mon.c] 2023-11-08T19:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.a] 2023-11-08T19:32:17.313 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.c] 2023-11-08T19:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.b] 2023-11-08T19:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout:[mds.d] 2023-11-08T19:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout:[mds] 2023-11-08T19:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds = 20 2023-11-08T19:32:17.314 INFO:teuthology.orchestra.run.smithi119.stdout: debug mds balancer = 20 2023-11-08T19:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: debug ms = 1 2023-11-08T19:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug frag = True 2023-11-08T19:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: mds debug scatterstat = True 2023-11-08T19:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: mds op complaint time = 180 2023-11-08T19:32:17.315 INFO:teuthology.orchestra.run.smithi119.stdout: mds verify scatter = True 2023-11-08T19:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: osd op complaint time = 180 2023-11-08T19:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: rados mon op timeout = 900 2023-11-08T19:32:17.316 INFO:teuthology.orchestra.run.smithi119.stdout: rados osd op timeout = 900 2023-11-08T19:32:17.317 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2023-11-08T19:32:17.317 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2023-11-08T19:32:17.317 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T19:32:17.317 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2023-11-08T19:32:17.317 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.318 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2023-11-08T19:32:17.318 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.318 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2023-11-08T19:32:17.318 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.318 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2023-11-08T19:32:17.319 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2023-11-08T19:32:17.319 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.319 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2023-11-08T19:32:17.319 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2023-11-08T19:32:17.319 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.320 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T19:32:17.320 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2023-11-08T19:32:17.320 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.320 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T19:32:17.320 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T19:32:17.320 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.321 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2023-11-08T19:32:17.321 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.321 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2023-11-08T19:32:17.321 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2023-11-08T19:32:17.321 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2023-11-08T19:32:17.322 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2023-11-08T19:32:17.322 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2023-11-08T19:32:17.322 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2023-11-08T19:32:17.322 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2023-11-08T19:32:17.322 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T19:32:17.323 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T19:32:17.323 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2023-11-08T19:32:17.323 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.323 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T19:32:17.323 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.323 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2023-11-08T19:32:17.324 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.324 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2023-11-08T19:32:17.324 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.324 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2023-11-08T19:32:17.324 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2023-11-08T19:32:17.325 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2023-11-08T19:32:17.325 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.119,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2023-11-08T19:32:17.325 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2023-11-08T19:32:17.325 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d 2023-11-08T19:32:17.325 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.326 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2023-11-08T19:32:17.326 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2023-11-08T19:32:17.326 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.326 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2023-11-08T19:32:17.326 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2023-11-08T19:32:17.327 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2023-11-08T19:32:17.327 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.327 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2023-11-08T19:32:17.327 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2023-11-08T19:32:17.327 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.328 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2023-11-08T19:32:17.328 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2023-11-08T19:32:17.328 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2023-11-08T19:32:17.328 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2023-11-08T19:32:17.328 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.329 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2023-11-08T19:32:17.329 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2023-11-08T19:32:17.329 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.329 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2023-11-08T19:32:17.329 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.330 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2023-11-08T19:32:17.330 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.330 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2023-11-08T19:32:17.330 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.330 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2023-11-08T19:32:17.331 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2023-11-08T19:32:17.331 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.331 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2023-11-08T19:32:17.331 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2023-11-08T19:32:17.331 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2023-11-08T19:32:17.332 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2023-11-08T19:32:17.332 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2023-11-08T19:32:17.332 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2023-11-08T19:32:17.332 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:32:17.332 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2023-11-08T19:32:17.333 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2023-11-08T19:32:17.333 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T19:32:17.333 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2023-11-08T19:32:17.333 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2023-11-08T19:32:17.333 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2023-11-08T19:32:17.333 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2023-11-08T19:32:17.334 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-08T19:32:17.334 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.334 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2023-11-08T19:32:17.334 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:32:17.335 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2023-11-08T19:32:17.335 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-08T19:32:17.335 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-08T19:32:17.335 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:32:17.335 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:32:17.336 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2023-11-08T19:32:17.336 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.336 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2023-11-08T19:32:17.336 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:32:17.336 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2023-11-08T19:32:17.336 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2023-11-08T19:32:17.337 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2023-11-08T19:32:17.337 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2023-11-08T19:32:17.337 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2023-11-08T19:32:17.337 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2023-11-08T19:32:17.337 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2023-11-08T19:32:17.338 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2023-11-08T19:32:17.338 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2023-11-08T19:32:17.338 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.338 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T19:32:17.338 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T19:32:17.339 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T19:32:17.339 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.339 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T19:32:17.339 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T19:32:17.339 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T19:32:17.340 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.340 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2023-11-08T19:32:17.340 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2023-11-08T19:32:17.340 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.340 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2023-11-08T19:32:17.341 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2023-11-08T19:32:17.341 INFO:teuthology.orchestra.run.smithi188.stdout: 2023-11-08T19:32:17.341 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2023-11-08T19:32:17.341 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2023-11-08T19:32:17.341 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2023-11-08T19:32:17.341 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2023-11-08T19:32:17.342 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T19:32:17.342 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T19:32:17.342 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2023-11-08T19:32:17.342 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2023-11-08T19:32:17.342 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:32:17.343 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = "" 2023-11-08T19:32:17.343 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-08T19:32:17.343 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-08T19:32:17.343 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2023-11-08T19:32:17.343 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2023-11-08T19:32:17.344 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2023-11-08T19:32:17.344 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2023-11-08T19:32:17.344 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2023-11-08T19:32:17.344 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2023-11-08T19:32:17.344 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2023-11-08T19:32:17.345 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2023-11-08T19:32:17.345 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2023-11-08T19:32:17.345 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2023-11-08T19:32:17.345 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2023-11-08T19:32:17.345 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2023-11-08T19:32:17.345 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2023-11-08T19:32:17.346 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2023-11-08T19:32:17.346 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2023-11-08T19:32:17.346 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2023-11-08T19:32:17.346 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2023-11-08T19:32:17.346 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2023-11-08T19:32:17.350 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T19:32:17.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.572+0000 7fc8a2bb6640 1 Processor -- start 2023-11-08T19:32:17.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.572+0000 7fc8a2bb6640 1 -- start start 2023-11-08T19:32:17.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.574+0000 7fc8a2bb6640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc89c144550 0x7fc89c0cb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:17.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.574+0000 7fc8a2bb6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc89c15bce0 0x7fc89c0c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:17.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.574+0000 7fc8a2bb6640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0cbd20 con 0x7fc89c0cb8c0 2023-11-08T19:32:17.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.574+0000 7fc8a2bb6640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c15c150 con 0x7fc89c144550 2023-11-08T19:32:17.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.574+0000 7fc8a2bb6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c0c43b0 con 0x7fc89c15bce0 2023-11-08T19:32:17.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc89c144550 0x7fc89c0cb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:17.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc89c144550 0x7fc89c0cb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56940/0 (socket says 172.21.15.119:56940) 2023-11-08T19:32:17.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 -- 172.21.15.119:0/4161990938 learned_addr learned my addr 172.21.15.119:0/4161990938 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:17.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 -- 172.21.15.119:0/4161990938 >> v2:172.21.15.188:3300/0 conn(0x7fc89c15bce0 msgr2=0x7fc89c0c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:17.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 --2- 172.21.15.119:0/4161990938 >> v2:172.21.15.188:3300/0 conn(0x7fc89c15bce0 0x7fc89c0c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 -- 172.21.15.119:0/4161990938 >> v1:172.21.15.188:6789/0 conn(0x7fc89c0cb8c0 legacy=0x7fc89c15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:17.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.575+0000 7fc8a0952640 1 -- 172.21.15.119:0/4161990938 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c15c340 con 0x7fc89c144550 2023-11-08T19:32:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.577+0000 7fc8a0952640 1 --2- 172.21.15.119:0/4161990938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c144550 0x7fc89c0cb380 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc89c06f900 tx=0x7fc88c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6851e08c8d94815d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.577+0000 7fc89b7fe640 1 -- 172.21.15.119:0/4161990938 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c02ca80 con 0x7fc89c144550 2023-11-08T19:32:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.577+0000 7fc89b7fe640 1 -- 172.21.15.119:0/4161990938 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c02cbe0 con 0x7fc89c144550 2023-11-08T19:32:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.577+0000 7fc89b7fe640 1 -- 172.21.15.119:0/4161990938 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c0343f0 con 0x7fc89c144550 2023-11-08T19:32:17.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/4161990938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c144550 msgr2=0x7fc89c0cb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/4161990938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c144550 0x7fc89c0cb380 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fc89c06f900 tx=0x7fc88c02bee0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/4161990938 shutdown_connections 2023-11-08T19:32:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/4161990938 >> v2:172.21.15.188:3300/0 conn(0x7fc89c15bce0 0x7fc89c0c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/4161990938 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c144550 0x7fc89c0cb380 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/4161990938 >> 172.21.15.119:0/4161990938 conn(0x7fc89c0b0950 msgr2=0x7fc89c0b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/4161990938 shutdown_connections 2023-11-08T19:32:17.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.578+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/4161990938 wait complete. 2023-11-08T19:32:17.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.580+0000 7fc8a2bb6640 1 Processor -- start 2023-11-08T19:32:17.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.580+0000 7fc8a2bb6640 1 -- start start 2023-11-08T19:32:17.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.581+0000 7fc8a2bb6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c0cb8c0 0x7fc89c13a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:17.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a2bb6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c144550 0x7fc89c0ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:17.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a2bb6640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 0x7fc89c13ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:17.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a2bb6640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c13a7f0 con 0x7fc89c15bce0 2023-11-08T19:32:17.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a2bb6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c13bf00 con 0x7fc89c144550 2023-11-08T19:32:17.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a2bb6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c13c1e0 con 0x7fc89c0cb8c0 2023-11-08T19:32:17.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 0x7fc89c13ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:17.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 0x7fc89c13ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56942/0 (socket says 172.21.15.119:56942) 2023-11-08T19:32:17.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 -- 172.21.15.119:0/3415816291 learned_addr learned my addr 172.21.15.119:0/3415816291 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:17.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc89bfff640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c144550 0x7fc89c0ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:17.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a0952640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c0cb8c0 0x7fc89c13a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:17.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 -- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c0cb8c0 msgr2=0x7fc89c13a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c0cb8c0 0x7fc89c13a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.587 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 -- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c144550 msgr2=0x7fc89c0ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c144550 0x7fc89c0ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.582+0000 7fc8a1153640 1 -- 172.21.15.119:0/3415816291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c009b30 con 0x7fc89c15bce0 2023-11-08T19:32:17.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.583+0000 7fc8a1153640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 0x7fc89c13ccc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc89400b410 tx=0x7fc89400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:17.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.583+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894011830 con 0x7fc89c15bce0 2023-11-08T19:32:17.588 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.583+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8940119b0 con 0x7fc89c15bce0 2023-11-08T19:32:17.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.583+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894011d60 con 0x7fc89c15bce0 2023-11-08T19:32:17.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.583+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c13c4c0 con 0x7fc89c15bce0 2023-11-08T19:32:17.589 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.583+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0ca820 con 0x7fc89c15bce0 2023-11-08T19:32:17.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.584+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c13c9e0 con 0x7fc89c15bce0 2023-11-08T19:32:17.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.585+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc89401a400 con 0x7fc89c15bce0 2023-11-08T19:32:17.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.586+0000 7fc8997fa640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc87c06ae70 0x7fc87c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:17.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.586+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fc894089cd0 con 0x7fc89c15bce0 2023-11-08T19:32:17.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.586+0000 7fc8a0952640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc87c06ae70 0x7fc87c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:17.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.587+0000 7fc8a0952640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc87c06ae70 0x7fc87c06d330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc88c035000 tx=0x7fc88c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:17.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.590+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc89c13c9e0 con 0x7fc89c15bce0 2023-11-08T19:32:17.794 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.792+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc89c13c9e0 con 0x7fc89c15bce0 2023-11-08T19:32:17.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.798+0000 7fc8997fa640 1 -- 172.21.15.119:0/3415816291 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v20) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc89c13c9e0 con 0x7fc89c15bce0 2023-11-08T19:32:17.809 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc87c06ae70 msgr2=0x7fc87c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc87c06ae70 0x7fc87c06d330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fc88c035000 tx=0x7fc88c030000 comp rx=0 tx=0).stop 2023-11-08T19:32:17.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 msgr2=0x7fc89c13ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:17.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 0x7fc89c13ccc0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7fc89400b410 tx=0x7fc89400b8d0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 shutdown_connections 2023-11-08T19:32:17.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc87c06ae70 0x7fc87c06d330 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc89c15bce0 0x7fc89c13ccc0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc89c144550 0x7fc89c0ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 --2- 172.21.15.119:0/3415816291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc89c0cb8c0 0x7fc89c13a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:17.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 >> 172.21.15.119:0/3415816291 conn(0x7fc89c0b0950 msgr2=0x7fc89c0b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:17.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 shutdown_connections 2023-11-08T19:32:17.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:17.801+0000 7fc8a2bb6640 1 -- 172.21.15.119:0/3415816291 wait complete. 2023-11-08T19:32:17.824 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable' 2023-11-08T19:32:18.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.052+0000 7f2a47f00640 1 Processor -- start 2023-11-08T19:32:18.054 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.053+0000 7f2a47f00640 1 -- start start 2023-11-08T19:32:18.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.054+0000 7f2a47f00640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a4006a120 0x7f2a4006a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.054+0000 7f2a47f00640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a4006aa70 con 0x7f2a4015f4c0 2023-11-08T19:32:18.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.054+0000 7f2a47f00640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a400c3a50 con 0x7f2a401447a0 2023-11-08T19:32:18.056 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.054+0000 7f2a47f00640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a4015f930 con 0x7f2a4006a120 2023-11-08T19:32:18.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.055+0000 7f2a4549b640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f2a4015f4c0 0x7f2a4006f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:42520/0 (socket says 172.21.15.119:42520) 2023-11-08T19:32:18.057 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.055+0000 7f2a4549b640 1 -- 172.21.15.119:0/4164734993 learned_addr learned my addr 172.21.15.119:0/4164734993 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:18.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.055+0000 7f2a4649d640 1 --2- 172.21.15.119:0/4164734993 >> v2:172.21.15.188:3300/0 conn(0x7f2a4006a120 0x7f2a4006a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.055+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869927128 0 0) 0x7f2a4006aa70 con 0x7f2a4015f4c0 2023-11-08T19:32:18.059 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.055+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a2c003520 con 0x7f2a4015f4c0 2023-11-08T19:32:18.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.055+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3881375606 0 0) 0x7f2a2c003520 con 0x7f2a4015f4c0 2023-11-08T19:32:18.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 >> v2:172.21.15.188:3300/0 conn(0x7f2a4006a120 msgr2=0x7f2a4006a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:18.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 --2- 172.21.15.119:0/4164734993 >> v2:172.21.15.188:3300/0 conn(0x7f2a4006a120 0x7f2a4006a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 >> v1:172.21.15.188:6789/0 conn(0x7f2a401447a0 legacy=0x7f2a400cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:18.060 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a40144b80 con 0x7f2a4015f4c0 2023-11-08T19:32:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f2a30001690 con 0x7f2a4015f4c0 2023-11-08T19:32:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 >> v1:172.21.15.119:6789/0 conn(0x7f2a4015f4c0 legacy=0x7f2a4006f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0039c0 0x7f2a2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a2c0047c0 0x7f2a2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.061 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c005180 0x7f2a2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4015f930 con 0x7f2a2c0047c0 2023-11-08T19:32:18.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c00c0b0 con 0x7f2a2c005180 2023-11-08T19:32:18.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c00c270 con 0x7f2a2c0039c0 2023-11-08T19:32:18.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4549b640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a2c0047c0 0x7f2a2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.062 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4549b640 1 -- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0039c0 msgr2=0x7f2a2c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4649d640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c005180 0x7f2a2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4549b640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0039c0 0x7f2a2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4549b640 1 -- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c005180 msgr2=0x7f2a2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4549b640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a2c005180 0x7f2a2c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4649d640 1 -- 172.21.15.119:0/4164734993 reap_dead start 2023-11-08T19:32:18.063 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.056+0000 7f2a4549b640 1 -- 172.21.15.119:0/4164734993 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2c003520 con 0x7f2a2c0047c0 2023-11-08T19:32:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.057+0000 7f2a4549b640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a2c0047c0 0x7f2a2c004c40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c160 tx=0x7f2a2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef90fb0c7063befc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30005ad0 con 0x7f2a2c0047c0 2023-11-08T19:32:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a30005c30 con 0x7f2a2c0047c0 2023-11-08T19:32:18.064 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a44c9a640 1 -- 172.21.15.119:0/4164734993 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002c7c0 con 0x7f2a2c0047c0 2023-11-08T19:32:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 -- 172.21.15.119:0/4164734993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a2c0047c0 msgr2=0x7f2a2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a2c0047c0 0x7f2a2c004c40 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f2a3000c160 tx=0x7f2a2c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 -- 172.21.15.119:0/4164734993 shutdown_connections 2023-11-08T19:32:18.067 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 --2- 172.21.15.119:0/4164734993 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a2c0047c0 0x7f2a2c004c40 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 -- 172.21.15.119:0/4164734993 >> 172.21.15.119:0/4164734993 conn(0x7f2a400cb610 msgr2=0x7f2a400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 -- 172.21.15.119:0/4164734993 shutdown_connections 2023-11-08T19:32:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.058+0000 7f2a47f00640 1 -- 172.21.15.119:0/4164734993 wait complete. 2023-11-08T19:32:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.060+0000 7f2a47f00640 1 Processor -- start 2023-11-08T19:32:18.068 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.060+0000 7f2a47f00640 1 -- start start 2023-11-08T19:32:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a47f00640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0047c0 0x7f2a400b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a47f00640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 0x7f2a4013d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a47f00640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4013cd80 0x7f2a4013d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a47f00640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400b19e0 con 0x7f2a4013c920 2023-11-08T19:32:18.069 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a47f00640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a400b1fa0 con 0x7f2a4013cd80 2023-11-08T19:32:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a47f00640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401f0c80 con 0x7f2a2c0047c0 2023-11-08T19:32:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a4549b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 0x7f2a4013d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a4549b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 0x7f2a4013d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:56970/0 (socket says 172.21.15.119:56970) 2023-11-08T19:32:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.061+0000 7f2a4549b640 1 -- 172.21.15.119:0/328584969 learned_addr learned my addr 172.21.15.119:0/328584969 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:18.070 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a45c9c640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0047c0 0x7f2a400b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4649d640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4013cd80 0x7f2a4013d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4549b640 1 -- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0047c0 msgr2=0x7f2a400b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4549b640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0047c0 0x7f2a400b42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.071 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4549b640 1 -- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4013cd80 msgr2=0x7f2a4013d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4549b640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4013cd80 0x7f2a4013d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4549b640 1 -- 172.21.15.119:0/328584969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400c8950 con 0x7f2a4013c920 2023-11-08T19:32:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a4549b640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 0x7f2a4013d190 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002c360 tx=0x7f2a30001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.062+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3003c070 con 0x7f2a4013c920 2023-11-08T19:32:18.072 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.063+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a3002c9d0 con 0x7f2a4013c920 2023-11-08T19:32:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.063+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3002cd40 con 0x7f2a4013c920 2023-11-08T19:32:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.063+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a400c8c30 con 0x7f2a4013c920 2023-11-08T19:32:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.063+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4013dc10 con 0x7f2a4013c920 2023-11-08T19:32:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.064+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40074430 con 0x7f2a4013c920 2023-11-08T19:32:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.065+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a30032070 con 0x7f2a4013c920 2023-11-08T19:32:18.073 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.066+0000 7f2a367fc640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a2406ae70 0x7f2a2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.066+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f2a300aab90 con 0x7f2a4013c920 2023-11-08T19:32:18.074 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.066+0000 7f2a45c9c640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a2406ae70 0x7f2a2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.067+0000 7f2a45c9c640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a2406ae70 0x7f2a2406d330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0015b0 tx=0x7f2a3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:18.075 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.070+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a300757e0 con 0x7f2a4013c920 2023-11-08T19:32:18.337 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:18.336+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]} v 0) v1 -- 0x7f2a400c3ce0 con 0x7f2a4013c920 2023-11-08T19:32:19.101 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.099+0000 7f2a367fc640 1 -- 172.21.15.119:0/328584969 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snapshots.TestSnapshots.test_kill_mdstable"]}]=0 v406) v1 ==== 139+0+0 (secure 0 0 0) 0x7f2a30078e00 con 0x7f2a4013c920 2023-11-08T19:32:19.106 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.104+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a2406ae70 msgr2=0x7f2a2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.104+0000 7f2a47f00640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a2406ae70 0x7f2a2406d330 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0015b0 tx=0x7f2a3c009040 comp rx=0 tx=0).stop 2023-11-08T19:32:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.104+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 msgr2=0x7f2a4013d190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:19.107 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.104+0000 7f2a47f00640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 0x7f2a4013d190 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002c360 tx=0x7f2a30001950 comp rx=0 tx=0).stop 2023-11-08T19:32:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.104+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 shutdown_connections 2023-11-08T19:32:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f2a2406ae70 0x7f2a2406d330 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a4013cd80 0x7f2a4013d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:19.108 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f2a4013c920 0x7f2a4013d190 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:19.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 --2- 172.21.15.119:0/328584969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a2c0047c0 0x7f2a400b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:19.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 >> 172.21.15.119:0/328584969 conn(0x7f2a400cb610 msgr2=0x7f2a400cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:19.109 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 shutdown_connections 2023-11-08T19:32:19.110 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:19.105+0000 7f2a47f00640 1 -- 172.21.15.119:0/328584969 wait complete. 2023-11-08T19:32:19.129 INFO:tasks.cephfs_test_runner:check snaptable transcation ... ERROR 2023-11-08T19:32:19.130 INFO:tasks.cephfs_test_runner: 2023-11-08T19:32:19.131 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T19:32:19.132 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:32:19.132 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-08T19:32:19.133 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T19:32:19.134 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T19:32:19.135 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs/test_snapshots.py", line 149, in test_kill_mdstable 2023-11-08T19:32:19.135 INFO:tasks.cephfs_test_runner: proc.wait() 2023-11-08T19:32:19.136 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-08T19:32:19.136 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-08T19:32:19.137 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-08T19:32:19.138 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-08T19:32:19.139 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi119 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22)' 2023-11-08T19:32:19.139 INFO:tasks.cephfs_test_runner: 2023-11-08T19:32:19.140 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T19:32:19.141 INFO:tasks.cephfs_test_runner:Ran 5 tests in 383.256s 2023-11-08T19:32:19.142 INFO:tasks.cephfs_test_runner: 2023-11-08T19:32:19.142 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-08T19:32:19.143 INFO:tasks.cephfs_test_runner: 2023-11-08T19:32:19.144 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T19:32:19.145 INFO:tasks.cephfs_test_runner:ERROR: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:32:19.145 INFO:tasks.cephfs_test_runner:check snaptable transcation 2023-11-08T19:32:19.146 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T19:32:19.147 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T19:32:19.147 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs/test_snapshots.py", line 149, in test_kill_mdstable 2023-11-08T19:32:19.148 INFO:tasks.cephfs_test_runner: proc.wait() 2023-11-08T19:32:19.149 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-08T19:32:19.150 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-08T19:32:19.150 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-08T19:32:19.151 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-08T19:32:19.152 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi119 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec mkdir d1/dir/.snap/s22)' 2023-11-08T19:32:19.152 INFO:tasks.cephfs_test_runner: 2023-11-08T19:32:19.153 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:32:19.267 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=a9fc6231aec34493b58e938c3b633200 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_e797689ab88984658d73bbc928b2ed06c676d14a/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots) 2023-11-08T19:32:19.269 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-08T19:32:19.277 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-08T19:32:19.285 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-08T19:32:19.286 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:32:19.286 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:19.350 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:32:19.350 DEBUG:teuthology.orchestra.run.smithi119:> set -ex 2023-11-08T19:32:19.351 DEBUG:teuthology.orchestra.run.smithi119:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:19.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T19:32:19.406 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:32:19.406 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:19.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:32:19.467 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2023-11-08T19:32:19.468 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T19:32:19.522 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054289,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T19:32:19.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:19.523 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:32:19.584 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T19:32:19.584 INFO:teuthology.orchestra.run.smithi119.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T19:32:19.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:19.585 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T19:32:19.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:19.685 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T19:32:19.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:19.780 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-11-08T19:32:19.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:19.926 DEBUG:teuthology.orchestra.run.smithi119:> ip netns list 2023-11-08T19:32:19.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:19.977 DEBUG:teuthology.orchestra.run.smithi119:> sudo ip link delete ceph-brx 2023-11-08T19:32:20.060 INFO:teuthology.orchestra.run.smithi119.stderr:Cannot find device "ceph-brx" 2023-11-08T19:32:20.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:20.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:20.065 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-08T19:32:20.124 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T19:32:20.125 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-08T19:32:20.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:20.127 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T19:32:20.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:20.223 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T19:32:20.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:20.316 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-08T19:32:20.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:20.428 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2023-11-08T19:32:20.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T19:32:20.480 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2023-11-08T19:32:20.562 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2023-11-08T19:32:20.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:32:20.566 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-08T19:32:20.575 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T19:32:20.576 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:20.730 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:20.740 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.794+0000 7fa8abf5e640 1 Processor -- start 2023-11-08T19:32:20.796 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.795+0000 7fa8abf5e640 1 -- start start 2023-11-08T19:32:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8abf5e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8a41447a0 0x7fa8a415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.798 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8abf5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cad30 0x7fa8a406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8abf5e640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8a40cb170 0x7fa8a406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8abf5e640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a40c3a50 con 0x7fa8a40cb170 2023-11-08T19:32:20.799 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8abf5e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a4144b80 con 0x7fa8a41447a0 2023-11-08T19:32:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8abf5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40c9150 con 0x7fa8a40cad30 2023-11-08T19:32:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8aa4fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8a40cb170 0x7fa8a406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8aa4fb640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fa8a40cb170 0x7fa8a406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39968/0 (socket says 172.21.15.119:39968) 2023-11-08T19:32:20.800 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.796+0000 7fa8aa4fb640 1 -- 172.21.15.119:0/551616675 learned_addr learned my addr 172.21.15.119:0/551616675 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:20.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8a94f9640 1 --2- 172.21.15.119:0/551616675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cad30 0x7fa8a406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8a9cfa640 1 --2- 172.21.15.119:0/551616675 >> v2:172.21.15.188:3300/0 conn(0x7fa8a41447a0 0x7fa8a415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.801 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8aa4fb640 1 -- 172.21.15.119:0/551616675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cad30 msgr2=0x7fa8a406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:20.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8aa4fb640 1 --2- 172.21.15.119:0/551616675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cad30 0x7fa8a406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8aa4fb640 1 -- 172.21.15.119:0/551616675 >> v2:172.21.15.188:3300/0 conn(0x7fa8a41447a0 msgr2=0x7fa8a415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:20.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8aa4fb640 1 --2- 172.21.15.119:0/551616675 >> v2:172.21.15.188:3300/0 conn(0x7fa8a41447a0 0x7fa8a415f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.802 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.797+0000 7fa8aa4fb640 1 -- 172.21.15.119:0/551616675 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a40c9410 con 0x7fa8a40cb170 2023-11-08T19:32:20.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.799+0000 7fa8aa4fb640 1 --2- 172.21.15.119:0/551616675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cb170 0x7fa8a406a120 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40cfbd0 tx=0x7fa89c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338fa366e823a8f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:20.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.799+0000 7fa8a8cf8640 1 -- 172.21.15.119:0/551616675 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c02ca70 con 0x7fa8a40cb170 2023-11-08T19:32:20.804 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.799+0000 7fa8a8cf8640 1 -- 172.21.15.119:0/551616675 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89c02cbd0 con 0x7fa8a40cb170 2023-11-08T19:32:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.799+0000 7fa8a8cf8640 1 -- 172.21.15.119:0/551616675 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa89c0343f0 con 0x7fa8a40cb170 2023-11-08T19:32:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 -- 172.21.15.119:0/551616675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cb170 msgr2=0x7fa8a406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:20.805 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/551616675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cb170 0x7fa8a406a120 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40cfbd0 tx=0x7fa89c02bed0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 -- 172.21.15.119:0/551616675 shutdown_connections 2023-11-08T19:32:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/551616675 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cb170 0x7fa8a406a120 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.806 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/551616675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cad30 0x7fa8a406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/551616675 >> v2:172.21.15.188:3300/0 conn(0x7fa8a41447a0 0x7fa8a415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 -- 172.21.15.119:0/551616675 >> 172.21.15.119:0/551616675 conn(0x7fa8a40cb610 msgr2=0x7fa8a40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 -- 172.21.15.119:0/551616675 shutdown_connections 2023-11-08T19:32:20.807 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.800+0000 7fa8abf5e640 1 -- 172.21.15.119:0/551616675 wait complete. 2023-11-08T19:32:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.803+0000 7fa8abf5e640 1 Processor -- start 2023-11-08T19:32:20.808 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.803+0000 7fa8abf5e640 1 -- start start 2023-11-08T19:32:20.810 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8abf5e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 0x7fa8a40b40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8abf5e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cb170 0x7fa8a40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8abf5e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8a41447a0 0x7fa8a40c6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.811 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8abf5e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a415c1c0 con 0x7fa8a40cad30 2023-11-08T19:32:20.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8abf5e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a415bbd0 con 0x7fa8a41447a0 2023-11-08T19:32:20.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8abf5e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40cdba0 con 0x7fa8a40cb170 2023-11-08T19:32:20.812 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8a9cfa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 0x7fa8a40b40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8a9cfa640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 0x7fa8a40b40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39972/0 (socket says 172.21.15.119:39972) 2023-11-08T19:32:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8a9cfa640 1 -- 172.21.15.119:0/1949691817 learned_addr learned my addr 172.21.15.119:0/1949691817 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.804+0000 7fa8a94f9640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cb170 0x7fa8a40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.813 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8aa4fb640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8a41447a0 0x7fa8a40c6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8a9cfa640 1 -- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cb170 msgr2=0x7fa8a40cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8a9cfa640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cb170 0x7fa8a40cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8a9cfa640 1 -- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8a41447a0 msgr2=0x7fa8a40c6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:20.814 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8a9cfa640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8a41447a0 0x7fa8a40c6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8a9cfa640 1 -- 172.21.15.119:0/1949691817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c009b30 con 0x7fa8a40cad30 2023-11-08T19:32:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8aa4fb640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8a41447a0 0x7fa8a40c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:32:20.815 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8a9cfa640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 0x7fa8a40b40a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000e9b0 tx=0x7fa8a000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0009670 con 0x7fa8a40cad30 2023-11-08T19:32:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8a000c9a0 con 0x7fa8a40cad30 2023-11-08T19:32:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8a000cd50 con 0x7fa8a40cad30 2023-11-08T19:32:20.816 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.805+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a40cde80 con 0x7fa8a40cad30 2023-11-08T19:32:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.806+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a40c8860 con 0x7fa8a40cad30 2023-11-08T19:32:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.807+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a40ce070 con 0x7fa8a40cad30 2023-11-08T19:32:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.808+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8a0013470 con 0x7fa8a40cad30 2023-11-08T19:32:20.817 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.809+0000 7fa89a7fc640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa88806ae70 0x7fa88806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.809+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fa8a0089c30 con 0x7fa8a40cad30 2023-11-08T19:32:20.818 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.809+0000 7fa8a94f9640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa88806ae70 0x7fa88806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:20.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.810+0000 7fa8a94f9640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa88806ae70 0x7fa88806d330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40ce690 tx=0x7fa894009190 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:20.819 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:20.815+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8a40ce070 con 0x7fa8a40cad30 2023-11-08T19:32:21.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.005+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8a40c3cf0 con 0x7fa88806ae70 2023-11-08T19:32:21.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.015+0000 7fa89a7fc640 1 -- 172.21.15.119:0/1949691817 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290803 (secure 0 0 0) 0x7fa8940119d0 con 0x7fa88806ae70 2023-11-08T19:32:21.021 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:21.022 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:32:21.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa88806ae70 msgr2=0x7fa88806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa88806ae70 0x7fa88806d330 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa8a40ce690 tx=0x7fa894009190 comp rx=0 tx=0).stop 2023-11-08T19:32:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 msgr2=0x7fa8a40b40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 0x7fa8a40b40a0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000e9b0 tx=0x7fa8a000ee70 comp rx=0 tx=0).stop 2023-11-08T19:32:21.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 shutdown_connections 2023-11-08T19:32:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa88806ae70 0x7fa88806d330 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8a41447a0 0x7fa8a40c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8a40cb170 0x7fa8a40cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 --2- 172.21.15.119:0/1949691817 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8a40cad30 0x7fa8a40b40a0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 >> 172.21.15.119:0/1949691817 conn(0x7fa8a40cb610 msgr2=0x7fa8a4142800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 shutdown_connections 2023-11-08T19:32:21.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.022+0000 7fa8abf5e640 1 -- 172.21.15.119:0/1949691817 wait complete. 2023-11-08T19:32:21.045 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-11-08T19:32:20.840051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261084,"kb_used_data":11636,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713636,"statfs":{"total":773094113280,"available":772826763264,"internally_reserved":0,"allocated":11915264,"data_stored":6682222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-78272,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-469,"num_read_kb":-623,"num_write":-182,"num_write_kb":-154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.762732"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082901+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:32:14.082901+0000","last_peered":"2023-11-08T19:32:14.082901+0000","last_clean":"2023-11-08T19:32:14.082901+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:14.082901+0000","last_undegraded":"2023-11-08T19:32:14.082901+0000","last_fullsized":"2023-11-08T19:32:14.082901+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326562+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:32:15.326562+0000","last_peered":"2023-11-08T19:32:15.326562+0000","last_clean":"2023-11-08T19:32:15.326562+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:15.326562+0000","last_undegraded":"2023-11-08T19:32:15.326562+0000","last_fullsized":"2023-11-08T19:32:15.326562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324561+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:32:15.324561+0000","last_peered":"2023-11-08T19:32:15.324561+0000","last_clean":"2023-11-08T19:32:15.324561+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:15.324561+0000","last_undegraded":"2023-11-08T19:32:15.324561+0000","last_fullsized":"2023-11-08T19:32:15.324561+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":43,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771284+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:32:14.771284+0000","last_peered":"2023-11-08T19:32:14.771284+0000","last_clean":"2023-11-08T19:32:14.771284+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:14.771284+0000","last_undegraded":"2023-11-08T19:32:14.771284+0000","last_fullsized":"2023-11-08T19:32:14.771284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326623+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:32:15.326623+0000","last_peered":"2023-11-08T19:32:15.326623+0000","last_clean":"2023-11-08T19:32:15.326623+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:32:15.326623+0000","last_undegraded":"2023-11-08T19:32:15.326623+0000","last_fullsized":"2023-11-08T19:32:15.326623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.084456+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:32:14.084456+0000","last_peered":"2023-11-08T19:32:14.084456+0000","last_clean":"2023-11-08T19:32:14.084456+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:14.084456+0000","last_undegraded":"2023-11-08T19:32:14.084456+0000","last_fullsized":"2023-11-08T19:32:14.084456+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.087661+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:32:14.087661+0000","last_peered":"2023-11-08T19:32:14.087661+0000","last_clean":"2023-11-08T19:32:14.087661+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:32:14.087661+0000","last_undegraded":"2023-11-08T19:32:14.087661+0000","last_fullsized":"2023-11-08T19:32:14.087661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.084090+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:32:14.084090+0000","last_peered":"2023-11-08T19:32:14.084090+0000","last_clean":"2023-11-08T19:32:14.084090+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:14.084090+0000","last_undegraded":"2023-11-08T19:32:14.084090+0000","last_fullsized":"2023-11-08T19:32:14.084090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.327008+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:32:15.327008+0000","last_peered":"2023-11-08T19:32:15.327008+0000","last_clean":"2023-11-08T19:32:15.327008+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:15.327008+0000","last_undegraded":"2023-11-08T19:32:15.327008+0000","last_fullsized":"2023-11-08T19:32:15.327008+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080981+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:32:14.080981+0000","last_peered":"2023-11-08T19:32:14.080981+0000","last_clean":"2023-11-08T19:32:14.080981+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:14.080981+0000","last_undegraded":"2023-11-08T19:32:14.080981+0000","last_fullsized":"2023-11-08T19:32:14.080981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325948+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:32:15.325948+0000","last_peered":"2023-11-08T19:32:15.325948+0000","last_clean":"2023-11-08T19:32:15.325948+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:32:15.325948+0000","last_undegraded":"2023-11-08T19:32:15.325948+0000","last_fullsized":"2023-11-08T19:32:15.325948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324984+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:32:15.324984+0000","last_peered":"2023-11-08T19:32:15.324984+0000","last_clean":"2023-11-08T19:32:15.324984+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:15.324984+0000","last_undegraded":"2023-11-08T19:32:15.324984+0000","last_fullsized":"2023-11-08T19:32:15.324984+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245918+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:32:14.245918+0000","last_peered":"2023-11-08T19:32:14.245918+0000","last_clean":"2023-11-08T19:32:14.245918+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:14.245918+0000","last_undegraded":"2023-11-08T19:32:14.245918+0000","last_fullsized":"2023-11-08T19:32:14.245918+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":50,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771029+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:32:14.771029+0000","last_peered":"2023-11-08T19:32:14.771029+0000","last_clean":"2023-11-08T19:32:14.771029+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:14.771029+0000","last_undegraded":"2023-11-08T19:32:14.771029+0000","last_fullsized":"2023-11-08T19:32:14.771029+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325529+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:32:15.325529+0000","last_peered":"2023-11-08T19:32:15.325529+0000","last_clean":"2023-11-08T19:32:15.325529+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:15.325529+0000","last_undegraded":"2023-11-08T19:32:15.325529+0000","last_fullsized":"2023-11-08T19:32:15.325529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080107+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:32:14.080107+0000","last_peered":"2023-11-08T19:32:14.080107+0000","last_clean":"2023-11-08T19:32:14.080107+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:32:14.080107+0000","last_undegraded":"2023-11-08T19:32:14.080107+0000","last_fullsized":"2023-11-08T19:32:14.080107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324313+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:32:15.324313+0000","last_peered":"2023-11-08T19:32:15.324313+0000","last_clean":"2023-11-08T19:32:15.324313+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:15.324313+0000","last_undegraded":"2023-11-08T19:32:15.324313+0000","last_fullsized":"2023-11-08T19:32:15.324313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079580+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:32:14.079580+0000","last_peered":"2023-11-08T19:32:14.079580+0000","last_clean":"2023-11-08T19:32:14.079580+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:14.079580+0000","last_undegraded":"2023-11-08T19:32:14.079580+0000","last_fullsized":"2023-11-08T19:32:14.079580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323815+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:32:15.323815+0000","last_peered":"2023-11-08T19:32:15.323815+0000","last_clean":"2023-11-08T19:32:15.323815+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:15.323815+0000","last_undegraded":"2023-11-08T19:32:15.323815+0000","last_fullsized":"2023-11-08T19:32:15.323815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079780+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:32:14.079780+0000","last_peered":"2023-11-08T19:32:14.079780+0000","last_clean":"2023-11-08T19:32:14.079780+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:14.079780+0000","last_undegraded":"2023-11-08T19:32:14.079780+0000","last_fullsized":"2023-11-08T19:32:14.079780+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080725+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:32:14.080725+0000","last_peered":"2023-11-08T19:32:14.080725+0000","last_clean":"2023-11-08T19:32:14.080725+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:14.080725+0000","last_undegraded":"2023-11-08T19:32:14.080725+0000","last_fullsized":"2023-11-08T19:32:14.080725+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323664+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:32:15.323664+0000","last_peered":"2023-11-08T19:32:15.323664+0000","last_clean":"2023-11-08T19:32:15.323664+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:15.323664+0000","last_undegraded":"2023-11-08T19:32:15.323664+0000","last_fullsized":"2023-11-08T19:32:15.323664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083600+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:32:14.083600+0000","last_peered":"2023-11-08T19:32:14.083600+0000","last_clean":"2023-11-08T19:32:14.083600+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:14.083600+0000","last_undegraded":"2023-11-08T19:32:14.083600+0000","last_fullsized":"2023-11-08T19:32:14.083600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086612+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:32:14.086612+0000","last_peered":"2023-11-08T19:32:14.086612+0000","last_clean":"2023-11-08T19:32:14.086612+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:14.086612+0000","last_undegraded":"2023-11-08T19:32:14.086612+0000","last_fullsized":"2023-11-08T19:32:14.086612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326896+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:32:15.326896+0000","last_peered":"2023-11-08T19:32:15.326896+0000","last_clean":"2023-11-08T19:32:15.326896+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:32:15.326896+0000","last_undegraded":"2023-11-08T19:32:15.326896+0000","last_fullsized":"2023-11-08T19:32:15.326896+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088580+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:32:14.088580+0000","last_peered":"2023-11-08T19:32:14.088580+0000","last_clean":"2023-11-08T19:32:14.088580+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:14.088580+0000","last_undegraded":"2023-11-08T19:32:14.088580+0000","last_fullsized":"2023-11-08T19:32:14.088580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246666+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:32:14.246666+0000","last_peered":"2023-11-08T19:32:14.246666+0000","last_clean":"2023-11-08T19:32:14.246666+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:14.246666+0000","last_undegraded":"2023-11-08T19:32:14.246666+0000","last_fullsized":"2023-11-08T19:32:14.246666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088147+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:32:14.088147+0000","last_peered":"2023-11-08T19:32:14.088147+0000","last_clean":"2023-11-08T19:32:14.088147+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:32:14.088147+0000","last_undegraded":"2023-11-08T19:32:14.088147+0000","last_fullsized":"2023-11-08T19:32:14.088147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326448+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:32:15.326448+0000","last_peered":"2023-11-08T19:32:15.326448+0000","last_clean":"2023-11-08T19:32:15.326448+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:32:15.326448+0000","last_undegraded":"2023-11-08T19:32:15.326448+0000","last_fullsized":"2023-11-08T19:32:15.326448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771506+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:32:14.771506+0000","last_peered":"2023-11-08T19:32:14.771506+0000","last_clean":"2023-11-08T19:32:14.771506+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:14.771506+0000","last_undegraded":"2023-11-08T19:32:14.771506+0000","last_fullsized":"2023-11-08T19:32:14.771506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246330+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:32:14.246330+0000","last_peered":"2023-11-08T19:32:14.246330+0000","last_clean":"2023-11-08T19:32:14.246330+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:32:14.246330+0000","last_undegraded":"2023-11-08T19:32:14.246330+0000","last_fullsized":"2023-11-08T19:32:14.246330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081842+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:32:14.081842+0000","last_peered":"2023-11-08T19:32:14.081842+0000","last_clean":"2023-11-08T19:32:14.081842+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:14.081842+0000","last_undegraded":"2023-11-08T19:32:14.081842+0000","last_fullsized":"2023-11-08T19:32:14.081842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246823+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:32:14.246823+0000","last_peered":"2023-11-08T19:32:14.246823+0000","last_clean":"2023-11-08T19:32:14.246823+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:14.246823+0000","last_undegraded":"2023-11-08T19:32:14.246823+0000","last_fullsized":"2023-11-08T19:32:14.246823+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325340+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:32:15.325340+0000","last_peered":"2023-11-08T19:32:15.325340+0000","last_clean":"2023-11-08T19:32:15.325340+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:32:15.325340+0000","last_undegraded":"2023-11-08T19:32:15.325340+0000","last_fullsized":"2023-11-08T19:32:15.325340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080658+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:32:14.080658+0000","last_peered":"2023-11-08T19:32:14.080658+0000","last_clean":"2023-11-08T19:32:14.080658+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:14.080658+0000","last_undegraded":"2023-11-08T19:32:14.080658+0000","last_fullsized":"2023-11-08T19:32:14.080658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088790+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:32:14.088790+0000","last_peered":"2023-11-08T19:32:14.088790+0000","last_clean":"2023-11-08T19:32:14.088790+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:14.088790+0000","last_undegraded":"2023-11-08T19:32:14.088790+0000","last_fullsized":"2023-11-08T19:32:14.088790+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085436+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:32:14.085436+0000","last_peered":"2023-11-08T19:32:14.085436+0000","last_clean":"2023-11-08T19:32:14.085436+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:14.085436+0000","last_undegraded":"2023-11-08T19:32:14.085436+0000","last_fullsized":"2023-11-08T19:32:14.085436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086428+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:32:14.086428+0000","last_peered":"2023-11-08T19:32:14.086428+0000","last_clean":"2023-11-08T19:32:14.086428+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:14.086428+0000","last_undegraded":"2023-11-08T19:32:14.086428+0000","last_fullsized":"2023-11-08T19:32:14.086428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245467+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:32:14.245467+0000","last_peered":"2023-11-08T19:32:14.245467+0000","last_clean":"2023-11-08T19:32:14.245467+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:14.245467+0000","last_undegraded":"2023-11-08T19:32:14.245467+0000","last_fullsized":"2023-11-08T19:32:14.245467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324366+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:32:15.324366+0000","last_peered":"2023-11-08T19:32:15.324366+0000","last_clean":"2023-11-08T19:32:15.324366+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:15.324366+0000","last_undegraded":"2023-11-08T19:32:15.324366+0000","last_fullsized":"2023-11-08T19:32:15.324366+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085906+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:32:14.085906+0000","last_peered":"2023-11-08T19:32:14.085906+0000","last_clean":"2023-11-08T19:32:14.085906+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:14.085906+0000","last_undegraded":"2023-11-08T19:32:14.085906+0000","last_fullsized":"2023-11-08T19:32:14.085906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.770306+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:32:14.770306+0000","last_peered":"2023-11-08T19:32:14.770306+0000","last_clean":"2023-11-08T19:32:14.770306+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:14.770306+0000","last_undegraded":"2023-11-08T19:32:14.770306+0000","last_fullsized":"2023-11-08T19:32:14.770306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":43,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.769959+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:32:14.769959+0000","last_peered":"2023-11-08T19:32:14.769959+0000","last_clean":"2023-11-08T19:32:14.769959+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:14.769959+0000","last_undegraded":"2023-11-08T19:32:14.769959+0000","last_fullsized":"2023-11-08T19:32:14.769959+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323721+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:32:15.323721+0000","last_peered":"2023-11-08T19:32:15.323721+0000","last_clean":"2023-11-08T19:32:15.323721+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:15.323721+0000","last_undegraded":"2023-11-08T19:32:15.323721+0000","last_fullsized":"2023-11-08T19:32:15.323721+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080836+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:32:14.080836+0000","last_peered":"2023-11-08T19:32:14.080836+0000","last_clean":"2023-11-08T19:32:14.080836+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:14.080836+0000","last_undegraded":"2023-11-08T19:32:14.080836+0000","last_fullsized":"2023-11-08T19:32:14.080836+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245560+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:32:14.245560+0000","last_peered":"2023-11-08T19:32:14.245560+0000","last_clean":"2023-11-08T19:32:14.245560+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:14.245560+0000","last_undegraded":"2023-11-08T19:32:14.245560+0000","last_fullsized":"2023-11-08T19:32:14.245560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079989+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:32:14.079989+0000","last_peered":"2023-11-08T19:32:14.079989+0000","last_clean":"2023-11-08T19:32:14.079989+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:14.079989+0000","last_undegraded":"2023-11-08T19:32:14.079989+0000","last_fullsized":"2023-11-08T19:32:14.079989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325680+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:32:15.325680+0000","last_peered":"2023-11-08T19:32:15.325680+0000","last_clean":"2023-11-08T19:32:15.325680+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:15.325680+0000","last_undegraded":"2023-11-08T19:32:15.325680+0000","last_fullsized":"2023-11-08T19:32:15.325680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246594+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:32:14.246594+0000","last_peered":"2023-11-08T19:32:14.246594+0000","last_clean":"2023-11-08T19:32:14.246594+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:32:14.246594+0000","last_undegraded":"2023-11-08T19:32:14.246594+0000","last_fullsized":"2023-11-08T19:32:14.246594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324910+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:32:15.324910+0000","last_peered":"2023-11-08T19:32:15.324910+0000","last_clean":"2023-11-08T19:32:15.324910+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:15.324910+0000","last_undegraded":"2023-11-08T19:32:15.324910+0000","last_fullsized":"2023-11-08T19:32:15.324910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326226+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:32:15.326226+0000","last_peered":"2023-11-08T19:32:15.326226+0000","last_clean":"2023-11-08T19:32:15.326226+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:15.326226+0000","last_undegraded":"2023-11-08T19:32:15.326226+0000","last_fullsized":"2023-11-08T19:32:15.326226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088471+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:32:14.088471+0000","last_peered":"2023-11-08T19:32:14.088471+0000","last_clean":"2023-11-08T19:32:14.088471+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:14.088471+0000","last_undegraded":"2023-11-08T19:32:14.088471+0000","last_fullsized":"2023-11-08T19:32:14.088471+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325063+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:32:15.325063+0000","last_peered":"2023-11-08T19:32:15.325063+0000","last_clean":"2023-11-08T19:32:15.325063+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:15.325063+0000","last_undegraded":"2023-11-08T19:32:15.325063+0000","last_fullsized":"2023-11-08T19:32:15.325063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325706+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:32:15.325706+0000","last_peered":"2023-11-08T19:32:15.325706+0000","last_clean":"2023-11-08T19:32:15.325706+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:32:15.325706+0000","last_undegraded":"2023-11-08T19:32:15.325706+0000","last_fullsized":"2023-11-08T19:32:15.325706+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088073+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:32:14.088073+0000","last_peered":"2023-11-08T19:32:14.088073+0000","last_clean":"2023-11-08T19:32:14.088073+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:32:14.088073+0000","last_undegraded":"2023-11-08T19:32:14.088073+0000","last_fullsized":"2023-11-08T19:32:14.088073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325460+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:32:15.325460+0000","last_peered":"2023-11-08T19:32:15.325460+0000","last_clean":"2023-11-08T19:32:15.325460+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:15.325460+0000","last_undegraded":"2023-11-08T19:32:15.325460+0000","last_fullsized":"2023-11-08T19:32:15.325460+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082006+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:32:14.082006+0000","last_peered":"2023-11-08T19:32:14.082006+0000","last_clean":"2023-11-08T19:32:14.082006+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:14.082006+0000","last_undegraded":"2023-11-08T19:32:14.082006+0000","last_fullsized":"2023-11-08T19:32:14.082006+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325608+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:32:15.325608+0000","last_peered":"2023-11-08T19:32:15.325608+0000","last_clean":"2023-11-08T19:32:15.325608+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:32:15.325608+0000","last_undegraded":"2023-11-08T19:32:15.325608+0000","last_fullsized":"2023-11-08T19:32:15.325608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325825+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:32:15.325825+0000","last_peered":"2023-11-08T19:32:15.325825+0000","last_clean":"2023-11-08T19:32:15.325825+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:32:15.325825+0000","last_undegraded":"2023-11-08T19:32:15.325825+0000","last_fullsized":"2023-11-08T19:32:15.325825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.970299+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:32:14.970299+0000","last_peered":"2023-11-08T19:32:14.970299+0000","last_clean":"2023-11-08T19:32:14.970299+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:14.970299+0000","last_undegraded":"2023-11-08T19:32:14.970299+0000","last_fullsized":"2023-11-08T19:32:14.970299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080424+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:32:14.080424+0000","last_peered":"2023-11-08T19:32:14.080424+0000","last_clean":"2023-11-08T19:32:14.080424+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:14.080424+0000","last_undegraded":"2023-11-08T19:32:14.080424+0000","last_fullsized":"2023-11-08T19:32:14.080424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086594+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:32:14.086594+0000","last_peered":"2023-11-08T19:32:14.086594+0000","last_clean":"2023-11-08T19:32:14.086594+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:14.086594+0000","last_undegraded":"2023-11-08T19:32:14.086594+0000","last_fullsized":"2023-11-08T19:32:14.086594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081053+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:32:14.081053+0000","last_peered":"2023-11-08T19:32:14.081053+0000","last_clean":"2023-11-08T19:32:14.081053+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:14.081053+0000","last_undegraded":"2023-11-08T19:32:14.081053+0000","last_fullsized":"2023-11-08T19:32:14.081053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082613+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:32:14.082613+0000","last_peered":"2023-11-08T19:32:14.082613+0000","last_clean":"2023-11-08T19:32:14.082613+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:14.082613+0000","last_undegraded":"2023-11-08T19:32:14.082613+0000","last_fullsized":"2023-11-08T19:32:14.082613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080615+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:32:14.080615+0000","last_peered":"2023-11-08T19:32:14.080615+0000","last_clean":"2023-11-08T19:32:14.080615+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:14.080615+0000","last_undegraded":"2023-11-08T19:32:14.080615+0000","last_fullsized":"2023-11-08T19:32:14.080615+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968861+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:32:14.968861+0000","last_peered":"2023-11-08T19:32:14.968861+0000","last_clean":"2023-11-08T19:32:14.968861+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:14.968861+0000","last_undegraded":"2023-11-08T19:32:14.968861+0000","last_fullsized":"2023-11-08T19:32:14.968861+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325069+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:32:15.325069+0000","last_peered":"2023-11-08T19:32:15.325069+0000","last_clean":"2023-11-08T19:32:15.325069+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:32:15.325069+0000","last_undegraded":"2023-11-08T19:32:15.325069+0000","last_fullsized":"2023-11-08T19:32:15.325069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083582+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:32:14.083582+0000","last_peered":"2023-11-08T19:32:14.083582+0000","last_clean":"2023-11-08T19:32:14.083582+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:14.083582+0000","last_undegraded":"2023-11-08T19:32:14.083582+0000","last_fullsized":"2023-11-08T19:32:14.083582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324550+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:32:15.324550+0000","last_peered":"2023-11-08T19:32:15.324550+0000","last_clean":"2023-11-08T19:32:15.324550+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:15.324550+0000","last_undegraded":"2023-11-08T19:32:15.324550+0000","last_fullsized":"2023-11-08T19:32:15.324550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245784+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:32:14.245784+0000","last_peered":"2023-11-08T19:32:14.245784+0000","last_clean":"2023-11-08T19:32:14.245784+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:14.245784+0000","last_undegraded":"2023-11-08T19:32:14.245784+0000","last_fullsized":"2023-11-08T19:32:14.245784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325075+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:32:15.325075+0000","last_peered":"2023-11-08T19:32:15.325075+0000","last_clean":"2023-11-08T19:32:15.325075+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:15.325075+0000","last_undegraded":"2023-11-08T19:32:15.325075+0000","last_fullsized":"2023-11-08T19:32:15.325075+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771424+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:32:14.771424+0000","last_peered":"2023-11-08T19:32:14.771424+0000","last_clean":"2023-11-08T19:32:14.771424+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:14.771424+0000","last_undegraded":"2023-11-08T19:32:14.771424+0000","last_fullsized":"2023-11-08T19:32:14.771424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326894+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:32:15.326894+0000","last_peered":"2023-11-08T19:32:15.326894+0000","last_clean":"2023-11-08T19:32:15.326894+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:32:15.326894+0000","last_undegraded":"2023-11-08T19:32:15.326894+0000","last_fullsized":"2023-11-08T19:32:15.326894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258452+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:32:16.258452+0000","last_peered":"2023-11-08T19:32:16.258452+0000","last_clean":"2023-11-08T19:32:16.258452+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:16.258452+0000","last_undegraded":"2023-11-08T19:32:16.258452+0000","last_fullsized":"2023-11-08T19:32:16.258452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246884+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:32:14.246884+0000","last_peered":"2023-11-08T19:32:14.246884+0000","last_clean":"2023-11-08T19:32:14.246884+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:14.246884+0000","last_undegraded":"2023-11-08T19:32:14.246884+0000","last_fullsized":"2023-11-08T19:32:14.246884+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245706+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:32:14.245706+0000","last_peered":"2023-11-08T19:32:14.245706+0000","last_clean":"2023-11-08T19:32:14.245706+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:14.245706+0000","last_undegraded":"2023-11-08T19:32:14.245706+0000","last_fullsized":"2023-11-08T19:32:14.245706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":54,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.255292+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:32:16.255292+0000","last_peered":"2023-11-08T19:32:16.255292+0000","last_clean":"2023-11-08T19:32:16.255292+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:32:16.255292+0000","last_undegraded":"2023-11-08T19:32:16.255292+0000","last_fullsized":"2023-11-08T19:32:16.255292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081818+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:32:14.081818+0000","last_peered":"2023-11-08T19:32:14.081818+0000","last_clean":"2023-11-08T19:32:14.081818+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:14.081818+0000","last_undegraded":"2023-11-08T19:32:14.081818+0000","last_fullsized":"2023-11-08T19:32:14.081818+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.087868+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:32:14.087868+0000","last_peered":"2023-11-08T19:32:14.087868+0000","last_clean":"2023-11-08T19:32:14.087868+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:14.087868+0000","last_undegraded":"2023-11-08T19:32:14.087868+0000","last_fullsized":"2023-11-08T19:32:14.087868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.256957+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:32:16.256957+0000","last_peered":"2023-11-08T19:32:16.256957+0000","last_clean":"2023-11-08T19:32:16.256957+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:16.256957+0000","last_undegraded":"2023-11-08T19:32:16.256957+0000","last_fullsized":"2023-11-08T19:32:16.256957+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.327036+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:32:15.327036+0000","last_peered":"2023-11-08T19:32:15.327036+0000","last_clean":"2023-11-08T19:32:15.327036+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:15.327036+0000","last_undegraded":"2023-11-08T19:32:15.327036+0000","last_fullsized":"2023-11-08T19:32:15.327036+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083682+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:32:14.083682+0000","last_peered":"2023-11-08T19:32:14.083682+0000","last_clean":"2023-11-08T19:32:14.083682+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:14.083682+0000","last_undegraded":"2023-11-08T19:32:14.083682+0000","last_fullsized":"2023-11-08T19:32:14.083682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968622+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:32:14.968622+0000","last_peered":"2023-11-08T19:32:14.968622+0000","last_clean":"2023-11-08T19:32:14.968622+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:14.968622+0000","last_undegraded":"2023-11-08T19:32:14.968622+0000","last_fullsized":"2023-11-08T19:32:14.968622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085702+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:32:14.085702+0000","last_peered":"2023-11-08T19:32:14.085702+0000","last_clean":"2023-11-08T19:32:14.085702+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:14.085702+0000","last_undegraded":"2023-11-08T19:32:14.085702+0000","last_fullsized":"2023-11-08T19:32:14.085702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968748+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:32:14.968748+0000","last_peered":"2023-11-08T19:32:14.968748+0000","last_clean":"2023-11-08T19:32:14.968748+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:14.968748+0000","last_undegraded":"2023-11-08T19:32:14.968748+0000","last_fullsized":"2023-11-08T19:32:14.968748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":51,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.252776+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:32:16.252776+0000","last_peered":"2023-11-08T19:32:16.252776+0000","last_clean":"2023-11-08T19:32:16.252776+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:16.252776+0000","last_undegraded":"2023-11-08T19:32:16.252776+0000","last_fullsized":"2023-11-08T19:32:16.252776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081711+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:32:14.081711+0000","last_peered":"2023-11-08T19:32:14.081711+0000","last_clean":"2023-11-08T19:32:14.081711+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:14.081711+0000","last_undegraded":"2023-11-08T19:32:14.081711+0000","last_fullsized":"2023-11-08T19:32:14.081711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082992+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:32:14.082992+0000","last_peered":"2023-11-08T19:32:14.082992+0000","last_clean":"2023-11-08T19:32:14.082992+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:14.082992+0000","last_undegraded":"2023-11-08T19:32:14.082992+0000","last_fullsized":"2023-11-08T19:32:14.082992+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.769987+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:32:14.769987+0000","last_peered":"2023-11-08T19:32:14.769987+0000","last_clean":"2023-11-08T19:32:14.769987+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:14.769987+0000","last_undegraded":"2023-11-08T19:32:14.769987+0000","last_fullsized":"2023-11-08T19:32:14.769987+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771136+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:32:14.771136+0000","last_peered":"2023-11-08T19:32:14.771136+0000","last_clean":"2023-11-08T19:32:14.771136+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:14.771136+0000","last_undegraded":"2023-11-08T19:32:14.771136+0000","last_fullsized":"2023-11-08T19:32:14.771136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968697+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:32:14.968697+0000","last_peered":"2023-11-08T19:32:14.968697+0000","last_clean":"2023-11-08T19:32:14.968697+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:14.968697+0000","last_undegraded":"2023-11-08T19:32:14.968697+0000","last_fullsized":"2023-11-08T19:32:14.968697+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968890+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:32:14.968890+0000","last_peered":"2023-11-08T19:32:14.968890+0000","last_clean":"2023-11-08T19:32:14.968890+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:14.968890+0000","last_undegraded":"2023-11-08T19:32:14.968890+0000","last_fullsized":"2023-11-08T19:32:14.968890+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771554+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:32:14.771554+0000","last_peered":"2023-11-08T19:32:14.771554+0000","last_clean":"2023-11-08T19:32:14.771554+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:14.771554+0000","last_undegraded":"2023-11-08T19:32:14.771554+0000","last_fullsized":"2023-11-08T19:32:14.771554+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325205+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:32:15.325205+0000","last_peered":"2023-11-08T19:32:15.325205+0000","last_clean":"2023-11-08T19:32:15.325205+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:15.325205+0000","last_undegraded":"2023-11-08T19:32:15.325205+0000","last_fullsized":"2023-11-08T19:32:15.325205+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":52,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.257983+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:32:16.257983+0000","last_peered":"2023-11-08T19:32:16.257983+0000","last_clean":"2023-11-08T19:32:16.257983+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:16.257983+0000","last_undegraded":"2023-11-08T19:32:16.257983+0000","last_fullsized":"2023-11-08T19:32:16.257983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080613+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:32:14.080613+0000","last_peered":"2023-11-08T19:32:14.080613+0000","last_clean":"2023-11-08T19:32:14.080613+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:14.080613+0000","last_undegraded":"2023-11-08T19:32:14.080613+0000","last_fullsized":"2023-11-08T19:32:14.080613+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246364+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:32:14.246364+0000","last_peered":"2023-11-08T19:32:14.246364+0000","last_clean":"2023-11-08T19:32:14.246364+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:14.246364+0000","last_undegraded":"2023-11-08T19:32:14.246364+0000","last_fullsized":"2023-11-08T19:32:14.246364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1929216,"data_stored":1273881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.77500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.599,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.252},"min":{"1min":0.72799999999999998,"5min":0.58599999999999997,"15min":0.58499999999999996},"max":{"1min":1.6619999999999999,"5min":2.1360000000000001,"15min":4.4589999999999996},"last":1.3560000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2010000000000001,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.548,"5min":1.728,"15min":4.5330000000000004},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.1910000000000001,"15min":1.1459999999999999},"min":{"1min":0.83299999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6160000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.2030000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.335,"15min":1.448},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5660000000000001,"5min":1.891,"15min":4.6980000000000004},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.194,"15min":1.3160000000000001},"min":{"1min":0.97999999999999998,"5min":0.67200000000000004,"15min":0.64700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":4.4269999999999996},"last":1.5780000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.294,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.7889999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.409,"15min":1.5149999999999999},"min":{"1min":0.95199999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.768,"5min":2.2200000000000002,"15min":4.6150000000000002},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.327},"min":{"1min":1.006,"5min":0.624,"15min":0.624},"max":{"1min":1.49,"5min":1.798,"15min":2.3170000000000002},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.323,"15min":1.4410000000000001},"min":{"1min":0.93200000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7909999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.403}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.95999999999999996,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.673,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.508},{"interface":"front","average":{"1min":1.379,"5min":1.363,"15min":1.4470000000000001},"min":{"1min":1.048,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.649,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":1.22,"5min":0.878,"15min":0.878},"max":{"1min":1.8109999999999999,"5min":1.9770000000000001,"15min":2.427},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.474,"15min":1.581},"min":{"1min":1.1180000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.98,"5min":2.1920000000000002,"15min":2.468},"last":1.5489999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.0940000000000001,"15min":0.98799999999999999},"min":{"1min":0.85399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.536,"5min":1.6279999999999999,"15min":1.752},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.202,"15min":1.1399999999999999},"min":{"1min":0.80200000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":8.391,"5min":8.391,"15min":8.391},"last":0.752}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.161,"15min":1.113},"min":{"1min":0.85599999999999998,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.679},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2050000000000001,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.665,"5min":1.665,"15min":4.6399999999999997},"last":1.099}]},{"osd":2,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.80300000000000005,"5min":0.68799999999999994,"15min":0.54600000000000004},"max":{"1min":1.4830000000000001,"5min":1.655,"15min":4.609},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2949999999999999,"15min":1.242},"min":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.861},"last":1.141}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.276,"15min":1.4239999999999999},"min":{"1min":1.0209999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.756,"15min":4.7190000000000003},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.256,"15min":1.2050000000000001},"min":{"1min":0.81200000000000006,"5min":0.625,"15min":0.625},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.784},"last":0.98299999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.83199999999999996,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.859,"5min":1.881,"15min":2.0449999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4770000000000001,"15min":1.4910000000000001},"min":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.998,"15min":2.3130000000000002},"last":1.5029999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.325,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.611,"5min":1.835,"15min":1.9079999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3999999999999999,"15min":1.353},"min":{"1min":0.92100000000000004,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":1.6819999999999999,"5min":1.8660000000000001,"15min":2.0489999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5569999999999999,"15min":1.55},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":1.99,"5min":1.99,"15min":2.2519999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4970000000000001,"15min":1.464},"min":{"1min":0.75700000000000001,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":1.7809999999999999,"5min":1.9870000000000001,"15min":3.0640000000000001},"last":1.4219999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1970000000000001,"15min":1.117},"min":{"1min":0.69599999999999995,"5min":0.59999999999999998,"15min":0.57699999999999996},"max":{"1min":1.603,"5min":1.724,"15min":1.76},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.274,"15min":1.252},"min":{"1min":0.77600000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.837,"5min":1.954,"15min":1.954},"last":1.6259999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.754,"5min":0.753,"15min":0.58699999999999997},"max":{"1min":1.552,"5min":1.6359999999999999,"15min":2.3039999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.179,"5min":1.1679999999999999,"15min":1.355},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.115}]},{"osd":2,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2350000000000001,"15min":1.194},"min":{"1min":0.82599999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5720000000000001,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.173,"15min":1.3959999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6990000000000001,"5min":1.806,"15min":7.6479999999999997},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.74,"5min":1.804,"15min":2.1920000000000002},"last":1.161},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3500000000000001,"15min":1.323},"min":{"1min":0.98799999999999999,"5min":0.91300000000000003,"15min":0.69699999999999995},"max":{"1min":1.6910000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.196,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7350000000000001,"5min":1.7969999999999999,"15min":2.347},"last":2.0430000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.347,"15min":1.488},"min":{"1min":0.94099999999999995,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.7350000000000001,"5min":2.097,"15min":3.153},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3859999999999999,"15min":1.4830000000000001},"min":{"1min":0.92100000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.923,"5min":1.923,"15min":2.8650000000000002},"last":1.448},{"interface":"front","average":{"1min":1.399,"5min":1.4019999999999999,"15min":1.47},"min":{"1min":0.94199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6990000000000001,"5min":1.899,"15min":2.6190000000000002},"last":1.742}]},{"osd":7,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.6359999999999999},"min":{"1min":1.0980000000000001,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":1.9259999999999999,"5min":2.0630000000000002,"15min":3.2730000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5229999999999999,"15min":1.5680000000000001},"min":{"1min":1.204,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.8600000000000001,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.7030000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1479999999999999,"15min":1.0760000000000001},"min":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.5900000000000001,"5min":1.782,"15min":2.5219999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.83499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.524,"5min":1.696,"15min":7.0119999999999996},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.0680000000000001,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.63100000000000001},"max":{"1min":1.752,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.206},{"interface":"front","average":{"1min":1.177,"5min":1.1499999999999999,"15min":1.1519999999999999},"min":{"1min":0.748,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.7310000000000001,"5min":1.7450000000000001,"15min":2.9940000000000002},"last":1.3879999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2490000000000001,"15min":1.3160000000000001},"min":{"1min":0.98199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.988,"5min":1.988,"15min":4.0410000000000004},"last":1.046},{"interface":"front","average":{"1min":1.341,"5min":1.319,"15min":1.3879999999999999},"min":{"1min":0.95999999999999996,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":3.2069999999999999},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1659999999999999,"15min":1.258},"min":{"1min":0.84199999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5169999999999999,"5min":1.696,"15min":4.4980000000000002},"last":1.105},{"interface":"front","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.345},"min":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":1.738,"15min":3.5710000000000002},"last":1.429}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.4330000000000001},"min":{"1min":0.81000000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.593,"5min":1.762,"15min":4.2350000000000003},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2549999999999999,"15min":1.4590000000000001},"min":{"1min":1.0189999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5800000000000001,"5min":1.8560000000000001,"15min":4.5880000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.365,"15min":1.5009999999999999},"min":{"1min":0.92300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":2.2130000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4419999999999999,"15min":1.5660000000000001},"min":{"1min":1.1639999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.841,"5min":1.9490000000000001,"15min":4.4130000000000003},"last":1.26}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.409,"15min":1.5640000000000001},"min":{"1min":1.137,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":4.6369999999999996},"last":1.325},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5669999999999999,"15min":1.6799999999999999},"min":{"1min":1.2969999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8999999999999999,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.488}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.99399999999999999},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.427,"5min":2.2320000000000002,"15min":3.972},"last":1.355},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93500000000000005,"15min":1.042},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.311},"max":{"1min":1.544,"5min":1.899,"15min":3.1520000000000001},"last":1.7829999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95299999999999996,"15min":1.1160000000000001},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.4299999999999999,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":1.0429999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8520000000000001,"15min":4.5750000000000002},"last":0.879}]},{"osd":2,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.153,"15min":1.272},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.619,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1020000000000001,"15min":1.218},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.6339999999999999,"5min":1.754,"15min":2.73},"last":1.294}]},{"osd":4,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.214,"15min":1.3779999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6599999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.24,"15min":1.3899999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9039999999999999,"5min":2.0449999999999999,"15min":3.052},"last":1.4530000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.4370000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":2.0430000000000001,"15min":3.0899999999999999},"last":1.9259999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.29,"15min":1.4570000000000001},"min":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.597,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.234}]},{"osd":6,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2769999999999999,"15min":1.3819999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7,"5min":3.21,"15min":4.2220000000000004},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.391,"15min":1.526},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.0880000000000001,"15min":3.1230000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.399,"15min":1.6160000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.925,"5min":2.1150000000000002,"15min":4.407},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.5209999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.671,"5min":2.0059999999999998,"15min":4.3849999999999998},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":674557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83699999999999997,"15min":0.85399999999999998},"min":{"1min":0.61699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2370000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0720000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.56799999999999995,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.371,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.66700000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.474,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.093,"15min":1.0620000000000001},"min":{"1min":0.80300000000000005,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.052,"15min":1.103},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.675,"5min":1.7330000000000001,"15min":1.7669999999999999},"last":1.1240000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2829999999999999,"15min":1.365},"min":{"1min":0.95499999999999996,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":1.883,"15min":3.2959999999999998},"last":1.268},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.284,"15min":1.3400000000000001},"min":{"1min":1.0609999999999999,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":1.704,"15min":2.278},"last":1.556}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.325,"15min":1.3899999999999999},"min":{"1min":1.1359999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.327,"15min":1.4179999999999999},"min":{"1min":1.048,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.7709999999999999,"5min":1.954,"15min":2.536},"last":1.4350000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":1.1120000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.657},{"interface":"front","average":{"1min":1.377,"5min":1.351,"15min":1.421},"min":{"1min":0.92200000000000004,"5min":0.83799999999999997,"15min":0.70099999999999996},"max":{"1min":1.9810000000000001,"5min":2.036,"15min":2.0910000000000002},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.409,"15min":1.522},"min":{"1min":0.95099999999999996,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":1.74,"5min":1.9339999999999999,"15min":2.6549999999999998},"last":1.323},{"interface":"front","average":{"1min":1.516,"5min":1.466,"15min":1.5269999999999999},"min":{"1min":1.111,"5min":0.84999999999999998,"15min":0.76600000000000001},"max":{"1min":1.8009999999999999,"5min":2.226,"15min":2.3439999999999999},"last":1.6080000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85599999999999998,"15min":0.78000000000000003},"min":{"1min":0.59199999999999997,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.284,"5min":1.327,"15min":1.327},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.88800000000000001,"15min":0.79900000000000004},"min":{"1min":0.82499999999999996,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":1.528,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1359999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84099999999999997,"15min":0.81100000000000005},"min":{"1min":0.51100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.256,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0249999999999999,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.47499999999999998,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.028}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2569999999999999,"15min":1.1240000000000001},"min":{"1min":0.80100000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.748,"5min":1.978,"15min":1.978},"last":1.302},{"interface":"front","average":{"1min":1.196,"5min":1.1439999999999999,"15min":1.1020000000000001},"min":{"1min":0.65300000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.6930000000000001,"5min":2.2690000000000001,"15min":2.5369999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.157},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.476,"5min":1.8620000000000001,"15min":2.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.294,"5min":1.2729999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.6870000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.1859999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.93799999999999994,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2689999999999999,"15min":1.3069999999999999},"min":{"1min":0.95199999999999996,"5min":0.73199999999999998,"15min":0.629},"max":{"1min":1.696,"5min":1.748,"15min":2.4089999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.375},"min":{"1min":1.194,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.7450000000000001,"5min":2.1509999999999998,"15min":2.6019999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.355,"15min":1.373},"min":{"1min":1.133,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.282},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.3560000000000001},"min":{"1min":1.0029999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.409,"15min":1.395},"min":{"1min":1.0940000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.865,"5min":2.4830000000000001,"15min":2.6699999999999999},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":1986560,"data_stored":1316219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.63200000000000001,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.0780000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2110000000000001,"5min":1.4770000000000001,"15min":1.7},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.876},"min":{"1min":0.57399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.319,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.998,"15min":0.98299999999999998},"min":{"1min":0.76200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5189999999999999,"5min":1.54,"15min":1.7270000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.016},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.748},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.089,"5min":1.0660000000000001,"15min":1.0940000000000001},"min":{"1min":0.60599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":1.8400000000000001,"15min":2.496},"last":1.276}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2709999999999999,"15min":1.244},"min":{"1min":0.90600000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6180000000000001,"5min":1.794,"15min":1.9239999999999999},"last":1.373},{"interface":"front","average":{"1min":1.427,"5min":1.29,"15min":1.3939999999999999},"min":{"1min":1.127,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.6399999999999999,"5min":1.706,"15min":2.4039999999999999},"last":1.3420000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.306,"15min":1.351},"min":{"1min":1.042,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7270000000000001,"5min":1.8620000000000001,"15min":2.4369999999999998},"last":1.643},{"interface":"front","average":{"1min":1.319,"5min":1.2949999999999999,"15min":1.3979999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.569,"5min":2.089,"15min":2.4239999999999999},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3500000000000001,"15min":1.4550000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.421},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.357,"15min":1.3959999999999999},"min":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7549999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.474}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.93400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.851,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.415,"15min":1.462},"min":{"1min":1.103,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7909999999999999,"5min":1.881,"15min":2.306},"last":1.524}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:21.052 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:21.286 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.284+0000 7f528b600640 1 Processor -- start 2023-11-08T19:32:21.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.285+0000 7f528b600640 1 -- start start 2023-11-08T19:32:21.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.286+0000 7f528b600640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f528415f4c0 0x7f528406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.286+0000 7f528b600640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f528406a120 0x7f528406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.286+0000 7f528b600640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f528406aa70 con 0x7f52841447a0 2023-11-08T19:32:21.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.286+0000 7f528b600640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f528415f930 con 0x7f528415f4c0 2023-11-08T19:32:21.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.286+0000 7f528b600640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52840c3a50 con 0x7f528406a120 2023-11-08T19:32:21.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f528415f4c0 0x7f528406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f528415f4c0 0x7f528406f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39980/0 (socket says 172.21.15.119:39980) 2023-11-08T19:32:21.291 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 -- 172.21.15.119:0/3917823632 learned_addr learned my addr 172.21.15.119:0/3917823632 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:21.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f528939c640 1 --1- 172.21.15.119:0/3917823632 >> v1:172.21.15.119:6789/0 conn(0x7f52841447a0 0x7f52840cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57878/0 (socket says 172.21.15.119:57878) 2023-11-08T19:32:21.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5289b9d640 1 --2- 172.21.15.119:0/3917823632 >> v2:172.21.15.188:3300/0 conn(0x7f528406a120 0x7f528406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.292 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 -- 172.21.15.119:0/3917823632 >> v2:172.21.15.188:3300/0 conn(0x7f528406a120 msgr2=0x7f528406a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:21.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 --2- 172.21.15.119:0/3917823632 >> v2:172.21.15.188:3300/0 conn(0x7f528406a120 0x7f528406a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 -- 172.21.15.119:0/3917823632 >> v1:172.21.15.119:6789/0 conn(0x7f52841447a0 legacy=0x7f52840cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.287+0000 7f5288b9b640 1 -- 172.21.15.119:0/3917823632 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5284144b80 con 0x7f528415f4c0 2023-11-08T19:32:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.289+0000 7f5288b9b640 1 --2- 172.21.15.119:0/3917823632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528415f4c0 0x7f528406f930 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f52840b1850 tx=0x7f527402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8359e7e0fa11e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.289+0000 7f527bfff640 1 -- 172.21.15.119:0/3917823632 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274030860 con 0x7f528415f4c0 2023-11-08T19:32:21.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.289+0000 7f527bfff640 1 -- 172.21.15.119:0/3917823632 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52740309c0 con 0x7f528415f4c0 2023-11-08T19:32:21.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.289+0000 7f527bfff640 1 -- 172.21.15.119:0/3917823632 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274030d50 con 0x7f528415f4c0 2023-11-08T19:32:21.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 -- 172.21.15.119:0/3917823632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528415f4c0 msgr2=0x7f528406f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 --2- 172.21.15.119:0/3917823632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528415f4c0 0x7f528406f930 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f52840b1850 tx=0x7f527402b9a0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 -- 172.21.15.119:0/3917823632 shutdown_connections 2023-11-08T19:32:21.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 --2- 172.21.15.119:0/3917823632 >> v2:172.21.15.188:3300/0 conn(0x7f528406a120 0x7f528406a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 --2- 172.21.15.119:0/3917823632 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528415f4c0 0x7f528406f930 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 -- 172.21.15.119:0/3917823632 >> 172.21.15.119:0/3917823632 conn(0x7f52840cb610 msgr2=0x7f52840cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:21.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 -- 172.21.15.119:0/3917823632 shutdown_connections 2023-11-08T19:32:21.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.290+0000 7f528b600640 1 -- 172.21.15.119:0/3917823632 wait complete. 2023-11-08T19:32:21.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.292+0000 7f528b600640 1 Processor -- start 2023-11-08T19:32:21.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.292+0000 7f528b600640 1 -- start start 2023-11-08T19:32:21.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528b600640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 0x7f5284140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528b600640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52841447a0 0x7f52840b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528b600640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f528415f4c0 0x7f5284143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528b600640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52840c6f70 con 0x7f528406a120 2023-11-08T19:32:21.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528b600640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52840b2c40 con 0x7f528415f4c0 2023-11-08T19:32:21.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528b600640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52840c7210 con 0x7f52841447a0 2023-11-08T19:32:21.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528939c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 0x7f5284140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528939c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 0x7f5284140180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39986/0 (socket says 172.21.15.119:39986) 2023-11-08T19:32:21.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.293+0000 7f528939c640 1 -- 172.21.15.119:0/2056606756 learned_addr learned my addr 172.21.15.119:0/2056606756 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:21.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f5288b9b640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52841447a0 0x7f52840b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f5289b9d640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f528415f4c0 0x7f5284143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f528939c640 1 -- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52841447a0 msgr2=0x7f52840b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f528939c640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52841447a0 0x7f52840b23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f528939c640 1 -- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f528415f4c0 msgr2=0x7f5284143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f528939c640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f528415f4c0 0x7f5284143500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f528939c640 1 -- 172.21.15.119:0/2056606756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52740094d0 con 0x7f528406a120 2023-11-08T19:32:21.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f528939c640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 0x7f5284140180 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f528000c450 tx=0x7f528000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:21.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.294+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280007480 con 0x7f528406a120 2023-11-08T19:32:21.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.295+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280007600 con 0x7f528406a120 2023-11-08T19:32:21.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.295+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280004a20 con 0x7f528406a120 2023-11-08T19:32:21.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.295+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52840c74f0 con 0x7f528406a120 2023-11-08T19:32:21.306 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.295+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52840b2f20 con 0x7f528406a120 2023-11-08T19:32:21.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.296+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5284074430 con 0x7f528406a120 2023-11-08T19:32:21.307 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.297+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5280004b80 con 0x7f528406a120 2023-11-08T19:32:21.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.298+0000 7f5279ffb640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f526806aea0 0x7f526806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.298+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f5280088c30 con 0x7f528406a120 2023-11-08T19:32:21.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.298+0000 7f5288b9b640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f526806aea0 0x7f526806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.299+0000 7f5288b9b640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f526806aea0 0x7f526806d360 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5274036000 tx=0x7f527403b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:21.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.306+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5280053730 con 0x7f528406a120 2023-11-08T19:32:21.493 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.492+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f528406a900 con 0x7f526806aea0 2023-11-08T19:32:21.502 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.501+0000 7f5279ffb640 1 -- 172.21.15.119:0/2056606756 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290803 (secure 0 0 0) 0x7f528406a900 con 0x7f526806aea0 2023-11-08T19:32:21.505 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:21.507 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:32:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.505+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f526806aea0 msgr2=0x7f526806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.505+0000 7f528b600640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f526806aea0 0x7f526806d360 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f5274036000 tx=0x7f527403b040 comp rx=0 tx=0).stop 2023-11-08T19:32:21.509 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.505+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 msgr2=0x7f5284140180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.505+0000 7f528b600640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 0x7f5284140180 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f528000c450 tx=0x7f528000c910 comp rx=0 tx=0).stop 2023-11-08T19:32:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 shutdown_connections 2023-11-08T19:32:21.510 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f526806aea0 0x7f526806d360 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f528415f4c0 0x7f5284143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52841447a0 0x7f52840b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 --2- 172.21.15.119:0/2056606756 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f528406a120 0x7f5284140180 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.511 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 >> 172.21.15.119:0/2056606756 conn(0x7f52840cb610 msgr2=0x7f52840c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 shutdown_connections 2023-11-08T19:32:21.512 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.506+0000 7f528b600640 1 -- 172.21.15.119:0/2056606756 wait complete. 2023-11-08T19:32:21.523 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-11-08T19:32:20.840051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261084,"kb_used_data":11636,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713636,"statfs":{"total":773094113280,"available":772826763264,"internally_reserved":0,"allocated":11915264,"data_stored":6682222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-78272,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-469,"num_read_kb":-623,"num_write":-182,"num_write_kb":-154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.762732"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082901+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:32:14.082901+0000","last_peered":"2023-11-08T19:32:14.082901+0000","last_clean":"2023-11-08T19:32:14.082901+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:14.082901+0000","last_undegraded":"2023-11-08T19:32:14.082901+0000","last_fullsized":"2023-11-08T19:32:14.082901+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326562+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:32:15.326562+0000","last_peered":"2023-11-08T19:32:15.326562+0000","last_clean":"2023-11-08T19:32:15.326562+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:15.326562+0000","last_undegraded":"2023-11-08T19:32:15.326562+0000","last_fullsized":"2023-11-08T19:32:15.326562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324561+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:32:15.324561+0000","last_peered":"2023-11-08T19:32:15.324561+0000","last_clean":"2023-11-08T19:32:15.324561+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:15.324561+0000","last_undegraded":"2023-11-08T19:32:15.324561+0000","last_fullsized":"2023-11-08T19:32:15.324561+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":43,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771284+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:32:14.771284+0000","last_peered":"2023-11-08T19:32:14.771284+0000","last_clean":"2023-11-08T19:32:14.771284+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:14.771284+0000","last_undegraded":"2023-11-08T19:32:14.771284+0000","last_fullsized":"2023-11-08T19:32:14.771284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326623+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:32:15.326623+0000","last_peered":"2023-11-08T19:32:15.326623+0000","last_clean":"2023-11-08T19:32:15.326623+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:32:15.326623+0000","last_undegraded":"2023-11-08T19:32:15.326623+0000","last_fullsized":"2023-11-08T19:32:15.326623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.084456+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:32:14.084456+0000","last_peered":"2023-11-08T19:32:14.084456+0000","last_clean":"2023-11-08T19:32:14.084456+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:14.084456+0000","last_undegraded":"2023-11-08T19:32:14.084456+0000","last_fullsized":"2023-11-08T19:32:14.084456+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.087661+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:32:14.087661+0000","last_peered":"2023-11-08T19:32:14.087661+0000","last_clean":"2023-11-08T19:32:14.087661+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:32:14.087661+0000","last_undegraded":"2023-11-08T19:32:14.087661+0000","last_fullsized":"2023-11-08T19:32:14.087661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.084090+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:32:14.084090+0000","last_peered":"2023-11-08T19:32:14.084090+0000","last_clean":"2023-11-08T19:32:14.084090+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:14.084090+0000","last_undegraded":"2023-11-08T19:32:14.084090+0000","last_fullsized":"2023-11-08T19:32:14.084090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.327008+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:32:15.327008+0000","last_peered":"2023-11-08T19:32:15.327008+0000","last_clean":"2023-11-08T19:32:15.327008+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:15.327008+0000","last_undegraded":"2023-11-08T19:32:15.327008+0000","last_fullsized":"2023-11-08T19:32:15.327008+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080981+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:32:14.080981+0000","last_peered":"2023-11-08T19:32:14.080981+0000","last_clean":"2023-11-08T19:32:14.080981+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:14.080981+0000","last_undegraded":"2023-11-08T19:32:14.080981+0000","last_fullsized":"2023-11-08T19:32:14.080981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325948+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:32:15.325948+0000","last_peered":"2023-11-08T19:32:15.325948+0000","last_clean":"2023-11-08T19:32:15.325948+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:32:15.325948+0000","last_undegraded":"2023-11-08T19:32:15.325948+0000","last_fullsized":"2023-11-08T19:32:15.325948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324984+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:32:15.324984+0000","last_peered":"2023-11-08T19:32:15.324984+0000","last_clean":"2023-11-08T19:32:15.324984+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:15.324984+0000","last_undegraded":"2023-11-08T19:32:15.324984+0000","last_fullsized":"2023-11-08T19:32:15.324984+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245918+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:32:14.245918+0000","last_peered":"2023-11-08T19:32:14.245918+0000","last_clean":"2023-11-08T19:32:14.245918+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:14.245918+0000","last_undegraded":"2023-11-08T19:32:14.245918+0000","last_fullsized":"2023-11-08T19:32:14.245918+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":50,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771029+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:32:14.771029+0000","last_peered":"2023-11-08T19:32:14.771029+0000","last_clean":"2023-11-08T19:32:14.771029+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:14.771029+0000","last_undegraded":"2023-11-08T19:32:14.771029+0000","last_fullsized":"2023-11-08T19:32:14.771029+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325529+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:32:15.325529+0000","last_peered":"2023-11-08T19:32:15.325529+0000","last_clean":"2023-11-08T19:32:15.325529+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:15.325529+0000","last_undegraded":"2023-11-08T19:32:15.325529+0000","last_fullsized":"2023-11-08T19:32:15.325529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080107+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:32:14.080107+0000","last_peered":"2023-11-08T19:32:14.080107+0000","last_clean":"2023-11-08T19:32:14.080107+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:32:14.080107+0000","last_undegraded":"2023-11-08T19:32:14.080107+0000","last_fullsized":"2023-11-08T19:32:14.080107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324313+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:32:15.324313+0000","last_peered":"2023-11-08T19:32:15.324313+0000","last_clean":"2023-11-08T19:32:15.324313+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:15.324313+0000","last_undegraded":"2023-11-08T19:32:15.324313+0000","last_fullsized":"2023-11-08T19:32:15.324313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079580+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:32:14.079580+0000","last_peered":"2023-11-08T19:32:14.079580+0000","last_clean":"2023-11-08T19:32:14.079580+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:14.079580+0000","last_undegraded":"2023-11-08T19:32:14.079580+0000","last_fullsized":"2023-11-08T19:32:14.079580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323815+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:32:15.323815+0000","last_peered":"2023-11-08T19:32:15.323815+0000","last_clean":"2023-11-08T19:32:15.323815+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:15.323815+0000","last_undegraded":"2023-11-08T19:32:15.323815+0000","last_fullsized":"2023-11-08T19:32:15.323815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079780+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:32:14.079780+0000","last_peered":"2023-11-08T19:32:14.079780+0000","last_clean":"2023-11-08T19:32:14.079780+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:14.079780+0000","last_undegraded":"2023-11-08T19:32:14.079780+0000","last_fullsized":"2023-11-08T19:32:14.079780+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080725+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:32:14.080725+0000","last_peered":"2023-11-08T19:32:14.080725+0000","last_clean":"2023-11-08T19:32:14.080725+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:14.080725+0000","last_undegraded":"2023-11-08T19:32:14.080725+0000","last_fullsized":"2023-11-08T19:32:14.080725+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323664+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:32:15.323664+0000","last_peered":"2023-11-08T19:32:15.323664+0000","last_clean":"2023-11-08T19:32:15.323664+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:15.323664+0000","last_undegraded":"2023-11-08T19:32:15.323664+0000","last_fullsized":"2023-11-08T19:32:15.323664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083600+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:32:14.083600+0000","last_peered":"2023-11-08T19:32:14.083600+0000","last_clean":"2023-11-08T19:32:14.083600+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:14.083600+0000","last_undegraded":"2023-11-08T19:32:14.083600+0000","last_fullsized":"2023-11-08T19:32:14.083600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086612+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:32:14.086612+0000","last_peered":"2023-11-08T19:32:14.086612+0000","last_clean":"2023-11-08T19:32:14.086612+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:14.086612+0000","last_undegraded":"2023-11-08T19:32:14.086612+0000","last_fullsized":"2023-11-08T19:32:14.086612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326896+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:32:15.326896+0000","last_peered":"2023-11-08T19:32:15.326896+0000","last_clean":"2023-11-08T19:32:15.326896+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:32:15.326896+0000","last_undegraded":"2023-11-08T19:32:15.326896+0000","last_fullsized":"2023-11-08T19:32:15.326896+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088580+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:32:14.088580+0000","last_peered":"2023-11-08T19:32:14.088580+0000","last_clean":"2023-11-08T19:32:14.088580+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:14.088580+0000","last_undegraded":"2023-11-08T19:32:14.088580+0000","last_fullsized":"2023-11-08T19:32:14.088580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246666+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:32:14.246666+0000","last_peered":"2023-11-08T19:32:14.246666+0000","last_clean":"2023-11-08T19:32:14.246666+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:14.246666+0000","last_undegraded":"2023-11-08T19:32:14.246666+0000","last_fullsized":"2023-11-08T19:32:14.246666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088147+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:32:14.088147+0000","last_peered":"2023-11-08T19:32:14.088147+0000","last_clean":"2023-11-08T19:32:14.088147+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:32:14.088147+0000","last_undegraded":"2023-11-08T19:32:14.088147+0000","last_fullsized":"2023-11-08T19:32:14.088147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326448+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:32:15.326448+0000","last_peered":"2023-11-08T19:32:15.326448+0000","last_clean":"2023-11-08T19:32:15.326448+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:32:15.326448+0000","last_undegraded":"2023-11-08T19:32:15.326448+0000","last_fullsized":"2023-11-08T19:32:15.326448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771506+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:32:14.771506+0000","last_peered":"2023-11-08T19:32:14.771506+0000","last_clean":"2023-11-08T19:32:14.771506+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:14.771506+0000","last_undegraded":"2023-11-08T19:32:14.771506+0000","last_fullsized":"2023-11-08T19:32:14.771506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246330+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:32:14.246330+0000","last_peered":"2023-11-08T19:32:14.246330+0000","last_clean":"2023-11-08T19:32:14.246330+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:32:14.246330+0000","last_undegraded":"2023-11-08T19:32:14.246330+0000","last_fullsized":"2023-11-08T19:32:14.246330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081842+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:32:14.081842+0000","last_peered":"2023-11-08T19:32:14.081842+0000","last_clean":"2023-11-08T19:32:14.081842+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:14.081842+0000","last_undegraded":"2023-11-08T19:32:14.081842+0000","last_fullsized":"2023-11-08T19:32:14.081842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246823+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:32:14.246823+0000","last_peered":"2023-11-08T19:32:14.246823+0000","last_clean":"2023-11-08T19:32:14.246823+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:14.246823+0000","last_undegraded":"2023-11-08T19:32:14.246823+0000","last_fullsized":"2023-11-08T19:32:14.246823+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325340+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:32:15.325340+0000","last_peered":"2023-11-08T19:32:15.325340+0000","last_clean":"2023-11-08T19:32:15.325340+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:32:15.325340+0000","last_undegraded":"2023-11-08T19:32:15.325340+0000","last_fullsized":"2023-11-08T19:32:15.325340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080658+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:32:14.080658+0000","last_peered":"2023-11-08T19:32:14.080658+0000","last_clean":"2023-11-08T19:32:14.080658+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:14.080658+0000","last_undegraded":"2023-11-08T19:32:14.080658+0000","last_fullsized":"2023-11-08T19:32:14.080658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088790+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:32:14.088790+0000","last_peered":"2023-11-08T19:32:14.088790+0000","last_clean":"2023-11-08T19:32:14.088790+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:14.088790+0000","last_undegraded":"2023-11-08T19:32:14.088790+0000","last_fullsized":"2023-11-08T19:32:14.088790+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085436+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:32:14.085436+0000","last_peered":"2023-11-08T19:32:14.085436+0000","last_clean":"2023-11-08T19:32:14.085436+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:14.085436+0000","last_undegraded":"2023-11-08T19:32:14.085436+0000","last_fullsized":"2023-11-08T19:32:14.085436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086428+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:32:14.086428+0000","last_peered":"2023-11-08T19:32:14.086428+0000","last_clean":"2023-11-08T19:32:14.086428+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:14.086428+0000","last_undegraded":"2023-11-08T19:32:14.086428+0000","last_fullsized":"2023-11-08T19:32:14.086428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245467+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:32:14.245467+0000","last_peered":"2023-11-08T19:32:14.245467+0000","last_clean":"2023-11-08T19:32:14.245467+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:14.245467+0000","last_undegraded":"2023-11-08T19:32:14.245467+0000","last_fullsized":"2023-11-08T19:32:14.245467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324366+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:32:15.324366+0000","last_peered":"2023-11-08T19:32:15.324366+0000","last_clean":"2023-11-08T19:32:15.324366+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:15.324366+0000","last_undegraded":"2023-11-08T19:32:15.324366+0000","last_fullsized":"2023-11-08T19:32:15.324366+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085906+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:32:14.085906+0000","last_peered":"2023-11-08T19:32:14.085906+0000","last_clean":"2023-11-08T19:32:14.085906+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:14.085906+0000","last_undegraded":"2023-11-08T19:32:14.085906+0000","last_fullsized":"2023-11-08T19:32:14.085906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.770306+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:32:14.770306+0000","last_peered":"2023-11-08T19:32:14.770306+0000","last_clean":"2023-11-08T19:32:14.770306+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:14.770306+0000","last_undegraded":"2023-11-08T19:32:14.770306+0000","last_fullsized":"2023-11-08T19:32:14.770306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":43,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.769959+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:32:14.769959+0000","last_peered":"2023-11-08T19:32:14.769959+0000","last_clean":"2023-11-08T19:32:14.769959+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:14.769959+0000","last_undegraded":"2023-11-08T19:32:14.769959+0000","last_fullsized":"2023-11-08T19:32:14.769959+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323721+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:32:15.323721+0000","last_peered":"2023-11-08T19:32:15.323721+0000","last_clean":"2023-11-08T19:32:15.323721+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:15.323721+0000","last_undegraded":"2023-11-08T19:32:15.323721+0000","last_fullsized":"2023-11-08T19:32:15.323721+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080836+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:32:14.080836+0000","last_peered":"2023-11-08T19:32:14.080836+0000","last_clean":"2023-11-08T19:32:14.080836+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:14.080836+0000","last_undegraded":"2023-11-08T19:32:14.080836+0000","last_fullsized":"2023-11-08T19:32:14.080836+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245560+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:32:14.245560+0000","last_peered":"2023-11-08T19:32:14.245560+0000","last_clean":"2023-11-08T19:32:14.245560+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:14.245560+0000","last_undegraded":"2023-11-08T19:32:14.245560+0000","last_fullsized":"2023-11-08T19:32:14.245560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079989+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:32:14.079989+0000","last_peered":"2023-11-08T19:32:14.079989+0000","last_clean":"2023-11-08T19:32:14.079989+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:14.079989+0000","last_undegraded":"2023-11-08T19:32:14.079989+0000","last_fullsized":"2023-11-08T19:32:14.079989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325680+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:32:15.325680+0000","last_peered":"2023-11-08T19:32:15.325680+0000","last_clean":"2023-11-08T19:32:15.325680+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:15.325680+0000","last_undegraded":"2023-11-08T19:32:15.325680+0000","last_fullsized":"2023-11-08T19:32:15.325680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246594+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:32:14.246594+0000","last_peered":"2023-11-08T19:32:14.246594+0000","last_clean":"2023-11-08T19:32:14.246594+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:32:14.246594+0000","last_undegraded":"2023-11-08T19:32:14.246594+0000","last_fullsized":"2023-11-08T19:32:14.246594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324910+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:32:15.324910+0000","last_peered":"2023-11-08T19:32:15.324910+0000","last_clean":"2023-11-08T19:32:15.324910+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:15.324910+0000","last_undegraded":"2023-11-08T19:32:15.324910+0000","last_fullsized":"2023-11-08T19:32:15.324910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326226+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:32:15.326226+0000","last_peered":"2023-11-08T19:32:15.326226+0000","last_clean":"2023-11-08T19:32:15.326226+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:15.326226+0000","last_undegraded":"2023-11-08T19:32:15.326226+0000","last_fullsized":"2023-11-08T19:32:15.326226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088471+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:32:14.088471+0000","last_peered":"2023-11-08T19:32:14.088471+0000","last_clean":"2023-11-08T19:32:14.088471+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:14.088471+0000","last_undegraded":"2023-11-08T19:32:14.088471+0000","last_fullsized":"2023-11-08T19:32:14.088471+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325063+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:32:15.325063+0000","last_peered":"2023-11-08T19:32:15.325063+0000","last_clean":"2023-11-08T19:32:15.325063+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:15.325063+0000","last_undegraded":"2023-11-08T19:32:15.325063+0000","last_fullsized":"2023-11-08T19:32:15.325063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325706+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:32:15.325706+0000","last_peered":"2023-11-08T19:32:15.325706+0000","last_clean":"2023-11-08T19:32:15.325706+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:32:15.325706+0000","last_undegraded":"2023-11-08T19:32:15.325706+0000","last_fullsized":"2023-11-08T19:32:15.325706+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088073+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:32:14.088073+0000","last_peered":"2023-11-08T19:32:14.088073+0000","last_clean":"2023-11-08T19:32:14.088073+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:32:14.088073+0000","last_undegraded":"2023-11-08T19:32:14.088073+0000","last_fullsized":"2023-11-08T19:32:14.088073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325460+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:32:15.325460+0000","last_peered":"2023-11-08T19:32:15.325460+0000","last_clean":"2023-11-08T19:32:15.325460+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:15.325460+0000","last_undegraded":"2023-11-08T19:32:15.325460+0000","last_fullsized":"2023-11-08T19:32:15.325460+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082006+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:32:14.082006+0000","last_peered":"2023-11-08T19:32:14.082006+0000","last_clean":"2023-11-08T19:32:14.082006+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:14.082006+0000","last_undegraded":"2023-11-08T19:32:14.082006+0000","last_fullsized":"2023-11-08T19:32:14.082006+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325608+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:32:15.325608+0000","last_peered":"2023-11-08T19:32:15.325608+0000","last_clean":"2023-11-08T19:32:15.325608+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:32:15.325608+0000","last_undegraded":"2023-11-08T19:32:15.325608+0000","last_fullsized":"2023-11-08T19:32:15.325608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325825+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:32:15.325825+0000","last_peered":"2023-11-08T19:32:15.325825+0000","last_clean":"2023-11-08T19:32:15.325825+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:32:15.325825+0000","last_undegraded":"2023-11-08T19:32:15.325825+0000","last_fullsized":"2023-11-08T19:32:15.325825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.970299+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:32:14.970299+0000","last_peered":"2023-11-08T19:32:14.970299+0000","last_clean":"2023-11-08T19:32:14.970299+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:14.970299+0000","last_undegraded":"2023-11-08T19:32:14.970299+0000","last_fullsized":"2023-11-08T19:32:14.970299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080424+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:32:14.080424+0000","last_peered":"2023-11-08T19:32:14.080424+0000","last_clean":"2023-11-08T19:32:14.080424+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:14.080424+0000","last_undegraded":"2023-11-08T19:32:14.080424+0000","last_fullsized":"2023-11-08T19:32:14.080424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086594+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:32:14.086594+0000","last_peered":"2023-11-08T19:32:14.086594+0000","last_clean":"2023-11-08T19:32:14.086594+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:14.086594+0000","last_undegraded":"2023-11-08T19:32:14.086594+0000","last_fullsized":"2023-11-08T19:32:14.086594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081053+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:32:14.081053+0000","last_peered":"2023-11-08T19:32:14.081053+0000","last_clean":"2023-11-08T19:32:14.081053+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:14.081053+0000","last_undegraded":"2023-11-08T19:32:14.081053+0000","last_fullsized":"2023-11-08T19:32:14.081053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082613+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:32:14.082613+0000","last_peered":"2023-11-08T19:32:14.082613+0000","last_clean":"2023-11-08T19:32:14.082613+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:14.082613+0000","last_undegraded":"2023-11-08T19:32:14.082613+0000","last_fullsized":"2023-11-08T19:32:14.082613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080615+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:32:14.080615+0000","last_peered":"2023-11-08T19:32:14.080615+0000","last_clean":"2023-11-08T19:32:14.080615+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:14.080615+0000","last_undegraded":"2023-11-08T19:32:14.080615+0000","last_fullsized":"2023-11-08T19:32:14.080615+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968861+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:32:14.968861+0000","last_peered":"2023-11-08T19:32:14.968861+0000","last_clean":"2023-11-08T19:32:14.968861+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:14.968861+0000","last_undegraded":"2023-11-08T19:32:14.968861+0000","last_fullsized":"2023-11-08T19:32:14.968861+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325069+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:32:15.325069+0000","last_peered":"2023-11-08T19:32:15.325069+0000","last_clean":"2023-11-08T19:32:15.325069+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:32:15.325069+0000","last_undegraded":"2023-11-08T19:32:15.325069+0000","last_fullsized":"2023-11-08T19:32:15.325069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083582+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:32:14.083582+0000","last_peered":"2023-11-08T19:32:14.083582+0000","last_clean":"2023-11-08T19:32:14.083582+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:14.083582+0000","last_undegraded":"2023-11-08T19:32:14.083582+0000","last_fullsized":"2023-11-08T19:32:14.083582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324550+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:32:15.324550+0000","last_peered":"2023-11-08T19:32:15.324550+0000","last_clean":"2023-11-08T19:32:15.324550+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:15.324550+0000","last_undegraded":"2023-11-08T19:32:15.324550+0000","last_fullsized":"2023-11-08T19:32:15.324550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245784+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:32:14.245784+0000","last_peered":"2023-11-08T19:32:14.245784+0000","last_clean":"2023-11-08T19:32:14.245784+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:14.245784+0000","last_undegraded":"2023-11-08T19:32:14.245784+0000","last_fullsized":"2023-11-08T19:32:14.245784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325075+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:32:15.325075+0000","last_peered":"2023-11-08T19:32:15.325075+0000","last_clean":"2023-11-08T19:32:15.325075+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:15.325075+0000","last_undegraded":"2023-11-08T19:32:15.325075+0000","last_fullsized":"2023-11-08T19:32:15.325075+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771424+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:32:14.771424+0000","last_peered":"2023-11-08T19:32:14.771424+0000","last_clean":"2023-11-08T19:32:14.771424+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:14.771424+0000","last_undegraded":"2023-11-08T19:32:14.771424+0000","last_fullsized":"2023-11-08T19:32:14.771424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326894+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:32:15.326894+0000","last_peered":"2023-11-08T19:32:15.326894+0000","last_clean":"2023-11-08T19:32:15.326894+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:32:15.326894+0000","last_undegraded":"2023-11-08T19:32:15.326894+0000","last_fullsized":"2023-11-08T19:32:15.326894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258452+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:32:16.258452+0000","last_peered":"2023-11-08T19:32:16.258452+0000","last_clean":"2023-11-08T19:32:16.258452+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:16.258452+0000","last_undegraded":"2023-11-08T19:32:16.258452+0000","last_fullsized":"2023-11-08T19:32:16.258452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246884+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:32:14.246884+0000","last_peered":"2023-11-08T19:32:14.246884+0000","last_clean":"2023-11-08T19:32:14.246884+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:14.246884+0000","last_undegraded":"2023-11-08T19:32:14.246884+0000","last_fullsized":"2023-11-08T19:32:14.246884+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245706+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:32:14.245706+0000","last_peered":"2023-11-08T19:32:14.245706+0000","last_clean":"2023-11-08T19:32:14.245706+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:14.245706+0000","last_undegraded":"2023-11-08T19:32:14.245706+0000","last_fullsized":"2023-11-08T19:32:14.245706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":54,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.255292+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:32:16.255292+0000","last_peered":"2023-11-08T19:32:16.255292+0000","last_clean":"2023-11-08T19:32:16.255292+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:32:16.255292+0000","last_undegraded":"2023-11-08T19:32:16.255292+0000","last_fullsized":"2023-11-08T19:32:16.255292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081818+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:32:14.081818+0000","last_peered":"2023-11-08T19:32:14.081818+0000","last_clean":"2023-11-08T19:32:14.081818+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:14.081818+0000","last_undegraded":"2023-11-08T19:32:14.081818+0000","last_fullsized":"2023-11-08T19:32:14.081818+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.087868+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:32:14.087868+0000","last_peered":"2023-11-08T19:32:14.087868+0000","last_clean":"2023-11-08T19:32:14.087868+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:14.087868+0000","last_undegraded":"2023-11-08T19:32:14.087868+0000","last_fullsized":"2023-11-08T19:32:14.087868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.256957+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:32:16.256957+0000","last_peered":"2023-11-08T19:32:16.256957+0000","last_clean":"2023-11-08T19:32:16.256957+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:16.256957+0000","last_undegraded":"2023-11-08T19:32:16.256957+0000","last_fullsized":"2023-11-08T19:32:16.256957+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.327036+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:32:15.327036+0000","last_peered":"2023-11-08T19:32:15.327036+0000","last_clean":"2023-11-08T19:32:15.327036+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:15.327036+0000","last_undegraded":"2023-11-08T19:32:15.327036+0000","last_fullsized":"2023-11-08T19:32:15.327036+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083682+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:32:14.083682+0000","last_peered":"2023-11-08T19:32:14.083682+0000","last_clean":"2023-11-08T19:32:14.083682+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:14.083682+0000","last_undegraded":"2023-11-08T19:32:14.083682+0000","last_fullsized":"2023-11-08T19:32:14.083682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968622+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:32:14.968622+0000","last_peered":"2023-11-08T19:32:14.968622+0000","last_clean":"2023-11-08T19:32:14.968622+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:14.968622+0000","last_undegraded":"2023-11-08T19:32:14.968622+0000","last_fullsized":"2023-11-08T19:32:14.968622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085702+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:32:14.085702+0000","last_peered":"2023-11-08T19:32:14.085702+0000","last_clean":"2023-11-08T19:32:14.085702+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:14.085702+0000","last_undegraded":"2023-11-08T19:32:14.085702+0000","last_fullsized":"2023-11-08T19:32:14.085702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968748+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:32:14.968748+0000","last_peered":"2023-11-08T19:32:14.968748+0000","last_clean":"2023-11-08T19:32:14.968748+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:14.968748+0000","last_undegraded":"2023-11-08T19:32:14.968748+0000","last_fullsized":"2023-11-08T19:32:14.968748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":51,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.252776+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:32:16.252776+0000","last_peered":"2023-11-08T19:32:16.252776+0000","last_clean":"2023-11-08T19:32:16.252776+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:16.252776+0000","last_undegraded":"2023-11-08T19:32:16.252776+0000","last_fullsized":"2023-11-08T19:32:16.252776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081711+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:32:14.081711+0000","last_peered":"2023-11-08T19:32:14.081711+0000","last_clean":"2023-11-08T19:32:14.081711+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:14.081711+0000","last_undegraded":"2023-11-08T19:32:14.081711+0000","last_fullsized":"2023-11-08T19:32:14.081711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082992+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:32:14.082992+0000","last_peered":"2023-11-08T19:32:14.082992+0000","last_clean":"2023-11-08T19:32:14.082992+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:14.082992+0000","last_undegraded":"2023-11-08T19:32:14.082992+0000","last_fullsized":"2023-11-08T19:32:14.082992+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.769987+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:32:14.769987+0000","last_peered":"2023-11-08T19:32:14.769987+0000","last_clean":"2023-11-08T19:32:14.769987+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:14.769987+0000","last_undegraded":"2023-11-08T19:32:14.769987+0000","last_fullsized":"2023-11-08T19:32:14.769987+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771136+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:32:14.771136+0000","last_peered":"2023-11-08T19:32:14.771136+0000","last_clean":"2023-11-08T19:32:14.771136+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:14.771136+0000","last_undegraded":"2023-11-08T19:32:14.771136+0000","last_fullsized":"2023-11-08T19:32:14.771136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968697+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:32:14.968697+0000","last_peered":"2023-11-08T19:32:14.968697+0000","last_clean":"2023-11-08T19:32:14.968697+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:14.968697+0000","last_undegraded":"2023-11-08T19:32:14.968697+0000","last_fullsized":"2023-11-08T19:32:14.968697+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968890+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:32:14.968890+0000","last_peered":"2023-11-08T19:32:14.968890+0000","last_clean":"2023-11-08T19:32:14.968890+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:14.968890+0000","last_undegraded":"2023-11-08T19:32:14.968890+0000","last_fullsized":"2023-11-08T19:32:14.968890+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771554+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:32:14.771554+0000","last_peered":"2023-11-08T19:32:14.771554+0000","last_clean":"2023-11-08T19:32:14.771554+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:14.771554+0000","last_undegraded":"2023-11-08T19:32:14.771554+0000","last_fullsized":"2023-11-08T19:32:14.771554+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325205+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:32:15.325205+0000","last_peered":"2023-11-08T19:32:15.325205+0000","last_clean":"2023-11-08T19:32:15.325205+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:15.325205+0000","last_undegraded":"2023-11-08T19:32:15.325205+0000","last_fullsized":"2023-11-08T19:32:15.325205+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":52,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.257983+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:32:16.257983+0000","last_peered":"2023-11-08T19:32:16.257983+0000","last_clean":"2023-11-08T19:32:16.257983+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:16.257983+0000","last_undegraded":"2023-11-08T19:32:16.257983+0000","last_fullsized":"2023-11-08T19:32:16.257983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080613+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:32:14.080613+0000","last_peered":"2023-11-08T19:32:14.080613+0000","last_clean":"2023-11-08T19:32:14.080613+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:14.080613+0000","last_undegraded":"2023-11-08T19:32:14.080613+0000","last_fullsized":"2023-11-08T19:32:14.080613+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246364+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:32:14.246364+0000","last_peered":"2023-11-08T19:32:14.246364+0000","last_clean":"2023-11-08T19:32:14.246364+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:14.246364+0000","last_undegraded":"2023-11-08T19:32:14.246364+0000","last_fullsized":"2023-11-08T19:32:14.246364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1929216,"data_stored":1273881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.77500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.599,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.252},"min":{"1min":0.72799999999999998,"5min":0.58599999999999997,"15min":0.58499999999999996},"max":{"1min":1.6619999999999999,"5min":2.1360000000000001,"15min":4.4589999999999996},"last":1.3560000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2010000000000001,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.548,"5min":1.728,"15min":4.5330000000000004},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.1910000000000001,"15min":1.1459999999999999},"min":{"1min":0.83299999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6160000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.2030000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.335,"15min":1.448},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5660000000000001,"5min":1.891,"15min":4.6980000000000004},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.194,"15min":1.3160000000000001},"min":{"1min":0.97999999999999998,"5min":0.67200000000000004,"15min":0.64700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":4.4269999999999996},"last":1.5780000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.294,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.7889999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.409,"15min":1.5149999999999999},"min":{"1min":0.95199999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.768,"5min":2.2200000000000002,"15min":4.6150000000000002},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.327},"min":{"1min":1.006,"5min":0.624,"15min":0.624},"max":{"1min":1.49,"5min":1.798,"15min":2.3170000000000002},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.323,"15min":1.4410000000000001},"min":{"1min":0.93200000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7909999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.403}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.95999999999999996,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.673,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.508},{"interface":"front","average":{"1min":1.379,"5min":1.363,"15min":1.4470000000000001},"min":{"1min":1.048,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.649,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":1.22,"5min":0.878,"15min":0.878},"max":{"1min":1.8109999999999999,"5min":1.9770000000000001,"15min":2.427},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.474,"15min":1.581},"min":{"1min":1.1180000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.98,"5min":2.1920000000000002,"15min":2.468},"last":1.5489999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.0940000000000001,"15min":0.98799999999999999},"min":{"1min":0.85399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.536,"5min":1.6279999999999999,"15min":1.752},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.202,"15min":1.1399999999999999},"min":{"1min":0.80200000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":8.391,"5min":8.391,"15min":8.391},"last":0.752}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.161,"15min":1.113},"min":{"1min":0.85599999999999998,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.679},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2050000000000001,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.665,"5min":1.665,"15min":4.6399999999999997},"last":1.099}]},{"osd":2,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.80300000000000005,"5min":0.68799999999999994,"15min":0.54600000000000004},"max":{"1min":1.4830000000000001,"5min":1.655,"15min":4.609},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2949999999999999,"15min":1.242},"min":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.861},"last":1.141}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.276,"15min":1.4239999999999999},"min":{"1min":1.0209999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.756,"15min":4.7190000000000003},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.256,"15min":1.2050000000000001},"min":{"1min":0.81200000000000006,"5min":0.625,"15min":0.625},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.784},"last":0.98299999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.83199999999999996,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.859,"5min":1.881,"15min":2.0449999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4770000000000001,"15min":1.4910000000000001},"min":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.998,"15min":2.3130000000000002},"last":1.5029999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.325,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.611,"5min":1.835,"15min":1.9079999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3999999999999999,"15min":1.353},"min":{"1min":0.92100000000000004,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":1.6819999999999999,"5min":1.8660000000000001,"15min":2.0489999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5569999999999999,"15min":1.55},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":1.99,"5min":1.99,"15min":2.2519999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4970000000000001,"15min":1.464},"min":{"1min":0.75700000000000001,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":1.7809999999999999,"5min":1.9870000000000001,"15min":3.0640000000000001},"last":1.4219999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1970000000000001,"15min":1.117},"min":{"1min":0.69599999999999995,"5min":0.59999999999999998,"15min":0.57699999999999996},"max":{"1min":1.603,"5min":1.724,"15min":1.76},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.274,"15min":1.252},"min":{"1min":0.77600000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.837,"5min":1.954,"15min":1.954},"last":1.6259999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.754,"5min":0.753,"15min":0.58699999999999997},"max":{"1min":1.552,"5min":1.6359999999999999,"15min":2.3039999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.179,"5min":1.1679999999999999,"15min":1.355},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.115}]},{"osd":2,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2350000000000001,"15min":1.194},"min":{"1min":0.82599999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5720000000000001,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.173,"15min":1.3959999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6990000000000001,"5min":1.806,"15min":7.6479999999999997},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.74,"5min":1.804,"15min":2.1920000000000002},"last":1.161},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3500000000000001,"15min":1.323},"min":{"1min":0.98799999999999999,"5min":0.91300000000000003,"15min":0.69699999999999995},"max":{"1min":1.6910000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.196,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7350000000000001,"5min":1.7969999999999999,"15min":2.347},"last":2.0430000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.347,"15min":1.488},"min":{"1min":0.94099999999999995,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.7350000000000001,"5min":2.097,"15min":3.153},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3859999999999999,"15min":1.4830000000000001},"min":{"1min":0.92100000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.923,"5min":1.923,"15min":2.8650000000000002},"last":1.448},{"interface":"front","average":{"1min":1.399,"5min":1.4019999999999999,"15min":1.47},"min":{"1min":0.94199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6990000000000001,"5min":1.899,"15min":2.6190000000000002},"last":1.742}]},{"osd":7,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.6359999999999999},"min":{"1min":1.0980000000000001,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":1.9259999999999999,"5min":2.0630000000000002,"15min":3.2730000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5229999999999999,"15min":1.5680000000000001},"min":{"1min":1.204,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.8600000000000001,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.7030000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1479999999999999,"15min":1.0760000000000001},"min":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.5900000000000001,"5min":1.782,"15min":2.5219999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.83499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.524,"5min":1.696,"15min":7.0119999999999996},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.0680000000000001,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.63100000000000001},"max":{"1min":1.752,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.206},{"interface":"front","average":{"1min":1.177,"5min":1.1499999999999999,"15min":1.1519999999999999},"min":{"1min":0.748,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.7310000000000001,"5min":1.7450000000000001,"15min":2.9940000000000002},"last":1.3879999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2490000000000001,"15min":1.3160000000000001},"min":{"1min":0.98199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.988,"5min":1.988,"15min":4.0410000000000004},"last":1.046},{"interface":"front","average":{"1min":1.341,"5min":1.319,"15min":1.3879999999999999},"min":{"1min":0.95999999999999996,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":3.2069999999999999},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1659999999999999,"15min":1.258},"min":{"1min":0.84199999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5169999999999999,"5min":1.696,"15min":4.4980000000000002},"last":1.105},{"interface":"front","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.345},"min":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":1.738,"15min":3.5710000000000002},"last":1.429}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.4330000000000001},"min":{"1min":0.81000000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.593,"5min":1.762,"15min":4.2350000000000003},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2549999999999999,"15min":1.4590000000000001},"min":{"1min":1.0189999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5800000000000001,"5min":1.8560000000000001,"15min":4.5880000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.365,"15min":1.5009999999999999},"min":{"1min":0.92300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":2.2130000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4419999999999999,"15min":1.5660000000000001},"min":{"1min":1.1639999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.841,"5min":1.9490000000000001,"15min":4.4130000000000003},"last":1.26}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.409,"15min":1.5640000000000001},"min":{"1min":1.137,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":4.6369999999999996},"last":1.325},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5669999999999999,"15min":1.6799999999999999},"min":{"1min":1.2969999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8999999999999999,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.488}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.99399999999999999},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.427,"5min":2.2320000000000002,"15min":3.972},"last":1.355},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93500000000000005,"15min":1.042},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.311},"max":{"1min":1.544,"5min":1.899,"15min":3.1520000000000001},"last":1.7829999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95299999999999996,"15min":1.1160000000000001},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.4299999999999999,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":1.0429999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8520000000000001,"15min":4.5750000000000002},"last":0.879}]},{"osd":2,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.153,"15min":1.272},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.619,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1020000000000001,"15min":1.218},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.6339999999999999,"5min":1.754,"15min":2.73},"last":1.294}]},{"osd":4,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.214,"15min":1.3779999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6599999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.24,"15min":1.3899999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9039999999999999,"5min":2.0449999999999999,"15min":3.052},"last":1.4530000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.4370000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":2.0430000000000001,"15min":3.0899999999999999},"last":1.9259999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.29,"15min":1.4570000000000001},"min":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.597,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.234}]},{"osd":6,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2769999999999999,"15min":1.3819999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7,"5min":3.21,"15min":4.2220000000000004},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.391,"15min":1.526},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.0880000000000001,"15min":3.1230000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.399,"15min":1.6160000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.925,"5min":2.1150000000000002,"15min":4.407},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.5209999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.671,"5min":2.0059999999999998,"15min":4.3849999999999998},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":674557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83699999999999997,"15min":0.85399999999999998},"min":{"1min":0.61699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2370000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0720000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.56799999999999995,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.371,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.66700000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.474,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.093,"15min":1.0620000000000001},"min":{"1min":0.80300000000000005,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.052,"15min":1.103},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.675,"5min":1.7330000000000001,"15min":1.7669999999999999},"last":1.1240000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2829999999999999,"15min":1.365},"min":{"1min":0.95499999999999996,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":1.883,"15min":3.2959999999999998},"last":1.268},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.284,"15min":1.3400000000000001},"min":{"1min":1.0609999999999999,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":1.704,"15min":2.278},"last":1.556}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.325,"15min":1.3899999999999999},"min":{"1min":1.1359999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.327,"15min":1.4179999999999999},"min":{"1min":1.048,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.7709999999999999,"5min":1.954,"15min":2.536},"last":1.4350000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":1.1120000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.657},{"interface":"front","average":{"1min":1.377,"5min":1.351,"15min":1.421},"min":{"1min":0.92200000000000004,"5min":0.83799999999999997,"15min":0.70099999999999996},"max":{"1min":1.9810000000000001,"5min":2.036,"15min":2.0910000000000002},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.409,"15min":1.522},"min":{"1min":0.95099999999999996,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":1.74,"5min":1.9339999999999999,"15min":2.6549999999999998},"last":1.323},{"interface":"front","average":{"1min":1.516,"5min":1.466,"15min":1.5269999999999999},"min":{"1min":1.111,"5min":0.84999999999999998,"15min":0.76600000000000001},"max":{"1min":1.8009999999999999,"5min":2.226,"15min":2.3439999999999999},"last":1.6080000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85599999999999998,"15min":0.78000000000000003},"min":{"1min":0.59199999999999997,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.284,"5min":1.327,"15min":1.327},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.88800000000000001,"15min":0.79900000000000004},"min":{"1min":0.82499999999999996,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":1.528,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1359999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84099999999999997,"15min":0.81100000000000005},"min":{"1min":0.51100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.256,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0249999999999999,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.47499999999999998,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.028}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2569999999999999,"15min":1.1240000000000001},"min":{"1min":0.80100000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.748,"5min":1.978,"15min":1.978},"last":1.302},{"interface":"front","average":{"1min":1.196,"5min":1.1439999999999999,"15min":1.1020000000000001},"min":{"1min":0.65300000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.6930000000000001,"5min":2.2690000000000001,"15min":2.5369999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.157},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.476,"5min":1.8620000000000001,"15min":2.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.294,"5min":1.2729999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.6870000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.1859999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.93799999999999994,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2689999999999999,"15min":1.3069999999999999},"min":{"1min":0.95199999999999996,"5min":0.73199999999999998,"15min":0.629},"max":{"1min":1.696,"5min":1.748,"15min":2.4089999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.375},"min":{"1min":1.194,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.7450000000000001,"5min":2.1509999999999998,"15min":2.6019999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.355,"15min":1.373},"min":{"1min":1.133,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.282},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.3560000000000001},"min":{"1min":1.0029999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.409,"15min":1.395},"min":{"1min":1.0940000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.865,"5min":2.4830000000000001,"15min":2.6699999999999999},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":1986560,"data_stored":1316219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.63200000000000001,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.0780000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2110000000000001,"5min":1.4770000000000001,"15min":1.7},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.876},"min":{"1min":0.57399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.319,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.998,"15min":0.98299999999999998},"min":{"1min":0.76200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5189999999999999,"5min":1.54,"15min":1.7270000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.016},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.748},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.089,"5min":1.0660000000000001,"15min":1.0940000000000001},"min":{"1min":0.60599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":1.8400000000000001,"15min":2.496},"last":1.276}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2709999999999999,"15min":1.244},"min":{"1min":0.90600000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6180000000000001,"5min":1.794,"15min":1.9239999999999999},"last":1.373},{"interface":"front","average":{"1min":1.427,"5min":1.29,"15min":1.3939999999999999},"min":{"1min":1.127,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.6399999999999999,"5min":1.706,"15min":2.4039999999999999},"last":1.3420000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.306,"15min":1.351},"min":{"1min":1.042,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7270000000000001,"5min":1.8620000000000001,"15min":2.4369999999999998},"last":1.643},{"interface":"front","average":{"1min":1.319,"5min":1.2949999999999999,"15min":1.3979999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.569,"5min":2.089,"15min":2.4239999999999999},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3500000000000001,"15min":1.4550000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.421},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.357,"15min":1.3959999999999999},"min":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7549999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.474}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.93400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.851,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.415,"15min":1.462},"min":{"1min":1.103,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7909999999999999,"5min":1.881,"15min":2.306},"last":1.524}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:21.529 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T19:32:21.530 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:21.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.760+0000 7fc80290c640 1 Processor -- start 2023-11-08T19:32:21.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.761+0000 7fc80290c640 1 -- start start 2023-11-08T19:32:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc80290c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7fc1447a0 0x7fc7fc15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc80290c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cad30 0x7fc7fc06f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc80290c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7fc0c3a50 con 0x7fc7fc0cb170 2023-11-08T19:32:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc80290c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7fc144b80 con 0x7fc7fc1447a0 2023-11-08T19:32:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc80290c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc0c9150 con 0x7fc7fc0cad30 2023-11-08T19:32:21.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc7fbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7fc1447a0 0x7fc7fc15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc7fbfff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc7fc1447a0 0x7fc7fc15f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:39996/0 (socket says 172.21.15.119:39996) 2023-11-08T19:32:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.762+0000 7fc7fbfff640 1 -- 172.21.15.119:0/3740663155 learned_addr learned my addr 172.21.15.119:0/3740663155 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:21.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.763+0000 7fc7f37fe640 1 --2- 172.21.15.119:0/3740663155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cad30 0x7fc7fc06f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.763+0000 7fc7fbfff640 1 -- 172.21.15.119:0/3740663155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cad30 msgr2=0x7fc7fc06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:21.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.763+0000 7fc7fbfff640 1 --2- 172.21.15.119:0/3740663155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cad30 0x7fc7fc06f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.763+0000 7fc7fbfff640 1 -- 172.21.15.119:0/3740663155 >> v1:172.21.15.188:6789/0 conn(0x7fc7fc0cb170 legacy=0x7fc7fc06a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:21.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.763+0000 7fc7fbfff640 1 -- 172.21.15.119:0/3740663155 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc0c9410 con 0x7fc7fc1447a0 2023-11-08T19:32:21.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.764+0000 7fc7fbfff640 1 --2- 172.21.15.119:0/3740663155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc15f4c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009870 tx=0x7fc7ec02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9af9eb32b343c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:21.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.764+0000 7fc7fb7fe640 1 -- 172.21.15.119:0/3740663155 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec030840 con 0x7fc7fc1447a0 2023-11-08T19:32:21.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.764+0000 7fc7fb7fe640 1 -- 172.21.15.119:0/3740663155 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec0309a0 con 0x7fc7fc1447a0 2023-11-08T19:32:21.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.764+0000 7fc7fb7fe640 1 -- 172.21.15.119:0/3740663155 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec030d30 con 0x7fc7fc1447a0 2023-11-08T19:32:21.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 -- 172.21.15.119:0/3740663155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 msgr2=0x7fc7fc15f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 --2- 172.21.15.119:0/3740663155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc15f4c0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec009870 tx=0x7fc7ec02b980 comp rx=0 tx=0).stop 2023-11-08T19:32:21.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 -- 172.21.15.119:0/3740663155 shutdown_connections 2023-11-08T19:32:21.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 --2- 172.21.15.119:0/3740663155 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cad30 0x7fc7fc06f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 --2- 172.21.15.119:0/3740663155 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc15f4c0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 -- 172.21.15.119:0/3740663155 >> 172.21.15.119:0/3740663155 conn(0x7fc7fc0cb610 msgr2=0x7fc7fc0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:21.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 -- 172.21.15.119:0/3740663155 shutdown_connections 2023-11-08T19:32:21.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.765+0000 7fc80290c640 1 -- 172.21.15.119:0/3740663155 wait complete. 2023-11-08T19:32:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.767+0000 7fc80290c640 1 Processor -- start 2023-11-08T19:32:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.767+0000 7fc80290c640 1 -- start start 2023-11-08T19:32:21.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc80290c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7fc0cad30 0x7fc7fc15a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc80290c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cb170 0x7fc7fc13d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc80290c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc13d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc80290c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc0b4010 con 0x7fc7fc1447a0 2023-11-08T19:32:21.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc80290c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc0b4250 con 0x7fc7fc0cad30 2023-11-08T19:32:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc80290c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc0b4530 con 0x7fc7fc0cb170 2023-11-08T19:32:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc800ea9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc13d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc800ea9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc13d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40000/0 (socket says 172.21.15.119:40000) 2023-11-08T19:32:21.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc800ea9640 1 -- 172.21.15.119:0/3783715799 learned_addr learned my addr 172.21.15.119:0/3783715799 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:21.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc7f37fe640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cb170 0x7fc7fc13d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.768+0000 7fc7fbfff640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7fc0cad30 0x7fc7fc15a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc800ea9640 1 -- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cb170 msgr2=0x7fc7fc13d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc800ea9640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cb170 0x7fc7fc13d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc800ea9640 1 -- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7fc0cad30 msgr2=0x7fc7fc15a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc800ea9640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7fc0cad30 0x7fc7fc15a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc800ea9640 1 -- 172.21.15.119:0/3783715799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec0094d0 con 0x7fc7fc1447a0 2023-11-08T19:32:21.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc7f37fe640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cb170 0x7fc7fc13d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:21.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc800ea9640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc13d8f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c730 tx=0x7fc7f400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:21.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4003b70 con 0x7fc7fc1447a0 2023-11-08T19:32:21.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f4003cd0 con 0x7fc7fc1447a0 2023-11-08T19:32:21.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc0c7690 con 0x7fc7fc1447a0 2023-11-08T19:32:21.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f400aa90 con 0x7fc7fc1447a0 2023-11-08T19:32:21.780 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.769+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc0c7340 con 0x7fc7fc1447a0 2023-11-08T19:32:21.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.770+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7fc0c7500 con 0x7fc7fc1447a0 2023-11-08T19:32:21.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.771+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc7f400abf0 con 0x7fc7fc1447a0 2023-11-08T19:32:21.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.773+0000 7fc7f97fa640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc7dc06ae70 0x7fc7dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:21.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.773+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fc7f4089a00 con 0x7fc7fc1447a0 2023-11-08T19:32:21.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.773+0000 7fc7fbfff640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc7dc06ae70 0x7fc7dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:21.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.774+0000 7fc7fbfff640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc7dc06ae70 0x7fc7dc06d330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec036000 tx=0x7fc7ec03b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:21.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.777+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7f4054500 con 0x7fc7fc1447a0 2023-11-08T19:32:21.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.955+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7fc0c3cf0 con 0x7fc7dc06ae70 2023-11-08T19:32:21.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.964+0000 7fc7f97fa640 1 -- 172.21.15.119:0/3783715799 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290803 (secure 0 0 0) 0x7fc7ec030b90 con 0x7fc7dc06ae70 2023-11-08T19:32:21.969 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:21.970 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:32:21.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.969+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc7dc06ae70 msgr2=0x7fc7dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.969+0000 7fc80290c640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc7dc06ae70 0x7fc7dc06d330 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec036000 tx=0x7fc7ec03b040 comp rx=0 tx=0).stop 2023-11-08T19:32:21.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.969+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 msgr2=0x7fc7fc13d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:21.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.969+0000 7fc80290c640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc13d8f0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c730 tx=0x7fc7f400cbf0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 shutdown_connections 2023-11-08T19:32:21.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc7dc06ae70 0x7fc7dc06d330 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc7fc1447a0 0x7fc7fc13d8f0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7fc0cb170 0x7fc7fc13d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 --2- 172.21.15.119:0/3783715799 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7fc0cad30 0x7fc7fc15a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:21.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 >> 172.21.15.119:0/3783715799 conn(0x7fc7fc0cb610 msgr2=0x7fc7fc143a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 shutdown_connections 2023-11-08T19:32:21.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:21.970+0000 7fc80290c640 1 -- 172.21.15.119:0/3783715799 wait complete. 2023-11-08T19:32:21.995 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-11-08T19:32:20.840051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":378,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261084,"kb_used_data":11636,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713636,"statfs":{"total":773094113280,"available":772826763264,"internally_reserved":0,"allocated":11915264,"data_stored":6682222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-78272,"num_objects":-41,"num_object_clones":0,"num_object_copies":-82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-41,"num_whiteouts":0,"num_read":-469,"num_read_kb":-623,"num_write":-182,"num_write_kb":-154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.762732"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082901+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:32:14.082901+0000","last_peered":"2023-11-08T19:32:14.082901+0000","last_clean":"2023-11-08T19:32:14.082901+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:14.082901+0000","last_undegraded":"2023-11-08T19:32:14.082901+0000","last_fullsized":"2023-11-08T19:32:14.082901+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326562+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:32:15.326562+0000","last_peered":"2023-11-08T19:32:15.326562+0000","last_clean":"2023-11-08T19:32:15.326562+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:15.326562+0000","last_undegraded":"2023-11-08T19:32:15.326562+0000","last_fullsized":"2023-11-08T19:32:15.326562+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324561+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:32:15.324561+0000","last_peered":"2023-11-08T19:32:15.324561+0000","last_clean":"2023-11-08T19:32:15.324561+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:15.324561+0000","last_undegraded":"2023-11-08T19:32:15.324561+0000","last_fullsized":"2023-11-08T19:32:15.324561+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:09.740785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":43,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771284+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:32:14.771284+0000","last_peered":"2023-11-08T19:32:14.771284+0000","last_clean":"2023-11-08T19:32:14.771284+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:14.771284+0000","last_undegraded":"2023-11-08T19:32:14.771284+0000","last_fullsized":"2023-11-08T19:32:14.771284+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:07:26.750150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326623+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:32:15.326623+0000","last_peered":"2023-11-08T19:32:15.326623+0000","last_clean":"2023-11-08T19:32:15.326623+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:32:15.326623+0000","last_undegraded":"2023-11-08T19:32:15.326623+0000","last_fullsized":"2023-11-08T19:32:15.326623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.084456+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:32:14.084456+0000","last_peered":"2023-11-08T19:32:14.084456+0000","last_clean":"2023-11-08T19:32:14.084456+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:14.084456+0000","last_undegraded":"2023-11-08T19:32:14.084456+0000","last_fullsized":"2023-11-08T19:32:14.084456+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.087661+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:32:14.087661+0000","last_peered":"2023-11-08T19:32:14.087661+0000","last_clean":"2023-11-08T19:32:14.087661+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:32:14.087661+0000","last_undegraded":"2023-11-08T19:32:14.087661+0000","last_fullsized":"2023-11-08T19:32:14.087661+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.084090+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:32:14.084090+0000","last_peered":"2023-11-08T19:32:14.084090+0000","last_clean":"2023-11-08T19:32:14.084090+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:14.084090+0000","last_undegraded":"2023-11-08T19:32:14.084090+0000","last_fullsized":"2023-11-08T19:32:14.084090+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.327008+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:32:15.327008+0000","last_peered":"2023-11-08T19:32:15.327008+0000","last_clean":"2023-11-08T19:32:15.327008+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:15.327008+0000","last_undegraded":"2023-11-08T19:32:15.327008+0000","last_fullsized":"2023-11-08T19:32:15.327008+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080981+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:32:14.080981+0000","last_peered":"2023-11-08T19:32:14.080981+0000","last_clean":"2023-11-08T19:32:14.080981+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:14.080981+0000","last_undegraded":"2023-11-08T19:32:14.080981+0000","last_fullsized":"2023-11-08T19:32:14.080981+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:56:51.502894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325948+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:32:15.325948+0000","last_peered":"2023-11-08T19:32:15.325948+0000","last_clean":"2023-11-08T19:32:15.325948+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:32:15.325948+0000","last_undegraded":"2023-11-08T19:32:15.325948+0000","last_fullsized":"2023-11-08T19:32:15.325948+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324984+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:32:15.324984+0000","last_peered":"2023-11-08T19:32:15.324984+0000","last_clean":"2023-11-08T19:32:15.324984+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:15.324984+0000","last_undegraded":"2023-11-08T19:32:15.324984+0000","last_fullsized":"2023-11-08T19:32:15.324984+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:55:20.355198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245918+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:32:14.245918+0000","last_peered":"2023-11-08T19:32:14.245918+0000","last_clean":"2023-11-08T19:32:14.245918+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:14.245918+0000","last_undegraded":"2023-11-08T19:32:14.245918+0000","last_fullsized":"2023-11-08T19:32:14.245918+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":50,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771029+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:32:14.771029+0000","last_peered":"2023-11-08T19:32:14.771029+0000","last_clean":"2023-11-08T19:32:14.771029+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:14.771029+0000","last_undegraded":"2023-11-08T19:32:14.771029+0000","last_fullsized":"2023-11-08T19:32:14.771029+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:55:11.719718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325529+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:32:15.325529+0000","last_peered":"2023-11-08T19:32:15.325529+0000","last_clean":"2023-11-08T19:32:15.325529+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:15.325529+0000","last_undegraded":"2023-11-08T19:32:15.325529+0000","last_fullsized":"2023-11-08T19:32:15.325529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080107+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:32:14.080107+0000","last_peered":"2023-11-08T19:32:14.080107+0000","last_clean":"2023-11-08T19:32:14.080107+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:32:14.080107+0000","last_undegraded":"2023-11-08T19:32:14.080107+0000","last_fullsized":"2023-11-08T19:32:14.080107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324313+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:32:15.324313+0000","last_peered":"2023-11-08T19:32:15.324313+0000","last_clean":"2023-11-08T19:32:15.324313+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:15.324313+0000","last_undegraded":"2023-11-08T19:32:15.324313+0000","last_fullsized":"2023-11-08T19:32:15.324313+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:56.949130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079580+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:32:14.079580+0000","last_peered":"2023-11-08T19:32:14.079580+0000","last_clean":"2023-11-08T19:32:14.079580+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:14.079580+0000","last_undegraded":"2023-11-08T19:32:14.079580+0000","last_fullsized":"2023-11-08T19:32:14.079580+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323815+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:32:15.323815+0000","last_peered":"2023-11-08T19:32:15.323815+0000","last_clean":"2023-11-08T19:32:15.323815+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:15.323815+0000","last_undegraded":"2023-11-08T19:32:15.323815+0000","last_fullsized":"2023-11-08T19:32:15.323815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:27:54.335831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079780+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:32:14.079780+0000","last_peered":"2023-11-08T19:32:14.079780+0000","last_clean":"2023-11-08T19:32:14.079780+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:14.079780+0000","last_undegraded":"2023-11-08T19:32:14.079780+0000","last_fullsized":"2023-11-08T19:32:14.079780+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080725+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:32:14.080725+0000","last_peered":"2023-11-08T19:32:14.080725+0000","last_clean":"2023-11-08T19:32:14.080725+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:14.080725+0000","last_undegraded":"2023-11-08T19:32:14.080725+0000","last_fullsized":"2023-11-08T19:32:14.080725+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:23.817534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323664+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:32:15.323664+0000","last_peered":"2023-11-08T19:32:15.323664+0000","last_clean":"2023-11-08T19:32:15.323664+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:15.323664+0000","last_undegraded":"2023-11-08T19:32:15.323664+0000","last_fullsized":"2023-11-08T19:32:15.323664+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:17.410646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083600+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:32:14.083600+0000","last_peered":"2023-11-08T19:32:14.083600+0000","last_clean":"2023-11-08T19:32:14.083600+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:14.083600+0000","last_undegraded":"2023-11-08T19:32:14.083600+0000","last_fullsized":"2023-11-08T19:32:14.083600+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086612+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:32:14.086612+0000","last_peered":"2023-11-08T19:32:14.086612+0000","last_clean":"2023-11-08T19:32:14.086612+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:14.086612+0000","last_undegraded":"2023-11-08T19:32:14.086612+0000","last_fullsized":"2023-11-08T19:32:14.086612+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326896+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:32:15.326896+0000","last_peered":"2023-11-08T19:32:15.326896+0000","last_clean":"2023-11-08T19:32:15.326896+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:32:15.326896+0000","last_undegraded":"2023-11-08T19:32:15.326896+0000","last_fullsized":"2023-11-08T19:32:15.326896+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088580+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:32:14.088580+0000","last_peered":"2023-11-08T19:32:14.088580+0000","last_clean":"2023-11-08T19:32:14.088580+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:14.088580+0000","last_undegraded":"2023-11-08T19:32:14.088580+0000","last_fullsized":"2023-11-08T19:32:14.088580+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246666+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:32:14.246666+0000","last_peered":"2023-11-08T19:32:14.246666+0000","last_clean":"2023-11-08T19:32:14.246666+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:14.246666+0000","last_undegraded":"2023-11-08T19:32:14.246666+0000","last_fullsized":"2023-11-08T19:32:14.246666+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088147+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:32:14.088147+0000","last_peered":"2023-11-08T19:32:14.088147+0000","last_clean":"2023-11-08T19:32:14.088147+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:32:14.088147+0000","last_undegraded":"2023-11-08T19:32:14.088147+0000","last_fullsized":"2023-11-08T19:32:14.088147+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326448+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:32:15.326448+0000","last_peered":"2023-11-08T19:32:15.326448+0000","last_clean":"2023-11-08T19:32:15.326448+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:32:15.326448+0000","last_undegraded":"2023-11-08T19:32:15.326448+0000","last_fullsized":"2023-11-08T19:32:15.326448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771506+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:32:14.771506+0000","last_peered":"2023-11-08T19:32:14.771506+0000","last_clean":"2023-11-08T19:32:14.771506+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:14.771506+0000","last_undegraded":"2023-11-08T19:32:14.771506+0000","last_fullsized":"2023-11-08T19:32:14.771506+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:03.191007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246330+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:32:14.246330+0000","last_peered":"2023-11-08T19:32:14.246330+0000","last_clean":"2023-11-08T19:32:14.246330+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:32:14.246330+0000","last_undegraded":"2023-11-08T19:32:14.246330+0000","last_fullsized":"2023-11-08T19:32:14.246330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081842+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:32:14.081842+0000","last_peered":"2023-11-08T19:32:14.081842+0000","last_clean":"2023-11-08T19:32:14.081842+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:14.081842+0000","last_undegraded":"2023-11-08T19:32:14.081842+0000","last_fullsized":"2023-11-08T19:32:14.081842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:54:02.851109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246823+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:32:14.246823+0000","last_peered":"2023-11-08T19:32:14.246823+0000","last_clean":"2023-11-08T19:32:14.246823+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:14.246823+0000","last_undegraded":"2023-11-08T19:32:14.246823+0000","last_fullsized":"2023-11-08T19:32:14.246823+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325340+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:32:15.325340+0000","last_peered":"2023-11-08T19:32:15.325340+0000","last_clean":"2023-11-08T19:32:15.325340+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:32:15.325340+0000","last_undegraded":"2023-11-08T19:32:15.325340+0000","last_fullsized":"2023-11-08T19:32:15.325340+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080658+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:32:14.080658+0000","last_peered":"2023-11-08T19:32:14.080658+0000","last_clean":"2023-11-08T19:32:14.080658+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:14.080658+0000","last_undegraded":"2023-11-08T19:32:14.080658+0000","last_fullsized":"2023-11-08T19:32:14.080658+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:12.885753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088790+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:32:14.088790+0000","last_peered":"2023-11-08T19:32:14.088790+0000","last_clean":"2023-11-08T19:32:14.088790+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:14.088790+0000","last_undegraded":"2023-11-08T19:32:14.088790+0000","last_fullsized":"2023-11-08T19:32:14.088790+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085436+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:32:14.085436+0000","last_peered":"2023-11-08T19:32:14.085436+0000","last_clean":"2023-11-08T19:32:14.085436+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:14.085436+0000","last_undegraded":"2023-11-08T19:32:14.085436+0000","last_fullsized":"2023-11-08T19:32:14.085436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086428+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:32:14.086428+0000","last_peered":"2023-11-08T19:32:14.086428+0000","last_clean":"2023-11-08T19:32:14.086428+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:14.086428+0000","last_undegraded":"2023-11-08T19:32:14.086428+0000","last_fullsized":"2023-11-08T19:32:14.086428+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245467+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:32:14.245467+0000","last_peered":"2023-11-08T19:32:14.245467+0000","last_clean":"2023-11-08T19:32:14.245467+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:14.245467+0000","last_undegraded":"2023-11-08T19:32:14.245467+0000","last_fullsized":"2023-11-08T19:32:14.245467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324366+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:32:15.324366+0000","last_peered":"2023-11-08T19:32:15.324366+0000","last_clean":"2023-11-08T19:32:15.324366+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:15.324366+0000","last_undegraded":"2023-11-08T19:32:15.324366+0000","last_fullsized":"2023-11-08T19:32:15.324366+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:40:41.307244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085906+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:32:14.085906+0000","last_peered":"2023-11-08T19:32:14.085906+0000","last_clean":"2023-11-08T19:32:14.085906+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:14.085906+0000","last_undegraded":"2023-11-08T19:32:14.085906+0000","last_fullsized":"2023-11-08T19:32:14.085906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.770306+0000","last_change":"2023-11-08T19:26:42.042613+0000","last_active":"2023-11-08T19:32:14.770306+0000","last_peered":"2023-11-08T19:32:14.770306+0000","last_clean":"2023-11-08T19:32:14.770306+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:14.770306+0000","last_undegraded":"2023-11-08T19:32:14.770306+0000","last_fullsized":"2023-11-08T19:32:14.770306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:05.676557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":43,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.769959+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:32:14.769959+0000","last_peered":"2023-11-08T19:32:14.769959+0000","last_clean":"2023-11-08T19:32:14.769959+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:14.769959+0000","last_undegraded":"2023-11-08T19:32:14.769959+0000","last_fullsized":"2023-11-08T19:32:14.769959+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:39.884153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.323721+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:32:15.323721+0000","last_peered":"2023-11-08T19:32:15.323721+0000","last_clean":"2023-11-08T19:32:15.323721+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:15.323721+0000","last_undegraded":"2023-11-08T19:32:15.323721+0000","last_fullsized":"2023-11-08T19:32:15.323721+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:55:57.467027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080836+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:32:14.080836+0000","last_peered":"2023-11-08T19:32:14.080836+0000","last_clean":"2023-11-08T19:32:14.080836+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:14.080836+0000","last_undegraded":"2023-11-08T19:32:14.080836+0000","last_fullsized":"2023-11-08T19:32:14.080836+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245560+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:32:14.245560+0000","last_peered":"2023-11-08T19:32:14.245560+0000","last_clean":"2023-11-08T19:32:14.245560+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:14.245560+0000","last_undegraded":"2023-11-08T19:32:14.245560+0000","last_fullsized":"2023-11-08T19:32:14.245560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.079989+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:32:14.079989+0000","last_peered":"2023-11-08T19:32:14.079989+0000","last_clean":"2023-11-08T19:32:14.079989+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:14.079989+0000","last_undegraded":"2023-11-08T19:32:14.079989+0000","last_fullsized":"2023-11-08T19:32:14.079989+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:10:37.978152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325680+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:32:15.325680+0000","last_peered":"2023-11-08T19:32:15.325680+0000","last_clean":"2023-11-08T19:32:15.325680+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:15.325680+0000","last_undegraded":"2023-11-08T19:32:15.325680+0000","last_fullsized":"2023-11-08T19:32:15.325680+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246594+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:32:14.246594+0000","last_peered":"2023-11-08T19:32:14.246594+0000","last_clean":"2023-11-08T19:32:14.246594+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:32:14.246594+0000","last_undegraded":"2023-11-08T19:32:14.246594+0000","last_fullsized":"2023-11-08T19:32:14.246594+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324910+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:32:15.324910+0000","last_peered":"2023-11-08T19:32:15.324910+0000","last_clean":"2023-11-08T19:32:15.324910+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:15.324910+0000","last_undegraded":"2023-11-08T19:32:15.324910+0000","last_fullsized":"2023-11-08T19:32:15.324910+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:01:04.379784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326226+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:32:15.326226+0000","last_peered":"2023-11-08T19:32:15.326226+0000","last_clean":"2023-11-08T19:32:15.326226+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:15.326226+0000","last_undegraded":"2023-11-08T19:32:15.326226+0000","last_fullsized":"2023-11-08T19:32:15.326226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088471+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:32:14.088471+0000","last_peered":"2023-11-08T19:32:14.088471+0000","last_clean":"2023-11-08T19:32:14.088471+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:14.088471+0000","last_undegraded":"2023-11-08T19:32:14.088471+0000","last_fullsized":"2023-11-08T19:32:14.088471+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":52,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325063+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:32:15.325063+0000","last_peered":"2023-11-08T19:32:15.325063+0000","last_clean":"2023-11-08T19:32:15.325063+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:15.325063+0000","last_undegraded":"2023-11-08T19:32:15.325063+0000","last_fullsized":"2023-11-08T19:32:15.325063+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:01.964005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325706+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:32:15.325706+0000","last_peered":"2023-11-08T19:32:15.325706+0000","last_clean":"2023-11-08T19:32:15.325706+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:32:15.325706+0000","last_undegraded":"2023-11-08T19:32:15.325706+0000","last_fullsized":"2023-11-08T19:32:15.325706+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.088073+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:32:14.088073+0000","last_peered":"2023-11-08T19:32:14.088073+0000","last_clean":"2023-11-08T19:32:14.088073+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:32:14.088073+0000","last_undegraded":"2023-11-08T19:32:14.088073+0000","last_fullsized":"2023-11-08T19:32:14.088073+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325460+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:32:15.325460+0000","last_peered":"2023-11-08T19:32:15.325460+0000","last_clean":"2023-11-08T19:32:15.325460+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:15.325460+0000","last_undegraded":"2023-11-08T19:32:15.325460+0000","last_fullsized":"2023-11-08T19:32:15.325460+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082006+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:32:14.082006+0000","last_peered":"2023-11-08T19:32:14.082006+0000","last_clean":"2023-11-08T19:32:14.082006+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:14.082006+0000","last_undegraded":"2023-11-08T19:32:14.082006+0000","last_fullsized":"2023-11-08T19:32:14.082006+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:45:01.533549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325608+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:32:15.325608+0000","last_peered":"2023-11-08T19:32:15.325608+0000","last_clean":"2023-11-08T19:32:15.325608+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:32:15.325608+0000","last_undegraded":"2023-11-08T19:32:15.325608+0000","last_fullsized":"2023-11-08T19:32:15.325608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325825+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:32:15.325825+0000","last_peered":"2023-11-08T19:32:15.325825+0000","last_clean":"2023-11-08T19:32:15.325825+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:32:15.325825+0000","last_undegraded":"2023-11-08T19:32:15.325825+0000","last_fullsized":"2023-11-08T19:32:15.325825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.970299+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:32:14.970299+0000","last_peered":"2023-11-08T19:32:14.970299+0000","last_clean":"2023-11-08T19:32:14.970299+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:14.970299+0000","last_undegraded":"2023-11-08T19:32:14.970299+0000","last_fullsized":"2023-11-08T19:32:14.970299+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:48:43.911462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080424+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:32:14.080424+0000","last_peered":"2023-11-08T19:32:14.080424+0000","last_clean":"2023-11-08T19:32:14.080424+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:14.080424+0000","last_undegraded":"2023-11-08T19:32:14.080424+0000","last_fullsized":"2023-11-08T19:32:14.080424+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:23:32.873475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.086594+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:32:14.086594+0000","last_peered":"2023-11-08T19:32:14.086594+0000","last_clean":"2023-11-08T19:32:14.086594+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:14.086594+0000","last_undegraded":"2023-11-08T19:32:14.086594+0000","last_fullsized":"2023-11-08T19:32:14.086594+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081053+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:32:14.081053+0000","last_peered":"2023-11-08T19:32:14.081053+0000","last_clean":"2023-11-08T19:32:14.081053+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:14.081053+0000","last_undegraded":"2023-11-08T19:32:14.081053+0000","last_fullsized":"2023-11-08T19:32:14.081053+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082613+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:32:14.082613+0000","last_peered":"2023-11-08T19:32:14.082613+0000","last_clean":"2023-11-08T19:32:14.082613+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:14.082613+0000","last_undegraded":"2023-11-08T19:32:14.082613+0000","last_fullsized":"2023-11-08T19:32:14.082613+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080615+0000","last_change":"2023-11-08T19:27:48.915018+0000","last_active":"2023-11-08T19:32:14.080615+0000","last_peered":"2023-11-08T19:32:14.080615+0000","last_clean":"2023-11-08T19:32:14.080615+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:14.080615+0000","last_undegraded":"2023-11-08T19:32:14.080615+0000","last_fullsized":"2023-11-08T19:32:14.080615+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:58:56.672134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968861+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:32:14.968861+0000","last_peered":"2023-11-08T19:32:14.968861+0000","last_clean":"2023-11-08T19:32:14.968861+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:14.968861+0000","last_undegraded":"2023-11-08T19:32:14.968861+0000","last_fullsized":"2023-11-08T19:32:14.968861+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:25.757270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325069+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:32:15.325069+0000","last_peered":"2023-11-08T19:32:15.325069+0000","last_clean":"2023-11-08T19:32:15.325069+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:32:15.325069+0000","last_undegraded":"2023-11-08T19:32:15.325069+0000","last_fullsized":"2023-11-08T19:32:15.325069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":55,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083582+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:32:14.083582+0000","last_peered":"2023-11-08T19:32:14.083582+0000","last_clean":"2023-11-08T19:32:14.083582+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:14.083582+0000","last_undegraded":"2023-11-08T19:32:14.083582+0000","last_fullsized":"2023-11-08T19:32:14.083582+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.324550+0000","last_change":"2023-11-08T19:26:41.726660+0000","last_active":"2023-11-08T19:32:15.324550+0000","last_peered":"2023-11-08T19:32:15.324550+0000","last_clean":"2023-11-08T19:32:15.324550+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:15.324550+0000","last_undegraded":"2023-11-08T19:32:15.324550+0000","last_fullsized":"2023-11-08T19:32:15.324550+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:06:16.326208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":45,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245784+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:32:14.245784+0000","last_peered":"2023-11-08T19:32:14.245784+0000","last_clean":"2023-11-08T19:32:14.245784+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:14.245784+0000","last_undegraded":"2023-11-08T19:32:14.245784+0000","last_fullsized":"2023-11-08T19:32:14.245784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":54,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325075+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:32:15.325075+0000","last_peered":"2023-11-08T19:32:15.325075+0000","last_clean":"2023-11-08T19:32:15.325075+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:15.325075+0000","last_undegraded":"2023-11-08T19:32:15.325075+0000","last_fullsized":"2023-11-08T19:32:15.325075+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771424+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:32:14.771424+0000","last_peered":"2023-11-08T19:32:14.771424+0000","last_clean":"2023-11-08T19:32:14.771424+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:14.771424+0000","last_undegraded":"2023-11-08T19:32:14.771424+0000","last_fullsized":"2023-11-08T19:32:14.771424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:16:03.307566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.326894+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:32:15.326894+0000","last_peered":"2023-11-08T19:32:15.326894+0000","last_clean":"2023-11-08T19:32:15.326894+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:32:15.326894+0000","last_undegraded":"2023-11-08T19:32:15.326894+0000","last_fullsized":"2023-11-08T19:32:15.326894+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258452+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:32:16.258452+0000","last_peered":"2023-11-08T19:32:16.258452+0000","last_clean":"2023-11-08T19:32:16.258452+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:16.258452+0000","last_undegraded":"2023-11-08T19:32:16.258452+0000","last_fullsized":"2023-11-08T19:32:16.258452+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:28:00.496098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":76,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246884+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:32:14.246884+0000","last_peered":"2023-11-08T19:32:14.246884+0000","last_clean":"2023-11-08T19:32:14.246884+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:14.246884+0000","last_undegraded":"2023-11-08T19:32:14.246884+0000","last_fullsized":"2023-11-08T19:32:14.246884+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.245706+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:32:14.245706+0000","last_peered":"2023-11-08T19:32:14.245706+0000","last_clean":"2023-11-08T19:32:14.245706+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:14.245706+0000","last_undegraded":"2023-11-08T19:32:14.245706+0000","last_fullsized":"2023-11-08T19:32:14.245706+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":54,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.255292+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:32:16.255292+0000","last_peered":"2023-11-08T19:32:16.255292+0000","last_clean":"2023-11-08T19:32:16.255292+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:32:16.255292+0000","last_undegraded":"2023-11-08T19:32:16.255292+0000","last_fullsized":"2023-11-08T19:32:16.255292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081818+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:32:14.081818+0000","last_peered":"2023-11-08T19:32:14.081818+0000","last_clean":"2023-11-08T19:32:14.081818+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:14.081818+0000","last_undegraded":"2023-11-08T19:32:14.081818+0000","last_fullsized":"2023-11-08T19:32:14.081818+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":47,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.087868+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:32:14.087868+0000","last_peered":"2023-11-08T19:32:14.087868+0000","last_clean":"2023-11-08T19:32:14.087868+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:14.087868+0000","last_undegraded":"2023-11-08T19:32:14.087868+0000","last_fullsized":"2023-11-08T19:32:14.087868+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.256957+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:32:16.256957+0000","last_peered":"2023-11-08T19:32:16.256957+0000","last_clean":"2023-11-08T19:32:16.256957+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:16.256957+0000","last_undegraded":"2023-11-08T19:32:16.256957+0000","last_fullsized":"2023-11-08T19:32:16.256957+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:49.694793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":77,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.327036+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:32:15.327036+0000","last_peered":"2023-11-08T19:32:15.327036+0000","last_clean":"2023-11-08T19:32:15.327036+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:15.327036+0000","last_undegraded":"2023-11-08T19:32:15.327036+0000","last_fullsized":"2023-11-08T19:32:15.327036+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":48,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.083682+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:32:14.083682+0000","last_peered":"2023-11-08T19:32:14.083682+0000","last_clean":"2023-11-08T19:32:14.083682+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:14.083682+0000","last_undegraded":"2023-11-08T19:32:14.083682+0000","last_fullsized":"2023-11-08T19:32:14.083682+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968622+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:32:14.968622+0000","last_peered":"2023-11-08T19:32:14.968622+0000","last_clean":"2023-11-08T19:32:14.968622+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:14.968622+0000","last_undegraded":"2023-11-08T19:32:14.968622+0000","last_fullsized":"2023-11-08T19:32:14.968622+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:43:00.506614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.085702+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:32:14.085702+0000","last_peered":"2023-11-08T19:32:14.085702+0000","last_clean":"2023-11-08T19:32:14.085702+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:14.085702+0000","last_undegraded":"2023-11-08T19:32:14.085702+0000","last_fullsized":"2023-11-08T19:32:14.085702+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968748+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:32:14.968748+0000","last_peered":"2023-11-08T19:32:14.968748+0000","last_clean":"2023-11-08T19:32:14.968748+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:14.968748+0000","last_undegraded":"2023-11-08T19:32:14.968748+0000","last_fullsized":"2023-11-08T19:32:14.968748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:18:15.988493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":51,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.252776+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:32:16.252776+0000","last_peered":"2023-11-08T19:32:16.252776+0000","last_clean":"2023-11-08T19:32:16.252776+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:16.252776+0000","last_undegraded":"2023-11-08T19:32:16.252776+0000","last_fullsized":"2023-11-08T19:32:16.252776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:26:53.983418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":81,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.081711+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:32:14.081711+0000","last_peered":"2023-11-08T19:32:14.081711+0000","last_clean":"2023-11-08T19:32:14.081711+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:14.081711+0000","last_undegraded":"2023-11-08T19:32:14.081711+0000","last_fullsized":"2023-11-08T19:32:14.081711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":49,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.082992+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:32:14.082992+0000","last_peered":"2023-11-08T19:32:14.082992+0000","last_clean":"2023-11-08T19:32:14.082992+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:14.082992+0000","last_undegraded":"2023-11-08T19:32:14.082992+0000","last_fullsized":"2023-11-08T19:32:14.082992+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":51,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.769987+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:32:14.769987+0000","last_peered":"2023-11-08T19:32:14.769987+0000","last_clean":"2023-11-08T19:32:14.769987+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:14.769987+0000","last_undegraded":"2023-11-08T19:32:14.769987+0000","last_fullsized":"2023-11-08T19:32:14.769987+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:28.392596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771136+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:32:14.771136+0000","last_peered":"2023-11-08T19:32:14.771136+0000","last_clean":"2023-11-08T19:32:14.771136+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:14.771136+0000","last_undegraded":"2023-11-08T19:32:14.771136+0000","last_fullsized":"2023-11-08T19:32:14.771136+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:32:23.410613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":46,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968697+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:32:14.968697+0000","last_peered":"2023-11-08T19:32:14.968697+0000","last_clean":"2023-11-08T19:32:14.968697+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:14.968697+0000","last_undegraded":"2023-11-08T19:32:14.968697+0000","last_fullsized":"2023-11-08T19:32:14.968697+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.664702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":53,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.968890+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:32:14.968890+0000","last_peered":"2023-11-08T19:32:14.968890+0000","last_clean":"2023-11-08T19:32:14.968890+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:14.968890+0000","last_undegraded":"2023-11-08T19:32:14.968890+0000","last_fullsized":"2023-11-08T19:32:14.968890+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:10:39.650776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":74,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.771554+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:32:14.771554+0000","last_peered":"2023-11-08T19:32:14.771554+0000","last_clean":"2023-11-08T19:32:14.771554+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:14.771554+0000","last_undegraded":"2023-11-08T19:32:14.771554+0000","last_fullsized":"2023-11-08T19:32:14.771554+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:43:34.352707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":44,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:15.325205+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:32:15.325205+0000","last_peered":"2023-11-08T19:32:15.325205+0000","last_clean":"2023-11-08T19:32:15.325205+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:15.325205+0000","last_undegraded":"2023-11-08T19:32:15.325205+0000","last_fullsized":"2023-11-08T19:32:15.325205+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:06:56.586505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":52,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.257983+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:32:16.257983+0000","last_peered":"2023-11-08T19:32:16.257983+0000","last_clean":"2023-11-08T19:32:16.257983+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:16.257983+0000","last_undegraded":"2023-11-08T19:32:16.257983+0000","last_fullsized":"2023-11-08T19:32:16.257983+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":75,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.080613+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:32:14.080613+0000","last_peered":"2023-11-08T19:32:14.080613+0000","last_clean":"2023-11-08T19:32:14.080613+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:14.080613+0000","last_undegraded":"2023-11-08T19:32:14.080613+0000","last_fullsized":"2023-11-08T19:32:14.080613+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":245,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-11-08T19:32:14.246364+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:32:14.246364+0000","last_peered":"2023-11-08T19:32:14.246364+0000","last_clean":"2023-11-08T19:32:14.246364+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:14.246364+0000","last_undegraded":"2023-11-08T19:32:14.246364+0000","last_fullsized":"2023-11-08T19:32:14.246364+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33064,"kb_used_data":1884,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338776,"statfs":{"total":96636764160,"available":96602906624,"internally_reserved":0,"allocated":1929216,"data_stored":1273881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.77500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.599,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.252},"min":{"1min":0.72799999999999998,"5min":0.58599999999999997,"15min":0.58499999999999996},"max":{"1min":1.6619999999999999,"5min":2.1360000000000001,"15min":4.4589999999999996},"last":1.3560000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2010000000000001,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.548,"5min":1.728,"15min":4.5330000000000004},"last":1.1160000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.1910000000000001,"15min":1.1459999999999999},"min":{"1min":0.83299999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6160000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.2030000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.335,"15min":1.448},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5660000000000001,"5min":1.891,"15min":4.6980000000000004},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.194,"15min":1.3160000000000001},"min":{"1min":0.97999999999999998,"5min":0.67200000000000004,"15min":0.64700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":4.4269999999999996},"last":1.5780000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.294,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.7889999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":0.72199999999999998},{"interface":"front","average":{"1min":1.383,"5min":1.409,"15min":1.5149999999999999},"min":{"1min":0.95199999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.768,"5min":2.2200000000000002,"15min":4.6150000000000002},"last":0.92200000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.327},"min":{"1min":1.006,"5min":0.624,"15min":0.624},"max":{"1min":1.49,"5min":1.798,"15min":2.3170000000000002},"last":1.0089999999999999},{"interface":"front","average":{"1min":1.335,"5min":1.323,"15min":1.4410000000000001},"min":{"1min":0.93200000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7909999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.403}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.95999999999999996,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.673,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.508},{"interface":"front","average":{"1min":1.379,"5min":1.363,"15min":1.4470000000000001},"min":{"1min":1.048,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.649,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.6850000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":1.22,"5min":0.878,"15min":0.878},"max":{"1min":1.8109999999999999,"5min":1.9770000000000001,"15min":2.427},"last":1.9550000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.474,"15min":1.581},"min":{"1min":1.1180000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.98,"5min":2.1920000000000002,"15min":2.468},"last":1.5489999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.0940000000000001,"15min":0.98799999999999999},"min":{"1min":0.85399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.536,"5min":1.6279999999999999,"15min":1.752},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.202,"15min":1.1399999999999999},"min":{"1min":0.80200000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":8.391,"5min":8.391,"15min":8.391},"last":0.752}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.161,"15min":1.113},"min":{"1min":0.85599999999999998,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.679},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2050000000000001,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.665,"5min":1.665,"15min":4.6399999999999997},"last":1.099}]},{"osd":2,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.80300000000000005,"5min":0.68799999999999994,"15min":0.54600000000000004},"max":{"1min":1.4830000000000001,"5min":1.655,"15min":4.609},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2949999999999999,"15min":1.242},"min":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.861},"last":1.141}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.276,"15min":1.4239999999999999},"min":{"1min":1.0209999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.756,"15min":4.7190000000000003},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.256,"15min":1.2050000000000001},"min":{"1min":0.81200000000000006,"5min":0.625,"15min":0.625},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.784},"last":0.98299999999999998}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.83199999999999996,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.859,"5min":1.881,"15min":2.0449999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4770000000000001,"15min":1.4910000000000001},"min":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.998,"15min":2.3130000000000002},"last":1.5029999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.325,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.611,"5min":1.835,"15min":1.9079999999999999},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3999999999999999,"15min":1.353},"min":{"1min":0.92100000000000004,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":1.6819999999999999,"5min":1.8660000000000001,"15min":2.0489999999999999},"last":1.2749999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5569999999999999,"15min":1.55},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":1.99,"5min":1.99,"15min":2.2519999999999998},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4970000000000001,"15min":1.464},"min":{"1min":0.75700000000000001,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":1.7809999999999999,"5min":1.9870000000000001,"15min":3.0640000000000001},"last":1.4219999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1970000000000001,"15min":1.117},"min":{"1min":0.69599999999999995,"5min":0.59999999999999998,"15min":0.57699999999999996},"max":{"1min":1.603,"5min":1.724,"15min":1.76},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.274,"15min":1.252},"min":{"1min":0.77600000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.837,"5min":1.954,"15min":1.954},"last":1.6259999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.754,"5min":0.753,"15min":0.58699999999999997},"max":{"1min":1.552,"5min":1.6359999999999999,"15min":2.3039999999999998},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.179,"5min":1.1679999999999999,"15min":1.355},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.115}]},{"osd":2,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2350000000000001,"15min":1.194},"min":{"1min":0.82599999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5720000000000001,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.173,"15min":1.3959999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6990000000000001,"5min":1.806,"15min":7.6479999999999997},"last":1.2430000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.74,"5min":1.804,"15min":2.1920000000000002},"last":1.161},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3500000000000001,"15min":1.323},"min":{"1min":0.98799999999999999,"5min":0.91300000000000003,"15min":0.69699999999999995},"max":{"1min":1.6910000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.196,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7350000000000001,"5min":1.7969999999999999,"15min":2.347},"last":2.0430000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.347,"15min":1.488},"min":{"1min":0.94099999999999995,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.7350000000000001,"5min":2.097,"15min":3.153},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3859999999999999,"15min":1.4830000000000001},"min":{"1min":0.92100000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.923,"5min":1.923,"15min":2.8650000000000002},"last":1.448},{"interface":"front","average":{"1min":1.399,"5min":1.4019999999999999,"15min":1.47},"min":{"1min":0.94199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6990000000000001,"5min":1.899,"15min":2.6190000000000002},"last":1.742}]},{"osd":7,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.6359999999999999},"min":{"1min":1.0980000000000001,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":1.9259999999999999,"5min":2.0630000000000002,"15min":3.2730000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5229999999999999,"15min":1.5680000000000001},"min":{"1min":1.204,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.8600000000000001,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.7030000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1479999999999999,"15min":1.0760000000000001},"min":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.5900000000000001,"5min":1.782,"15min":2.5219999999999998},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.206,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.83499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.524,"5min":1.696,"15min":7.0119999999999996},"last":0.95299999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.0680000000000001,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.63100000000000001},"max":{"1min":1.752,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.206},{"interface":"front","average":{"1min":1.177,"5min":1.1499999999999999,"15min":1.1519999999999999},"min":{"1min":0.748,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.7310000000000001,"5min":1.7450000000000001,"15min":2.9940000000000002},"last":1.3879999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2490000000000001,"15min":1.3160000000000001},"min":{"1min":0.98199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.988,"5min":1.988,"15min":4.0410000000000004},"last":1.046},{"interface":"front","average":{"1min":1.341,"5min":1.319,"15min":1.3879999999999999},"min":{"1min":0.95999999999999996,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":3.2069999999999999},"last":0.77600000000000002}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1659999999999999,"15min":1.258},"min":{"1min":0.84199999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5169999999999999,"5min":1.696,"15min":4.4980000000000002},"last":1.105},{"interface":"front","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.345},"min":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":1.738,"15min":3.5710000000000002},"last":1.429}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.4330000000000001},"min":{"1min":0.81000000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.593,"5min":1.762,"15min":4.2350000000000003},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2549999999999999,"15min":1.4590000000000001},"min":{"1min":1.0189999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5800000000000001,"5min":1.8560000000000001,"15min":4.5880000000000001},"last":1.163}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.365,"15min":1.5009999999999999},"min":{"1min":0.92300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":2.2130000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4419999999999999,"15min":1.5660000000000001},"min":{"1min":1.1639999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.841,"5min":1.9490000000000001,"15min":4.4130000000000003},"last":1.26}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.409,"15min":1.5640000000000001},"min":{"1min":1.137,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":4.6369999999999996},"last":1.325},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5669999999999999,"15min":1.6799999999999999},"min":{"1min":1.2969999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8999999999999999,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.488}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32484,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339356,"statfs":{"total":96636764160,"available":96603500544,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.99399999999999999},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.427,"5min":2.2320000000000002,"15min":3.972},"last":1.355},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93500000000000005,"15min":1.042},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.311},"max":{"1min":1.544,"5min":1.899,"15min":3.1520000000000001},"last":1.7829999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95299999999999996,"15min":1.1160000000000001},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.4299999999999999,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":1.0429999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8520000000000001,"15min":4.5750000000000002},"last":0.879}]},{"osd":2,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.153,"15min":1.272},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.619,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1020000000000001,"15min":1.218},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.6339999999999999,"5min":1.754,"15min":2.73},"last":1.294}]},{"osd":4,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.214,"15min":1.3779999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6599999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.24,"15min":1.3899999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9039999999999999,"5min":2.0449999999999999,"15min":3.052},"last":1.4530000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.4370000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":2.0430000000000001,"15min":3.0899999999999999},"last":1.9259999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.29,"15min":1.4570000000000001},"min":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.597,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.234}]},{"osd":6,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2769999999999999,"15min":1.3819999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7,"5min":3.21,"15min":4.2220000000000004},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.391,"15min":1.526},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.0880000000000001,"15min":3.1230000000000002},"last":1.5880000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.399,"15min":1.6160000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.925,"5min":2.1150000000000002,"15min":4.407},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.5209999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.671,"5min":2.0059999999999998,"15min":4.3849999999999998},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1323008,"data_stored":674557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83699999999999997,"15min":0.85399999999999998},"min":{"1min":0.61699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2370000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.0720000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.56799999999999995,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.371,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.66700000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.474,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.093,"15min":1.0620000000000001},"min":{"1min":0.80300000000000005,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.052,"15min":1.103},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.675,"5min":1.7330000000000001,"15min":1.7669999999999999},"last":1.1240000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2829999999999999,"15min":1.365},"min":{"1min":0.95499999999999996,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":1.883,"15min":3.2959999999999998},"last":1.268},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.284,"15min":1.3400000000000001},"min":{"1min":1.0609999999999999,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":1.704,"15min":2.278},"last":1.556}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.325,"15min":1.3899999999999999},"min":{"1min":1.1359999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.327,"15min":1.4179999999999999},"min":{"1min":1.048,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.7709999999999999,"5min":1.954,"15min":2.536},"last":1.4350000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":1.1120000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.657},{"interface":"front","average":{"1min":1.377,"5min":1.351,"15min":1.421},"min":{"1min":0.92200000000000004,"5min":0.83799999999999997,"15min":0.70099999999999996},"max":{"1min":1.9810000000000001,"5min":2.036,"15min":2.0910000000000002},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.409,"15min":1.522},"min":{"1min":0.95099999999999996,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":1.74,"5min":1.9339999999999999,"15min":2.6549999999999998},"last":1.323},{"interface":"front","average":{"1min":1.516,"5min":1.466,"15min":1.5269999999999999},"min":{"1min":1.111,"5min":0.84999999999999998,"15min":0.76600000000000001},"max":{"1min":1.8009999999999999,"5min":2.226,"15min":2.3439999999999999},"last":1.6080000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32488,"kb_used_data":1304,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339352,"statfs":{"total":96636764160,"available":96603496448,"internally_reserved":0,"allocated":1335296,"data_stored":683513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85599999999999998,"15min":0.78000000000000003},"min":{"1min":0.59199999999999997,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.284,"5min":1.327,"15min":1.327},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.88800000000000001,"15min":0.79900000000000004},"min":{"1min":0.82499999999999996,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":1.528,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.1359999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84099999999999997,"15min":0.81100000000000005},"min":{"1min":0.51100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.256,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0249999999999999,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.47499999999999998,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.028}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2569999999999999,"15min":1.1240000000000001},"min":{"1min":0.80100000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.748,"5min":1.978,"15min":1.978},"last":1.302},{"interface":"front","average":{"1min":1.196,"5min":1.1439999999999999,"15min":1.1020000000000001},"min":{"1min":0.65300000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.6930000000000001,"5min":2.2690000000000001,"15min":2.5369999999999999},"last":1.2210000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.157},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.476,"5min":1.8620000000000001,"15min":2.165},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.294,"5min":1.2729999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.6870000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.1859999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.93799999999999994,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2689999999999999,"15min":1.3069999999999999},"min":{"1min":0.95199999999999996,"5min":0.73199999999999998,"15min":0.629},"max":{"1min":1.696,"5min":1.748,"15min":2.4089999999999998},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.375},"min":{"1min":1.194,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.7450000000000001,"5min":2.1509999999999998,"15min":2.6019999999999999},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.355,"15min":1.373},"min":{"1min":1.133,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.282},"last":1.2549999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.3560000000000001},"min":{"1min":1.0029999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.454,"5min":1.409,"15min":1.395},"min":{"1min":1.0940000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.865,"5min":2.4830000000000001,"15min":2.6699999999999999},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33120,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338720,"statfs":{"total":96636764160,"available":96602849280,"internally_reserved":0,"allocated":1986560,"data_stored":1316219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.63200000000000001,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.0780000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2110000000000001,"5min":1.4770000000000001,"15min":1.7},"last":1.0609999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.876},"min":{"1min":0.57399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.319,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.998,"15min":0.98299999999999998},"min":{"1min":0.76200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5189999999999999,"5min":1.54,"15min":1.7270000000000001},"last":1.2130000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.016},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.748},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.089,"5min":1.0660000000000001,"15min":1.0940000000000001},"min":{"1min":0.60599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":1.8400000000000001,"15min":2.496},"last":1.276}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2709999999999999,"15min":1.244},"min":{"1min":0.90600000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6180000000000001,"5min":1.794,"15min":1.9239999999999999},"last":1.373},{"interface":"front","average":{"1min":1.427,"5min":1.29,"15min":1.3939999999999999},"min":{"1min":1.127,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.6399999999999999,"5min":1.706,"15min":2.4039999999999999},"last":1.3420000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.306,"15min":1.351},"min":{"1min":1.042,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7270000000000001,"5min":1.8620000000000001,"15min":2.4369999999999998},"last":1.643},{"interface":"front","average":{"1min":1.319,"5min":1.2949999999999999,"15min":1.3979999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.569,"5min":2.089,"15min":2.4239999999999999},"last":1.5980000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3500000000000001,"15min":1.4550000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.421},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.357,"15min":1.3959999999999999},"min":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7549999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.474}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.93400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.851,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.415,"15min":1.462},"min":{"1min":1.103,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7909999999999999,"5min":1.881,"15min":2.306},"last":1.524}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:22.001 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T19:32:22.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.233+0000 7ff505e21640 1 Processor -- start 2023-11-08T19:32:22.235 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.233+0000 7ff505e21640 1 -- start start 2023-11-08T19:32:22.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff505e21640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff5000ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:22.236 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff505e21640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff500161250 0x7ff5001616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:22.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff505e21640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5000cda70 con 0x7ff5000cf8c0 2023-11-08T19:32:22.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff505e21640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff500161fc0 con 0x7ff500161250 2023-11-08T19:32:22.237 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff505e21640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000cdcc0 con 0x7ff50015b080 2023-11-08T19:32:22.238 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff4fffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff500161250 0x7ff5001616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:22.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff4fffff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7ff500161250 0x7ff5001616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40010/0 (socket says 172.21.15.119:40010) 2023-11-08T19:32:22.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.235+0000 7ff4fffff640 1 -- 172.21.15.119:0/2482859147 learned_addr learned my addr 172.21.15.119:0/2482859147 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:22.239 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.236+0000 7ff4fffff640 1 -- 172.21.15.119:0/2482859147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 msgr2=0x7ff5000ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:22.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.236+0000 7ff4fffff640 1 --2- 172.21.15.119:0/2482859147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff5000ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.236+0000 7ff4fffff640 1 -- 172.21.15.119:0/2482859147 >> v1:172.21.15.188:6789/0 conn(0x7ff5000cf8c0 legacy=0x7ff5000cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:22.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.236+0000 7ff4fffff640 1 -- 172.21.15.119:0/2482859147 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5000cdfa0 con 0x7ff500161250 2023-11-08T19:32:22.240 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.237+0000 7ff4fffff640 1 --2- 172.21.15.119:0/2482859147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff500161250 0x7ff5001616c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009e50 tx=0x7ff4f802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff97d5521410b9a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:22.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.237+0000 7ff4feffd640 1 -- 172.21.15.119:0/2482859147 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f802c9e0 con 0x7ff500161250 2023-11-08T19:32:22.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.237+0000 7ff4feffd640 1 -- 172.21.15.119:0/2482859147 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f802cb40 con 0x7ff500161250 2023-11-08T19:32:22.241 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.237+0000 7ff4feffd640 1 -- 172.21.15.119:0/2482859147 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f80343f0 con 0x7ff500161250 2023-11-08T19:32:22.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 -- 172.21.15.119:0/2482859147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff500161250 msgr2=0x7ff5001616c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:22.244 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 --2- 172.21.15.119:0/2482859147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff500161250 0x7ff5001616c0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009e50 tx=0x7ff4f802be40 comp rx=0 tx=0).stop 2023-11-08T19:32:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 -- 172.21.15.119:0/2482859147 shutdown_connections 2023-11-08T19:32:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 --2- 172.21.15.119:0/2482859147 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff500161250 0x7ff5001616c0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.245 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 --2- 172.21.15.119:0/2482859147 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff5000ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 -- 172.21.15.119:0/2482859147 >> 172.21.15.119:0/2482859147 conn(0x7ff50006a9f0 msgr2=0x7ff50006add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 -- 172.21.15.119:0/2482859147 shutdown_connections 2023-11-08T19:32:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.238+0000 7ff505e21640 1 -- 172.21.15.119:0/2482859147 wait complete. 2023-11-08T19:32:22.246 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.240+0000 7ff505e21640 1 Processor -- start 2023-11-08T19:32:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.240+0000 7ff505e21640 1 -- start start 2023-11-08T19:32:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff505e21640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 0x7ff50014d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:22.247 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff505e21640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff50015dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff505e21640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff500161250 0x7ff5000c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff505e21640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500155ac0 con 0x7ff5000cf8c0 2023-11-08T19:32:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff505e21640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50014de50 con 0x7ff500161250 2023-11-08T19:32:22.248 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff505e21640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5001f7d60 con 0x7ff50015b080 2023-11-08T19:32:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff4ff7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 0x7ff50014d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff4ff7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 0x7ff50014d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40018/0 (socket says 172.21.15.119:40018) 2023-11-08T19:32:22.249 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.241+0000 7ff4ff7fe640 1 -- 172.21.15.119:0/3150367973 learned_addr learned my addr 172.21.15.119:0/3150367973 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4f7fff640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff50015dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4ff7fe640 1 -- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 msgr2=0x7ff50015dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4ff7fe640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff50015dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.250 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4fffff640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff500161250 0x7ff5000c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4ff7fe640 1 -- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff500161250 msgr2=0x7ff5000c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4ff7fe640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff500161250 0x7ff5000c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.251 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4ff7fe640 1 -- 172.21.15.119:0/3150367973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f8009b30 con 0x7ff5000cf8c0 2023-11-08T19:32:22.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4f7fff640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff50015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:22.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4ff7fe640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 0x7ff50014d910 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000b4a0 tx=0x7ff4f000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:22.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0010610 con 0x7ff5000cf8c0 2023-11-08T19:32:22.252 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0010770 con 0x7ff5000cf8c0 2023-11-08T19:32:22.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff500155190 con 0x7ff5000cf8c0 2023-11-08T19:32:22.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0010b20 con 0x7ff5000cf8c0 2023-11-08T19:32:22.253 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.242+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff500155750 con 0x7ff5000cf8c0 2023-11-08T19:32:22.254 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.244+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff500074430 con 0x7ff5000cf8c0 2023-11-08T19:32:22.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.244+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff4f0010c80 con 0x7ff5000cf8c0 2023-11-08T19:32:22.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.245+0000 7ff4fcff9640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff4e006aea0 0x7ff4e006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:22.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.246+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7ff4f0089700 con 0x7ff5000cf8c0 2023-11-08T19:32:22.255 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.246+0000 7ff4f7fff640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff4e006aea0 0x7ff4e006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:22.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.247+0000 7ff4f7fff640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff4e006aea0 0x7ff4e006d360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007fd0 tx=0x7ff4e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:22.256 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.250+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff4f0054290 con 0x7ff5000cf8c0 2023-11-08T19:32:22.489 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.488+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5000d1120 con 0x7ff5000cf8c0 2023-11-08T19:32:22.491 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.490+0000 7ff4fcff9640 1 -- 172.21.15.119:0/3150367973 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+19460 (secure 0 0 0) 0x7ff4f00578b0 con 0x7ff5000cf8c0 2023-11-08T19:32:22.492 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:22.492 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":109,"fsid":"ffe0b4d9-6ece-4b5f-928d-c0e5f4e3c55d","created":"2023-11-08T19:24:33.595108+0000","modified":"2023-11-08T19:32:17.082682+0000","last_up_change":"2023-11-08T19:24:45.959621+0000","last_in_change":"2023-11-08T19:24:37.062283+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T19:24:46.723787+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"snap-test-pool","create_time":"2023-11-08T19:26:39.314051+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"test_data_pool","create_time":"2023-11-08T19:27:45.435414+0000","flags":16385,"flags_names":"hashpspool,pool_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"pool","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"test_metadata_pool","create_time":"2023-11-08T19:27:46.245481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f44842af-30d5-463b-bdab-3bf51ed607e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6800","nonce":337690},{"type":"v1","addr":"172.21.15.119:6801","nonce":337690}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6802","nonce":337690},{"type":"v1","addr":"172.21.15.119:6803","nonce":337690}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6806","nonce":337690},{"type":"v1","addr":"172.21.15.119:6807","nonce":337690}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6804","nonce":337690},{"type":"v1","addr":"172.21.15.119:6805","nonce":337690}]},"public_addr":"172.21.15.119:6801/337690","cluster_addr":"172.21.15.119:6803/337690","heartbeat_back_addr":"172.21.15.119:6807/337690","heartbeat_front_addr":"172.21.15.119:6805/337690","state":["exists","up"]},{"osd":1,"uuid":"9c1f24ae-f05b-4d9c-8867-fa908bd72d75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6824","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6825","nonce":1204926209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6826","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6827","nonce":1204926209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6830","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6831","nonce":1204926209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6828","nonce":1204926209},{"type":"v1","addr":"172.21.15.119:6829","nonce":1204926209}]},"public_addr":"172.21.15.119:6825/1204926209","cluster_addr":"172.21.15.119:6827/1204926209","heartbeat_back_addr":"172.21.15.119:6831/1204926209","heartbeat_front_addr":"172.21.15.119:6829/1204926209","state":["exists","up"]},{"osd":2,"uuid":"f623705e-975f-4df5-aa4d-61abed82929f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6809","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6811","nonce":3756217836}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6813","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6815","nonce":3756217836}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6821","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6823","nonce":3756217836}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6817","nonce":3756217836},{"type":"v1","addr":"172.21.15.119:6819","nonce":3756217836}]},"public_addr":"172.21.15.119:6811/3756217836","cluster_addr":"172.21.15.119:6815/3756217836","heartbeat_back_addr":"172.21.15.119:6823/3756217836","heartbeat_front_addr":"172.21.15.119:6819/3756217836","state":["exists","up"]},{"osd":3,"uuid":"c2435256-2139-4515-ab1c-cdef186bd730","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6808","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6810","nonce":2410755625}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6812","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6814","nonce":2410755625}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6820","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6822","nonce":2410755625}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6816","nonce":2410755625},{"type":"v1","addr":"172.21.15.119:6818","nonce":2410755625}]},"public_addr":"172.21.15.119:6810/2410755625","cluster_addr":"172.21.15.119:6814/2410755625","heartbeat_back_addr":"172.21.15.119:6822/2410755625","heartbeat_front_addr":"172.21.15.119:6818/2410755625","state":["exists","up"]},{"osd":4,"uuid":"4591576b-2c35-457c-8790-d2e9af31a168","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6801","nonce":4217594962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6803","nonce":4217594962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6807","nonce":4217594962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":4217594962},{"type":"v1","addr":"172.21.15.188:6805","nonce":4217594962}]},"public_addr":"172.21.15.188:6801/4217594962","cluster_addr":"172.21.15.188:6803/4217594962","heartbeat_back_addr":"172.21.15.188:6807/4217594962","heartbeat_front_addr":"172.21.15.188:6805/4217594962","state":["exists","up"]},{"osd":5,"uuid":"e224dfe7-9d85-4a0e-b408-43df239741d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6821","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6823","nonce":704788363}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6825","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6827","nonce":704788363}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6831","nonce":704788363}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":704788363},{"type":"v1","addr":"172.21.15.188:6829","nonce":704788363}]},"public_addr":"172.21.15.188:6823/704788363","cluster_addr":"172.21.15.188:6827/704788363","heartbeat_back_addr":"172.21.15.188:6831/704788363","heartbeat_front_addr":"172.21.15.188:6829/704788363","state":["exists","up"]},{"osd":6,"uuid":"a680ca11-7326-4758-9067-d74680391317","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6809","nonce":4161471119}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6811","nonce":4161471119}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6815","nonce":4161471119}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4161471119},{"type":"v1","addr":"172.21.15.188:6813","nonce":4161471119}]},"public_addr":"172.21.15.188:6809/4161471119","cluster_addr":"172.21.15.188:6811/4161471119","heartbeat_back_addr":"172.21.15.188:6815/4161471119","heartbeat_front_addr":"172.21.15.188:6813/4161471119","state":["exists","up"]},{"osd":7,"uuid":"2c7ab92b-e835-4319-88c7-2b4e829783ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6817","nonce":2104156129}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6819","nonce":2104156129}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6826","nonce":2104156129}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":2104156129},{"type":"v1","addr":"172.21.15.188:6822","nonce":2104156129}]},"public_addr":"172.21.15.188:6817/2104156129","cluster_addr":"172.21.15.188:6819/2104156129","heartbeat_back_addr":"172.21.15.188:6826/2104156129","heartbeat_front_addr":"172.21.15.188:6822/2104156129","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.620120+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.375922+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.417194+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.643467+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:41.670940+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.481023+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.632063+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T19:24:42.398014+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.119:6837/2823709152":"2023-11-09T19:32:14.070811+0000","172.21.15.119:6835/2823709152":"2023-11-09T19:32:14.070811+0000","172.21.15.188:6833/168553637":"2023-11-09T19:29:41.161236+0000","172.21.15.188:6835/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.188:6835/168553637":"2023-11-09T19:29:41.161236+0000","172.21.15.119:6835/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6833/865908207":"2023-11-09T19:32:14.070775+0000","172.21.15.188:6832/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.188:6832/865908207":"2023-11-09T19:32:14.070775+0000","172.21.15.119:6837/3498735251":"2023-11-09T19:30:09.035506+0000","172.21.15.188:6833/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.188:6835/310147451":"2023-11-09T19:30:37.434843+0000","172.21.15.119:6837/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.188:6833/2339927924":"2023-11-09T19:31:35.619931+0000","172.21.15.119:6835/1517894744":"2023-11-09T19:31:05.198475+0000","172.21.15.188:6834/4246462419":"2023-11-09T19:31:43.291623+0000","172.21.15.119:0/1419527548":"2023-11-08T20:31:50.849149+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T19:32:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff4e006aea0 msgr2=0x7ff4e006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:22.494 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff4e006aea0 0x7ff4e006d360 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8007fd0 tx=0x7ff4e8009040 comp rx=0 tx=0).stop 2023-11-08T19:32:22.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 msgr2=0x7ff50014d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:22.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 0x7ff50014d910 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000b4a0 tx=0x7ff4f000b960 comp rx=0 tx=0).stop 2023-11-08T19:32:22.495 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 shutdown_connections 2023-11-08T19:32:22.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7ff4e006aea0 0x7ff4e006d360 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff500161250 0x7ff5000c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff50015b080 0x7ff50015dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.496 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 --2- 172.21.15.119:0/3150367973 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7ff5000cf8c0 0x7ff50014d910 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:22.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 >> 172.21.15.119:0/3150367973 conn(0x7ff50006a9f0 msgr2=0x7ff5000d2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:22.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 shutdown_connections 2023-11-08T19:32:22.497 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:22.493+0000 7ff505e21640 1 -- 172.21.15.119:0/3150367973 wait complete. 2023-11-08T19:32:23.514 INFO:tasks.ceph:Scrubbing osd.0 2023-11-08T19:32:23.515 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:23.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.742+0000 7fa87dc89640 1 Processor -- start 2023-11-08T19:32:23.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.743+0000 7fa87dc89640 1 -- start start 2023-11-08T19:32:23.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.744+0000 7fa87dc89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 0x7fa87815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.751 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.744+0000 7fa87dc89640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780c43b0 con 0x7fa878144550 2023-11-08T19:32:23.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.744+0000 7fa87dc89640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87815c340 con 0x7fa87815bb10 2023-11-08T19:32:23.752 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.744+0000 7fa87dc89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c3a80 con 0x7fa8780cba90 2023-11-08T19:32:23.753 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.745+0000 7fa8777fe640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa878144550 0x7fa8780cb380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57890/0 (socket says 172.21.15.119:57890) 2023-11-08T19:32:23.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.745+0000 7fa8777fe640 1 -- 172.21.15.119:0/631179820 learned_addr learned my addr 172.21.15.119:0/631179820 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:23.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.745+0000 7fa86effd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 0x7fa87815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.754 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.745+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338557231 0 0) 0x7fa8780c43b0 con 0x7fa878144550 2023-11-08T19:32:23.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.745+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa864003520 con 0x7fa878144550 2023-11-08T19:32:23.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.745+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3731254169 0 0) 0x7fa864003520 con 0x7fa878144550 2023-11-08T19:32:23.755 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 msgr2=0x7fa87815b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 0x7fa87815b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 >> v1:172.21.15.188:6789/0 conn(0x7fa87815bb10 legacy=0x7fa8780c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780c3d60 con 0x7fa878144550 2023-11-08T19:32:23.756 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fa8680019f0 con 0x7fa878144550 2023-11-08T19:32:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 >> v1:172.21.15.119:6789/0 conn(0x7fa878144550 legacy=0x7fa8780cb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8640039a0 0x7fa864003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa864004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.757 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 0x7fa864005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c3a80 con 0x7fa8640047c0 2023-11-08T19:32:23.758 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86400ba30 con 0x7fa8640039a0 2023-11-08T19:32:23.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86400bbf0 con 0x7fa8780cba90 2023-11-08T19:32:23.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa86effd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa864004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa8777fe640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8640039a0 0x7fa864003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.759 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa86effd640 1 -- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 msgr2=0x7fa864005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:23.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa86effd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8780cba90 0x7fa864005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.760 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa86effd640 1 -- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8640039a0 msgr2=0x7fa864003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa86effd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8640039a0 0x7fa864003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.746+0000 7fa86effd640 1 -- 172.21.15.119:0/631179820 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa864003520 con 0x7fa8640047c0 2023-11-08T19:32:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.747+0000 7fa86effd640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa864004c40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa860009a90 tx=0x7fa8640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ede4678aa5a285 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86002c6b0 con 0x7fa8640047c0 2023-11-08T19:32:23.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86002c810 con 0x7fa8640047c0 2023-11-08T19:32:23.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa876ffd640 1 -- 172.21.15.119:0/631179820 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86002cba0 con 0x7fa8640047c0 2023-11-08T19:32:23.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa87dc89640 1 -- 172.21.15.119:0/631179820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 msgr2=0x7fa864004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa87dc89640 1 --2- 172.21.15.119:0/631179820 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa864004c40 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fa860009a90 tx=0x7fa8640057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa877fff640 1 -- 172.21.15.119:0/631179820 reap_dead start 2023-11-08T19:32:23.763 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa87dc89640 1 -- 172.21.15.119:0/631179820 shutdown_connections 2023-11-08T19:32:23.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa87dc89640 1 -- 172.21.15.119:0/631179820 >> 172.21.15.119:0/631179820 conn(0x7fa8780b0950 msgr2=0x7fa8780b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:23.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.748+0000 7fa87dc89640 1 -- 172.21.15.119:0/631179820 shutdown_connections 2023-11-08T19:32:23.764 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.749+0000 7fa87dc89640 1 -- 172.21.15.119:0/631179820 wait complete. 2023-11-08T19:32:23.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.750+0000 7fa87dc89640 1 Processor -- start 2023-11-08T19:32:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.750+0000 7fa87dc89640 1 -- start start 2023-11-08T19:32:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.751+0000 7fa87dc89640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa87813b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.751+0000 7fa87dc89640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa878140410 0x7fa8781f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.751+0000 7fa87dc89640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87813cb30 0x7fa8781f0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.751+0000 7fa87dc89640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780c6550 con 0x7fa8640047c0 2023-11-08T19:32:23.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.751+0000 7fa87dc89640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87815b8d0 con 0x7fa878140410 2023-11-08T19:32:23.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.751+0000 7fa87dc89640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87813ba90 con 0x7fa87813cb30 2023-11-08T19:32:23.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa87813b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa87813b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40030/0 (socket says 172.21.15.119:40030) 2023-11-08T19:32:23.770 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 -- 172.21.15.119:0/2753803277 learned_addr learned my addr 172.21.15.119:0/2753803277 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:23.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa877fff640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87813cb30 0x7fa8781f0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 -- 172.21.15.119:0/2753803277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87813cb30 msgr2=0x7fa8781f0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa86effd640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa878140410 0x7fa8781f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.771 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa87813cb30 0x7fa8781f0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 -- 172.21.15.119:0/2753803277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa878140410 msgr2=0x7fa8781f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa878140410 0x7fa8781f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.772 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 -- 172.21.15.119:0/2753803277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87813bd70 con 0x7fa8640047c0 2023-11-08T19:32:23.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.752+0000 7fa8777fe640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa87813b240 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa868007b30 tx=0x7fa86800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.753+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868013420 con 0x7fa8640047c0 2023-11-08T19:32:23.773 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.753+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8680135a0 con 0x7fa8640047c0 2023-11-08T19:32:23.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.753+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87813c050 con 0x7fa8640047c0 2023-11-08T19:32:23.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.753+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa868013950 con 0x7fa8640047c0 2023-11-08T19:32:23.774 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.753+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87813c610 con 0x7fa8640047c0 2023-11-08T19:32:23.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.754+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa8780c4040 con 0x7fa8640047c0 2023-11-08T19:32:23.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.755+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa868013ab0 con 0x7fa8640047c0 2023-11-08T19:32:23.775 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa874ff9640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa85806ae70 0x7fa85806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fa86808a460 con 0x7fa8640047c0 2023-11-08T19:32:23.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa86effd640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa85806ae70 0x7fa85806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa874ff9640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fa858073de0 0x7fa858076260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:23.776 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 --> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa858076930 con 0x7fa858073de0 2023-11-08T19:32:23.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa86808ca40 con 0x7fa8640047c0 2023-11-08T19:32:23.777 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.756+0000 7fa877fff640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fa858073de0 0x7fa858076260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:23.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.757+0000 7fa86effd640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa85806ae70 0x7fa85806d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa860036000 tx=0x7fa860030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.757+0000 7fa877fff640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fa858073de0 0x7fa858076260 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:23.778 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.758+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== osd.0 v2:172.21.15.119:6800/337690 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fa858076930 con 0x7fa858073de0 2023-11-08T19:32:23.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.772+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 --> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa87804ece0 con 0x7fa858073de0 2023-11-08T19:32:23.779 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa874ff9640 1 -- 172.21.15.119:0/2753803277 <== osd.0 v2:172.21.15.119:6800/337690 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa87804ece0 con 0x7fa858073de0 2023-11-08T19:32:23.780 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:23.780 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:23.780 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:23.781 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fa858073de0 msgr2=0x7fa858076260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6800/337690,v1:172.21.15.119:6801/337690] conn(0x7fa858073de0 0x7fa858076260 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa85806ae70 msgr2=0x7fa85806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.782 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa85806ae70 0x7fa85806d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa860036000 tx=0x7fa860030000 comp rx=0 tx=0).stop 2023-11-08T19:32:23.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 msgr2=0x7fa87813b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:23.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 --2- 172.21.15.119:0/2753803277 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa8640047c0 0x7fa87813b240 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fa868007b30 tx=0x7fa86800e530 comp rx=0 tx=0).stop 2023-11-08T19:32:23.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa877fff640 1 -- 172.21.15.119:0/2753803277 reap_dead start 2023-11-08T19:32:23.783 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 shutdown_connections 2023-11-08T19:32:23.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 >> 172.21.15.119:0/2753803277 conn(0x7fa8780b0950 msgr2=0x7fa8780b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:23.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.773+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 shutdown_connections 2023-11-08T19:32:23.784 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:23.774+0000 7fa87dc89640 1 -- 172.21.15.119:0/2753803277 wait complete. 2023-11-08T19:32:23.792 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-08T19:32:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.000+0000 7fe153490640 1 Processor -- start 2023-11-08T19:32:24.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.001+0000 7fe153490640 1 -- start start 2023-11-08T19:32:24.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.002+0000 7fe153490640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c1447a0 0x7fe14c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.002+0000 7fe153490640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe14c0cb170 0x7fe14c06a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.002+0000 7fe153490640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c0c3a50 con 0x7fe14c0cad30 2023-11-08T19:32:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.002+0000 7fe153490640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c070040 con 0x7fe14c0cb170 2023-11-08T19:32:24.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.002+0000 7fe153490640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c144b80 con 0x7fe14c1447a0 2023-11-08T19:32:24.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe14c0cb170 0x7fe14c06a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fe14c0cb170 0x7fe14c06a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40046/0 (socket says 172.21.15.119:40046) 2023-11-08T19:32:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 -- 172.21.15.119:0/2643958045 learned_addr learned my addr 172.21.15.119:0/2643958045 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:24.007 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe15122c640 1 --2- 172.21.15.119:0/2643958045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c1447a0 0x7fe14c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 -- 172.21.15.119:0/2643958045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c1447a0 msgr2=0x7fe14c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 --2- 172.21.15.119:0/2643958045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c1447a0 0x7fe14c15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 -- 172.21.15.119:0/2643958045 >> v1:172.21.15.188:6789/0 conn(0x7fe14c0cad30 legacy=0x7fe14c06f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.008 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.003+0000 7fe151a2d640 1 -- 172.21.15.119:0/2643958045 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0c9150 con 0x7fe14c0cb170 2023-11-08T19:32:24.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.004+0000 7fe151a2d640 1 --2- 172.21.15.119:0/2643958045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c06a120 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe144009e50 tx=0x7fe14402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6604ad73e21deec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.004+0000 7fe143fff640 1 -- 172.21.15.119:0/2643958045 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14402ca80 con 0x7fe14c0cb170 2023-11-08T19:32:24.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe143fff640 1 -- 172.21.15.119:0/2643958045 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14402cbe0 con 0x7fe14c0cb170 2023-11-08T19:32:24.009 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe143fff640 1 -- 172.21.15.119:0/2643958045 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1440343f0 con 0x7fe14c0cb170 2023-11-08T19:32:24.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 -- 172.21.15.119:0/2643958045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 msgr2=0x7fe14c06a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 --2- 172.21.15.119:0/2643958045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c06a120 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7fe144009e50 tx=0x7fe14402bee0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 -- 172.21.15.119:0/2643958045 shutdown_connections 2023-11-08T19:32:24.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 --2- 172.21.15.119:0/2643958045 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c06a120 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 --2- 172.21.15.119:0/2643958045 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c1447a0 0x7fe14c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 -- 172.21.15.119:0/2643958045 >> 172.21.15.119:0/2643958045 conn(0x7fe14c0cb610 msgr2=0x7fe14c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.011 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 -- 172.21.15.119:0/2643958045 shutdown_connections 2023-11-08T19:32:24.012 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.005+0000 7fe153490640 1 -- 172.21.15.119:0/2643958045 wait complete. 2023-11-08T19:32:24.014 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.007+0000 7fe153490640 1 Processor -- start 2023-11-08T19:32:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.007+0000 7fe153490640 1 -- start start 2023-11-08T19:32:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe153490640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0cad30 0x7fe14c15aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.015 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe153490640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe153490640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c1447a0 0x7fe14c1433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe153490640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c6450 con 0x7fe14c0cb170 2023-11-08T19:32:24.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe153490640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c159dd0 con 0x7fe14c1447a0 2023-11-08T19:32:24.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe153490640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c6630 con 0x7fe14c0cad30 2023-11-08T19:32:24.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.008+0000 7fe150a2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c159890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40062/0 (socket says 172.21.15.119:40062) 2023-11-08T19:32:24.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 -- 172.21.15.119:0/2140476968 learned_addr learned my addr 172.21.15.119:0/2140476968 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:24.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe15122c640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0cad30 0x7fe14c15aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe151a2d640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c1447a0 0x7fe14c1433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 -- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0cad30 msgr2=0x7fe14c15aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0cad30 0x7fe14c15aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 -- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c1447a0 msgr2=0x7fe14c1433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c1447a0 0x7fe14c1433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 -- 172.21.15.119:0/2140476968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009b30 con 0x7fe14c0cb170 2023-11-08T19:32:24.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe15122c640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0cad30 0x7fe14c15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:24.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe151a2d640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c1447a0 0x7fe14c1433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:24.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe150a2b640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c159890 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b3e0 tx=0x7fe13c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.009+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011830 con 0x7fe14c0cb170 2023-11-08T19:32:24.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.010+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13c0119b0 con 0x7fe14c0cb170 2023-11-08T19:32:24.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.010+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13c011d60 con 0x7fe14c0cb170 2023-11-08T19:32:24.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.010+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c0c6910 con 0x7fe14c0cb170 2023-11-08T19:32:24.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.010+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c0c6ed0 con 0x7fe14c0cb170 2023-11-08T19:32:24.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.011+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c074430 con 0x7fe14c0cb170 2023-11-08T19:32:24.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.012+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe13c01a400 con 0x7fe14c0cb170 2023-11-08T19:32:24.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.013+0000 7fe141ffb640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe13006ae70 0x7fe13006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.013+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fe13c089c50 con 0x7fe14c0cb170 2023-11-08T19:32:24.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.013+0000 7fe15122c640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe13006ae70 0x7fe13006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.014+0000 7fe15122c640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe13006ae70 0x7fe13006d330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe148006fd0 tx=0x7fe148008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.017+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe13c08f020 con 0x7fe14c0cb170 2023-11-08T19:32:24.208 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.206+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fe14c0c96f0 con 0x7fe13006ae70 2023-11-08T19:32:24.209 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.207+0000 7fe141ffb640 1 -- 172.21.15.119:0/2140476968 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe148009e10 con 0x7fe13006ae70 2023-11-08T19:32:24.209 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 0 to deep-scrub 2023-11-08T19:32:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe13006ae70 msgr2=0x7fe13006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.212 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe13006ae70 0x7fe13006d330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe148006fd0 tx=0x7fe148008040 comp rx=0 tx=0).stop 2023-11-08T19:32:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 msgr2=0x7fe14c159890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.213 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c159890 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fe13c00b3e0 tx=0x7fe13c00b8a0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.214 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 shutdown_connections 2023-11-08T19:32:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fe13006ae70 0x7fe13006d330 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.215 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe14c1447a0 0x7fe14c1433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe14c0cb170 0x7fe14c159890 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 --2- 172.21.15.119:0/2140476968 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe14c0cad30 0x7fe14c15aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.216 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 >> 172.21.15.119:0/2140476968 conn(0x7fe14c0cb610 msgr2=0x7fe14c0711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.210+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 shutdown_connections 2023-11-08T19:32:24.217 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.211+0000 7fe153490640 1 -- 172.21.15.119:0/2140476968 wait complete. 2023-11-08T19:32:24.230 INFO:tasks.ceph:Scrubbing osd.1 2023-11-08T19:32:24.231 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.448+0000 7f625a7e2640 1 Processor -- start 2023-11-08T19:32:24.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.448+0000 7f625a7e2640 1 -- start start 2023-11-08T19:32:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f625a7e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540716a0 0x7f62540c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f625a7e2640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f62540c9e70 0x7f62540cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f625a7e2640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62540711a0 con 0x7f6254150350 2023-11-08T19:32:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f625a7e2640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62540ca300 con 0x7f62540c9e70 2023-11-08T19:32:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f625a7e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625406fe30 con 0x7f62540716a0 2023-11-08T19:32:24.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f6253fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f6254150350 0x7f625415a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57902/0 (socket says 172.21.15.119:57902) 2023-11-08T19:32:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f6253fff640 1 -- 172.21.15.119:0/2708287379 learned_addr learned my addr 172.21.15.119:0/2708287379 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.449+0000 7f6258d7f640 1 --2- 172.21.15.119:0/2708287379 >> v2:172.21.15.119:3300/0 conn(0x7f62540c9e70 0x7f62540cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f624bfff640 1 --2- 172.21.15.119:0/2708287379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540716a0 0x7f62540c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f62537fe640 1 -- 172.21.15.119:0/2708287379 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3998253807 0 0) 0x7f62540711a0 con 0x7f6254150350 2023-11-08T19:32:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f62537fe640 1 -- 172.21.15.119:0/2708287379 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6240005730 con 0x7f6254150350 2023-11-08T19:32:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f6258d7f640 1 -- 172.21.15.119:0/2708287379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540716a0 msgr2=0x7f62540c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f6258d7f640 1 --2- 172.21.15.119:0/2708287379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540716a0 0x7f62540c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f6258d7f640 1 -- 172.21.15.119:0/2708287379 >> v1:172.21.15.119:6789/0 conn(0x7f6254150350 legacy=0x7f625415a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.450+0000 7f6258d7f640 1 -- 172.21.15.119:0/2708287379 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254070100 con 0x7f62540c9e70 2023-11-08T19:32:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.451+0000 7f6258d7f640 1 --2- 172.21.15.119:0/2708287379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540c9e70 0x7f62540cdca0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f624c009d90 tx=0x7f624c02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e62f2c52ad7077a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.451+0000 7f62537fe640 1 -- 172.21.15.119:0/2708287379 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c02cc50 con 0x7f62540c9e70 2023-11-08T19:32:24.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.451+0000 7f62537fe640 1 -- 172.21.15.119:0/2708287379 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c02cdb0 con 0x7f62540c9e70 2023-11-08T19:32:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.451+0000 7f62537fe640 1 -- 172.21.15.119:0/2708287379 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c034660 con 0x7f62540c9e70 2023-11-08T19:32:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 -- 172.21.15.119:0/2708287379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540c9e70 msgr2=0x7f62540cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 --2- 172.21.15.119:0/2708287379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540c9e70 0x7f62540cdca0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f624c009d90 tx=0x7f624c02bea0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 -- 172.21.15.119:0/2708287379 shutdown_connections 2023-11-08T19:32:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 --2- 172.21.15.119:0/2708287379 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540c9e70 0x7f62540cdca0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 --2- 172.21.15.119:0/2708287379 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540716a0 0x7f62540c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 -- 172.21.15.119:0/2708287379 >> 172.21.15.119:0/2708287379 conn(0x7f62540b0950 msgr2=0x7f62540b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 -- 172.21.15.119:0/2708287379 shutdown_connections 2023-11-08T19:32:24.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.452+0000 7f625a7e2640 1 -- 172.21.15.119:0/2708287379 wait complete. 2023-11-08T19:32:24.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.454+0000 7f625a7e2640 1 Processor -- start 2023-11-08T19:32:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.454+0000 7f625a7e2640 1 -- start start 2023-11-08T19:32:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f625a7e2640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540716a0 0x7f625414e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f625a7e2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540c9e70 0x7f6254158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f625a7e2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6254150350 0x7f6254158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f625a7e2640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c6d90 con 0x7f62540716a0 2023-11-08T19:32:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f625a7e2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c6180 con 0x7f6254150350 2023-11-08T19:32:24.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f625a7e2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62540c6460 con 0x7f62540c9e70 2023-11-08T19:32:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f6253fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540716a0 0x7f625414e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f6253fff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540716a0 0x7f625414e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40084/0 (socket says 172.21.15.119:40084) 2023-11-08T19:32:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.455+0000 7f6253fff640 1 -- 172.21.15.119:0/59190141 learned_addr learned my addr 172.21.15.119:0/59190141 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:24.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f624bfff640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540c9e70 0x7f6254158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6258d7f640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6254150350 0x7f6254158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6253fff640 1 -- 172.21.15.119:0/59190141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540c9e70 msgr2=0x7f6254158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.465 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6253fff640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f62540c9e70 0x7f6254158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6253fff640 1 -- 172.21.15.119:0/59190141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6254150350 msgr2=0x7f6254158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6253fff640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6254150350 0x7f6254158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6253fff640 1 -- 172.21.15.119:0/59190141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009a70 con 0x7f62540716a0 2023-11-08T19:32:24.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f6253fff640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540716a0 0x7f625414e230 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f624400c410 tx=0x7f624400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6244005ce0 con 0x7f62540716a0 2023-11-08T19:32:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6244005e40 con 0x7f62540716a0 2023-11-08T19:32:24.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62440072f0 con 0x7f62540716a0 2023-11-08T19:32:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.456+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62540c6740 con 0x7f62540716a0 2023-11-08T19:32:24.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.457+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625414e770 con 0x7f62540716a0 2023-11-08T19:32:24.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.457+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f62540706c0 con 0x7f62540716a0 2023-11-08T19:32:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.459+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6244007450 con 0x7f62540716a0 2023-11-08T19:32:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.459+0000 7f62517fa640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f623406aea0 0x7f623406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f62440891d0 con 0x7f62540716a0 2023-11-08T19:32:24.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f624bfff640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f623406aea0 0x7f623406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f62517fa640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f6234073e10 0x7f6234076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 --> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6234076960 con 0x7f6234073e10 2023-11-08T19:32:24.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624408b7b0 con 0x7f62540716a0 2023-11-08T19:32:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f6258d7f640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f6234073e10 0x7f6234076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.460+0000 7f624bfff640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f623406aea0 0x7f623406d360 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f62540b29e0 tx=0x7f623c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.461+0000 7f6258d7f640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f6234073e10 0x7f6234076290 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.461+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== osd.1 v2:172.21.15.119:6824/1204926209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f6234076960 con 0x7f6234073e10 2023-11-08T19:32:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.476+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 --> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f62540703e0 con 0x7f6234073e10 2023-11-08T19:32:24.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.476+0000 7f62517fa640 1 -- 172.21.15.119:0/59190141 <== osd.1 v2:172.21.15.119:6824/1204926209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f624c009450 con 0x7f6234073e10 2023-11-08T19:32:24.478 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:24.478 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:24.479 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:24.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.476+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f6234073e10 msgr2=0x7f6234076290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.476+0000 7f625a7e2640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6824/1204926209,v1:172.21.15.119:6825/1204926209] conn(0x7f6234073e10 0x7f6234076290 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f623406aea0 msgr2=0x7f623406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f623406aea0 0x7f623406d360 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f62540b29e0 tx=0x7f623c008040 comp rx=0 tx=0).stop 2023-11-08T19:32:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540716a0 msgr2=0x7f625414e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 --2- 172.21.15.119:0/59190141 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f62540716a0 0x7f625414e230 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f624400c410 tx=0x7f624400c8d0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.481 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f6258d7f640 1 -- 172.21.15.119:0/59190141 reap_dead start 2023-11-08T19:32:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 shutdown_connections 2023-11-08T19:32:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 >> 172.21.15.119:0/59190141 conn(0x7f62540b0950 msgr2=0x7f625414f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.482 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 shutdown_connections 2023-11-08T19:32:24.483 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.477+0000 7f625a7e2640 1 -- 172.21.15.119:0/59190141 wait complete. 2023-11-08T19:32:24.495 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-08T19:32:24.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.713+0000 7f33fc88a640 1 Processor -- start 2023-11-08T19:32:24.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.714+0000 7f33fc88a640 1 -- start start 2023-11-08T19:32:24.716 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33fc88a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33f80d1c50 0x7f33f80b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33fc88a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f33f80d0d40 0x7f33f80b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33fc88a640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80b2fb0 con 0x7f33f815d2e0 2023-11-08T19:32:24.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33fc88a640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80ceeb0 con 0x7f33f80d1c50 2023-11-08T19:32:24.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33fc88a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80cebd0 con 0x7f33f80d0d40 2023-11-08T19:32:24.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33f6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33f80d1c50 0x7f33f80b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33f6ffd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f33f80d1c50 0x7f33f80b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40088/0 (socket says 172.21.15.119:40088) 2023-11-08T19:32:24.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.715+0000 7f33f6ffd640 1 -- 172.21.15.119:0/2189671185 learned_addr learned my addr 172.21.15.119:0/2189671185 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:24.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.716+0000 7f33f7fff640 1 --2- 172.21.15.119:0/2189671185 >> v2:172.21.15.188:3300/0 conn(0x7f33f80d0d40 0x7f33f80b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.716+0000 7f33f6ffd640 1 -- 172.21.15.119:0/2189671185 >> v2:172.21.15.188:3300/0 conn(0x7f33f80d0d40 msgr2=0x7f33f80b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:24.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.716+0000 7f33f6ffd640 1 --2- 172.21.15.119:0/2189671185 >> v2:172.21.15.188:3300/0 conn(0x7f33f80d0d40 0x7f33f80b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.716+0000 7f33f6ffd640 1 -- 172.21.15.119:0/2189671185 >> v1:172.21.15.119:6789/0 conn(0x7f33f815d2e0 legacy=0x7f33f80d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.716+0000 7f33f6ffd640 1 -- 172.21.15.119:0/2189671185 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80b2960 con 0x7f33f80d1c50 2023-11-08T19:32:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.717+0000 7f33f6ffd640 1 --2- 172.21.15.119:0/2189671185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f80b3440 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f33f8160d80 tx=0x7f33e002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec112e2b079ffd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.722 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.717+0000 7f33f67fc640 1 -- 172.21.15.119:0/2189671185 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e002f850 con 0x7f33f80d1c50 2023-11-08T19:32:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.717+0000 7f33f67fc640 1 -- 172.21.15.119:0/2189671185 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e002f9b0 con 0x7f33f80d1c50 2023-11-08T19:32:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.717+0000 7f33f67fc640 1 -- 172.21.15.119:0/2189671185 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e002fd40 con 0x7f33f80d1c50 2023-11-08T19:32:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 -- 172.21.15.119:0/2189671185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 msgr2=0x7f33f80b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.723 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 --2- 172.21.15.119:0/2189671185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f80b3440 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f33f8160d80 tx=0x7f33e002ba30 comp rx=0 tx=0).stop 2023-11-08T19:32:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 -- 172.21.15.119:0/2189671185 shutdown_connections 2023-11-08T19:32:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 --2- 172.21.15.119:0/2189671185 >> v2:172.21.15.188:3300/0 conn(0x7f33f80d0d40 0x7f33f80b3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 --2- 172.21.15.119:0/2189671185 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f80b3440 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 -- 172.21.15.119:0/2189671185 >> 172.21.15.119:0/2189671185 conn(0x7f33f80d2f60 msgr2=0x7f33f80d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 -- 172.21.15.119:0/2189671185 shutdown_connections 2023-11-08T19:32:24.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.718+0000 7f33fc88a640 1 -- 172.21.15.119:0/2189671185 wait complete. 2023-11-08T19:32:24.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.720+0000 7f33fc88a640 1 Processor -- start 2023-11-08T19:32:24.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.720+0000 7f33fc88a640 1 -- start start 2023-11-08T19:32:24.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33fc88a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f80d0d40 0x7f33f815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33fc88a640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f806b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33fc88a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f815d2e0 0x7f33f806bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33fc88a640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f81e7620 con 0x7f33f80d1c50 2023-11-08T19:32:24.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33fc88a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c74a0 con 0x7f33f815d2e0 2023-11-08T19:32:24.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33fc88a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f81e6a10 con 0x7f33f80d0d40 2023-11-08T19:32:24.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f806b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f806b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40096/0 (socket says 172.21.15.119:40096) 2023-11-08T19:32:24.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 -- 172.21.15.119:0/1952117142 learned_addr learned my addr 172.21.15.119:0/1952117142 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:24.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f7fff640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f815d2e0 0x7f33f806bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f77fe640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f80d0d40 0x7f33f815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 -- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f80d0d40 msgr2=0x7f33f815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f80d0d40 0x7f33f815a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 -- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f815d2e0 msgr2=0x7f33f806bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.721+0000 7f33f6ffd640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f815d2e0 0x7f33f806bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33f6ffd640 1 -- 172.21.15.119:0/1952117142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00094d0 con 0x7f33f80d1c50 2023-11-08T19:32:24.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33f6ffd640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f806b870 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f33e002ba70 tx=0x7f33e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0037d10 con 0x7f33f80d1c50 2023-11-08T19:32:24.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0037e90 con 0x7f33f80d1c50 2023-11-08T19:32:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f81e6cf0 con 0x7f33f80d1c50 2023-11-08T19:32:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0041640 con 0x7f33f80d1c50 2023-11-08T19:32:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.722+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f81e72b0 con 0x7f33f80d1c50 2023-11-08T19:32:24.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.723+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f804ece0 con 0x7f33f80d1c50 2023-11-08T19:32:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.724+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33e00417a0 con 0x7f33f80d1c50 2023-11-08T19:32:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.726+0000 7f33effff640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33d806ae70 0x7f33d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:24.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.726+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f33e003c070 con 0x7f33f80d1c50 2023-11-08T19:32:24.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.726+0000 7f33f77fe640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33d806ae70 0x7f33d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:24.736 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.727+0000 7f33f77fe640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33d806ae70 0x7f33d806d330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f33e8007fd0 tx=0x7f33e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:24.737 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.730+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33e00774d0 con 0x7f33f80d1c50 2023-11-08T19:32:24.926 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.925+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f33f815d6c0 con 0x7f33d806ae70 2023-11-08T19:32:24.927 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.926+0000 7f33effff640 1 -- 172.21.15.119:0/1952117142 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f33e8003990 con 0x7f33d806ae70 2023-11-08T19:32:24.928 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 1 to deep-scrub 2023-11-08T19:32:24.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33d806ae70 msgr2=0x7f33d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.930 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33d806ae70 0x7f33d806d330 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f33e8007fd0 tx=0x7f33e8009040 comp rx=0 tx=0).stop 2023-11-08T19:32:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 msgr2=0x7f33f806b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f806b870 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f33e002ba70 tx=0x7f33e0004730 comp rx=0 tx=0).stop 2023-11-08T19:32:24.931 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 shutdown_connections 2023-11-08T19:32:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f33d806ae70 0x7f33d806d330 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f33f815d2e0 0x7f33f806bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.932 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f33f80d1c50 0x7f33f806b870 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 --2- 172.21.15.119:0/1952117142 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f33f80d0d40 0x7f33f815a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:24.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 >> 172.21.15.119:0/1952117142 conn(0x7f33f80d2f60 msgr2=0x7f33f815c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:24.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 shutdown_connections 2023-11-08T19:32:24.933 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:24.929+0000 7f33fc88a640 1 -- 172.21.15.119:0/1952117142 wait complete. 2023-11-08T19:32:24.947 INFO:tasks.ceph:Scrubbing osd.2 2023-11-08T19:32:24.947 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:25.166 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.165+0000 7fa6a824f640 1 Processor -- start 2023-11-08T19:32:25.167 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.165+0000 7fa6a824f640 1 -- start start 2023-11-08T19:32:25.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.167+0000 7fa6a824f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 0x7fa6a016c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.168 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.167+0000 7fa6a824f640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00c42a0 con 0x7fa6a014a1e0 2023-11-08T19:32:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.167+0000 7fa6a824f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00c4580 con 0x7fa6a014af50 2023-11-08T19:32:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.167+0000 7fa6a824f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00cdf40 con 0x7fa6a00cb990 2023-11-08T19:32:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.167+0000 7fa6a5feb640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fa6a014a1e0 0x7fa6a00c39e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57920/0 (socket says 172.21.15.119:57920) 2023-11-08T19:32:25.169 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.167+0000 7fa6a5feb640 1 -- 172.21.15.119:0/2896081779 learned_addr learned my addr 172.21.15.119:0/2896081779 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:25.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 0x7fa6a016c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 607215239 0 0) 0x7fa6a00c42a0 con 0x7fa6a014a1e0 2023-11-08T19:32:25.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa68c003520 con 0x7fa6a014a1e0 2023-11-08T19:32:25.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 936655359 0 0) 0x7fa68c003520 con 0x7fa6a014a1e0 2023-11-08T19:32:25.174 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 msgr2=0x7fa6a016c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 0x7fa6a016c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 >> v1:172.21.15.188:6789/0 conn(0x7fa6a014af50 legacy=0x7fa6a00cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00c4860 con 0x7fa6a014a1e0 2023-11-08T19:32:25.175 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fa69c001690 con 0x7fa6a014a1e0 2023-11-08T19:32:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 >> v1:172.21.15.119:6789/0 conn(0x7fa6a014a1e0 legacy=0x7fa6a00c39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.176 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68c0039a0 0x7fa68c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 0x7fa68c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa68c004d00 0x7fa68c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00cdf40 con 0x7fa68c004d00 2023-11-08T19:32:25.177 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c00bb40 con 0x7fa68c0039a0 2023-11-08T19:32:25.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.168+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c00bd00 con 0x7fa6a00cb990 2023-11-08T19:32:25.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa68c004d00 0x7fa68c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.178 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a57ea640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 0x7fa68c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a67ec640 1 -- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 msgr2=0x7fa68c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00cb990 0x7fa68c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a67ec640 1 -- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68c0039a0 msgr2=0x7fa68c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.179 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa68c0039a0 0x7fa68c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.169+0000 7fa6a67ec640 1 -- 172.21.15.119:0/2896081779 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c003520 con 0x7fa68c004d00 2023-11-08T19:32:25.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.170+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa68c004d00 0x7fa68c005180 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa698009fc0 tx=0x7fa68c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2021d5869a0c30a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.180 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69802cc20 con 0x7fa68c004d00 2023-11-08T19:32:25.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69800f040 con 0x7fa68c004d00 2023-11-08T19:32:25.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a4fe9640 1 -- 172.21.15.119:0/2896081779 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa698034440 con 0x7fa68c004d00 2023-11-08T19:32:25.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a824f640 1 -- 172.21.15.119:0/2896081779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa68c004d00 msgr2=0x7fa68c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a824f640 1 --2- 172.21.15.119:0/2896081779 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa68c004d00 0x7fa68c005180 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa698009fc0 tx=0x7fa68c0057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a67ec640 1 -- 172.21.15.119:0/2896081779 reap_dead start 2023-11-08T19:32:25.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a824f640 1 -- 172.21.15.119:0/2896081779 shutdown_connections 2023-11-08T19:32:25.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a824f640 1 -- 172.21.15.119:0/2896081779 >> 172.21.15.119:0/2896081779 conn(0x7fa6a00b0950 msgr2=0x7fa6a00b0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:25.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a824f640 1 -- 172.21.15.119:0/2896081779 shutdown_connections 2023-11-08T19:32:25.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.171+0000 7fa6a824f640 1 -- 172.21.15.119:0/2896081779 wait complete. 2023-11-08T19:32:25.186 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.173+0000 7fa6a824f640 1 Processor -- start 2023-11-08T19:32:25.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.173+0000 7fa6a824f640 1 -- start start 2023-11-08T19:32:25.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a824f640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a00b4050 0x7fa6a00b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a824f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6a00b4430 0x7fa6a00b24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.187 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a824f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00b29e0 0x7fa6a01fa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a824f640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00b3c40 con 0x7fa6a00b4050 2023-11-08T19:32:25.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a824f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00b1a00 con 0x7fa6a00b4430 2023-11-08T19:32:25.188 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a824f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00b3960 con 0x7fa6a00b29e0 2023-11-08T19:32:25.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a5feb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a00b4050 0x7fa6a00b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a5feb640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a00b4050 0x7fa6a00b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40104/0 (socket says 172.21.15.119:40104) 2023-11-08T19:32:25.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.174+0000 7fa6a5feb640 1 -- 172.21.15.119:0/409800192 learned_addr learned my addr 172.21.15.119:0/409800192 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:25.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a57ea640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6a00b4430 0x7fa6a00b24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00b29e0 0x7fa6a01fa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a5feb640 1 -- 172.21.15.119:0/409800192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00b29e0 msgr2=0x7fa6a01fa700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a5feb640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa6a00b29e0 0x7fa6a01fa700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.191 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a5feb640 1 -- 172.21.15.119:0/409800192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6a00b4430 msgr2=0x7fa6a00b24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a5feb640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa6a00b4430 0x7fa6a00b24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a5feb640 1 -- 172.21.15.119:0/409800192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00b1ca0 con 0x7fa6a00b4050 2023-11-08T19:32:25.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a5feb640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a00b4050 0x7fa6a00b3060 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c6c0 tx=0x7fa69c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.192 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c004aa0 con 0x7fa6a00b4050 2023-11-08T19:32:25.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69c004c20 con 0x7fa6a00b4050 2023-11-08T19:32:25.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a00b1f80 con 0x7fa6a00b4050 2023-11-08T19:32:25.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.175+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69c0070d0 con 0x7fa6a00b4050 2023-11-08T19:32:25.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.176+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01fac40 con 0x7fa6a00b4050 2023-11-08T19:32:25.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.176+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa6a00cca10 con 0x7fa6a00b4050 2023-11-08T19:32:25.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.177+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa69c007230 con 0x7fa6a00b4050 2023-11-08T19:32:25.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.178+0000 7fa6967fc640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa68406aec0 0x7fa68406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.178+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fa69c089320 con 0x7fa6a00b4050 2023-11-08T19:32:25.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.179+0000 7fa6a57ea640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa68406aec0 0x7fa68406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.179+0000 7fa6967fc640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa684073e30 0x7fa6840762b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.179+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 --> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa684076980 con 0x7fa684073e30 2023-11-08T19:32:25.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.179+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa69c08ba30 con 0x7fa6a00b4050 2023-11-08T19:32:25.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.179+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa684073e30 0x7fa6840762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.179+0000 7fa6a57ea640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa68406aec0 0x7fa68406d380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa6a015e310 tx=0x7fa690009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.180+0000 7fa6a67ec640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa684073e30 0x7fa6840762b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.197 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.180+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== osd.2 v2:172.21.15.119:6809/3756217836 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fa684076980 con 0x7fa684073e30 2023-11-08T19:32:25.198 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.196+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 --> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa6a00cc730 con 0x7fa684073e30 2023-11-08T19:32:25.199 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.196+0000 7fa6967fc640 1 -- 172.21.15.119:0/409800192 <== osd.2 v2:172.21.15.119:6809/3756217836 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa69802cb40 con 0x7fa684073e30 2023-11-08T19:32:25.199 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:25.200 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:25.200 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:25.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa684073e30 msgr2=0x7fa6840762b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.201 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6809/3756217836,v1:172.21.15.119:6811/3756217836] conn(0x7fa684073e30 0x7fa6840762b0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa68406aec0 msgr2=0x7fa68406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fa68406aec0 0x7fa68406d380 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fa6a015e310 tx=0x7fa690009040 comp rx=0 tx=0).stop 2023-11-08T19:32:25.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a00b4050 msgr2=0x7fa6a00b3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.202 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 --2- 172.21.15.119:0/409800192 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fa6a00b4050 0x7fa6a00b3060 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fa69c00c6c0 tx=0x7fa69c00cb80 comp rx=0 tx=0).stop 2023-11-08T19:32:25.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a67ec640 1 -- 172.21.15.119:0/409800192 reap_dead start 2023-11-08T19:32:25.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 shutdown_connections 2023-11-08T19:32:25.203 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 >> 172.21.15.119:0/409800192 conn(0x7fa6a00b0950 msgr2=0x7fa6a00c9240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:25.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 shutdown_connections 2023-11-08T19:32:25.204 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.197+0000 7fa6a824f640 1 -- 172.21.15.119:0/409800192 wait complete. 2023-11-08T19:32:25.215 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-08T19:32:25.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.429+0000 7fe00cb90640 1 Processor -- start 2023-11-08T19:32:25.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.429+0000 7fe00cb90640 1 -- start start 2023-11-08T19:32:25.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00cb90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 0x7fe00806f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00cb90640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe00815f900 con 0x7fe0081447a0 2023-11-08T19:32:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00cb90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0080c3a50 con 0x7fe00806a120 2023-11-08T19:32:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00cb90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe008144b80 con 0x7fe00815f4c0 2023-11-08T19:32:25.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00659c640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fe0081447a0 0x7fe0080cad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57924/0 (socket says 172.21.15.119:57924) 2023-11-08T19:32:25.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00659c640 1 -- 172.21.15.119:0/441736369 learned_addr learned my addr 172.21.15.119:0/441736369 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe005d9b640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 0x7fe00806f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.431+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033425491 0 0) 0x7fdff8001110 con 0x7fe0081447a0 2023-11-08T19:32:25.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfe8003520 con 0x7fe0081447a0 2023-11-08T19:32:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070286773 0 0) 0x7fdfe8003520 con 0x7fe0081447a0 2023-11-08T19:32:25.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 msgr2=0x7fe00806f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 0x7fe00806f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 >> v1:172.21.15.188:6789/0 conn(0x7fe00806a120 legacy=0x7fe0080c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0080c9150 con 0x7fe0081447a0 2023-11-08T19:32:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fdff8001800 con 0x7fe0081447a0 2023-11-08T19:32:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 >> v1:172.21.15.119:6789/0 conn(0x7fe0081447a0 legacy=0x7fe0080cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 0x7fdfe80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8003f70 0x7fdfe80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfe8004d00 0x7fdfe8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008144b80 con 0x7fdfe8004d00 2023-11-08T19:32:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe800bac0 con 0x7fdfe8003f70 2023-11-08T19:32:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe800bc80 con 0x7fe00815f4c0 2023-11-08T19:32:25.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.432+0000 7fe006d9d640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfe8004d00 0x7fdfe8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe005d9b640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8003f70 0x7fdfe80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe00659c640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 0x7fdfe80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe006d9d640 1 -- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 msgr2=0x7fdfe80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe006d9d640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815f4c0 0x7fdfe80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe006d9d640 1 -- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8003f70 msgr2=0x7fdfe80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe006d9d640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8003f70 0x7fdfe80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.433+0000 7fe006d9d640 1 -- 172.21.15.119:0/441736369 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe8003520 con 0x7fdfe8004d00 2023-11-08T19:32:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.434+0000 7fe006d9d640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfe8004d00 0x7fdfe8005180 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fe000001750 tx=0x7fdfe80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c39674f48acaa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.434+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000052d0 con 0x7fdfe8004d00 2023-11-08T19:32:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.434+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00002e050 con 0x7fdfe8004d00 2023-11-08T19:32:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.434+0000 7fe00559a640 1 -- 172.21.15.119:0/441736369 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000003830 con 0x7fdfe8004d00 2023-11-08T19:32:25.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe00cb90640 1 -- 172.21.15.119:0/441736369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfe8004d00 msgr2=0x7fdfe8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe00cb90640 1 --2- 172.21.15.119:0/441736369 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fdfe8004d00 0x7fdfe8005180 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fe000001750 tx=0x7fdfe80057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe006d9d640 1 -- 172.21.15.119:0/441736369 reap_dead start 2023-11-08T19:32:25.447 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe00cb90640 1 -- 172.21.15.119:0/441736369 shutdown_connections 2023-11-08T19:32:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe00cb90640 1 -- 172.21.15.119:0/441736369 >> 172.21.15.119:0/441736369 conn(0x7fe0080cb610 msgr2=0x7fe0080cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe00cb90640 1 -- 172.21.15.119:0/441736369 shutdown_connections 2023-11-08T19:32:25.448 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.435+0000 7fe00cb90640 1 -- 172.21.15.119:0/441736369 wait complete. 2023-11-08T19:32:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.437+0000 7fe00cb90640 1 Processor -- start 2023-11-08T19:32:25.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.437+0000 7fe00cb90640 1 -- start start 2023-11-08T19:32:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00cb90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8004d00 0x7fe00815b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00cb90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815a8b0 0x7fe00815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00cb90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 0x7fe0080cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00cb90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0080cda70 con 0x7fe0080cd0b0 2023-11-08T19:32:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00cb90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe008143f60 con 0x7fdfe8004d00 2023-11-08T19:32:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00cb90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe008141e30 con 0x7fe00815a8b0 2023-11-08T19:32:25.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 0x7fe0080cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 0x7fe0080cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40118/0 (socket says 172.21.15.119:40118) 2023-11-08T19:32:25.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 -- 172.21.15.119:0/4058480287 learned_addr learned my addr 172.21.15.119:0/4058480287 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:25.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 -- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815a8b0 msgr2=0x7fe00815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815a8b0 0x7fe00815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 -- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8004d00 msgr2=0x7fe00815b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe00659c640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8004d00 0x7fe00815b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8004d00 0x7fe00815b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.438+0000 7fe006d9d640 1 -- 172.21.15.119:0/4058480287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00815bb90 con 0x7fe0080cd0b0 2023-11-08T19:32:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.439+0000 7fe006d9d640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 0x7fe0080cd530 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe000001870 tx=0x7fe000001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.439+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000043070 con 0x7fe0080cd0b0 2023-11-08T19:32:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.439+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe000007a30 con 0x7fe0080cd0b0 2023-11-08T19:32:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.439+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0000067d0 con 0x7fe0080cd0b0 2023-11-08T19:32:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.439+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00815be70 con 0x7fe0080cd0b0 2023-11-08T19:32:25.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.439+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe00815a5c0 con 0x7fe0080cd0b0 2023-11-08T19:32:25.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.440+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe008142030 con 0x7fe0080cd0b0 2023-11-08T19:32:25.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.441+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe000006930 con 0x7fe0080cd0b0 2023-11-08T19:32:25.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.442+0000 7fdffeffd640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfec06ae70 0x7fdfec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.442+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fe0000aad40 con 0x7fe0080cd0b0 2023-11-08T19:32:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.443+0000 7fe00659c640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfec06ae70 0x7fdfec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.443+0000 7fe00659c640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfec06ae70 0x7fdfec06d330 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdff8001930 tx=0x7fdff8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.446+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe008142030 con 0x7fe0080cd0b0 2023-11-08T19:32:25.641 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.640+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0080c96f0 con 0x7fdfec06ae70 2023-11-08T19:32:25.642 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.641+0000 7fdffeffd640 1 -- 172.21.15.119:0/4058480287 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdff800d040 con 0x7fdfec06ae70 2023-11-08T19:32:25.643 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 2 to deep-scrub 2023-11-08T19:32:25.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfec06ae70 msgr2=0x7fdfec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfec06ae70 0x7fdfec06d330 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdff8001930 tx=0x7fdff8009040 comp rx=0 tx=0).stop 2023-11-08T19:32:25.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 msgr2=0x7fe0080cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 0x7fe0080cd530 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fe000001870 tx=0x7fe000001120 comp rx=0 tx=0).stop 2023-11-08T19:32:25.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 shutdown_connections 2023-11-08T19:32:25.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fdfec06ae70 0x7fdfec06d330 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fe0080cd0b0 0x7fe0080cd530 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe00815a8b0 0x7fe00815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 --2- 172.21.15.119:0/4058480287 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdfe8004d00 0x7fe00815b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 >> 172.21.15.119:0/4058480287 conn(0x7fe0080cb610 msgr2=0x7fe0080cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:25.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.644+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 shutdown_connections 2023-11-08T19:32:25.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.645+0000 7fe00cb90640 1 -- 172.21.15.119:0/4058480287 wait complete. 2023-11-08T19:32:25.664 INFO:tasks.ceph:Scrubbing osd.3 2023-11-08T19:32:25.665 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.876+0000 7f1053303640 1 Processor -- start 2023-11-08T19:32:25.878 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.877+0000 7f1053303640 1 -- start start 2023-11-08T19:32:25.879 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1053303640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f104c0d1c50 0x7f104c0b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1053303640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 0x7f104c0b3980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1053303640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104c0b2fb0 con 0x7f104c15d2e0 2023-11-08T19:32:25.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1053303640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104c0ceeb0 con 0x7f104c0d1c50 2023-11-08T19:32:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1053303640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0cebd0 con 0x7f104c0d0d40 2023-11-08T19:32:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1052301640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f104c15d2e0 0x7f104c0d1540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57936/0 (socket says 172.21.15.119:57936) 2023-11-08T19:32:25.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.878+0000 7f1052301640 1 -- 172.21.15.119:0/197721383 learned_addr learned my addr 172.21.15.119:0/197721383 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:25.884 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276953481 0 0) 0x7f104c0b2fb0 con 0x7f104c15d2e0 2023-11-08T19:32:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f1051b00640 1 --2- 172.21.15.119:0/197721383 >> v2:172.21.15.188:3300/0 conn(0x7f104c0d1c50 0x7f104c0b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1034003520 con 0x7f104c15d2e0 2023-11-08T19:32:25.885 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f1052b02640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 0x7f104c0b3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1231033672 0 0) 0x7f1034003520 con 0x7f104c15d2e0 2023-11-08T19:32:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 msgr2=0x7f104c0b3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 0x7f104c0b3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 >> v2:172.21.15.188:3300/0 conn(0x7f104c0d1c50 msgr2=0x7f104c0b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 --2- 172.21.15.119:0/197721383 >> v2:172.21.15.188:3300/0 conn(0x7f104c0d1c50 0x7f104c0b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c0b2960 con 0x7f104c15d2e0 2023-11-08T19:32:25.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f103c001670 con 0x7f104c15d2e0 2023-11-08T19:32:25.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 >> v1:172.21.15.119:6789/0 conn(0x7f104c15d2e0 legacy=0x7f104c0d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1034003970 0x7f1034003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 0x7f10340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f1034005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0ceeb0 con 0x7f1034004d00 2023-11-08T19:32:25.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c0cebd0 con 0x7f1034003970 2023-11-08T19:32:25.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.879+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103400bab0 con 0x7f104c0d0d40 2023-11-08T19:32:25.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052b02640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f1034005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1051b00640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 0x7f10340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052301640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1034003970 0x7f1034003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052b02640 1 -- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 msgr2=0x7f10340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052b02640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c0d0d40 0x7f10340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052b02640 1 -- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1034003970 msgr2=0x7f1034003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:25.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052b02640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1034003970 0x7f1034003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052b02640 1 -- 172.21.15.119:0/197721383 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1034003520 con 0x7f1034004d00 2023-11-08T19:32:25.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.880+0000 7f1052301640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1034003970 0x7f1034003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:25.892 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.881+0000 7f1052b02640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f1034005180 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1044007360 tx=0x7f10340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2b5065acaad7b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104402ccd0 con 0x7f1034004d00 2023-11-08T19:32:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104402ce30 con 0x7f1034004d00 2023-11-08T19:32:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f10512ff640 1 -- 172.21.15.119:0/197721383 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044034660 con 0x7f1034004d00 2023-11-08T19:32:25.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1053303640 1 -- 172.21.15.119:0/197721383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 msgr2=0x7f1034005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1053303640 1 --2- 172.21.15.119:0/197721383 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f1034005180 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f1044007360 tx=0x7f10340057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1052b02640 1 -- 172.21.15.119:0/197721383 reap_dead start 2023-11-08T19:32:25.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1053303640 1 -- 172.21.15.119:0/197721383 shutdown_connections 2023-11-08T19:32:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1053303640 1 -- 172.21.15.119:0/197721383 >> 172.21.15.119:0/197721383 conn(0x7f104c0d2f60 msgr2=0x7f104c0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1053303640 1 -- 172.21.15.119:0/197721383 shutdown_connections 2023-11-08T19:32:25.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.882+0000 7f1053303640 1 -- 172.21.15.119:0/197721383 wait complete. 2023-11-08T19:32:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.884+0000 7f1053303640 1 Processor -- start 2023-11-08T19:32:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.884+0000 7f1053303640 1 -- start start 2023-11-08T19:32:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1053303640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f104c14e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.899 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1053303640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c151ab0 0x7f104c151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1053303640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c14f8a0 0x7f104c15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1053303640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c14e980 con 0x7f1034004d00 2023-11-08T19:32:25.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1053303640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f104c1501a0 con 0x7f104c151ab0 2023-11-08T19:32:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1053303640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f104c150480 con 0x7f104c14f8a0 2023-11-08T19:32:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.885+0000 7f1052301640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f104c14e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f104c14e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40136/0 (socket says 172.21.15.119:40136) 2023-11-08T19:32:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 -- 172.21.15.119:0/1708005259 learned_addr learned my addr 172.21.15.119:0/1708005259 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052b02640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c14f8a0 0x7f104c15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1051b00640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c151ab0 0x7f104c151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 -- 172.21.15.119:0/1708005259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c14f8a0 msgr2=0x7f104c15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f104c14f8a0 0x7f104c15d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 -- 172.21.15.119:0/1708005259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c151ab0 msgr2=0x7f104c151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f104c151ab0 0x7f104c151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 -- 172.21.15.119:0/1708005259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c150760 con 0x7f1034004d00 2023-11-08T19:32:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1052301640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f104c14e440 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f103c00d8e0 tx=0x7f103c00dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0074e0 con 0x7f1034004d00 2023-11-08T19:32:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c007640 con 0x7f1034004d00 2023-11-08T19:32:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0049e0 con 0x7f1034004d00 2023-11-08T19:32:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.886+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f104c150a40 con 0x7f1034004d00 2023-11-08T19:32:25.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.887+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104c151780 con 0x7f1034004d00 2023-11-08T19:32:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.888+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f104c15d6c0 con 0x7f1034004d00 2023-11-08T19:32:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.889+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f103c004b40 con 0x7f1034004d00 2023-11-08T19:32:25.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.889+0000 7f104affd640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f103806aea0 0x7f103806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f103c0894b0 con 0x7f1034004d00 2023-11-08T19:32:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f1051b00640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f103806aea0 0x7f103806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f104affd640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f1038073e10 0x7f1038076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 --> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1038076960 con 0x7f1038073e10 2023-11-08T19:32:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f103c08ba90 con 0x7f1034004d00 2023-11-08T19:32:25.908 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f1052b02640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f1038073e10 0x7f1038076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.890+0000 7f1051b00640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f103806aea0 0x7f103806d360 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f104c1606e0 tx=0x7f1040008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.891+0000 7f1052b02640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f1038073e10 0x7f1038076290 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:25.909 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.891+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== osd.3 v2:172.21.15.119:6808/2410755625 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f1038076960 con 0x7f1038073e10 2023-11-08T19:32:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.906+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 --> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f104c0b2c40 con 0x7f1038073e10 2023-11-08T19:32:25.910 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.906+0000 7f104affd640 1 -- 172.21.15.119:0/1708005259 <== osd.3 v2:172.21.15.119:6808/2410755625 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f104402cbf0 con 0x7f1038073e10 2023-11-08T19:32:25.911 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:25.911 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:25.912 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f1038073e10 msgr2=0x7f1038076290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6808/2410755625,v1:172.21.15.119:6810/2410755625] conn(0x7f1038073e10 0x7f1038076290 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.913 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f103806aea0 msgr2=0x7f103806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f103806aea0 0x7f103806d360 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f104c1606e0 tx=0x7f1040008040 comp rx=0 tx=0).stop 2023-11-08T19:32:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 msgr2=0x7f104c14e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:25.914 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 --2- 172.21.15.119:0/1708005259 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f1034004d00 0x7f104c14e440 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f103c00d8e0 tx=0x7f103c00dda0 comp rx=0 tx=0).stop 2023-11-08T19:32:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1052b02640 1 -- 172.21.15.119:0/1708005259 reap_dead start 2023-11-08T19:32:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 shutdown_connections 2023-11-08T19:32:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 >> 172.21.15.119:0/1708005259 conn(0x7f104c0d2f60 msgr2=0x7f104c0d1540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:25.915 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 shutdown_connections 2023-11-08T19:32:25.916 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:25.907+0000 7f1053303640 1 -- 172.21.15.119:0/1708005259 wait complete. 2023-11-08T19:32:25.927 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-08T19:32:26.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.134+0000 7f49b31ac640 1 Processor -- start 2023-11-08T19:32:26.136 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.134+0000 7f49b31ac640 1 -- start start 2023-11-08T19:32:26.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b31ac640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f49ac15cf30 0x7f49ac0d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.137 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b31ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac15d310 0x7f49ac0cf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b31ac640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49ac0b24c0 con 0x7f49ac0cfd50 2023-11-08T19:32:26.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b31ac640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49ac0b2b80 con 0x7f49ac15cf30 2023-11-08T19:32:26.138 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b31ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac0d01c0 con 0x7f49ac15d310 2023-11-08T19:32:26.139 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b29ab640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f49ac0cfd50 0x7f49ac0cd950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57950/0 (socket says 172.21.15.119:57950) 2023-11-08T19:32:26.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b29ab640 1 -- 172.21.15.119:0/381393399 learned_addr learned my addr 172.21.15.119:0/381393399 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:26.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.136+0000 7f49b21aa640 1 --2- 172.21.15.119:0/381393399 >> v2:172.21.15.119:3300/0 conn(0x7f49ac15cf30 0x7f49ac0d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.140 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b19a9640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac15d310 0x7f49ac0cf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b11a8640 1 -- 172.21.15.119:0/381393399 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025802255 0 0) 0x7f49ac0b24c0 con 0x7f49ac0cfd50 2023-11-08T19:32:26.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b11a8640 1 -- 172.21.15.119:0/381393399 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4998003520 con 0x7f49ac0cfd50 2023-11-08T19:32:26.141 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b21aa640 1 -- 172.21.15.119:0/381393399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac15d310 msgr2=0x7f49ac0cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:26.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b21aa640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac15d310 0x7f49ac0cf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b21aa640 1 -- 172.21.15.119:0/381393399 >> v1:172.21.15.119:6789/0 conn(0x7f49ac0cfd50 legacy=0x7f49ac0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b21aa640 1 -- 172.21.15.119:0/381393399 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0d1800 con 0x7f49ac15cf30 2023-11-08T19:32:26.142 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.137+0000 7f49b19a9640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac15d310 0x7f49ac0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:32:26.144 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.139+0000 7f49b21aa640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0d3960 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f49a80097c0 tx=0x7f49a802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7877e0ae2ade038 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.139+0000 7f49b11a8640 1 -- 172.21.15.119:0/381393399 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a80316f0 con 0x7f49ac15cf30 2023-11-08T19:32:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.139+0000 7f49b11a8640 1 -- 172.21.15.119:0/381393399 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8031850 con 0x7f49ac15cf30 2023-11-08T19:32:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.139+0000 7f49b11a8640 1 -- 172.21.15.119:0/381393399 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8031bc0 con 0x7f49ac15cf30 2023-11-08T19:32:26.145 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 -- 172.21.15.119:0/381393399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 msgr2=0x7f49ac0d3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0d3960 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f49a80097c0 tx=0x7f49a802b790 comp rx=0 tx=0).stop 2023-11-08T19:32:26.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 -- 172.21.15.119:0/381393399 shutdown_connections 2023-11-08T19:32:26.146 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac15d310 0x7f49ac0cf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 --2- 172.21.15.119:0/381393399 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0d3960 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 -- 172.21.15.119:0/381393399 >> 172.21.15.119:0/381393399 conn(0x7f49ac0d2e40 msgr2=0x7f49ac0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 -- 172.21.15.119:0/381393399 shutdown_connections 2023-11-08T19:32:26.147 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.140+0000 7f49b31ac640 1 -- 172.21.15.119:0/381393399 wait complete. 2023-11-08T19:32:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.142+0000 7f49b31ac640 1 Processor -- start 2023-11-08T19:32:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.142+0000 7f49b31ac640 1 -- start start 2023-11-08T19:32:26.148 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b31ac640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac0cfd50 0x7f49ac157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b31ac640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b31ac640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac15d310 0x7f49ac0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b31ac640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac15d720 con 0x7f49ac15cf30 2023-11-08T19:32:26.149 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b31ac640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac15d970 con 0x7f49ac15d310 2023-11-08T19:32:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b31ac640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac15dc50 con 0x7f49ac0cfd50 2023-11-08T19:32:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b19a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b19a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0b0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40160/0 (socket says 172.21.15.119:40160) 2023-11-08T19:32:26.150 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.143+0000 7f49b19a9640 1 -- 172.21.15.119:0/3389589581 learned_addr learned my addr 172.21.15.119:0/3389589581 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:26.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b29ab640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac15d310 0x7f49ac0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b21aa640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac0cfd50 0x7f49ac157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b19a9640 1 -- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac0cfd50 msgr2=0x7f49ac157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.153 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b19a9640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac0cfd50 0x7f49ac157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b19a9640 1 -- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac15d310 msgr2=0x7f49ac0b0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b19a9640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac15d310 0x7f49ac0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.154 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b19a9640 1 -- 172.21.15.119:0/3389589581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49a80093a0 con 0x7f49ac15cf30 2023-11-08T19:32:26.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b19a9640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0b0670 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d7d0 tx=0x7f499c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c004030 con 0x7f49ac15cf30 2023-11-08T19:32:26.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f499c004190 con 0x7f49ac15cf30 2023-11-08T19:32:26.155 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f499c004520 con 0x7f49ac15cf30 2023-11-08T19:32:26.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49ac15df30 con 0x7f49ac15cf30 2023-11-08T19:32:26.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.144+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49ac1581d0 con 0x7f49ac15cf30 2023-11-08T19:32:26.156 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.145+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49ac158390 con 0x7f49ac15cf30 2023-11-08T19:32:26.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.147+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f499c0096a0 con 0x7f49ac15cf30 2023-11-08T19:32:26.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.147+0000 7f49a2ffd640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f499006ae70 0x7f499006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.148+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f499c089c30 con 0x7f49ac15cf30 2023-11-08T19:32:26.157 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.148+0000 7f49b21aa640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f499006ae70 0x7f499006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.149+0000 7f49b21aa640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f499006ae70 0x7f499006d330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f49a8036000 tx=0x7f49a8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.158 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.152+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49ac158390 con 0x7f49ac15cf30 2023-11-08T19:32:26.342 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.341+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f49ac0b27a0 con 0x7f499006ae70 2023-11-08T19:32:26.343 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.342+0000 7f49a2ffd640 1 -- 172.21.15.119:0/3389589581 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f49a8004020 con 0x7f499006ae70 2023-11-08T19:32:26.344 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 3 to deep-scrub 2023-11-08T19:32:26.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.345+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f499006ae70 msgr2=0x7f499006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.347 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.345+0000 7f49b31ac640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f499006ae70 0x7f499006d330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f49a8036000 tx=0x7f49a8008030 comp rx=0 tx=0).stop 2023-11-08T19:32:26.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.345+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 msgr2=0x7f49ac0b0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.345+0000 7f49b31ac640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0b0670 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f499c00d7d0 tx=0x7f499c00dc90 comp rx=0 tx=0).stop 2023-11-08T19:32:26.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 shutdown_connections 2023-11-08T19:32:26.348 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f499006ae70 0x7f499006d330 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f49ac15d310 0x7f49ac0b0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f49ac15cf30 0x7f49ac0b0670 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 --2- 172.21.15.119:0/3389589581 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f49ac0cfd50 0x7f49ac157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.349 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 >> 172.21.15.119:0/3389589581 conn(0x7f49ac0d2e40 msgr2=0x7f49ac0b1660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:26.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 shutdown_connections 2023-11-08T19:32:26.350 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.346+0000 7f49b31ac640 1 -- 172.21.15.119:0/3389589581 wait complete. 2023-11-08T19:32:26.366 INFO:tasks.ceph:Scrubbing osd.4 2023-11-08T19:32:26.367 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:26.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.573+0000 7f421544b640 1 Processor -- start 2023-11-08T19:32:26.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.574+0000 7f421544b640 1 -- start start 2023-11-08T19:32:26.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.575+0000 7f421544b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 0x7f4210168210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.575+0000 7f421544b640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100cf1a0 con 0x7f42100ce720 2023-11-08T19:32:26.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.575+0000 7f421544b640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42100d03d0 con 0x7f4210160a00 2023-11-08T19:32:26.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.575+0000 7f421544b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100d0910 con 0x7f42100b42b0 2023-11-08T19:32:26.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.575+0000 7f420e7fc640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f42100ce720 0x7f42100cfcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57956/0 (socket says 172.21.15.119:57956) 2023-11-08T19:32:26.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.575+0000 7f420e7fc640 1 -- 172.21.15.119:0/1156020349 learned_addr learned my addr 172.21.15.119:0/1156020349 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:26.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420f7fe640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 0x7f4210168210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323726622 0 0) 0x7f42100cf1a0 con 0x7f42100ce720 2023-11-08T19:32:26.580 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41fc003520 con 0x7f42100ce720 2023-11-08T19:32:26.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 612076600 0 0) 0x7f41fc003520 con 0x7f42100ce720 2023-11-08T19:32:26.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 msgr2=0x7f4210168210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:26.581 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 0x7f4210168210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 >> v1:172.21.15.188:6789/0 conn(0x7f4210160a00 legacy=0x7f42100cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100cebe0 con 0x7f42100ce720 2023-11-08T19:32:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f41f8001670 con 0x7f42100ce720 2023-11-08T19:32:26.582 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 >> v1:172.21.15.119:6789/0 conn(0x7f42100ce720 legacy=0x7f42100cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc0039a0 0x7f41fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 0x7f41fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.583 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f41fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42100d0910 con 0x7f41fc004d00 2023-11-08T19:32:26.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc00bb10 con 0x7f41fc0039a0 2023-11-08T19:32:26.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.576+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc00bcd0 con 0x7f42100b42b0 2023-11-08T19:32:26.584 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420f7fe640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f41fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420e7fc640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 0x7f41fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420effd640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc0039a0 0x7f41fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420f7fe640 1 -- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 msgr2=0x7f41fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:26.585 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420f7fe640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100b42b0 0x7f41fc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420f7fe640 1 -- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc0039a0 msgr2=0x7f41fc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:26.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420f7fe640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41fc0039a0 0x7f41fc003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.586 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.577+0000 7f420f7fe640 1 -- 172.21.15.119:0/1156020349 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc003520 con 0x7f41fc004d00 2023-11-08T19:32:26.590 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.578+0000 7f420f7fe640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f41fc005180 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4208007330 tx=0x7f41fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfac5a8e1a2bea7e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.578+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420802cca0 con 0x7f41fc004d00 2023-11-08T19:32:26.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.578+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420802ce00 con 0x7f41fc004d00 2023-11-08T19:32:26.591 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.578+0000 7f420dffb640 1 -- 172.21.15.119:0/1156020349 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4208034680 con 0x7f41fc004d00 2023-11-08T19:32:26.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f421544b640 1 -- 172.21.15.119:0/1156020349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 msgr2=0x7f41fc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f421544b640 1 --2- 172.21.15.119:0/1156020349 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f41fc005180 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f4208007330 tx=0x7f41fc0057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f420f7fe640 1 -- 172.21.15.119:0/1156020349 reap_dead start 2023-11-08T19:32:26.592 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f421544b640 1 -- 172.21.15.119:0/1156020349 shutdown_connections 2023-11-08T19:32:26.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f421544b640 1 -- 172.21.15.119:0/1156020349 >> 172.21.15.119:0/1156020349 conn(0x7f42100d3070 msgr2=0x7f42100d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:26.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f421544b640 1 -- 172.21.15.119:0/1156020349 shutdown_connections 2023-11-08T19:32:26.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.579+0000 7f421544b640 1 -- 172.21.15.119:0/1156020349 wait complete. 2023-11-08T19:32:26.593 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.581+0000 7f421544b640 1 Processor -- start 2023-11-08T19:32:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.581+0000 7f421544b640 1 -- start start 2023-11-08T19:32:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f421544b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f4210153510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.594 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f421544b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100cbb70 0x7f42100ca140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f421544b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421006aa90 0x7f421006af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f421544b640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421015c8f0 con 0x7f41fc004d00 2023-11-08T19:32:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f421544b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f421006b450 con 0x7f421006aa90 2023-11-08T19:32:26.595 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f421544b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42100ca680 con 0x7f42100cbb70 2023-11-08T19:32:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f420effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f4210153510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f420effd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f4210153510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40178/0 (socket says 172.21.15.119:40178) 2023-11-08T19:32:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.582+0000 7f420effd640 1 -- 172.21.15.119:0/3777314470 learned_addr learned my addr 172.21.15.119:0/3777314470 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:26.596 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420f7fe640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421006aa90 0x7f421006af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420e7fc640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100cbb70 0x7f42100ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420effd640 1 -- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100cbb70 msgr2=0x7f42100ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.597 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420effd640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f42100cbb70 0x7f42100ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420effd640 1 -- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421006aa90 msgr2=0x7f421006af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420effd640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f421006aa90 0x7f421006af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420effd640 1 -- 172.21.15.119:0/3777314470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42100ca8b0 con 0x7f41fc004d00 2023-11-08T19:32:26.598 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f420effd640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f4210153510 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f420000d5a0 tx=0x7f420000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200003610 con 0x7f41fc004d00 2023-11-08T19:32:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4200003770 con 0x7f41fc004d00 2023-11-08T19:32:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4200003b00 con 0x7f41fc004d00 2023-11-08T19:32:26.599 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.583+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42100cab90 con 0x7f41fc004d00 2023-11-08T19:32:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.584+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42100c7c50 con 0x7f41fc004d00 2023-11-08T19:32:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.584+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f42100b2aa0 con 0x7f41fc004d00 2023-11-08T19:32:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.585+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4200003c60 con 0x7f41fc004d00 2023-11-08T19:32:26.600 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.586+0000 7f42077fe640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f41f006aea0 0x7f41f006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.586+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f4200089940 con 0x7f41fc004d00 2023-11-08T19:32:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.587+0000 7f420e7fc640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f41f006aea0 0x7f41f006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.601 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.587+0000 7f42077fe640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7f41f0073e10 0x7f41f0076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.587+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 --> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f41f0076960 con 0x7f41f0073e10 2023-11-08T19:32:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.587+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f420008e2a0 con 0x7f41fc004d00 2023-11-08T19:32:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.587+0000 7f420e7fc640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f41f006aea0 0x7f41f006d360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000f60 tx=0x7f41f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.602 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.587+0000 7f420f7fe640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7f41f0073e10 0x7f41f0076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.588+0000 7f420f7fe640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7f41f0073e10 0x7f41f0076290 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.603 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.591+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== osd.4 v2:172.21.15.188:6800/4217594962 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f41f0076960 con 0x7f41f0073e10 2023-11-08T19:32:26.606 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.605+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 --> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f42100ceec0 con 0x7f41f0073e10 2023-11-08T19:32:26.607 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.606+0000 7f42077fe640 1 -- 172.21.15.119:0/3777314470 <== osd.4 v2:172.21.15.188:6800/4217594962 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f420802cbc0 con 0x7f41f0073e10 2023-11-08T19:32:26.608 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:26.608 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:26.608 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:26.609 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.606+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7f41f0073e10 msgr2=0x7f41f0076290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.606+0000 7f421544b640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.188:6800/4217594962,v1:172.21.15.188:6801/4217594962] conn(0x7f41f0073e10 0x7f41f0076290 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f41f006aea0 msgr2=0x7f41f006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f41f006aea0 0x7f41f006d360 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f41f8000f60 tx=0x7f41f8009040 comp rx=0 tx=0).stop 2023-11-08T19:32:26.610 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 msgr2=0x7f4210153510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 --2- 172.21.15.119:0/3777314470 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f41fc004d00 0x7f4210153510 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f420000d5a0 tx=0x7f420000da60 comp rx=0 tx=0).stop 2023-11-08T19:32:26.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f420f7fe640 1 -- 172.21.15.119:0/3777314470 reap_dead start 2023-11-08T19:32:26.611 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 shutdown_connections 2023-11-08T19:32:26.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 >> 172.21.15.119:0/3777314470 conn(0x7f42100d3070 msgr2=0x7f42100b22f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:26.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 shutdown_connections 2023-11-08T19:32:26.612 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.607+0000 7f421544b640 1 -- 172.21.15.119:0/3777314470 wait complete. 2023-11-08T19:32:26.626 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-08T19:32:26.836 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.835+0000 7fc9d987c640 1 Processor -- start 2023-11-08T19:32:26.837 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.835+0000 7fc9d987c640 1 -- start start 2023-11-08T19:32:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.836+0000 7fc9d987c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d41447a0 0x7fc9d415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.838 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.836+0000 7fc9d987c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc9d40cad30 0x7fc9d406f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d987c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc9d40cb170 0x7fc9d406a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d987c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d40c3a50 con 0x7fc9d40cb170 2023-11-08T19:32:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d987c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d4144b80 con 0x7fc9d40cad30 2023-11-08T19:32:26.839 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d987c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c9150 con 0x7fc9d41447a0 2023-11-08T19:32:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc9d40cb170 0x7fc9d406a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fc9d40cb170 0x7fc9d406a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40184/0 (socket says 172.21.15.119:40184) 2023-11-08T19:32:26.840 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 -- 172.21.15.119:0/1424143160 learned_addr learned my addr 172.21.15.119:0/1424143160 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:26.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d2ffd640 1 --2- 172.21.15.119:0/1424143160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d41447a0 0x7fc9d415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d27fc640 1 --2- 172.21.15.119:0/1424143160 >> v2:172.21.15.188:3300/0 conn(0x7fc9d40cad30 0x7fc9d406f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 -- 172.21.15.119:0/1424143160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d41447a0 msgr2=0x7fc9d415f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:26.843 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 --2- 172.21.15.119:0/1424143160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d41447a0 0x7fc9d415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 -- 172.21.15.119:0/1424143160 >> v2:172.21.15.188:3300/0 conn(0x7fc9d40cad30 msgr2=0x7fc9d406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:26.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 --2- 172.21.15.119:0/1424143160 >> v2:172.21.15.188:3300/0 conn(0x7fc9d40cad30 0x7fc9d406f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.844 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d37fe640 1 -- 172.21.15.119:0/1424143160 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d40c9410 con 0x7fc9d40cb170 2023-11-08T19:32:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.837+0000 7fc9d27fc640 1 --2- 172.21.15.119:0/1424143160 >> v2:172.21.15.188:3300/0 conn(0x7fc9d40cad30 0x7fc9d406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.839+0000 7fc9d37fe640 1 --2- 172.21.15.119:0/1424143160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d40cb170 0x7fc9d406a120 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc9d406b200 tx=0x7fc9cc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a9bb1625cfb5b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.845 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.839+0000 7fc9d1ffb640 1 -- 172.21.15.119:0/1424143160 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc02ca80 con 0x7fc9d40cb170 2023-11-08T19:32:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.839+0000 7fc9d1ffb640 1 -- 172.21.15.119:0/1424143160 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc02cbe0 con 0x7fc9d40cb170 2023-11-08T19:32:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.839+0000 7fc9d1ffb640 1 -- 172.21.15.119:0/1424143160 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc0343f0 con 0x7fc9d40cb170 2023-11-08T19:32:26.846 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.839+0000 7fc9d987c640 1 -- 172.21.15.119:0/1424143160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d40cb170 msgr2=0x7fc9d406a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.839+0000 7fc9d987c640 1 --2- 172.21.15.119:0/1424143160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d40cb170 0x7fc9d406a120 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc9d406b200 tx=0x7fc9cc02bee0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 -- 172.21.15.119:0/1424143160 shutdown_connections 2023-11-08T19:32:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 --2- 172.21.15.119:0/1424143160 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d40cb170 0x7fc9d406a120 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.847 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 --2- 172.21.15.119:0/1424143160 >> v2:172.21.15.188:3300/0 conn(0x7fc9d40cad30 0x7fc9d406f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 --2- 172.21.15.119:0/1424143160 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d41447a0 0x7fc9d415f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 -- 172.21.15.119:0/1424143160 >> 172.21.15.119:0/1424143160 conn(0x7fc9d40cb610 msgr2=0x7fc9d40cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 -- 172.21.15.119:0/1424143160 shutdown_connections 2023-11-08T19:32:26.848 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.840+0000 7fc9d987c640 1 -- 172.21.15.119:0/1424143160 wait complete. 2023-11-08T19:32:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.841+0000 7fc9d987c640 1 Processor -- start 2023-11-08T19:32:26.849 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.842+0000 7fc9d987c640 1 -- start start 2023-11-08T19:32:26.851 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d987c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d40cad30 0x7fc9d40c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d987c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d40cb170 0x7fc9d41f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d987c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 0x7fc9d41f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d987c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40b3cf0 con 0x7fc9d41447a0 2023-11-08T19:32:26.852 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d987c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40ce060 con 0x7fc9d40cad30 2023-11-08T19:32:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d987c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40cd450 con 0x7fc9d40cb170 2023-11-08T19:32:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 0x7fc9d41f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 0x7fc9d41f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40192/0 (socket says 172.21.15.119:40192) 2023-11-08T19:32:26.853 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 -- 172.21.15.119:0/3715191434 learned_addr learned my addr 172.21.15.119:0/3715191434 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d27fc640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d40cb170 0x7fc9d41f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.854 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d2ffd640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d40cad30 0x7fc9d40c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 -- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d40cb170 msgr2=0x7fc9d41f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d40cb170 0x7fc9d41f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 -- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d40cad30 msgr2=0x7fc9d40c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:26.855 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d40cad30 0x7fc9d40c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d37fe640 1 -- 172.21.15.119:0/3715191434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc009b30 con 0x7fc9d41447a0 2023-11-08T19:32:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d2ffd640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d40cad30 0x7fc9d40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:26.856 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.843+0000 7fc9d27fc640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d40cb170 0x7fc9d41f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.844+0000 7fc9d37fe640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 0x7fc9d41f6280 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc036040 tx=0x7fc9cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.844+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc009510 con 0x7fc9d41447a0 2023-11-08T19:32:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.844+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9cc03e450 con 0x7fc9d41447a0 2023-11-08T19:32:26.857 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.844+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d40cd730 con 0x7fc9d41447a0 2023-11-08T19:32:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.844+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc03e800 con 0x7fc9d41447a0 2023-11-08T19:32:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.844+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d40cdcf0 con 0x7fc9d41447a0 2023-11-08T19:32:26.858 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.845+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d40cde80 con 0x7fc9d41447a0 2023-11-08T19:32:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.846+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9cc02cd50 con 0x7fc9d41447a0 2023-11-08T19:32:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.847+0000 7fc9bbfff640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc9c006ae70 0x7fc9c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.847+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fc9cc0ac7e0 con 0x7fc9d41447a0 2023-11-08T19:32:26.859 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.847+0000 7fc9d2ffd640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc9c006ae70 0x7fc9c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.848+0000 7fc9d2ffd640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc9c006ae70 0x7fc9c006d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b840 tx=0x7fc9c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:26.860 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:26.851+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9cc077390 con 0x7fc9d41447a0 2023-11-08T19:32:27.045 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.044+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9d40c3cf0 con 0x7fc9c006ae70 2023-11-08T19:32:27.046 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.044+0000 7fc9bbfff640 1 -- 172.21.15.119:0/3715191434 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc9c400c040 con 0x7fc9c006ae70 2023-11-08T19:32:27.046 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 4 to deep-scrub 2023-11-08T19:32:27.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc9c006ae70 msgr2=0x7fc9c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc9c006ae70 0x7fc9c006d330 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400b840 tx=0x7fc9c4008040 comp rx=0 tx=0).stop 2023-11-08T19:32:27.049 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 msgr2=0x7fc9d41f6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 0x7fc9d41f6280 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc036040 tx=0x7fc9cc004a40 comp rx=0 tx=0).stop 2023-11-08T19:32:27.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 shutdown_connections 2023-11-08T19:32:27.050 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc9c006ae70 0x7fc9c006d330 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc9d41447a0 0x7fc9d41f6280 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc9d40cb170 0x7fc9d41f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.051 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 --2- 172.21.15.119:0/3715191434 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc9d40cad30 0x7fc9d40c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 >> 172.21.15.119:0/3715191434 conn(0x7fc9d40cb610 msgr2=0x7fc9d4141e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:27.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 shutdown_connections 2023-11-08T19:32:27.052 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.047+0000 7fc9d987c640 1 -- 172.21.15.119:0/3715191434 wait complete. 2023-11-08T19:32:27.068 INFO:tasks.ceph:Scrubbing osd.5 2023-11-08T19:32:27.068 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.285+0000 7f920cc40640 1 Processor -- start 2023-11-08T19:32:27.287 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.286+0000 7f920cc40640 1 -- start start 2023-11-08T19:32:27.288 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f920cc40640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f92080c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f920cc40640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92080b18c0 0x7f92080b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f920cc40640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f920806be20 con 0x7f920806b650 2023-11-08T19:32:27.289 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f920cc40640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f920806c100 con 0x7f92080b18c0 2023-11-08T19:32:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f920cc40640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920806c3e0 con 0x7f920815aea0 2023-11-08T19:32:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f9206ffd640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f920806b650 0x7f920806cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57966/0 (socket says 172.21.15.119:57966) 2023-11-08T19:32:27.290 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.287+0000 7f9206ffd640 1 -- 172.21.15.119:0/2627968168 learned_addr learned my addr 172.21.15.119:0/2627968168 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:27.293 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f9207fff640 1 --2- 172.21.15.119:0/2627968168 >> v2:172.21.15.188:3300/0 conn(0x7f92080b18c0 0x7f92080b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92077fe640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f92080c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 981554305 0 0) 0x7f920806be20 con 0x7f920806b650 2023-11-08T19:32:27.294 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f4003520 con 0x7f920806b650 2023-11-08T19:32:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2602846185 0 0) 0x7f91f4003520 con 0x7f920806b650 2023-11-08T19:32:27.295 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 msgr2=0x7f92080c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f92080c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 >> v2:172.21.15.188:3300/0 conn(0x7f92080b18c0 msgr2=0x7f92080b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:27.296 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 --2- 172.21.15.119:0/2627968168 >> v2:172.21.15.188:3300/0 conn(0x7f92080b18c0 0x7f92080b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920806c6c0 con 0x7f920806b650 2023-11-08T19:32:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f91f80019f0 con 0x7f920806b650 2023-11-08T19:32:27.297 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 >> v1:172.21.15.119:6789/0 conn(0x7f920806b650 legacy=0x7f920806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.288+0000 7f92067fc640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f91f4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92067fc640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f91f40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.298 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92067fc640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91f4004d00 0x7f91f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920806c100 con 0x7f91f4003ef0 2023-11-08T19:32:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920806c3e0 con 0x7f91f4004d00 2023-11-08T19:32:27.299 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f400b980 con 0x7f920815aea0 2023-11-08T19:32:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9206ffd640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f91f40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92077fe640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f91f4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9206ffd640 1 -- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 msgr2=0x7f91f4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:27.300 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9207fff640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91f4004d00 0x7f91f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9206ffd640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f91f4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9206ffd640 1 -- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91f4004d00 msgr2=0x7f91f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:27.301 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9206ffd640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f91f4004d00 0x7f91f4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f9206ffd640 1 -- 172.21.15.119:0/2627968168 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f4003520 con 0x7f91f4003ef0 2023-11-08T19:32:27.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.289+0000 7f92077fe640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f920815aea0 0x7f91f4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:27.302 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.290+0000 7f9206ffd640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f91f40047c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f91f800a8d0 tx=0x7f91f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c6a5e77e552732 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f8034630 con 0x7f91f4003ef0 2023-11-08T19:32:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f802f070 con 0x7f91f4003ef0 2023-11-08T19:32:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f92067fc640 1 -- 172.21.15.119:0/2627968168 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f80349d0 con 0x7f91f4003ef0 2023-11-08T19:32:27.303 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f920cc40640 1 -- 172.21.15.119:0/2627968168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 msgr2=0x7f91f40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f920cc40640 1 --2- 172.21.15.119:0/2627968168 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f91f40047c0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f91f800a8d0 tx=0x7f91f40057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f9207fff640 1 -- 172.21.15.119:0/2627968168 reap_dead start 2023-11-08T19:32:27.304 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f920cc40640 1 -- 172.21.15.119:0/2627968168 shutdown_connections 2023-11-08T19:32:27.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f920cc40640 1 -- 172.21.15.119:0/2627968168 >> 172.21.15.119:0/2627968168 conn(0x7f92080b0560 msgr2=0x7f92080b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:27.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f920cc40640 1 -- 172.21.15.119:0/2627968168 shutdown_connections 2023-11-08T19:32:27.305 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.291+0000 7f920cc40640 1 -- 172.21.15.119:0/2627968168 wait complete. 2023-11-08T19:32:27.308 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.293+0000 7f920cc40640 1 Processor -- start 2023-11-08T19:32:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.294+0000 7f920cc40640 1 -- start start 2023-11-08T19:32:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.294+0000 7f920cc40640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f920815e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.309 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f920cc40640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080c8640 0x7f920814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f920cc40640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f920815c6b0 0x7f92081e4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f920cc40640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920815d010 con 0x7f91f4003ef0 2023-11-08T19:32:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f920cc40640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920815d2f0 con 0x7f920815c6b0 2023-11-08T19:32:27.310 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f920cc40640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920815d5d0 con 0x7f92080c8640 2023-11-08T19:32:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f920815e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f920815e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40206/0 (socket says 172.21.15.119:40206) 2023-11-08T19:32:27.311 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 -- 172.21.15.119:0/2074150042 learned_addr learned my addr 172.21.15.119:0/2074150042 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 -- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080c8640 msgr2=0x7f920814be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f92080c8640 0x7f920814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.312 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 -- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f920815c6b0 msgr2=0x7f92081e4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f920815c6b0 0x7f92081e4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 -- 172.21.15.119:0/2074150042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f920815d8b0 con 0x7f91f4003ef0 2023-11-08T19:32:27.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.295+0000 7f92077fe640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f920815e0b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f91f00108b0 tx=0x7f91f0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.313 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.296+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f000eae0 con 0x7f91f4003ef0 2023-11-08T19:32:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.296+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91f000ec40 con 0x7f91f4003ef0 2023-11-08T19:32:27.314 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.296+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f001c400 con 0x7f91f4003ef0 2023-11-08T19:32:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.296+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f920815db90 con 0x7f91f4003ef0 2023-11-08T19:32:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.296+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f920815cb60 con 0x7f91f4003ef0 2023-11-08T19:32:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.297+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f920806ba90 con 0x7f91f4003ef0 2023-11-08T19:32:27.315 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.298+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f91f001c560 con 0x7f91f4003ef0 2023-11-08T19:32:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.299+0000 7f91fffff640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f91e806aea0 0x7f91e806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.299+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f91f008a110 con 0x7f91f4003ef0 2023-11-08T19:32:27.316 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.299+0000 7f9206ffd640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f91e806aea0 0x7f91e806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.299+0000 7f91fffff640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7f91e8073e10 0x7f91e8076290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.299+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 --> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91e8076960 con 0x7f91e8073e10 2023-11-08T19:32:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.299+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91f008c6f0 con 0x7f91f4003ef0 2023-11-08T19:32:27.317 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.300+0000 7f9207fff640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7f91e8073e10 0x7f91e8076290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.300+0000 7f9206ffd640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f91e806aea0 0x7f91e806d360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f91f8034000 tx=0x7f91f8007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.301+0000 7f9207fff640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7f91e8073e10 0x7f91e8076290 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.318 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.303+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== osd.5 v2:172.21.15.188:6821/704788363 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f91e8076960 con 0x7f91e8073e10 2023-11-08T19:32:27.319 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.317+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 --> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f920806c9a0 con 0x7f91e8073e10 2023-11-08T19:32:27.320 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.318+0000 7f91fffff640 1 -- 172.21.15.119:0/2074150042 <== osd.5 v2:172.21.15.188:6821/704788363 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9200005990 con 0x7f91e8073e10 2023-11-08T19:32:27.320 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:27.320 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:27.321 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:27.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7f91e8073e10 msgr2=0x7f91e8076290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.322 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.188:6821/704788363,v1:172.21.15.188:6823/704788363] conn(0x7f91e8073e10 0x7f91e8076290 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f91e806aea0 msgr2=0x7f91e806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f91e806aea0 0x7f91e806d360 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f91f8034000 tx=0x7f91f8007000 comp rx=0 tx=0).stop 2023-11-08T19:32:27.323 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 msgr2=0x7f920815e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 --2- 172.21.15.119:0/2074150042 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f91f4003ef0 0x7f920815e0b0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f91f00108b0 tx=0x7f91f0010d70 comp rx=0 tx=0).stop 2023-11-08T19:32:27.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f9207fff640 1 -- 172.21.15.119:0/2074150042 reap_dead start 2023-11-08T19:32:27.324 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 shutdown_connections 2023-11-08T19:32:27.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 >> 172.21.15.119:0/2074150042 conn(0x7f92080b0560 msgr2=0x7f92080ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:27.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 shutdown_connections 2023-11-08T19:32:27.325 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.319+0000 7f920cc40640 1 -- 172.21.15.119:0/2074150042 wait complete. 2023-11-08T19:32:27.338 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-08T19:32:27.553 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.552+0000 7f6fffd68640 1 Processor -- start 2023-11-08T19:32:27.554 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.552+0000 7f6fffd68640 1 -- start start 2023-11-08T19:32:27.555 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.553+0000 7f6fffd68640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ff8142f80 0x7f6ff80cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.553+0000 7f6fffd68640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff806cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.553+0000 7f6fffd68640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ff80c4960 con 0x7f6ff815d9f0 2023-11-08T19:32:27.556 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.553+0000 7f6fffd68640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ff8143a40 con 0x7f6ff8142f80 2023-11-08T19:32:27.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.553+0000 7f6fffd68640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff80ca3e0 con 0x7f6ff806cb40 2023-11-08T19:32:27.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ff8142f80 0x7f6ff80cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.557 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f6ff8142f80 0x7f6ff80cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40210/0 (socket says 172.21.15.119:40210) 2023-11-08T19:32:27.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 -- 172.21.15.119:0/352266781 learned_addr learned my addr 172.21.15.119:0/352266781 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:27.558 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffd303640 1 --1- 172.21.15.119:0/352266781 >> v1:172.21.15.119:6789/0 conn(0x7f6ff815d9f0 0x7f6ff80c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57982/0 (socket says 172.21.15.119:57982) 2023-11-08T19:32:27.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffe305640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff806cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 -- 172.21.15.119:0/352266781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 msgr2=0x7f6ff806cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:27.559 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff806cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 -- 172.21.15.119:0/352266781 >> v1:172.21.15.119:6789/0 conn(0x7f6ff815d9f0 legacy=0x7f6ff80c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.560 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffdb04640 1 -- 172.21.15.119:0/352266781 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff815df60 con 0x7f6ff8142f80 2023-11-08T19:32:27.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffcb02640 1 -- 172.21.15.119:0/352266781 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116453568 0 0) 0x7f6ff80c4960 con 0x7f6ff815d9f0 2023-11-08T19:32:27.561 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.554+0000 7f6ffe305640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:27.565 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.555+0000 7f6ffdb04640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff8142f80 0x7f6ff80cc6d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400bb10 tx=0x7f6ff402e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb79fd733df87d72 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6ffcb02640 1 -- 172.21.15.119:0/352266781 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4036440 con 0x7f6ff8142f80 2023-11-08T19:32:27.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6ffcb02640 1 -- 172.21.15.119:0/352266781 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff40365a0 con 0x7f6ff8142f80 2023-11-08T19:32:27.566 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6ffcb02640 1 -- 172.21.15.119:0/352266781 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff4036930 con 0x7f6ff8142f80 2023-11-08T19:32:27.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 -- 172.21.15.119:0/352266781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff8142f80 msgr2=0x7f6ff80cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff8142f80 0x7f6ff80cc6d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6ff400bb10 tx=0x7f6ff402e150 comp rx=0 tx=0).stop 2023-11-08T19:32:27.567 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 -- 172.21.15.119:0/352266781 shutdown_connections 2023-11-08T19:32:27.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff806cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 --2- 172.21.15.119:0/352266781 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff8142f80 0x7f6ff80cc6d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.568 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 -- 172.21.15.119:0/352266781 >> 172.21.15.119:0/352266781 conn(0x7f6ff80cb610 msgr2=0x7f6ff80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:27.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.556+0000 7f6fffd68640 1 -- 172.21.15.119:0/352266781 shutdown_connections 2023-11-08T19:32:27.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.557+0000 7f6fffd68640 1 -- 172.21.15.119:0/352266781 wait complete. 2023-11-08T19:32:27.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.558+0000 7f6fffd68640 1 Processor -- start 2023-11-08T19:32:27.569 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.558+0000 7f6fffd68640 1 -- start start 2023-11-08T19:32:27.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.559+0000 7f6fffd68640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff80aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.559+0000 7f6fffd68640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ff8142f80 0x7f6ff80b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.570 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.559+0000 7f6fffd68640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 0x7f6ff80b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.559+0000 7f6fffd68640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff81f6c90 con 0x7f6ff815d9f0 2023-11-08T19:32:27.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.559+0000 7f6fffd68640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff81f2c30 con 0x7f6ff8142f80 2023-11-08T19:32:27.571 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.559+0000 7f6fffd68640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff81f2f10 con 0x7f6ff806cb40 2023-11-08T19:32:27.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 0x7f6ff80b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 0x7f6ff80b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40220/0 (socket says 172.21.15.119:40220) 2023-11-08T19:32:27.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 -- 172.21.15.119:0/2195759678 learned_addr learned my addr 172.21.15.119:0/2195759678 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:27.572 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffdb04640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff80aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffd303640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ff8142f80 0x7f6ff80b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 -- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 msgr2=0x7f6ff80aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.573 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff80aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 -- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ff8142f80 msgr2=0x7f6ff80b07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ff8142f80 0x7f6ff80b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 -- 172.21.15.119:0/2195759678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff400b770 con 0x7f6ff815d9f0 2023-11-08T19:32:27.574 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffdb04640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff80aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:27.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.560+0000 7f6ffe305640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 0x7f6ff80b0d00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000cd50 tx=0x7f6ff000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.561+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000f680 con 0x7f6ff815d9f0 2023-11-08T19:32:27.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.561+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff000f800 con 0x7f6ff815d9f0 2023-11-08T19:32:27.575 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.561+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000fbb0 con 0x7f6ff815d9f0 2023-11-08T19:32:27.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.561+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff81f31f0 con 0x7f6ff815d9f0 2023-11-08T19:32:27.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.561+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff80b04d0 con 0x7f6ff815d9f0 2023-11-08T19:32:27.576 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.562+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff81f33e0 con 0x7f6ff815d9f0 2023-11-08T19:32:27.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.563+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ff000b040 con 0x7f6ff815d9f0 2023-11-08T19:32:27.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.564+0000 7f6fee7fc640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fdc06ae70 0x7f6fdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.564+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f6ff008a430 con 0x7f6ff815d9f0 2023-11-08T19:32:27.577 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.564+0000 7f6ffdb04640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fdc06ae70 0x7f6fdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:27.578 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.565+0000 7f6ffdb04640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fdc06ae70 0x7f6fdc06d330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4038000 tx=0x7f6ff4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:27.579 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.568+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ff0055060 con 0x7f6ff815d9f0 2023-11-08T19:32:27.761 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.760+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ff81f2da0 con 0x7f6fdc06ae70 2023-11-08T19:32:27.762 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.761+0000 7f6fee7fc640 1 -- 172.21.15.119:0/2195759678 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6ff81f2da0 con 0x7f6fdc06ae70 2023-11-08T19:32:27.763 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 5 to deep-scrub 2023-11-08T19:32:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fdc06ae70 msgr2=0x7f6fdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fdc06ae70 0x7f6fdc06d330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4038000 tx=0x7f6ff4008040 comp rx=0 tx=0).stop 2023-11-08T19:32:27.765 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 msgr2=0x7f6ff80b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:27.766 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 0x7f6ff80b0d00 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000cd50 tx=0x7f6ff000a650 comp rx=0 tx=0).stop 2023-11-08T19:32:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 shutdown_connections 2023-11-08T19:32:27.767 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f6fdc06ae70 0x7f6fdc06d330 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f6ff815d9f0 0x7f6ff80b0d00 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6ff8142f80 0x7f6ff80b07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.768 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 --2- 172.21.15.119:0/2195759678 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6ff806cb40 0x7f6ff80aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 >> 172.21.15.119:0/2195759678 conn(0x7f6ff80cb610 msgr2=0x7f6ff80cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 shutdown_connections 2023-11-08T19:32:27.769 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.764+0000 7f6fffd68640 1 -- 172.21.15.119:0/2195759678 wait complete. 2023-11-08T19:32:27.785 INFO:tasks.ceph:Scrubbing osd.6 2023-11-08T19:32:27.785 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:27.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.992+0000 7fc38cc2e640 1 Processor -- start 2023-11-08T19:32:27.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.992+0000 7fc38cc2e640 1 -- start start 2023-11-08T19:32:27.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc38cc2e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3880c9c90 0x7fc38806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:27.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc38cc2e640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38806be20 con 0x7fc38806d400 2023-11-08T19:32:27.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc38cc2e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38806c100 con 0x7fc38815aea0 2023-11-08T19:32:27.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc38cc2e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38806c3e0 con 0x7fc3880c9c90 2023-11-08T19:32:27.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc387fff640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fc38806d400 0x7fc3880b18c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57994/0 (socket says 172.21.15.119:57994) 2023-11-08T19:32:27.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc387fff640 1 -- 172.21.15.119:0/2240906026 learned_addr learned my addr 172.21.15.119:0/2240906026 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:27.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.994+0000 7fc386ffd640 1 --2- 172.21.15.119:0/2240906026 >> v2:172.21.15.188:3300/0 conn(0x7fc3880c9c90 0x7fc38806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463737082 0 0) 0x7fc38806be20 con 0x7fc38806d400 2023-11-08T19:32:28.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc370003520 con 0x7fc38806d400 2023-11-08T19:32:28.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764423231 0 0) 0x7fc370003520 con 0x7fc38806d400 2023-11-08T19:32:28.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 >> v2:172.21.15.188:3300/0 conn(0x7fc3880c9c90 msgr2=0x7fc38806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 --2- 172.21.15.119:0/2240906026 >> v2:172.21.15.188:3300/0 conn(0x7fc3880c9c90 0x7fc38806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 >> v1:172.21.15.188:6789/0 conn(0x7fc38815aea0 legacy=0x7fc38806b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38806c6c0 con 0x7fc38806d400 2023-11-08T19:32:28.002 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fc3800019f0 con 0x7fc38806d400 2023-11-08T19:32:28.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 >> v1:172.21.15.119:6789/0 conn(0x7fc38806d400 legacy=0x7fc3880b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370003970 0x7fc370003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3700047c0 0x7fc370004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc370005180 0x7fc37000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38806c3e0 con 0x7fc370003970 2023-11-08T19:32:28.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000c020 con 0x7fc3700047c0 2023-11-08T19:32:28.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc37000c1e0 con 0x7fc370005180 2023-11-08T19:32:28.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.995+0000 7fc3877fe640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370003970 0x7fc370003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc3877fe640 1 -- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc370005180 msgr2=0x7fc37000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc3877fe640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc370005180 0x7fc37000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.005 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc3877fe640 1 -- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3700047c0 msgr2=0x7fc370004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc386ffd640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3700047c0 0x7fc370004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc3877fe640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3700047c0 0x7fc370004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.006 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc387fff640 1 -- 172.21.15.119:0/2240906026 reap_dead start 2023-11-08T19:32:28.010 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.996+0000 7fc3877fe640 1 -- 172.21.15.119:0/2240906026 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc370003520 con 0x7fc370003970 2023-11-08T19:32:28.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.997+0000 7fc3877fe640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370003970 0x7fc370003d90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc378001750 tx=0x7fc3700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b15469557469dcd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.016 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.997+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3780052d0 con 0x7fc370003970 2023-11-08T19:32:28.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.997+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37802e050 con 0x7fc370003970 2023-11-08T19:32:28.017 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.997+0000 7fc3867fc640 1 -- 172.21.15.119:0/2240906026 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc378003830 con 0x7fc370003970 2023-11-08T19:32:28.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 -- 172.21.15.119:0/2240906026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370003970 msgr2=0x7fc370003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370003970 0x7fc370003d90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fc378001750 tx=0x7fc3700057f0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.018 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 -- 172.21.15.119:0/2240906026 shutdown_connections 2023-11-08T19:32:28.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 --2- 172.21.15.119:0/2240906026 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc370003970 0x7fc370003d90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 -- 172.21.15.119:0/2240906026 >> 172.21.15.119:0/2240906026 conn(0x7fc3880b0560 msgr2=0x7fc3880b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.019 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 -- 172.21.15.119:0/2240906026 shutdown_connections 2023-11-08T19:32:28.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:27.998+0000 7fc38cc2e640 1 -- 172.21.15.119:0/2240906026 wait complete. 2023-11-08T19:32:28.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.000+0000 7fc38cc2e640 1 Processor -- start 2023-11-08T19:32:28.020 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.000+0000 7fc38cc2e640 1 -- start start 2023-11-08T19:32:28.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc38cc2e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370003970 0x7fc388146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.021 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc38cc2e640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc38815e160 0x7fc3881e4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc38cc2e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3881e4840 0x7fc3880c5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.022 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc38cc2e640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc388146650 con 0x7fc38815e160 2023-11-08T19:32:28.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc38cc2e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815d0c0 con 0x7fc370003970 2023-11-08T19:32:28.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc38cc2e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38815d3a0 con 0x7fc3881e4840 2023-11-08T19:32:28.023 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc38815e160 0x7fc3881e4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc38815e160 0x7fc3881e4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40236/0 (socket says 172.21.15.119:40236) 2023-11-08T19:32:28.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 -- 172.21.15.119:0/1977430826 learned_addr learned my addr 172.21.15.119:0/1977430826 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:28.024 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 -- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3881e4840 msgr2=0x7fc3880c5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc387fff640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3881e4840 0x7fc3880c5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3881e4840 0x7fc3880c5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.025 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc3877fe640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370003970 0x7fc388146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 -- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370003970 msgr2=0x7fc388146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc370003970 0x7fc388146110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.026 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.001+0000 7fc386ffd640 1 -- 172.21.15.119:0/1977430826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38815d680 con 0x7fc38815e160 2023-11-08T19:32:28.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.002+0000 7fc386ffd640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc38815e160 0x7fc3881e4300 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc37c010790 tx=0x7fc37c010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.027 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.002+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c009630 con 0x7fc38815e160 2023-11-08T19:32:28.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.002+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c00eb60 con 0x7fc38815e160 2023-11-08T19:32:28.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.002+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c010ea0 con 0x7fc38815e160 2023-11-08T19:32:28.028 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.002+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38815d960 con 0x7fc38815e160 2023-11-08T19:32:28.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.002+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38815cce0 con 0x7fc38815e160 2023-11-08T19:32:28.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.003+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc38806c9a0 con 0x7fc38815e160 2023-11-08T19:32:28.029 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.004+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc37c01c400 con 0x7fc38815e160 2023-11-08T19:32:28.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.005+0000 7fc36ffff640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc37406ae70 0x7fc37406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.030 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.005+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fc37c08a1e0 con 0x7fc38815e160 2023-11-08T19:32:28.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.006+0000 7fc3877fe640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc37406ae70 0x7fc37406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.006+0000 7fc36ffff640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7fc374073de0 0x7fc374076260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.031 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.006+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 --> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc374076930 con 0x7fc374073de0 2023-11-08T19:32:28.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.006+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc37c08a550 con 0x7fc38815e160 2023-11-08T19:32:28.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.006+0000 7fc3877fe640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc37406ae70 0x7fc37406d330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc378033000 tx=0x7fc378009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.006+0000 7fc387fff640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7fc374073de0 0x7fc374076260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.032 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.008+0000 7fc387fff640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7fc374073de0 0x7fc374076260 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.010+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== osd.6 v2:172.21.15.188:6808/4161471119 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fc374076930 con 0x7fc374073de0 2023-11-08T19:32:28.033 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.025+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 --> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc38804ece0 con 0x7fc374073de0 2023-11-08T19:32:28.034 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.026+0000 7fc36ffff640 1 -- 172.21.15.119:0/1977430826 <== osd.6 v2:172.21.15.188:6808/4161471119 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc38804ece0 con 0x7fc374073de0 2023-11-08T19:32:28.034 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:28.035 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:28.035 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:28.036 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.026+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7fc374073de0 msgr2=0x7fc374076260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.026+0000 7fc38cc2e640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.188:6808/4161471119,v1:172.21.15.188:6809/4161471119] conn(0x7fc374073de0 0x7fc374076260 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc37406ae70 msgr2=0x7fc37406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fc37406ae70 0x7fc37406d330 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fc378033000 tx=0x7fc378009030 comp rx=0 tx=0).stop 2023-11-08T19:32:28.037 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc38815e160 msgr2=0x7fc3881e4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 --2- 172.21.15.119:0/1977430826 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fc38815e160 0x7fc3881e4300 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fc37c010790 tx=0x7fc37c010c50 comp rx=0 tx=0).stop 2023-11-08T19:32:28.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc387fff640 1 -- 172.21.15.119:0/1977430826 reap_dead start 2023-11-08T19:32:28.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 shutdown_connections 2023-11-08T19:32:28.038 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 >> 172.21.15.119:0/1977430826 conn(0x7fc3880b0560 msgr2=0x7fc3881e3760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 shutdown_connections 2023-11-08T19:32:28.039 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.027+0000 7fc38cc2e640 1 -- 172.21.15.119:0/1977430826 wait complete. 2023-11-08T19:32:28.045 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-08T19:32:28.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.259+0000 7f740c797640 1 Processor -- start 2023-11-08T19:32:28.261 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.260+0000 7f740c797640 1 -- start start 2023-11-08T19:32:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.261+0000 7f740c797640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f740406a120 0x7f740406a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.261+0000 7f740c797640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740406aa70 con 0x7f740415f4c0 2023-11-08T19:32:28.263 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.261+0000 7f740c797640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74040c3a50 con 0x7f74041447a0 2023-11-08T19:32:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.261+0000 7f740c797640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740415f930 con 0x7f740406a120 2023-11-08T19:32:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.261+0000 7f740ad34640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f740406a120 0x7f740406a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.264 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.262+0000 7f7409d32640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f740415f4c0 0x7f740406f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:58008/0 (socket says 172.21.15.119:58008) 2023-11-08T19:32:28.265 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.262+0000 7f7409d32640 1 -- 172.21.15.119:0/867777007 learned_addr learned my addr 172.21.15.119:0/867777007 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:32:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.262+0000 7f740ad34640 1 -- 172.21.15.119:0/867777007 >> v1:172.21.15.188:6789/0 conn(0x7f74041447a0 legacy=0x7f74040cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.266 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.262+0000 7f740ad34640 1 -- 172.21.15.119:0/867777007 >> v1:172.21.15.119:6789/0 conn(0x7f740415f4c0 legacy=0x7f740406f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.262+0000 7f740ad34640 1 -- 172.21.15.119:0/867777007 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7404144b80 con 0x7f740406a120 2023-11-08T19:32:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.263+0000 7f740ad34640 1 --2- 172.21.15.119:0/867777007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740406a120 0x7f740406a530 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f740415fa70 tx=0x7f73fc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74543840b925944 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.267 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.263+0000 7f7409531640 1 -- 172.21.15.119:0/867777007 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc02cc60 con 0x7f740406a120 2023-11-08T19:32:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.263+0000 7f7409531640 1 -- 172.21.15.119:0/867777007 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc02cdc0 con 0x7f740406a120 2023-11-08T19:32:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.263+0000 7f7409531640 1 -- 172.21.15.119:0/867777007 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc034660 con 0x7f740406a120 2023-11-08T19:32:28.268 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 -- 172.21.15.119:0/867777007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740406a120 msgr2=0x7f740406a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 --2- 172.21.15.119:0/867777007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740406a120 0x7f740406a530 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f740415fa70 tx=0x7f73fc02beb0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 -- 172.21.15.119:0/867777007 shutdown_connections 2023-11-08T19:32:28.269 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 --2- 172.21.15.119:0/867777007 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740406a120 0x7f740406a530 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 -- 172.21.15.119:0/867777007 >> 172.21.15.119:0/867777007 conn(0x7f74040cb610 msgr2=0x7f74040cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 -- 172.21.15.119:0/867777007 shutdown_connections 2023-11-08T19:32:28.270 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.264+0000 7f740c797640 1 -- 172.21.15.119:0/867777007 wait complete. 2023-11-08T19:32:28.273 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.266+0000 7f740c797640 1 Processor -- start 2023-11-08T19:32:28.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.266+0000 7f740c797640 1 -- start start 2023-11-08T19:32:28.274 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740c797640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f740406a120 0x7f740413d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740c797640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74041447a0 0x7f7404140860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740c797640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 0x7f7404140da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.275 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740c797640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040c65a0 con 0x7f740415f4c0 2023-11-08T19:32:28.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740c797640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740413dc70 con 0x7f74041447a0 2023-11-08T19:32:28.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740c797640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74040ce460 con 0x7f740406a120 2023-11-08T19:32:28.276 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740ad34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 0x7f7404140da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.267+0000 7f740ad34640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 0x7f7404140da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:40240/0 (socket says 172.21.15.119:40240) 2023-11-08T19:32:28.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 -- 172.21.15.119:0/97397933 learned_addr learned my addr 172.21.15.119:0/97397933 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:28.277 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740a533640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f740406a120 0x7f740413d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f7409d32640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74041447a0 0x7f7404140860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 -- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f740406a120 msgr2=0x7f740413d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.278 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f740406a120 0x7f740413d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 -- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74041447a0 msgr2=0x7f7404140860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74041447a0 0x7f7404140860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.279 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 -- 172.21.15.119:0/97397933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73fc009a70 con 0x7f740415f4c0 2023-11-08T19:32:28.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.268+0000 7f740ad34640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 0x7f7404140da0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009e80 tx=0x7f73fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.269+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc040650 con 0x7f740415f4c0 2023-11-08T19:32:28.280 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.269+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73fc0407b0 con 0x7f740415f4c0 2023-11-08T19:32:28.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.269+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73fc040b20 con 0x7f740415f4c0 2023-11-08T19:32:28.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.269+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74040c5c70 con 0x7f740415f4c0 2023-11-08T19:32:28.281 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.269+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74040c6230 con 0x7f740415f4c0 2023-11-08T19:32:28.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.270+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74040c6750 con 0x7f740415f4c0 2023-11-08T19:32:28.282 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.271+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73fc040c80 con 0x7f740415f4c0 2023-11-08T19:32:28.283 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.272+0000 7f73faffd640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73e806ae70 0x7f73e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.272+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f73fc0acb30 con 0x7f740415f4c0 2023-11-08T19:32:28.284 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.272+0000 7f740a533640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73e806ae70 0x7f73e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.273+0000 7f740a533640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73e806ae70 0x7f73e806d330 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7400004ea0 tx=0x7f7400009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.285 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.276+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74040c6750 con 0x7f740415f4c0 2023-11-08T19:32:28.466 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.465+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f74040c9150 con 0x7f73e806ae70 2023-11-08T19:32:28.467 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.466+0000 7f73faffd640 1 -- 172.21.15.119:0/97397933 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7400005420 con 0x7f73e806ae70 2023-11-08T19:32:28.468 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 6 to deep-scrub 2023-11-08T19:32:28.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73e806ae70 msgr2=0x7f73e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73e806ae70 0x7f73e806d330 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7400004ea0 tx=0x7f7400009040 comp rx=0 tx=0).stop 2023-11-08T19:32:28.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 msgr2=0x7f7404140da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 0x7f7404140da0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f73fc009e80 tx=0x7f73fc004a40 comp rx=0 tx=0).stop 2023-11-08T19:32:28.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 shutdown_connections 2023-11-08T19:32:28.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f73e806ae70 0x7f73e806d330 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f740415f4c0 0x7f7404140da0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f74041447a0 0x7f7404140860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 --2- 172.21.15.119:0/97397933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f740406a120 0x7f740413d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.468+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 >> 172.21.15.119:0/97397933 conn(0x7f74040cb610 msgr2=0x7f74040cbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.469+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 shutdown_connections 2023-11-08T19:32:28.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.469+0000 7f740c797640 1 -- 172.21.15.119:0/97397933 wait complete. 2023-11-08T19:32:28.489 INFO:tasks.ceph:Scrubbing osd.7 2023-11-08T19:32:28.490 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-08T19:32:28.710 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.709+0000 7f1561d8c640 1 Processor -- start 2023-11-08T19:32:28.711 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.709+0000 7f1561d8c640 1 -- start start 2023-11-08T19:32:28.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.710+0000 7f1561d8c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f155c15aea0 0x7f155c0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.712 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.710+0000 7f1561d8c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f155c06b650 0x7f155c06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1561d8c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f155c06be20 con 0x7f155c06d400 2023-11-08T19:32:28.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1561d8c640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c06c100 con 0x7f155c15aea0 2023-11-08T19:32:28.713 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1561d8c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155c06c3e0 con 0x7f155c06b650 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f155c15aea0 0x7f155c0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f155c15aea0 0x7f155c0c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58830/0 (socket says 172.21.15.119:58830) 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 -- 172.21.15.119:0/3171699077 learned_addr learned my addr 172.21.15.119:0/3171699077 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:28.714 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 -- 172.21.15.119:0/3171699077 >> v2:172.21.15.188:3300/0 conn(0x7f155c06b650 msgr2=0x7f155c06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 --2- 172.21.15.119:0/3171699077 >> v2:172.21.15.188:3300/0 conn(0x7f155c06b650 0x7f155c06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 -- 172.21.15.119:0/3171699077 >> v1:172.21.15.188:6789/0 conn(0x7f155c06d400 legacy=0x7f155c0b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.715 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.711+0000 7f1560d8a640 1 -- 172.21.15.119:0/3171699077 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c06c6c0 con 0x7f155c15aea0 2023-11-08T19:32:28.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.712+0000 7f1560d8a640 1 --2- 172.21.15.119:0/3171699077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c15aea0 0x7f155c0c9c90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f155c06f2e0 tx=0x7f155002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df5843f43a4d37c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.717 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f155b7fe640 1 -- 172.21.15.119:0/3171699077 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550031870 con 0x7f155c15aea0 2023-11-08T19:32:28.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f155b7fe640 1 -- 172.21.15.119:0/3171699077 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15500319d0 con 0x7f155c15aea0 2023-11-08T19:32:28.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f155b7fe640 1 -- 172.21.15.119:0/3171699077 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1550031d60 con 0x7f155c15aea0 2023-11-08T19:32:28.718 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 -- 172.21.15.119:0/3171699077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c15aea0 msgr2=0x7f155c0c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 --2- 172.21.15.119:0/3171699077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c15aea0 0x7f155c0c9c90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f155c06f2e0 tx=0x7f155002b910 comp rx=0 tx=0).stop 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 -- 172.21.15.119:0/3171699077 shutdown_connections 2023-11-08T19:32:28.719 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 --2- 172.21.15.119:0/3171699077 >> v2:172.21.15.188:3300/0 conn(0x7f155c06b650 0x7f155c06cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 --2- 172.21.15.119:0/3171699077 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c15aea0 0x7f155c0c9c90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 -- 172.21.15.119:0/3171699077 >> 172.21.15.119:0/3171699077 conn(0x7f155c0b0560 msgr2=0x7f155c0b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.720 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 -- 172.21.15.119:0/3171699077 shutdown_connections 2023-11-08T19:32:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.713+0000 7f1561d8c640 1 -- 172.21.15.119:0/3171699077 wait complete. 2023-11-08T19:32:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.715+0000 7f1561d8c640 1 Processor -- start 2023-11-08T19:32:28.721 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.715+0000 7f1561d8c640 1 -- start start 2023-11-08T19:32:28.724 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f1561d8c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c06b650 0x7f155c151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f1561d8c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c06d400 0x7f155c15d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f1561d8c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c15aea0 0x7f155c071120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f1561d8c640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c1523f0 con 0x7f155c06d400 2023-11-08T19:32:28.725 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f1561d8c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0c60b0 con 0x7f155c06b650 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f1561d8c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155c0c6390 con 0x7f155c15aea0 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.716+0000 7f155bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c06d400 0x7f155c15d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.726 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c06d400 0x7f155c15d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58836/0 (socket says 172.21.15.119:58836) 2023-11-08T19:32:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 -- 172.21.15.119:0/1011601934 learned_addr learned my addr 172.21.15.119:0/1011601934 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f1560d8a640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c06b650 0x7f155c151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.727 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 -- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c15aea0 msgr2=0x7f155c071120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f156158b640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c15aea0 0x7f155c071120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c15aea0 0x7f155c071120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 -- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c06b650 msgr2=0x7f155c151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.728 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c06b650 0x7f155c151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 -- 172.21.15.119:0/1011601934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15500094d0 con 0x7f155c06d400 2023-11-08T19:32:28.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f1560d8a640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f155c06b650 0x7f155c151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:28.729 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f156158b640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f155c15aea0 0x7f155c071120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f155bfff640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c06d400 0x7f155c15d2f0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f154800b4a0 tx=0x7f154800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.717+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548010610 con 0x7f155c06d400 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.718+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1548010770 con 0x7f155c06d400 2023-11-08T19:32:28.730 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.718+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548010b20 con 0x7f155c06d400 2023-11-08T19:32:28.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.718+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c0c6670 con 0x7f155c06d400 2023-11-08T19:32:28.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.718+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c1f2a40 con 0x7f155c06d400 2023-11-08T19:32:28.731 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.719+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f155c06bac0 con 0x7f155c06d400 2023-11-08T19:32:28.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.720+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1548010c80 con 0x7f155c06d400 2023-11-08T19:32:28.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.721+0000 7f15597fa640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f154006ae70 0x7f154006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.721+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f1548089720 con 0x7f155c06d400 2023-11-08T19:32:28.732 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.721+0000 7f1560d8a640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f154006ae70 0x7f154006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.721+0000 7f15597fa640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7f1540073de0 0x7f1540076260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.721+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 --> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1540076930 con 0x7f1540073de0 2023-11-08T19:32:28.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.721+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_get_version_reply(handle=1 version=109) v2 ==== 24+0+0 (secure 0 0 0) 0x7f154808bde0 con 0x7f155c06d400 2023-11-08T19:32:28.733 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.722+0000 7f1560d8a640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f154006ae70 0x7f154006d330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1550036000 tx=0x7f155003c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.722+0000 7f156158b640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7f1540073de0 0x7f1540076260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.734 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.723+0000 7f156158b640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7f1540073de0 0x7f1540076260 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.735 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.725+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== osd.7 v2:172.21.15.188:6816/2104156129 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f1540076930 con 0x7f1540073de0 2023-11-08T19:32:28.740 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.739+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 --> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f155c06c9a0 con 0x7f1540073de0 2023-11-08T19:32:28.742 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.740+0000 7f15597fa640 1 -- 172.21.15.119:0/1011601934 <== osd.7 v2:172.21.15.188:6816/2104156129 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f15540019d0 con 0x7f1540073de0 2023-11-08T19:32:28.742 INFO:teuthology.orchestra.run.smithi119.stdout:{ 2023-11-08T19:32:28.743 INFO:teuthology.orchestra.run.smithi119.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T19:32:28.743 INFO:teuthology.orchestra.run.smithi119.stdout:} 2023-11-08T19:32:28.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7f1540073de0 msgr2=0x7f1540076260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.744 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.188:6816/2104156129,v1:172.21.15.188:6817/2104156129] conn(0x7f1540073de0 0x7f1540076260 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f154006ae70 msgr2=0x7f154006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f154006ae70 0x7f154006d330 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1550036000 tx=0x7f155003c040 comp rx=0 tx=0).stop 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c06d400 msgr2=0x7f155c15d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.745 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 --2- 172.21.15.119:0/1011601934 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f155c06d400 0x7f155c15d2f0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f154800b4a0 tx=0x7f154800b960 comp rx=0 tx=0).stop 2023-11-08T19:32:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f156158b640 1 -- 172.21.15.119:0/1011601934 reap_dead start 2023-11-08T19:32:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 shutdown_connections 2023-11-08T19:32:28.746 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 >> 172.21.15.119:0/1011601934 conn(0x7f155c0b0560 msgr2=0x7f155c0ce060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 shutdown_connections 2023-11-08T19:32:28.747 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.741+0000 7f1561d8c640 1 -- 172.21.15.119:0/1011601934 wait complete. 2023-11-08T19:32:28.759 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-08T19:32:28.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.979+0000 7f7c5efb8640 1 Processor -- start 2023-11-08T19:32:28.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.980+0000 7f7c5efb8640 1 -- start start 2023-11-08T19:32:28.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5efb8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5815aea0 0x7f7c580c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5efb8640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c580b18c0 0x7f7c580b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5efb8640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5806be20 con 0x7f7c5806b650 2023-11-08T19:32:28.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5efb8640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c5806c100 con 0x7f7c580b18c0 2023-11-08T19:32:28.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5efb8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5806c3e0 con 0x7f7c5815aea0 2023-11-08T19:32:28.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5e7b7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c580b18c0 0x7f7c580b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5e7b7640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f7c580b18c0 0x7f7c580b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58846/0 (socket says 172.21.15.119:58846) 2023-11-08T19:32:28.985 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/3644106313 learned_addr learned my addr 172.21.15.119:0/3644106313 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.981+0000 7f7c5d7b5640 1 --1- 172.21.15.119:0/3644106313 >> v1:172.21.15.119:6789/0 conn(0x7f7c5806b650 0x7f7c5806cec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57016/0 (socket says 172.21.15.119:57016) 2023-11-08T19:32:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.982+0000 7f7c5dfb6640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5815aea0 0x7f7c580c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.986 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.982+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/3644106313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5815aea0 msgr2=0x7f7c580c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.982+0000 7f7c5e7b7640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5815aea0 0x7f7c580c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.982+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/3644106313 >> v1:172.21.15.119:6789/0 conn(0x7f7c5806b650 legacy=0x7f7c5806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.982+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/3644106313 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5806c6c0 con 0x7f7c580b18c0 2023-11-08T19:32:28.987 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.982+0000 7f7c5dfb6640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5815aea0 0x7f7c580c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:28.990 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.983+0000 7f7c5e7b7640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c580b18c0 0x7f7c580b1d50 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009870 tx=0x7f7c5002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d1c56c54ad0df6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:28.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.983+0000 7f7c5cfb4640 1 -- 172.21.15.119:0/3644106313 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50030840 con 0x7f7c580b18c0 2023-11-08T19:32:28.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.983+0000 7f7c5cfb4640 1 -- 172.21.15.119:0/3644106313 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c500309a0 con 0x7f7c580b18c0 2023-11-08T19:32:28.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.983+0000 7f7c5cfb4640 1 -- 172.21.15.119:0/3644106313 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50030d30 con 0x7f7c580b18c0 2023-11-08T19:32:28.991 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 -- 172.21.15.119:0/3644106313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c580b18c0 msgr2=0x7f7c580b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c580b18c0 0x7f7c580b1d50 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009870 tx=0x7f7c5002b980 comp rx=0 tx=0).stop 2023-11-08T19:32:28.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 -- 172.21.15.119:0/3644106313 shutdown_connections 2023-11-08T19:32:28.992 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c580b18c0 0x7f7c580b1d50 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/3644106313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5815aea0 0x7f7c580c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 -- 172.21.15.119:0/3644106313 >> 172.21.15.119:0/3644106313 conn(0x7f7c580b0560 msgr2=0x7f7c580b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:28.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 -- 172.21.15.119:0/3644106313 shutdown_connections 2023-11-08T19:32:28.993 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.984+0000 7f7c5efb8640 1 -- 172.21.15.119:0/3644106313 wait complete. 2023-11-08T19:32:28.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.986+0000 7f7c5efb8640 1 Processor -- start 2023-11-08T19:32:28.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.986+0000 7f7c5efb8640 1 -- start start 2023-11-08T19:32:28.994 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5efb8640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5806b650 0x7f7c5815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5efb8640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c580b18c0 0x7f7c580acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5efb8640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 0x7f7c58150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:28.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5efb8640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cd2f0 con 0x7f7c5815aea0 2023-11-08T19:32:28.995 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5efb8640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cc6e0 con 0x7f7c580b18c0 2023-11-08T19:32:28.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5efb8640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c580cc9c0 con 0x7f7c5806b650 2023-11-08T19:32:28.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 0x7f7c58150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.996 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 0x7f7c58150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58862/0 (socket says 172.21.15.119:58862) 2023-11-08T19:32:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/983724355 learned_addr learned my addr 172.21.15.119:0/983724355 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5dfb6640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5806b650 0x7f7c5815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.997 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5806b650 msgr2=0x7f7c5815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5d7b5640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c580b18c0 0x7f7c580acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:28.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5806b650 0x7f7c5815c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.998 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c580b18c0 msgr2=0x7f7c580acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c580b18c0 0x7f7c580acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.987+0000 7f7c5e7b7640 1 -- 172.21.15.119:0/983724355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c500094d0 con 0x7f7c5815aea0 2023-11-08T19:32:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c5d7b5640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c580b18c0 0x7f7c580acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:28.999 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c5e7b7640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 0x7f7c58150200 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009870 tx=0x7f7c50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c500307d0 con 0x7f7c5815aea0 2023-11-08T19:32:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c50039d90 con 0x7f7c5815aea0 2023-11-08T19:32:29.000 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c580ccca0 con 0x7f7c5815aea0 2023-11-08T19:32:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c50043650 con 0x7f7c5815aea0 2023-11-08T19:32:29.001 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.988+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5815cb30 con 0x7f7c5815aea0 2023-11-08T19:32:29.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.989+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c580cd4a0 con 0x7f7c5815aea0 2023-11-08T19:32:29.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.990+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c5002f390 con 0x7f7c5815aea0 2023-11-08T19:32:29.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.991+0000 7f7c4e7fc640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c3c06ae70 0x7f7c3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.003 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.991+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f7c500abd60 con 0x7f7c5815aea0 2023-11-08T19:32:29.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.991+0000 7f7c5dfb6640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c3c06ae70 0x7f7c3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:29.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.992+0000 7f7c5dfb6640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c3c06ae70 0x7f7c3c06d330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7c54008f80 tx=0x7f7c540084d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:29.004 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:28.996+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c50076ab0 con 0x7f7c5815aea0 2023-11-08T19:32:29.189 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.188+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c5806fea0 con 0x7f7c3c06ae70 2023-11-08T19:32:29.190 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.189+0000 7f7c4e7fc640 1 -- 172.21.15.119:0/983724355 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7c5806fea0 con 0x7f7c3c06ae70 2023-11-08T19:32:29.191 INFO:teuthology.orchestra.run.smithi119.stderr:instructed osd(s) 7 to deep-scrub 2023-11-08T19:32:29.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.191+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c3c06ae70 msgr2=0x7f7c3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.193 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.191+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c3c06ae70 0x7f7c3c06d330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f7c54008f80 tx=0x7f7c540084d0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.191+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 msgr2=0x7f7c58150200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.191+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 0x7f7c58150200 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009870 tx=0x7f7c50004730 comp rx=0 tx=0).stop 2023-11-08T19:32:29.194 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 shutdown_connections 2023-11-08T19:32:29.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f7c3c06ae70 0x7f7c3c06d330 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f7c5815aea0 0x7f7c58150200 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7c580b18c0 0x7f7c580acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.195 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 --2- 172.21.15.119:0/983724355 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7c5806b650 0x7f7c5815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 >> 172.21.15.119:0/983724355 conn(0x7f7c580b0560 msgr2=0x7f7c58070100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 shutdown_connections 2023-11-08T19:32:29.196 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.192+0000 7f7c5efb8640 1 -- 172.21.15.119:0/983724355 wait complete. 2023-11-08T19:32:29.212 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.425+0000 7efe5597d640 1 Processor -- start 2023-11-08T19:32:29.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.426+0000 7efe5597d640 1 -- start start 2023-11-08T19:32:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.427+0000 7efe5597d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe5006a390 0x7efe500c4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.429 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.427+0000 7efe5597d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efe500cc810 0x7efe5006cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.427+0000 7efe5597d640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe500ae410 con 0x7efe5006a770 2023-11-08T19:32:29.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.427+0000 7efe5597d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5006ba30 con 0x7efe5006a390 2023-11-08T19:32:29.430 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.427+0000 7efe5597d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5006bcf0 con 0x7efe500cc810 2023-11-08T19:32:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.427+0000 7efe4effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe5006a390 0x7efe500c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4effd640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7efe5006a390 0x7efe500c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58868/0 (socket says 172.21.15.119:58868) 2023-11-08T19:32:29.431 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4effd640 1 -- 172.21.15.119:0/3167422814 learned_addr learned my addr 172.21.15.119:0/3167422814 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4f7fe640 1 --1- 172.21.15.119:0/3167422814 >> v1:172.21.15.119:6789/0 conn(0x7efe5006a770 0x7efe5015ca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:57026/0 (socket says 172.21.15.119:57026) 2023-11-08T19:32:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe467fc640 1 --2- 172.21.15.119:0/3167422814 >> v2:172.21.15.188:3300/0 conn(0x7efe500cc810 0x7efe5006cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:29.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4effd640 1 -- 172.21.15.119:0/3167422814 >> v2:172.21.15.188:3300/0 conn(0x7efe500cc810 msgr2=0x7efe5006cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:29.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4effd640 1 --2- 172.21.15.119:0/3167422814 >> v2:172.21.15.188:3300/0 conn(0x7efe500cc810 0x7efe5006cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4effd640 1 -- 172.21.15.119:0/3167422814 >> v1:172.21.15.119:6789/0 conn(0x7efe5006a770 legacy=0x7efe5015ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4e7fc640 1 -- 172.21.15.119:0/3167422814 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1384178630 0 0) 0x7efe500ae410 con 0x7efe5006a770 2023-11-08T19:32:29.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.428+0000 7efe4effd640 1 -- 172.21.15.119:0/3167422814 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5006bfd0 con 0x7efe5006a390 2023-11-08T19:32:29.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.429+0000 7efe4effd640 1 --2- 172.21.15.119:0/3167422814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a390 0x7efe500c4460 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efe500b1850 tx=0x7efe4002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ddafe617194dae server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:29.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe4e7fc640 1 -- 172.21.15.119:0/3167422814 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40030860 con 0x7efe5006a390 2023-11-08T19:32:29.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe4e7fc640 1 -- 172.21.15.119:0/3167422814 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe400309c0 con 0x7efe5006a390 2023-11-08T19:32:29.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe4e7fc640 1 -- 172.21.15.119:0/3167422814 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe40030d50 con 0x7efe5006a390 2023-11-08T19:32:29.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 -- 172.21.15.119:0/3167422814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a390 msgr2=0x7efe500c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 --2- 172.21.15.119:0/3167422814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a390 0x7efe500c4460 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7efe500b1850 tx=0x7efe4002b9a0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 -- 172.21.15.119:0/3167422814 shutdown_connections 2023-11-08T19:32:29.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 --2- 172.21.15.119:0/3167422814 >> v2:172.21.15.188:3300/0 conn(0x7efe500cc810 0x7efe5006cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 --2- 172.21.15.119:0/3167422814 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a390 0x7efe500c4460 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 -- 172.21.15.119:0/3167422814 >> 172.21.15.119:0/3167422814 conn(0x7efe500cb610 msgr2=0x7efe500cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:29.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 -- 172.21.15.119:0/3167422814 shutdown_connections 2023-11-08T19:32:29.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.430+0000 7efe5597d640 1 -- 172.21.15.119:0/3167422814 wait complete. 2023-11-08T19:32:29.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.432+0000 7efe5597d640 1 Processor -- start 2023-11-08T19:32:29.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.432+0000 7efe5597d640 1 -- start start 2023-11-08T19:32:29.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.433+0000 7efe5597d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe5006a390 0x7efe5015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.433+0000 7efe5597d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 0x7efe5015fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.433+0000 7efe5597d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe500cc810 0x7efe5014a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.433+0000 7efe5597d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe500b37b0 con 0x7efe5006a770 2023-11-08T19:32:29.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.433+0000 7efe5597d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe500b45d0 con 0x7efe500cc810 2023-11-08T19:32:29.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.433+0000 7efe5597d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe500b39c0 con 0x7efe5006a390 2023-11-08T19:32:29.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 0x7efe5015fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:29.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 0x7efe5015fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:58880/0 (socket says 172.21.15.119:58880) 2023-11-08T19:32:29.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 -- 172.21.15.119:0/1789079227 learned_addr learned my addr 172.21.15.119:0/1789079227 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:29.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 -- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe5006a390 msgr2=0x7efe5015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe5006a390 0x7efe5015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 -- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe500cc810 msgr2=0x7efe5014a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe4f7fe640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe500cc810 0x7efe5014a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:29.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe500cc810 0x7efe5014a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 -- 172.21.15.119:0/1789079227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe400094d0 con 0x7efe5006a770 2023-11-08T19:32:29.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.434+0000 7efe467fc640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 0x7efe5015fb70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c7c0 tx=0x7efe3800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:29.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.435+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38007820 con 0x7efe5006a770 2023-11-08T19:32:29.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.435+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38007980 con 0x7efe5006a770 2023-11-08T19:32:29.446 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.435+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38007cf0 con 0x7efe5006a770 2023-11-08T19:32:29.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.435+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe500b3ca0 con 0x7efe5006a770 2023-11-08T19:32:29.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.435+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe500b4260 con 0x7efe5006a770 2023-11-08T19:32:29.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.436+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe50074430 con 0x7efe5006a770 2023-11-08T19:32:29.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.437+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe380113f0 con 0x7efe5006a770 2023-11-08T19:32:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.438+0000 7efe4cff9640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.438+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7efe38089780 con 0x7efe5006a770 2023-11-08T19:32:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.438+0000 7efe4effd640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:29.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.439+0000 7efe4effd640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efe40036000 tx=0x7efe4003b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:29.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.443+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe38054320 con 0x7efe5006a770 2023-11-08T19:32:29.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.632+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efe500b4740 con 0x7efe3006aea0 2023-11-08T19:32:29.656 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.654+0000 7efe4cff9640 1 -- 172.21.15.119:0/1789079227 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289132 (secure 0 0 0) 0x7efe500b4740 con 0x7efe3006aea0 2023-11-08T19:32:29.659 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:29.661 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:32:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 msgr2=0x7efe3006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efe40036000 tx=0x7efe4003b040 comp rx=0 tx=0).stop 2023-11-08T19:32:29.663 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 msgr2=0x7efe5015fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:29.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 0x7efe5015fb70 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c7c0 tx=0x7efe3800cc80 comp rx=0 tx=0).stop 2023-11-08T19:32:29.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 shutdown_connections 2023-11-08T19:32:29.664 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7efe3006aea0 0x7efe3006d360 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efe500cc810 0x7efe5014a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7efe5006a770 0x7efe5015fb70 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.665 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 --2- 172.21.15.119:0/1789079227 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efe5006a390 0x7efe5015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 >> 172.21.15.119:0/1789079227 conn(0x7efe500cb610 msgr2=0x7efe500ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.659+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 shutdown_connections 2023-11-08T19:32:29.666 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:29.660+0000 7efe5597d640 1 -- 172.21.15.119:0/1789079227 wait complete. 2023-11-08T19:32:29.680 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2023-11-08T19:32:28.842968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261164,"kb_used_data":11712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713556,"statfs":{"total":773094113280,"available":772826681344,"internally_reserved":0,"allocated":11993088,"data_stored":6708042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.746713"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":49,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.093812+0000","last_change":"2023-11-08T19:27:47.802097+0000","last_active":"2023-11-08T19:32:16.093812+0000","last_peered":"2023-11-08T19:32:16.093812+0000","last_clean":"2023-11-08T19:32:16.093812+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:16.093812+0000","last_undegraded":"2023-11-08T19:32:16.093812+0000","last_fullsized":"2023-11-08T19:32:16.093812+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":54,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.259160+0000","last_change":"2023-11-08T19:27:49.258256+0000","last_active":"2023-11-08T19:32:16.259160+0000","last_peered":"2023-11-08T19:32:16.259160+0000","last_clean":"2023-11-08T19:32:16.259160+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:16.259160+0000","last_undegraded":"2023-11-08T19:32:16.259160+0000","last_fullsized":"2023-11-08T19:32:16.259160+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:39.328571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.994792+0000","last_change":"2023-11-08T19:26:41.727442+0000","last_active":"2023-11-08T19:32:18.994792+0000","last_peered":"2023-11-08T19:32:18.994792+0000","last_clean":"2023-11-08T19:32:18.994792+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:18.994792+0000","last_undegraded":"2023-11-08T19:32:18.994792+0000","last_fullsized":"2023-11-08T19:32:18.994792+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":45,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:19.000034+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:32:19.000034+0000","last_peered":"2023-11-08T19:32:19.000034+0000","last_clean":"2023-11-08T19:32:19.000034+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:19.000034+0000","last_undegraded":"2023-11-08T19:32:19.000034+0000","last_fullsized":"2023-11-08T19:32:19.000034+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":54,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.255077+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:32:16.255077+0000","last_peered":"2023-11-08T19:32:16.255077+0000","last_clean":"2023-11-08T19:32:16.255077+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:32:16.255077+0000","last_undegraded":"2023-11-08T19:32:16.255077+0000","last_fullsized":"2023-11-08T19:32:16.255077+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.092870+0000","last_change":"2023-11-08T19:26:41.729586+0000","last_active":"2023-11-08T19:32:16.092870+0000","last_peered":"2023-11-08T19:32:16.092870+0000","last_clean":"2023-11-08T19:32:16.092870+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:16.092870+0000","last_undegraded":"2023-11-08T19:32:16.092870+0000","last_fullsized":"2023-11-08T19:32:16.092870+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:54:09.506365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":48,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.092175+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:32:16.092175+0000","last_peered":"2023-11-08T19:32:16.092175+0000","last_clean":"2023-11-08T19:32:16.092175+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:32:16.092175+0000","last_undegraded":"2023-11-08T19:32:16.092175+0000","last_fullsized":"2023-11-08T19:32:16.092175+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:08.296724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.100970+0000","last_change":"2023-11-08T19:27:48.787970+0000","last_active":"2023-11-08T19:32:17.100970+0000","last_peered":"2023-11-08T19:32:17.100970+0000","last_clean":"2023-11-08T19:32:17.100970+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:17.100970+0000","last_undegraded":"2023-11-08T19:32:17.100970+0000","last_fullsized":"2023-11-08T19:32:17.100970+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":78,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.256173+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:32:16.256173+0000","last_peered":"2023-11-08T19:32:16.256173+0000","last_clean":"2023-11-08T19:32:16.256173+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:16.256173+0000","last_undegraded":"2023-11-08T19:32:16.256173+0000","last_fullsized":"2023-11-08T19:32:16.256173+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:14:13.359070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.829303+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:32:18.829303+0000","last_peered":"2023-11-08T19:32:18.829303+0000","last_clean":"2023-11-08T19:32:18.829303+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:18.829303+0000","last_undegraded":"2023-11-08T19:32:18.829303+0000","last_fullsized":"2023-11-08T19:32:18.829303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":54,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258571+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:32:16.258571+0000","last_peered":"2023-11-08T19:32:16.258571+0000","last_clean":"2023-11-08T19:32:16.258571+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:32:16.258571+0000","last_undegraded":"2023-11-08T19:32:16.258571+0000","last_fullsized":"2023-11-08T19:32:16.258571+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.995098+0000","last_change":"2023-11-08T19:26:41.726645+0000","last_active":"2023-11-08T19:32:18.995098+0000","last_peered":"2023-11-08T19:32:18.995098+0000","last_clean":"2023-11-08T19:32:18.995098+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:18.995098+0000","last_undegraded":"2023-11-08T19:32:18.995098+0000","last_fullsized":"2023-11-08T19:32:18.995098+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":45,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253871+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:32:16.253871+0000","last_peered":"2023-11-08T19:32:16.253871+0000","last_clean":"2023-11-08T19:32:16.253871+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:16.253871+0000","last_undegraded":"2023-11-08T19:32:16.253871+0000","last_fullsized":"2023-11-08T19:32:16.253871+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":53,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.343549+0000","last_change":"2023-11-08T19:27:49.198015+0000","last_active":"2023-11-08T19:32:26.343549+0000","last_peered":"2023-11-08T19:32:26.343549+0000","last_clean":"2023-11-08T19:32:26.343549+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:26.343549+0000","last_undegraded":"2023-11-08T19:32:26.343549+0000","last_fullsized":"2023-11-08T19:32:26.343549+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":78,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253587+0000","last_change":"2023-11-08T19:26:41.727324+0000","last_active":"2023-11-08T19:32:16.253587+0000","last_peered":"2023-11-08T19:32:16.253587+0000","last_clean":"2023-11-08T19:32:16.253587+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:16.253587+0000","last_undegraded":"2023-11-08T19:32:16.253587+0000","last_fullsized":"2023-11-08T19:32:16.253587+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:39.563504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.827031+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:32:18.827031+0000","last_peered":"2023-11-08T19:32:18.827031+0000","last_clean":"2023-11-08T19:32:18.827031+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:32:18.827031+0000","last_undegraded":"2023-11-08T19:32:18.827031+0000","last_fullsized":"2023-11-08T19:32:18.827031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.927331+0000","last_change":"2023-11-08T19:27:48.786692+0000","last_active":"2023-11-08T19:32:24.927331+0000","last_peered":"2023-11-08T19:32:24.927331+0000","last_clean":"2023-11-08T19:32:24.927331+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:24.927331+0000","last_undegraded":"2023-11-08T19:32:24.927331+0000","last_fullsized":"2023-11-08T19:32:24.927331+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.827046+0000","last_change":"2023-11-08T19:26:41.840432+0000","last_active":"2023-11-08T19:32:18.827046+0000","last_peered":"2023-11-08T19:32:18.827046+0000","last_clean":"2023-11-08T19:32:18.827046+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:18.827046+0000","last_undegraded":"2023-11-08T19:32:18.827046+0000","last_fullsized":"2023-11-08T19:32:18.827046+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.991590+0000","last_change":"2023-11-08T19:27:47.804907+0000","last_active":"2023-11-08T19:32:18.991590+0000","last_peered":"2023-11-08T19:32:18.991590+0000","last_clean":"2023-11-08T19:32:18.991590+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:18.991590+0000","last_undegraded":"2023-11-08T19:32:18.991590+0000","last_fullsized":"2023-11-08T19:32:18.991590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.208903+0000","last_change":"2023-11-08T19:27:48.914055+0000","last_active":"2023-11-08T19:32:24.208903+0000","last_peered":"2023-11-08T19:32:24.208903+0000","last_clean":"2023-11-08T19:32:24.208903+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:24.208903+0000","last_undegraded":"2023-11-08T19:32:24.208903+0000","last_fullsized":"2023-11-08T19:32:24.208903+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.826795+0000","last_change":"2023-11-08T19:26:41.841438+0000","last_active":"2023-11-08T19:32:18.826795+0000","last_peered":"2023-11-08T19:32:18.826795+0000","last_clean":"2023-11-08T19:32:18.826795+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:18.826795+0000","last_undegraded":"2023-11-08T19:32:18.826795+0000","last_fullsized":"2023-11-08T19:32:18.826795+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.990618+0000","last_change":"2023-11-08T19:27:47.795963+0000","last_active":"2023-11-08T19:32:18.990618+0000","last_peered":"2023-11-08T19:32:18.990618+0000","last_clean":"2023-11-08T19:32:18.990618+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:18.990618+0000","last_undegraded":"2023-11-08T19:32:18.990618+0000","last_fullsized":"2023-11-08T19:32:18.990618+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.105842+0000","last_change":"2023-11-08T19:27:48.791586+0000","last_active":"2023-11-08T19:32:17.105842+0000","last_peered":"2023-11-08T19:32:17.105842+0000","last_clean":"2023-11-08T19:32:17.105842+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:17.105842+0000","last_undegraded":"2023-11-08T19:32:17.105842+0000","last_fullsized":"2023-11-08T19:32:17.105842+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:06.670569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.093878+0000","last_change":"2023-11-08T19:26:42.042527+0000","last_active":"2023-11-08T19:32:16.093878+0000","last_peered":"2023-11-08T19:32:16.093878+0000","last_clean":"2023-11-08T19:32:16.093878+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:16.093878+0000","last_undegraded":"2023-11-08T19:32:16.093878+0000","last_fullsized":"2023-11-08T19:32:16.093878+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:01:03.470958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":48,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.255935+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:32:16.255935+0000","last_peered":"2023-11-08T19:32:16.255935+0000","last_clean":"2023-11-08T19:32:16.255935+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:32:16.255935+0000","last_undegraded":"2023-11-08T19:32:16.255935+0000","last_fullsized":"2023-11-08T19:32:16.255935+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":56,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.096906+0000","last_change":"2023-11-08T19:27:48.792992+0000","last_active":"2023-11-08T19:32:16.096906+0000","last_peered":"2023-11-08T19:32:16.096906+0000","last_clean":"2023-11-08T19:32:16.096906+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:16.096906+0000","last_undegraded":"2023-11-08T19:32:16.096906+0000","last_fullsized":"2023-11-08T19:32:16.096906+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:07.961783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":77,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258427+0000","last_change":"2023-11-08T19:26:41.761224+0000","last_active":"2023-11-08T19:32:16.258427+0000","last_peered":"2023-11-08T19:32:16.258427+0000","last_clean":"2023-11-08T19:32:16.258427+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:16.258427+0000","last_undegraded":"2023-11-08T19:32:16.258427+0000","last_fullsized":"2023-11-08T19:32:16.258427+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:22:57.302151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":49,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.096307+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:32:16.096307+0000","last_peered":"2023-11-08T19:32:16.096307+0000","last_clean":"2023-11-08T19:32:16.096307+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:32:16.096307+0000","last_undegraded":"2023-11-08T19:32:16.096307+0000","last_fullsized":"2023-11-08T19:32:16.096307+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.254631+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:32:16.254631+0000","last_peered":"2023-11-08T19:32:16.254631+0000","last_clean":"2023-11-08T19:32:16.254631+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:32:16.254631+0000","last_undegraded":"2023-11-08T19:32:16.254631+0000","last_fullsized":"2023-11-08T19:32:16.254631+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:37.340234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":76,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:19.002243+0000","last_change":"2023-11-08T19:26:42.044335+0000","last_active":"2023-11-08T19:32:19.002243+0000","last_peered":"2023-11-08T19:32:19.002243+0000","last_clean":"2023-11-08T19:32:19.002243+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:19.002243+0000","last_undegraded":"2023-11-08T19:32:19.002243+0000","last_fullsized":"2023-11-08T19:32:19.002243+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":45,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.255021+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:32:16.255021+0000","last_peered":"2023-11-08T19:32:16.255021+0000","last_clean":"2023-11-08T19:32:16.255021+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:32:16.255021+0000","last_undegraded":"2023-11-08T19:32:16.255021+0000","last_fullsized":"2023-11-08T19:32:16.255021+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:05:07.684742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825130+0000","last_change":"2023-11-08T19:27:48.916608+0000","last_active":"2023-11-08T19:32:18.825130+0000","last_peered":"2023-11-08T19:32:18.825130+0000","last_clean":"2023-11-08T19:32:18.825130+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:18.825130+0000","last_undegraded":"2023-11-08T19:32:18.825130+0000","last_fullsized":"2023-11-08T19:32:18.825130+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":77,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.259303+0000","last_change":"2023-11-08T19:26:41.761890+0000","last_active":"2023-11-08T19:32:16.259303+0000","last_peered":"2023-11-08T19:32:16.259303+0000","last_clean":"2023-11-08T19:32:16.259303+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:16.259303+0000","last_undegraded":"2023-11-08T19:32:16.259303+0000","last_fullsized":"2023-11-08T19:32:16.259303+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:46.394448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":47,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253058+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:32:16.253058+0000","last_peered":"2023-11-08T19:32:16.253058+0000","last_clean":"2023-11-08T19:32:16.253058+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:32:16.253058+0000","last_undegraded":"2023-11-08T19:32:16.253058+0000","last_fullsized":"2023-11-08T19:32:16.253058+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:24:43.245410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825158+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:32:18.825158+0000","last_peered":"2023-11-08T19:32:18.825158+0000","last_clean":"2023-11-08T19:32:18.825158+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:18.825158+0000","last_undegraded":"2023-11-08T19:32:18.825158+0000","last_fullsized":"2023-11-08T19:32:18.825158+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.097650+0000","last_change":"2023-11-08T19:26:42.043381+0000","last_active":"2023-11-08T19:32:16.097650+0000","last_peered":"2023-11-08T19:32:16.097650+0000","last_clean":"2023-11-08T19:32:16.097650+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:16.097650+0000","last_undegraded":"2023-11-08T19:32:16.097650+0000","last_fullsized":"2023-11-08T19:32:16.097650+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":48,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.096244+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:32:16.096244+0000","last_peered":"2023-11-08T19:32:16.096244+0000","last_clean":"2023-11-08T19:32:16.096244+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:16.096244+0000","last_undegraded":"2023-11-08T19:32:16.096244+0000","last_fullsized":"2023-11-08T19:32:16.096244+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:38.354406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.093317+0000","last_change":"2023-11-08T19:27:48.788766+0000","last_active":"2023-11-08T19:32:16.093317+0000","last_peered":"2023-11-08T19:32:16.093317+0000","last_clean":"2023-11-08T19:32:16.093317+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:16.093317+0000","last_undegraded":"2023-11-08T19:32:16.093317+0000","last_fullsized":"2023-11-08T19:32:16.093317+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":77,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.251913+0000","last_change":"2023-11-08T19:26:41.760333+0000","last_active":"2023-11-08T19:32:16.251913+0000","last_peered":"2023-11-08T19:32:16.251913+0000","last_clean":"2023-11-08T19:32:16.251913+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:16.251913+0000","last_undegraded":"2023-11-08T19:32:16.251913+0000","last_fullsized":"2023-11-08T19:32:16.251913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:00:00.317242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.990685+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:32:18.990685+0000","last_peered":"2023-11-08T19:32:18.990685+0000","last_clean":"2023-11-08T19:32:18.990685+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:18.990685+0000","last_undegraded":"2023-11-08T19:32:18.990685+0000","last_fullsized":"2023-11-08T19:32:18.990685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.096669+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:32:16.096669+0000","last_peered":"2023-11-08T19:32:16.096669+0000","last_clean":"2023-11-08T19:32:16.096669+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:16.096669+0000","last_undegraded":"2023-11-08T19:32:16.096669+0000","last_fullsized":"2023-11-08T19:32:16.096669+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:55.726407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.631351+0000","last_change":"2023-11-08T19:32:26.631371+0000","last_active":"2023-11-08T19:32:26.631351+0000","last_peered":"2023-11-08T19:32:26.631351+0000","last_clean":"2023-11-08T19:32:26.631351+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:26.631351+0000","last_undegraded":"2023-11-08T19:32:26.631351+0000","last_fullsized":"2023-11-08T19:32:26.631351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.631277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:31:19.710068+0000","scrub_duration":0.0020387719999999999,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":45,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.991220+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:32:18.991220+0000","last_peered":"2023-11-08T19:32:18.991220+0000","last_clean":"2023-11-08T19:32:18.991220+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:18.991220+0000","last_undegraded":"2023-11-08T19:32:18.991220+0000","last_fullsized":"2023-11-08T19:32:18.991220+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":53,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.988461+0000","last_change":"2023-11-08T19:27:48.786529+0000","last_active":"2023-11-08T19:32:18.988461+0000","last_peered":"2023-11-08T19:32:18.988461+0000","last_clean":"2023-11-08T19:32:18.988461+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:18.988461+0000","last_undegraded":"2023-11-08T19:32:18.988461+0000","last_fullsized":"2023-11-08T19:32:18.988461+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.827040+0000","last_change":"2023-11-08T19:26:41.841874+0000","last_active":"2023-11-08T19:32:18.827040+0000","last_peered":"2023-11-08T19:32:18.827040+0000","last_clean":"2023-11-08T19:32:18.827040+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:18.827040+0000","last_undegraded":"2023-11-08T19:32:18.827040+0000","last_fullsized":"2023-11-08T19:32:18.827040+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":45,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.251823+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:32:16.251823+0000","last_peered":"2023-11-08T19:32:16.251823+0000","last_clean":"2023-11-08T19:32:16.251823+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:16.251823+0000","last_undegraded":"2023-11-08T19:32:16.251823+0000","last_fullsized":"2023-11-08T19:32:16.251823+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:06:08.975955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825077+0000","last_change":"2023-11-08T19:27:48.914081+0000","last_active":"2023-11-08T19:32:18.825077+0000","last_peered":"2023-11-08T19:32:18.825077+0000","last_clean":"2023-11-08T19:32:18.825077+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:18.825077+0000","last_undegraded":"2023-11-08T19:32:18.825077+0000","last_fullsized":"2023-11-08T19:32:18.825077+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":78,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.256524+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:32:16.256524+0000","last_peered":"2023-11-08T19:32:16.256524+0000","last_clean":"2023-11-08T19:32:16.256524+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:16.256524+0000","last_undegraded":"2023-11-08T19:32:16.256524+0000","last_fullsized":"2023-11-08T19:32:16.256524+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:13:21.945988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":46,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258606+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:32:16.258606+0000","last_peered":"2023-11-08T19:32:16.258606+0000","last_clean":"2023-11-08T19:32:16.258606+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:32:16.258606+0000","last_undegraded":"2023-11-08T19:32:16.258606+0000","last_fullsized":"2023-11-08T19:32:16.258606+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:46.487562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.994885+0000","last_change":"2023-11-08T19:27:48.787043+0000","last_active":"2023-11-08T19:32:18.994885+0000","last_peered":"2023-11-08T19:32:18.994885+0000","last_clean":"2023-11-08T19:32:18.994885+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:18.994885+0000","last_undegraded":"2023-11-08T19:32:18.994885+0000","last_fullsized":"2023-11-08T19:32:18.994885+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":78,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258830+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:32:16.258830+0000","last_peered":"2023-11-08T19:32:16.258830+0000","last_clean":"2023-11-08T19:32:16.258830+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:16.258830+0000","last_undegraded":"2023-11-08T19:32:16.258830+0000","last_fullsized":"2023-11-08T19:32:16.258830+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:14:15.070954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":49,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.096685+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:32:16.096685+0000","last_peered":"2023-11-08T19:32:16.096685+0000","last_clean":"2023-11-08T19:32:16.096685+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:16.096685+0000","last_undegraded":"2023-11-08T19:32:16.096685+0000","last_fullsized":"2023-11-08T19:32:16.096685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.994578+0000","last_change":"2023-11-08T19:27:48.787246+0000","last_active":"2023-11-08T19:32:18.994578+0000","last_peered":"2023-11-08T19:32:18.994578+0000","last_clean":"2023-11-08T19:32:18.994578+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:18.994578+0000","last_undegraded":"2023-11-08T19:32:18.994578+0000","last_fullsized":"2023-11-08T19:32:18.994578+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":78,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253704+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:32:16.253704+0000","last_peered":"2023-11-08T19:32:16.253704+0000","last_clean":"2023-11-08T19:32:16.253704+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:32:16.253704+0000","last_undegraded":"2023-11-08T19:32:16.253704+0000","last_fullsized":"2023-11-08T19:32:16.253704+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:45.197535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":49,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.092579+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:32:16.092579+0000","last_peered":"2023-11-08T19:32:16.092579+0000","last_clean":"2023-11-08T19:32:16.092579+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:32:16.092579+0000","last_undegraded":"2023-11-08T19:32:16.092579+0000","last_fullsized":"2023-11-08T19:32:16.092579+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:32.935992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253259+0000","last_change":"2023-11-08T19:27:49.257192+0000","last_active":"2023-11-08T19:32:16.253259+0000","last_peered":"2023-11-08T19:32:16.253259+0000","last_clean":"2023-11-08T19:32:16.253259+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:16.253259+0000","last_undegraded":"2023-11-08T19:32:16.253259+0000","last_fullsized":"2023-11-08T19:32:16.253259+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:07.282552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.836787+0000","last_change":"2023-11-08T19:26:41.841299+0000","last_active":"2023-11-08T19:32:18.836787+0000","last_peered":"2023-11-08T19:32:18.836787+0000","last_clean":"2023-11-08T19:32:18.836787+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:18.836787+0000","last_undegraded":"2023-11-08T19:32:18.836787+0000","last_fullsized":"2023-11-08T19:32:18.836787+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":48,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253460+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:32:16.253460+0000","last_peered":"2023-11-08T19:32:16.253460+0000","last_clean":"2023-11-08T19:32:16.253460+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:32:16.253460+0000","last_undegraded":"2023-11-08T19:32:16.253460+0000","last_fullsized":"2023-11-08T19:32:16.253460+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:20.527143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.258474+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:32:16.258474+0000","last_peered":"2023-11-08T19:32:16.258474+0000","last_clean":"2023-11-08T19:32:16.258474+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:32:16.258474+0000","last_undegraded":"2023-11-08T19:32:16.258474+0000","last_fullsized":"2023-11-08T19:32:16.258474+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:07:43.093787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":78,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.101786+0000","last_change":"2023-11-08T19:26:41.934223+0000","last_active":"2023-11-08T19:32:17.101786+0000","last_peered":"2023-11-08T19:32:17.101786+0000","last_clean":"2023-11-08T19:32:17.101786+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:17.101786+0000","last_undegraded":"2023-11-08T19:32:17.101786+0000","last_fullsized":"2023-11-08T19:32:17.101786+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.826195+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:32:18.826195+0000","last_peered":"2023-11-08T19:32:18.826195+0000","last_clean":"2023-11-08T19:32:18.826195+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:18.826195+0000","last_undegraded":"2023-11-08T19:32:18.826195+0000","last_fullsized":"2023-11-08T19:32:18.826195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":56,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.093692+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:32:16.093692+0000","last_peered":"2023-11-08T19:32:16.093692+0000","last_clean":"2023-11-08T19:32:16.093692+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:16.093692+0000","last_undegraded":"2023-11-08T19:32:16.093692+0000","last_fullsized":"2023-11-08T19:32:16.093692+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:28.014086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.828769+0000","last_change":"2023-11-08T19:26:41.840288+0000","last_active":"2023-11-08T19:32:18.828769+0000","last_peered":"2023-11-08T19:32:18.828769+0000","last_clean":"2023-11-08T19:32:18.828769+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:18.828769+0000","last_undegraded":"2023-11-08T19:32:18.828769+0000","last_fullsized":"2023-11-08T19:32:18.828769+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":50,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.091751+0000","last_change":"2023-11-08T19:27:47.806105+0000","last_active":"2023-11-08T19:32:16.091751+0000","last_peered":"2023-11-08T19:32:16.091751+0000","last_clean":"2023-11-08T19:32:16.091751+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:16.091751+0000","last_undegraded":"2023-11-08T19:32:16.091751+0000","last_fullsized":"2023-11-08T19:32:16.091751+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:06.262589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.848482+0000","last_change":"2023-11-08T19:32:24.848528+0000","last_active":"2023-11-08T19:32:24.848482+0000","last_peered":"2023-11-08T19:32:24.848482+0000","last_clean":"2023-11-08T19:32:24.848482+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:24.848482+0000","last_undegraded":"2023-11-08T19:32:24.848482+0000","last_fullsized":"2023-11-08T19:32:24.848482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_clean_scrub_stamp":"2023-11-08T19:32:24.848327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:22.967345+0000","scrub_duration":0.0035992749999999999,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":78,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.099308+0000","last_change":"2023-11-08T19:26:41.933671+0000","last_active":"2023-11-08T19:32:17.099308+0000","last_peered":"2023-11-08T19:32:17.099308+0000","last_clean":"2023-11-08T19:32:17.099308+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:17.099308+0000","last_undegraded":"2023-11-08T19:32:17.099308+0000","last_fullsized":"2023-11-08T19:32:17.099308+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":48,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.252742+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:32:16.252742+0000","last_peered":"2023-11-08T19:32:16.252742+0000","last_clean":"2023-11-08T19:32:16.252742+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:32:16.252742+0000","last_undegraded":"2023-11-08T19:32:16.252742+0000","last_fullsized":"2023-11-08T19:32:16.252742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:01:35.911874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":56,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.089261+0000","last_change":"2023-11-08T19:27:48.789157+0000","last_active":"2023-11-08T19:32:16.089261+0000","last_peered":"2023-11-08T19:32:16.089261+0000","last_clean":"2023-11-08T19:32:16.089261+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:16.089261+0000","last_undegraded":"2023-11-08T19:32:16.089261+0000","last_fullsized":"2023-11-08T19:32:16.089261+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:09:19.512970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:25.703825+0000","last_change":"2023-11-08T19:32:25.703850+0000","last_active":"2023-11-08T19:32:25.703825+0000","last_peered":"2023-11-08T19:32:25.703825+0000","last_clean":"2023-11-08T19:32:25.703825+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:25.703825+0000","last_undegraded":"2023-11-08T19:32:25.703825+0000","last_fullsized":"2023-11-08T19:32:25.703825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_clean_scrub_stamp":"2023-11-08T19:32:25.703734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:40.388881+0000","scrub_duration":0.0027277820000000002,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":46,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.251919+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:32:16.251919+0000","last_peered":"2023-11-08T19:32:16.251919+0000","last_clean":"2023-11-08T19:32:16.251919+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:16.251919+0000","last_undegraded":"2023-11-08T19:32:16.251919+0000","last_fullsized":"2023-11-08T19:32:16.251919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:56:57.732162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.253093+0000","last_change":"2023-11-08T19:27:49.255748+0000","last_active":"2023-11-08T19:32:16.253093+0000","last_peered":"2023-11-08T19:32:16.253093+0000","last_clean":"2023-11-08T19:32:16.253093+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:16.253093+0000","last_undegraded":"2023-11-08T19:32:16.253093+0000","last_fullsized":"2023-11-08T19:32:16.253093+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:00:24.016131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":76,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:19.001438+0000","last_change":"2023-11-08T19:26:42.041825+0000","last_active":"2023-11-08T19:32:19.001438+0000","last_peered":"2023-11-08T19:32:19.001438+0000","last_clean":"2023-11-08T19:32:19.001438+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:19.001438+0000","last_undegraded":"2023-11-08T19:32:19.001438+0000","last_fullsized":"2023-11-08T19:32:19.001438+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.109748+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:32:17.109748+0000","last_peered":"2023-11-08T19:32:17.109748+0000","last_clean":"2023-11-08T19:32:17.109748+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:32:17.109748+0000","last_undegraded":"2023-11-08T19:32:17.109748+0000","last_fullsized":"2023-11-08T19:32:17.109748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825240+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:32:18.825240+0000","last_peered":"2023-11-08T19:32:18.825240+0000","last_clean":"2023-11-08T19:32:18.825240+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:18.825240+0000","last_undegraded":"2023-11-08T19:32:18.825240+0000","last_fullsized":"2023-11-08T19:32:18.825240+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":77,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.260311+0000","last_change":"2023-11-08T19:26:41.761261+0000","last_active":"2023-11-08T19:32:16.260311+0000","last_peered":"2023-11-08T19:32:16.260311+0000","last_clean":"2023-11-08T19:32:16.260311+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:16.260311+0000","last_undegraded":"2023-11-08T19:32:16.260311+0000","last_fullsized":"2023-11-08T19:32:16.260311+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:37:20.983273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.948950+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:32:18.948950+0000","last_peered":"2023-11-08T19:32:18.948950+0000","last_clean":"2023-11-08T19:32:18.948950+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:18.948950+0000","last_undegraded":"2023-11-08T19:32:18.948950+0000","last_fullsized":"2023-11-08T19:32:18.948950+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:31:29.170378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":55,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.812448+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:32:16.812448+0000","last_peered":"2023-11-08T19:32:16.812448+0000","last_clean":"2023-11-08T19:32:16.812448+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:32:16.812448+0000","last_undegraded":"2023-11-08T19:32:16.812448+0000","last_fullsized":"2023-11-08T19:32:16.812448+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:37:31.921819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.829643+0000","last_change":"2023-11-08T19:26:41.841041+0000","last_active":"2023-11-08T19:32:18.829643+0000","last_peered":"2023-11-08T19:32:18.829643+0000","last_clean":"2023-11-08T19:32:18.829643+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:18.829643+0000","last_undegraded":"2023-11-08T19:32:18.829643+0000","last_fullsized":"2023-11-08T19:32:18.829643+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.490329+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:32:18.490329+0000","last_peered":"2023-11-08T19:32:18.490329+0000","last_clean":"2023-11-08T19:32:18.490329+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:18.490329+0000","last_undegraded":"2023-11-08T19:32:18.490329+0000","last_fullsized":"2023-11-08T19:32:18.490329+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:40.342743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825318+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:32:18.825318+0000","last_peered":"2023-11-08T19:32:18.825318+0000","last_clean":"2023-11-08T19:32:18.825318+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:18.825318+0000","last_undegraded":"2023-11-08T19:32:18.825318+0000","last_fullsized":"2023-11-08T19:32:18.825318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":78,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.256389+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:32:16.256389+0000","last_peered":"2023-11-08T19:32:16.256389+0000","last_clean":"2023-11-08T19:32:16.256389+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:16.256389+0000","last_undegraded":"2023-11-08T19:32:16.256389+0000","last_fullsized":"2023-11-08T19:32:16.256389+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:17:09.136166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.100704+0000","last_change":"2023-11-08T19:27:47.803388+0000","last_active":"2023-11-08T19:32:17.100704+0000","last_peered":"2023-11-08T19:32:17.100704+0000","last_clean":"2023-11-08T19:32:17.100704+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:17.100704+0000","last_undegraded":"2023-11-08T19:32:17.100704+0000","last_fullsized":"2023-11-08T19:32:17.100704+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:38.916289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.099135+0000","last_change":"2023-11-08T19:27:49.252514+0000","last_active":"2023-11-08T19:32:17.099135+0000","last_peered":"2023-11-08T19:32:17.099135+0000","last_clean":"2023-11-08T19:32:17.099135+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:17.099135+0000","last_undegraded":"2023-11-08T19:32:17.099135+0000","last_fullsized":"2023-11-08T19:32:17.099135+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.092334+0000","last_change":"2023-11-08T19:26:42.041860+0000","last_active":"2023-11-08T19:32:16.092334+0000","last_peered":"2023-11-08T19:32:16.092334+0000","last_clean":"2023-11-08T19:32:16.092334+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:16.092334+0000","last_undegraded":"2023-11-08T19:32:16.092334+0000","last_fullsized":"2023-11-08T19:32:16.092334+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:29:17.455330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.099392+0000","last_change":"2023-11-08T19:27:47.799290+0000","last_active":"2023-11-08T19:32:17.099392+0000","last_peered":"2023-11-08T19:32:17.099392+0000","last_clean":"2023-11-08T19:32:17.099392+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:17.099392+0000","last_undegraded":"2023-11-08T19:32:17.099392+0000","last_fullsized":"2023-11-08T19:32:17.099392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":53,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.991929+0000","last_change":"2023-11-08T19:27:49.199254+0000","last_active":"2023-11-08T19:32:18.991929+0000","last_peered":"2023-11-08T19:32:18.991929+0000","last_clean":"2023-11-08T19:32:18.991929+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:18.991929+0000","last_undegraded":"2023-11-08T19:32:18.991929+0000","last_fullsized":"2023-11-08T19:32:18.991929+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.089181+0000","last_change":"2023-11-08T19:26:41.725798+0000","last_active":"2023-11-08T19:32:16.089181+0000","last_peered":"2023-11-08T19:32:16.089181+0000","last_clean":"2023-11-08T19:32:16.089181+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:16.089181+0000","last_undegraded":"2023-11-08T19:32:16.089181+0000","last_fullsized":"2023-11-08T19:32:16.089181+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:58.881401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":50,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.090199+0000","last_change":"2023-11-08T19:27:47.801496+0000","last_active":"2023-11-08T19:32:16.090199+0000","last_peered":"2023-11-08T19:32:16.090199+0000","last_clean":"2023-11-08T19:32:16.090199+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:16.090199+0000","last_undegraded":"2023-11-08T19:32:16.090199+0000","last_fullsized":"2023-11-08T19:32:16.090199+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:41.050155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":53,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.989160+0000","last_change":"2023-11-08T19:27:49.198261+0000","last_active":"2023-11-08T19:32:18.989160+0000","last_peered":"2023-11-08T19:32:18.989160+0000","last_clean":"2023-11-08T19:32:18.989160+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:18.989160+0000","last_undegraded":"2023-11-08T19:32:18.989160+0000","last_fullsized":"2023-11-08T19:32:18.989160+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":76,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:19.000220+0000","last_change":"2023-11-08T19:26:42.041259+0000","last_active":"2023-11-08T19:32:19.000220+0000","last_peered":"2023-11-08T19:32:19.000220+0000","last_clean":"2023-11-08T19:32:19.000220+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:19.000220+0000","last_undegraded":"2023-11-08T19:32:19.000220+0000","last_fullsized":"2023-11-08T19:32:19.000220+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.098887+0000","last_change":"2023-11-08T19:27:47.805406+0000","last_active":"2023-11-08T19:32:17.098887+0000","last_peered":"2023-11-08T19:32:17.098887+0000","last_clean":"2023-11-08T19:32:17.098887+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:17.098887+0000","last_undegraded":"2023-11-08T19:32:17.098887+0000","last_fullsized":"2023-11-08T19:32:17.098887+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.098608+0000","last_change":"2023-11-08T19:27:49.252418+0000","last_active":"2023-11-08T19:32:17.098608+0000","last_peered":"2023-11-08T19:32:17.098608+0000","last_clean":"2023-11-08T19:32:17.098608+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:17.098608+0000","last_undegraded":"2023-11-08T19:32:17.098608+0000","last_fullsized":"2023-11-08T19:32:17.098608+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":76,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:19.001714+0000","last_change":"2023-11-08T19:26:42.043129+0000","last_active":"2023-11-08T19:32:19.001714+0000","last_peered":"2023-11-08T19:32:19.001714+0000","last_clean":"2023-11-08T19:32:19.001714+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:19.001714+0000","last_undegraded":"2023-11-08T19:32:19.001714+0000","last_fullsized":"2023-11-08T19:32:19.001714+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:21.804522+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:32:21.804522+0000","last_peered":"2023-11-08T19:32:21.804522+0000","last_clean":"2023-11-08T19:32:21.804522+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:21.804522+0000","last_undegraded":"2023-11-08T19:32:21.804522+0000","last_fullsized":"2023-11-08T19:32:21.804522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":53,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.813629+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:32:16.813629+0000","last_peered":"2023-11-08T19:32:16.813629+0000","last_clean":"2023-11-08T19:32:16.813629+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:16.813629+0000","last_undegraded":"2023-11-08T19:32:16.813629+0000","last_fullsized":"2023-11-08T19:32:16.813629+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:37.316637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":77,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.832604+0000","last_change":"2023-11-08T19:26:41.840848+0000","last_active":"2023-11-08T19:32:18.832604+0000","last_peered":"2023-11-08T19:32:18.832604+0000","last_clean":"2023-11-08T19:32:18.832604+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:18.832604+0000","last_undegraded":"2023-11-08T19:32:18.832604+0000","last_fullsized":"2023-11-08T19:32:18.832604+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":246,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-08T19:32:16.252520+0000","last_change":"2023-11-08T19:24:48.613674+0000","last_active":"2023-11-08T19:32:16.252520+0000","last_peered":"2023-11-08T19:32:16.252520+0000","last_clean":"2023-11-08T19:32:16.252520+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:16.252520+0000","last_undegraded":"2023-11-08T19:32:16.252520+0000","last_fullsized":"2023-11-08T19:32:16.252520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:24:46.987711+0000","last_clean_scrub_stamp":"2023-11-08T19:24:46.987711+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:49:54.469110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574944,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33080,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338760,"statfs":{"total":96636764160,"available":96602890240,"internally_reserved":0,"allocated":1945600,"data_stored":1281536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.77500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.599,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.252},"min":{"1min":0.72799999999999998,"5min":0.58599999999999997,"15min":0.58499999999999996},"max":{"1min":1.6619999999999999,"5min":2.1360000000000001,"15min":4.4589999999999996},"last":0.94699999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2010000000000001,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.548,"5min":1.728,"15min":4.5330000000000004},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.1910000000000001,"15min":1.1459999999999999},"min":{"1min":0.83299999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6160000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":1.0569999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.335,"15min":1.448},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5660000000000001,"5min":1.891,"15min":4.6980000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.28,"5min":1.194,"15min":1.3160000000000001},"min":{"1min":0.97999999999999998,"5min":0.67200000000000004,"15min":0.64700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":4.4269999999999996},"last":0.88600000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.294,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.7889999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":1.292},{"interface":"front","average":{"1min":1.383,"5min":1.409,"15min":1.5149999999999999},"min":{"1min":0.95199999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.768,"5min":2.2200000000000002,"15min":4.6150000000000002},"last":1.242}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.327},"min":{"1min":1.006,"5min":0.624,"15min":0.624},"max":{"1min":1.49,"5min":1.798,"15min":2.3170000000000002},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.335,"5min":1.323,"15min":1.4410000000000001},"min":{"1min":0.93200000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7909999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.4099999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.95999999999999996,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.673,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.363,"15min":1.4470000000000001},"min":{"1min":1.048,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.649,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.621}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":1.22,"5min":0.878,"15min":0.878},"max":{"1min":1.8109999999999999,"5min":1.9770000000000001,"15min":2.427},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.474,"15min":1.581},"min":{"1min":1.1180000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.98,"5min":2.1920000000000002,"15min":2.468},"last":1.661}]}]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32508,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339332,"statfs":{"total":96636764160,"available":96603475968,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.0940000000000001,"15min":0.98799999999999999},"min":{"1min":0.85399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.536,"5min":1.6279999999999999,"15min":1.752},"last":0.73699999999999999},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.202,"15min":1.1399999999999999},"min":{"1min":0.80200000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":8.391,"5min":8.391,"15min":8.391},"last":1.0580000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.161,"15min":1.113},"min":{"1min":0.85599999999999998,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.679},"last":0.876},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2050000000000001,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.665,"5min":1.665,"15min":4.6399999999999997},"last":1.155}]},{"osd":2,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.80300000000000005,"5min":0.68799999999999994,"15min":0.54600000000000004},"max":{"1min":1.4830000000000001,"5min":1.655,"15min":4.609},"last":0.68100000000000005},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2949999999999999,"15min":1.242},"min":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.861},"last":0.98199999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.276,"15min":1.4239999999999999},"min":{"1min":1.0209999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.756,"15min":4.7190000000000003},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.256,"15min":1.2050000000000001},"min":{"1min":0.81200000000000006,"5min":0.625,"15min":0.625},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.784},"last":0.79300000000000004}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.83199999999999996,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.859,"5min":1.881,"15min":2.0449999999999999},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4770000000000001,"15min":1.4910000000000001},"min":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.998,"15min":2.3130000000000002},"last":1.319}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.325,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.611,"5min":1.835,"15min":1.9079999999999999},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3999999999999999,"15min":1.353},"min":{"1min":0.92100000000000004,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":1.6819999999999999,"5min":1.8660000000000001,"15min":2.0489999999999999},"last":1.121}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5569999999999999,"15min":1.55},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":1.99,"5min":1.99,"15min":2.2519999999999998},"last":1.371},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4970000000000001,"15min":1.464},"min":{"1min":0.75700000000000001,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":1.7809999999999999,"5min":1.9870000000000001,"15min":3.0640000000000001},"last":1.0329999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574943,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1970000000000001,"15min":1.117},"min":{"1min":0.69599999999999995,"5min":0.59999999999999998,"15min":0.57699999999999996},"max":{"1min":1.603,"5min":1.724,"15min":1.76},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.274,"15min":1.252},"min":{"1min":0.77600000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.837,"5min":1.954,"15min":1.954},"last":1.056}]},{"osd":1,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.754,"5min":0.753,"15min":0.58699999999999997},"max":{"1min":1.552,"5min":1.6359999999999999,"15min":2.3039999999999998},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.179,"5min":1.1679999999999999,"15min":1.355},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.3}]},{"osd":2,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2350000000000001,"15min":1.194},"min":{"1min":0.82599999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5720000000000001,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.173,"15min":1.3959999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6990000000000001,"5min":1.806,"15min":7.6479999999999997},"last":0.87}]},{"osd":3,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.74,"5min":1.804,"15min":2.1920000000000002},"last":1.155},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3500000000000001,"15min":1.323},"min":{"1min":0.98799999999999999,"5min":0.91300000000000003,"15min":0.69699999999999995},"max":{"1min":1.6910000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.5169999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.196,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7350000000000001,"5min":1.7969999999999999,"15min":2.347},"last":1.341},{"interface":"front","average":{"1min":1.337,"5min":1.347,"15min":1.488},"min":{"1min":0.94099999999999995,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.7350000000000001,"5min":2.097,"15min":3.153},"last":0.92300000000000004}]},{"osd":6,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3859999999999999,"15min":1.4830000000000001},"min":{"1min":0.92100000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.923,"5min":1.923,"15min":2.8650000000000002},"last":1.393},{"interface":"front","average":{"1min":1.399,"5min":1.4019999999999999,"15min":1.47},"min":{"1min":0.94199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6990000000000001,"5min":1.899,"15min":2.6190000000000002},"last":1.2669999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.6359999999999999},"min":{"1min":1.0980000000000001,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":1.9259999999999999,"5min":2.0630000000000002,"15min":3.2730000000000001},"last":1.244},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5229999999999999,"15min":1.5680000000000001},"min":{"1min":1.204,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.8600000000000001,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.466}]}]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1479999999999999,"15min":1.0760000000000001},"min":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.5900000000000001,"5min":1.782,"15min":2.5219999999999998},"last":1.099},{"interface":"front","average":{"1min":1.206,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.83499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.524,"5min":1.696,"15min":7.0119999999999996},"last":1.397}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.0680000000000001,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.63100000000000001},"max":{"1min":1.752,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.579},{"interface":"front","average":{"1min":1.177,"5min":1.1499999999999999,"15min":1.1519999999999999},"min":{"1min":0.748,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.7310000000000001,"5min":1.7450000000000001,"15min":2.9940000000000002},"last":1.2150000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2490000000000001,"15min":1.3160000000000001},"min":{"1min":0.98199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.988,"5min":1.988,"15min":4.0410000000000004},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.319,"15min":1.3879999999999999},"min":{"1min":0.95999999999999996,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":3.2069999999999999},"last":1.446}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1659999999999999,"15min":1.258},"min":{"1min":0.84199999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5169999999999999,"5min":1.696,"15min":4.4980000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.345},"min":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":1.738,"15min":3.5710000000000002},"last":1.714}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.4330000000000001},"min":{"1min":0.81000000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.593,"5min":1.762,"15min":4.2350000000000003},"last":1.357},{"interface":"front","average":{"1min":1.268,"5min":1.2549999999999999,"15min":1.4590000000000001},"min":{"1min":1.0189999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5800000000000001,"5min":1.8560000000000001,"15min":4.5880000000000001},"last":1.5489999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.365,"15min":1.5009999999999999},"min":{"1min":0.92300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":2.2130000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4419999999999999,"15min":1.5660000000000001},"min":{"1min":1.1639999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.841,"5min":1.9490000000000001,"15min":4.4130000000000003},"last":1.611}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.409,"15min":1.5640000000000001},"min":{"1min":1.137,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":4.6369999999999996},"last":1.6739999999999999},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5669999999999999,"15min":1.6799999999999999},"min":{"1min":1.2969999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8999999999999999,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.7809999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32500,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339340,"statfs":{"total":96636764160,"available":96603484160,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.99399999999999999},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.427,"5min":2.2320000000000002,"15min":3.972},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93500000000000005,"15min":1.042},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.311},"max":{"1min":1.544,"5min":1.899,"15min":3.1520000000000001},"last":0.94499999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95299999999999996,"15min":1.1160000000000001},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.4299999999999999,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":1.03},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":1.0429999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8520000000000001,"15min":4.5750000000000002},"last":1.0740000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.153,"15min":1.272},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.619,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1020000000000001,"15min":1.218},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.6339999999999999,"5min":1.754,"15min":2.73},"last":1.202}]},{"osd":4,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.214,"15min":1.3779999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6599999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.24,"15min":1.3899999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9039999999999999,"5min":2.0449999999999999,"15min":3.052},"last":1.272}]},{"osd":5,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.4370000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":2.0430000000000001,"15min":3.0899999999999999},"last":1.381},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.29,"15min":1.4570000000000001},"min":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.597,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.4930000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2769999999999999,"15min":1.3819999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7,"5min":3.21,"15min":4.2220000000000004},"last":1.728},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.391,"15min":1.526},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.0880000000000001,"15min":3.1230000000000002},"last":1.323}]},{"osd":7,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.399,"15min":1.6160000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.925,"5min":2.1150000000000002,"15min":4.407},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.5209999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.671,"5min":2.0059999999999998,"15min":4.3849999999999998},"last":1.792}]}]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1347584,"data_stored":690149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83699999999999997,"15min":0.85399999999999998},"min":{"1min":0.61699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2370000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":1.04},{"interface":"front","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.81699999999999995}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.56799999999999995,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.371,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":1.1040000000000001},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.66700000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.474,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.093,"15min":1.0620000000000001},"min":{"1min":0.80300000000000005,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.052,"15min":1.103},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.675,"5min":1.7330000000000001,"15min":1.7669999999999999},"last":0.64300000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2829999999999999,"15min":1.365},"min":{"1min":0.95499999999999996,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":1.883,"15min":3.2959999999999998},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.284,"15min":1.3400000000000001},"min":{"1min":1.0609999999999999,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":1.704,"15min":2.278},"last":1.179}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.325,"15min":1.3899999999999999},"min":{"1min":1.1359999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.327,"15min":1.4179999999999999},"min":{"1min":1.048,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.7709999999999999,"5min":1.954,"15min":2.536},"last":1.4470000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":1.1120000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.351,"15min":1.421},"min":{"1min":0.92200000000000004,"5min":0.83799999999999997,"15min":0.70099999999999996},"max":{"1min":1.9810000000000001,"5min":2.036,"15min":2.0910000000000002},"last":1.3640000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.409,"15min":1.522},"min":{"1min":0.95099999999999996,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":1.74,"5min":1.9339999999999999,"15min":2.6549999999999998},"last":1.145},{"interface":"front","average":{"1min":1.516,"5min":1.466,"15min":1.5269999999999999},"min":{"1min":1.111,"5min":0.84999999999999998,"15min":0.76600000000000001},"max":{"1min":1.8009999999999999,"5min":2.226,"15min":2.3439999999999999},"last":1.4930000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85599999999999998,"15min":0.78000000000000003},"min":{"1min":0.59199999999999997,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.284,"5min":1.327,"15min":1.327},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.88800000000000001,"15min":0.79900000000000004},"min":{"1min":0.82499999999999996,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":1.528,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.60299999999999998}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84099999999999997,"15min":0.81100000000000005},"min":{"1min":0.51100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.256,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.504},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0249999999999999,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.47499999999999998,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2569999999999999,"15min":1.1240000000000001},"min":{"1min":0.80100000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.748,"5min":1.978,"15min":1.978},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.196,"5min":1.1439999999999999,"15min":1.1020000000000001},"min":{"1min":0.65300000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.6930000000000001,"5min":2.2690000000000001,"15min":2.5369999999999999},"last":0.68899999999999995}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.157},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.476,"5min":1.8620000000000001,"15min":2.165},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.294,"5min":1.2729999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.6870000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.1000000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.93799999999999994,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.046},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2689999999999999,"15min":1.3069999999999999},"min":{"1min":0.95199999999999996,"5min":0.73199999999999998,"15min":0.629},"max":{"1min":1.696,"5min":1.748,"15min":2.4089999999999998},"last":0.93899999999999995}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.375},"min":{"1min":1.194,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.7450000000000001,"5min":2.1509999999999998,"15min":2.6019999999999999},"last":1.129},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.355,"15min":1.373},"min":{"1min":1.133,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.282},"last":1.085}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.3560000000000001},"min":{"1min":1.0029999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1},{"interface":"front","average":{"1min":1.454,"5min":1.409,"15min":1.395},"min":{"1min":1.0940000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.865,"5min":2.4830000000000001,"15min":2.6699999999999999},"last":0.95799999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1941504,"data_stored":1280517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.63200000000000001,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.0780000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2110000000000001,"5min":1.4770000000000001,"15min":1.7},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.876},"min":{"1min":0.57399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.319,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.998,"15min":0.98299999999999998},"min":{"1min":0.76200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5189999999999999,"5min":1.54,"15min":1.7270000000000001},"last":0.88800000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.016},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.748},"last":1.149},{"interface":"front","average":{"1min":1.089,"5min":1.0660000000000001,"15min":1.0940000000000001},"min":{"1min":0.60599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":1.8400000000000001,"15min":2.496},"last":1.0069999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2709999999999999,"15min":1.244},"min":{"1min":0.90600000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6180000000000001,"5min":1.794,"15min":1.9239999999999999},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.29,"15min":1.3939999999999999},"min":{"1min":1.127,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.6399999999999999,"5min":1.706,"15min":2.4039999999999999},"last":1.4670000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.306,"15min":1.351},"min":{"1min":1.042,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7270000000000001,"5min":1.8620000000000001,"15min":2.4369999999999998},"last":1.075},{"interface":"front","average":{"1min":1.319,"5min":1.2949999999999999,"15min":1.3979999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.569,"5min":2.089,"15min":2.4239999999999999},"last":1.101}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3500000000000001,"15min":1.4550000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.357,"15min":1.3959999999999999},"min":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7549999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.254}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.93400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.851,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.415,"15min":1.462},"min":{"1min":1.103,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7909999999999999,"5min":1.881,"15min":2.306},"last":1.391}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:29.690 INFO:tasks.ceph:pgid 13.17 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.690 INFO:tasks.ceph:pgid 12.16 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.691 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.691 INFO:tasks.ceph:pgid 13.16 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.691 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.692 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.692 INFO:tasks.ceph:pgid 13.15 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.693 INFO:tasks.ceph:pgid 12.14 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.693 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.693 INFO:tasks.ceph:pgid 13.14 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.694 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.694 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.694 INFO:tasks.ceph:pgid 13.13 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.695 INFO:tasks.ceph:pgid 12.12 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.695 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.696 INFO:tasks.ceph:pgid 13.12 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.696 INFO:tasks.ceph:pgid 12.13 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.696 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.697 INFO:tasks.ceph:pgid 13.11 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.697 INFO:tasks.ceph:pgid 12.10 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.697 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.698 INFO:tasks.ceph:pgid 13.10 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.698 INFO:tasks.ceph:pgid 12.11 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.699 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.699 INFO:tasks.ceph:pgid 13.1f last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.699 INFO:tasks.ceph:pgid 12.1e last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.700 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.700 INFO:tasks.ceph:pgid 13.1e last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.700 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.701 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.701 INFO:tasks.ceph:pgid 13.1d last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.702 INFO:tasks.ceph:pgid 12.1c last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.702 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.702 INFO:tasks.ceph:pgid 13.1c last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.703 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.703 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.704 INFO:tasks.ceph:pgid 13.1b last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.704 INFO:tasks.ceph:pgid 12.1a last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.704 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.705 INFO:tasks.ceph:pgid 13.1a last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.705 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.705 INFO:tasks.ceph:pgid 13.19 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.706 INFO:tasks.ceph:pgid 12.18 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.706 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.707 INFO:tasks.ceph:pgid 13.18 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.707 INFO:tasks.ceph:pgid 12.19 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.708 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.708 INFO:tasks.ceph:pgid 13.7 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.708 INFO:tasks.ceph:pgid 12.6 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.709 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.709 INFO:tasks.ceph:pgid 13.6 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.709 INFO:tasks.ceph:pgid 12.7 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.710 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.710 INFO:tasks.ceph:pgid 13.5 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.711 INFO:tasks.ceph:pgid 12.4 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.711 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.711 INFO:tasks.ceph:pgid 13.4 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.712 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.712 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.713 INFO:tasks.ceph:pgid 13.3 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.713 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.713 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.714 INFO:tasks.ceph:pgid 13.2 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.714 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.715 INFO:tasks.ceph:pgid 13.1 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.715 INFO:tasks.ceph:pgid 12.0 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.715 INFO:tasks.ceph:pgid 13.0 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.716 INFO:tasks.ceph:pgid 12.1 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.716 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.717 INFO:tasks.ceph:pgid 13.f last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.717 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.717 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.718 INFO:tasks.ceph:pgid 13.e last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.718 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.718 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.719 INFO:tasks.ceph:pgid 13.d last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.719 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.720 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.720 INFO:tasks.ceph:pgid 13.b last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.720 INFO:tasks.ceph:pgid 12.a last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.721 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.721 INFO:tasks.ceph:pgid 13.a last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.722 INFO:tasks.ceph:pgid 12.b last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.722 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.722 INFO:tasks.ceph:pgid 13.9 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.723 INFO:tasks.ceph:pgid 12.8 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.723 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.724 INFO:tasks.ceph:pgid 13.8 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.724 INFO:tasks.ceph:pgid 12.9 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.724 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.725 INFO:tasks.ceph:pgid 13.c last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.725 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.726 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.726 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-08T19:24:46.987711+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=24, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:29.726 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T19:32:49.728 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:32:49.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.958+0000 7fbe436e1640 1 Processor -- start 2023-11-08T19:32:49.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.958+0000 7fbe436e1640 1 -- start start 2023-11-08T19:32:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.960+0000 7fbe436e1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbe3c06a970 0x7fbe3c06cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.960+0000 7fbe436e1640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe3c0c9150 0x7fbe3c0cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.960+0000 7fbe436e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c1653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.960+0000 7fbe436e1640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c1619b0 con 0x7fbe3c0c9150 2023-11-08T19:32:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.960+0000 7fbe436e1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cd0a0 con 0x7fbe3c06a970 2023-11-08T19:32:49.964 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.960+0000 7fbe436e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06d4b0 con 0x7fbe3c0ccc10 2023-11-08T19:32:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe3c0c9150 0x7fbe3c0cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7fbe3c0c9150 0x7fbe3c0cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52716/0 (socket says 172.21.15.119:52716) 2023-11-08T19:32:49.965 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 -- 172.21.15.119:0/1746752020 learned_addr learned my addr 172.21.15.119:0/1746752020 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe4147d640 1 --2- 172.21.15.119:0/1746752020 >> v2:172.21.15.188:3300/0 conn(0x7fbe3c06a970 0x7fbe3c06cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe41c7e640 1 --2- 172.21.15.119:0/1746752020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c1653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.966 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 -- 172.21.15.119:0/1746752020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 msgr2=0x7fbe3c1653f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/1746752020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c1653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 -- 172.21.15.119:0/1746752020 >> v2:172.21.15.188:3300/0 conn(0x7fbe3c06a970 msgr2=0x7fbe3c06cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:32:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/1746752020 >> v2:172.21.15.188:3300/0 conn(0x7fbe3c06a970 0x7fbe3c06cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.967 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.961+0000 7fbe40c7c640 1 -- 172.21.15.119:0/1746752020 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c147b80 con 0x7fbe3c0c9150 2023-11-08T19:32:49.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.962+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/1746752020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c0cc6d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6452c5775d6780 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.962+0000 7fbe33fff640 1 -- 172.21.15.119:0/1746752020 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0317c0 con 0x7fbe3c0c9150 2023-11-08T19:32:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe33fff640 1 -- 172.21.15.119:0/1746752020 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c031920 con 0x7fbe3c0c9150 2023-11-08T19:32:49.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe33fff640 1 -- 172.21.15.119:0/1746752020 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c031cb0 con 0x7fbe3c0c9150 2023-11-08T19:32:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 -- 172.21.15.119:0/1746752020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0c9150 msgr2=0x7fbe3c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 --2- 172.21.15.119:0/1746752020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c0cc6d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009870 tx=0x7fbe2c02b860 comp rx=0 tx=0).stop 2023-11-08T19:32:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 -- 172.21.15.119:0/1746752020 shutdown_connections 2023-11-08T19:32:49.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 --2- 172.21.15.119:0/1746752020 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c1653f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 --2- 172.21.15.119:0/1746752020 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c0cc6d0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 --2- 172.21.15.119:0/1746752020 >> v2:172.21.15.188:3300/0 conn(0x7fbe3c06a970 0x7fbe3c06cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 -- 172.21.15.119:0/1746752020 >> 172.21.15.119:0/1746752020 conn(0x7fbe3c0cb610 msgr2=0x7fbe3c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 -- 172.21.15.119:0/1746752020 shutdown_connections 2023-11-08T19:32:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.963+0000 7fbe436e1640 1 -- 172.21.15.119:0/1746752020 wait complete. 2023-11-08T19:32:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.965+0000 7fbe436e1640 1 Processor -- start 2023-11-08T19:32:49.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.965+0000 7fbe436e1640 1 -- start start 2023-11-08T19:32:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe436e1640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 0x7fbe3c1f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe436e1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe436e1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c0b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe436e1640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0b1680 con 0x7fbe3c06a970 2023-11-08T19:32:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe436e1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1493d0 con 0x7fbe3c0c9150 2023-11-08T19:32:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe436e1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1462d0 con 0x7fbe3c0ccc10 2023-11-08T19:32:49.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe4147d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 0x7fbe3c1f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe4147d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 0x7fbe3c1f7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:52728/0 (socket says 172.21.15.119:52728) 2023-11-08T19:32:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.966+0000 7fbe4147d640 1 -- 172.21.15.119:0/599333810 learned_addr learned my addr 172.21.15.119:0/599333810 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:32:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe41c7e640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c0b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe4147d640 1 -- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 msgr2=0x7fbe3c0b0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe4147d640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c0b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe4147d640 1 -- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe3c0c9150 msgr2=0x7fbe3c145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe4147d640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe4147d640 1 -- 172.21.15.119:0/599333810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c0094d0 con 0x7fbe3c06a970 2023-11-08T19:32:49.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:32:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe4147d640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 0x7fbe3c1f7f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800d800 tx=0x7fbe3800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800b6c0 con 0x7fbe3c06a970 2023-11-08T19:32:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3800b820 con 0x7fbe3c06a970 2023-11-08T19:32:49.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.967+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3800bbb0 con 0x7fbe3c06a970 2023-11-08T19:32:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.968+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c149670 con 0x7fbe3c06a970 2023-11-08T19:32:49.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.968+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c149c30 con 0x7fbe3c06a970 2023-11-08T19:32:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.969+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c074430 con 0x7fbe3c06a970 2023-11-08T19:32:49.982 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.970+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe3800bd10 con 0x7fbe3c06a970 2023-11-08T19:32:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.970+0000 7fbe31ffb640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe2006ae70 0x7fbe2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:32:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.971+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fbe3808a0d0 con 0x7fbe3c06a970 2023-11-08T19:32:49.983 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.971+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe2006ae70 0x7fbe2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:32:49.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.971+0000 7fbe40c7c640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe2006ae70 0x7fbe2006d330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c036000 tx=0x7fbe2c03d040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:32:49.984 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:49.975+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe38054c60 con 0x7fbe3c06a970 2023-11-08T19:32:50.164 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.162+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe3c1472e0 con 0x7fbe2006ae70 2023-11-08T19:32:50.173 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.171+0000 7fbe31ffb640 1 -- 172.21.15.119:0/599333810 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+290071 (secure 0 0 0) 0x7fbe2c031b10 con 0x7fbe2006ae70 2023-11-08T19:32:50.177 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:32:50.179 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:32:50.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe2006ae70 msgr2=0x7fbe2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:50.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe2006ae70 0x7fbe2006d330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c036000 tx=0x7fbe2c03d040 comp rx=0 tx=0).stop 2023-11-08T19:32:50.181 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 msgr2=0x7fbe3c1f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:32:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 0x7fbe3c1f7f30 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fbe3800d800 tx=0x7fbe3800dcc0 comp rx=0 tx=0).stop 2023-11-08T19:32:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 shutdown_connections 2023-11-08T19:32:50.182 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbe2006ae70 0x7fbe2006d330 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbe3c0ccc10 0x7fbe3c0b0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbe3c0c9150 0x7fbe3c145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 --2- 172.21.15.119:0/599333810 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbe3c06a970 0x7fbe3c1f7f30 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:32:50.183 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 >> 172.21.15.119:0/599333810 conn(0x7fbe3c0cb610 msgr2=0x7fbe3c06f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:32:50.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 shutdown_connections 2023-11-08T19:32:50.184 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:32:50.178+0000 7fbe436e1640 1 -- 172.21.15.119:0/599333810 wait complete. 2023-11-08T19:32:50.200 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2023-11-08T19:32:48.851018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261120,"kb_used_data":11712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713600,"statfs":{"total":773094113280,"available":772826726400,"internally_reserved":0,"allocated":11993088,"data_stored":6708042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004894"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.525576+0000","last_change":"2023-11-08T19:32:46.525623+0000","last_active":"2023-11-08T19:32:46.525576+0000","last_peered":"2023-11-08T19:32:46.525576+0000","last_clean":"2023-11-08T19:32:46.525576+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:46.525576+0000","last_undegraded":"2023-11-08T19:32:46.525576+0000","last_fullsized":"2023-11-08T19:32:46.525576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.525310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.525310+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.525310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:58:08.797703+0000","scrub_duration":0.0036410370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:32.935524+0000","last_change":"2023-11-08T19:32:32.935524+0000","last_active":"2023-11-08T19:32:32.935524+0000","last_peered":"2023-11-08T19:32:32.935524+0000","last_clean":"2023-11-08T19:32:32.935524+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:32.935524+0000","last_undegraded":"2023-11-08T19:32:32.935524+0000","last_fullsized":"2023-11-08T19:32:32.935524+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:32.935290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:32.935290+0000","last_clean_scrub_stamp":"2023-11-08T19:32:32.935290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:23.985335+0000","scrub_duration":0.0032534619999999999,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.771234+0000","last_change":"2023-11-08T19:32:29.771285+0000","last_active":"2023-11-08T19:32:29.771234+0000","last_peered":"2023-11-08T19:32:29.771234+0000","last_clean":"2023-11-08T19:32:29.771234+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:29.771234+0000","last_undegraded":"2023-11-08T19:32:29.771234+0000","last_fullsized":"2023-11-08T19:32:29.771234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.771060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.771060+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.771060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:57:45.945310+0000","scrub_duration":0.0045387559999999997,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.343466+0000","last_change":"2023-11-08T19:27:47.801983+0000","last_active":"2023-11-08T19:32:26.343466+0000","last_peered":"2023-11-08T19:32:26.343466+0000","last_clean":"2023-11-08T19:32:26.343466+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:26.343466+0000","last_undegraded":"2023-11-08T19:32:26.343466+0000","last_fullsized":"2023-11-08T19:32:26.343466+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.468282+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:32:28.468282+0000","last_peered":"2023-11-08T19:32:28.468282+0000","last_clean":"2023-11-08T19:32:28.468282+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:32:28.468282+0000","last_undegraded":"2023-11-08T19:32:28.468282+0000","last_fullsized":"2023-11-08T19:32:28.468282+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.652067+0000","last_change":"2023-11-08T19:32:28.652115+0000","last_active":"2023-11-08T19:32:28.652067+0000","last_peered":"2023-11-08T19:32:28.652067+0000","last_clean":"2023-11-08T19:32:28.652067+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:28.652067+0000","last_undegraded":"2023-11-08T19:32:28.652067+0000","last_fullsized":"2023-11-08T19:32:28.652067+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.651902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.651902+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.651902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:39:39.497609+0000","scrub_duration":0.0036776040000000001,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.046809+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:32:27.046809+0000","last_peered":"2023-11-08T19:32:27.046809+0000","last_clean":"2023-11-08T19:32:27.046809+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:32:27.046809+0000","last_undegraded":"2023-11-08T19:32:27.046809+0000","last_fullsized":"2023-11-08T19:32:27.046809+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":66,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:38.643087+0000","last_change":"2023-11-08T19:32:38.643143+0000","last_active":"2023-11-08T19:32:38.643087+0000","last_peered":"2023-11-08T19:32:38.643087+0000","last_clean":"2023-11-08T19:32:38.643087+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:38.643087+0000","last_undegraded":"2023-11-08T19:32:38.643087+0000","last_fullsized":"2023-11-08T19:32:38.643087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:38.642935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:38.642935+0000","last_clean_scrub_stamp":"2023-11-08T19:32:38.642935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:19.775448+0000","scrub_duration":0.0035460359999999998,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.110153+0000","last_change":"2023-11-08T19:26:41.731909+0000","last_active":"2023-11-08T19:32:17.110153+0000","last_peered":"2023-11-08T19:32:17.110153+0000","last_clean":"2023-11-08T19:32:17.110153+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:17.110153+0000","last_undegraded":"2023-11-08T19:32:17.110153+0000","last_fullsized":"2023-11-08T19:32:17.110153+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.209144+0000","last_change":"2023-11-08T19:27:47.810846+0000","last_active":"2023-11-08T19:32:24.209144+0000","last_peered":"2023-11-08T19:32:24.209144+0000","last_clean":"2023-11-08T19:32:24.209144+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:24.209144+0000","last_undegraded":"2023-11-08T19:32:24.209144+0000","last_fullsized":"2023-11-08T19:32:24.209144+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.468194+0000","last_change":"2023-11-08T19:27:49.258272+0000","last_active":"2023-11-08T19:32:28.468194+0000","last_peered":"2023-11-08T19:32:28.468194+0000","last_clean":"2023-11-08T19:32:28.468194+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:32:28.468194+0000","last_undegraded":"2023-11-08T19:32:28.468194+0000","last_fullsized":"2023-11-08T19:32:28.468194+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.727558+0000","last_change":"2023-11-08T19:32:27.727558+0000","last_active":"2023-11-08T19:32:27.727558+0000","last_peered":"2023-11-08T19:32:27.727558+0000","last_clean":"2023-11-08T19:32:27.727558+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:27.727558+0000","last_undegraded":"2023-11-08T19:32:27.727558+0000","last_fullsized":"2023-11-08T19:32:27.727558+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:27.727335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:27.727335+0000","last_clean_scrub_stamp":"2023-11-08T19:32:27.727335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:55.266502+0000","scrub_duration":0.0027330219999999999,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.191069+0000","last_change":"2023-11-08T19:27:47.812037+0000","last_active":"2023-11-08T19:32:29.191069+0000","last_peered":"2023-11-08T19:32:29.191069+0000","last_clean":"2023-11-08T19:32:29.191069+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:29.191069+0000","last_undegraded":"2023-11-08T19:32:29.191069+0000","last_fullsized":"2023-11-08T19:32:29.191069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.625497+0000","last_change":"2023-11-08T19:32:45.625543+0000","last_active":"2023-11-08T19:32:45.625497+0000","last_peered":"2023-11-08T19:32:45.625497+0000","last_clean":"2023-11-08T19:32:45.625497+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:45.625497+0000","last_undegraded":"2023-11-08T19:32:45.625497+0000","last_fullsized":"2023-11-08T19:32:45.625497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.625348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.625348+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.625348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:33.970967+0000","scrub_duration":0.0030484169999999999,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:42.920729+0000","last_change":"2023-11-08T19:32:42.920778+0000","last_active":"2023-11-08T19:32:42.920729+0000","last_peered":"2023-11-08T19:32:42.920729+0000","last_clean":"2023-11-08T19:32:42.920729+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:42.920729+0000","last_undegraded":"2023-11-08T19:32:42.920729+0000","last_fullsized":"2023-11-08T19:32:42.920729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:42.920564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:42.920564+0000","last_clean_scrub_stamp":"2023-11-08T19:32:42.920564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:15:38.876790+0000","scrub_duration":0.0033816290000000001,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.209084+0000","last_change":"2023-11-08T19:27:47.807339+0000","last_active":"2023-11-08T19:32:24.209084+0000","last_peered":"2023-11-08T19:32:24.209084+0000","last_clean":"2023-11-08T19:32:24.209084+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:32:24.209084+0000","last_undegraded":"2023-11-08T19:32:24.209084+0000","last_fullsized":"2023-11-08T19:32:24.209084+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.753355+0000","last_change":"2023-11-08T19:32:30.753400+0000","last_active":"2023-11-08T19:32:30.753355+0000","last_peered":"2023-11-08T19:32:30.753355+0000","last_clean":"2023-11-08T19:32:30.753355+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:30.753355+0000","last_undegraded":"2023-11-08T19:32:30.753355+0000","last_fullsized":"2023-11-08T19:32:30.753355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.753206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.753206+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.753206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:15:08.346214+0000","scrub_duration":0.0030208409999999998,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.821586+0000","last_change":"2023-11-08T19:32:34.821633+0000","last_active":"2023-11-08T19:32:34.821586+0000","last_peered":"2023-11-08T19:32:34.821586+0000","last_clean":"2023-11-08T19:32:34.821586+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:34.821586+0000","last_undegraded":"2023-11-08T19:32:34.821586+0000","last_fullsized":"2023-11-08T19:32:34.821586+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.821364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.821364+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.821364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:07.177396+0000","scrub_duration":0.0040781000000000003,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.703378+0000","last_change":"2023-11-08T19:32:45.703425+0000","last_active":"2023-11-08T19:32:45.703378+0000","last_peered":"2023-11-08T19:32:45.703378+0000","last_clean":"2023-11-08T19:32:45.703378+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:45.703378+0000","last_undegraded":"2023-11-08T19:32:45.703378+0000","last_fullsized":"2023-11-08T19:32:45.703378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.703215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.703215+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.703215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:29.552613+0000","scrub_duration":0.0041161319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.780034+0000","last_change":"2023-11-08T19:32:45.780081+0000","last_active":"2023-11-08T19:32:45.780034+0000","last_peered":"2023-11-08T19:32:45.780034+0000","last_clean":"2023-11-08T19:32:45.780034+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:45.780034+0000","last_undegraded":"2023-11-08T19:32:45.780034+0000","last_fullsized":"2023-11-08T19:32:45.780034+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.779844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.779844+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.779844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:03.630656+0000","scrub_duration":0.003794487,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.874214+0000","last_change":"2023-11-08T19:32:30.874260+0000","last_active":"2023-11-08T19:32:30.874214+0000","last_peered":"2023-11-08T19:32:30.874214+0000","last_clean":"2023-11-08T19:32:30.874214+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:30.874214+0000","last_undegraded":"2023-11-08T19:32:30.874214+0000","last_fullsized":"2023-11-08T19:32:30.874214+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.874063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.874063+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.874063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:33.344505+0000","scrub_duration":0.0032893110000000001,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:39.724808+0000","last_change":"2023-11-08T19:32:39.724860+0000","last_active":"2023-11-08T19:32:39.724808+0000","last_peered":"2023-11-08T19:32:39.724808+0000","last_clean":"2023-11-08T19:32:39.724808+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:39.724808+0000","last_undegraded":"2023-11-08T19:32:39.724808+0000","last_fullsized":"2023-11-08T19:32:39.724808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:39.724643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:39.724643+0000","last_clean_scrub_stamp":"2023-11-08T19:32:39.724643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:16.175462+0000","scrub_duration":0.0033653490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":66,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.588081+0000","last_change":"2023-11-08T19:32:35.588132+0000","last_active":"2023-11-08T19:32:35.588081+0000","last_peered":"2023-11-08T19:32:35.588081+0000","last_clean":"2023-11-08T19:32:35.588081+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:35.588081+0000","last_undegraded":"2023-11-08T19:32:35.588081+0000","last_fullsized":"2023-11-08T19:32:35.588081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.587924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.587924+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.587924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:11:25.638388+0000","scrub_duration":0.003687957,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.650654+0000","last_change":"2023-11-08T19:32:29.650701+0000","last_active":"2023-11-08T19:32:29.650654+0000","last_peered":"2023-11-08T19:32:29.650654+0000","last_clean":"2023-11-08T19:32:29.650654+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:29.650654+0000","last_undegraded":"2023-11-08T19:32:29.650654+0000","last_fullsized":"2023-11-08T19:32:29.650654+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.650428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.650428+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.650428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:44:57.186304+0000","scrub_duration":0.0035263999999999998,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.109947+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:32:17.109947+0000","last_peered":"2023-11-08T19:32:17.109947+0000","last_clean":"2023-11-08T19:32:17.109947+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:32:17.109947+0000","last_undegraded":"2023-11-08T19:32:17.109947+0000","last_fullsized":"2023-11-08T19:32:17.109947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.645920+0000","last_change":"2023-11-08T19:32:46.645975+0000","last_active":"2023-11-08T19:32:46.645920+0000","last_peered":"2023-11-08T19:32:46.645920+0000","last_clean":"2023-11-08T19:32:46.645920+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:46.645920+0000","last_undegraded":"2023-11-08T19:32:46.645920+0000","last_fullsized":"2023-11-08T19:32:46.645920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.645739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.645739+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.645739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:57.261846+0000","scrub_duration":0.0036408650000000001,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.559388+0000","last_change":"2023-11-08T19:32:41.559435+0000","last_active":"2023-11-08T19:32:41.559388+0000","last_peered":"2023-11-08T19:32:41.559388+0000","last_clean":"2023-11-08T19:32:41.559388+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:41.559388+0000","last_undegraded":"2023-11-08T19:32:41.559388+0000","last_fullsized":"2023-11-08T19:32:41.559388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.559226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.559226+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.559226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:42.827112+0000","scrub_duration":0.0037686489999999998,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.491581+0000","last_change":"2023-11-08T19:27:47.808716+0000","last_active":"2023-11-08T19:32:18.491581+0000","last_peered":"2023-11-08T19:32:18.491581+0000","last_clean":"2023-11-08T19:32:18.491581+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:32:18.491581+0000","last_undegraded":"2023-11-08T19:32:18.491581+0000","last_fullsized":"2023-11-08T19:32:18.491581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.107795+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:32:17.107795+0000","last_peered":"2023-11-08T19:32:17.107795+0000","last_clean":"2023-11-08T19:32:17.107795+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:32:17.107795+0000","last_undegraded":"2023-11-08T19:32:17.107795+0000","last_fullsized":"2023-11-08T19:32:17.107795+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.675611+0000","last_change":"2023-11-08T19:32:28.675655+0000","last_active":"2023-11-08T19:32:28.675611+0000","last_peered":"2023-11-08T19:32:28.675611+0000","last_clean":"2023-11-08T19:32:28.675611+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:28.675611+0000","last_undegraded":"2023-11-08T19:32:28.675611+0000","last_fullsized":"2023-11-08T19:32:28.675611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.675467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.675467+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.675467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:32:57.443158+0000","scrub_duration":0.00292019,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.951632+0000","last_change":"2023-11-08T19:27:47.815508+0000","last_active":"2023-11-08T19:32:18.951632+0000","last_peered":"2023-11-08T19:32:18.951632+0000","last_clean":"2023-11-08T19:32:18.951632+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:32:18.951632+0000","last_undegraded":"2023-11-08T19:32:18.951632+0000","last_fullsized":"2023-11-08T19:32:18.951632+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.780194+0000","last_change":"2023-11-08T19:32:46.780194+0000","last_active":"2023-11-08T19:32:46.780194+0000","last_peered":"2023-11-08T19:32:46.780194+0000","last_clean":"2023-11-08T19:32:46.780194+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:46.780194+0000","last_undegraded":"2023-11-08T19:32:46.780194+0000","last_fullsized":"2023-11-08T19:32:46.780194+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.779991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.779991+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.779991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:08.986106+0000","scrub_duration":0.0036491200000000001,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.589623+0000","last_change":"2023-11-08T19:32:44.589675+0000","last_active":"2023-11-08T19:32:44.589623+0000","last_peered":"2023-11-08T19:32:44.589623+0000","last_clean":"2023-11-08T19:32:44.589623+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:44.589623+0000","last_undegraded":"2023-11-08T19:32:44.589623+0000","last_fullsized":"2023-11-08T19:32:44.589623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.589412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.589412+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.589412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:03.472044+0000","scrub_duration":0.0032550370000000001,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.103193+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:32:17.103193+0000","last_peered":"2023-11-08T19:32:17.103193+0000","last_clean":"2023-11-08T19:32:17.103193+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:32:17.103193+0000","last_undegraded":"2023-11-08T19:32:17.103193+0000","last_fullsized":"2023-11-08T19:32:17.103193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.209008+0000","last_change":"2023-11-08T19:27:48.916252+0000","last_active":"2023-11-08T19:32:24.209008+0000","last_peered":"2023-11-08T19:32:24.209008+0000","last_clean":"2023-11-08T19:32:24.209008+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:24.209008+0000","last_undegraded":"2023-11-08T19:32:24.209008+0000","last_fullsized":"2023-11-08T19:32:24.209008+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.663302+0000","last_change":"2023-11-08T19:32:35.663353+0000","last_active":"2023-11-08T19:32:35.663302+0000","last_peered":"2023-11-08T19:32:35.663302+0000","last_clean":"2023-11-08T19:32:35.663302+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:35.663302+0000","last_undegraded":"2023-11-08T19:32:35.663302+0000","last_fullsized":"2023-11-08T19:32:35.663302+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.663149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.663149+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.663149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:08:01.980513+0000","scrub_duration":0.0040296780000000001,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.490994+0000","last_change":"2023-11-08T19:27:47.797783+0000","last_active":"2023-11-08T19:32:18.490994+0000","last_peered":"2023-11-08T19:32:18.490994+0000","last_clean":"2023-11-08T19:32:18.490994+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:18.490994+0000","last_undegraded":"2023-11-08T19:32:18.490994+0000","last_fullsized":"2023-11-08T19:32:18.490994+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.691774+0000","last_change":"2023-11-08T19:32:44.691774+0000","last_active":"2023-11-08T19:32:44.691774+0000","last_peered":"2023-11-08T19:32:44.691774+0000","last_clean":"2023-11-08T19:32:44.691774+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:44.691774+0000","last_undegraded":"2023-11-08T19:32:44.691774+0000","last_fullsized":"2023-11-08T19:32:44.691774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.691532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.691532+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.691532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:22.520571+0000","scrub_duration":0.0039118929999999996,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.500518+0000","last_change":"2023-11-08T19:32:34.500610+0000","last_active":"2023-11-08T19:32:34.500518+0000","last_peered":"2023-11-08T19:32:34.500518+0000","last_clean":"2023-11-08T19:32:34.500518+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:34.500518+0000","last_undegraded":"2023-11-08T19:32:34.500518+0000","last_fullsized":"2023-11-08T19:32:34.500518+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.500295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.500295+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.500295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:07:20.761692+0000","scrub_duration":0.0032186850000000002,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.990685+0000","last_change":"2023-11-08T19:27:47.802132+0000","last_active":"2023-11-08T19:32:18.990685+0000","last_peered":"2023-11-08T19:32:18.990685+0000","last_clean":"2023-11-08T19:32:18.990685+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:18.990685+0000","last_undegraded":"2023-11-08T19:32:18.990685+0000","last_fullsized":"2023-11-08T19:32:18.990685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.046523+0000","last_change":"2023-11-08T19:27:48.791706+0000","last_active":"2023-11-08T19:32:27.046523+0000","last_peered":"2023-11-08T19:32:27.046523+0000","last_clean":"2023-11-08T19:32:27.046523+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:27.046523+0000","last_undegraded":"2023-11-08T19:32:27.046523+0000","last_fullsized":"2023-11-08T19:32:27.046523+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.631351+0000","last_change":"2023-11-08T19:32:26.631371+0000","last_active":"2023-11-08T19:32:26.631351+0000","last_peered":"2023-11-08T19:32:26.631351+0000","last_clean":"2023-11-08T19:32:26.631351+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:26.631351+0000","last_undegraded":"2023-11-08T19:32:26.631351+0000","last_fullsized":"2023-11-08T19:32:26.631351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.631277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:31:19.710068+0000","scrub_duration":0.0020387719999999999,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":46,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.343490+0000","last_change":"2023-11-08T19:27:47.812525+0000","last_active":"2023-11-08T19:32:26.343490+0000","last_peered":"2023-11-08T19:32:26.343490+0000","last_clean":"2023-11-08T19:32:26.343490+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:26.343490+0000","last_undegraded":"2023-11-08T19:32:26.343490+0000","last_fullsized":"2023-11-08T19:32:26.343490+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:32.749081+0000","last_change":"2023-11-08T19:32:32.749081+0000","last_active":"2023-11-08T19:32:32.749081+0000","last_peered":"2023-11-08T19:32:32.749081+0000","last_clean":"2023-11-08T19:32:32.749081+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:32.749081+0000","last_undegraded":"2023-11-08T19:32:32.749081+0000","last_fullsized":"2023-11-08T19:32:32.749081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:32.748790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:32.748790+0000","last_clean_scrub_stamp":"2023-11-08T19:32:32.748790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:07.459426+0000","scrub_duration":0.0049107079999999997,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.898760+0000","last_change":"2023-11-08T19:32:29.898810+0000","last_active":"2023-11-08T19:32:29.898760+0000","last_peered":"2023-11-08T19:32:29.898760+0000","last_clean":"2023-11-08T19:32:29.898760+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:29.898760+0000","last_undegraded":"2023-11-08T19:32:29.898760+0000","last_fullsized":"2023-11-08T19:32:29.898760+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.898611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.898611+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.898611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:10.217632+0000","scrub_duration":0.0034633910000000001,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.191063+0000","last_change":"2023-11-08T19:27:47.808329+0000","last_active":"2023-11-08T19:32:29.191063+0000","last_peered":"2023-11-08T19:32:29.191063+0000","last_clean":"2023-11-08T19:32:29.191063+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:29.191063+0000","last_undegraded":"2023-11-08T19:32:29.191063+0000","last_fullsized":"2023-11-08T19:32:29.191063+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:43.834776+0000","last_change":"2023-11-08T19:32:43.834821+0000","last_active":"2023-11-08T19:32:43.834776+0000","last_peered":"2023-11-08T19:32:43.834776+0000","last_clean":"2023-11-08T19:32:43.834776+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:43.834776+0000","last_undegraded":"2023-11-08T19:32:43.834776+0000","last_fullsized":"2023-11-08T19:32:43.834776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:43.834622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:43.834622+0000","last_clean_scrub_stamp":"2023-11-08T19:32:43.834622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:57:53.189317+0000","scrub_duration":0.0032581189999999999,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.102935+0000","last_change":"2023-11-08T19:26:41.727242+0000","last_active":"2023-11-08T19:32:17.102935+0000","last_peered":"2023-11-08T19:32:17.102935+0000","last_clean":"2023-11-08T19:32:17.102935+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:17.102935+0000","last_undegraded":"2023-11-08T19:32:17.102935+0000","last_fullsized":"2023-11-08T19:32:17.102935+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.951788+0000","last_change":"2023-11-08T19:27:47.815820+0000","last_active":"2023-11-08T19:32:18.951788+0000","last_peered":"2023-11-08T19:32:18.951788+0000","last_clean":"2023-11-08T19:32:18.951788+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:32:18.951788+0000","last_undegraded":"2023-11-08T19:32:18.951788+0000","last_fullsized":"2023-11-08T19:32:18.951788+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.707482+0000","last_change":"2023-11-08T19:32:35.707531+0000","last_active":"2023-11-08T19:32:35.707482+0000","last_peered":"2023-11-08T19:32:35.707482+0000","last_clean":"2023-11-08T19:32:35.707482+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:35.707482+0000","last_undegraded":"2023-11-08T19:32:35.707482+0000","last_fullsized":"2023-11-08T19:32:35.707482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.707332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.707332+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.707332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:36:47.701015+0000","scrub_duration":0.0033186940000000001,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.105906+0000","last_change":"2023-11-08T19:26:41.731256+0000","last_active":"2023-11-08T19:32:17.105906+0000","last_peered":"2023-11-08T19:32:17.105906+0000","last_clean":"2023-11-08T19:32:17.105906+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:17.105906+0000","last_undegraded":"2023-11-08T19:32:17.105906+0000","last_fullsized":"2023-11-08T19:32:17.105906+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.493575+0000","last_change":"2023-11-08T19:27:47.804958+0000","last_active":"2023-11-08T19:32:18.493575+0000","last_peered":"2023-11-08T19:32:18.493575+0000","last_clean":"2023-11-08T19:32:18.493575+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:18.493575+0000","last_undegraded":"2023-11-08T19:32:18.493575+0000","last_fullsized":"2023-11-08T19:32:18.493575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.755805+0000","last_change":"2023-11-08T19:32:36.755856+0000","last_active":"2023-11-08T19:32:36.755805+0000","last_peered":"2023-11-08T19:32:36.755805+0000","last_clean":"2023-11-08T19:32:36.755805+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:36.755805+0000","last_undegraded":"2023-11-08T19:32:36.755805+0000","last_fullsized":"2023-11-08T19:32:36.755805+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.755645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.755645+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.755645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:34.826462+0000","scrub_duration":0.0037998400000000001,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.103732+0000","last_change":"2023-11-08T19:26:41.731181+0000","last_active":"2023-11-08T19:32:17.103732+0000","last_peered":"2023-11-08T19:32:17.103732+0000","last_clean":"2023-11-08T19:32:17.103732+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:32:17.103732+0000","last_undegraded":"2023-11-08T19:32:17.103732+0000","last_fullsized":"2023-11-08T19:32:17.103732+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.490577+0000","last_change":"2023-11-08T19:27:47.803547+0000","last_active":"2023-11-08T19:32:18.490577+0000","last_peered":"2023-11-08T19:32:18.490577+0000","last_clean":"2023-11-08T19:32:18.490577+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:32:18.490577+0000","last_undegraded":"2023-11-08T19:32:18.490577+0000","last_fullsized":"2023-11-08T19:32:18.490577+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.901318+0000","last_change":"2023-11-08T19:32:29.901364+0000","last_active":"2023-11-08T19:32:29.901318+0000","last_peered":"2023-11-08T19:32:29.901318+0000","last_clean":"2023-11-08T19:32:29.901318+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:29.901318+0000","last_undegraded":"2023-11-08T19:32:29.901318+0000","last_fullsized":"2023-11-08T19:32:29.901318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.901159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.901159+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.901159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:18.596622+0000","scrub_duration":0.0039081070000000001,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.887981+0000","last_change":"2023-11-08T19:32:27.887999+0000","last_active":"2023-11-08T19:32:27.887981+0000","last_peered":"2023-11-08T19:32:27.887981+0000","last_clean":"2023-11-08T19:32:27.887981+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:27.887981+0000","last_undegraded":"2023-11-08T19:32:27.887981+0000","last_fullsized":"2023-11-08T19:32:27.887981+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:27.887914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:27.887914+0000","last_clean_scrub_stamp":"2023-11-08T19:32:27.887914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:54.774642+0000","scrub_duration":0.0022926560000000001,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.103630+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:32:17.103630+0000","last_peered":"2023-11-08T19:32:17.103630+0000","last_clean":"2023-11-08T19:32:17.103630+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:32:17.103630+0000","last_undegraded":"2023-11-08T19:32:17.103630+0000","last_fullsized":"2023-11-08T19:32:17.103630+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.104373+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:32:17.104373+0000","last_peered":"2023-11-08T19:32:17.104373+0000","last_clean":"2023-11-08T19:32:17.104373+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:32:17.104373+0000","last_undegraded":"2023-11-08T19:32:17.104373+0000","last_fullsized":"2023-11-08T19:32:17.104373+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.981855+0000","last_change":"2023-11-08T19:32:26.981899+0000","last_active":"2023-11-08T19:32:26.981855+0000","last_peered":"2023-11-08T19:32:26.981855+0000","last_clean":"2023-11-08T19:32:26.981855+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:26.981855+0000","last_undegraded":"2023-11-08T19:32:26.981855+0000","last_fullsized":"2023-11-08T19:32:26.981855+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.981756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.981756+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.981756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:20.096248+0000","scrub_duration":0.0030939209999999999,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":48,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.826195+0000","last_change":"2023-11-08T19:27:47.804620+0000","last_active":"2023-11-08T19:32:18.826195+0000","last_peered":"2023-11-08T19:32:18.826195+0000","last_clean":"2023-11-08T19:32:18.826195+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:18.826195+0000","last_undegraded":"2023-11-08T19:32:18.826195+0000","last_fullsized":"2023-11-08T19:32:18.826195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.489618+0000","last_change":"2023-11-08T19:27:48.789263+0000","last_active":"2023-11-08T19:32:18.489618+0000","last_peered":"2023-11-08T19:32:18.489618+0000","last_clean":"2023-11-08T19:32:18.489618+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:18.489618+0000","last_undegraded":"2023-11-08T19:32:18.489618+0000","last_fullsized":"2023-11-08T19:32:18.489618+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:39.855542+0000","last_change":"2023-11-08T19:32:39.855592+0000","last_active":"2023-11-08T19:32:39.855542+0000","last_peered":"2023-11-08T19:32:39.855542+0000","last_clean":"2023-11-08T19:32:39.855542+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:39.855542+0000","last_undegraded":"2023-11-08T19:32:39.855542+0000","last_fullsized":"2023-11-08T19:32:39.855542+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:39.855386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:39.855386+0000","last_clean_scrub_stamp":"2023-11-08T19:32:39.855386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:11.250998+0000","scrub_duration":0.0035043140000000001,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.598433+0000","last_change":"2023-11-08T19:32:44.598517+0000","last_active":"2023-11-08T19:32:44.598433+0000","last_peered":"2023-11-08T19:32:44.598433+0000","last_clean":"2023-11-08T19:32:44.598433+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:44.598433+0000","last_undegraded":"2023-11-08T19:32:44.598433+0000","last_fullsized":"2023-11-08T19:32:44.598433+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.598277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.598277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.598277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:33.293740+0000","scrub_duration":0.0035716250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.848482+0000","last_change":"2023-11-08T19:32:24.848528+0000","last_active":"2023-11-08T19:32:24.848482+0000","last_peered":"2023-11-08T19:32:24.848482+0000","last_clean":"2023-11-08T19:32:24.848482+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:24.848482+0000","last_undegraded":"2023-11-08T19:32:24.848482+0000","last_fullsized":"2023-11-08T19:32:24.848482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_clean_scrub_stamp":"2023-11-08T19:32:24.848327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:22.967345+0000","scrub_duration":0.0035992749999999999,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.019582+0000","last_change":"2023-11-08T19:32:28.019603+0000","last_active":"2023-11-08T19:32:28.019582+0000","last_peered":"2023-11-08T19:32:28.019582+0000","last_clean":"2023-11-08T19:32:28.019582+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:28.019582+0000","last_undegraded":"2023-11-08T19:32:28.019582+0000","last_fullsized":"2023-11-08T19:32:28.019582+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.019513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.019513+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.019513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:18.418880+0000","scrub_duration":0.0027169690000000001,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.109057+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:32:17.109057+0000","last_peered":"2023-11-08T19:32:17.109057+0000","last_clean":"2023-11-08T19:32:17.109057+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:32:17.109057+0000","last_undegraded":"2023-11-08T19:32:17.109057+0000","last_fullsized":"2023-11-08T19:32:17.109057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:40.655839+0000","last_change":"2023-11-08T19:32:40.655884+0000","last_active":"2023-11-08T19:32:40.655839+0000","last_peered":"2023-11-08T19:32:40.655839+0000","last_clean":"2023-11-08T19:32:40.655839+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:40.655839+0000","last_undegraded":"2023-11-08T19:32:40.655839+0000","last_fullsized":"2023-11-08T19:32:40.655839+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:40.655688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:40.655688+0000","last_clean_scrub_stamp":"2023-11-08T19:32:40.655688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:41.923574+0000","scrub_duration":0.003438927,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:25.703825+0000","last_change":"2023-11-08T19:32:25.703850+0000","last_active":"2023-11-08T19:32:25.703825+0000","last_peered":"2023-11-08T19:32:25.703825+0000","last_clean":"2023-11-08T19:32:25.703825+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:25.703825+0000","last_undegraded":"2023-11-08T19:32:25.703825+0000","last_fullsized":"2023-11-08T19:32:25.703825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_clean_scrub_stamp":"2023-11-08T19:32:25.703734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:40.388881+0000","scrub_duration":0.0027277820000000002,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.949249+0000","last_change":"2023-11-08T19:27:47.812758+0000","last_active":"2023-11-08T19:32:18.949249+0000","last_peered":"2023-11-08T19:32:18.949249+0000","last_clean":"2023-11-08T19:32:18.949249+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:18.949249+0000","last_undegraded":"2023-11-08T19:32:18.949249+0000","last_fullsized":"2023-11-08T19:32:18.949249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.950835+0000","last_change":"2023-11-08T19:32:30.950919+0000","last_active":"2023-11-08T19:32:30.950835+0000","last_peered":"2023-11-08T19:32:30.950835+0000","last_clean":"2023-11-08T19:32:30.950835+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:30.950835+0000","last_undegraded":"2023-11-08T19:32:30.950835+0000","last_fullsized":"2023-11-08T19:32:30.950835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.950623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.950623+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.950623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:53.223016+0000","scrub_duration":0.0042637869999999998,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.657467+0000","last_change":"2023-11-08T19:32:30.657516+0000","last_active":"2023-11-08T19:32:30.657467+0000","last_peered":"2023-11-08T19:32:30.657467+0000","last_clean":"2023-11-08T19:32:30.657467+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:30.657467+0000","last_undegraded":"2023-11-08T19:32:30.657467+0000","last_fullsized":"2023-11-08T19:32:30.657467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.657305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.657305+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.657305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:01:57.318648+0000","scrub_duration":0.003567045,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:21.813461+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:32:21.813461+0000","last_peered":"2023-11-08T19:32:21.813461+0000","last_clean":"2023-11-08T19:32:21.813461+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:32:21.813461+0000","last_undegraded":"2023-11-08T19:32:21.813461+0000","last_fullsized":"2023-11-08T19:32:21.813461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825240+0000","last_change":"2023-11-08T19:27:48.916790+0000","last_active":"2023-11-08T19:32:18.825240+0000","last_peered":"2023-11-08T19:32:18.825240+0000","last_clean":"2023-11-08T19:32:18.825240+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:18.825240+0000","last_undegraded":"2023-11-08T19:32:18.825240+0000","last_fullsized":"2023-11-08T19:32:18.825240+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.652889+0000","last_change":"2023-11-08T19:32:46.652935+0000","last_active":"2023-11-08T19:32:46.652889+0000","last_peered":"2023-11-08T19:32:46.652889+0000","last_clean":"2023-11-08T19:32:46.652889+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:46.652889+0000","last_undegraded":"2023-11-08T19:32:46.652889+0000","last_fullsized":"2023-11-08T19:32:46.652889+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.652731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.652731+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.652731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:57.268837+0000","scrub_duration":0.0036855339999999999,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:21.814536+0000","last_change":"2023-11-08T19:27:47.812610+0000","last_active":"2023-11-08T19:32:21.814536+0000","last_peered":"2023-11-08T19:32:21.814536+0000","last_clean":"2023-11-08T19:32:21.814536+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:21.814536+0000","last_undegraded":"2023-11-08T19:32:21.814536+0000","last_fullsized":"2023-11-08T19:32:21.814536+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.108429+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:32:17.108429+0000","last_peered":"2023-11-08T19:32:17.108429+0000","last_clean":"2023-11-08T19:32:17.108429+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:32:17.108429+0000","last_undegraded":"2023-11-08T19:32:17.108429+0000","last_fullsized":"2023-11-08T19:32:17.108429+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:42.878234+0000","last_change":"2023-11-08T19:32:42.878282+0000","last_active":"2023-11-08T19:32:42.878234+0000","last_peered":"2023-11-08T19:32:42.878234+0000","last_clean":"2023-11-08T19:32:42.878234+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:42.878234+0000","last_undegraded":"2023-11-08T19:32:42.878234+0000","last_fullsized":"2023-11-08T19:32:42.878234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:42.878052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:42.878052+0000","last_clean_scrub_stamp":"2023-11-08T19:32:42.878052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:41.200867+0000","scrub_duration":0.003533516,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:21.803841+0000","last_change":"2023-11-08T19:27:47.802556+0000","last_active":"2023-11-08T19:32:21.803841+0000","last_peered":"2023-11-08T19:32:21.803841+0000","last_clean":"2023-11-08T19:32:21.803841+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:21.803841+0000","last_undegraded":"2023-11-08T19:32:21.803841+0000","last_fullsized":"2023-11-08T19:32:21.803841+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.825318+0000","last_change":"2023-11-08T19:27:48.916124+0000","last_active":"2023-11-08T19:32:18.825318+0000","last_peered":"2023-11-08T19:32:18.825318+0000","last_clean":"2023-11-08T19:32:18.825318+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:18.825318+0000","last_undegraded":"2023-11-08T19:32:18.825318+0000","last_fullsized":"2023-11-08T19:32:18.825318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":79,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.110349+0000","last_change":"2023-11-08T19:26:41.732128+0000","last_active":"2023-11-08T19:32:17.110349+0000","last_peered":"2023-11-08T19:32:17.110349+0000","last_clean":"2023-11-08T19:32:17.110349+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:17.110349+0000","last_undegraded":"2023-11-08T19:32:17.110349+0000","last_fullsized":"2023-11-08T19:32:17.110349+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:26:39.680044+0000","last_clean_scrub_stamp":"2023-11-08T19:26:39.680044+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.633079+0000","last_change":"2023-11-08T19:32:41.633132+0000","last_active":"2023-11-08T19:32:41.633079+0000","last_peered":"2023-11-08T19:32:41.633079+0000","last_clean":"2023-11-08T19:32:41.633079+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:41.633079+0000","last_undegraded":"2023-11-08T19:32:41.633079+0000","last_fullsized":"2023-11-08T19:32:41.633079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.632927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.632927+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.632927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:36.100474+0000","scrub_duration":0.0039924180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.088025+0000","last_change":"2023-11-08T19:32:30.088072+0000","last_active":"2023-11-08T19:32:30.088025+0000","last_peered":"2023-11-08T19:32:30.088025+0000","last_clean":"2023-11-08T19:32:30.088025+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:30.088025+0000","last_undegraded":"2023-11-08T19:32:30.088025+0000","last_fullsized":"2023-11-08T19:32:30.088025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.087841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.087841+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.087841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:05.489237+0000","scrub_duration":0.003235796,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.685837+0000","last_change":"2023-11-08T19:32:34.685883+0000","last_active":"2023-11-08T19:32:34.685837+0000","last_peered":"2023-11-08T19:32:34.685837+0000","last_clean":"2023-11-08T19:32:34.685837+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:34.685837+0000","last_undegraded":"2023-11-08T19:32:34.685837+0000","last_fullsized":"2023-11-08T19:32:34.685837+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.685687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.685687+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.685687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:39.814571+0000","scrub_duration":0.0041719410000000002,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.057312+0000","last_change":"2023-11-08T19:32:36.057360+0000","last_active":"2023-11-08T19:32:36.057312+0000","last_peered":"2023-11-08T19:32:36.057312+0000","last_clean":"2023-11-08T19:32:36.057312+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:36.057312+0000","last_undegraded":"2023-11-08T19:32:36.057312+0000","last_fullsized":"2023-11-08T19:32:36.057312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.057153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.057153+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.057153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:57:50.126042+0000","scrub_duration":0.0035364089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:43.628590+0000","last_change":"2023-11-08T19:32:43.628647+0000","last_active":"2023-11-08T19:32:43.628590+0000","last_peered":"2023-11-08T19:32:43.628590+0000","last_clean":"2023-11-08T19:32:43.628590+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:43.628590+0000","last_undegraded":"2023-11-08T19:32:43.628590+0000","last_fullsized":"2023-11-08T19:32:43.628590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:43.628419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:43.628419+0000","last_clean_scrub_stamp":"2023-11-08T19:32:43.628419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:42:45.712641+0000","scrub_duration":0.0032515249999999999,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":92,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.618782+0000","last_change":"2023-11-08T19:32:30.618782+0000","last_active":"2023-11-08T19:32:30.618782+0000","last_peered":"2023-11-08T19:32:30.618782+0000","last_clean":"2023-11-08T19:32:30.618782+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:30.618782+0000","last_undegraded":"2023-11-08T19:32:30.618782+0000","last_fullsized":"2023-11-08T19:32:30.618782+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.618568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.618568+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.618568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:54:51.352530+0000","scrub_duration":0.0037122639999999998,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:40.632718+0000","last_change":"2023-11-08T19:32:40.632765+0000","last_active":"2023-11-08T19:32:40.632718+0000","last_peered":"2023-11-08T19:32:40.632718+0000","last_clean":"2023-11-08T19:32:40.632718+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:40.632718+0000","last_undegraded":"2023-11-08T19:32:40.632718+0000","last_fullsized":"2023-11-08T19:32:40.632718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:40.632500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:40.632500+0000","last_clean_scrub_stamp":"2023-11-08T19:32:40.632500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:28:55.639010+0000","scrub_duration":0.0035395969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.613815+0000","last_change":"2023-11-08T19:32:41.613901+0000","last_active":"2023-11-08T19:32:41.613815+0000","last_peered":"2023-11-08T19:32:41.613815+0000","last_clean":"2023-11-08T19:32:41.613815+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:41.613815+0000","last_undegraded":"2023-11-08T19:32:41.613815+0000","last_fullsized":"2023-11-08T19:32:41.613815+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.613660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.613660+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.613660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:59.049648+0000","scrub_duration":0.0031596810000000001,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.662218+0000","last_change":"2023-11-08T19:32:36.662271+0000","last_active":"2023-11-08T19:32:36.662218+0000","last_peered":"2023-11-08T19:32:36.662218+0000","last_clean":"2023-11-08T19:32:36.662218+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:36.662218+0000","last_undegraded":"2023-11-08T19:32:36.662218+0000","last_fullsized":"2023-11-08T19:32:36.662218+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.662065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.662065+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.662065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:59.279634+0000","scrub_duration":0.0035784039999999999,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:33.011945+0000","last_change":"2023-11-08T19:32:33.011999+0000","last_active":"2023-11-08T19:32:33.011945+0000","last_peered":"2023-11-08T19:32:33.011945+0000","last_clean":"2023-11-08T19:32:33.011945+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:33.011945+0000","last_undegraded":"2023-11-08T19:32:33.011945+0000","last_fullsized":"2023-11-08T19:32:33.011945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:33.011743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:33.011743+0000","last_clean_scrub_stamp":"2023-11-08T19:32:33.011743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:06.887936+0000","scrub_duration":0.0036639329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.051468+0000","last_change":"2023-11-08T19:32:29.051518+0000","last_active":"2023-11-08T19:32:29.051468+0000","last_peered":"2023-11-08T19:32:29.051468+0000","last_clean":"2023-11-08T19:32:29.051468+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:29.051468+0000","last_undegraded":"2023-11-08T19:32:29.051468+0000","last_fullsized":"2023-11-08T19:32:29.051468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.051307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.051307+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.051307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:38.939562+0000","scrub_duration":0.0038472430000000002,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:37.646395+0000","last_change":"2023-11-08T19:32:37.646449+0000","last_active":"2023-11-08T19:32:37.646395+0000","last_peered":"2023-11-08T19:32:37.646395+0000","last_clean":"2023-11-08T19:32:37.646395+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:37.646395+0000","last_undegraded":"2023-11-08T19:32:37.646395+0000","last_fullsized":"2023-11-08T19:32:37.646395+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:37.646227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:37.646227+0000","last_clean_scrub_stamp":"2023-11-08T19:32:37.646227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:07.065709+0000","scrub_duration":0.0035584530000000001,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":47,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:21.804522+0000","last_change":"2023-11-08T19:27:47.803965+0000","last_active":"2023-11-08T19:32:21.804522+0000","last_peered":"2023-11-08T19:32:21.804522+0000","last_clean":"2023-11-08T19:32:21.804522+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:21.804522+0000","last_undegraded":"2023-11-08T19:32:21.804522+0000","last_fullsized":"2023-11-08T19:32:21.804522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:18.951849+0000","last_change":"2023-11-08T19:27:48.797143+0000","last_active":"2023-11-08T19:32:18.951849+0000","last_peered":"2023-11-08T19:32:18.951849+0000","last_clean":"2023-11-08T19:32:18.951849+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:18.951849+0000","last_undegraded":"2023-11-08T19:32:18.951849+0000","last_fullsized":"2023-11-08T19:32:18.951849+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.792001+0000","last_change":"2023-11-08T19:32:35.792054+0000","last_active":"2023-11-08T19:32:35.792001+0000","last_peered":"2023-11-08T19:32:35.792001+0000","last_clean":"2023-11-08T19:32:35.792001+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:35.792001+0000","last_undegraded":"2023-11-08T19:32:35.792001+0000","last_fullsized":"2023-11-08T19:32:35.792001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.791811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.791811+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.791811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:51:30.989602+0000","scrub_duration":0.003741978,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"109'119","reported_seq":257,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.572544+0000","last_change":"2023-11-08T19:32:29.572619+0000","last_active":"2023-11-08T19:32:29.572544+0000","last_peered":"2023-11-08T19:32:29.572544+0000","last_clean":"2023-11-08T19:32:29.572544+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:29.572544+0000","last_undegraded":"2023-11-08T19:32:29.572544+0000","last_fullsized":"2023-11-08T19:32:29.572544+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'119","last_scrub_stamp":"2023-11-08T19:32:29.572367+0000","last_deep_scrub":"109'119","last_deep_scrub_stamp":"2023-11-08T19:32:29.572367+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.572367+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:21.179114+0000","scrub_duration":0.0093265610000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574948,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1945600,"data_stored":1281536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.77500000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.599,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.268},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.125,"15min":1.252},"min":{"1min":0.72799999999999998,"5min":0.58599999999999997,"15min":0.58499999999999996},"max":{"1min":1.6619999999999999,"5min":2.1360000000000001,"15min":4.4589999999999996},"last":1.1040000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2010000000000001,"15min":1.24},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.548,"5min":1.728,"15min":4.5330000000000004},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.1910000000000001,"15min":1.1459999999999999},"min":{"1min":0.83299999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.6160000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.88900000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.335,"15min":1.448},"min":{"1min":0.98999999999999999,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.5660000000000001,"5min":1.891,"15min":4.6980000000000004},"last":1.306},{"interface":"front","average":{"1min":1.28,"5min":1.194,"15min":1.3160000000000001},"min":{"1min":0.97999999999999998,"5min":0.67200000000000004,"15min":0.64700000000000002},"max":{"1min":1.71,"5min":1.71,"15min":4.4269999999999996},"last":0.98099999999999998}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.294,"15min":1.343},"min":{"1min":0.749,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.7889999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":1.389},{"interface":"front","average":{"1min":1.383,"5min":1.409,"15min":1.5149999999999999},"min":{"1min":0.95199999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.768,"5min":2.2200000000000002,"15min":4.6150000000000002},"last":1.2290000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.246,"15min":1.327},"min":{"1min":1.006,"5min":0.624,"15min":0.624},"max":{"1min":1.49,"5min":1.798,"15min":2.3170000000000002},"last":1.347},{"interface":"front","average":{"1min":1.335,"5min":1.323,"15min":1.4410000000000001},"min":{"1min":0.93200000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7909999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.0269999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.389,"15min":1.4370000000000001},"min":{"1min":0.95999999999999996,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.673,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":1.4239999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.363,"15min":1.4470000000000001},"min":{"1min":1.048,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.649,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.0669999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4790000000000001,"15min":1.548},"min":{"1min":1.22,"5min":0.878,"15min":0.878},"max":{"1min":1.8109999999999999,"5min":1.9770000000000001,"15min":2.427},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.474,"15min":1.581},"min":{"1min":1.1180000000000001,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.98,"5min":2.1920000000000002,"15min":2.468},"last":1.4490000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574947,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.0940000000000001,"15min":0.98799999999999999},"min":{"1min":0.85399999999999998,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.536,"5min":1.6279999999999999,"15min":1.752},"last":1.202},{"interface":"front","average":{"1min":1.5720000000000001,"5min":1.202,"15min":1.1399999999999999},"min":{"1min":0.80200000000000005,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":8.391,"5min":8.391,"15min":8.391},"last":1.397}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.161,"15min":1.113},"min":{"1min":0.85599999999999998,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.679},"last":1.071},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.2050000000000001,"15min":1.3009999999999999},"min":{"1min":0.91000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.665,"5min":1.665,"15min":4.6399999999999997},"last":1.256}]},{"osd":2,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.2010000000000001,"15min":1.2649999999999999},"min":{"1min":0.80300000000000005,"5min":0.68799999999999994,"15min":0.54600000000000004},"max":{"1min":1.4830000000000001,"5min":1.655,"15min":4.609},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.2949999999999999,"15min":1.242},"min":{"1min":0.66800000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.861},"last":1.4370000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.276,"15min":1.4239999999999999},"min":{"1min":1.0209999999999999,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.5509999999999999,"5min":1.756,"15min":4.7190000000000003},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.256,"15min":1.2050000000000001},"min":{"1min":0.81200000000000006,"5min":0.625,"15min":0.625},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.784},"last":1.141}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.288,"15min":1.3500000000000001},"min":{"1min":0.83199999999999996,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.859,"5min":1.881,"15min":2.0449999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.4770000000000001,"15min":1.4910000000000001},"min":{"1min":1.0449999999999999,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.9279999999999999,"5min":1.998,"15min":2.3130000000000002},"last":1.5449999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.325,"15min":1.2689999999999999},"min":{"1min":0.68799999999999994,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.611,"5min":1.835,"15min":1.9079999999999999},"last":1.306},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.3999999999999999,"15min":1.353},"min":{"1min":0.92100000000000004,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":1.6819999999999999,"5min":1.8660000000000001,"15min":2.0489999999999999},"last":0.92100000000000004}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.52,"5min":1.5569999999999999,"15min":1.55},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":1.99,"5min":1.99,"15min":2.2519999999999998},"last":1.49},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.4970000000000001,"15min":1.464},"min":{"1min":0.75700000000000001,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":1.7809999999999999,"5min":1.9870000000000001,"15min":3.0640000000000001},"last":1.3700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574947,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1970000000000001,"15min":1.117},"min":{"1min":0.69599999999999995,"5min":0.59999999999999998,"15min":0.57699999999999996},"max":{"1min":1.603,"5min":1.724,"15min":1.76},"last":1.379},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.274,"15min":1.252},"min":{"1min":0.77600000000000002,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.837,"5min":1.954,"15min":1.954},"last":1.4159999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.1279999999999999,"15min":1.1599999999999999},"min":{"1min":0.754,"5min":0.753,"15min":0.58699999999999997},"max":{"1min":1.552,"5min":1.6359999999999999,"15min":2.3039999999999998},"last":1.165},{"interface":"front","average":{"1min":1.179,"5min":1.1679999999999999,"15min":1.355},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.8009999999999999,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.274}]},{"osd":2,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.2350000000000001,"15min":1.194},"min":{"1min":0.82599999999999996,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.5720000000000001,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":1.117},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.173,"15min":1.3959999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6990000000000001,"5min":1.806,"15min":7.6479999999999997},"last":1.1870000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2010000000000001,"15min":1.2629999999999999},"min":{"1min":0.89900000000000002,"5min":0.76900000000000002,"15min":0.71899999999999997},"max":{"1min":1.74,"5min":1.804,"15min":2.1920000000000002},"last":1.333},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3500000000000001,"15min":1.323},"min":{"1min":0.98799999999999999,"5min":0.91300000000000003,"15min":0.69699999999999995},"max":{"1min":1.6910000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.458}]},{"osd":4,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.196,"15min":1.2589999999999999},"min":{"1min":0.79600000000000004,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7350000000000001,"5min":1.7969999999999999,"15min":2.347},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.347,"15min":1.488},"min":{"1min":0.94099999999999995,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.7350000000000001,"5min":2.097,"15min":3.153},"last":1.0089999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.3859999999999999,"15min":1.4830000000000001},"min":{"1min":0.92100000000000004,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.923,"5min":1.923,"15min":2.8650000000000002},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4019999999999999,"15min":1.47},"min":{"1min":0.94199999999999995,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.6990000000000001,"5min":1.899,"15min":2.6190000000000002},"last":1.492}]},{"osd":7,"last update":"Wed Nov 8 19:31:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.4950000000000001,"15min":1.6359999999999999},"min":{"1min":1.0980000000000001,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":1.9259999999999999,"5min":2.0630000000000002,"15min":3.2730000000000001},"last":1.232},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5229999999999999,"15min":1.5680000000000001},"min":{"1min":1.204,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.8600000000000001,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.583}]}]},{"osd":4,"up_from":13,"seq":55834574949,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1479999999999999,"15min":1.0760000000000001},"min":{"1min":0.76400000000000001,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.5900000000000001,"5min":1.782,"15min":2.5219999999999998},"last":1.294},{"interface":"front","average":{"1min":1.206,"5min":1.2150000000000001,"15min":1.216},"min":{"1min":0.83499999999999996,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.524,"5min":1.696,"15min":7.0119999999999996},"last":1.496}]},{"osd":1,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.0680000000000001,"15min":1.091},"min":{"1min":0.74299999999999999,"5min":0.64300000000000002,"15min":0.63100000000000001},"max":{"1min":1.752,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.073},{"interface":"front","average":{"1min":1.177,"5min":1.1499999999999999,"15min":1.1519999999999999},"min":{"1min":0.748,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.7310000000000001,"5min":1.7450000000000001,"15min":2.9940000000000002},"last":0.91200000000000003}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2490000000000001,"15min":1.3160000000000001},"min":{"1min":0.98199999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.988,"5min":1.988,"15min":4.0410000000000004},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.319,"15min":1.3879999999999999},"min":{"1min":0.95999999999999996,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.6739999999999999,"5min":1.867,"15min":3.2069999999999999},"last":1.397}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.1659999999999999,"15min":1.258},"min":{"1min":0.84199999999999997,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.5169999999999999,"5min":1.696,"15min":4.4980000000000002},"last":1.353},{"interface":"front","average":{"1min":1.226,"5min":1.2549999999999999,"15min":1.345},"min":{"1min":0.91400000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":1.738,"15min":3.5710000000000002},"last":1.3240000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.2629999999999999,"15min":1.4330000000000001},"min":{"1min":0.81000000000000005,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.593,"5min":1.762,"15min":4.2350000000000003},"last":1.117},{"interface":"front","average":{"1min":1.268,"5min":1.2549999999999999,"15min":1.4590000000000001},"min":{"1min":1.0189999999999999,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.5800000000000001,"5min":1.8560000000000001,"15min":4.5880000000000001},"last":1.222}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.365,"15min":1.5009999999999999},"min":{"1min":0.92300000000000004,"5min":0.745,"15min":0.745},"max":{"1min":2.2130000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4419999999999999,"15min":1.5660000000000001},"min":{"1min":1.1639999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":1.841,"5min":1.9490000000000001,"15min":4.4130000000000003},"last":1.5700000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.409,"15min":1.5640000000000001},"min":{"1min":1.137,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":4.6369999999999996},"last":1.905},{"interface":"front","average":{"1min":1.5640000000000001,"5min":1.5669999999999999,"15min":1.6799999999999999},"min":{"1min":1.2969999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.8999999999999999,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.538}]}]},{"osd":3,"up_from":13,"seq":55834574947,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.81100000000000005,"15min":0.99399999999999999},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":1.427,"5min":2.2320000000000002,"15min":3.972},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93500000000000005,"15min":1.042},"min":{"1min":0.48699999999999999,"5min":0.372,"15min":0.311},"max":{"1min":1.544,"5min":1.899,"15min":3.1520000000000001},"last":1.177}]},{"osd":1,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95299999999999996,"15min":1.1160000000000001},"min":{"1min":0.46500000000000002,"5min":0.41699999999999998,"15min":0.39500000000000002},"max":{"1min":1.4299999999999999,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90200000000000002,"15min":1.0429999999999999},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.5409999999999999,"5min":1.8520000000000001,"15min":4.5750000000000002},"last":1.0369999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.153,"15min":1.272},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.619,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.256},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1020000000000001,"15min":1.218},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.6339999999999999,"5min":1.754,"15min":2.73},"last":1.006}]},{"osd":4,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.214,"15min":1.3779999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6599999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.399},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.24,"15min":1.3899999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.9039999999999999,"5min":2.0449999999999999,"15min":3.052},"last":1.1020000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.306,"15min":1.4370000000000001},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.8360000000000001,"5min":2.0430000000000001,"15min":3.0899999999999999},"last":1.4510000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.29,"15min":1.4570000000000001},"min":{"1min":0.72799999999999998,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.597,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.0760000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2769999999999999,"15min":1.3819999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.7,"5min":3.21,"15min":4.2220000000000004},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.391,"15min":1.526},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9490000000000001,"5min":2.0880000000000001,"15min":3.1230000000000002},"last":1.329}]},{"osd":7,"last update":"Wed Nov 8 19:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.399,"15min":1.6160000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.925,"5min":2.1150000000000002,"15min":4.407},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.5209999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.671,"5min":2.0059999999999998,"15min":4.3849999999999998},"last":1.48}]}]},{"osd":2,"up_from":13,"seq":55834574949,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1347584,"data_stored":690149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.83699999999999997,"15min":0.85399999999999998},"min":{"1min":0.61699999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.2370000000000001,"5min":1.7809999999999999,"15min":1.7809999999999999},"last":0.749},{"interface":"front","average":{"1min":1.024,"5min":0.90900000000000003,"15min":0.92000000000000004},"min":{"1min":0.40600000000000003,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":1.4059999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.88600000000000001,"15min":0.873},"min":{"1min":0.56799999999999995,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.371,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0169999999999999,"15min":1.0129999999999999},"min":{"1min":0.66700000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.474,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":1.0029999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.093,"15min":1.0620000000000001},"min":{"1min":0.80300000000000005,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.052,"15min":1.103},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.675,"5min":1.7330000000000001,"15min":1.7669999999999999},"last":1.145}]},{"osd":4,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2829999999999999,"15min":1.365},"min":{"1min":0.95499999999999996,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.6850000000000001,"5min":1.883,"15min":3.2959999999999998},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.284,"15min":1.3400000000000001},"min":{"1min":1.0609999999999999,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.6359999999999999,"5min":1.704,"15min":2.278},"last":1.246}]},{"osd":5,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.353,"5min":1.325,"15min":1.3899999999999999},"min":{"1min":1.1359999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":1.869,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.427},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.327,"15min":1.4179999999999999},"min":{"1min":1.048,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.7709999999999999,"5min":1.954,"15min":2.536},"last":1.2030000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3680000000000001,"15min":1.381},"min":{"1min":1.1120000000000001,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.377,"5min":1.351,"15min":1.421},"min":{"1min":0.92200000000000004,"5min":0.83799999999999997,"15min":0.70099999999999996},"max":{"1min":1.9810000000000001,"5min":2.036,"15min":2.0910000000000002},"last":1.302}]},{"osd":7,"last update":"Wed Nov 8 19:32:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4299999999999999,"5min":1.409,"15min":1.522},"min":{"1min":0.95099999999999996,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":1.74,"5min":1.9339999999999999,"15min":2.6549999999999998},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.466,"15min":1.5269999999999999},"min":{"1min":1.111,"5min":0.84999999999999998,"15min":0.76600000000000001},"max":{"1min":1.8009999999999999,"5min":2.226,"15min":2.3439999999999999},"last":1.571}]}]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.85599999999999998,"15min":0.78000000000000003},"min":{"1min":0.59199999999999997,"5min":0.47199999999999998,"15min":0.45700000000000002},"max":{"1min":1.284,"5min":1.327,"15min":1.327},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0629999999999999,"5min":0.88800000000000001,"15min":0.79900000000000004},"min":{"1min":0.82499999999999996,"5min":0.38700000000000001,"15min":0.374},"max":{"1min":1.528,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.81699999999999995}]},{"osd":2,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.84099999999999997,"15min":0.81100000000000005},"min":{"1min":0.51100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.256,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.872},{"interface":"front","average":{"1min":1.0589999999999999,"5min":1.0249999999999999,"15min":0.93200000000000005},"min":{"1min":0.623,"5min":0.47499999999999998,"15min":0.46400000000000002},"max":{"1min":1.5980000000000001,"5min":1.863,"15min":1.863},"last":1.014}]},{"osd":3,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2569999999999999,"15min":1.1240000000000001},"min":{"1min":0.80100000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.748,"5min":1.978,"15min":1.978},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.1439999999999999,"15min":1.1020000000000001},"min":{"1min":0.65300000000000002,"5min":0.5,"15min":0.49299999999999999},"max":{"1min":1.6930000000000001,"5min":2.2690000000000001,"15min":2.5369999999999999},"last":1.0740000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.159,"15min":1.157},"min":{"1min":0.81499999999999995,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.476,"5min":1.8620000000000001,"15min":2.165},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2729999999999999,"15min":1.26},"min":{"1min":0.95599999999999996,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.6870000000000001,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.397}]},{"osd":5,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.93799999999999994,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.802,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.2689999999999999,"15min":1.3069999999999999},"min":{"1min":0.95199999999999996,"5min":0.73199999999999998,"15min":0.629},"max":{"1min":1.696,"5min":1.748,"15min":2.4089999999999998},"last":1.347}]},{"osd":6,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3939999999999999,"15min":1.375},"min":{"1min":1.194,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.7450000000000001,"5min":2.1509999999999998,"15min":2.6019999999999999},"last":1.26},{"interface":"front","average":{"1min":1.4359999999999999,"5min":1.355,"15min":1.373},"min":{"1min":1.133,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":2.282},"last":1.4870000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.415,"15min":1.3560000000000001},"min":{"1min":1.0029999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":1.915,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.522},{"interface":"front","average":{"1min":1.454,"5min":1.409,"15min":1.395},"min":{"1min":1.0940000000000001,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.865,"5min":2.4830000000000001,"15min":2.6699999999999999},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834574948,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1941504,"data_stored":1280517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.80900000000000005,"15min":0.78700000000000003},"min":{"1min":0.63200000000000001,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.0780000000000001,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":1.577},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.92200000000000004,"15min":0.93400000000000005},"min":{"1min":0.53100000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2110000000000001,"5min":1.4770000000000001,"15min":1.7},"last":0.79500000000000004}]},{"osd":2,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.89200000000000002,"15min":0.876},"min":{"1min":0.57399999999999995,"5min":0.432,"15min":0.432},"max":{"1min":1.319,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.1299999999999999,"5min":0.998,"15min":0.98299999999999998},"min":{"1min":0.76200000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5189999999999999,"5min":1.54,"15min":1.7270000000000001},"last":1.1419999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.016},"min":{"1min":0.501,"5min":0.38,"15min":0.38},"max":{"1min":1.611,"5min":1.6599999999999999,"15min":1.748},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.089,"5min":1.0660000000000001,"15min":1.0940000000000001},"min":{"1min":0.60599999999999998,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.552,"5min":1.8400000000000001,"15min":2.496},"last":1.2749999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.2709999999999999,"15min":1.244},"min":{"1min":0.90600000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.6180000000000001,"5min":1.794,"15min":1.9239999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.29,"15min":1.3939999999999999},"min":{"1min":1.127,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.6399999999999999,"5min":1.706,"15min":2.4039999999999999},"last":1.3839999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.306,"15min":1.351},"min":{"1min":1.042,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.7270000000000001,"5min":1.8620000000000001,"15min":2.4369999999999998},"last":1.659},{"interface":"front","average":{"1min":1.319,"5min":1.2949999999999999,"15min":1.3979999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.569,"5min":2.089,"15min":2.4239999999999999},"last":1.194}]},{"osd":6,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3500000000000001,"15min":1.4550000000000001},"min":{"1min":0.93600000000000005,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.7110000000000001,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.3480000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.357,"15min":1.3959999999999999},"min":{"1min":0.79200000000000004,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.7549999999999999,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.22}]},{"osd":7,"last update":"Wed Nov 8 19:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.4119999999999999,"15min":1.405},"min":{"1min":0.93400000000000005,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.851,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.415,"15min":1.462},"min":{"1min":1.103,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.7909999999999999,"5min":1.881,"15min":2.306},"last":1.49}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:32:50.207 INFO:tasks.ceph:pgid 13.16 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.208 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.208 INFO:tasks.ceph:pgid 13.15 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.209 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.209 INFO:tasks.ceph:pgid 13.14 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.209 INFO:tasks.ceph:pgid 12.15 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.210 INFO:tasks.ceph:pgid 13.13 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.210 INFO:tasks.ceph:pgid 13.12 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.211 INFO:tasks.ceph:pgid 13.1f last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.211 INFO:tasks.ceph:pgid 13.1e last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.212 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.212 INFO:tasks.ceph:pgid 13.1d last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.212 INFO:tasks.ceph:pgid 13.1c last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.213 INFO:tasks.ceph:pgid 12.1d last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.213 INFO:tasks.ceph:pgid 13.1b last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.214 INFO:tasks.ceph:pgid 13.1a last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.214 INFO:tasks.ceph:pgid 12.1b last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.214 INFO:tasks.ceph:pgid 13.19 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.215 INFO:tasks.ceph:pgid 13.18 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.216 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.216 INFO:tasks.ceph:pgid 13.7 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.216 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.217 INFO:tasks.ceph:pgid 13.6 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.217 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.217 INFO:tasks.ceph:pgid 13.5 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.218 INFO:tasks.ceph:pgid 13.4 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.218 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.219 INFO:tasks.ceph:pgid 13.3 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.219 INFO:tasks.ceph:pgid 12.2 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.220 INFO:tasks.ceph:pgid 13.1 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.220 INFO:tasks.ceph:pgid 13.0 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.220 INFO:tasks.ceph:pgid 13.f last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.221 INFO:tasks.ceph:pgid 12.e last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.221 INFO:tasks.ceph:pgid 13.e last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.221 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.222 INFO:tasks.ceph:pgid 13.d last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.222 INFO:tasks.ceph:pgid 12.c last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.223 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2023-11-08T19:26:39.680044+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=26, tm_sec=39, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.223 INFO:tasks.ceph:pgid 13.c last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.224 INFO:tasks.ceph:pgid 12.d last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:32:50.224 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T19:32:50.894 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:32:50.903 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:10.225 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:33:10.449 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.447+0000 7fbbf4c90640 1 Processor -- start 2023-11-08T19:33:10.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.448+0000 7fbbf4c90640 1 -- start start 2023-11-08T19:33:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbf4c90640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbbf006ebd0 0x7fbbf0161900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbf4c90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 0x7fbbf0070b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbf4c90640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbf00cf3e0 con 0x7fbbf006a500 2023-11-08T19:33:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbf4c90640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbf00cfbd0 con 0x7fbbf006ebd0 2023-11-08T19:33:10.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbf4c90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0161e40 con 0x7fbbf006efb0 2023-11-08T19:33:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbeed9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7fbbf006a500 0x7fbbf006a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:41962/0 (socket says 172.21.15.119:41962) 2023-11-08T19:33:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.449+0000 7fbbeed9d640 1 -- 172.21.15.119:0/4104008115 learned_addr learned my addr 172.21.15.119:0/4104008115 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:33:10.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbee59c640 1 --2- 172.21.15.119:0/4104008115 >> v2:172.21.15.188:3300/0 conn(0x7fbbf006ebd0 0x7fbbf0161900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871152332 0 0) 0x7fbbf00cf3e0 con 0x7fbbf006a500 2023-11-08T19:33:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 0x7fbbf0070b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbdc003520 con 0x7fbbf006a500 2023-11-08T19:33:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2098684212 0 0) 0x7fbbdc003520 con 0x7fbbf006a500 2023-11-08T19:33:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 msgr2=0x7fbbf0070b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 0x7fbbf0070b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 >> v2:172.21.15.188:3300/0 conn(0x7fbbf006ebd0 msgr2=0x7fbbf0161900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:10.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 --2- 172.21.15.119:0/4104008115 >> v2:172.21.15.188:3300/0 conn(0x7fbbf006ebd0 0x7fbbf0161900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf006b0a0 con 0x7fbbf006a500 2023-11-08T19:33:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7fbbe8001670 con 0x7fbbf006a500 2023-11-08T19:33:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbee59c640 1 --2- 172.21.15.119:0/4104008115 >> v2:172.21.15.188:3300/0 conn(0x7fbbf006ebd0 0x7fbbf0161900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T19:33:10.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 >> v1:172.21.15.119:6789/0 conn(0x7fbbf006a500 legacy=0x7fbbf006a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 0x7fbbdc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbdc003f40 0x7fbbdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0161e40 con 0x7fbbdc003f40 2023-11-08T19:33:10.461 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00ba60 con 0x7fbbdc004d00 2023-11-08T19:33:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.450+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc00bc20 con 0x7fbbf006efb0 2023-11-08T19:33:10.462 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbdc003f40 0x7fbbdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbee59c640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 0x7fbbdc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbeed9d640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbedd9b640 1 -- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 msgr2=0x7fbbdc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:10.463 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbf006efb0 0x7fbbdc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbedd9b640 1 -- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc004d00 msgr2=0x7fbbdc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:10.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbdc004d00 0x7fbbdc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.464 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.451+0000 7fbbedd9b640 1 -- 172.21.15.119:0/4104008115 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc003520 con 0x7fbbdc003f40 2023-11-08T19:33:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.452+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbdc003f40 0x7fbbdc0047c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fc0 tx=0x7fbbdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73baf1b61de2788 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:10.468 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd802cc20 con 0x7fbbdc003f40 2023-11-08T19:33:10.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd800f040 con 0x7fbbdc003f40 2023-11-08T19:33:10.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbed59a640 1 -- 172.21.15.119:0/4104008115 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8034440 con 0x7fbbdc003f40 2023-11-08T19:33:10.469 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbf4c90640 1 -- 172.21.15.119:0/4104008115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbdc003f40 msgr2=0x7fbbdc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:10.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/4104008115 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbdc003f40 0x7fbbdc0047c0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009fc0 tx=0x7fbbdc0057f0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbeed9d640 1 -- 172.21.15.119:0/4104008115 reap_dead start 2023-11-08T19:33:10.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbf4c90640 1 -- 172.21.15.119:0/4104008115 shutdown_connections 2023-11-08T19:33:10.470 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.453+0000 7fbbf4c90640 1 -- 172.21.15.119:0/4104008115 >> 172.21.15.119:0/4104008115 conn(0x7fbbf00cfd20 msgr2=0x7fbbf00d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:10.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.454+0000 7fbbf4c90640 1 -- 172.21.15.119:0/4104008115 shutdown_connections 2023-11-08T19:33:10.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.454+0000 7fbbf4c90640 1 -- 172.21.15.119:0/4104008115 wait complete. 2023-11-08T19:33:10.471 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.455+0000 7fbbf4c90640 1 Processor -- start 2023-11-08T19:33:10.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.455+0000 7fbbf4c90640 1 -- start start 2023-11-08T19:33:10.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.456+0000 7fbbf4c90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc003f40 0x7fbbf00c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.456+0000 7fbbf4c90640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 0x7fbbf01530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.472 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.456+0000 7fbbf4c90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf00b3050 0x7fbbf00b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.456+0000 7fbbf4c90640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf0153600 con 0x7fbbf00c5a20 2023-11-08T19:33:10.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.456+0000 7fbbf4c90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00c5e80 con 0x7fbbf00b3050 2023-11-08T19:33:10.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.456+0000 7fbbf4c90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbf00c7ee0 con 0x7fbbdc003f40 2023-11-08T19:33:10.473 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 0x7fbbf01530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 0x7fbbf01530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:59240/0 (socket says 172.21.15.119:59240) 2023-11-08T19:33:10.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 -- 172.21.15.119:0/1901026562 learned_addr learned my addr 172.21.15.119:0/1901026562 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:33:10.474 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbee59c640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc003f40 0x7fbbf00c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbeed9d640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf00b3050 0x7fbbf00b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 -- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc003f40 msgr2=0x7fbbf00c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:10.475 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc003f40 0x7fbbf00c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 -- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf00b3050 msgr2=0x7fbbf00b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf00b3050 0x7fbbf00b34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.476 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 -- 172.21.15.119:0/1901026562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf00c81c0 con 0x7fbbf00c5a20 2023-11-08T19:33:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.457+0000 7fbbedd9b640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 0x7fbbf01530c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e230 tx=0x7fbbd8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.458+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8034bd0 con 0x7fbbf00c5a20 2023-11-08T19:33:10.477 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.458+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd8034d50 con 0x7fbbf00c5a20 2023-11-08T19:33:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.458+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd8003bd0 con 0x7fbbf00c5a20 2023-11-08T19:33:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.458+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbf00c84a0 con 0x7fbbf00c5a20 2023-11-08T19:33:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.458+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbf00c8a60 con 0x7fbbf00c5a20 2023-11-08T19:33:10.478 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.459+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbf0074430 con 0x7fbbf00c5a20 2023-11-08T19:33:10.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.460+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbbd8040450 con 0x7fbbf00c5a20 2023-11-08T19:33:10.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.461+0000 7fbbe6ffd640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbbd006aea0 0x7fbbd006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:10.479 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.461+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7fbbd80ac240 con 0x7fbbf00c5a20 2023-11-08T19:33:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.461+0000 7fbbee59c640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbbd006aea0 0x7fbbd006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.462+0000 7fbbee59c640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbbd006aea0 0x7fbbd006d360 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0006fd0 tx=0x7fbbe0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:10.480 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.466+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbbd8076d40 con 0x7fbbf00c5a20 2023-11-08T19:33:10.653 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.651+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbbf00cdbd0 con 0x7fbbd006aea0 2023-11-08T19:33:10.662 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.661+0000 7fbbe6ffd640 1 -- 172.21.15.119:0/1901026562 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+291603 (secure 0 0 0) 0x7fbbe000c040 con 0x7fbbd006aea0 2023-11-08T19:33:10.667 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:33:10.669 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:33:10.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbbd006aea0 msgr2=0x7fbbd006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:10.671 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbbd006aea0 0x7fbbd006d360 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0006fd0 tx=0x7fbbe0008040 comp rx=0 tx=0).stop 2023-11-08T19:33:10.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 msgr2=0x7fbbf01530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:10.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 0x7fbbf01530c0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fbbd800e230 tx=0x7fbbd8009750 comp rx=0 tx=0).stop 2023-11-08T19:33:10.672 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 shutdown_connections 2023-11-08T19:33:10.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7fbbd006aea0 0x7fbbd006d360 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbbf00b3050 0x7fbbf00b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.673 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7fbbf00c5a20 0x7fbbf01530c0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 --2- 172.21.15.119:0/1901026562 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbbdc003f40 0x7fbbf00c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:10.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.668+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 >> 172.21.15.119:0/1901026562 conn(0x7fbbf00cfd20 msgr2=0x7fbbf006e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:10.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.669+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 shutdown_connections 2023-11-08T19:33:10.674 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:10.669+0000 7fbbf4c90640 1 -- 172.21.15.119:0/1901026562 wait complete. 2023-11-08T19:33:10.689 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2023-11-08T19:33:08.859186+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261120,"kb_used_data":11712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713600,"statfs":{"total":773094113280,"available":772826726400,"internally_reserved":0,"allocated":11993088,"data_stored":6708042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005033"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.525576+0000","last_change":"2023-11-08T19:32:46.525623+0000","last_active":"2023-11-08T19:32:46.525576+0000","last_peered":"2023-11-08T19:32:46.525576+0000","last_clean":"2023-11-08T19:32:46.525576+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:46.525576+0000","last_undegraded":"2023-11-08T19:32:46.525576+0000","last_fullsized":"2023-11-08T19:32:46.525576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.525310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.525310+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.525310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:58:08.797703+0000","scrub_duration":0.0036410370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:32.935524+0000","last_change":"2023-11-08T19:32:32.935524+0000","last_active":"2023-11-08T19:32:32.935524+0000","last_peered":"2023-11-08T19:32:32.935524+0000","last_clean":"2023-11-08T19:32:32.935524+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:32.935524+0000","last_undegraded":"2023-11-08T19:32:32.935524+0000","last_fullsized":"2023-11-08T19:32:32.935524+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:32.935290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:32.935290+0000","last_clean_scrub_stamp":"2023-11-08T19:32:32.935290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:23.985335+0000","scrub_duration":0.0032534619999999999,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.771234+0000","last_change":"2023-11-08T19:32:29.771285+0000","last_active":"2023-11-08T19:32:29.771234+0000","last_peered":"2023-11-08T19:32:29.771234+0000","last_clean":"2023-11-08T19:32:29.771234+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:29.771234+0000","last_undegraded":"2023-11-08T19:32:29.771234+0000","last_fullsized":"2023-11-08T19:32:29.771234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.771060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.771060+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.771060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:57:45.945310+0000","scrub_duration":0.0045387559999999997,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:50.683999+0000","last_change":"2023-11-08T19:32:50.684086+0000","last_active":"2023-11-08T19:32:50.683999+0000","last_peered":"2023-11-08T19:32:50.683999+0000","last_clean":"2023-11-08T19:32:50.683999+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:50.683999+0000","last_undegraded":"2023-11-08T19:32:50.683999+0000","last_fullsized":"2023-11-08T19:32:50.683999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:50.683777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:50.683777+0000","last_clean_scrub_stamp":"2023-11-08T19:32:50.683777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:40.297282+0000","scrub_duration":0.0032965569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.468282+0000","last_change":"2023-11-08T19:27:49.259555+0000","last_active":"2023-11-08T19:32:28.468282+0000","last_peered":"2023-11-08T19:32:28.468282+0000","last_clean":"2023-11-08T19:32:28.468282+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:32:28.468282+0000","last_undegraded":"2023-11-08T19:32:28.468282+0000","last_fullsized":"2023-11-08T19:32:28.468282+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.652067+0000","last_change":"2023-11-08T19:32:28.652115+0000","last_active":"2023-11-08T19:32:28.652067+0000","last_peered":"2023-11-08T19:32:28.652067+0000","last_clean":"2023-11-08T19:32:28.652067+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:28.652067+0000","last_undegraded":"2023-11-08T19:32:28.652067+0000","last_fullsized":"2023-11-08T19:32:28.652067+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.651902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.651902+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.651902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:39:39.497609+0000","scrub_duration":0.0036776040000000001,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":50,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.046809+0000","last_change":"2023-11-08T19:27:47.803800+0000","last_active":"2023-11-08T19:32:27.046809+0000","last_peered":"2023-11-08T19:32:27.046809+0000","last_clean":"2023-11-08T19:32:27.046809+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:32:27.046809+0000","last_undegraded":"2023-11-08T19:32:27.046809+0000","last_fullsized":"2023-11-08T19:32:27.046809+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":66,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:38.643087+0000","last_change":"2023-11-08T19:32:38.643143+0000","last_active":"2023-11-08T19:32:38.643087+0000","last_peered":"2023-11-08T19:32:38.643087+0000","last_clean":"2023-11-08T19:32:38.643087+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:38.643087+0000","last_undegraded":"2023-11-08T19:32:38.643087+0000","last_fullsized":"2023-11-08T19:32:38.643087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:38.642935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:38.642935+0000","last_clean_scrub_stamp":"2023-11-08T19:32:38.642935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:19.775448+0000","scrub_duration":0.0035460359999999998,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":88,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:54.750163+0000","last_change":"2023-11-08T19:32:54.750163+0000","last_active":"2023-11-08T19:32:54.750163+0000","last_peered":"2023-11-08T19:32:54.750163+0000","last_clean":"2023-11-08T19:32:54.750163+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:54.750163+0000","last_undegraded":"2023-11-08T19:32:54.750163+0000","last_fullsized":"2023-11-08T19:32:54.750163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:54.749958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:54.749958+0000","last_clean_scrub_stamp":"2023-11-08T19:32:54.749958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:37:41.693384+0000","scrub_duration":0.003451188,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:56.878255+0000","last_change":"2023-11-08T19:32:56.878341+0000","last_active":"2023-11-08T19:32:56.878255+0000","last_peered":"2023-11-08T19:32:56.878255+0000","last_clean":"2023-11-08T19:32:56.878255+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:56.878255+0000","last_undegraded":"2023-11-08T19:32:56.878255+0000","last_fullsized":"2023-11-08T19:32:56.878255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:56.878015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:56.878015+0000","last_clean_scrub_stamp":"2023-11-08T19:32:56.878015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:06.790201+0000","scrub_duration":0.0038922980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:01.722245+0000","last_change":"2023-11-08T19:33:01.722291+0000","last_active":"2023-11-08T19:33:01.722245+0000","last_peered":"2023-11-08T19:33:01.722245+0000","last_clean":"2023-11-08T19:33:01.722245+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:33:01.722245+0000","last_undegraded":"2023-11-08T19:33:01.722245+0000","last_fullsized":"2023-11-08T19:33:01.722245+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:01.722092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:01.722092+0000","last_clean_scrub_stamp":"2023-11-08T19:33:01.722092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:39.551131+0000","scrub_duration":0.003627941,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.727558+0000","last_change":"2023-11-08T19:32:27.727558+0000","last_active":"2023-11-08T19:32:27.727558+0000","last_peered":"2023-11-08T19:32:27.727558+0000","last_clean":"2023-11-08T19:32:27.727558+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:27.727558+0000","last_undegraded":"2023-11-08T19:32:27.727558+0000","last_fullsized":"2023-11-08T19:32:27.727558+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:27.727335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:27.727335+0000","last_clean_scrub_stamp":"2023-11-08T19:32:27.727335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:55.266502+0000","scrub_duration":0.0027330219999999999,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:57.645193+0000","last_change":"2023-11-08T19:32:57.645239+0000","last_active":"2023-11-08T19:32:57.645193+0000","last_peered":"2023-11-08T19:32:57.645193+0000","last_clean":"2023-11-08T19:32:57.645193+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:57.645193+0000","last_undegraded":"2023-11-08T19:32:57.645193+0000","last_fullsized":"2023-11-08T19:32:57.645193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:57.645028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:57.645028+0000","last_clean_scrub_stamp":"2023-11-08T19:32:57.645028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:23.575294+0000","scrub_duration":0.0039615750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.625497+0000","last_change":"2023-11-08T19:32:45.625543+0000","last_active":"2023-11-08T19:32:45.625497+0000","last_peered":"2023-11-08T19:32:45.625497+0000","last_clean":"2023-11-08T19:32:45.625497+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:45.625497+0000","last_undegraded":"2023-11-08T19:32:45.625497+0000","last_fullsized":"2023-11-08T19:32:45.625497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.625348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.625348+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.625348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:33.970967+0000","scrub_duration":0.0030484169999999999,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:42.920729+0000","last_change":"2023-11-08T19:32:42.920778+0000","last_active":"2023-11-08T19:32:42.920729+0000","last_peered":"2023-11-08T19:32:42.920729+0000","last_clean":"2023-11-08T19:32:42.920729+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:42.920729+0000","last_undegraded":"2023-11-08T19:32:42.920729+0000","last_fullsized":"2023-11-08T19:32:42.920729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:42.920564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:42.920564+0000","last_clean_scrub_stamp":"2023-11-08T19:32:42.920564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:15:38.876790+0000","scrub_duration":0.0033816290000000001,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.728642+0000","last_change":"2023-11-08T19:33:00.728688+0000","last_active":"2023-11-08T19:33:00.728642+0000","last_peered":"2023-11-08T19:33:00.728642+0000","last_clean":"2023-11-08T19:33:00.728642+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:33:00.728642+0000","last_undegraded":"2023-11-08T19:33:00.728642+0000","last_fullsized":"2023-11-08T19:33:00.728642+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.728493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.728493+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.728493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:39.952049+0000","scrub_duration":0.0036203329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.753355+0000","last_change":"2023-11-08T19:32:30.753400+0000","last_active":"2023-11-08T19:32:30.753355+0000","last_peered":"2023-11-08T19:32:30.753355+0000","last_clean":"2023-11-08T19:32:30.753355+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:30.753355+0000","last_undegraded":"2023-11-08T19:32:30.753355+0000","last_fullsized":"2023-11-08T19:32:30.753355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.753206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.753206+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.753206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:15:08.346214+0000","scrub_duration":0.0030208409999999998,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.821586+0000","last_change":"2023-11-08T19:32:34.821633+0000","last_active":"2023-11-08T19:32:34.821586+0000","last_peered":"2023-11-08T19:32:34.821586+0000","last_clean":"2023-11-08T19:32:34.821586+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:34.821586+0000","last_undegraded":"2023-11-08T19:32:34.821586+0000","last_fullsized":"2023-11-08T19:32:34.821586+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.821364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.821364+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.821364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:07.177396+0000","scrub_duration":0.0040781000000000003,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.703378+0000","last_change":"2023-11-08T19:32:45.703425+0000","last_active":"2023-11-08T19:32:45.703378+0000","last_peered":"2023-11-08T19:32:45.703378+0000","last_clean":"2023-11-08T19:32:45.703378+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:45.703378+0000","last_undegraded":"2023-11-08T19:32:45.703378+0000","last_fullsized":"2023-11-08T19:32:45.703378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.703215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.703215+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.703215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:29.552613+0000","scrub_duration":0.0041161319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.780034+0000","last_change":"2023-11-08T19:32:45.780081+0000","last_active":"2023-11-08T19:32:45.780034+0000","last_peered":"2023-11-08T19:32:45.780034+0000","last_clean":"2023-11-08T19:32:45.780034+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:45.780034+0000","last_undegraded":"2023-11-08T19:32:45.780034+0000","last_fullsized":"2023-11-08T19:32:45.780034+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.779844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.779844+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.779844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:03.630656+0000","scrub_duration":0.003794487,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.874214+0000","last_change":"2023-11-08T19:32:30.874260+0000","last_active":"2023-11-08T19:32:30.874214+0000","last_peered":"2023-11-08T19:32:30.874214+0000","last_clean":"2023-11-08T19:32:30.874214+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:30.874214+0000","last_undegraded":"2023-11-08T19:32:30.874214+0000","last_fullsized":"2023-11-08T19:32:30.874214+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.874063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.874063+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.874063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:33.344505+0000","scrub_duration":0.0032893110000000001,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:39.724808+0000","last_change":"2023-11-08T19:32:39.724860+0000","last_active":"2023-11-08T19:32:39.724808+0000","last_peered":"2023-11-08T19:32:39.724808+0000","last_clean":"2023-11-08T19:32:39.724808+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:39.724808+0000","last_undegraded":"2023-11-08T19:32:39.724808+0000","last_fullsized":"2023-11-08T19:32:39.724808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:39.724643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:39.724643+0000","last_clean_scrub_stamp":"2023-11-08T19:32:39.724643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:16.175462+0000","scrub_duration":0.0033653490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":66,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.588081+0000","last_change":"2023-11-08T19:32:35.588132+0000","last_active":"2023-11-08T19:32:35.588081+0000","last_peered":"2023-11-08T19:32:35.588081+0000","last_clean":"2023-11-08T19:32:35.588081+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:35.588081+0000","last_undegraded":"2023-11-08T19:32:35.588081+0000","last_fullsized":"2023-11-08T19:32:35.588081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.587924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.587924+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.587924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:11:25.638388+0000","scrub_duration":0.003687957,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.650654+0000","last_change":"2023-11-08T19:32:29.650701+0000","last_active":"2023-11-08T19:32:29.650654+0000","last_peered":"2023-11-08T19:32:29.650654+0000","last_clean":"2023-11-08T19:32:29.650654+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:29.650654+0000","last_undegraded":"2023-11-08T19:32:29.650654+0000","last_fullsized":"2023-11-08T19:32:29.650654+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.650428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.650428+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.650428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:44:57.186304+0000","scrub_duration":0.0035263999999999998,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.109947+0000","last_change":"2023-11-08T19:27:47.810555+0000","last_active":"2023-11-08T19:32:17.109947+0000","last_peered":"2023-11-08T19:32:17.109947+0000","last_clean":"2023-11-08T19:32:17.109947+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:32:17.109947+0000","last_undegraded":"2023-11-08T19:32:17.109947+0000","last_fullsized":"2023-11-08T19:32:17.109947+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.645920+0000","last_change":"2023-11-08T19:32:46.645975+0000","last_active":"2023-11-08T19:32:46.645920+0000","last_peered":"2023-11-08T19:32:46.645920+0000","last_clean":"2023-11-08T19:32:46.645920+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:46.645920+0000","last_undegraded":"2023-11-08T19:32:46.645920+0000","last_fullsized":"2023-11-08T19:32:46.645920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.645739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.645739+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.645739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:57.261846+0000","scrub_duration":0.0036408650000000001,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.559388+0000","last_change":"2023-11-08T19:32:41.559435+0000","last_active":"2023-11-08T19:32:41.559388+0000","last_peered":"2023-11-08T19:32:41.559388+0000","last_clean":"2023-11-08T19:32:41.559388+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:41.559388+0000","last_undegraded":"2023-11-08T19:32:41.559388+0000","last_fullsized":"2023-11-08T19:32:41.559388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.559226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.559226+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.559226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:42.827112+0000","scrub_duration":0.0037686489999999998,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.601945+0000","last_change":"2023-11-08T19:33:00.601992+0000","last_active":"2023-11-08T19:33:00.601945+0000","last_peered":"2023-11-08T19:33:00.601945+0000","last_clean":"2023-11-08T19:33:00.601945+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:33:00.601945+0000","last_undegraded":"2023-11-08T19:33:00.601945+0000","last_fullsized":"2023-11-08T19:33:00.601945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.601784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.601784+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.601784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:27.398867+0000","scrub_duration":0.0037468319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.107795+0000","last_change":"2023-11-08T19:27:49.259191+0000","last_active":"2023-11-08T19:32:17.107795+0000","last_peered":"2023-11-08T19:32:17.107795+0000","last_clean":"2023-11-08T19:32:17.107795+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:32:17.107795+0000","last_undegraded":"2023-11-08T19:32:17.107795+0000","last_fullsized":"2023-11-08T19:32:17.107795+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.675611+0000","last_change":"2023-11-08T19:32:28.675655+0000","last_active":"2023-11-08T19:32:28.675611+0000","last_peered":"2023-11-08T19:32:28.675611+0000","last_clean":"2023-11-08T19:32:28.675611+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:28.675611+0000","last_undegraded":"2023-11-08T19:32:28.675611+0000","last_fullsized":"2023-11-08T19:32:28.675611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.675467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.675467+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.675467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:32:57.443158+0000","scrub_duration":0.00292019,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.627590+0000","last_change":"2023-11-08T19:33:00.627635+0000","last_active":"2023-11-08T19:33:00.627590+0000","last_peered":"2023-11-08T19:33:00.627590+0000","last_clean":"2023-11-08T19:33:00.627590+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:33:00.627590+0000","last_undegraded":"2023-11-08T19:33:00.627590+0000","last_fullsized":"2023-11-08T19:33:00.627590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.627409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.627409+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.627409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:27.424492+0000","scrub_duration":0.0036601170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.780194+0000","last_change":"2023-11-08T19:32:46.780194+0000","last_active":"2023-11-08T19:32:46.780194+0000","last_peered":"2023-11-08T19:32:46.780194+0000","last_clean":"2023-11-08T19:32:46.780194+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:46.780194+0000","last_undegraded":"2023-11-08T19:32:46.780194+0000","last_fullsized":"2023-11-08T19:32:46.780194+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.779991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.779991+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.779991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:08.986106+0000","scrub_duration":0.0036491200000000001,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.589623+0000","last_change":"2023-11-08T19:32:44.589675+0000","last_active":"2023-11-08T19:32:44.589623+0000","last_peered":"2023-11-08T19:32:44.589623+0000","last_clean":"2023-11-08T19:32:44.589623+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:44.589623+0000","last_undegraded":"2023-11-08T19:32:44.589623+0000","last_fullsized":"2023-11-08T19:32:44.589623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.589412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.589412+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.589412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:03.472044+0000","scrub_duration":0.0032550370000000001,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.468294+0000","last_change":"2023-11-08T19:27:47.807659+0000","last_active":"2023-11-08T19:32:28.468294+0000","last_peered":"2023-11-08T19:32:28.468294+0000","last_clean":"2023-11-08T19:32:28.468294+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:32:28.468294+0000","last_undegraded":"2023-11-08T19:32:28.468294+0000","last_fullsized":"2023-11-08T19:32:28.468294+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:51.893384+0000","last_change":"2023-11-08T19:32:51.893433+0000","last_active":"2023-11-08T19:32:51.893384+0000","last_peered":"2023-11-08T19:32:51.893384+0000","last_clean":"2023-11-08T19:32:51.893384+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:51.893384+0000","last_undegraded":"2023-11-08T19:32:51.893384+0000","last_fullsized":"2023-11-08T19:32:51.893384+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:51.893226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:51.893226+0000","last_clean_scrub_stamp":"2023-11-08T19:32:51.893226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:39.105087+0000","scrub_duration":0.003348733,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.663302+0000","last_change":"2023-11-08T19:32:35.663353+0000","last_active":"2023-11-08T19:32:35.663302+0000","last_peered":"2023-11-08T19:32:35.663302+0000","last_clean":"2023-11-08T19:32:35.663302+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:35.663302+0000","last_undegraded":"2023-11-08T19:32:35.663302+0000","last_fullsized":"2023-11-08T19:32:35.663302+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.663149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.663149+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.663149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:08:01.980513+0000","scrub_duration":0.0040296780000000001,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:52.620414+0000","last_change":"2023-11-08T19:32:52.620458+0000","last_active":"2023-11-08T19:32:52.620414+0000","last_peered":"2023-11-08T19:32:52.620414+0000","last_clean":"2023-11-08T19:32:52.620414+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:52.620414+0000","last_undegraded":"2023-11-08T19:32:52.620414+0000","last_fullsized":"2023-11-08T19:32:52.620414+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:52.620263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:52.620263+0000","last_clean_scrub_stamp":"2023-11-08T19:32:52.620263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:19.281606+0000","scrub_duration":0.0038208080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.691774+0000","last_change":"2023-11-08T19:32:44.691774+0000","last_active":"2023-11-08T19:32:44.691774+0000","last_peered":"2023-11-08T19:32:44.691774+0000","last_clean":"2023-11-08T19:32:44.691774+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:44.691774+0000","last_undegraded":"2023-11-08T19:32:44.691774+0000","last_fullsized":"2023-11-08T19:32:44.691774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.691532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.691532+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.691532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:22.520571+0000","scrub_duration":0.0039118929999999996,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.500518+0000","last_change":"2023-11-08T19:32:34.500610+0000","last_active":"2023-11-08T19:32:34.500518+0000","last_peered":"2023-11-08T19:32:34.500518+0000","last_clean":"2023-11-08T19:32:34.500518+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:34.500518+0000","last_undegraded":"2023-11-08T19:32:34.500518+0000","last_fullsized":"2023-11-08T19:32:34.500518+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.500295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.500295+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.500295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:07:20.761692+0000","scrub_duration":0.0032186850000000002,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:47.682570+0000","last_change":"2023-11-08T19:32:47.682570+0000","last_active":"2023-11-08T19:32:47.682570+0000","last_peered":"2023-11-08T19:32:47.682570+0000","last_clean":"2023-11-08T19:32:47.682570+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:47.682570+0000","last_undegraded":"2023-11-08T19:32:47.682570+0000","last_fullsized":"2023-11-08T19:32:47.682570+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:47.682372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:47.682372+0000","last_clean_scrub_stamp":"2023-11-08T19:32:47.682372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:34:23.371271+0000","scrub_duration":0.0034803009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:48.606882+0000","last_change":"2023-11-08T19:32:48.606932+0000","last_active":"2023-11-08T19:32:48.606882+0000","last_peered":"2023-11-08T19:32:48.606882+0000","last_clean":"2023-11-08T19:32:48.606882+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:48.606882+0000","last_undegraded":"2023-11-08T19:32:48.606882+0000","last_fullsized":"2023-11-08T19:32:48.606882+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:48.606714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:48.606714+0000","last_clean_scrub_stamp":"2023-11-08T19:32:48.606714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:42.003937+0000","scrub_duration":0.003633795,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.631351+0000","last_change":"2023-11-08T19:32:26.631371+0000","last_active":"2023-11-08T19:32:26.631351+0000","last_peered":"2023-11-08T19:32:26.631351+0000","last_clean":"2023-11-08T19:32:26.631351+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:26.631351+0000","last_undegraded":"2023-11-08T19:32:26.631351+0000","last_fullsized":"2023-11-08T19:32:26.631351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.631277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:31:19.710068+0000","scrub_duration":0.0020387719999999999,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:51.696393+0000","last_change":"2023-11-08T19:32:51.696440+0000","last_active":"2023-11-08T19:32:51.696393+0000","last_peered":"2023-11-08T19:32:51.696393+0000","last_clean":"2023-11-08T19:32:51.696393+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:51.696393+0000","last_undegraded":"2023-11-08T19:32:51.696393+0000","last_fullsized":"2023-11-08T19:32:51.696393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:51.696242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:51.696242+0000","last_clean_scrub_stamp":"2023-11-08T19:32:51.696242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:47.507076+0000","scrub_duration":0.003286059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:32.749081+0000","last_change":"2023-11-08T19:32:32.749081+0000","last_active":"2023-11-08T19:32:32.749081+0000","last_peered":"2023-11-08T19:32:32.749081+0000","last_clean":"2023-11-08T19:32:32.749081+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:32.749081+0000","last_undegraded":"2023-11-08T19:32:32.749081+0000","last_fullsized":"2023-11-08T19:32:32.749081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:32.748790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:32.748790+0000","last_clean_scrub_stamp":"2023-11-08T19:32:32.748790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:07.459426+0000","scrub_duration":0.0049107079999999997,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.898760+0000","last_change":"2023-11-08T19:32:29.898810+0000","last_active":"2023-11-08T19:32:29.898760+0000","last_peered":"2023-11-08T19:32:29.898760+0000","last_clean":"2023-11-08T19:32:29.898760+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:29.898760+0000","last_undegraded":"2023-11-08T19:32:29.898760+0000","last_fullsized":"2023-11-08T19:32:29.898760+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.898611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.898611+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.898611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:10.217632+0000","scrub_duration":0.0034633910000000001,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:51.646194+0000","last_change":"2023-11-08T19:32:51.646238+0000","last_active":"2023-11-08T19:32:51.646194+0000","last_peered":"2023-11-08T19:32:51.646194+0000","last_clean":"2023-11-08T19:32:51.646194+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:51.646194+0000","last_undegraded":"2023-11-08T19:32:51.646194+0000","last_fullsized":"2023-11-08T19:32:51.646194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:51.646032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:51.646032+0000","last_clean_scrub_stamp":"2023-11-08T19:32:51.646032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:20.413723+0000","scrub_duration":0.0038116949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:43.834776+0000","last_change":"2023-11-08T19:32:43.834821+0000","last_active":"2023-11-08T19:32:43.834776+0000","last_peered":"2023-11-08T19:32:43.834776+0000","last_clean":"2023-11-08T19:32:43.834776+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:43.834776+0000","last_undegraded":"2023-11-08T19:32:43.834776+0000","last_fullsized":"2023-11-08T19:32:43.834776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:43.834622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:43.834622+0000","last_clean_scrub_stamp":"2023-11-08T19:32:43.834622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:57:53.189317+0000","scrub_duration":0.0032581189999999999,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:59.705826+0000","last_change":"2023-11-08T19:32:59.705879+0000","last_active":"2023-11-08T19:32:59.705826+0000","last_peered":"2023-11-08T19:32:59.705826+0000","last_clean":"2023-11-08T19:32:59.705826+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:59.705826+0000","last_undegraded":"2023-11-08T19:32:59.705826+0000","last_fullsized":"2023-11-08T19:32:59.705826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:59.705654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:59.705654+0000","last_clean_scrub_stamp":"2023-11-08T19:32:59.705654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:00.973540+0000","scrub_duration":0.0040140590000000004,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:01.608751+0000","last_change":"2023-11-08T19:33:01.608805+0000","last_active":"2023-11-08T19:33:01.608751+0000","last_peered":"2023-11-08T19:33:01.608751+0000","last_clean":"2023-11-08T19:33:01.608751+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:33:01.608751+0000","last_undegraded":"2023-11-08T19:33:01.608751+0000","last_fullsized":"2023-11-08T19:33:01.608751+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:01.608590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:01.608590+0000","last_clean_scrub_stamp":"2023-11-08T19:33:01.608590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:19.044577+0000","scrub_duration":0.0034235480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.707482+0000","last_change":"2023-11-08T19:32:35.707531+0000","last_active":"2023-11-08T19:32:35.707482+0000","last_peered":"2023-11-08T19:32:35.707482+0000","last_clean":"2023-11-08T19:32:35.707482+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:35.707482+0000","last_undegraded":"2023-11-08T19:32:35.707482+0000","last_fullsized":"2023-11-08T19:32:35.707482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.707332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.707332+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.707332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:36:47.701015+0000","scrub_duration":0.0033186940000000001,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:50.853276+0000","last_change":"2023-11-08T19:32:50.853323+0000","last_active":"2023-11-08T19:32:50.853276+0000","last_peered":"2023-11-08T19:32:50.853276+0000","last_clean":"2023-11-08T19:32:50.853276+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:50.853276+0000","last_undegraded":"2023-11-08T19:32:50.853276+0000","last_fullsized":"2023-11-08T19:32:50.853276+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:50.853102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:50.853102+0000","last_clean_scrub_stamp":"2023-11-08T19:32:50.853102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:45:18.388978+0000","scrub_duration":0.004043474,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:58.588361+0000","last_change":"2023-11-08T19:32:58.588408+0000","last_active":"2023-11-08T19:32:58.588361+0000","last_peered":"2023-11-08T19:32:58.588361+0000","last_clean":"2023-11-08T19:32:58.588361+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:58.588361+0000","last_undegraded":"2023-11-08T19:32:58.588361+0000","last_fullsized":"2023-11-08T19:32:58.588361+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:58.588189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:58.588189+0000","last_clean_scrub_stamp":"2023-11-08T19:32:58.588189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:24.518455+0000","scrub_duration":0.00330525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.755805+0000","last_change":"2023-11-08T19:32:36.755856+0000","last_active":"2023-11-08T19:32:36.755805+0000","last_peered":"2023-11-08T19:32:36.755805+0000","last_clean":"2023-11-08T19:32:36.755805+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:36.755805+0000","last_undegraded":"2023-11-08T19:32:36.755805+0000","last_fullsized":"2023-11-08T19:32:36.755805+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.755645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.755645+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.755645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:34.826462+0000","scrub_duration":0.0037998400000000001,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.726327+0000","last_change":"2023-11-08T19:33:00.726374+0000","last_active":"2023-11-08T19:33:00.726327+0000","last_peered":"2023-11-08T19:33:00.726327+0000","last_clean":"2023-11-08T19:33:00.726327+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:33:00.726327+0000","last_undegraded":"2023-11-08T19:33:00.726327+0000","last_fullsized":"2023-11-08T19:33:00.726327+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.726176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.726176+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.726176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:36:33.504769+0000","scrub_duration":0.0035468000000000001,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:05.707629+0000","last_change":"2023-11-08T19:33:05.707676+0000","last_active":"2023-11-08T19:33:05.707629+0000","last_peered":"2023-11-08T19:33:05.707629+0000","last_clean":"2023-11-08T19:33:05.707629+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:33:05.707629+0000","last_undegraded":"2023-11-08T19:33:05.707629+0000","last_fullsized":"2023-11-08T19:33:05.707629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:05.707419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:05.707419+0000","last_clean_scrub_stamp":"2023-11-08T19:33:05.707419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:24.111269+0000","scrub_duration":0.0038339580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.901318+0000","last_change":"2023-11-08T19:32:29.901364+0000","last_active":"2023-11-08T19:32:29.901318+0000","last_peered":"2023-11-08T19:32:29.901318+0000","last_clean":"2023-11-08T19:32:29.901318+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:29.901318+0000","last_undegraded":"2023-11-08T19:32:29.901318+0000","last_fullsized":"2023-11-08T19:32:29.901318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.901159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.901159+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.901159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:18.596622+0000","scrub_duration":0.0039081070000000001,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.887981+0000","last_change":"2023-11-08T19:32:27.887999+0000","last_active":"2023-11-08T19:32:27.887981+0000","last_peered":"2023-11-08T19:32:27.887981+0000","last_clean":"2023-11-08T19:32:27.887981+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:27.887981+0000","last_undegraded":"2023-11-08T19:32:27.887981+0000","last_fullsized":"2023-11-08T19:32:27.887981+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:27.887914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:27.887914+0000","last_clean_scrub_stamp":"2023-11-08T19:32:27.887914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:54.774642+0000","scrub_duration":0.0022926560000000001,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.103630+0000","last_change":"2023-11-08T19:27:47.810001+0000","last_active":"2023-11-08T19:32:17.103630+0000","last_peered":"2023-11-08T19:32:17.103630+0000","last_clean":"2023-11-08T19:32:17.103630+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:32:17.103630+0000","last_undegraded":"2023-11-08T19:32:17.103630+0000","last_fullsized":"2023-11-08T19:32:17.103630+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.104373+0000","last_change":"2023-11-08T19:27:49.257347+0000","last_active":"2023-11-08T19:32:17.104373+0000","last_peered":"2023-11-08T19:32:17.104373+0000","last_clean":"2023-11-08T19:32:17.104373+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:32:17.104373+0000","last_undegraded":"2023-11-08T19:32:17.104373+0000","last_fullsized":"2023-11-08T19:32:17.104373+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.981855+0000","last_change":"2023-11-08T19:32:26.981899+0000","last_active":"2023-11-08T19:32:26.981855+0000","last_peered":"2023-11-08T19:32:26.981855+0000","last_clean":"2023-11-08T19:32:26.981855+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:26.981855+0000","last_undegraded":"2023-11-08T19:32:26.981855+0000","last_fullsized":"2023-11-08T19:32:26.981855+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.981756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.981756+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.981756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:20.096248+0000","scrub_duration":0.0030939209999999999,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:54.895016+0000","last_change":"2023-11-08T19:32:54.895068+0000","last_active":"2023-11-08T19:32:54.895016+0000","last_peered":"2023-11-08T19:32:54.895016+0000","last_clean":"2023-11-08T19:32:54.895016+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:54.895016+0000","last_undegraded":"2023-11-08T19:32:54.895016+0000","last_fullsized":"2023-11-08T19:32:54.895016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:54.894835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:54.894835+0000","last_clean_scrub_stamp":"2023-11-08T19:32:54.894835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:08.993720+0000","scrub_duration":0.0035580619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:49.643705+0000","last_change":"2023-11-08T19:32:49.643753+0000","last_active":"2023-11-08T19:32:49.643705+0000","last_peered":"2023-11-08T19:32:49.643705+0000","last_clean":"2023-11-08T19:32:49.643705+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:49.643705+0000","last_undegraded":"2023-11-08T19:32:49.643705+0000","last_fullsized":"2023-11-08T19:32:49.643705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:49.643546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:49.643546+0000","last_clean_scrub_stamp":"2023-11-08T19:32:49.643546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:18.411237+0000","scrub_duration":0.0038580799999999998,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:39.855542+0000","last_change":"2023-11-08T19:32:39.855592+0000","last_active":"2023-11-08T19:32:39.855542+0000","last_peered":"2023-11-08T19:32:39.855542+0000","last_clean":"2023-11-08T19:32:39.855542+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:39.855542+0000","last_undegraded":"2023-11-08T19:32:39.855542+0000","last_fullsized":"2023-11-08T19:32:39.855542+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:39.855386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:39.855386+0000","last_clean_scrub_stamp":"2023-11-08T19:32:39.855386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:11.250998+0000","scrub_duration":0.0035043140000000001,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.598433+0000","last_change":"2023-11-08T19:32:44.598517+0000","last_active":"2023-11-08T19:32:44.598433+0000","last_peered":"2023-11-08T19:32:44.598433+0000","last_clean":"2023-11-08T19:32:44.598433+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:44.598433+0000","last_undegraded":"2023-11-08T19:32:44.598433+0000","last_fullsized":"2023-11-08T19:32:44.598433+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.598277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.598277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.598277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:33.293740+0000","scrub_duration":0.0035716250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.848482+0000","last_change":"2023-11-08T19:32:24.848528+0000","last_active":"2023-11-08T19:32:24.848482+0000","last_peered":"2023-11-08T19:32:24.848482+0000","last_clean":"2023-11-08T19:32:24.848482+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:24.848482+0000","last_undegraded":"2023-11-08T19:32:24.848482+0000","last_fullsized":"2023-11-08T19:32:24.848482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_clean_scrub_stamp":"2023-11-08T19:32:24.848327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:22.967345+0000","scrub_duration":0.0035992749999999999,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.019582+0000","last_change":"2023-11-08T19:32:28.019603+0000","last_active":"2023-11-08T19:32:28.019582+0000","last_peered":"2023-11-08T19:32:28.019582+0000","last_clean":"2023-11-08T19:32:28.019582+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:28.019582+0000","last_undegraded":"2023-11-08T19:32:28.019582+0000","last_fullsized":"2023-11-08T19:32:28.019582+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.019513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.019513+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.019513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:18.418880+0000","scrub_duration":0.0027169690000000001,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.109057+0000","last_change":"2023-11-08T19:27:47.806842+0000","last_active":"2023-11-08T19:32:17.109057+0000","last_peered":"2023-11-08T19:32:17.109057+0000","last_clean":"2023-11-08T19:32:17.109057+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:32:17.109057+0000","last_undegraded":"2023-11-08T19:32:17.109057+0000","last_fullsized":"2023-11-08T19:32:17.109057+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:40.655839+0000","last_change":"2023-11-08T19:32:40.655884+0000","last_active":"2023-11-08T19:32:40.655839+0000","last_peered":"2023-11-08T19:32:40.655839+0000","last_clean":"2023-11-08T19:32:40.655839+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:40.655839+0000","last_undegraded":"2023-11-08T19:32:40.655839+0000","last_fullsized":"2023-11-08T19:32:40.655839+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:40.655688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:40.655688+0000","last_clean_scrub_stamp":"2023-11-08T19:32:40.655688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:41.923574+0000","scrub_duration":0.003438927,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:25.703825+0000","last_change":"2023-11-08T19:32:25.703850+0000","last_active":"2023-11-08T19:32:25.703825+0000","last_peered":"2023-11-08T19:32:25.703825+0000","last_clean":"2023-11-08T19:32:25.703825+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:25.703825+0000","last_undegraded":"2023-11-08T19:32:25.703825+0000","last_fullsized":"2023-11-08T19:32:25.703825+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_clean_scrub_stamp":"2023-11-08T19:32:25.703734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:40.388881+0000","scrub_duration":0.0027277820000000002,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:52.662805+0000","last_change":"2023-11-08T19:32:52.662835+0000","last_active":"2023-11-08T19:32:52.662805+0000","last_peered":"2023-11-08T19:32:52.662805+0000","last_clean":"2023-11-08T19:32:52.662805+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:52.662805+0000","last_undegraded":"2023-11-08T19:32:52.662805+0000","last_fullsized":"2023-11-08T19:32:52.662805+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:52.662726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:52.662726+0000","last_clean_scrub_stamp":"2023-11-08T19:32:52.662726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:19.324070+0000","scrub_duration":0.003021709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.950835+0000","last_change":"2023-11-08T19:32:30.950919+0000","last_active":"2023-11-08T19:32:30.950835+0000","last_peered":"2023-11-08T19:32:30.950835+0000","last_clean":"2023-11-08T19:32:30.950835+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:30.950835+0000","last_undegraded":"2023-11-08T19:32:30.950835+0000","last_fullsized":"2023-11-08T19:32:30.950835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.950623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.950623+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.950623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:53.223016+0000","scrub_duration":0.0042637869999999998,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.657467+0000","last_change":"2023-11-08T19:32:30.657516+0000","last_active":"2023-11-08T19:32:30.657467+0000","last_peered":"2023-11-08T19:32:30.657467+0000","last_clean":"2023-11-08T19:32:30.657467+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:30.657467+0000","last_undegraded":"2023-11-08T19:32:30.657467+0000","last_fullsized":"2023-11-08T19:32:30.657467+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.657305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.657305+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.657305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:01:57.318648+0000","scrub_duration":0.003567045,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":49,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:21.813461+0000","last_change":"2023-11-08T19:27:47.816577+0000","last_active":"2023-11-08T19:32:21.813461+0000","last_peered":"2023-11-08T19:32:21.813461+0000","last_clean":"2023-11-08T19:32:21.813461+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:32:21.813461+0000","last_undegraded":"2023-11-08T19:32:21.813461+0000","last_fullsized":"2023-11-08T19:32:21.813461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:46.764558+0000","last_clean_scrub_stamp":"2023-11-08T19:27:46.764558+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:52.908195+0000","last_change":"2023-11-08T19:32:52.908241+0000","last_active":"2023-11-08T19:32:52.908195+0000","last_peered":"2023-11-08T19:32:52.908195+0000","last_clean":"2023-11-08T19:32:52.908195+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:52.908195+0000","last_undegraded":"2023-11-08T19:32:52.908195+0000","last_fullsized":"2023-11-08T19:32:52.908195+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:52.908042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:52.908042+0000","last_clean_scrub_stamp":"2023-11-08T19:32:52.908042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:27.527625+0000","scrub_duration":0.003437729,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.652935+0000","last_change":"2023-11-08T19:32:46.652935+0000","last_active":"2023-11-08T19:32:46.652935+0000","last_peered":"2023-11-08T19:32:46.652935+0000","last_clean":"2023-11-08T19:32:46.652935+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:46.652935+0000","last_undegraded":"2023-11-08T19:32:46.652935+0000","last_fullsized":"2023-11-08T19:32:46.652935+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.652731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.652731+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.652731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:57.268837+0000","scrub_duration":0.0036855339999999999,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:54.587249+0000","last_change":"2023-11-08T19:32:54.587295+0000","last_active":"2023-11-08T19:32:54.587249+0000","last_peered":"2023-11-08T19:32:54.587249+0000","last_clean":"2023-11-08T19:32:54.587249+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:54.587249+0000","last_undegraded":"2023-11-08T19:32:54.587249+0000","last_fullsized":"2023-11-08T19:32:54.587249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:54.587090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:54.587090+0000","last_clean_scrub_stamp":"2023-11-08T19:32:54.587090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:17.204659+0000","scrub_duration":0.003566053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:17.108429+0000","last_change":"2023-11-08T19:27:49.259848+0000","last_active":"2023-11-08T19:32:17.108429+0000","last_peered":"2023-11-08T19:32:17.108429+0000","last_clean":"2023-11-08T19:32:17.108429+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:32:17.108429+0000","last_undegraded":"2023-11-08T19:32:17.108429+0000","last_fullsized":"2023-11-08T19:32:17.108429+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:27:45.750386+0000","last_clean_scrub_stamp":"2023-11-08T19:27:45.750386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:42.878282+0000","last_change":"2023-11-08T19:32:42.878282+0000","last_active":"2023-11-08T19:32:42.878282+0000","last_peered":"2023-11-08T19:32:42.878282+0000","last_clean":"2023-11-08T19:32:42.878282+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:42.878282+0000","last_undegraded":"2023-11-08T19:32:42.878282+0000","last_fullsized":"2023-11-08T19:32:42.878282+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:42.878052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:42.878052+0000","last_clean_scrub_stamp":"2023-11-08T19:32:42.878052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:41.200867+0000","scrub_duration":0.003533516,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:56.571922+0000","last_change":"2023-11-08T19:32:56.571973+0000","last_active":"2023-11-08T19:32:56.571922+0000","last_peered":"2023-11-08T19:32:56.571922+0000","last_clean":"2023-11-08T19:32:56.571922+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:56.571922+0000","last_undegraded":"2023-11-08T19:32:56.571922+0000","last_fullsized":"2023-11-08T19:32:56.571922+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:56.571754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:56.571754+0000","last_clean_scrub_stamp":"2023-11-08T19:32:56.571754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:19.189323+0000","scrub_duration":0.004365865,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:50.888488+0000","last_change":"2023-11-08T19:32:50.888537+0000","last_active":"2023-11-08T19:32:50.888488+0000","last_peered":"2023-11-08T19:32:50.888488+0000","last_clean":"2023-11-08T19:32:50.888488+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:50.888488+0000","last_undegraded":"2023-11-08T19:32:50.888488+0000","last_fullsized":"2023-11-08T19:32:50.888488+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:50.888333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:50.888333+0000","last_clean_scrub_stamp":"2023-11-08T19:32:50.888333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:40.360814+0000","scrub_duration":0.003364494,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:47.920708+0000","last_change":"2023-11-08T19:32:47.920754+0000","last_active":"2023-11-08T19:32:47.920708+0000","last_peered":"2023-11-08T19:32:47.920708+0000","last_clean":"2023-11-08T19:32:47.920708+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:47.920708+0000","last_undegraded":"2023-11-08T19:32:47.920708+0000","last_fullsized":"2023-11-08T19:32:47.920708+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:47.920450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:47.920450+0000","last_clean_scrub_stamp":"2023-11-08T19:32:47.920450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:28.639203+0000","scrub_duration":0.003623958,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.633079+0000","last_change":"2023-11-08T19:32:41.633132+0000","last_active":"2023-11-08T19:32:41.633079+0000","last_peered":"2023-11-08T19:32:41.633079+0000","last_clean":"2023-11-08T19:32:41.633079+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:41.633079+0000","last_undegraded":"2023-11-08T19:32:41.633079+0000","last_fullsized":"2023-11-08T19:32:41.633079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.632927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.632927+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.632927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:36.100474+0000","scrub_duration":0.0039924180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.088025+0000","last_change":"2023-11-08T19:32:30.088072+0000","last_active":"2023-11-08T19:32:30.088025+0000","last_peered":"2023-11-08T19:32:30.088025+0000","last_clean":"2023-11-08T19:32:30.088025+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:30.088025+0000","last_undegraded":"2023-11-08T19:32:30.088025+0000","last_fullsized":"2023-11-08T19:32:30.088025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.087841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.087841+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.087841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:05.489237+0000","scrub_duration":0.003235796,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":92,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.685883+0000","last_change":"2023-11-08T19:32:34.685883+0000","last_active":"2023-11-08T19:32:34.685883+0000","last_peered":"2023-11-08T19:32:34.685883+0000","last_clean":"2023-11-08T19:32:34.685883+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:34.685883+0000","last_undegraded":"2023-11-08T19:32:34.685883+0000","last_fullsized":"2023-11-08T19:32:34.685883+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.685687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.685687+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.685687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:39.814571+0000","scrub_duration":0.0041719410000000002,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.057312+0000","last_change":"2023-11-08T19:32:36.057360+0000","last_active":"2023-11-08T19:32:36.057312+0000","last_peered":"2023-11-08T19:32:36.057312+0000","last_clean":"2023-11-08T19:32:36.057312+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:36.057312+0000","last_undegraded":"2023-11-08T19:32:36.057312+0000","last_fullsized":"2023-11-08T19:32:36.057312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.057153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.057153+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.057153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:57:50.126042+0000","scrub_duration":0.0035364089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:43.628590+0000","last_change":"2023-11-08T19:32:43.628647+0000","last_active":"2023-11-08T19:32:43.628590+0000","last_peered":"2023-11-08T19:32:43.628590+0000","last_clean":"2023-11-08T19:32:43.628590+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:43.628590+0000","last_undegraded":"2023-11-08T19:32:43.628590+0000","last_fullsized":"2023-11-08T19:32:43.628590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:43.628419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:43.628419+0000","last_clean_scrub_stamp":"2023-11-08T19:32:43.628419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:42:45.712641+0000","scrub_duration":0.0032515249999999999,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":92,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.618782+0000","last_change":"2023-11-08T19:32:30.618782+0000","last_active":"2023-11-08T19:32:30.618782+0000","last_peered":"2023-11-08T19:32:30.618782+0000","last_clean":"2023-11-08T19:32:30.618782+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:30.618782+0000","last_undegraded":"2023-11-08T19:32:30.618782+0000","last_fullsized":"2023-11-08T19:32:30.618782+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.618568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.618568+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.618568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:54:51.352530+0000","scrub_duration":0.0037122639999999998,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:40.632718+0000","last_change":"2023-11-08T19:32:40.632765+0000","last_active":"2023-11-08T19:32:40.632718+0000","last_peered":"2023-11-08T19:32:40.632718+0000","last_clean":"2023-11-08T19:32:40.632718+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:40.632718+0000","last_undegraded":"2023-11-08T19:32:40.632718+0000","last_fullsized":"2023-11-08T19:32:40.632718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:40.632500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:40.632500+0000","last_clean_scrub_stamp":"2023-11-08T19:32:40.632500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:28:55.639010+0000","scrub_duration":0.0035395969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.613815+0000","last_change":"2023-11-08T19:32:41.613901+0000","last_active":"2023-11-08T19:32:41.613815+0000","last_peered":"2023-11-08T19:32:41.613815+0000","last_clean":"2023-11-08T19:32:41.613815+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:41.613815+0000","last_undegraded":"2023-11-08T19:32:41.613815+0000","last_fullsized":"2023-11-08T19:32:41.613815+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.613660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.613660+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.613660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:59.049648+0000","scrub_duration":0.0031596810000000001,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.662271+0000","last_change":"2023-11-08T19:32:36.662271+0000","last_active":"2023-11-08T19:32:36.662271+0000","last_peered":"2023-11-08T19:32:36.662271+0000","last_clean":"2023-11-08T19:32:36.662271+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:36.662271+0000","last_undegraded":"2023-11-08T19:32:36.662271+0000","last_fullsized":"2023-11-08T19:32:36.662271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.662065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.662065+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.662065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:59.279634+0000","scrub_duration":0.0035784039999999999,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:33.011945+0000","last_change":"2023-11-08T19:32:33.011999+0000","last_active":"2023-11-08T19:32:33.011945+0000","last_peered":"2023-11-08T19:32:33.011945+0000","last_clean":"2023-11-08T19:32:33.011945+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:33.011945+0000","last_undegraded":"2023-11-08T19:32:33.011945+0000","last_fullsized":"2023-11-08T19:32:33.011945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:33.011743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:33.011743+0000","last_clean_scrub_stamp":"2023-11-08T19:32:33.011743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:06.887936+0000","scrub_duration":0.0036639329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.051468+0000","last_change":"2023-11-08T19:32:29.051518+0000","last_active":"2023-11-08T19:32:29.051468+0000","last_peered":"2023-11-08T19:32:29.051468+0000","last_clean":"2023-11-08T19:32:29.051468+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:29.051468+0000","last_undegraded":"2023-11-08T19:32:29.051468+0000","last_fullsized":"2023-11-08T19:32:29.051468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.051307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.051307+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.051307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:38.939562+0000","scrub_duration":0.0038472430000000002,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:37.646449+0000","last_change":"2023-11-08T19:32:37.646449+0000","last_active":"2023-11-08T19:32:37.646449+0000","last_peered":"2023-11-08T19:32:37.646449+0000","last_clean":"2023-11-08T19:32:37.646449+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:37.646449+0000","last_undegraded":"2023-11-08T19:32:37.646449+0000","last_fullsized":"2023-11-08T19:32:37.646449+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:37.646227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:37.646227+0000","last_clean_scrub_stamp":"2023-11-08T19:32:37.646227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:07.065709+0000","scrub_duration":0.0035584530000000001,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:57.687729+0000","last_change":"2023-11-08T19:32:57.687775+0000","last_active":"2023-11-08T19:32:57.687729+0000","last_peered":"2023-11-08T19:32:57.687729+0000","last_clean":"2023-11-08T19:32:57.687729+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:57.687729+0000","last_undegraded":"2023-11-08T19:32:57.687729+0000","last_fullsized":"2023-11-08T19:32:57.687729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:57.687571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:57.687571+0000","last_clean_scrub_stamp":"2023-11-08T19:32:57.687571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:43.215813+0000","scrub_duration":0.0041095289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:48.677527+0000","last_change":"2023-11-08T19:32:48.677578+0000","last_active":"2023-11-08T19:32:48.677527+0000","last_peered":"2023-11-08T19:32:48.677527+0000","last_clean":"2023-11-08T19:32:48.677527+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:48.677527+0000","last_undegraded":"2023-11-08T19:32:48.677527+0000","last_fullsized":"2023-11-08T19:32:48.677527+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:48.677329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:48.677329+0000","last_clean_scrub_stamp":"2023-11-08T19:32:48.677329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:42.074552+0000","scrub_duration":0.0036287210000000001,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.792001+0000","last_change":"2023-11-08T19:32:35.792054+0000","last_active":"2023-11-08T19:32:35.792001+0000","last_peered":"2023-11-08T19:32:35.792001+0000","last_clean":"2023-11-08T19:32:35.792001+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:35.792001+0000","last_undegraded":"2023-11-08T19:32:35.792001+0000","last_fullsized":"2023-11-08T19:32:35.792001+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.791811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.791811+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.791811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:51:30.989602+0000","scrub_duration":0.003741978,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"109'119","reported_seq":258,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.572619+0000","last_change":"2023-11-08T19:32:29.572619+0000","last_active":"2023-11-08T19:32:29.572619+0000","last_peered":"2023-11-08T19:32:29.572619+0000","last_clean":"2023-11-08T19:32:29.572619+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:29.572619+0000","last_undegraded":"2023-11-08T19:32:29.572619+0000","last_fullsized":"2023-11-08T19:32:29.572619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'119","last_scrub_stamp":"2023-11-08T19:32:29.572367+0000","last_deep_scrub":"109'119","last_deep_scrub_stamp":"2023-11-08T19:32:29.572367+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.572367+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:21.179114+0000","scrub_duration":0.0093265610000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1945600,"data_stored":1281536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.71599999999999997,"5min":0.66500000000000004,"15min":0.53000000000000003},"max":{"1min":1.3009999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1279999999999999,"15min":1.24},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.58499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":4.4589999999999996},"last":1.081}]},{"osd":1,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5309999999999999,"5min":1.728,"15min":4.5330000000000004},"last":1.149},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.147},"min":{"1min":0.69299999999999995,"5min":0.63900000000000001,"15min":0.57399999999999995},"max":{"1min":1.706,"5min":1.706,"15min":2.0150000000000001},"last":1.2909999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3300000000000001,"15min":1.4259999999999999},"min":{"1min":0.83199999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.593,"5min":1.891,"15min":4.6980000000000004},"last":1.1950000000000001},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2350000000000001,"15min":1.306},"min":{"1min":0.88600000000000001,"5min":0.78500000000000003,"15min":0.64700000000000002},"max":{"1min":1.671,"5min":1.71,"15min":4.4269999999999996},"last":1.4159999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3,"15min":1.3360000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":1.657},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.405,"15min":1.504},"min":{"1min":0.92200000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.831,"5min":1.8560000000000001,"15min":4.6150000000000002},"last":1.46}]},{"osd":4,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.232,"15min":1.3149999999999999},"min":{"1min":0.89100000000000001,"5min":0.624,"15min":0.624},"max":{"1min":1.5429999999999999,"5min":1.75,"15min":2.3170000000000002},"last":1.601},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.3460000000000001,"15min":1.4350000000000001},"min":{"1min":1.0269999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7969999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.486}]},{"osd":5,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3839999999999999,"15min":1.4339999999999999},"min":{"1min":0.97499999999999998,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.784,"5min":1.8740000000000001,"15min":2.3260000000000001},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.397,"15min":1.4470000000000001},"min":{"1min":1.0669999999999999,"5min":0.873,"15min":0.81699999999999995},"max":{"1min":1.706,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.333}]},{"osd":6,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4910000000000001,"15min":1.544},"min":{"1min":1.133,"5min":0.93700000000000006,"15min":0.878},"max":{"1min":1.97,"5min":1.97,"15min":2.427},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.55,"5min":1.4890000000000001,"15min":1.573},"min":{"1min":1.361,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.8620000000000001,"5min":2.1920000000000002,"15min":2.468},"last":1.375}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1220000000000001,"15min":1.0029999999999999},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.54600000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.752},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.236,"15min":1.151},"min":{"1min":0.752,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6080000000000001,"5min":8.391,"15min":8.391},"last":0.83599999999999997}]},{"osd":1,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.163,"15min":1.115},"min":{"1min":0.83599999999999997,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.254,"5min":1.2350000000000001,"15min":1.2929999999999999},"min":{"1min":0.91800000000000004,"5min":0.69999999999999996,"15min":0.51200000000000001},"max":{"1min":1.6020000000000001,"5min":1.665,"15min":4.6399999999999997},"last":1.21}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2170000000000001,"15min":1.258},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.54600000000000004},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":4.609},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.3069999999999999,"15min":1.2509999999999999},"min":{"1min":0.95999999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.861},"last":1.3420000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.2929999999999999,"15min":1.4139999999999999},"min":{"1min":0.92400000000000004,"5min":0.77900000000000003,"15min":0.56599999999999995},"max":{"1min":1.714,"5min":1.714,"15min":4.7190000000000003},"last":1.3879999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.272,"15min":1.2110000000000001},"min":{"1min":0.79300000000000004,"5min":0.625,"15min":0.625},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.266}]},{"osd":4,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.286,"15min":1.345},"min":{"1min":0.89500000000000002,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.8140000000000001,"5min":1.881,"15min":2.0449999999999999},"last":1.546},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.464,"15min":1.4930000000000001},"min":{"1min":1.226,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.913,"5min":1.998,"15min":2.3130000000000002},"last":1.617}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.294,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.835,"15min":1.9079999999999999},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.377,"15min":1.351},"min":{"1min":0.84599999999999997,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0489999999999999},"last":1.514}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.552,"15min":1.5529999999999999},"min":{"1min":1.359,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.468,"15min":1.4590000000000001},"min":{"1min":1.0329999999999999,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":3.0640000000000001},"last":1.147}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.228,"15min":1.1279999999999999},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.724,"15min":1.76},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.323,"5min":1.3069999999999999,"15min":1.258},"min":{"1min":0.89500000000000002,"5min":0.73599999999999999,"15min":0.61499999999999999},"max":{"1min":1.6679999999999999,"5min":1.837,"15min":1.954},"last":1.2729999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1519999999999999,"15min":1.159},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.58699999999999997},"max":{"1min":1.655,"5min":1.655,"15min":2.3039999999999998},"last":1.036},{"interface":"front","average":{"1min":1.238,"5min":1.1799999999999999,"15min":1.3380000000000001},"min":{"1min":0.78100000000000003,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.679,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.1659999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.258,"15min":1.202},"min":{"1min":1.0609999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.643,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.226,"5min":1.179,"15min":1.375},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.599,"5min":1.6990000000000001,"15min":7.6479999999999997},"last":1.107}]},{"osd":3,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.224,"15min":1.2649999999999999},"min":{"1min":0.89800000000000002,"5min":0.81399999999999995,"15min":0.71899999999999997},"max":{"1min":1.7130000000000001,"5min":1.74,"15min":2.1920000000000002},"last":1.464},{"interface":"front","average":{"1min":1.391,"5min":1.373,"15min":1.329},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.69699999999999995},"max":{"1min":1.6870000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.5720000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.216,"15min":1.262},"min":{"1min":1.0840000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.347},"last":1.006},{"interface":"front","average":{"1min":1.343,"5min":1.329,"15min":1.472},"min":{"1min":0.89500000000000002,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.6519999999999999,"5min":1.7989999999999999,"15min":3.153},"last":1.613}]},{"osd":6,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.3959999999999999,"15min":1.478},"min":{"1min":1.121,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8720000000000001,"5min":1.923,"15min":2.8650000000000002},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.393,"15min":1.4610000000000001},"min":{"1min":1.012,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.742,"5min":1.879,"15min":2.6190000000000002},"last":1.5129999999999999}]},{"osd":7,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.494,"15min":1.6220000000000001},"min":{"1min":1.016,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":3.2730000000000001},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.53,"5min":1.516,"15min":1.5640000000000001},"min":{"1min":0.97899999999999998,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.7849999999999999,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.6499999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.1779999999999999,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.6040000000000001,"5min":1.782,"15min":2.5219999999999998},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.92200000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":7.0119999999999996},"last":1.2849999999999999}]},{"osd":1,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63100000000000001},"max":{"1min":1.579,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.091},{"interface":"front","average":{"1min":1.244,"5min":1.171,"15min":1.1559999999999999},"min":{"1min":0.91200000000000003,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.5660000000000001,"5min":1.7310000000000001,"15min":2.9940000000000002},"last":1.1830000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.262,"15min":1.3120000000000001},"min":{"1min":0.752,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5129999999999999,"5min":1.988,"15min":4.0410000000000004},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3340000000000001,"15min":1.3839999999999999},"min":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.79,"5min":1.867,"15min":3.2069999999999999},"last":1.637}]},{"osd":3,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.198,"15min":1.258},"min":{"1min":0.95099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4980000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.246,"15min":1.3360000000000001},"min":{"1min":0.94199999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.714,"5min":1.738,"15min":3.5710000000000002},"last":1.323}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.417},"min":{"1min":0.81399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":4.2350000000000003},"last":1.042},{"interface":"front","average":{"1min":1.298,"5min":1.2490000000000001,"15min":1.4410000000000001},"min":{"1min":0.84899999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.194,"5min":2.194,"15min":4.5880000000000001},"last":1.129}]},{"osd":6,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.335,"15min":1.48},"min":{"1min":0.84999999999999998,"5min":0.745,"15min":0.745},"max":{"1min":1.6080000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4370000000000001,"15min":1.556},"min":{"1min":1.149,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":4.4130000000000003},"last":1.345}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.421,"15min":1.556},"min":{"1min":1.0629999999999999,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":4.6369999999999996},"last":1.47},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.554,"15min":1.665},"min":{"1min":1.1259999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.9370000000000001,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.5169999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.98099999999999998},"min":{"1min":0.67000000000000004,"5min":0.34100000000000003,"15min":0.312},"max":{"1min":1.9099999999999999,"5min":2.2320000000000002,"15min":3.972},"last":0.84399999999999997},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.94599999999999995,"15min":1.0349999999999999},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.311},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":3.1520000000000001},"last":0.95099999999999996}]},{"osd":1,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.995,"15min":1.1080000000000001},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.39500000000000002},"max":{"1min":1.79,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.90100000000000002,"15min":1.0289999999999999},"min":{"1min":0.43099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.744,"5min":1.744,"15min":4.5750000000000002},"last":0.88400000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1679999999999999,"15min":1.264},"min":{"1min":0.66900000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.6299999999999999,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.113,"15min":1.206},"min":{"1min":0.60999999999999999,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.5569999999999999,"5min":1.7190000000000001,"15min":2.73},"last":1.2070000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.234,"15min":1.365},"min":{"1min":0.80800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6439999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.637},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.264,"15min":1.379},"min":{"1min":0.97899999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":2.0449999999999999,"15min":3.052},"last":1.3680000000000001}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.323,"15min":1.429},"min":{"1min":0.77200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":1.972,"15min":3.0899999999999999},"last":1.5},{"interface":"front","average":{"1min":1.294,"5min":1.286,"15min":1.4390000000000001},"min":{"1min":1.012,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5980000000000001,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.403}]},{"osd":6,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2290000000000001,"15min":1.3680000000000001},"min":{"1min":0.78400000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.728,"5min":1.841,"15min":4.2220000000000004},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.393,"15min":1.5109999999999999},"min":{"1min":0.93799999999999994,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.889,"5min":1.9490000000000001,"15min":3.1230000000000002},"last":1.4410000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4179999999999999,"15min":1.597},"min":{"1min":0.91300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.157,"5min":2.157,"15min":4.407},"last":1.538},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.5049999999999999},"min":{"1min":0.83299999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.792,"5min":1.871,"15min":4.3849999999999998},"last":1.472}]}]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1347584,"data_stored":690149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.274,"5min":1.3460000000000001,"15min":1.7809999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.91900000000000004},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.36899999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.141}]},{"osd":1,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.91100000000000003,"15min":0.876},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.2310000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.0269999999999999,"15min":1.012},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4650000000000001,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":1.2250000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.125,"15min":1.0700000000000001},"min":{"1min":0.67200000000000004,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.5029999999999999,"5min":1.821,"15min":1.821},"last":0.92400000000000004},{"interface":"front","average":{"1min":1.101,"5min":1.079,"15min":1.099},"min":{"1min":0.64300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.0660000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2969999999999999,"15min":1.355},"min":{"1min":0.96299999999999997,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.2959999999999998},"last":1.365},{"interface":"front","average":{"1min":1.327,"5min":1.3100000000000001,"15min":1.335},"min":{"1min":0.82099999999999995,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.748,"5min":1.748,"15min":2.278},"last":1.115}]},{"osd":5,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3839999999999999},"min":{"1min":1.0209999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":2.0840000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.345,"15min":1.407},"min":{"1min":1.0269999999999999,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.889,"5min":1.9450000000000001,"15min":2.536},"last":1.5069999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.387,"15min":1.381},"min":{"1min":0.94299999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6910000000000001,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.409},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.367,"15min":1.417},"min":{"1min":1.1220000000000001,"5min":0.84699999999999998,"15min":0.70099999999999996},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.29}]},{"osd":7,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4139999999999999,"15min":1.5089999999999999},"min":{"1min":0.98599999999999999,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.6549999999999998},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.528,"5min":1.4910000000000001,"15min":1.5229999999999999},"min":{"1min":1.0980000000000001,"5min":0.93500000000000005,"15min":0.76600000000000001},"max":{"1min":2.2200000000000002,"5min":2.226,"15min":2.3439999999999999},"last":1.3320000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88,"15min":0.79100000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93500000000000005,"15min":0.81100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.374},"max":{"1min":1.45,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.45}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.872,"15min":0.81699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":1.288},{"interface":"front","average":{"1min":1.046,"5min":1.052,"15min":0.94199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.512,"5min":1.863,"15min":1.863},"last":1.046}]},{"osd":3,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3049999999999999,"15min":1.1419999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8440000000000001,"5min":1.978,"15min":1.978},"last":1.139},{"interface":"front","average":{"1min":1.044,"5min":1.131,"15min":1.099},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.583,"5min":1.778,"15min":2.5369999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.175,"15min":1.1599999999999999},"min":{"1min":0.79000000000000004,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.649,"5min":1.8620000000000001,"15min":2.165},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3029999999999999,"15min":1.2649999999999999},"min":{"1min":0.80600000000000005,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.256}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2949999999999999,"15min":1.264},"min":{"1min":0.79600000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0489999999999999,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.347},{"interface":"front","average":{"1min":1.258,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.629},"max":{"1min":1.7070000000000001,"5min":1.748,"15min":2.4089999999999998},"last":1.7070000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.373},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.9159999999999999,"5min":2.097,"15min":2.6019999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3740000000000001,"15min":1.373},"min":{"1min":0.94999999999999996,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.748,"5min":1.8240000000000001,"15min":2.282},"last":1.635}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4359999999999999,"15min":1.3640000000000001},"min":{"1min":0.85899999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":2.3860000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.409,"15min":1.393},"min":{"1min":0.72999999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.6699999999999999},"last":1.5920000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574952,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1941504,"data_stored":1280517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83599999999999997,"15min":0.79200000000000004},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.6559999999999999},"last":0.91800000000000004},{"interface":"front","average":{"1min":1.026,"5min":0.97299999999999998,"15min":0.93899999999999995},"min":{"1min":0.70999999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.7},"last":1.048}]},{"osd":2,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90600000000000003,"15min":0.876},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.77400000000000002},{"interface":"front","average":{"1min":1.115,"5min":1.0469999999999999,"15min":0.99099999999999999},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5309999999999999,"5min":1.54,"15min":1.7270000000000001},"last":1.1970000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0620000000000001,"15min":1.0209999999999999},"min":{"1min":0.54000000000000004,"5min":0.38,"15min":0.38},"max":{"1min":1.444,"5min":1.6599999999999999,"15min":1.748},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.0720000000000001,"15min":1.091},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8400000000000001,"15min":2.496},"last":1.127}]},{"osd":4,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.282,"15min":1.252},"min":{"1min":0.79200000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.663,"5min":1.774,"15min":1.9239999999999999},"last":1.502},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3400000000000001,"15min":1.391},"min":{"1min":0.85899999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.4039999999999999},"last":1.321}]},{"osd":5,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3420000000000001,"15min":1.3520000000000001},"min":{"1min":0.84599999999999997,"5min":0.68899999999999995,"15min":0.61099999999999999},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":2.4369999999999998},"last":1.556},{"interface":"front","average":{"1min":1.343,"5min":1.3109999999999999,"15min":1.389},"min":{"1min":0.879,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.9079999999999999,"5min":2.089,"15min":2.4239999999999999},"last":1.2490000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3700000000000001,"15min":1.444},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":2.0990000000000002,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.359},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.3939999999999999,"15min":1.3979999999999999},"min":{"1min":0.93600000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.0870000000000002,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.6080000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4399999999999999,"15min":1.411},"min":{"1min":1.002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.1629999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.381},{"interface":"front","average":{"1min":1.504,"5min":1.4430000000000001,"15min":1.462},"min":{"1min":0.90800000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.839,"5min":1.881,"15min":2.306},"last":1.4370000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:33:10.698 INFO:tasks.ceph:pgid 12.17 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.698 INFO:tasks.ceph:pgid 13.15 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.699 INFO:tasks.ceph:pgid 13.1f last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.699 INFO:tasks.ceph:pgid 12.1f last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.700 INFO:tasks.ceph:pgid 13.1c last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.701 INFO:tasks.ceph:pgid 13.4 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.701 INFO:tasks.ceph:pgid 12.5 last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.703 INFO:tasks.ceph:pgid 13.1 last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.703 INFO:tasks.ceph:pgid 13.f last_scrub_stamp 2023-11-08T19:27:46.764558+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=46, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.704 INFO:tasks.ceph:pgid 12.f last_scrub_stamp 2023-11-08T19:27:45.750386+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=27, tm_sec=45, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=19, tm_min=32, tm_sec=22, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T19:33:10.704 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T19:33:21.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:21.049 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:30.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T19:33:30.936 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.926+0000 7f24fca25640 1 Processor -- start 2023-11-08T19:33:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.927+0000 7f24fca25640 1 -- start start 2023-11-08T19:33:30.937 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24fca25640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f24f80ae2d0 0x7f24f80afa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24fca25640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 0x7f24f815ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.938 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24fca25640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f806f400 con 0x7f24f815e440 2023-11-08T19:33:30.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24fca25640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80b01d0 con 0x7f24f80ae2d0 2023-11-08T19:33:30.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24fca25640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f806ee80 con 0x7f24f806e790 2023-11-08T19:33:30.939 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24f6d9d640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f24f815e440 0x7f24f80cf320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60132/0 (socket says 172.21.15.119:60132) 2023-11-08T19:33:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.928+0000 7f24f6d9d640 1 -- 172.21.15.119:0/175582102 learned_addr learned my addr 172.21.15.119:0/175582102 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:33:30.940 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f659c640 1 --2- 172.21.15.119:0/175582102 >> v2:172.21.15.188:3300/0 conn(0x7f24f80ae2d0 0x7f24f80afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f759e640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 0x7f24f815ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.941 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952129957 0 0) 0x7f24f806f400 con 0x7f24f815e440 2023-11-08T19:33:30.942 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e4003520 con 0x7f24f815e440 2023-11-08T19:33:30.943 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251564803 0 0) 0x7f24e4003520 con 0x7f24f815e440 2023-11-08T19:33:30.944 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 msgr2=0x7f24f815ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:30.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 0x7f24f815ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 >> v2:172.21.15.188:3300/0 conn(0x7f24f80ae2d0 msgr2=0x7f24f80afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:30.945 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 --2- 172.21.15.119:0/175582102 >> v2:172.21.15.188:3300/0 conn(0x7f24f80ae2d0 0x7f24f80afa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f806ebe0 con 0x7f24f815e440 2023-11-08T19:33:30.946 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.929+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f24e8001b70 con 0x7f24f815e440 2023-11-08T19:33:30.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 >> v1:172.21.15.119:6789/0 conn(0x7f24f815e440 legacy=0x7f24f80cf320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:30.947 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24e40047c0 0x7f24e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.948 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 0x7f24e4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80b01d0 con 0x7f24e40047c0 2023-11-08T19:33:30.949 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f806ee80 con 0x7f24e4003970 2023-11-08T19:33:30.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e400b980 con 0x7f24f806e790 2023-11-08T19:33:30.950 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f659c640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24e40047c0 0x7f24e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.951 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f6d9d640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.952 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f659c640 1 -- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 msgr2=0x7f24e4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:30.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f659c640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f806e790 0x7f24e4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.953 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f659c640 1 -- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e4003970 msgr2=0x7f24e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:30.954 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f659c640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.955 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f659c640 1 -- 172.21.15.119:0/175582102 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4003520 con 0x7f24e40047c0 2023-11-08T19:33:30.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.930+0000 7f24f6d9d640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e4003970 0x7f24e4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T19:33:30.956 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.932+0000 7f24f659c640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24e40047c0 0x7f24e4004c40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009a90 tx=0x7f24e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2f61cba4bf7099 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:30.957 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.933+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e002c680 con 0x7f24e40047c0 2023-11-08T19:33:30.959 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.933+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e002c7e0 con 0x7f24e40047c0 2023-11-08T19:33:30.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.933+0000 7f24f5d9b640 1 -- 172.21.15.119:0/175582102 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e002cb90 con 0x7f24e40047c0 2023-11-08T19:33:30.960 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24fca25640 1 -- 172.21.15.119:0/175582102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24e40047c0 msgr2=0x7f24e4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:30.961 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24fca25640 1 --2- 172.21.15.119:0/175582102 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24e40047c0 0x7f24e4004c40 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009a90 tx=0x7f24e40057f0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.963 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24f759e640 1 -- 172.21.15.119:0/175582102 reap_dead start 2023-11-08T19:33:30.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24fca25640 1 -- 172.21.15.119:0/175582102 shutdown_connections 2023-11-08T19:33:30.970 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24fca25640 1 -- 172.21.15.119:0/175582102 >> 172.21.15.119:0/175582102 conn(0x7f24f8099fd0 msgr2=0x7f24f80d0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:30.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24fca25640 1 -- 172.21.15.119:0/175582102 shutdown_connections 2023-11-08T19:33:30.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.934+0000 7f24fca25640 1 -- 172.21.15.119:0/175582102 wait complete. 2023-11-08T19:33:30.971 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.936+0000 7f24fca25640 1 Processor -- start 2023-11-08T19:33:30.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.936+0000 7f24fca25640 1 -- start start 2023-11-08T19:33:30.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24fca25640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e40047c0 0x7f24f80c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24fca25640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 0x7f24f80b16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.972 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24fca25640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f81531e0 0x7f24f80c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24fca25640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80b1c00 con 0x7f24f80b2e90 2023-11-08T19:33:30.973 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24fca25640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80b1df0 con 0x7f24e40047c0 2023-11-08T19:33:30.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24fca25640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80b20d0 con 0x7f24f81531e0 2023-11-08T19:33:30.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24f659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 0x7f24f80b16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.974 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24f659c640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 0x7f24f80b16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36872/0 (socket says 172.21.15.119:36872) 2023-11-08T19:33:30.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.937+0000 7f24f659c640 1 -- 172.21.15.119:0/2046430135 learned_addr learned my addr 172.21.15.119:0/2046430135 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:33:30.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f759e640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f81531e0 0x7f24f80c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.975 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f659c640 1 -- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f81531e0 msgr2=0x7f24f80c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:30.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f659c640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f81531e0 0x7f24f80c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f659c640 1 -- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e40047c0 msgr2=0x7f24f80c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:30.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f6d9d640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e40047c0 0x7f24f80c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.976 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f659c640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e40047c0 0x7f24f80c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:30.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f659c640 1 -- 172.21.15.119:0/2046430135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80b23b0 con 0x7f24f80b2e90 2023-11-08T19:33:30.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24f659c640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 0x7f24f80b16c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009bb0 tx=0x7f24e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:30.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0034a90 con 0x7f24f80b2e90 2023-11-08T19:33:30.977 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0034c10 con 0x7f24f80b2e90 2023-11-08T19:33:30.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f80b2690 con 0x7f24f80b2e90 2023-11-08T19:33:30.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.938+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e003e400 con 0x7f24f80b2e90 2023-11-08T19:33:30.978 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.939+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f8153690 con 0x7f24f80b2e90 2023-11-08T19:33:30.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.940+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f804ece0 con 0x7f24f80b2e90 2023-11-08T19:33:30.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.940+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24e003e560 con 0x7f24f80b2e90 2023-11-08T19:33:30.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.941+0000 7f24ef7fe640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24d806ae70 0x7f24d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:30.979 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.941+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f24e00abaa0 con 0x7f24f80b2e90 2023-11-08T19:33:30.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.942+0000 7f24f6d9d640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24d806ae70 0x7f24d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:30.980 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.942+0000 7f24f6d9d640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24d806ae70 0x7f24d806d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f24e8001ca0 tx=0x7f24e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:30.981 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:30.946+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24e00766e0 con 0x7f24f80b2e90 2023-11-08T19:33:31.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.127+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 --> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24f8153820 con 0x7f24d806ae70 2023-11-08T19:33:31.152 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.149+0000 7f24ef7fe640 1 -- 172.21.15.119:0/2046430135 <== mgr.4107 v2:172.21.15.119:6832/2270580327 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+292184 (secure 0 0 0) 0x7f24f8153820 con 0x7f24d806ae70 2023-11-08T19:33:31.153 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:33:31.157 INFO:teuthology.orchestra.run.smithi119.stderr:dumped all 2023-11-08T19:33:31.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24d806ae70 msgr2=0x7f24d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24d806ae70 0x7f24d806d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f24e8001ca0 tx=0x7f24e8009040 comp rx=0 tx=0).stop 2023-11-08T19:33:31.159 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 msgr2=0x7f24f80b16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 0x7f24f80b16c0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009bb0 tx=0x7f24e0004730 comp rx=0 tx=0).stop 2023-11-08T19:33:31.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 shutdown_connections 2023-11-08T19:33:31.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f24d806ae70 0x7f24d806d330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.160 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24f81531e0 0x7f24f80c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f24f80b2e90 0x7f24f80b16c0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 --2- 172.21.15.119:0/2046430135 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24e40047c0 0x7f24f80c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.161 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 >> 172.21.15.119:0/2046430135 conn(0x7f24f8099fd0 msgr2=0x7f24f80b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.156+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 shutdown_connections 2023-11-08T19:33:31.162 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.157+0000 7f24fca25640 1 -- 172.21.15.119:0/2046430135 wait complete. 2023-11-08T19:33:31.176 INFO:teuthology.orchestra.run.smithi119.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2023-11-08T19:33:30.868128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261120,"kb_used_data":11712,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713600,"statfs":{"total":773094113280,"available":772826726400,"internally_reserved":0,"allocated":11993088,"data_stored":6708042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004744"},"pg_stats":[{"pgid":"13.17","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.525576+0000","last_change":"2023-11-08T19:32:46.525623+0000","last_active":"2023-11-08T19:32:46.525576+0000","last_peered":"2023-11-08T19:32:46.525576+0000","last_clean":"2023-11-08T19:32:46.525576+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:46.525576+0000","last_undegraded":"2023-11-08T19:32:46.525576+0000","last_fullsized":"2023-11-08T19:32:46.525576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.525310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.525310+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.525310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:58:08.797703+0000","scrub_duration":0.0036410370000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:32.935524+0000","last_change":"2023-11-08T19:32:32.935524+0000","last_active":"2023-11-08T19:32:32.935524+0000","last_peered":"2023-11-08T19:32:32.935524+0000","last_clean":"2023-11-08T19:32:32.935524+0000","last_became_active":"2023-11-08T19:27:46.810048+0000","last_became_peered":"2023-11-08T19:27:46.810048+0000","last_unstale":"2023-11-08T19:32:32.935524+0000","last_undegraded":"2023-11-08T19:32:32.935524+0000","last_fullsized":"2023-11-08T19:32:32.935524+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:32.935290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:32.935290+0000","last_clean_scrub_stamp":"2023-11-08T19:32:32.935290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:26:23.985335+0000","scrub_duration":0.0032534619999999999,"objects_trimmed":0,"snaptrim_duration":0.0016468489999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.771234+0000","last_change":"2023-11-08T19:32:29.771285+0000","last_active":"2023-11-08T19:32:29.771234+0000","last_peered":"2023-11-08T19:32:29.771234+0000","last_clean":"2023-11-08T19:32:29.771234+0000","last_became_active":"2023-11-08T19:26:40.728195+0000","last_became_peered":"2023-11-08T19:26:40.728195+0000","last_unstale":"2023-11-08T19:32:29.771234+0000","last_undegraded":"2023-11-08T19:32:29.771234+0000","last_fullsized":"2023-11-08T19:32:29.771234+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.771060+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.771060+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.771060+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:57:45.945310+0000","scrub_duration":0.0045387559999999997,"objects_trimmed":0,"snaptrim_duration":0.0015778630000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:50.683999+0000","last_change":"2023-11-08T19:32:50.684086+0000","last_active":"2023-11-08T19:32:50.683999+0000","last_peered":"2023-11-08T19:32:50.683999+0000","last_clean":"2023-11-08T19:32:50.683999+0000","last_became_active":"2023-11-08T19:27:47.801689+0000","last_became_peered":"2023-11-08T19:27:47.801689+0000","last_unstale":"2023-11-08T19:32:50.683999+0000","last_undegraded":"2023-11-08T19:32:50.683999+0000","last_fullsized":"2023-11-08T19:32:50.683999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:50.683777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:50.683777+0000","last_clean_scrub_stamp":"2023-11-08T19:32:50.683777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:48:40.297282+0000","scrub_duration":0.0032965569999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:13.837041+0000","last_change":"2023-11-08T19:33:13.837123+0000","last_active":"2023-11-08T19:33:13.837041+0000","last_peered":"2023-11-08T19:33:13.837041+0000","last_clean":"2023-11-08T19:33:13.837041+0000","last_became_active":"2023-11-08T19:27:46.808316+0000","last_became_peered":"2023-11-08T19:27:46.808316+0000","last_unstale":"2023-11-08T19:33:13.837041+0000","last_undegraded":"2023-11-08T19:33:13.837041+0000","last_fullsized":"2023-11-08T19:33:13.837041+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:13.836856+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:13.836856+0000","last_clean_scrub_stamp":"2023-11-08T19:33:13.836856+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:40.498199+0000","scrub_duration":0.0036399309999999999,"objects_trimmed":0,"snaptrim_duration":0.0020956640000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.652067+0000","last_change":"2023-11-08T19:32:28.652115+0000","last_active":"2023-11-08T19:32:28.652067+0000","last_peered":"2023-11-08T19:32:28.652067+0000","last_clean":"2023-11-08T19:32:28.652067+0000","last_became_active":"2023-11-08T19:26:40.732065+0000","last_became_peered":"2023-11-08T19:26:40.732065+0000","last_unstale":"2023-11-08T19:32:28.652067+0000","last_undegraded":"2023-11-08T19:32:28.652067+0000","last_fullsized":"2023-11-08T19:32:28.652067+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.651902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.651902+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.651902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:39:39.497609+0000","scrub_duration":0.0036776040000000001,"objects_trimmed":0,"snaptrim_duration":0.0036321859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:08.768150+0000","last_change":"2023-11-08T19:33:08.768200+0000","last_active":"2023-11-08T19:33:08.768150+0000","last_peered":"2023-11-08T19:33:08.768150+0000","last_clean":"2023-11-08T19:33:08.768150+0000","last_became_active":"2023-11-08T19:27:47.803218+0000","last_became_peered":"2023-11-08T19:27:47.803218+0000","last_unstale":"2023-11-08T19:33:08.768150+0000","last_undegraded":"2023-11-08T19:33:08.768150+0000","last_fullsized":"2023-11-08T19:33:08.768150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:08.767986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:08.767986+0000","last_clean_scrub_stamp":"2023-11-08T19:33:08.767986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:53.489180+0000","scrub_duration":0.0036415649999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":66,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:38.643087+0000","last_change":"2023-11-08T19:32:38.643143+0000","last_active":"2023-11-08T19:32:38.643087+0000","last_peered":"2023-11-08T19:32:38.643087+0000","last_clean":"2023-11-08T19:32:38.643087+0000","last_became_active":"2023-11-08T19:27:46.798170+0000","last_became_peered":"2023-11-08T19:27:46.798170+0000","last_unstale":"2023-11-08T19:32:38.643087+0000","last_undegraded":"2023-11-08T19:32:38.643087+0000","last_fullsized":"2023-11-08T19:32:38.643087+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:38.642935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:38.642935+0000","last_clean_scrub_stamp":"2023-11-08T19:32:38.642935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:12:19.775448+0000","scrub_duration":0.0035460359999999998,"objects_trimmed":0,"snaptrim_duration":0.0019046110000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":88,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:54.750163+0000","last_change":"2023-11-08T19:32:54.750163+0000","last_active":"2023-11-08T19:32:54.750163+0000","last_peered":"2023-11-08T19:32:54.750163+0000","last_clean":"2023-11-08T19:32:54.750163+0000","last_became_active":"2023-11-08T19:26:40.726090+0000","last_became_peered":"2023-11-08T19:26:40.726090+0000","last_unstale":"2023-11-08T19:32:54.750163+0000","last_undegraded":"2023-11-08T19:32:54.750163+0000","last_fullsized":"2023-11-08T19:32:54.750163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:54.749958+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:54.749958+0000","last_clean_scrub_stamp":"2023-11-08T19:32:54.749958+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:37:41.693384+0000","scrub_duration":0.003451188,"objects_trimmed":0,"snaptrim_duration":0.0022513519999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:56.878255+0000","last_change":"2023-11-08T19:32:56.878341+0000","last_active":"2023-11-08T19:32:56.878255+0000","last_peered":"2023-11-08T19:32:56.878255+0000","last_clean":"2023-11-08T19:32:56.878255+0000","last_became_active":"2023-11-08T19:27:47.810559+0000","last_became_peered":"2023-11-08T19:27:47.810559+0000","last_unstale":"2023-11-08T19:32:56.878255+0000","last_undegraded":"2023-11-08T19:32:56.878255+0000","last_fullsized":"2023-11-08T19:32:56.878255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:56.878015+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:56.878015+0000","last_clean_scrub_stamp":"2023-11-08T19:32:56.878015+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:06.790201+0000","scrub_duration":0.0038922980000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:01.722245+0000","last_change":"2023-11-08T19:33:01.722291+0000","last_active":"2023-11-08T19:33:01.722245+0000","last_peered":"2023-11-08T19:33:01.722245+0000","last_clean":"2023-11-08T19:33:01.722245+0000","last_became_active":"2023-11-08T19:27:46.804745+0000","last_became_peered":"2023-11-08T19:27:46.804745+0000","last_unstale":"2023-11-08T19:33:01.722245+0000","last_undegraded":"2023-11-08T19:33:01.722245+0000","last_fullsized":"2023-11-08T19:33:01.722245+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:01.722092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:01.722092+0000","last_clean_scrub_stamp":"2023-11-08T19:33:01.722092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:39.551131+0000","scrub_duration":0.003627941,"objects_trimmed":0,"snaptrim_duration":0.0025153039999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.727558+0000","last_change":"2023-11-08T19:32:27.727558+0000","last_active":"2023-11-08T19:32:27.727558+0000","last_peered":"2023-11-08T19:32:27.727558+0000","last_clean":"2023-11-08T19:32:27.727558+0000","last_became_active":"2023-11-08T19:26:40.723570+0000","last_became_peered":"2023-11-08T19:26:40.723570+0000","last_unstale":"2023-11-08T19:32:27.727558+0000","last_undegraded":"2023-11-08T19:32:27.727558+0000","last_fullsized":"2023-11-08T19:32:27.727558+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:27.727335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:27.727335+0000","last_clean_scrub_stamp":"2023-11-08T19:32:27.727335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:55.266502+0000","scrub_duration":0.0027330219999999999,"objects_trimmed":0,"snaptrim_duration":0.00098962000000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:57.645193+0000","last_change":"2023-11-08T19:32:57.645239+0000","last_active":"2023-11-08T19:32:57.645193+0000","last_peered":"2023-11-08T19:32:57.645193+0000","last_clean":"2023-11-08T19:32:57.645193+0000","last_became_active":"2023-11-08T19:27:47.811717+0000","last_became_peered":"2023-11-08T19:27:47.811717+0000","last_unstale":"2023-11-08T19:32:57.645193+0000","last_undegraded":"2023-11-08T19:32:57.645193+0000","last_fullsized":"2023-11-08T19:32:57.645193+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:57.645028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:57.645028+0000","last_clean_scrub_stamp":"2023-11-08T19:32:57.645028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:23.575294+0000","scrub_duration":0.0039615750000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.625497+0000","last_change":"2023-11-08T19:32:45.625543+0000","last_active":"2023-11-08T19:32:45.625497+0000","last_peered":"2023-11-08T19:32:45.625497+0000","last_clean":"2023-11-08T19:32:45.625497+0000","last_became_active":"2023-11-08T19:27:46.799907+0000","last_became_peered":"2023-11-08T19:27:46.799907+0000","last_unstale":"2023-11-08T19:32:45.625497+0000","last_undegraded":"2023-11-08T19:32:45.625497+0000","last_fullsized":"2023-11-08T19:32:45.625497+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.625348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.625348+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.625348+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:17:33.970967+0000","scrub_duration":0.0030484169999999999,"objects_trimmed":0,"snaptrim_duration":0.00056458799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:42.920729+0000","last_change":"2023-11-08T19:32:42.920778+0000","last_active":"2023-11-08T19:32:42.920729+0000","last_peered":"2023-11-08T19:32:42.920729+0000","last_clean":"2023-11-08T19:32:42.920729+0000","last_became_active":"2023-11-08T19:26:40.722312+0000","last_became_peered":"2023-11-08T19:26:40.722312+0000","last_unstale":"2023-11-08T19:32:42.920729+0000","last_undegraded":"2023-11-08T19:32:42.920729+0000","last_fullsized":"2023-11-08T19:32:42.920729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:42.920564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:42.920564+0000","last_clean_scrub_stamp":"2023-11-08T19:32:42.920564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:15:38.876790+0000","scrub_duration":0.0033816290000000001,"objects_trimmed":0,"snaptrim_duration":0.0015198239999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.728642+0000","last_change":"2023-11-08T19:33:00.728688+0000","last_active":"2023-11-08T19:33:00.728642+0000","last_peered":"2023-11-08T19:33:00.728642+0000","last_clean":"2023-11-08T19:33:00.728642+0000","last_became_active":"2023-11-08T19:27:47.806767+0000","last_became_peered":"2023-11-08T19:27:47.806767+0000","last_unstale":"2023-11-08T19:33:00.728642+0000","last_undegraded":"2023-11-08T19:33:00.728642+0000","last_fullsized":"2023-11-08T19:33:00.728642+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.728493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.728493+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.728493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:39.952049+0000","scrub_duration":0.0036203329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.753355+0000","last_change":"2023-11-08T19:32:30.753400+0000","last_active":"2023-11-08T19:32:30.753355+0000","last_peered":"2023-11-08T19:32:30.753355+0000","last_clean":"2023-11-08T19:32:30.753355+0000","last_became_active":"2023-11-08T19:27:46.794983+0000","last_became_peered":"2023-11-08T19:27:46.794983+0000","last_unstale":"2023-11-08T19:32:30.753355+0000","last_undegraded":"2023-11-08T19:32:30.753355+0000","last_fullsized":"2023-11-08T19:32:30.753355+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.753206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.753206+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.753206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:15:08.346214+0000","scrub_duration":0.0030208409999999998,"objects_trimmed":0,"snaptrim_duration":0.0018282870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.821586+0000","last_change":"2023-11-08T19:32:34.821633+0000","last_active":"2023-11-08T19:32:34.821586+0000","last_peered":"2023-11-08T19:32:34.821586+0000","last_clean":"2023-11-08T19:32:34.821586+0000","last_became_active":"2023-11-08T19:26:40.727205+0000","last_became_peered":"2023-11-08T19:26:40.727205+0000","last_unstale":"2023-11-08T19:32:34.821586+0000","last_undegraded":"2023-11-08T19:32:34.821586+0000","last_fullsized":"2023-11-08T19:32:34.821586+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.821364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.821364+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.821364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:32:07.177396+0000","scrub_duration":0.0040781000000000003,"objects_trimmed":0,"snaptrim_duration":0.001128682,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.703378+0000","last_change":"2023-11-08T19:32:45.703425+0000","last_active":"2023-11-08T19:32:45.703378+0000","last_peered":"2023-11-08T19:32:45.703378+0000","last_clean":"2023-11-08T19:32:45.703378+0000","last_became_active":"2023-11-08T19:27:47.804558+0000","last_became_peered":"2023-11-08T19:27:47.804558+0000","last_unstale":"2023-11-08T19:32:45.703378+0000","last_undegraded":"2023-11-08T19:32:45.703378+0000","last_fullsized":"2023-11-08T19:32:45.703378+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.703215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.703215+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.703215+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:29.552613+0000","scrub_duration":0.0041161319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:45.780034+0000","last_change":"2023-11-08T19:32:45.780081+0000","last_active":"2023-11-08T19:32:45.780034+0000","last_peered":"2023-11-08T19:32:45.780034+0000","last_clean":"2023-11-08T19:32:45.780034+0000","last_became_active":"2023-11-08T19:27:46.809244+0000","last_became_peered":"2023-11-08T19:27:46.809244+0000","last_unstale":"2023-11-08T19:32:45.780034+0000","last_undegraded":"2023-11-08T19:32:45.780034+0000","last_fullsized":"2023-11-08T19:32:45.780034+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:45.779844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:45.779844+0000","last_clean_scrub_stamp":"2023-11-08T19:32:45.779844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:00:03.630656+0000","scrub_duration":0.003794487,"objects_trimmed":0,"snaptrim_duration":0.001081055,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.874214+0000","last_change":"2023-11-08T19:32:30.874260+0000","last_active":"2023-11-08T19:32:30.874214+0000","last_peered":"2023-11-08T19:32:30.874214+0000","last_clean":"2023-11-08T19:32:30.874214+0000","last_became_active":"2023-11-08T19:26:40.726775+0000","last_became_peered":"2023-11-08T19:26:40.726775+0000","last_unstale":"2023-11-08T19:32:30.874214+0000","last_undegraded":"2023-11-08T19:32:30.874214+0000","last_fullsized":"2023-11-08T19:32:30.874214+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.874063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.874063+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.874063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:25:33.344505+0000","scrub_duration":0.0032893110000000001,"objects_trimmed":0,"snaptrim_duration":0.001359545,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:39.724808+0000","last_change":"2023-11-08T19:32:39.724860+0000","last_active":"2023-11-08T19:32:39.724808+0000","last_peered":"2023-11-08T19:32:39.724808+0000","last_clean":"2023-11-08T19:32:39.724808+0000","last_became_active":"2023-11-08T19:27:47.795653+0000","last_became_peered":"2023-11-08T19:27:47.795653+0000","last_unstale":"2023-11-08T19:32:39.724808+0000","last_undegraded":"2023-11-08T19:32:39.724808+0000","last_fullsized":"2023-11-08T19:32:39.724808+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:39.724643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:39.724643+0000","last_clean_scrub_stamp":"2023-11-08T19:32:39.724643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:02:16.175462+0000","scrub_duration":0.0033653490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":66,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.588081+0000","last_change":"2023-11-08T19:32:35.588132+0000","last_active":"2023-11-08T19:32:35.588081+0000","last_peered":"2023-11-08T19:32:35.588081+0000","last_clean":"2023-11-08T19:32:35.588081+0000","last_became_active":"2023-11-08T19:27:46.809312+0000","last_became_peered":"2023-11-08T19:27:46.809312+0000","last_unstale":"2023-11-08T19:32:35.588081+0000","last_undegraded":"2023-11-08T19:32:35.588081+0000","last_fullsized":"2023-11-08T19:32:35.588081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.587924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.587924+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.587924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:11:25.638388+0000","scrub_duration":0.003687957,"objects_trimmed":0,"snaptrim_duration":0.0034928870000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.650654+0000","last_change":"2023-11-08T19:32:29.650701+0000","last_active":"2023-11-08T19:32:29.650654+0000","last_peered":"2023-11-08T19:32:29.650654+0000","last_clean":"2023-11-08T19:32:29.650654+0000","last_became_active":"2023-11-08T19:26:40.727821+0000","last_became_peered":"2023-11-08T19:26:40.727821+0000","last_unstale":"2023-11-08T19:32:29.650654+0000","last_undegraded":"2023-11-08T19:32:29.650654+0000","last_fullsized":"2023-11-08T19:32:29.650654+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.650428+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.650428+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.650428+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:44:57.186304+0000","scrub_duration":0.0035263999999999998,"objects_trimmed":0,"snaptrim_duration":0.001507918,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:23.673155+0000","last_change":"2023-11-08T19:33:23.673236+0000","last_active":"2023-11-08T19:33:23.673155+0000","last_peered":"2023-11-08T19:33:23.673155+0000","last_clean":"2023-11-08T19:33:23.673155+0000","last_became_active":"2023-11-08T19:27:47.810192+0000","last_became_peered":"2023-11-08T19:27:47.810192+0000","last_unstale":"2023-11-08T19:33:23.673155+0000","last_undegraded":"2023-11-08T19:33:23.673155+0000","last_fullsized":"2023-11-08T19:33:23.673155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:23.672916+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:23.672916+0000","last_clean_scrub_stamp":"2023-11-08T19:33:23.672916+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:50:08.394110+0000","scrub_duration":0.0042419069999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.645920+0000","last_change":"2023-11-08T19:32:46.645975+0000","last_active":"2023-11-08T19:32:46.645920+0000","last_peered":"2023-11-08T19:32:46.645920+0000","last_clean":"2023-11-08T19:32:46.645920+0000","last_became_active":"2023-11-08T19:27:46.800108+0000","last_became_peered":"2023-11-08T19:27:46.800108+0000","last_unstale":"2023-11-08T19:32:46.645920+0000","last_undegraded":"2023-11-08T19:32:46.645920+0000","last_fullsized":"2023-11-08T19:32:46.645920+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.645739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.645739+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.645739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:57.261846+0000","scrub_duration":0.0036408650000000001,"objects_trimmed":0,"snaptrim_duration":0.0015149880000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.559388+0000","last_change":"2023-11-08T19:32:41.559435+0000","last_active":"2023-11-08T19:32:41.559388+0000","last_peered":"2023-11-08T19:32:41.559388+0000","last_clean":"2023-11-08T19:32:41.559388+0000","last_became_active":"2023-11-08T19:26:40.724569+0000","last_became_peered":"2023-11-08T19:26:40.724569+0000","last_unstale":"2023-11-08T19:32:41.559388+0000","last_undegraded":"2023-11-08T19:32:41.559388+0000","last_fullsized":"2023-11-08T19:32:41.559388+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.559226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.559226+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.559226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:42.827112+0000","scrub_duration":0.0037686489999999998,"objects_trimmed":0,"snaptrim_duration":0.001483725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.601945+0000","last_change":"2023-11-08T19:33:00.601992+0000","last_active":"2023-11-08T19:33:00.601945+0000","last_peered":"2023-11-08T19:33:00.601945+0000","last_clean":"2023-11-08T19:33:00.601945+0000","last_became_active":"2023-11-08T19:27:47.808380+0000","last_became_peered":"2023-11-08T19:27:47.808380+0000","last_unstale":"2023-11-08T19:33:00.601945+0000","last_undegraded":"2023-11-08T19:33:00.601945+0000","last_fullsized":"2023-11-08T19:33:00.601945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.601784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.601784+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.601784+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:27.398867+0000","scrub_duration":0.0037468319999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:14.841454+0000","last_change":"2023-11-08T19:33:14.841534+0000","last_active":"2023-11-08T19:33:14.841454+0000","last_peered":"2023-11-08T19:33:14.841454+0000","last_clean":"2023-11-08T19:33:14.841454+0000","last_became_active":"2023-11-08T19:27:46.807108+0000","last_became_peered":"2023-11-08T19:27:46.807108+0000","last_unstale":"2023-11-08T19:33:14.841454+0000","last_undegraded":"2023-11-08T19:33:14.841454+0000","last_fullsized":"2023-11-08T19:33:14.841454+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:14.841320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:14.841320+0000","last_clean_scrub_stamp":"2023-11-08T19:33:14.841320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:36:55.243365+0000","scrub_duration":0.003420476,"objects_trimmed":0,"snaptrim_duration":0.0019525670000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.675611+0000","last_change":"2023-11-08T19:32:28.675655+0000","last_active":"2023-11-08T19:32:28.675611+0000","last_peered":"2023-11-08T19:32:28.675611+0000","last_clean":"2023-11-08T19:32:28.675611+0000","last_became_active":"2023-11-08T19:26:40.723202+0000","last_became_peered":"2023-11-08T19:26:40.723202+0000","last_unstale":"2023-11-08T19:32:28.675611+0000","last_undegraded":"2023-11-08T19:32:28.675611+0000","last_fullsized":"2023-11-08T19:32:28.675611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.675467+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.675467+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.675467+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:32:57.443158+0000","scrub_duration":0.00292019,"objects_trimmed":0,"snaptrim_duration":0.0023737519999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.627590+0000","last_change":"2023-11-08T19:33:00.627635+0000","last_active":"2023-11-08T19:33:00.627590+0000","last_peered":"2023-11-08T19:33:00.627590+0000","last_clean":"2023-11-08T19:33:00.627590+0000","last_became_active":"2023-11-08T19:27:47.815157+0000","last_became_peered":"2023-11-08T19:27:47.815157+0000","last_unstale":"2023-11-08T19:33:00.627590+0000","last_undegraded":"2023-11-08T19:33:00.627590+0000","last_fullsized":"2023-11-08T19:33:00.627590+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.627409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.627409+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.627409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:27.424492+0000","scrub_duration":0.0036601170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.780194+0000","last_change":"2023-11-08T19:32:46.780194+0000","last_active":"2023-11-08T19:32:46.780194+0000","last_peered":"2023-11-08T19:32:46.780194+0000","last_clean":"2023-11-08T19:32:46.780194+0000","last_became_active":"2023-11-08T19:27:46.801396+0000","last_became_peered":"2023-11-08T19:27:46.801396+0000","last_unstale":"2023-11-08T19:32:46.780194+0000","last_undegraded":"2023-11-08T19:32:46.780194+0000","last_fullsized":"2023-11-08T19:32:46.780194+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.779991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.779991+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.779991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:19:08.986106+0000","scrub_duration":0.0036491200000000001,"objects_trimmed":0,"snaptrim_duration":0.001406998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.589623+0000","last_change":"2023-11-08T19:32:44.589675+0000","last_active":"2023-11-08T19:32:44.589623+0000","last_peered":"2023-11-08T19:32:44.589623+0000","last_clean":"2023-11-08T19:32:44.589623+0000","last_became_active":"2023-11-08T19:26:40.733288+0000","last_became_peered":"2023-11-08T19:26:40.733288+0000","last_unstale":"2023-11-08T19:32:44.589623+0000","last_undegraded":"2023-11-08T19:32:44.589623+0000","last_fullsized":"2023-11-08T19:32:44.589623+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.589412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.589412+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.589412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:19:03.472044+0000","scrub_duration":0.0032550370000000001,"objects_trimmed":0,"snaptrim_duration":0.0016157319999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:20.795455+0000","last_change":"2023-11-08T19:33:20.795571+0000","last_active":"2023-11-08T19:33:20.795455+0000","last_peered":"2023-11-08T19:33:20.795455+0000","last_clean":"2023-11-08T19:33:20.795455+0000","last_became_active":"2023-11-08T19:27:47.807264+0000","last_became_peered":"2023-11-08T19:27:47.807264+0000","last_unstale":"2023-11-08T19:33:20.795455+0000","last_undegraded":"2023-11-08T19:33:20.795455+0000","last_fullsized":"2023-11-08T19:33:20.795455+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:20.795210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:20.795210+0000","last_clean_scrub_stamp":"2023-11-08T19:33:20.795210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:02:47.592293+0000","scrub_duration":0.0037079190000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:51.893384+0000","last_change":"2023-11-08T19:32:51.893433+0000","last_active":"2023-11-08T19:32:51.893384+0000","last_peered":"2023-11-08T19:32:51.893384+0000","last_clean":"2023-11-08T19:32:51.893384+0000","last_became_active":"2023-11-08T19:27:46.795350+0000","last_became_peered":"2023-11-08T19:27:46.795350+0000","last_unstale":"2023-11-08T19:32:51.893384+0000","last_undegraded":"2023-11-08T19:32:51.893384+0000","last_fullsized":"2023-11-08T19:32:51.893384+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:51.893226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:51.893226+0000","last_clean_scrub_stamp":"2023-11-08T19:32:51.893226+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:23:39.105087+0000","scrub_duration":0.003348733,"objects_trimmed":0,"snaptrim_duration":0.0012749580000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":91,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.663302+0000","last_change":"2023-11-08T19:32:35.663353+0000","last_active":"2023-11-08T19:32:35.663302+0000","last_peered":"2023-11-08T19:32:35.663302+0000","last_clean":"2023-11-08T19:32:35.663302+0000","last_became_active":"2023-11-08T19:26:40.730264+0000","last_became_peered":"2023-11-08T19:26:40.730264+0000","last_unstale":"2023-11-08T19:32:35.663302+0000","last_undegraded":"2023-11-08T19:32:35.663302+0000","last_fullsized":"2023-11-08T19:32:35.663302+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.663149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.663149+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.663149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:08:01.980513+0000","scrub_duration":0.0040296780000000001,"objects_trimmed":0,"snaptrim_duration":0.00117168,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:52.620414+0000","last_change":"2023-11-08T19:32:52.620458+0000","last_active":"2023-11-08T19:32:52.620414+0000","last_peered":"2023-11-08T19:32:52.620414+0000","last_clean":"2023-11-08T19:32:52.620414+0000","last_became_active":"2023-11-08T19:27:47.797350+0000","last_became_peered":"2023-11-08T19:27:47.797350+0000","last_unstale":"2023-11-08T19:32:52.620414+0000","last_undegraded":"2023-11-08T19:32:52.620414+0000","last_fullsized":"2023-11-08T19:32:52.620414+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:52.620263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:52.620263+0000","last_clean_scrub_stamp":"2023-11-08T19:32:52.620263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:19.281606+0000","scrub_duration":0.0038208080000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.691774+0000","last_change":"2023-11-08T19:32:44.691774+0000","last_active":"2023-11-08T19:32:44.691774+0000","last_peered":"2023-11-08T19:32:44.691774+0000","last_clean":"2023-11-08T19:32:44.691774+0000","last_became_active":"2023-11-08T19:27:46.794145+0000","last_became_peered":"2023-11-08T19:27:46.794145+0000","last_unstale":"2023-11-08T19:32:44.691774+0000","last_undegraded":"2023-11-08T19:32:44.691774+0000","last_fullsized":"2023-11-08T19:32:44.691774+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.691532+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.691532+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.691532+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:08:22.520571+0000","scrub_duration":0.0039118929999999996,"objects_trimmed":0,"snaptrim_duration":0.0016887689999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.500518+0000","last_change":"2023-11-08T19:32:34.500610+0000","last_active":"2023-11-08T19:32:34.500518+0000","last_peered":"2023-11-08T19:32:34.500518+0000","last_clean":"2023-11-08T19:32:34.500518+0000","last_became_active":"2023-11-08T19:26:40.723326+0000","last_became_peered":"2023-11-08T19:26:40.723326+0000","last_unstale":"2023-11-08T19:32:34.500518+0000","last_undegraded":"2023-11-08T19:32:34.500518+0000","last_fullsized":"2023-11-08T19:32:34.500518+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.500295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.500295+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.500295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:07:20.761692+0000","scrub_duration":0.0032186850000000002,"objects_trimmed":0,"snaptrim_duration":0.0012868980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:47.682570+0000","last_change":"2023-11-08T19:32:47.682570+0000","last_active":"2023-11-08T19:32:47.682570+0000","last_peered":"2023-11-08T19:32:47.682570+0000","last_clean":"2023-11-08T19:32:47.682570+0000","last_became_active":"2023-11-08T19:27:47.801818+0000","last_became_peered":"2023-11-08T19:27:47.801818+0000","last_unstale":"2023-11-08T19:32:47.682570+0000","last_undegraded":"2023-11-08T19:32:47.682570+0000","last_fullsized":"2023-11-08T19:32:47.682570+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:47.682372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:47.682372+0000","last_clean_scrub_stamp":"2023-11-08T19:32:47.682372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:34:23.371271+0000","scrub_duration":0.0034803009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:48.606882+0000","last_change":"2023-11-08T19:32:48.606932+0000","last_active":"2023-11-08T19:32:48.606882+0000","last_peered":"2023-11-08T19:32:48.606882+0000","last_clean":"2023-11-08T19:32:48.606882+0000","last_became_active":"2023-11-08T19:27:46.796217+0000","last_became_peered":"2023-11-08T19:27:46.796217+0000","last_unstale":"2023-11-08T19:32:48.606882+0000","last_undegraded":"2023-11-08T19:32:48.606882+0000","last_fullsized":"2023-11-08T19:32:48.606882+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:48.606714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:48.606714+0000","last_clean_scrub_stamp":"2023-11-08T19:32:48.606714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:42.003937+0000","scrub_duration":0.003633795,"objects_trimmed":0,"snaptrim_duration":0.002083681,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":84,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.631351+0000","last_change":"2023-11-08T19:32:26.631371+0000","last_active":"2023-11-08T19:32:26.631351+0000","last_peered":"2023-11-08T19:32:26.631351+0000","last_clean":"2023-11-08T19:32:26.631351+0000","last_became_active":"2023-11-08T19:26:40.724376+0000","last_became_peered":"2023-11-08T19:26:40.724376+0000","last_unstale":"2023-11-08T19:32:26.631351+0000","last_undegraded":"2023-11-08T19:32:26.631351+0000","last_fullsized":"2023-11-08T19:32:26.631351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.631277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.631277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:31:19.710068+0000","scrub_duration":0.0020387719999999999,"objects_trimmed":0,"snaptrim_duration":0.0021562999999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":54,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:51.696393+0000","last_change":"2023-11-08T19:32:51.696440+0000","last_active":"2023-11-08T19:32:51.696393+0000","last_peered":"2023-11-08T19:32:51.696393+0000","last_clean":"2023-11-08T19:32:51.696393+0000","last_became_active":"2023-11-08T19:27:47.812087+0000","last_became_peered":"2023-11-08T19:27:47.812087+0000","last_unstale":"2023-11-08T19:32:51.696393+0000","last_undegraded":"2023-11-08T19:32:51.696393+0000","last_fullsized":"2023-11-08T19:32:51.696393+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:51.696242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:51.696242+0000","last_clean_scrub_stamp":"2023-11-08T19:32:51.696242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:47.507076+0000","scrub_duration":0.003286059,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:32.749081+0000","last_change":"2023-11-08T19:32:32.749081+0000","last_active":"2023-11-08T19:32:32.749081+0000","last_peered":"2023-11-08T19:32:32.749081+0000","last_clean":"2023-11-08T19:32:32.749081+0000","last_became_active":"2023-11-08T19:27:46.803891+0000","last_became_peered":"2023-11-08T19:27:46.803891+0000","last_unstale":"2023-11-08T19:32:32.749081+0000","last_undegraded":"2023-11-08T19:32:32.749081+0000","last_fullsized":"2023-11-08T19:32:32.749081+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:32.748790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:32.748790+0000","last_clean_scrub_stamp":"2023-11-08T19:32:32.748790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:56:07.459426+0000","scrub_duration":0.0049107079999999997,"objects_trimmed":0,"snaptrim_duration":0.001399101,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.898760+0000","last_change":"2023-11-08T19:32:29.898810+0000","last_active":"2023-11-08T19:32:29.898760+0000","last_peered":"2023-11-08T19:32:29.898760+0000","last_clean":"2023-11-08T19:32:29.898760+0000","last_became_active":"2023-11-08T19:26:40.724848+0000","last_became_peered":"2023-11-08T19:26:40.724848+0000","last_unstale":"2023-11-08T19:32:29.898760+0000","last_undegraded":"2023-11-08T19:32:29.898760+0000","last_fullsized":"2023-11-08T19:32:29.898760+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.898611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.898611+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.898611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:17:10.217632+0000","scrub_duration":0.0034633910000000001,"objects_trimmed":0,"snaptrim_duration":0.0015907269999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:51.646194+0000","last_change":"2023-11-08T19:32:51.646238+0000","last_active":"2023-11-08T19:32:51.646194+0000","last_peered":"2023-11-08T19:32:51.646194+0000","last_clean":"2023-11-08T19:32:51.646194+0000","last_became_active":"2023-11-08T19:27:47.808023+0000","last_became_peered":"2023-11-08T19:27:47.808023+0000","last_unstale":"2023-11-08T19:32:51.646194+0000","last_undegraded":"2023-11-08T19:32:51.646194+0000","last_fullsized":"2023-11-08T19:32:51.646194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:51.646032+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:51.646032+0000","last_clean_scrub_stamp":"2023-11-08T19:32:51.646032+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:20.413723+0000","scrub_duration":0.0038116949999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:43.834776+0000","last_change":"2023-11-08T19:32:43.834821+0000","last_active":"2023-11-08T19:32:43.834776+0000","last_peered":"2023-11-08T19:32:43.834776+0000","last_clean":"2023-11-08T19:32:43.834776+0000","last_became_active":"2023-11-08T19:27:46.795453+0000","last_became_peered":"2023-11-08T19:27:46.795453+0000","last_unstale":"2023-11-08T19:32:43.834776+0000","last_undegraded":"2023-11-08T19:32:43.834776+0000","last_fullsized":"2023-11-08T19:32:43.834776+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:43.834622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:43.834622+0000","last_clean_scrub_stamp":"2023-11-08T19:32:43.834622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:57:53.189317+0000","scrub_duration":0.0032581189999999999,"objects_trimmed":0,"snaptrim_duration":0.0010516309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:59.705826+0000","last_change":"2023-11-08T19:32:59.705879+0000","last_active":"2023-11-08T19:32:59.705826+0000","last_peered":"2023-11-08T19:32:59.705826+0000","last_clean":"2023-11-08T19:32:59.705826+0000","last_became_active":"2023-11-08T19:26:40.730010+0000","last_became_peered":"2023-11-08T19:26:40.730010+0000","last_unstale":"2023-11-08T19:32:59.705826+0000","last_undegraded":"2023-11-08T19:32:59.705826+0000","last_fullsized":"2023-11-08T19:32:59.705826+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:59.705654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:59.705654+0000","last_clean_scrub_stamp":"2023-11-08T19:32:59.705654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:00.973540+0000","scrub_duration":0.0040140590000000004,"objects_trimmed":0,"snaptrim_duration":0.001901254,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:01.608751+0000","last_change":"2023-11-08T19:33:01.608805+0000","last_active":"2023-11-08T19:33:01.608751+0000","last_peered":"2023-11-08T19:33:01.608751+0000","last_clean":"2023-11-08T19:33:01.608751+0000","last_became_active":"2023-11-08T19:27:47.815360+0000","last_became_peered":"2023-11-08T19:27:47.815360+0000","last_unstale":"2023-11-08T19:33:01.608751+0000","last_undegraded":"2023-11-08T19:33:01.608751+0000","last_fullsized":"2023-11-08T19:33:01.608751+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:01.608590+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:01.608590+0000","last_clean_scrub_stamp":"2023-11-08T19:33:01.608590+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:19.044577+0000","scrub_duration":0.0034235480000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.707482+0000","last_change":"2023-11-08T19:32:35.707531+0000","last_active":"2023-11-08T19:32:35.707482+0000","last_peered":"2023-11-08T19:32:35.707482+0000","last_clean":"2023-11-08T19:32:35.707482+0000","last_became_active":"2023-11-08T19:27:46.796466+0000","last_became_peered":"2023-11-08T19:27:46.796466+0000","last_unstale":"2023-11-08T19:32:35.707482+0000","last_undegraded":"2023-11-08T19:32:35.707482+0000","last_fullsized":"2023-11-08T19:32:35.707482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.707332+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.707332+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.707332+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:36:47.701015+0000","scrub_duration":0.0033186940000000001,"objects_trimmed":0,"snaptrim_duration":0.0015255990000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":88,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:50.853323+0000","last_change":"2023-11-08T19:32:50.853323+0000","last_active":"2023-11-08T19:32:50.853323+0000","last_peered":"2023-11-08T19:32:50.853323+0000","last_clean":"2023-11-08T19:32:50.853323+0000","last_became_active":"2023-11-08T19:26:40.726812+0000","last_became_peered":"2023-11-08T19:26:40.726812+0000","last_unstale":"2023-11-08T19:32:50.853323+0000","last_undegraded":"2023-11-08T19:32:50.853323+0000","last_fullsized":"2023-11-08T19:32:50.853323+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:50.853102+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:50.853102+0000","last_clean_scrub_stamp":"2023-11-08T19:32:50.853102+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:45:18.388978+0000","scrub_duration":0.004043474,"objects_trimmed":0,"snaptrim_duration":0.001908121,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:58.588361+0000","last_change":"2023-11-08T19:32:58.588408+0000","last_active":"2023-11-08T19:32:58.588361+0000","last_peered":"2023-11-08T19:32:58.588361+0000","last_clean":"2023-11-08T19:32:58.588361+0000","last_became_active":"2023-11-08T19:27:47.804651+0000","last_became_peered":"2023-11-08T19:27:47.804651+0000","last_unstale":"2023-11-08T19:32:58.588361+0000","last_undegraded":"2023-11-08T19:32:58.588361+0000","last_fullsized":"2023-11-08T19:32:58.588361+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:58.588189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:58.588189+0000","last_clean_scrub_stamp":"2023-11-08T19:32:58.588189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:24.518455+0000","scrub_duration":0.00330525,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.755805+0000","last_change":"2023-11-08T19:32:36.755856+0000","last_active":"2023-11-08T19:32:36.755805+0000","last_peered":"2023-11-08T19:32:36.755805+0000","last_clean":"2023-11-08T19:32:36.755805+0000","last_became_active":"2023-11-08T19:27:46.805532+0000","last_became_peered":"2023-11-08T19:27:46.805532+0000","last_unstale":"2023-11-08T19:32:36.755805+0000","last_undegraded":"2023-11-08T19:32:36.755805+0000","last_fullsized":"2023-11-08T19:32:36.755805+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.755645+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.755645+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.755645+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:59:34.826462+0000","scrub_duration":0.0037998400000000001,"objects_trimmed":0,"snaptrim_duration":0.0012459019999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":88,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:00.726374+0000","last_change":"2023-11-08T19:33:00.726374+0000","last_active":"2023-11-08T19:33:00.726374+0000","last_peered":"2023-11-08T19:33:00.726374+0000","last_clean":"2023-11-08T19:33:00.726374+0000","last_became_active":"2023-11-08T19:26:40.723876+0000","last_became_peered":"2023-11-08T19:26:40.723876+0000","last_unstale":"2023-11-08T19:33:00.726374+0000","last_undegraded":"2023-11-08T19:33:00.726374+0000","last_fullsized":"2023-11-08T19:33:00.726374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:00.726176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:00.726176+0000","last_clean_scrub_stamp":"2023-11-08T19:33:00.726176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:36:33.504769+0000","scrub_duration":0.0035468000000000001,"objects_trimmed":0,"snaptrim_duration":0.0020565620000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:05.707629+0000","last_change":"2023-11-08T19:33:05.707676+0000","last_active":"2023-11-08T19:33:05.707629+0000","last_peered":"2023-11-08T19:33:05.707629+0000","last_clean":"2023-11-08T19:33:05.707629+0000","last_became_active":"2023-11-08T19:27:47.802850+0000","last_became_peered":"2023-11-08T19:27:47.802850+0000","last_unstale":"2023-11-08T19:33:05.707629+0000","last_undegraded":"2023-11-08T19:33:05.707629+0000","last_fullsized":"2023-11-08T19:33:05.707629+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:05.707419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:05.707419+0000","last_clean_scrub_stamp":"2023-11-08T19:33:05.707419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T07:14:24.111269+0000","scrub_duration":0.0038339580000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.901318+0000","last_change":"2023-11-08T19:32:29.901364+0000","last_active":"2023-11-08T19:32:29.901318+0000","last_peered":"2023-11-08T19:32:29.901318+0000","last_clean":"2023-11-08T19:32:29.901318+0000","last_became_active":"2023-11-08T19:27:46.808032+0000","last_became_peered":"2023-11-08T19:27:46.808032+0000","last_unstale":"2023-11-08T19:32:29.901318+0000","last_undegraded":"2023-11-08T19:32:29.901318+0000","last_fullsized":"2023-11-08T19:32:29.901318+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.901159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.901159+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.901159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:18.596622+0000","scrub_duration":0.0039081070000000001,"objects_trimmed":0,"snaptrim_duration":0.0017030070000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:27.887999+0000","last_change":"2023-11-08T19:32:27.887999+0000","last_active":"2023-11-08T19:32:27.887999+0000","last_peered":"2023-11-08T19:32:27.887999+0000","last_clean":"2023-11-08T19:32:27.887999+0000","last_became_active":"2023-11-08T19:26:40.722128+0000","last_became_peered":"2023-11-08T19:26:40.722128+0000","last_unstale":"2023-11-08T19:32:27.887999+0000","last_undegraded":"2023-11-08T19:32:27.887999+0000","last_fullsized":"2023-11-08T19:32:27.887999+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:27.887914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:27.887914+0000","last_clean_scrub_stamp":"2023-11-08T19:32:27.887914+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:04:54.774642+0000","scrub_duration":0.0022926560000000001,"objects_trimmed":0,"snaptrim_duration":0.001173767,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:21.747288+0000","last_change":"2023-11-08T19:33:21.747335+0000","last_active":"2023-11-08T19:33:21.747288+0000","last_peered":"2023-11-08T19:33:21.747288+0000","last_clean":"2023-11-08T19:33:21.747288+0000","last_became_active":"2023-11-08T19:27:47.808118+0000","last_became_peered":"2023-11-08T19:27:47.808118+0000","last_unstale":"2023-11-08T19:33:21.747288+0000","last_undegraded":"2023-11-08T19:33:21.747288+0000","last_fullsized":"2023-11-08T19:33:21.747288+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:21.747114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:21.747114+0000","last_clean_scrub_stamp":"2023-11-08T19:33:21.747114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:27:39.183101+0000","scrub_duration":0.0034870470000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:09.816958+0000","last_change":"2023-11-08T19:33:09.817044+0000","last_active":"2023-11-08T19:33:09.816958+0000","last_peered":"2023-11-08T19:33:09.816958+0000","last_clean":"2023-11-08T19:33:09.816958+0000","last_became_active":"2023-11-08T19:27:46.805247+0000","last_became_peered":"2023-11-08T19:27:46.805247+0000","last_unstale":"2023-11-08T19:33:09.816958+0000","last_undegraded":"2023-11-08T19:33:09.816958+0000","last_fullsized":"2023-11-08T19:33:09.816958+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:09.816763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:09.816763+0000","last_clean_scrub_stamp":"2023-11-08T19:33:09.816763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:47:03.213986+0000","scrub_duration":0.0037446630000000001,"objects_trimmed":0,"snaptrim_duration":0.0017149450000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:26.981899+0000","last_change":"2023-11-08T19:32:26.981899+0000","last_active":"2023-11-08T19:32:26.981899+0000","last_peered":"2023-11-08T19:32:26.981899+0000","last_clean":"2023-11-08T19:32:26.981899+0000","last_became_active":"2023-11-08T19:26:40.732461+0000","last_became_peered":"2023-11-08T19:26:40.732461+0000","last_unstale":"2023-11-08T19:32:26.981899+0000","last_undegraded":"2023-11-08T19:32:26.981899+0000","last_fullsized":"2023-11-08T19:32:26.981899+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:26.981756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:26.981756+0000","last_clean_scrub_stamp":"2023-11-08T19:32:26.981756+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:20.096248+0000","scrub_duration":0.0030939209999999999,"objects_trimmed":0,"snaptrim_duration":0.00057714799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:54.895016+0000","last_change":"2023-11-08T19:32:54.895068+0000","last_active":"2023-11-08T19:32:54.895016+0000","last_peered":"2023-11-08T19:32:54.895016+0000","last_clean":"2023-11-08T19:32:54.895016+0000","last_became_active":"2023-11-08T19:27:47.803312+0000","last_became_peered":"2023-11-08T19:27:47.803312+0000","last_unstale":"2023-11-08T19:32:54.895016+0000","last_undegraded":"2023-11-08T19:32:54.895016+0000","last_fullsized":"2023-11-08T19:32:54.895016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:54.894835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:54.894835+0000","last_clean_scrub_stamp":"2023-11-08T19:32:54.894835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:03:08.993720+0000","scrub_duration":0.0035580619999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:49.643705+0000","last_change":"2023-11-08T19:32:49.643753+0000","last_active":"2023-11-08T19:32:49.643705+0000","last_peered":"2023-11-08T19:32:49.643705+0000","last_clean":"2023-11-08T19:32:49.643705+0000","last_became_active":"2023-11-08T19:27:46.794527+0000","last_became_peered":"2023-11-08T19:27:46.794527+0000","last_unstale":"2023-11-08T19:32:49.643705+0000","last_undegraded":"2023-11-08T19:32:49.643705+0000","last_fullsized":"2023-11-08T19:32:49.643705+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:49.643546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:49.643546+0000","last_clean_scrub_stamp":"2023-11-08T19:32:49.643546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:18.411237+0000","scrub_duration":0.0038580799999999998,"objects_trimmed":0,"snaptrim_duration":0.0023295099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:39.855592+0000","last_change":"2023-11-08T19:32:39.855592+0000","last_active":"2023-11-08T19:32:39.855592+0000","last_peered":"2023-11-08T19:32:39.855592+0000","last_clean":"2023-11-08T19:32:39.855592+0000","last_became_active":"2023-11-08T19:26:40.728011+0000","last_became_peered":"2023-11-08T19:26:40.728011+0000","last_unstale":"2023-11-08T19:32:39.855592+0000","last_undegraded":"2023-11-08T19:32:39.855592+0000","last_fullsized":"2023-11-08T19:32:39.855592+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:39.855386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:39.855386+0000","last_clean_scrub_stamp":"2023-11-08T19:32:39.855386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:57:11.250998+0000","scrub_duration":0.0035043140000000001,"objects_trimmed":0,"snaptrim_duration":0.00090141400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:44.598433+0000","last_change":"2023-11-08T19:32:44.598517+0000","last_active":"2023-11-08T19:32:44.598433+0000","last_peered":"2023-11-08T19:32:44.598433+0000","last_clean":"2023-11-08T19:32:44.598433+0000","last_became_active":"2023-11-08T19:27:47.805769+0000","last_became_peered":"2023-11-08T19:27:47.805769+0000","last_unstale":"2023-11-08T19:32:44.598433+0000","last_undegraded":"2023-11-08T19:32:44.598433+0000","last_fullsized":"2023-11-08T19:32:44.598433+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:44.598277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:44.598277+0000","last_clean_scrub_stamp":"2023-11-08T19:32:44.598277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:32:33.293740+0000","scrub_duration":0.0035716250000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:24.848482+0000","last_change":"2023-11-08T19:32:24.848528+0000","last_active":"2023-11-08T19:32:24.848482+0000","last_peered":"2023-11-08T19:32:24.848482+0000","last_clean":"2023-11-08T19:32:24.848482+0000","last_became_active":"2023-11-08T19:27:46.806577+0000","last_became_peered":"2023-11-08T19:27:46.806577+0000","last_unstale":"2023-11-08T19:32:24.848482+0000","last_undegraded":"2023-11-08T19:32:24.848482+0000","last_fullsized":"2023-11-08T19:32:24.848482+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:24.848327+0000","last_clean_scrub_stamp":"2023-11-08T19:32:24.848327+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:04:22.967345+0000","scrub_duration":0.0035992749999999999,"objects_trimmed":0,"snaptrim_duration":0.00080549500000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:28.019603+0000","last_change":"2023-11-08T19:32:28.019603+0000","last_active":"2023-11-08T19:32:28.019603+0000","last_peered":"2023-11-08T19:32:28.019603+0000","last_clean":"2023-11-08T19:32:28.019603+0000","last_became_active":"2023-11-08T19:26:40.722837+0000","last_became_peered":"2023-11-08T19:26:40.722837+0000","last_unstale":"2023-11-08T19:32:28.019603+0000","last_undegraded":"2023-11-08T19:32:28.019603+0000","last_fullsized":"2023-11-08T19:32:28.019603+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:28.019513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:28.019513+0000","last_clean_scrub_stamp":"2023-11-08T19:32:28.019513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:31:18.418880+0000","scrub_duration":0.0027169690000000001,"objects_trimmed":0,"snaptrim_duration":0.00067612,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:19.828686+0000","last_change":"2023-11-08T19:33:19.828737+0000","last_active":"2023-11-08T19:33:19.828686+0000","last_peered":"2023-11-08T19:33:19.828686+0000","last_clean":"2023-11-08T19:33:19.828686+0000","last_became_active":"2023-11-08T19:27:47.806508+0000","last_became_peered":"2023-11-08T19:27:47.806508+0000","last_unstale":"2023-11-08T19:33:19.828686+0000","last_undegraded":"2023-11-08T19:33:19.828686+0000","last_fullsized":"2023-11-08T19:33:19.828686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:19.828524+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:19.828524+0000","last_clean_scrub_stamp":"2023-11-08T19:33:19.828524+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:49.248006+0000","scrub_duration":0.0038991400000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":65,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:40.655839+0000","last_change":"2023-11-08T19:32:40.655884+0000","last_active":"2023-11-08T19:32:40.655839+0000","last_peered":"2023-11-08T19:32:40.655839+0000","last_clean":"2023-11-08T19:32:40.655839+0000","last_became_active":"2023-11-08T19:27:46.805618+0000","last_became_peered":"2023-11-08T19:27:46.805618+0000","last_unstale":"2023-11-08T19:32:40.655839+0000","last_undegraded":"2023-11-08T19:32:40.655839+0000","last_fullsized":"2023-11-08T19:32:40.655839+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:40.655688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:40.655688+0000","last_clean_scrub_stamp":"2023-11-08T19:32:40.655688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:03:41.923574+0000","scrub_duration":0.003438927,"objects_trimmed":0,"snaptrim_duration":0.0031022340000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:25.703850+0000","last_change":"2023-11-08T19:32:25.703850+0000","last_active":"2023-11-08T19:32:25.703850+0000","last_peered":"2023-11-08T19:32:25.703850+0000","last_clean":"2023-11-08T19:32:25.703850+0000","last_became_active":"2023-11-08T19:26:40.734463+0000","last_became_peered":"2023-11-08T19:26:40.734463+0000","last_unstale":"2023-11-08T19:32:25.703850+0000","last_undegraded":"2023-11-08T19:32:25.703850+0000","last_fullsized":"2023-11-08T19:32:25.703850+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:25.703734+0000","last_clean_scrub_stamp":"2023-11-08T19:32:25.703734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:21:40.388881+0000","scrub_duration":0.0027277820000000002,"objects_trimmed":0,"snaptrim_duration":0.001351362,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:52.662805+0000","last_change":"2023-11-08T19:32:52.662835+0000","last_active":"2023-11-08T19:32:52.662805+0000","last_peered":"2023-11-08T19:32:52.662805+0000","last_clean":"2023-11-08T19:32:52.662805+0000","last_became_active":"2023-11-08T19:27:47.812065+0000","last_became_peered":"2023-11-08T19:27:47.812065+0000","last_unstale":"2023-11-08T19:32:52.662805+0000","last_undegraded":"2023-11-08T19:32:52.662805+0000","last_fullsized":"2023-11-08T19:32:52.662805+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:52.662726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:52.662726+0000","last_clean_scrub_stamp":"2023-11-08T19:32:52.662726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:19.324070+0000","scrub_duration":0.003021709,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.950835+0000","last_change":"2023-11-08T19:32:30.950919+0000","last_active":"2023-11-08T19:32:30.950835+0000","last_peered":"2023-11-08T19:32:30.950835+0000","last_clean":"2023-11-08T19:32:30.950835+0000","last_became_active":"2023-11-08T19:27:46.808822+0000","last_became_peered":"2023-11-08T19:27:46.808822+0000","last_unstale":"2023-11-08T19:32:30.950835+0000","last_undegraded":"2023-11-08T19:32:30.950835+0000","last_fullsized":"2023-11-08T19:32:30.950835+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.950623+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.950623+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.950623+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:53.223016+0000","scrub_duration":0.0042637869999999998,"objects_trimmed":0,"snaptrim_duration":0.001299297,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.657516+0000","last_change":"2023-11-08T19:32:30.657516+0000","last_active":"2023-11-08T19:32:30.657516+0000","last_peered":"2023-11-08T19:32:30.657516+0000","last_clean":"2023-11-08T19:32:30.657516+0000","last_became_active":"2023-11-08T19:26:40.718357+0000","last_became_peered":"2023-11-08T19:26:40.718357+0000","last_unstale":"2023-11-08T19:32:30.657516+0000","last_undegraded":"2023-11-08T19:32:30.657516+0000","last_fullsized":"2023-11-08T19:32:30.657516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.657305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.657305+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.657305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:01:57.318648+0000","scrub_duration":0.003567045,"objects_trimmed":0,"snaptrim_duration":0.0011027820000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":57,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:24.702978+0000","last_change":"2023-11-08T19:33:24.703024+0000","last_active":"2023-11-08T19:33:24.702978+0000","last_peered":"2023-11-08T19:33:24.702978+0000","last_clean":"2023-11-08T19:33:24.702978+0000","last_became_active":"2023-11-08T19:27:47.816224+0000","last_became_peered":"2023-11-08T19:27:47.816224+0000","last_unstale":"2023-11-08T19:33:24.702978+0000","last_undegraded":"2023-11-08T19:33:24.702978+0000","last_fullsized":"2023-11-08T19:33:24.702978+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:24.702817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:24.702817+0000","last_clean_scrub_stamp":"2023-11-08T19:33:24.702817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:41:20.144989+0000","scrub_duration":0.0035337459999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:52.908195+0000","last_change":"2023-11-08T19:32:52.908241+0000","last_active":"2023-11-08T19:32:52.908195+0000","last_peered":"2023-11-08T19:32:52.908195+0000","last_clean":"2023-11-08T19:32:52.908195+0000","last_became_active":"2023-11-08T19:27:46.809426+0000","last_became_peered":"2023-11-08T19:27:46.809426+0000","last_unstale":"2023-11-08T19:32:52.908195+0000","last_undegraded":"2023-11-08T19:32:52.908195+0000","last_fullsized":"2023-11-08T19:32:52.908195+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:52.908042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:52.908042+0000","last_clean_scrub_stamp":"2023-11-08T19:32:52.908042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:59:27.527625+0000","scrub_duration":0.003437729,"objects_trimmed":0,"snaptrim_duration":0.0014608309999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":87,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:46.652935+0000","last_change":"2023-11-08T19:32:46.652935+0000","last_active":"2023-11-08T19:32:46.652935+0000","last_peered":"2023-11-08T19:32:46.652935+0000","last_clean":"2023-11-08T19:32:46.652935+0000","last_became_active":"2023-11-08T19:26:40.724967+0000","last_became_peered":"2023-11-08T19:26:40.724967+0000","last_unstale":"2023-11-08T19:32:46.652935+0000","last_undegraded":"2023-11-08T19:32:46.652935+0000","last_fullsized":"2023-11-08T19:32:46.652935+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:46.652731+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:46.652731+0000","last_clean_scrub_stamp":"2023-11-08T19:32:46.652731+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:05:57.268837+0000","scrub_duration":0.0036855339999999999,"objects_trimmed":0,"snaptrim_duration":0.001134997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:54.587249+0000","last_change":"2023-11-08T19:32:54.587295+0000","last_active":"2023-11-08T19:32:54.587249+0000","last_peered":"2023-11-08T19:32:54.587249+0000","last_clean":"2023-11-08T19:32:54.587249+0000","last_became_active":"2023-11-08T19:27:47.812271+0000","last_became_peered":"2023-11-08T19:27:47.812271+0000","last_unstale":"2023-11-08T19:32:54.587249+0000","last_undegraded":"2023-11-08T19:32:54.587249+0000","last_fullsized":"2023-11-08T19:32:54.587249+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:54.587090+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:54.587090+0000","last_clean_scrub_stamp":"2023-11-08T19:32:54.587090+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:17.204659+0000","scrub_duration":0.003566053,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":64,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:33:11.879306+0000","last_change":"2023-11-08T19:33:11.879356+0000","last_active":"2023-11-08T19:33:11.879306+0000","last_peered":"2023-11-08T19:33:11.879306+0000","last_clean":"2023-11-08T19:33:11.879306+0000","last_became_active":"2023-11-08T19:27:46.809032+0000","last_became_peered":"2023-11-08T19:27:46.809032+0000","last_unstale":"2023-11-08T19:33:11.879306+0000","last_undegraded":"2023-11-08T19:33:11.879306+0000","last_fullsized":"2023-11-08T19:33:11.879306+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:33:11.879134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:33:11.879134+0000","last_clean_scrub_stamp":"2023-11-08T19:33:11.879134+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:33:40.646825+0000","scrub_duration":0.003326533,"objects_trimmed":0,"snaptrim_duration":0.0019952329999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:42.878282+0000","last_change":"2023-11-08T19:32:42.878282+0000","last_active":"2023-11-08T19:32:42.878282+0000","last_peered":"2023-11-08T19:32:42.878282+0000","last_clean":"2023-11-08T19:32:42.878282+0000","last_became_active":"2023-11-08T19:26:40.727218+0000","last_became_peered":"2023-11-08T19:26:40.727218+0000","last_unstale":"2023-11-08T19:32:42.878282+0000","last_undegraded":"2023-11-08T19:32:42.878282+0000","last_fullsized":"2023-11-08T19:32:42.878282+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:42.878052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:42.878052+0000","last_clean_scrub_stamp":"2023-11-08T19:32:42.878052+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:41.200867+0000","scrub_duration":0.003533516,"objects_trimmed":0,"snaptrim_duration":0.0013685279999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":58,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:56.571922+0000","last_change":"2023-11-08T19:32:56.571973+0000","last_active":"2023-11-08T19:32:56.571922+0000","last_peered":"2023-11-08T19:32:56.571922+0000","last_clean":"2023-11-08T19:32:56.571922+0000","last_became_active":"2023-11-08T19:27:47.802111+0000","last_became_peered":"2023-11-08T19:27:47.802111+0000","last_unstale":"2023-11-08T19:32:56.571922+0000","last_undegraded":"2023-11-08T19:32:56.571922+0000","last_fullsized":"2023-11-08T19:32:56.571922+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:56.571754+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:56.571754+0000","last_clean_scrub_stamp":"2023-11-08T19:32:56.571754+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:45:19.189323+0000","scrub_duration":0.004365865,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:50.888488+0000","last_change":"2023-11-08T19:32:50.888537+0000","last_active":"2023-11-08T19:32:50.888488+0000","last_peered":"2023-11-08T19:32:50.888488+0000","last_clean":"2023-11-08T19:32:50.888488+0000","last_became_active":"2023-11-08T19:27:46.797315+0000","last_became_peered":"2023-11-08T19:27:46.797315+0000","last_unstale":"2023-11-08T19:32:50.888488+0000","last_undegraded":"2023-11-08T19:32:50.888488+0000","last_fullsized":"2023-11-08T19:32:50.888488+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:50.888333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:50.888333+0000","last_clean_scrub_stamp":"2023-11-08T19:32:50.888333+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:40.360814+0000","scrub_duration":0.003364494,"objects_trimmed":0,"snaptrim_duration":0.001667595,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":88,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:47.920754+0000","last_change":"2023-11-08T19:32:47.920754+0000","last_active":"2023-11-08T19:32:47.920754+0000","last_peered":"2023-11-08T19:32:47.920754+0000","last_clean":"2023-11-08T19:32:47.920754+0000","last_became_active":"2023-11-08T19:26:40.730589+0000","last_became_peered":"2023-11-08T19:26:40.730589+0000","last_unstale":"2023-11-08T19:32:47.920754+0000","last_undegraded":"2023-11-08T19:32:47.920754+0000","last_fullsized":"2023-11-08T19:32:47.920754+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:47.920450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:47.920450+0000","last_clean_scrub_stamp":"2023-11-08T19:32:47.920450+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T05:33:28.639203+0000","scrub_duration":0.003623958,"objects_trimmed":0,"snaptrim_duration":0.0019018139999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.633079+0000","last_change":"2023-11-08T19:32:41.633132+0000","last_active":"2023-11-08T19:32:41.633079+0000","last_peered":"2023-11-08T19:32:41.633079+0000","last_clean":"2023-11-08T19:32:41.633079+0000","last_became_active":"2023-11-08T19:27:47.803108+0000","last_became_peered":"2023-11-08T19:27:47.803108+0000","last_unstale":"2023-11-08T19:32:41.633079+0000","last_undegraded":"2023-11-08T19:32:41.633079+0000","last_fullsized":"2023-11-08T19:32:41.633079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.632927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.632927+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.632927+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:56:36.100474+0000","scrub_duration":0.0039924180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.088025+0000","last_change":"2023-11-08T19:32:30.088072+0000","last_active":"2023-11-08T19:32:30.088025+0000","last_peered":"2023-11-08T19:32:30.088025+0000","last_clean":"2023-11-08T19:32:30.088025+0000","last_became_active":"2023-11-08T19:27:46.793019+0000","last_became_peered":"2023-11-08T19:27:46.793019+0000","last_unstale":"2023-11-08T19:32:30.088025+0000","last_undegraded":"2023-11-08T19:32:30.088025+0000","last_fullsized":"2023-11-08T19:32:30.088025+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.087841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.087841+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.087841+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:31:05.489237+0000","scrub_duration":0.003235796,"objects_trimmed":0,"snaptrim_duration":0.00071545099999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":92,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:34.685883+0000","last_change":"2023-11-08T19:32:34.685883+0000","last_active":"2023-11-08T19:32:34.685883+0000","last_peered":"2023-11-08T19:32:34.685883+0000","last_clean":"2023-11-08T19:32:34.685883+0000","last_became_active":"2023-11-08T19:26:40.727773+0000","last_became_peered":"2023-11-08T19:26:40.727773+0000","last_unstale":"2023-11-08T19:32:34.685883+0000","last_undegraded":"2023-11-08T19:32:34.685883+0000","last_fullsized":"2023-11-08T19:32:34.685883+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:34.685687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:34.685687+0000","last_clean_scrub_stamp":"2023-11-08T19:32:34.685687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:39.814571+0000","scrub_duration":0.0041719410000000002,"objects_trimmed":0,"snaptrim_duration":0.0010800149999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.057312+0000","last_change":"2023-11-08T19:32:36.057360+0000","last_active":"2023-11-08T19:32:36.057312+0000","last_peered":"2023-11-08T19:32:36.057312+0000","last_clean":"2023-11-08T19:32:36.057312+0000","last_became_active":"2023-11-08T19:27:47.798916+0000","last_became_peered":"2023-11-08T19:27:47.798916+0000","last_unstale":"2023-11-08T19:32:36.057312+0000","last_undegraded":"2023-11-08T19:32:36.057312+0000","last_fullsized":"2023-11-08T19:32:36.057312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.057153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.057153+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.057153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:57:50.126042+0000","scrub_duration":0.0035364089999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:43.628590+0000","last_change":"2023-11-08T19:32:43.628647+0000","last_active":"2023-11-08T19:32:43.628590+0000","last_peered":"2023-11-08T19:32:43.628590+0000","last_clean":"2023-11-08T19:32:43.628590+0000","last_became_active":"2023-11-08T19:27:46.791959+0000","last_became_peered":"2023-11-08T19:27:46.791959+0000","last_unstale":"2023-11-08T19:32:43.628590+0000","last_undegraded":"2023-11-08T19:32:43.628590+0000","last_fullsized":"2023-11-08T19:32:43.628590+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:43.628419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:43.628419+0000","last_clean_scrub_stamp":"2023-11-08T19:32:43.628419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:42:45.712641+0000","scrub_duration":0.0032515249999999999,"objects_trimmed":0,"snaptrim_duration":0.00063634,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":92,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:30.618782+0000","last_change":"2023-11-08T19:32:30.618782+0000","last_active":"2023-11-08T19:32:30.618782+0000","last_peered":"2023-11-08T19:32:30.618782+0000","last_clean":"2023-11-08T19:32:30.618782+0000","last_became_active":"2023-11-08T19:26:40.734133+0000","last_became_peered":"2023-11-08T19:26:40.734133+0000","last_unstale":"2023-11-08T19:32:30.618782+0000","last_undegraded":"2023-11-08T19:32:30.618782+0000","last_fullsized":"2023-11-08T19:32:30.618782+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:30.618568+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:30.618568+0000","last_clean_scrub_stamp":"2023-11-08T19:32:30.618568+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:54:51.352530+0000","scrub_duration":0.0037122639999999998,"objects_trimmed":0,"snaptrim_duration":0.0013619260000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":59,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:40.632718+0000","last_change":"2023-11-08T19:32:40.632765+0000","last_active":"2023-11-08T19:32:40.632718+0000","last_peered":"2023-11-08T19:32:40.632718+0000","last_clean":"2023-11-08T19:32:40.632718+0000","last_became_active":"2023-11-08T19:27:47.801179+0000","last_became_peered":"2023-11-08T19:27:47.801179+0000","last_unstale":"2023-11-08T19:32:40.632718+0000","last_undegraded":"2023-11-08T19:32:40.632718+0000","last_fullsized":"2023-11-08T19:32:40.632718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:40.632500+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:40.632500+0000","last_clean_scrub_stamp":"2023-11-08T19:32:40.632500+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:28:55.639010+0000","scrub_duration":0.0035395969999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":61,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:41.613815+0000","last_change":"2023-11-08T19:32:41.613901+0000","last_active":"2023-11-08T19:32:41.613815+0000","last_peered":"2023-11-08T19:32:41.613815+0000","last_clean":"2023-11-08T19:32:41.613815+0000","last_became_active":"2023-11-08T19:27:46.797459+0000","last_became_peered":"2023-11-08T19:27:46.797459+0000","last_unstale":"2023-11-08T19:32:41.613815+0000","last_undegraded":"2023-11-08T19:32:41.613815+0000","last_fullsized":"2023-11-08T19:32:41.613815+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:41.613660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:41.613660+0000","last_clean_scrub_stamp":"2023-11-08T19:32:41.613660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:26:59.049648+0000","scrub_duration":0.0031596810000000001,"objects_trimmed":0,"snaptrim_duration":0.00088339899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:36.662271+0000","last_change":"2023-11-08T19:32:36.662271+0000","last_active":"2023-11-08T19:32:36.662271+0000","last_peered":"2023-11-08T19:32:36.662271+0000","last_clean":"2023-11-08T19:32:36.662271+0000","last_became_active":"2023-11-08T19:26:40.721331+0000","last_became_peered":"2023-11-08T19:26:40.721331+0000","last_unstale":"2023-11-08T19:32:36.662271+0000","last_undegraded":"2023-11-08T19:32:36.662271+0000","last_fullsized":"2023-11-08T19:32:36.662271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:36.662065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:36.662065+0000","last_clean_scrub_stamp":"2023-11-08T19:32:36.662065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T06:44:59.279634+0000","scrub_duration":0.0035784039999999999,"objects_trimmed":0,"snaptrim_duration":0.00052128700000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":56,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:33.011945+0000","last_change":"2023-11-08T19:32:33.011999+0000","last_active":"2023-11-08T19:32:33.011945+0000","last_peered":"2023-11-08T19:32:33.011945+0000","last_clean":"2023-11-08T19:32:33.011945+0000","last_became_active":"2023-11-08T19:27:47.805097+0000","last_became_peered":"2023-11-08T19:27:47.805097+0000","last_unstale":"2023-11-08T19:32:33.011945+0000","last_undegraded":"2023-11-08T19:32:33.011945+0000","last_fullsized":"2023-11-08T19:32:33.011945+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:33.011743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:33.011743+0000","last_clean_scrub_stamp":"2023-11-08T19:32:33.011743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:06.887936+0000","scrub_duration":0.0036639329999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":63,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.051468+0000","last_change":"2023-11-08T19:32:29.051518+0000","last_active":"2023-11-08T19:32:29.051468+0000","last_peered":"2023-11-08T19:32:29.051468+0000","last_clean":"2023-11-08T19:32:29.051468+0000","last_became_active":"2023-11-08T19:27:46.803757+0000","last_became_peered":"2023-11-08T19:27:46.803757+0000","last_unstale":"2023-11-08T19:32:29.051468+0000","last_undegraded":"2023-11-08T19:32:29.051468+0000","last_fullsized":"2023-11-08T19:32:29.051468+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:29.051307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:29.051307+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.051307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:38.939562+0000","scrub_duration":0.0038472430000000002,"objects_trimmed":0,"snaptrim_duration":0.00088318700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":85,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:37.646449+0000","last_change":"2023-11-08T19:32:37.646449+0000","last_active":"2023-11-08T19:32:37.646449+0000","last_peered":"2023-11-08T19:32:37.646449+0000","last_clean":"2023-11-08T19:32:37.646449+0000","last_became_active":"2023-11-08T19:26:40.720611+0000","last_became_peered":"2023-11-08T19:26:40.720611+0000","last_unstale":"2023-11-08T19:32:37.646449+0000","last_undegraded":"2023-11-08T19:32:37.646449+0000","last_fullsized":"2023-11-08T19:32:37.646449+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:37.646227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:37.646227+0000","last_clean_scrub_stamp":"2023-11-08T19:32:37.646227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:07.065709+0000","scrub_duration":0.0035584530000000001,"objects_trimmed":0,"snaptrim_duration":0.0021886029999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":55,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:57.687729+0000","last_change":"2023-11-08T19:32:57.687775+0000","last_active":"2023-11-08T19:32:57.687729+0000","last_peered":"2023-11-08T19:32:57.687729+0000","last_clean":"2023-11-08T19:32:57.687729+0000","last_became_active":"2023-11-08T19:27:47.803680+0000","last_became_peered":"2023-11-08T19:27:47.803680+0000","last_unstale":"2023-11-08T19:32:57.687729+0000","last_undegraded":"2023-11-08T19:32:57.687729+0000","last_fullsized":"2023-11-08T19:32:57.687729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:57.687571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:57.687571+0000","last_clean_scrub_stamp":"2023-11-08T19:32:57.687571+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:25:43.215813+0000","scrub_duration":0.0041095289999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":62,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:48.677527+0000","last_change":"2023-11-08T19:32:48.677578+0000","last_active":"2023-11-08T19:32:48.677527+0000","last_peered":"2023-11-08T19:32:48.677527+0000","last_clean":"2023-11-08T19:32:48.677527+0000","last_became_active":"2023-11-08T19:27:46.811367+0000","last_became_peered":"2023-11-08T19:27:46.811367+0000","last_unstale":"2023-11-08T19:32:48.677527+0000","last_undegraded":"2023-11-08T19:32:48.677527+0000","last_fullsized":"2023-11-08T19:32:48.677527+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:48.677329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:48.677329+0000","last_clean_scrub_stamp":"2023-11-08T19:32:48.677329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:46:42.074552+0000","scrub_duration":0.0036287210000000001,"objects_trimmed":0,"snaptrim_duration":0.00095506400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":86,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:35.792054+0000","last_change":"2023-11-08T19:32:35.792054+0000","last_active":"2023-11-08T19:32:35.792054+0000","last_peered":"2023-11-08T19:32:35.792054+0000","last_clean":"2023-11-08T19:32:35.792054+0000","last_became_active":"2023-11-08T19:26:40.728148+0000","last_became_peered":"2023-11-08T19:26:40.728148+0000","last_unstale":"2023-11-08T19:32:35.792054+0000","last_undegraded":"2023-11-08T19:32:35.792054+0000","last_fullsized":"2023-11-08T19:32:35.792054+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T19:32:35.791811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T19:32:35.791811+0000","last_clean_scrub_stamp":"2023-11-08T19:32:35.791811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:51:30.989602+0000","scrub_duration":0.003741978,"objects_trimmed":0,"snaptrim_duration":0.0010996859999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"109'119","reported_seq":258,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-08T19:32:29.572619+0000","last_change":"2023-11-08T19:32:29.572619+0000","last_active":"2023-11-08T19:32:29.572619+0000","last_peered":"2023-11-08T19:32:29.572619+0000","last_clean":"2023-11-08T19:32:29.572619+0000","last_became_active":"2023-11-08T19:24:48.612992+0000","last_became_peered":"2023-11-08T19:24:48.612992+0000","last_unstale":"2023-11-08T19:32:29.572619+0000","last_undegraded":"2023-11-08T19:32:29.572619+0000","last_fullsized":"2023-11-08T19:32:29.572619+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"109'119","last_scrub_stamp":"2023-11-08T19:32:29.572367+0000","last_deep_scrub":"109'119","last_deep_scrub_stamp":"2023-11-08T19:32:29.572367+0000","last_clean_scrub_stamp":"2023-11-08T19:32:29.572367+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:21.179114+0000","scrub_duration":0.0093265610000000006,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574956,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33076,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338764,"statfs":{"total":96636764160,"available":96602894336,"internally_reserved":0,"allocated":1945600,"data_stored":1281536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":1.099,"15min":1.0960000000000001},"min":{"1min":0.71599999999999997,"5min":0.66500000000000004,"15min":0.53000000000000003},"max":{"1min":1.3009999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.25},{"interface":"front","average":{"1min":1.1579999999999999,"5min":1.1279999999999999,"15min":1.24},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.58499999999999996},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":4.4589999999999996},"last":1.387}]},{"osd":1,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.202,"15min":1.2290000000000001},"min":{"1min":0.71599999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5309999999999999,"5min":1.728,"15min":4.5330000000000004},"last":1.504},{"interface":"front","average":{"1min":1.1559999999999999,"5min":1.175,"15min":1.147},"min":{"1min":0.69299999999999995,"5min":0.63900000000000001,"15min":0.57399999999999995},"max":{"1min":1.706,"5min":1.706,"15min":2.0150000000000001},"last":1.649}]},{"osd":2,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3300000000000001,"15min":1.4259999999999999},"min":{"1min":0.83199999999999996,"5min":0.71199999999999997,"15min":0.71199999999999997},"max":{"1min":1.593,"5min":1.891,"15min":4.6980000000000004},"last":1.476},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.2350000000000001,"15min":1.306},"min":{"1min":0.88600000000000001,"5min":0.78500000000000003,"15min":0.64700000000000002},"max":{"1min":1.671,"5min":1.71,"15min":4.4269999999999996},"last":1.5580000000000001}]},{"osd":3,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3,"15min":1.3360000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.69799999999999995},"max":{"1min":1.6459999999999999,"5min":2.0459999999999998,"15min":2.2240000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.405,"15min":1.504},"min":{"1min":0.92200000000000004,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.831,"5min":1.8560000000000001,"15min":4.6150000000000002},"last":1.5960000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.232,"15min":1.3149999999999999},"min":{"1min":0.89100000000000001,"5min":0.624,"15min":0.624},"max":{"1min":1.5429999999999999,"5min":1.75,"15min":2.3170000000000002},"last":1.4299999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.3460000000000001,"15min":1.4350000000000001},"min":{"1min":1.0269999999999999,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7969999999999999,"5min":2.141,"15min":2.2869999999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.3839999999999999,"15min":1.4339999999999999},"min":{"1min":0.97499999999999998,"5min":0.57599999999999996,"15min":0.56699999999999995},"max":{"1min":1.784,"5min":1.8740000000000001,"15min":2.3260000000000001},"last":1.206},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.397,"15min":1.4470000000000001},"min":{"1min":1.0669999999999999,"5min":0.873,"15min":0.81699999999999995},"max":{"1min":1.706,"5min":1.8169999999999999,"15min":2.1680000000000001},"last":1.292}]},{"osd":6,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.4910000000000001,"15min":1.544},"min":{"1min":1.133,"5min":0.93700000000000006,"15min":0.878},"max":{"1min":1.97,"5min":1.97,"15min":2.427},"last":1.681},{"interface":"front","average":{"1min":1.55,"5min":1.4890000000000001,"15min":1.573},"min":{"1min":1.361,"5min":0.81799999999999995,"15min":0.81799999999999995},"max":{"1min":1.8620000000000001,"5min":2.1920000000000002,"15min":2.468},"last":1.3440000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574955,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.1220000000000001,"15min":1.0029999999999999},"min":{"1min":0.71499999999999997,"5min":0.65600000000000003,"15min":0.54600000000000004},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.752},"last":1.224},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.236,"15min":1.151},"min":{"1min":0.752,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.6080000000000001,"5min":8.391,"15min":8.391},"last":0.874}]},{"osd":1,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.163,"15min":1.115},"min":{"1min":0.83599999999999997,"5min":0.68899999999999995,"15min":0.68100000000000005},"max":{"1min":1.752,"5min":1.752,"15min":1.752},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.2350000000000001,"15min":1.2929999999999999},"min":{"1min":0.91800000000000004,"5min":0.69999999999999996,"15min":0.51200000000000001},"max":{"1min":1.6020000000000001,"5min":1.665,"15min":4.6399999999999997},"last":1.387}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2170000000000001,"15min":1.258},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.54600000000000004},"max":{"1min":2.5150000000000001,"5min":2.5150000000000001,"15min":4.609},"last":1.4119999999999999},{"interface":"front","average":{"1min":1.341,"5min":1.3069999999999999,"15min":1.2509999999999999},"min":{"1min":0.95999999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.861},"last":1.145}]},{"osd":3,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.2929999999999999,"15min":1.4139999999999999},"min":{"1min":0.92400000000000004,"5min":0.77900000000000003,"15min":0.56599999999999995},"max":{"1min":1.714,"5min":1.714,"15min":4.7190000000000003},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.272,"15min":1.2110000000000001},"min":{"1min":0.79300000000000004,"5min":0.625,"15min":0.625},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.5700000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.286,"15min":1.345},"min":{"1min":0.89500000000000002,"5min":0.64100000000000001,"15min":0.51900000000000002},"max":{"1min":1.8140000000000001,"5min":1.881,"15min":2.0449999999999999},"last":1.091},{"interface":"front","average":{"1min":1.5169999999999999,"5min":1.464,"15min":1.4930000000000001},"min":{"1min":1.226,"5min":0.75700000000000001,"15min":0.40899999999999997},"max":{"1min":1.913,"5min":1.998,"15min":2.3130000000000002},"last":1.296}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.294,"15min":1.268},"min":{"1min":0.86599999999999999,"5min":0.54900000000000004,"15min":0.35499999999999998},"max":{"1min":1.5780000000000001,"5min":1.835,"15min":1.9079999999999999},"last":1.641},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.377,"15min":1.351},"min":{"1min":0.84599999999999997,"5min":0.47999999999999998,"15min":0.38700000000000001},"max":{"1min":2.0299999999999998,"5min":2.0299999999999998,"15min":2.0489999999999999},"last":1.3500000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.5960000000000001,"5min":1.552,"15min":1.5529999999999999},"min":{"1min":1.359,"5min":0.82699999999999996,"15min":0.56399999999999995},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.468,"15min":1.4590000000000001},"min":{"1min":1.0329999999999999,"5min":0.505,"15min":0.45000000000000001},"max":{"1min":2.1890000000000001,"5min":2.1890000000000001,"15min":3.0640000000000001},"last":1.696}]}]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.228,"15min":1.1279999999999999},"min":{"1min":0.82399999999999995,"5min":0.64500000000000002,"15min":0.57699999999999996},"max":{"1min":1.5740000000000001,"5min":1.724,"15min":1.76},"last":1.661},{"interface":"front","average":{"1min":1.323,"5min":1.3069999999999999,"15min":1.258},"min":{"1min":0.89500000000000002,"5min":0.73599999999999999,"15min":0.61499999999999999},"max":{"1min":1.6679999999999999,"5min":1.837,"15min":1.954},"last":1.3720000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":1.1519999999999999,"15min":1.159},"min":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.58699999999999997},"max":{"1min":1.655,"5min":1.655,"15min":2.3039999999999998},"last":1.145},{"interface":"front","average":{"1min":1.238,"5min":1.1799999999999999,"15min":1.3380000000000001},"min":{"1min":0.78100000000000003,"5min":0.68999999999999995,"15min":0.67100000000000004},"max":{"1min":1.679,"5min":1.8009999999999999,"15min":7.6779999999999999},"last":1.4470000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.258,"15min":1.202},"min":{"1min":1.0609999999999999,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.643,"5min":1.7729999999999999,"15min":2.0640000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.226,"5min":1.179,"15min":1.375},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.599,"5min":1.6990000000000001,"15min":7.6479999999999997},"last":1.3939999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.323,"5min":1.224,"15min":1.2649999999999999},"min":{"1min":0.89800000000000002,"5min":0.81399999999999995,"15min":0.71899999999999997},"max":{"1min":1.7130000000000001,"5min":1.74,"15min":2.1920000000000002},"last":1.597},{"interface":"front","average":{"1min":1.391,"5min":1.373,"15min":1.329},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.69699999999999995},"max":{"1min":1.6870000000000001,"5min":1.776,"15min":2.2120000000000002},"last":1.1080000000000001}]},{"osd":4,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.216,"15min":1.262},"min":{"1min":1.0840000000000001,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0430000000000001,"5min":2.0430000000000001,"15min":2.347},"last":1.502},{"interface":"front","average":{"1min":1.343,"5min":1.329,"15min":1.472},"min":{"1min":0.89500000000000002,"5min":0.78100000000000003,"15min":0.74399999999999999},"max":{"1min":1.6519999999999999,"5min":1.7989999999999999,"15min":3.153},"last":1.3}]},{"osd":6,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.3959999999999999,"15min":1.478},"min":{"1min":1.121,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.8720000000000001,"5min":1.923,"15min":2.8650000000000002},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.393,"15min":1.4610000000000001},"min":{"1min":1.012,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.742,"5min":1.879,"15min":2.6190000000000002},"last":1.768}]},{"osd":7,"last update":"Wed Nov 8 19:33:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.494,"15min":1.6220000000000001},"min":{"1min":1.016,"5min":0.94399999999999995,"15min":0.85799999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":3.2730000000000001},"last":1.544},{"interface":"front","average":{"1min":1.53,"5min":1.516,"15min":1.5640000000000001},"min":{"1min":0.97899999999999998,"5min":0.96799999999999997,"15min":0.95099999999999996},"max":{"1min":1.7849999999999999,"5min":1.9219999999999999,"15min":2.4670000000000001},"last":1.6279999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.1779999999999999,"15min":1.0920000000000001},"min":{"1min":0.73999999999999999,"5min":0.66900000000000004,"15min":0.58799999999999997},"max":{"1min":1.6040000000000001,"5min":1.782,"15min":2.5219999999999998},"last":1.038},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2270000000000001,"15min":1.2230000000000001},"min":{"1min":0.92200000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":7.0119999999999996},"last":1.3580000000000001}]},{"osd":1,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0840000000000001,"15min":1.085},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63100000000000001},"max":{"1min":1.579,"5min":1.7729999999999999,"15min":2.9790000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.171,"15min":1.1559999999999999},"min":{"1min":0.91200000000000003,"5min":0.67200000000000004,"15min":0.499},"max":{"1min":1.5660000000000001,"5min":1.7310000000000001,"15min":2.9940000000000002},"last":0.90100000000000002}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.262,"15min":1.3120000000000001},"min":{"1min":0.752,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5129999999999999,"5min":1.988,"15min":4.0410000000000004},"last":1.577},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3340000000000001,"15min":1.3839999999999999},"min":{"1min":0.77600000000000002,"5min":0.70199999999999996,"15min":0.53800000000000003},"max":{"1min":1.79,"5min":1.867,"15min":3.2069999999999999},"last":1.153}]},{"osd":3,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.198,"15min":1.258},"min":{"1min":0.95099999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0840000000000001,"5min":2.0840000000000001,"15min":4.4980000000000002},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.246,"15min":1.3360000000000001},"min":{"1min":0.94199999999999995,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.714,"5min":1.738,"15min":3.5710000000000002},"last":1.5289999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3089999999999999,"5min":1.2609999999999999,"15min":1.417},"min":{"1min":0.81399999999999995,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":4.2350000000000003},"last":1.413},{"interface":"front","average":{"1min":1.298,"5min":1.2490000000000001,"15min":1.4410000000000001},"min":{"1min":0.84899999999999998,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":2.194,"5min":2.194,"15min":4.5880000000000001},"last":1.496}]},{"osd":6,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.335,"15min":1.48},"min":{"1min":0.84999999999999998,"5min":0.745,"15min":0.745},"max":{"1min":1.6080000000000001,"5min":3.0139999999999998,"15min":4.335},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.4370000000000001,"15min":1.556},"min":{"1min":1.149,"5min":0.83299999999999996,"15min":0.83299999999999996},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":4.4130000000000003},"last":1.319}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.421,"15min":1.556},"min":{"1min":1.0629999999999999,"5min":0.876,"15min":0.56699999999999995},"max":{"1min":2.2829999999999999,"5min":2.2829999999999999,"15min":4.6369999999999996},"last":1.613},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.554,"15min":1.665},"min":{"1min":1.1259999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"max":{"1min":1.9370000000000001,"5min":3.1549999999999998,"15min":4.4950000000000001},"last":1.6419999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574955,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.84099999999999997,"15min":0.98099999999999998},"min":{"1min":0.67000000000000004,"5min":0.34100000000000003,"15min":0.312},"max":{"1min":1.9099999999999999,"5min":2.2320000000000002,"15min":3.972},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.94599999999999995,"15min":1.0349999999999999},"min":{"1min":0.39800000000000002,"5min":0.38300000000000001,"15min":0.311},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":3.1520000000000001},"last":0.745}]},{"osd":1,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":0.995,"15min":1.1080000000000001},"min":{"1min":0.64600000000000002,"5min":0.44800000000000001,"15min":0.39500000000000002},"max":{"1min":1.79,"5min":1.8480000000000001,"15min":4.3700000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.90100000000000002,"15min":1.0289999999999999},"min":{"1min":0.43099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.744,"5min":1.744,"15min":4.5750000000000002},"last":1.1060000000000001}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.1679999999999999,"15min":1.264},"min":{"1min":0.66900000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.6299999999999999,"5min":1.6850000000000001,"15min":3.9079999999999999},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.113,"15min":1.206},"min":{"1min":0.60999999999999999,"5min":0.50800000000000001,"15min":0.42199999999999999},"max":{"1min":1.5569999999999999,"5min":1.7190000000000001,"15min":2.73},"last":1.157}]},{"osd":4,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.234,"15min":1.365},"min":{"1min":0.80800000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.6439999999999999,"5min":1.752,"15min":4.2770000000000001},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.264,"15min":1.379},"min":{"1min":0.97899999999999998,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.7290000000000001,"5min":2.0449999999999999,"15min":3.052},"last":1.5529999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.323,"15min":1.429},"min":{"1min":0.77200000000000002,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.9259999999999999,"5min":1.972,"15min":3.0899999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.286,"15min":1.4390000000000001},"min":{"1min":1.012,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.5980000000000001,"5min":1.9950000000000001,"15min":4.2380000000000004},"last":1.2150000000000001}]},{"osd":6,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2290000000000001,"15min":1.3680000000000001},"min":{"1min":0.78400000000000003,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.728,"5min":1.841,"15min":4.2220000000000004},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.393,"15min":1.5109999999999999},"min":{"1min":0.93799999999999994,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.889,"5min":1.9490000000000001,"15min":3.1230000000000002},"last":1.4650000000000001}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4179999999999999,"15min":1.597},"min":{"1min":0.91300000000000003,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.157,"5min":2.157,"15min":4.407},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.3720000000000001,"5min":1.379,"15min":1.5049999999999999},"min":{"1min":0.83299999999999996,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.792,"5min":1.871,"15min":4.3849999999999998},"last":1.6220000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1316,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1347584,"data_stored":690149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84799999999999998},"min":{"1min":0.51400000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.274,"5min":1.3460000000000001,"15min":1.7809999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.91900000000000004},"min":{"1min":0.47999999999999998,"5min":0.40600000000000003,"15min":0.36899999999999999},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.79600000000000004}]},{"osd":1,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.91100000000000003,"15min":0.876},"min":{"1min":0.55800000000000005,"5min":0.40200000000000002,"15min":0.34899999999999998},"max":{"1min":1.2310000000000001,"5min":1.8759999999999999,"15min":1.8759999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.022,"5min":1.0269999999999999,"15min":1.012},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.4650000000000001,"5min":1.5209999999999999,"15min":1.5820000000000001},"last":1.141}]},{"osd":3,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.125,"15min":1.0700000000000001},"min":{"1min":0.67200000000000004,"5min":0.46600000000000003,"15min":0.44800000000000001},"max":{"1min":1.5029999999999999,"5min":1.821,"15min":1.821},"last":1.571},{"interface":"front","average":{"1min":1.101,"5min":1.079,"15min":1.099},"min":{"1min":0.64300000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":1.0189999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2969999999999999,"15min":1.355},"min":{"1min":0.96299999999999997,"5min":0.70499999999999996,"15min":0.67100000000000004},"max":{"1min":1.9670000000000001,"5min":1.9670000000000001,"15min":3.2959999999999998},"last":1.238},{"interface":"front","average":{"1min":1.327,"5min":1.3100000000000001,"15min":1.335},"min":{"1min":0.82099999999999995,"5min":0.77700000000000002,"15min":0.59299999999999997},"max":{"1min":1.748,"5min":1.748,"15min":2.278},"last":1.288}]},{"osd":5,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3340000000000001,"15min":1.3839999999999999},"min":{"1min":1.0209999999999999,"5min":0.751,"15min":0.60899999999999999},"max":{"1min":2.0840000000000001,"5min":3.0529999999999999,"15min":3.0529999999999999},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.345,"15min":1.407},"min":{"1min":1.0269999999999999,"5min":0.73899999999999999,"15min":0.64100000000000001},"max":{"1min":1.889,"5min":1.9450000000000001,"15min":2.536},"last":1.329}]},{"osd":6,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.387,"15min":1.381},"min":{"1min":0.94299999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.6910000000000001,"5min":2.1869999999999998,"15min":2.1869999999999998},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.367,"15min":1.417},"min":{"1min":1.1220000000000001,"5min":0.84699999999999998,"15min":0.70099999999999996},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.196}]},{"osd":7,"last update":"Wed Nov 8 19:33:04 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.4139999999999999,"15min":1.5089999999999999},"min":{"1min":0.98599999999999999,"5min":0.76700000000000002,"15min":0.70299999999999996},"max":{"1min":2.0230000000000001,"5min":2.0230000000000001,"15min":2.6549999999999998},"last":1.401},{"interface":"front","average":{"1min":1.528,"5min":1.4910000000000001,"15min":1.5229999999999999},"min":{"1min":1.0980000000000001,"5min":0.93500000000000005,"15min":0.76600000000000001},"max":{"1min":2.2200000000000002,"5min":2.226,"15min":2.3439999999999999},"last":1.516}]}]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32496,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339344,"statfs":{"total":96636764160,"available":96603488256,"internally_reserved":0,"allocated":1351680,"data_stored":691168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.88,"15min":0.79100000000000004},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.93500000000000005,"15min":0.81100000000000005},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.374},"max":{"1min":1.45,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.01}]},{"osd":2,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.872,"15min":0.81699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.046,"5min":1.052,"15min":0.94199999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46400000000000002},"max":{"1min":1.512,"5min":1.863,"15min":1.863},"last":1.081}]},{"osd":3,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.3049999999999999,"15min":1.1419999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.8440000000000001,"5min":1.978,"15min":1.978},"last":1.5640000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.131,"15min":1.099},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.583,"5min":1.778,"15min":2.5369999999999999},"last":1.296}]},{"osd":4,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.175,"15min":1.1599999999999999},"min":{"1min":0.79000000000000004,"5min":0.72099999999999997,"15min":0.65200000000000002},"max":{"1min":1.649,"5min":1.8620000000000001,"15min":2.165},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.3029999999999999,"15min":1.2649999999999999},"min":{"1min":0.80600000000000005,"5min":0.64900000000000002,"15min":0.64800000000000002},"max":{"1min":1.696,"5min":2.0830000000000002,"15min":2.0830000000000002},"last":1.335}]},{"osd":5,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.2949999999999999,"15min":1.264},"min":{"1min":0.79600000000000004,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0489999999999999,"5min":2.1930000000000001,"15min":2.4569999999999999},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.284,"15min":1.3009999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.629},"max":{"1min":1.7070000000000001,"5min":1.748,"15min":2.4089999999999998},"last":1.5049999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.391,"15min":1.373},"min":{"1min":0.875,"5min":0.82099999999999995,"15min":0.67200000000000004},"max":{"1min":1.9159999999999999,"5min":2.097,"15min":2.6019999999999999},"last":1.379},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3740000000000001,"15min":1.373},"min":{"1min":0.94999999999999996,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.748,"5min":1.8240000000000001,"15min":2.282},"last":1.24}]},{"osd":7,"last update":"Wed Nov 8 19:33:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4359999999999999,"15min":1.3640000000000001},"min":{"1min":0.85899999999999999,"5min":0.74199999999999999,"15min":0.68999999999999995},"max":{"1min":2.3860000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":1.6359999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.409,"15min":1.393},"min":{"1min":0.72999999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.9950000000000001,"5min":1.9950000000000001,"15min":2.6699999999999999},"last":1.6020000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574956,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33072,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338768,"statfs":{"total":96636764160,"available":96602898432,"internally_reserved":0,"allocated":1941504,"data_stored":1280517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.83599999999999997,"15min":0.79200000000000004},"min":{"1min":0.42399999999999999,"5min":0.38500000000000001,"15min":0.35099999999999998},"max":{"1min":1.577,"5min":1.577,"15min":1.6559999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.026,"5min":0.97299999999999998,"15min":0.93899999999999995},"min":{"1min":0.70999999999999996,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.7},"last":1.0129999999999999}]},{"osd":2,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.90600000000000003,"15min":0.876},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.115,"5min":1.0469999999999999,"15min":0.99099999999999999},"min":{"1min":0.498,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.5309999999999999,"5min":1.54,"15min":1.7270000000000001},"last":1.3149999999999999}]},{"osd":3,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.091,"5min":1.0620000000000001,"15min":1.0209999999999999},"min":{"1min":0.54000000000000004,"5min":0.38,"15min":0.38},"max":{"1min":1.444,"5min":1.6599999999999999,"15min":1.748},"last":1.1100000000000001},{"interface":"front","average":{"1min":1.083,"5min":1.0720000000000001,"15min":1.091},"min":{"1min":0.54100000000000004,"5min":0.48499999999999999,"15min":0.46200000000000002},"max":{"1min":1.4410000000000001,"5min":1.8400000000000001,"15min":2.496},"last":1.0569999999999999}]},{"osd":4,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.282,"15min":1.252},"min":{"1min":0.79200000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.663,"5min":1.774,"15min":1.9239999999999999},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3400000000000001,"15min":1.391},"min":{"1min":0.85899999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.4039999999999999},"last":1.1739999999999999}]},{"osd":5,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.3420000000000001,"15min":1.3520000000000001},"min":{"1min":0.84599999999999997,"5min":0.68899999999999995,"15min":0.61099999999999999},"max":{"1min":1.9419999999999999,"5min":1.9419999999999999,"15min":2.4369999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.3109999999999999,"15min":1.389},"min":{"1min":0.879,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.9079999999999999,"5min":2.089,"15min":2.4239999999999999},"last":1.1459999999999999}]},{"osd":6,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3700000000000001,"15min":1.444},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":2.0990000000000002,"5min":2.1619999999999999,"15min":2.5710000000000002},"last":1.23},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.3939999999999999,"15min":1.3979999999999999},"min":{"1min":0.93600000000000005,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.0870000000000002,"5min":2.4249999999999998,"15min":2.4249999999999998},"last":1.452}]},{"osd":7,"last update":"Wed Nov 8 19:32:57 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.4399999999999999,"15min":1.411},"min":{"1min":1.002,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.1629999999999998,"5min":2.3239999999999998,"15min":2.3239999999999998},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.504,"5min":1.4430000000000001,"15min":1.462},"min":{"1min":0.90800000000000003,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.839,"5min":1.881,"15min":2.306},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T19:33:31.184 DEBUG:teuthology.orchestra.run.smithi119:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-08T19:33:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.424+0000 7f66a5f1d640 1 Processor -- start 2023-11-08T19:33:31.427 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.424+0000 7f66a5f1d640 1 -- start start 2023-11-08T19:33:31.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f66a5f1d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f66a0144750 0x7f66a015bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.432 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f66a5f1d640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66a00cd000 0x7f66a006a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f66a5f1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a006aab0 con 0x7f66a006a660 2023-11-08T19:33:31.433 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f66a5f1d640 1 -- --> v2:172.21.15.119:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a00ce380 con 0x7f66a00cd000 2023-11-08T19:33:31.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f66a5f1d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a00cd770 con 0x7f66a0144750 2023-11-08T19:33:31.434 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f6697fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66a00cd000 0x7f66a006a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f6697fff640 1 --2- >> v2:172.21.15.119:3300/0 conn(0x7f66a00cd000 0x7f66a006a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36888/0 (socket says 172.21.15.119:36888) 2023-11-08T19:33:31.435 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f6697fff640 1 -- 172.21.15.119:0/3777421945 learned_addr learned my addr 172.21.15.119:0/3777421945 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:33:31.436 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f6697fff640 1 -- 172.21.15.119:0/3777421945 >> v2:172.21.15.188:3300/0 conn(0x7f66a0144750 msgr2=0x7f66a015bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:31.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f669f7fe640 1 --2- 172.21.15.119:0/3777421945 >> v2:172.21.15.188:3300/0 conn(0x7f66a0144750 0x7f66a015bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.437 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.426+0000 7f6697fff640 1 --2- 172.21.15.119:0/3777421945 >> v2:172.21.15.188:3300/0 conn(0x7f66a0144750 0x7f66a015bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.427+0000 7f6697fff640 1 -- 172.21.15.119:0/3777421945 >> v1:172.21.15.188:6789/0 conn(0x7f66a006a660 legacy=0x7f66a015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:31.438 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.427+0000 7f6697fff640 1 -- 172.21.15.119:0/3777421945 --> v2:172.21.15.119:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a00cda50 con 0x7f66a00cd000 2023-11-08T19:33:31.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.428+0000 7f6697fff640 1 --2- 172.21.15.119:0/3777421945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a00cd000 0x7f66a006a120 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f66a006dd50 tx=0x7f668802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=909cd66b8f115d74 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:31.439 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.428+0000 7f669effd640 1 -- 172.21.15.119:0/3777421945 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688031870 con 0x7f66a00cd000 2023-11-08T19:33:31.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.428+0000 7f669effd640 1 -- 172.21.15.119:0/3777421945 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66880319d0 con 0x7f66a00cd000 2023-11-08T19:33:31.440 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.428+0000 7f669effd640 1 -- 172.21.15.119:0/3777421945 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688031d60 con 0x7f66a00cd000 2023-11-08T19:33:31.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 -- 172.21.15.119:0/3777421945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a00cd000 msgr2=0x7f66a006a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.441 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/3777421945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a00cd000 0x7f66a006a120 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f66a006dd50 tx=0x7f668802b910 comp rx=0 tx=0).stop 2023-11-08T19:33:31.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 -- 172.21.15.119:0/3777421945 shutdown_connections 2023-11-08T19:33:31.442 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/3777421945 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a00cd000 0x7f66a006a120 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/3777421945 >> v2:172.21.15.188:3300/0 conn(0x7f66a0144750 0x7f66a015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.443 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 -- 172.21.15.119:0/3777421945 >> 172.21.15.119:0/3777421945 conn(0x7f66a00b08e0 msgr2=0x7f66a00b0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:31.444 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 -- 172.21.15.119:0/3777421945 shutdown_connections 2023-11-08T19:33:31.445 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.429+0000 7f66a5f1d640 1 -- 172.21.15.119:0/3777421945 wait complete. 2023-11-08T19:33:31.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.430+0000 7f66a5f1d640 1 Processor -- start 2023-11-08T19:33:31.450 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.431+0000 7f66a5f1d640 1 -- start start 2023-11-08T19:33:31.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.431+0000 7f66a5f1d640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 0x7f66a01365e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.451 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f66a5f1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66a00cd000 0x7f66a013e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f66a5f1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0144750 0x7f66a01e54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.452 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f66a5f1d640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a013f5e0 con 0x7f66a006a660 2023-11-08T19:33:31.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f66a5f1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a013fa00 con 0x7f66a0144750 2023-11-08T19:33:31.453 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f66a5f1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a0136b20 con 0x7f66a00cd000 2023-11-08T19:33:31.454 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 0x7f66a01365e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 0x7f66a01365e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36890/0 (socket says 172.21.15.119:36890) 2023-11-08T19:33:31.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 -- 172.21.15.119:0/611350266 learned_addr learned my addr 172.21.15.119:0/611350266 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:33:31.455 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 -- 172.21.15.119:0/611350266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66a00cd000 msgr2=0x7f66a013e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66a00cd000 0x7f66a013e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.456 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 -- 172.21.15.119:0/611350266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0144750 msgr2=0x7f66a01e54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0144750 0x7f66a01e54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.432+0000 7f669f7fe640 1 -- 172.21.15.119:0/611350266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880094d0 con 0x7f66a006a660 2023-11-08T19:33:31.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.433+0000 7f669f7fe640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 0x7f66a01365e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f669000c7f0 tx=0x7f669000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:31.457 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.433+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690004220 con 0x7f66a006a660 2023-11-08T19:33:31.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.433+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6690004380 con 0x7f66a006a660 2023-11-08T19:33:31.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.433+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690007840 con 0x7f66a006a660 2023-11-08T19:33:31.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.433+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a013e960 con 0x7f66a006a660 2023-11-08T19:33:31.458 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.433+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a013ef20 con 0x7f66a006a660 2023-11-08T19:33:31.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.434+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a013f720 con 0x7f66a006a660 2023-11-08T19:33:31.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.435+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6690007bb0 con 0x7f66a006a660 2023-11-08T19:33:31.459 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.436+0000 7f669cff9640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f668006ae70 0x7f668006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.436+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f6690089a20 con 0x7f66a006a660 2023-11-08T19:33:31.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.436+0000 7f6697fff640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f668006ae70 0x7f668006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.437+0000 7f6697fff640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f668006ae70 0x7f668006d330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6688036000 tx=0x7f6688030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:31.460 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.440+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66a013f720 con 0x7f66a006a660 2023-11-08T19:33:31.633 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.631+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f66a013f720 con 0x7f66a006a660 2023-11-08T19:33:31.640 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.638+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v21)=0 v21) v1 ==== 123+0+0 (secure 0 0 0) 0x7f66a013f720 con 0x7f66a006a660 2023-11-08T19:33:31.643 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.641+0000 7f669cff9640 1 -- 172.21.15.119:0/611350266 <== mon.0 v2:172.21.15.119:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f66900044f0 con 0x7f66a006a660 2023-11-08T19:33:31.646 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f668006ae70 msgr2=0x7f668006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f668006ae70 0x7f668006d330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f6688036000 tx=0x7f6688030000 comp rx=0 tx=0).stop 2023-11-08T19:33:31.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 msgr2=0x7f66a01365e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.647 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 0x7f66a01365e0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f669000c7f0 tx=0x7f669000ccb0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 shutdown_connections 2023-11-08T19:33:31.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f668006ae70 0x7f668006d330 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.648 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f66a0144750 0x7f66a01e54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f66a00cd000 0x7f66a013e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 --2- 172.21.15.119:0/611350266 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f66a006a660 0x7f66a01365e0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.649 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 >> 172.21.15.119:0/611350266 conn(0x7f66a00b08e0 msgr2=0x7f66a006d060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:31.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 shutdown_connections 2023-11-08T19:33:31.650 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.643+0000 7f66a5f1d640 1 -- 172.21.15.119:0/611350266 wait complete. 2023-11-08T19:33:31.663 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T19:33:31.664 DEBUG:teuthology.orchestra.run.smithi119:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T19:33:31.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.878+0000 7f968b2a9640 1 Processor -- start 2023-11-08T19:33:31.880 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.878+0000 7f968b2a9640 1 -- start start 2023-11-08T19:33:31.881 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.879+0000 7f968b2a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 0x7f96840cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.879+0000 7f968b2a9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f968406cb40 0x7f968406cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.879+0000 7f968b2a9640 1 -- --> v1:172.21.15.119:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96840c4960 con 0x7f9684142f80 2023-11-08T19:33:31.882 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.879+0000 7f968b2a9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9684143a40 con 0x7f968406cb40 2023-11-08T19:33:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.879+0000 7f968b2a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840ca3e0 con 0x7f968415d9f0 2023-11-08T19:33:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.879+0000 7f9689045640 1 --1- >> v1:172.21.15.119:6789/0 conn(0x7f9684142f80 0x7f96840c9cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.119:6789/0 says I am v1:172.21.15.119:60134/0 (socket says 172.21.15.119:60134) 2023-11-08T19:33:31.883 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9689045640 1 -- 172.21.15.119:0/602309651 learned_addr learned my addr 172.21.15.119:0/602309651 (peer_addr_for_me v1:172.21.15.119:0/0) 2023-11-08T19:33:31.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 <== mon.0 v1:172.21.15.119:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069265325 0 0) 0x7f96840c4960 con 0x7f9684142f80 2023-11-08T19:33:31.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 --> v1:172.21.15.119:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9670003520 con 0x7f9684142f80 2023-11-08T19:33:31.886 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9689846640 1 --2- 172.21.15.119:0/602309651 >> v2:172.21.15.188:3300/0 conn(0x7f968406cb40 0x7f968406cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 <== mon.0 v1:172.21.15.119:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 878261779 0 0) 0x7f9670003520 con 0x7f9684142f80 2023-11-08T19:33:31.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 msgr2=0x7f96840cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:31.887 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 0x7f96840cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 >> v2:172.21.15.188:3300/0 conn(0x7f968406cb40 msgr2=0x7f968406cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:31.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 --2- 172.21.15.119:0/602309651 >> v2:172.21.15.188:3300/0 conn(0x7f968406cb40 0x7f968406cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 --> v1:172.21.15.119:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968415df60 con 0x7f9684142f80 2023-11-08T19:33:31.888 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 <== mon.0 v1:172.21.15.119:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 985027024 0 0) 0x7f9680001670 con 0x7f9684142f80 2023-11-08T19:33:31.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 >> v1:172.21.15.119:6789/0 conn(0x7f9684142f80 legacy=0x7f96840c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9670003970 0x7f9670003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 0x7f96700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.889 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f9670005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.880+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684143a40 con 0x7f9670004d00 2023-11-08T19:33:31.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840ca3e0 con 0x7f9670003970 2023-11-08T19:33:31.890 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f967000bc10 con 0x7f968415d9f0 2023-11-08T19:33:31.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689846640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f9670005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.891 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9688844640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 0x7f96700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689045640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9670003970 0x7f9670003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689846640 1 -- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 msgr2=0x7f96700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:31.893 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689846640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f968415d9f0 0x7f96700047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689846640 1 -- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9670003970 msgr2=0x7f9670003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T19:33:31.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689846640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9670003970 0x7f9670003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.894 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.881+0000 7f9689846640 1 -- 172.21.15.119:0/602309651 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9670003520 con 0x7f9670004d00 2023-11-08T19:33:31.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.882+0000 7f9689846640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f9670005180 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f967c009ba0 tx=0x7f96700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46443508b6dbfca1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:31.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.882+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c036440 con 0x7f9670004d00 2023-11-08T19:33:31.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.882+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f967c0365a0 con 0x7f9670004d00 2023-11-08T19:33:31.895 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.882+0000 7f9677fff640 1 -- 172.21.15.119:0/602309651 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c036950 con 0x7f9670004d00 2023-11-08T19:33:31.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f968b2a9640 1 -- 172.21.15.119:0/602309651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 msgr2=0x7f9670005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f968b2a9640 1 --2- 172.21.15.119:0/602309651 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f9670005180 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f967c009ba0 tx=0x7f96700057f0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f9689846640 1 -- 172.21.15.119:0/602309651 reap_dead start 2023-11-08T19:33:31.896 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f968b2a9640 1 -- 172.21.15.119:0/602309651 shutdown_connections 2023-11-08T19:33:31.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f968b2a9640 1 -- 172.21.15.119:0/602309651 >> 172.21.15.119:0/602309651 conn(0x7f96840cb610 msgr2=0x7f96840cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:31.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f968b2a9640 1 -- 172.21.15.119:0/602309651 shutdown_connections 2023-11-08T19:33:31.897 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.883+0000 7f968b2a9640 1 -- 172.21.15.119:0/602309651 wait complete. 2023-11-08T19:33:31.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.885+0000 7f968b2a9640 1 Processor -- start 2023-11-08T19:33:31.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.885+0000 7f968b2a9640 1 -- start start 2023-11-08T19:33:31.900 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f968b2a9640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f96841f20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f968b2a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96840c5bd0 0x7f96840c6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f968b2a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96840c7a40 0x7f96840c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f968b2a9640 1 -- --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96841f2620 con 0x7f9670004d00 2023-11-08T19:33:31.901 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f968b2a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c8400 con 0x7f96840c7a40 2023-11-08T19:33:31.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f968b2a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96840c6670 con 0x7f96840c5bd0 2023-11-08T19:33:31.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f9689045640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f96841f20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f9689045640 1 --2- >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f96841f20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.119:3300/0 says I am v2:172.21.15.119:36910/0 (socket says 172.21.15.119:36910) 2023-11-08T19:33:31.902 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.886+0000 7f9689045640 1 -- 172.21.15.119:0/224594842 learned_addr learned my addr 172.21.15.119:0/224594842 (peer_addr_for_me v2:172.21.15.119:0/0) 2023-11-08T19:33:31.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9689045640 1 -- 172.21.15.119:0/224594842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96840c5bd0 msgr2=0x7f96840c6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9689045640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96840c5bd0 0x7f96840c6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.903 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9689045640 1 -- 172.21.15.119:0/224594842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96840c7a40 msgr2=0x7f96840c7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:31.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9689045640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96840c7a40 0x7f96840c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:31.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9689045640 1 -- 172.21.15.119:0/224594842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840c6950 con 0x7f9670004d00 2023-11-08T19:33:31.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9689045640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f96841f20e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f968000d7c0 tx=0x7f968000dc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:31.904 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96800074e0 con 0x7f9670004d00 2023-11-08T19:33:31.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9680007640 con 0x7f9670004d00 2023-11-08T19:33:31.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9680004a40 con 0x7f9670004d00 2023-11-08T19:33:31.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.887+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96840c6c30 con 0x7f9670004d00 2023-11-08T19:33:31.905 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.888+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96840c71f0 con 0x7f9670004d00 2023-11-08T19:33:31.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.889+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9684074430 con 0x7f9670004d00 2023-11-08T19:33:31.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.890+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9680004ba0 con 0x7f9670004d00 2023-11-08T19:33:31.906 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.890+0000 7f9675ffb640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f966806aea0 0x7f966806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T19:33:31.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.891+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6999+0+0 (secure 0 0 0) 0x7f9680089330 con 0x7f9670004d00 2023-11-08T19:33:31.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.891+0000 7f9688844640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f966806aea0 0x7f966806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T19:33:31.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.891+0000 7f9688844640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f966806aea0 0x7f966806d360 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f968415d310 tx=0x7f9678008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T19:33:31.907 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:31.895+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9680053e40 con 0x7f9670004d00 2023-11-08T19:33:32.122 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.120+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 --> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96840c4230 con 0x7f9670004d00 2023-11-08T19:33:32.124 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.121+0000 7f9675ffb640 1 -- 172.21.15.119:0/224594842 <== mon.0 v2:172.21.15.119:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 103 v103) v1 ==== 95+0+3714 (secure 0 0 0) 0x7f9680019090 con 0x7f9670004d00 2023-11-08T19:33:32.124 INFO:teuthology.orchestra.run.smithi119.stdout: 2023-11-08T19:33:32.124 INFO:teuthology.orchestra.run.smithi119.stdout:{"epoch":103,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5057,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6836/3170218997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6834","nonce":3170218997},{"type":"v1","addr":"172.21.15.119:6836","nonce":3170218997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49},{"gid":5333,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/4200281915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":4200281915},{"type":"v1","addr":"172.21.15.188:6833","nonce":4200281915}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":5842,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4070216955","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4070216955},{"type":"v1","addr":"172.21.15.188:6835","nonce":4070216955}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8112,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.119:6837/1184972475","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.119:6835","nonce":1184972475},{"type":"v1","addr":"172.21.15.119:6837","nonce":1184972475}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[]} 2023-11-08T19:33:32.125 INFO:teuthology.orchestra.run.smithi119.stderr:dumped fsmap epoch 103 2023-11-08T19:33:32.126 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f966806aea0 msgr2=0x7f966806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:32.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f966806aea0 0x7f966806d360 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f968415d310 tx=0x7f9678008040 comp rx=0 tx=0).stop 2023-11-08T19:33:32.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 msgr2=0x7f96841f20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T19:33:32.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f96841f20e0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f968000d7c0 tx=0x7f968000dc80 comp rx=0 tx=0).stop 2023-11-08T19:33:32.127 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 shutdown_connections 2023-11-08T19:33:32.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:6832/2270580327,v1:172.21.15.119:6833/2270580327] conn(0x7f966806aea0 0x7f966806d360 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:32.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f96840c7a40 0x7f96840c7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:32.128 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f96840c5bd0 0x7f96840c6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 --2- 172.21.15.119:0/224594842 >> [v2:172.21.15.119:3300/0,v1:172.21.15.119:6789/0] conn(0x7f9670004d00 0x7f96841f20e0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T19:33:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 >> 172.21.15.119:0/224594842 conn(0x7f96840cb610 msgr2=0x7f968406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T19:33:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 shutdown_connections 2023-11-08T19:33:32.129 INFO:teuthology.orchestra.run.smithi119.stderr:2023-11-08T19:33:32.124+0000 7f968b2a9640 1 -- 172.21.15.119:0/224594842 wait complete. 2023-11-08T19:33:32.145 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-08T19:33:32.146 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-08T19:33:32.146 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T19:33:32.147 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.166 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T19:33:32.167 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T19:33:32.167 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.225 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T19:33:32.226 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T19:33:32.226 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.259 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T19:33:32.260 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T19:33:32.260 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.319 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T19:33:32.319 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-08T19:33:32.320 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-08T19:33:32.320 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.369 INFO:tasks.ceph.osd.0:Stopped 2023-11-08T19:33:32.369 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-08T19:33:32.370 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.433 INFO:tasks.ceph.osd.1:Stopped 2023-11-08T19:33:32.434 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-08T19:33:32.434 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.507 INFO:tasks.ceph.osd.2:Stopped 2023-11-08T19:33:32.507 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-08T19:33:32.508 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.521 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-08T19:33:32.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-08T19:33:32.522 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-08T19:33:32.573 INFO:tasks.ceph.osd.3:Stopped 2023-11-08T19:33:32.573 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-08T19:33:32.574 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.624 INFO:tasks.ceph.osd.4:Stopped 2023-11-08T19:33:32.624 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-08T19:33:32.625 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.692 INFO:tasks.ceph.osd.5:Stopped 2023-11-08T19:33:32.692 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-08T19:33:32.693 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.755 INFO:tasks.ceph.osd.6:Stopped 2023-11-08T19:33:32.755 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-08T19:33:32.756 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:32.834 INFO:tasks.ceph.osd.7:Stopped 2023-11-08T19:33:32.835 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-08T19:33:32.835 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-08T19:33:32.835 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:37.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y is failed for ~0s 2023-11-08T19:33:37.926 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-08T19:33:37.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-08T19:33:37.927 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-08T19:33:38.938 INFO:tasks.ceph.mgr.y:Stopped 2023-11-08T19:33:38.939 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-08T19:33:38.939 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:39.031 INFO:tasks.ceph.mgr.x:Stopped 2023-11-08T19:33:39.031 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-08T19:33:39.032 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-08T19:33:39.032 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:39.081 INFO:tasks.ceph.mon.a:Stopped 2023-11-08T19:33:39.082 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-08T19:33:39.082 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:39.116 INFO:tasks.ceph.mon.b:Stopped 2023-11-08T19:33:39.116 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-08T19:33:39.117 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T19:33:39.192 INFO:tasks.ceph.mon.c:Stopped 2023-11-08T19:33:39.192 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-08T19:33:39.193 DEBUG:teuthology.orchestra.run.smithi119:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'evicting unresponsive client' | egrep -v RECENT_CRASH | head -n 1 2023-11-08T19:33:39.300 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:33:39.301 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-08T19:33:39.509 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:33:39.510 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-08T19:33:39.673 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:33:39.673 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-08T19:33:39.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:33:39.821 DEBUG:teuthology.orchestra.run.smithi119:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-08T19:33:39.999 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:33:39.999 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-08T19:33:40.255 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:33:40.255 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-08T19:33:40.446 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:33:40.446 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-08T19:33:40.602 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:33:40.603 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-08T19:33:40.780 INFO:tasks.ceph:Archiving mon data... 2023-11-08T19:33:40.781 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/data/mon.a.tgz 2023-11-08T19:33:40.782 DEBUG:teuthology.orchestra.run.smithi119:> mktemp 2023-11-08T19:33:40.841 INFO:teuthology.orchestra.run.smithi119.stdout:/tmp/tmp.Rfq1JSDi9X 2023-11-08T19:33:40.842 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Rfq1JSDi9X 2023-11-08T19:33:41.038 DEBUG:teuthology.orchestra.run.smithi119:> sudo chmod 0666 /tmp/tmp.Rfq1JSDi9X 2023-11-08T19:33:41.190 DEBUG:teuthology.orchestra.remote:smithi119:/tmp/tmp.Rfq1JSDi9X is 371KB 2023-11-08T19:33:41.301 DEBUG:teuthology.orchestra.run.smithi119:> rm -fr /tmp/tmp.Rfq1JSDi9X 2023-11-08T19:33:41.360 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/data/mon.b.tgz 2023-11-08T19:33:41.360 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2023-11-08T19:33:41.420 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.Z1J8SdsH04 2023-11-08T19:33:41.455 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Z1J8SdsH04 2023-11-08T19:33:41.662 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.Z1J8SdsH04 2023-11-08T19:33:41.816 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.Z1J8SdsH04 is 373KB 2023-11-08T19:33:42.097 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.Z1J8SdsH04 2023-11-08T19:33:42.157 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/data/mon.c.tgz 2023-11-08T19:33:42.157 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2023-11-08T19:33:42.211 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.BNbKVZvuNb 2023-11-08T19:33:42.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.BNbKVZvuNb 2023-11-08T19:33:42.384 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.BNbKVZvuNb 2023-11-08T19:33:42.535 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.BNbKVZvuNb is 373KB 2023-11-08T19:33:42.645 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.BNbKVZvuNb 2023-11-08T19:33:42.704 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-08T19:33:42.705 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T19:33:42.714 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T19:33:42.813 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-08T19:33:42.813 DEBUG:teuthology.orchestra.run.smithi119:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T19:33:42.822 DEBUG:teuthology.orchestra.run.smithi188:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T19:33:42.924 INFO:teuthology.orchestra.run.smithi119.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T19:33:42.924 INFO:teuthology.orchestra.run.smithi188.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T19:33:42.928 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mgr.y has been restored 2023-11-08T19:33:42.937 INFO:tasks.ceph:Archiving crash dumps... 2023-11-08T19:33:42.942 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/remote/smithi119/crash 2023-11-08T19:33:42.944 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T19:33:43.048 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/remote/smithi188/crash 2023-11-08T19:33:43.049 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T19:33:43.201 INFO:tasks.ceph:Shutting down logrotate 2023-11-08T19:33:43.202 DEBUG:teuthology.orchestra.run.smithi119:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:43.211 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:43.356 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:43.457 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T19:33:43.561 INFO:tasks.ceph:Compressing logs... 2023-11-08T19:33:43.562 DEBUG:teuthology.orchestra.run.smithi119:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-08T19:33:43.571 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-08T19:33:43.660 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108976.log 2023-11-08T19:33:43.661 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100148.log 2023-11-08T19:33:43.662 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99146.log 2023-11-08T19:33:43.662 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.108976.log.gz 2023-11-08T19:33:43.663 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121014.log 2023-11-08T19:33:43.664 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100148.log.gz 2023-11-08T19:33:43.664 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-08T19:33:43.665 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99146.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99146.log.gz 2023-11-08T19:33:43.666 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120720.log 2023-11-08T19:33:43.666 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107923.log 2023-11-08T19:33:43.666 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121014.log.gz 2023-11-08T19:33:43.668 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109060.log 2023-11-08T19:33:43.669 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120720.log.gz 2023-11-08T19:33:43.669 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107923.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.107923.log.gz 2023-11-08T19:33:43.669 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114863.log 2023-11-08T19:33:43.670 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87527.log 2023-11-08T19:33:43.670 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87687.log 2023-11-08T19:33:43.670 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93286.log 2023-11-08T19:33:43.670 INFO:teuthology.orchestra.run.smithi188.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87527.log.gz 2023-11-08T19:33:43.674 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87567.log 2023-11-08T19:33:43.675 INFO:teuthology.orchestra.run.smithi188.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.87687.log.gz 2023-11-08T19:33:43.675 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.93286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.93286.log.gz 2023-11-08T19:33:43.675 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-08T19:33:43.675 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87567.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.87567.log.gz 2023-11-08T19:33:43.676 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-08T19:33:43.676 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87607.log 2023-11-08T19:33:43.676 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-08T19:33:43.678 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120050.log 2023-11-08T19:33:43.678 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109060.log.gz 2023-11-08T19:33:43.679 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118159.log 2023-11-08T19:33:43.679 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.114863.log.gz 2023-11-08T19:33:43.679 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120050.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120050.log.gz 2023-11-08T19:33:43.680 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117467.log 2023-11-08T19:33:43.680 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117019.log 2023-11-08T19:33:43.680 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.118159.log.gz 2023-11-08T19:33:43.680 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117467.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-08T19:33:43.681 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117467.log.gz 2023-11-08T19:33:43.681 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117019.log.gz 2023-11-08T19:33:43.681 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105325.log 2023-11-08T19:33:43.682 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100652.log 2023-11-08T19:33:43.682 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105409.log 2023-11-08T19:33:43.682 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105325.log.gz 2023-11-08T19:33:43.683 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100652.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100652.log.gz 2023-11-08T19:33:43.684 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97913.log 2023-11-08T19:33:43.688 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87607.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87607.log.gz 2023-11-08T19:33:43.689 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87727.log 2023-11-08T19:33:43.689 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-08T19:33:43.689 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-08T19:33:43.689 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87727.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-08T19:33:43.690 INFO:teuthology.orchestra.run.smithi188.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87727.log.gz 2023-11-08T19:33:43.690 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93679.log 2023-11-08T19:33:43.690 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-08T19:33:43.691 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.3.93679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.93679.log.gz 2023-11-08T19:33:43.691 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87807.log 2023-11-08T19:33:43.691 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-08T19:33:43.691 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87807.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87807.log.gz 2023-11-08T19:33:43.693 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-08T19:33:43.697 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100442.log 2023-11-08T19:33:43.698 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105409.log.gz 2023-11-08T19:33:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97913.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97913.log.gz 2023-11-08T19:33:43.698 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93258.log 2023-11-08T19:33:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109743.log 2023-11-08T19:33:43.699 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100442.log.gz 2023-11-08T19:33:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93258.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93258.log.gz 2023-11-08T19:33:43.699 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93088.log 2023-11-08T19:33:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106973.log 2023-11-08T19:33:43.700 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109743.log.gz 2023-11-08T19:33:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93088.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93088.log.gz 2023-11-08T19:33:43.700 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102680.log 2023-11-08T19:33:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105367.log 2023-11-08T19:33:43.701 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106973.log.gz 2023-11-08T19:33:43.701 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102680.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102680.log.gz 2023-11-08T19:33:43.702 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116017.log 2023-11-08T19:33:43.702 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112158.log 2023-11-08T19:33:43.702 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105367.log.gz 2023-11-08T19:33:43.703 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116017.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116017.log.gz 2023-11-08T19:33:43.703 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99020.log 2023-11-08T19:33:43.703 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94255.log 2023-11-08T19:33:43.704 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112158.log.gz 2023-11-08T19:33:43.704 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99020.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99020.log.gz 2023-11-08T19:33:43.704 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101592.log 2023-11-08T19:33:43.704 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116440.log 2023-11-08T19:33:43.705 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94255.log.gz 2023-11-08T19:33:43.705 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101592.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101592.log.gz 2023-11-08T19:33:43.705 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102816.log 2023-11-08T19:33:43.705 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116440.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116440.log.gz 2023-11-08T19:33:43.706 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109144.log 2023-11-08T19:33:43.709 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87647.log 2023-11-08T19:33:43.710 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-08T19:33:43.710 INFO:teuthology.orchestra.run.smithi188.stderr: 94.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-08T19:33:43.710 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.d.log: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-08T19:33:43.711 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87647.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87647.log.gz 2023-11-08T19:33:43.711 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87767.log 2023-11-08T19:33:43.711 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-08T19:33:43.711 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.87767.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.87767.log.gz 2023-11-08T19:33:43.714 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120888.log 2023-11-08T19:33:43.715 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102816.log.gz 2023-11-08T19:33:43.715 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109144.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109144.log.gz 2023-11-08T19:33:43.715 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107229.log 2023-11-08T19:33:43.715 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106681.log 2023-11-08T19:33:43.716 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120888.log.gz 2023-11-08T19:33:43.716 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107229.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107229.log.gz 2023-11-08T19:33:43.716 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81586.log 2023-11-08T19:33:43.717 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117081.log 2023-11-08T19:33:43.717 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.106681.log.gz 2023-11-08T19:33:43.717 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.tmp-client.admin.81586.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81586.log.gz 2023-11-08T19:33:43.717 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99896.log 2023-11-08T19:33:43.718 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117081.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117081.log.gz 2023-11-08T19:33:43.718 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111204.log 2023-11-08T19:33:43.718 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101084.log 2023-11-08T19:33:43.719 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99896.log.gz 2023-11-08T19:33:43.719 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111204.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111204.log.gz 2023-11-08T19:33:43.719 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116140.log 2023-11-08T19:33:43.719 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98825.log 2023-11-08T19:33:43.720 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101084.log.gz 2023-11-08T19:33:43.720 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111864.log 2023-11-08T19:33:43.720 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116140.log.gz 2023-11-08T19:33:43.721 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98825.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98825.log.gz 2023-11-08T19:33:43.721 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120593.log 2023-11-08T19:33:43.721 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110337.log 2023-11-08T19:33:43.721 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111864.log.gz 2023-11-08T19:33:43.726 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120593.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120593.log.gz 2023-11-08T19:33:43.726 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96228.log 2023-11-08T19:33:43.726 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98363.log 2023-11-08T19:33:43.727 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110337.log.gz 2023-11-08T19:33:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96228.log.gz 2023-11-08T19:33:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93468.log 2023-11-08T19:33:43.727 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104533.log 2023-11-08T19:33:43.728 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98363.log.gz 2023-11-08T19:33:43.728 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101126.log 2023-11-08T19:33:43.729 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93468.log.gz 2023-11-08T19:33:43.729 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104533.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104533.log.gz 2023-11-08T19:33:43.729 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102722.log 2023-11-08T19:33:43.730 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114679.log 2023-11-08T19:33:43.730 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101126.log.gz 2023-11-08T19:33:43.730 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102722.log: gzip -5 --verbose -- 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102722.log.gz /var/log/ceph/ceph-client.admin.103506.log 2023-11-08T19:33:43.730 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:43.731 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119140.log 2023-11-08T19:33:43.734 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103506.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.114679.log.gz 2023-11-08T19:33:43.734 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103506.log.gzgzip 2023-11-08T19:33:43.734 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99434.log 2023-11-08T19:33:43.734 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116078.log 2023-11-08T19:33:43.735 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119140.log.gz 2023-11-08T19:33:43.735 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99392.log 2023-11-08T19:33:43.735 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99434.log.gz 2023-11-08T19:33:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115501.log 2023-11-08T19:33:43.736 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116078.log.gz 2023-11-08T19:33:43.736 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112344.log 2023-11-08T19:33:43.736 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99392.log.gz 2023-11-08T19:33:43.737 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113506.log 2023-11-08T19:33:43.737 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.115501.log.gz 2023-11-08T19:33:43.739 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107854.log 2023-11-08T19:33:43.739 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112344.log.gz 2023-11-08T19:33:43.740 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113506.log.gz 2023-11-08T19:33:43.740 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106804.log 2023-11-08T19:33:43.740 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109438.log 2023-11-08T19:33:43.740 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107854.log.gz 2023-11-08T19:33:43.741 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119444.log 2023-11-08T19:33:43.741 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106804.log.gz 2023-11-08T19:33:43.742 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117590.log 2023-11-08T19:33:43.743 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109438.log.gz 2023-11-08T19:33:43.743 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104449.log 2023-11-08T19:33:43.743 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119444.log.gz 2023-11-08T19:33:43.744 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117590.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.117590.log.gz 2023-11-08T19:33:43.744 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116519.log 2023-11-08T19:33:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104449.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104449.log.gz 2023-11-08T19:33:43.745 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118772.log 2023-11-08T19:33:43.746 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111644.log 2023-11-08T19:33:43.746 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116519.log.gz 2023-11-08T19:33:43.747 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118772.log: gzip 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118772.log.gz -5 2023-11-08T19:33:43.747 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94649.log 2023-11-08T19:33:43.748 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107271.log 2023-11-08T19:33:43.748 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111644.log.gz 2023-11-08T19:33:43.749 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99062.log 2023-11-08T19:33:43.749 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94649.log.gz 2023-11-08T19:33:43.750 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104743.log 2023-11-08T19:33:43.751 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107271.log.gz 2023-11-08T19:33:43.751 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102900.log 2023-11-08T19:33:43.752 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99062.log.gz 2023-11-08T19:33:43.752 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99938.log 2023-11-08T19:33:43.753 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.104743.log.gz 2023-11-08T19:33:43.754 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102900.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102900.log.gz /var/log/ceph/ceph-client.admin.121098.log 2023-11-08T19:33:43.754 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:43.758 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97220.log 2023-11-08T19:33:43.759 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99938.log.gz 2023-11-08T19:33:43.759 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106847.log 2023-11-08T19:33:43.759 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121098.log.gz 2023-11-08T19:33:43.760 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95343.log 2023-11-08T19:33:43.760 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97220.log.gz 2023-11-08T19:33:43.761 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112471.log 2023-11-08T19:33:43.761 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106847.log.gz 2023-11-08T19:33:43.762 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95343.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95343.log.gz 2023-11-08T19:33:43.762 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118920.log 2023-11-08T19:33:43.763 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109823.log 2023-11-08T19:33:43.763 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112471.log.gz 2023-11-08T19:33:43.764 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94213.log 2023-11-08T19:33:43.764 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118920.log.gz 2023-11-08T19:33:43.765 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116664.log 2023-11-08T19:33:43.765 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109823.log.gz 2023-11-08T19:33:43.766 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94213.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94213.log.gzgzip 2023-11-08T19:33:43.766 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96270.log 2023-11-08T19:33:43.767 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100694.log 2023-11-08T19:33:43.767 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116664.log.gz 2023-11-08T19:33:43.768 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.c.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-08T19:33:43.769 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94926.log 2023-11-08T19:33:43.770 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96270.log.gz 2023-11-08T19:33:43.770 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106254.log 2023-11-08T19:33:43.770 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100694.log.gz 2023-11-08T19:33:43.771 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120438.log 2023-11-08T19:33:43.771 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94926.log.gz 2023-11-08T19:33:43.772 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118586.log 2023-11-08T19:33:43.772 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.106254.log.gz 2023-11-08T19:33:43.773 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120438.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.120438.log.gz 2023-11-08T19:33:43.774 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105283.log 2023-11-08T19:33:43.774 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110157.log 2023-11-08T19:33:43.775 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.118586.log.gz 2023-11-08T19:33:43.776 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105283.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.105283.log.gz 2023-11-08T19:33:43.776 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112597.log 2023-11-08T19:33:43.777 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120678.log 2023-11-08T19:33:43.777 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110157.log.gz 2023-11-08T19:33:43.778 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112597.log: gzip 88.8% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.112597.log.gz -- 2023-11-08T19:33:43.778 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.114923.log 2023-11-08T19:33:43.779 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115212.log 2023-11-08T19:33:43.779 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120678.log.gz 2023-11-08T19:33:43.780 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99476.log 2023-11-08T19:33:43.781 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114923.log.gz 2023-11-08T19:33:43.781 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98615.log 2023-11-08T19:33:43.782 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115212.log.gz 2023-11-08T19:33:43.783 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114965.log 88.9% 2023-11-08T19:33:43.783 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99476.log.gz 2023-11-08T19:33:43.784 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115916.log 2023-11-08T19:33:43.784 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98615.log.gz 2023-11-08T19:33:43.785 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95353.log 2023-11-08T19:33:43.786 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114965.log.gz 2023-11-08T19:33:43.786 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108378.log 2023-11-08T19:33:43.787 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.115916.log.gz 2023-11-08T19:33:43.788 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107898.log 2023-11-08T19:33:43.788 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95353.log.gz 2023-11-08T19:33:43.789 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108378.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108378.log.gz 2023-11-08T19:33:43.790 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103254.log 2023-11-08T19:33:43.791 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107898.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107898.log.gz 2023-11-08T19:33:43.791 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102362.log 2023-11-08T19:33:43.792 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102284.log 2023-11-08T19:33:43.792 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103254.log.gz 2023-11-08T19:33:43.793 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94968.log 2023-11-08T19:33:43.793 INFO:teuthology.orchestra.run.smithi119.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.102362.log.gz 2023-11-08T19:33:43.794 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108420.log 2023-11-08T19:33:43.795 INFO:teuthology.orchestra.run.smithi119.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.102284.log.gz 2023-11-08T19:33:43.796 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100400.log 2023-11-08T19:33:43.796 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94968.log.gz 2023-11-08T19:33:43.797 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119696.log 2023-11-08T19:33:43.797 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108420.log.gz 2023-11-08T19:33:43.798 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101676.log 2023-11-08T19:33:43.798 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100400.log.gz 2023-11-08T19:33:43.799 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119696.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.119696.log.gz 2023-11-08T19:33:43.800 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95976.log 2023-11-08T19:33:43.801 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101676.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101676.log.gz 2023-11-08T19:33:43.801 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109228.log 2023-11-08T19:33:43.802 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112681.log 2023-11-08T19:33:43.803 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95976.log.gz 2023-11-08T19:33:43.804 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119822.log 2023-11-08T19:33:43.804 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109228.log.gz 2023-11-08T19:33:43.805 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98699.log 2023-11-08T19:33:43.805 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112681.log.gz 2023-11-08T19:33:43.806 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100754.log 2023-11-08T19:33:43.806 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119822.log.gz 2023-11-08T19:33:43.807 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105983.log 2023-11-08T19:33:43.807 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98699.log.gz 2023-11-08T19:33:43.808 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96396.log 2023-11-08T19:33:43.808 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100754.log.gz 2023-11-08T19:33:43.809 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115170.log 2023-11-08T19:33:43.809 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105983.log.gz 2023-11-08T19:33:43.810 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93044.log 2023-11-08T19:33:43.811 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96396.log.gz 2023-11-08T19:33:43.812 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115435.log 2023-11-08T19:33:43.812 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115170.log.gz 2023-11-08T19:33:43.813 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109564.log 2023-11-08T19:33:43.813 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93044.log.gz 2023-11-08T19:33:43.814 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118243.log 2023-11-08T19:33:43.814 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.115435.log.gz 2023-11-08T19:33:43.815 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109564.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109564.log.gz 2023-11-08T19:33:43.815 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.121511.log 2023-11-08T19:33:43.817 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117740.log 2023-11-08T19:33:43.817 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118243.log.gz 2023-11-08T19:33:43.818 INFO:teuthology.orchestra.run.smithi188.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-08T19:33:43.818 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121511.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121511.log.gz 2023-11-08T19:33:43.819 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114779.log 2023-11-08T19:33:43.819 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103068.log 2023-11-08T19:33:43.820 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.117740.log.gz 2023-11-08T19:33:43.821 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118544.log 2023-11-08T19:33:43.821 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114779.log.gz 2023-11-08T19:33:43.822 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116182.log 2023-11-08T19:33:43.822 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103068.log.gz 2023-11-08T19:33:43.823 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118544.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118544.log.gz 2023-11-08T19:33:43.824 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116706.log 2023-11-08T19:33:43.825 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116182.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116182.log.gz 2023-11-08T19:33:43.825 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87871.log 2023-11-08T19:33:43.826 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100484.log 2023-11-08T19:33:43.826 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116706.log.gz 2023-11-08T19:33:43.827 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.87871.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.87871.log.gz 2023-11-08T19:33:43.827 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95346.log 2023-11-08T19:33:43.828 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108598.log 2023-11-08T19:33:43.829 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100484.log.gz 2023-11-08T19:33:43.830 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101336.log 2023-11-08T19:33:43.830 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95346.log.gz 2023-11-08T19:33:43.831 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120545.log 2023-11-08T19:33:43.832 INFO:teuthology.orchestra.run.smithi119.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108598.log.gz 2023-11-08T19:33:43.832 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101336.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101336.log.gz 2023-11-08T19:33:43.833 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117674.log 2023-11-08T19:33:43.834 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109186.log 2023-11-08T19:33:43.834 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120545.log.gz 2023-11-08T19:33:43.835 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108504.log 2023-11-08T19:33:43.836 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.117674.log.gz 2023-11-08T19:33:43.836 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109946.log 2023-11-08T19:33:43.836 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109186.log.gz 2023-11-08T19:33:43.837 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120093.log 2023-11-08T19:33:43.838 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108504.log.gz 2023-11-08T19:33:43.838 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109946.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109946.log.gz 2023-11-08T19:33:43.839 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100957.log 2023-11-08T19:33:43.840 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109270.log 2023-11-08T19:33:43.840 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120093.log.gz 2023-11-08T19:33:43.841 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99776.log 2023-11-08T19:33:43.842 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100957.log.gz 2023-11-08T19:33:43.842 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111434.log 2023-11-08T19:33:43.843 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109270.log.gz 2023-11-08T19:33:43.844 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111738.log 2023-11-08T19:33:43.844 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99776.log.gz 2023-11-08T19:33:43.845 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111392.log 2023-11-08T19:33:43.845 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111434.log.gz 2023-11-08T19:33:43.846 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111602.log 2023-11-08T19:33:43.847 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111738.log.gz 2023-11-08T19:33:43.847 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98110.log 2023-11-08T19:33:43.848 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111392.log.gz 2023-11-08T19:33:43.849 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119224.log 2023-11-08T19:33:43.849 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111602.log.gz 2023-11-08T19:33:43.850 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100876.log 2023-11-08T19:33:43.850 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98110.log.gz 2023-11-08T19:33:43.851 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118201.log 2023-11-08T19:33:43.851 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119224.log.gz 2023-11-08T19:33:43.852 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114160.log 2023-11-08T19:33:43.853 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100876.log.gz 2023-11-08T19:33:43.854 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111145.log 2023-11-08T19:33:43.854 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118201.log.gz 2023-11-08T19:33:43.855 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99188.log 2023-11-08T19:33:43.855 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.114160.log.gz 2023-11-08T19:33:43.856 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111822.log 2023-11-08T19:33:43.856 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111145.log.gz 2023-11-08T19:33:43.857 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99188.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99188.log.gz 2023-11-08T19:33:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93130.log 2023-11-08T19:33:43.858 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102554.log 2023-11-08T19:33:43.859 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111822.log.gz 2023-11-08T19:33:43.860 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94728.log 2023-11-08T19:33:43.860 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93130.log.gz 2023-11-08T19:33:43.861 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95358.log 2023-11-08T19:33:43.861 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102554.log.gz 2023-11-08T19:33:43.862 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.102181.log 2023-11-08T19:33:43.863 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94728.log.gz 2023-11-08T19:33:43.863 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98783.log 2023-11-08T19:33:43.864 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95358.log.gz 2023-11-08T19:33:43.865 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.102181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.102181.log.gz 2023-11-08T19:33:43.865 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114821.log 2023-11-08T19:33:43.866 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112242.log 2023-11-08T19:33:43.866 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98783.log.gz 2023-11-08T19:33:43.867 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98279.log 2023-11-08T19:33:43.868 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114821.log.gz 2023-11-08T19:33:43.869 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121182.log 2023-11-08T19:33:43.869 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112242.log.gz 2023-11-08T19:33:43.870 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119098.log 2023-11-08T19:33:43.870 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98279.log.gz 2023-11-08T19:33:43.871 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100316.log 2023-11-08T19:33:43.871 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121182.log.gz 2023-11-08T19:33:43.872 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100274.log 2023-11-08T19:33:43.873 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119098.log.gz 2023-11-08T19:33:43.874 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96770.log 2023-11-08T19:33:43.874 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100316.log.gz 2023-11-08T19:33:43.875 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101802.log 2023-11-08T19:33:43.875 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100274.log.gz 2023-11-08T19:33:43.876 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102596.log 2023-11-08T19:33:43.876 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96770.log.gz 2023-11-08T19:33:43.877 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98909.log 2023-11-08T19:33:43.878 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101802.log.gz 2023-11-08T19:33:43.878 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113842.log 2023-11-08T19:33:43.879 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102596.log.gz 2023-11-08T19:33:43.880 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107017.log 2023-11-08T19:33:43.880 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98909.log.gz 2023-11-08T19:33:43.881 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105493.log 2023-11-08T19:33:43.881 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113842.log.gz 2023-11-08T19:33:43.882 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107017.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107017.log.gz 2023-11-08T19:33:43.883 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100568.log 2023-11-08T19:33:43.883 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105493.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.105493.log.gz /var/log/ceph/ceph-client.admin.112429.log 2023-11-08T19:33:43.884 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100568.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100568.log.gz 2023-11-08T19:33:43.885 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120499.log 2023-11-08T19:33:43.886 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109904.log 2023-11-08T19:33:43.887 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.112429.log.gz 2023-11-08T19:33:43.887 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117123.log 2023-11-08T19:33:43.888 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120499.log.gz 2023-11-08T19:33:43.888 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115351.log 2023-11-08T19:33:43.889 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109904.log.gz 2023-11-08T19:33:43.890 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117123.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117123.log.gz 2023-11-08T19:33:43.890 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108934.log 2023-11-08T19:33:43.891 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95693.log 2023-11-08T19:33:43.892 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115351.log.gz 2023-11-08T19:33:43.892 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94770.log 2023-11-08T19:33:43.893 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108934.log.gz 2023-11-08T19:33:43.894 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105092.log 2023-11-08T19:33:43.894 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95693.log.gz 2023-11-08T19:33:43.895 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94770.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94770.log.gz 2023-11-08T19:33:43.896 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119990.log 2023-11-08T19:33:43.897 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110421.log 2023-11-08T19:33:43.897 INFO:teuthology.orchestra.run.smithi119.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.105092.log.gz 2023-11-08T19:33:43.898 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119990.log: gzip -5 --verbose -- 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119990.log.gz /var/log/ceph/ceph-client.admin.104099.log 2023-11-08T19:33:43.898 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:43.900 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115788.log 2023-11-08T19:33:43.900 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110421.log.gz 2023-11-08T19:33:43.901 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104099.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104099.log.gz 2023-11-08T19:33:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.97084.log 2023-11-08T19:33:43.902 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115788.log: gzip -5 --verbose -- 88.7% /var/log/ceph/ceph-client.admin.121056.log -- replaced with /var/log/ceph/ceph-client.admin.115788.log.gz 2023-11-08T19:33:43.903 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:43.903 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.97084.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.97084.log.gz 2023-11-08T19:33:43.904 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96480.log 2023-11-08T19:33:43.905 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99518.log 2023-11-08T19:33:43.906 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121056.log.gz 2023-11-08T19:33:43.906 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104141.log 2023-11-08T19:33:43.907 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96480.log.gz 2023-11-08T19:33:43.908 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99518.log.gz 2023-11-08T19:33:43.908 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103422.log 2023-11-08T19:33:43.909 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112116.log 2023-11-08T19:33:43.909 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104141.log.gz 2023-11-08T19:33:43.910 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100064.log 2023-11-08T19:33:43.910 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103422.log.gz 2023-11-08T19:33:43.911 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103632.log 2023-11-08T19:33:43.912 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112116.log.gz 2023-11-08T19:33:43.913 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118398.log 2023-11-08T19:33:43.913 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100064.log.gz 2023-11-08T19:33:43.914 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93510.log 2023-11-08T19:33:43.914 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103632.log.gz 2023-11-08T19:33:43.915 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117893.log 2023-11-08T19:33:43.915 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118398.log.gz 2023-11-08T19:33:43.916 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99650.log 2023-11-08T19:33:43.917 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.93510.log.gz 2023-11-08T19:33:43.918 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117893.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 88.6% 2023-11-08T19:33:43.918 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117893.log.gz 2023-11-08T19:33:43.919 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118816.log 2023-11-08T19:33:43.920 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99650.log.gz 2023-11-08T19:33:43.921 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118440.log 2023-11-08T19:33:43.922 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118816.log: gzip -5 --verbose -- 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118816.log.gz /var/log/ceph/ceph-client.admin.101422.log 2023-11-08T19:33:43.922 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:43.923 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118440.log: 89.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-08T19:33:43.924 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110549.log 2023-11-08T19:33:43.924 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118440.log.gz 2023-11-08T19:33:43.925 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101422.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101422.log.gz 2023-11-08T19:33:43.925 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96606.log 2023-11-08T19:33:43.926 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104363.log 2023-11-08T19:33:43.927 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110549.log.gz 2023-11-08T19:33:43.928 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111476.log 2023-11-08T19:33:43.928 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.96606.log.gz 2023-11-08T19:33:43.929 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104363.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104363.log.gz 2023-11-08T19:33:43.929 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117935.log 2023-11-08T19:33:43.931 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94381.log 2023-11-08T19:33:43.931 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111476.log.gz 2023-11-08T19:33:43.932 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117935.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117935.log.gz 2023-11-08T19:33:43.932 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114094.log 2023-11-08T19:33:43.933 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106889.log 2023-11-08T19:33:43.934 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94381.log.gz 2023-11-08T19:33:43.934 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119182.log 2023-11-08T19:33:43.935 INFO:teuthology.orchestra.run.smithi119.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.114094.log.gz 2023-11-08T19:33:43.936 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106889.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106889.log.gz 2023-11-08T19:33:43.936 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101928.log 2023-11-08T19:33:43.937 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92708.log 2023-11-08T19:33:43.938 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119182.log.gz 2023-11-08T19:33:43.939 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119654.log 2023-11-08T19:33:43.939 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.101928.log.gz 2023-11-08T19:33:43.940 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101634.log 2023-11-08T19:33:43.941 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92708.log.gz 2023-11-08T19:33:43.941 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107103.log 2023-11-08T19:33:43.942 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119654.log.gz 2023-11-08T19:33:43.943 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107589.log 2023-11-08T19:33:43.943 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101634.log.gz 2023-11-08T19:33:43.944 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114010.log 2023-11-08T19:33:43.944 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107103.log.gz 2023-11-08T19:33:43.945 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107061.log 2023-11-08T19:33:43.945 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107589.log.gz 2023-11-08T19:33:43.946 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110693.log 2023-11-08T19:33:43.947 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114010.log.gz 2023-11-08T19:33:43.947 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100232.log 2023-11-08T19:33:43.948 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107061.log.gz 2023-11-08T19:33:43.949 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119266.log 2023-11-08T19:33:43.949 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110693.log.gz 2023-11-08T19:33:43.950 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103716.log 2023-11-08T19:33:43.950 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100232.log.gz 2023-11-08T19:33:43.951 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104491.log 2023-11-08T19:33:43.951 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119266.log.gz 2023-11-08T19:33:43.952 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103716.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.103716.log.gz -5 2023-11-08T19:33:43.953 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98489.log 2023-11-08T19:33:43.954 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95696.log 2023-11-08T19:33:43.954 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104491.log.gz 2023-11-08T19:33:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98489.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98489.log.gz 2023-11-08T19:33:43.955 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107355.log 2023-11-08T19:33:43.956 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109522.log 2023-11-08T19:33:43.957 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95696.log.gz 2023-11-08T19:33:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107355.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107355.log.gz 2023-11-08T19:33:43.958 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113926.log 2023-11-08T19:33:43.959 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109522.log: gzip 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109522.log.gz -5 2023-11-08T19:33:43.959 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.117977.log 2023-11-08T19:33:43.962 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94297.log 2023-11-08T19:33:43.962 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113926.log.gz 2023-11-08T19:33:43.963 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111560.log 2023-11-08T19:33:43.963 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.117977.log.gz 2023-11-08T19:33:43.964 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120247.log 2023-11-08T19:33:43.964 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94297.log.gz 2023-11-08T19:33:43.965 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103464.log 2023-11-08T19:33:43.965 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111560.log.gz 2023-11-08T19:33:43.966 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105815.log 2023-11-08T19:33:43.966 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120247.log.gz 2023-11-08T19:33:43.967 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103464.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103464.log.gz 2023-11-08T19:33:43.967 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115830.log 2023-11-08T19:33:43.968 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110243.log 2023-11-08T19:33:43.968 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105815.log.gz 2023-11-08T19:33:43.969 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120972.log 2023-11-08T19:33:43.969 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115830.log.gz 2023-11-08T19:33:43.970 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105773.log 2023-11-08T19:33:43.971 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110243.log.gz 2023-11-08T19:33:43.971 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105713.log 2023-11-08T19:33:43.972 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120972.log.gz 2023-11-08T19:33:43.973 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115306.log 2023-11-08T19:33:43.973 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105773.log.gz 2023-11-08T19:33:43.974 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105713.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105713.log.gz 2023-11-08T19:33:43.974 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113548.log 2023-11-08T19:33:43.975 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115306.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115306.log.gz 2023-11-08T19:33:43.975 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120930.log 2023-11-08T19:33:43.976 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110843.log 2023-11-08T19:33:43.977 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113548.log.gz 2023-11-08T19:33:43.978 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93216.log 2023-11-08T19:33:43.978 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120930.log.gz 2023-11-08T19:33:43.979 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112639.log 2023-11-08T19:33:43.979 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.110843.log.gz 2023-11-08T19:33:43.980 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95178.log 2023-11-08T19:33:43.980 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93216.log.gz 2023-11-08T19:33:43.981 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112639.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112639.log.gzgzip 2023-11-08T19:33:43.981 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.99818.log 2023-11-08T19:33:43.982 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.111030.log 2023-11-08T19:33:43.983 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95178.log.gz 2023-11-08T19:33:43.984 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118628.log 2023-11-08T19:33:43.984 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99818.log.gz 2023-11-08T19:33:43.985 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.111030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.111030.log.gz 2023-11-08T19:33:43.985 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103170.log 2023-11-08T19:33:43.986 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103973.log 2023-11-08T19:33:43.986 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118628.log.gz 2023-11-08T19:33:43.987 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109480.log 2023-11-08T19:33:43.987 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103170.log.gz 2023-11-08T19:33:43.988 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103973.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103973.log.gz 2023-11-08T19:33:43.989 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96522.log 2023-11-08T19:33:43.990 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119906.log 2023-11-08T19:33:43.990 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109480.log.gz 2023-11-08T19:33:43.991 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114529.log 2023-11-08T19:33:43.991 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96522.log.gz 2023-11-08T19:33:43.992 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116952.log 2023-11-08T19:33:43.992 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119906.log.gz 2023-11-08T19:33:43.993 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113334.log 2023-11-08T19:33:43.993 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114529.log.gz 2023-11-08T19:33:43.994 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96102.log 2023-11-08T19:33:43.995 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.116952.log.gz 2023-11-08T19:33:43.996 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113334.log: gzip 88.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.113334.log.gz --verbose 2023-11-08T19:33:43.996 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.99104.log 2023-11-08T19:33:43.997 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112032.log 2023-11-08T19:33:43.998 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96102.log.gz 2023-11-08T19:33:43.998 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99104.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99104.log.gz 2023-11-08T19:33:43.999 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95052.log 2023-11-08T19:33:44.000 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112032.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112032.log.gz 2023-11-08T19:33:44.001 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110777.log 2023-11-08T19:33:44.001 INFO:teuthology.orchestra.run.smithi188.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-08T19:33:44.002 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104015.log 2023-11-08T19:33:44.002 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95052.log.gz 2023-11-08T19:33:44.003 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115629.log 2023-11-08T19:33:44.003 INFO:teuthology.orchestra.run.smithi119.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110777.log.gz 2023-11-08T19:33:44.004 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98321.log 2023-11-08T19:33:44.005 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104015.log.gz 2023-11-08T19:33:44.006 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115629.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115629.log.gz 2023-11-08T19:33:44.007 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101466.log 2023-11-08T19:33:44.008 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110465.log 2023-11-08T19:33:44.008 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98321.log.gz 2023-11-08T19:33:44.009 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98741.log 2023-11-08T19:33:44.010 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101466.log.gz 2023-11-08T19:33:44.010 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115873.log 2023-11-08T19:33:44.011 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110465.log.gz 2023-11-08T19:33:44.012 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108850.log 2023-11-08T19:33:44.012 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98741.log.gz 2023-11-08T19:33:44.013 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113190.log 2023-11-08T19:33:44.013 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115873.log.gz 2023-11-08T19:33:44.014 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108850.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108850.log.gz 2023-11-08T19:33:44.014 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119612.log 2023-11-08T19:33:44.016 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115254.log 2023-11-08T19:33:44.016 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113190.log.gz 2023-11-08T19:33:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119612.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119612.log.gz 2023-11-08T19:33:44.017 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98573.log 2023-11-08T19:33:44.019 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119056.log 2023-11-08T19:33:44.019 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115254.log.gz 2023-11-08T19:33:44.020 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98573.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.98573.log.gz /var/log/ceph/ceph-client.admin.100610.log 2023-11-08T19:33:44.020 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:44.021 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107439.log 2023-11-08T19:33:44.021 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119056.log.gz 2023-11-08T19:33:44.022 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105587.log 2023-11-08T19:33:44.023 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100610.log.gz 2023-11-08T19:33:44.023 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95685.log 2023-11-08T19:33:44.024 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107439.log.gz 2023-11-08T19:33:44.025 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121266.log 2023-11-08T19:33:44.025 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.105587.log.gz 2023-11-08T19:33:44.026 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112387.log 2023-11-08T19:33:44.026 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95685.log.gz 2023-11-08T19:33:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121266.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121266.log.gz 2023-11-08T19:33:44.027 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113106.log 2023-11-08T19:33:44.028 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112387.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-08T19:33:44.029 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112387.log.gz 2023-11-08T19:33:44.030 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113106.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113106.log.gz 2023-11-08T19:33:44.030 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119528.log 2023-11-08T19:33:44.031 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99980.log 2023-11-08T19:33:44.032 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119528.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119528.log.gz 2023-11-08T19:33:44.033 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94481.log 2023-11-08T19:33:44.034 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99980.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99980.log.gz 2023-11-08T19:33:44.035 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103212.log 2023-11-08T19:33:44.036 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94481.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94481.log.gz 2023-11-08T19:33:44.036 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104405.log 2023-11-08T19:33:44.037 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103212.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103212.log.gz 2023-11-08T19:33:44.038 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119486.log 2023-11-08T19:33:44.038 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104405.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104405.log.gz 2023-11-08T19:33:44.039 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116579.log 2023-11-08T19:33:44.040 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119486.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119486.log.gz 2023-11-08T19:33:44.041 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103296.log 2023-11-08T19:33:44.042 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116579.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116579.log.gz 2023-11-08T19:33:44.042 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104701.log 2023-11-08T19:33:44.043 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103296.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103296.log.gz 2023-11-08T19:33:44.044 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109396.log 2023-11-08T19:33:44.045 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104701.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104701.log.gz 2023-11-08T19:33:44.045 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96564.log 2023-11-08T19:33:44.046 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105857.log 2023-11-08T19:33:44.047 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109396.log.gz 2023-11-08T19:33:44.048 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96564.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96564.log.gz -5 2023-11-08T19:33:44.048 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.87613.log 2023-11-08T19:33:44.049 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105857.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105857.log.gz 2023-11-08T19:33:44.050 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.97531.log 2023-11-08T19:33:44.051 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.87613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108724.log 2023-11-08T19:33:44.051 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87613.log.gz 2023-11-08T19:33:44.052 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.2.97531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.97531.log.gz 2023-11-08T19:33:44.053 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119350.log 2023-11-08T19:33:44.054 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95355.log 2023-11-08T19:33:44.055 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108724.log.gz 2023-11-08T19:33:44.055 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119350.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119350.log.gz 2023-11-08T19:33:44.056 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92789.log 2023-11-08T19:33:44.057 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101252.log 2023-11-08T19:33:44.057 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95355.log.gz 2023-11-08T19:33:44.058 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92789.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92789.log.gz 2023-11-08T19:33:44.059 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116802.log 2023-11-08T19:33:44.060 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100190.log 2023-11-08T19:33:44.060 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101252.log.gz 2023-11-08T19:33:44.061 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116802.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116802.log.gz 2023-11-08T19:33:44.062 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102259.log 2023-11-08T19:33:44.063 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100358.log 2023-11-08T19:33:44.064 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100190.log.gz 2023-11-08T19:33:44.064 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102259.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102259.log.gz 2023-11-08T19:33:44.065 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116844.log 2023-11-08T19:33:44.066 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107523.log 2023-11-08T19:33:44.066 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100358.log.gz 2023-11-08T19:33:44.067 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116844.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.116844.log.gz 2023-11-08T19:33:44.068 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117806.log 2023-11-08T19:33:44.069 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98405.log 2023-11-08T19:33:44.070 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.107523.log.gz 2023-11-08T19:33:44.071 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117806.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117806.log.gz 2023-11-08T19:33:44.071 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99692.log 2023-11-08T19:33:44.072 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110115.log 2023-11-08T19:33:44.073 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98405.log.gz 2023-11-08T19:33:44.073 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101042.log 2023-11-08T19:33:44.074 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99692.log.gz 2023-11-08T19:33:44.075 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118056.log 2023-11-08T19:33:44.075 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110115.log.gz 2023-11-08T19:33:44.076 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118304.log 2023-11-08T19:33:44.077 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101042.log.gz 2023-11-08T19:33:44.078 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94339.log 2023-11-08T19:33:44.078 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118056.log.gz 2023-11-08T19:33:44.079 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118304.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.118304.log.gz 2023-11-08T19:33:44.079 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111948.log 2023-11-08T19:33:44.080 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114445.log 2023-11-08T19:33:44.081 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94339.log.gz 2023-11-08T19:33:44.081 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111948.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111948.log.gz 2023-11-08T19:33:44.082 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98867.log 2023-11-08T19:33:44.083 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107776.log 2023-11-08T19:33:44.083 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114445.log.gz 2023-11-08T19:33:44.085 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98867.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98867.log.gz 2023-11-08T19:33:44.085 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98447.log 2023-11-08T19:33:44.085 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.107776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107776.log.gz 2023-11-08T19:33:44.086 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108057.log 2023-11-08T19:33:44.087 INFO:teuthology.orchestra.run.smithi188.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-08T19:33:44.088 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100796.log 2023-11-08T19:33:44.088 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98447.log.gz 2023-11-08T19:33:44.089 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108098.log 2023-11-08T19:33:44.089 INFO:teuthology.orchestra.run.smithi119.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.108057.log.gz 2023-11-08T19:33:44.090 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100796.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100796.log.gz 2023-11-08T19:33:44.090 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118482.log 2023-11-08T19:33:44.091 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110379.log 2023-11-08T19:33:44.091 INFO:teuthology.orchestra.run.smithi119.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.108098.log.gz 2023-11-08T19:33:44.092 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118482.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118482.log.gz 2023-11-08T19:33:44.093 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101508.log 2023-11-08T19:33:44.094 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106212.log 2023-11-08T19:33:44.094 INFO:teuthology.orchestra.run.smithi119.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110379.log.gz 2023-11-08T19:33:44.095 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101508.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.101508.log.gz 2023-11-08T19:33:44.095 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108001.log 2023-11-08T19:33:44.096 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110609.log 2023-11-08T19:33:44.097 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.106212.log.gz 2023-11-08T19:33:44.098 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107187.log 2023-11-08T19:33:44.098 INFO:teuthology.orchestra.run.smithi119.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.108001.log.gz 2023-11-08T19:33:44.099 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.114347.log 2023-11-08T19:33:44.099 INFO:teuthology.orchestra.run.smithi119.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.110609.log.gz 2023-11-08T19:33:44.100 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107187.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107187.log.gz 2023-11-08T19:33:44.101 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120135.log 2023-11-08T19:33:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.114347.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.114347.log.gz 2023-11-08T19:33:44.102 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99308.log 2023-11-08T19:33:44.103 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105451.log 2023-11-08T19:33:44.103 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120135.log.gz 2023-11-08T19:33:44.104 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113420.log 2023-11-08T19:33:44.105 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99308.log.gz 2023-11-08T19:33:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105451.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105451.log.gz 2023-11-08T19:33:44.106 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112200.log 2023-11-08T19:33:44.107 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102858.log 2023-11-08T19:33:44.107 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113420.log.gz 2023-11-08T19:33:44.108 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112200.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112200.log.gz 2023-11-08T19:33:44.109 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109648.log 2023-11-08T19:33:44.110 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113884.log 2023-11-08T19:33:44.110 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102858.log.gz 2023-11-08T19:33:44.111 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111906.log 2023-11-08T19:33:44.111 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109648.log.gz 2023-11-08T19:33:44.112 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113884.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113884.log.gz 2023-11-08T19:33:44.113 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95344.log 2023-11-08T19:33:44.114 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111906.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111906.log.gz 2023-11-08T19:33:44.114 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113632.log 2023-11-08T19:33:44.115 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95344.log: gzip -5 --verbose 88.1% -- -- replaced with /var/log/ceph/ceph-client.admin.95344.log.gz /var/log/ceph/ceph-client.admin.115128.log 2023-11-08T19:33:44.115 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:44.116 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112982.log 2023-11-08T19:33:44.116 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113632.log.gz 2023-11-08T19:33:44.117 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115128.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115128.log.gz 2023-11-08T19:33:44.118 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98016.log 2023-11-08T19:33:44.119 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112982.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112982.log.gz 2023-11-08T19:33:44.119 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108210.log 2023-11-08T19:33:44.120 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98016.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98016.log.gz 2023-11-08T19:33:44.120 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.118730.log 2023-11-08T19:33:44.121 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108210.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108210.log.gz 2023-11-08T19:33:44.122 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98195.log 2023-11-08T19:33:44.122 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118730.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118730.log.gz 2023-11-08T19:33:44.123 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95683.log 2023-11-08T19:33:44.124 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106931.log 2023-11-08T19:33:44.124 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98195.log.gz 2023-11-08T19:33:44.125 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95683.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95683.log.gz -5 2023-11-08T19:33:44.125 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.87981.log 2023-11-08T19:33:44.126 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104321.log 2023-11-08T19:33:44.127 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106931.log.gz 2023-11-08T19:33:44.127 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.87981.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.87981.log.gz 2023-11-08T19:33:44.128 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117632.log 2023-11-08T19:33:44.129 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96060.log 2023-11-08T19:33:44.129 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104321.log.gz 2023-11-08T19:33:44.130 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117632.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117632.log.gz 2023-11-08T19:33:44.130 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-08T19:33:44.131 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116398.log 2023-11-08T19:33:44.131 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96060.log.gz 2023-11-08T19:33:44.132 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112723.log 2023-11-08T19:33:44.134 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116398.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116398.log.gz 2023-11-08T19:33:44.134 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112765.log 2023-11-08T19:33:44.135 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118670.log 2023-11-08T19:33:44.136 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112723.log.gz 2023-11-08T19:33:44.137 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112765.log: gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.112765.log.gz 2023-11-08T19:33:44.137 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.101718.log 2023-11-08T19:33:44.139 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97787.log 2023-11-08T19:33:44.139 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118670.log.gz 2023-11-08T19:33:44.140 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112513.log 2023-11-08T19:33:44.141 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101718.log.gz 2023-11-08T19:33:44.142 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110199.log 2023-11-08T19:33:44.142 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97787.log.gz 2023-11-08T19:33:44.143 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112513.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112513.log.gz -5 2023-11-08T19:33:44.144 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.106296.log 2023-11-08T19:33:44.145 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101378.log 2023-11-08T19:33:44.145 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110199.log.gz 2023-11-08T19:33:44.147 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98657.log 2023-11-08T19:33:44.147 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106296.log.gz 2023-11-08T19:33:44.148 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101378.log: gzip -5 89.1% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.101378.log.gz 2023-11-08T19:33:44.149 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.99230.log 2023-11-08T19:33:44.150 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97703.log 88.7% 2023-11-08T19:33:44.150 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98657.log.gz 2023-11-08T19:33:44.152 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104617.log 2023-11-08T19:33:44.152 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99230.log.gz 2023-11-08T19:33:44.153 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106422.log 2023-11-08T19:33:44.153 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97703.log.gz 2023-11-08T19:33:44.155 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103811.log 2023-11-08T19:33:44.155 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104617.log.gz 2023-11-08T19:33:44.156 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114487.log 2023-11-08T19:33:44.157 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106422.log.gz 2023-11-08T19:33:44.158 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116622.log 2023-11-08T19:33:44.158 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.103811.log.gz 2023-11-08T19:33:44.160 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114487.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.114487.log.gz -5 2023-11-08T19:33:44.160 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111990.log 2023-11-08T19:33:44.161 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111780.log 2023-11-08T19:33:44.161 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116622.log.gz 2023-11-08T19:33:44.166 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118356.log 2023-11-08T19:33:44.167 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111990.log.gz 2023-11-08T19:33:44.167 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111780.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111780.log.gz 2023-11-08T19:33:44.168 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121411.log 2023-11-08T19:33:44.168 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118356.log: gzip 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118356.log.gz -5 2023-11-08T19:33:44.168 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.95934.log 2023-11-08T19:33:44.169 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99608.log 2023-11-08T19:33:44.169 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121411.log.gz 2023-11-08T19:33:44.170 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104575.log 2023-11-08T19:33:44.171 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95934.log.gz 2023-11-08T19:33:44.171 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97743.log 2023-11-08T19:33:44.172 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99608.log.gz 2023-11-08T19:33:44.172 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119570.log 2023-11-08T19:33:44.173 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104575.log.gz 2023-11-08T19:33:44.174 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97743.log.gz 2023-11-08T19:33:44.174 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113590.log 2023-11-08T19:33:44.175 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106067.log 2023-11-08T19:33:44.176 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119570.log.gz 2023-11-08T19:33:44.177 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95360.log 2023-11-08T19:33:44.177 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113590.log.gz 2023-11-08T19:33:44.178 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106506.log 89.0% 2023-11-08T19:33:44.179 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106067.log.gz 2023-11-08T19:33:44.180 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95136.log 2023-11-08T19:33:44.180 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95360.log.gz 2023-11-08T19:33:44.181 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114052.log 2023-11-08T19:33:44.182 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106506.log.gz 2023-11-08T19:33:44.183 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95136.log: gzip 89.5% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.95136.log.gz -- 2023-11-08T19:33:44.183 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.121351.log 2023-11-08T19:33:44.185 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110287.log 2023-11-08T19:33:44.185 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114052.log.gz 2023-11-08T19:33:44.186 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115007.log 2023-11-08T19:33:44.186 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121351.log.gz 2023-11-08T19:33:44.188 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112074.log 2023-11-08T19:33:44.188 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110287.log.gz 2023-11-08T19:33:44.189 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106464.log 2023-11-08T19:33:44.189 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115007.log.gz 2023-11-08T19:33:44.190 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108294.log 2023-11-08T19:33:44.191 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112074.log.gz 2023-11-08T19:33:44.192 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106464.log: gzip 89.1% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.106464.log.gz -- 2023-11-08T19:33:44.192 INFO:teuthology.orchestra.run.smithi119.stderr: /var/log/ceph/ceph-client.admin.116356.log 2023-11-08T19:33:44.193 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110735.log 88.8% 2023-11-08T19:33:44.194 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108294.log.gz 2023-11-08T19:33:44.195 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111518.log 2023-11-08T19:33:44.195 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116356.log.gz 2023-11-08T19:33:44.197 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106025.log 2023-11-08T19:33:44.197 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110735.log.gz 2023-11-08T19:33:44.198 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103110.log 2023-11-08T19:33:44.198 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111518.log.gz 2023-11-08T19:33:44.199 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100526.log 2023-11-08T19:33:44.200 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106025.log.gz 2023-11-08T19:33:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103110.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103110.log.gz 2023-11-08T19:33:44.201 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108682.log 2023-11-08T19:33:44.203 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93426.log 2023-11-08T19:33:44.203 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100526.log.gz 2023-11-08T19:33:44.204 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113148.log 2023-11-08T19:33:44.205 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108682.log.gz 2023-11-08T19:33:44.206 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94439.log 2023-11-08T19:33:44.206 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93426.log.gz 2023-11-08T19:33:44.207 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113148.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113148.log.gz 2023-11-08T19:33:44.208 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102340.log 2023-11-08T19:33:44.209 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94439.log: gzip -5 --verbose -- 88.8% /var/log/ceph/ceph-client.admin.109102.log 2023-11-08T19:33:44.209 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94439.log.gz 2023-11-08T19:33:44.210 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100999.log 2023-11-08T19:33:44.210 INFO:teuthology.orchestra.run.smithi119.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102340.log.gz 2023-11-08T19:33:44.211 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109102.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109102.log.gz 2023-11-08T19:33:44.212 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120205.log 2023-11-08T19:33:44.213 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93384.log 2023-11-08T19:33:44.213 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100999.log.gz 2023-11-08T19:33:44.215 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113716.log 2023-11-08T19:33:44.215 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120205.log.gz 2023-11-08T19:33:44.216 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106127.log 2023-11-08T19:33:44.216 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93384.log.gz 2023-11-08T19:33:44.217 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99350.log 2023-11-08T19:33:44.218 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113716.log.gz 2023-11-08T19:33:44.219 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113968.log 2023-11-08T19:33:44.219 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106127.log.gz 2023-11-08T19:33:44.220 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99350.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99350.log.gz 2023-11-08T19:33:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103338.log 2023-11-08T19:33:44.221 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108892.log 2023-11-08T19:33:44.222 INFO:teuthology.orchestra.run.smithi119.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113968.log.gz 2023-11-08T19:33:44.223 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103338.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103338.log.gz 2023-11-08T19:33:44.223 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117528.log 2023-11-08T19:33:44.224 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108892.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.108892.log.gz 2023-11-08T19:33:44.224 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109312.log 2023-11-08T19:33:44.226 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100022.log 2023-11-08T19:33:44.226 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117528.log.gz 2023-11-08T19:33:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109312.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109312.log.gz 2023-11-08T19:33:44.227 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95357.log 2023-11-08T19:33:44.228 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100022.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100022.log.gz 2023-11-08T19:33:44.229 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100106.log 2023-11-08T19:33:44.230 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106601.log 2023-11-08T19:33:44.230 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95357.log.gz 2023-11-08T19:33:44.231 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.100106.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100106.log.gz 2023-11-08T19:33:44.231 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112807.log 2023-11-08T19:33:44.232 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106601.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106601.log.gz 2023-11-08T19:33:44.233 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95682.log 2023-11-08T19:33:44.234 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104809.log 2023-11-08T19:33:44.235 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112807.log.gz 2023-11-08T19:33:44.235 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95682.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95682.log.gz 2023-11-08T19:33:44.236 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96144.log 2023-11-08T19:33:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104809.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104809.log.gz 2023-11-08T19:33:44.237 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95678.log 2023-11-08T19:33:44.239 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101550.log 2023-11-08T19:33:44.239 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.96144.log.gz 2023-11-08T19:33:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95678.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95678.log.gz 2023-11-08T19:33:44.240 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99734.log 2023-11-08T19:33:44.241 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101168.log 2023-11-08T19:33:44.242 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101550.log.gz 2023-11-08T19:33:44.242 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.99734.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99734.log.gz 2023-11-08T19:33:44.243 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119780.log 2023-11-08T19:33:44.244 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101168.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101168.log.gz 2023-11-08T19:33:44.244 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107397.log 2023-11-08T19:33:44.245 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102984.log 2023-11-08T19:33:44.246 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119780.log.gz 2023-11-08T19:33:44.246 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107397.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107397.log.gz 2023-11-08T19:33:44.247 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108640.log 2023-11-08T19:33:44.248 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102984.log.gz 2023-11-08T19:33:44.249 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101760.log 2023-11-08T19:33:44.251 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108640.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-08T19:33:44.251 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108640.log.gz 2023-11-08T19:33:44.252 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101760.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101760.log.gz 2023-11-08T19:33:44.252 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105148.log 2023-11-08T19:33:44.254 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92916.log 2023-11-08T19:33:44.255 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109606.log 2023-11-08T19:33:44.255 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.105148.log.gz 2023-11-08T19:33:44.256 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114571.log 2023-11-08T19:33:44.257 INFO:teuthology.orchestra.run.smithi119.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92916.log.gz 2023-11-08T19:33:44.257 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109354.log 2023-11-08T19:33:44.258 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109606.log.gz 2023-11-08T19:33:44.259 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.114571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108252.log 2023-11-08T19:33:44.259 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114571.log.gz 2023-11-08T19:33:44.260 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106170.log 2023-11-08T19:33:44.260 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109354.log.gz 2023-11-08T19:33:44.261 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108252.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108252.log.gzgzip 2023-11-08T19:33:44.261 INFO:teuthology.orchestra.run.smithi119.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93172.log 2023-11-08T19:33:44.262 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116290.log 2023-11-08T19:33:44.264 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106170.log.gz 2023-11-08T19:33:44.264 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121224.log 2023-11-08T19:33:44.265 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93172.log.gz 2023-11-08T19:33:44.267 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101886.log 2023-11-08T19:33:44.267 INFO:teuthology.orchestra.run.smithi119.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.116290.log.gz 2023-11-08T19:33:44.269 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121553.log 2023-11-08T19:33:44.269 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121224.log.gz 2023-11-08T19:33:44.270 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101886.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101886.log.gz 2023-11-08T19:33:44.270 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93976.log 2023-11-08T19:33:44.272 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95892.log 2023-11-08T19:33:44.273 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121553.log.gz 2023-11-08T19:33:44.273 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103548.log 2023-11-08T19:33:44.275 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110073.log 2023-11-08T19:33:44.275 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95892.log.gz 2023-11-08T19:33:44.276 INFO:teuthology.orchestra.run.smithi119.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.93976.log.gz 2023-11-08T19:33:44.276 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121308.log 2023-11-08T19:33:44.277 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103548.log.gz 2023-11-08T19:33:44.278 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110073.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.110073.log.gz -5 2023-11-08T19:33:44.279 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.119308.log 2023-11-08T19:33:44.279 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95694.log 2023-11-08T19:33:44.280 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121308.log.gz 2023-11-08T19:33:44.281 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116758.log 2023-11-08T19:33:44.281 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119308.log.gz 2023-11-08T19:33:44.282 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110507.log 2023-11-08T19:33:44.282 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95694.log.gz 2023-11-08T19:33:44.283 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105629.log 2023-11-08T19:33:44.284 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116758.log.gz 2023-11-08T19:33:44.284 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119864.log 2023-11-08T19:33:44.285 INFO:teuthology.orchestra.run.smithi119.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110507.log.gz 2023-11-08T19:33:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96354.log 2023-11-08T19:33:44.287 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105629.log.gz 2023-11-08T19:33:44.287 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121471.log 2023-11-08T19:33:44.288 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119864.log.gz 2023-11-08T19:33:44.289 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106338.log 2023-11-08T19:33:44.289 INFO:teuthology.orchestra.run.smithi119.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96354.log.gz 2023-11-08T19:33:44.290 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121471.log: gzip -5 88.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.121471.log.gz 2023-11-08T19:33:44.290 INFO:teuthology.orchestra.run.smithi119.stderr: -- /var/log/ceph/ceph-client.admin.97871.log 2023-11-08T19:33:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106338.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106338.log.gz 2023-11-08T19:33:44.291 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98237.log 2023-11-08T19:33:44.292 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97973.log 2023-11-08T19:33:44.292 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97871.log.gz 2023-11-08T19:33:44.293 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103674.log 2023-11-08T19:33:44.294 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98237.log.gz 2023-11-08T19:33:44.295 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104996.log 2023-11-08T19:33:44.295 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97973.log.gz 2023-11-08T19:33:44.296 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94607.log 2023-11-08T19:33:44.297 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103674.log.gz 2023-11-08T19:33:44.298 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.0.104996.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.104996.log.gz 2023-11-08T19:33:44.299 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92612.log 2023-11-08T19:33:44.299 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120636.log 2023-11-08T19:33:44.300 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94607.log.gz 2023-11-08T19:33:44.301 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101994.log 2023-11-08T19:33:44.301 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92612.log.gz 2023-11-08T19:33:44.302 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104227.log 2023-11-08T19:33:44.303 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120636.log.gz 2023-11-08T19:33:44.304 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92874.log 2023-11-08T19:33:44.304 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101994.log.gz 2023-11-08T19:33:44.305 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104227.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-08T19:33:44.305 INFO:teuthology.orchestra.run.smithi119.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104227.log.gz 2023-11-08T19:33:44.306 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110651.log 2023-11-08T19:33:44.307 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92874.log.gz 2023-11-08T19:33:44.308 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113464.log 2023-11-08T19:33:44.310 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113274.log 2023-11-08T19:33:44.310 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110651.log.gz 2023-11-08T19:33:44.311 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108336.log 2023-11-08T19:33:44.311 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113464.log.gz 2023-11-08T19:33:44.312 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107979.log 2023-11-08T19:33:44.313 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.113274.log.gz 2023-11-08T19:33:44.314 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108808.log 2023-11-08T19:33:44.314 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108336.log.gz 2023-11-08T19:33:44.315 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117366.log 2023-11-08T19:33:44.316 INFO:teuthology.orchestra.run.smithi119.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.107979.log.gz 2023-11-08T19:33:44.317 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101210.log 2023-11-08T19:33:44.317 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108808.log.gz 2023-11-08T19:33:44.319 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117366.log: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93000.log 2023-11-08T19:33:44.319 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.117366.log.gz 2023-11-08T19:33:44.320 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101210.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101210.log.gz 2023-11-08T19:33:44.320 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103380.log 2023-11-08T19:33:44.322 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92958.log 2023-11-08T19:33:44.322 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93000.log.gz 2023-11-08T19:33:44.324 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103380.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103380.log.gz 2023-11-08T19:33:44.324 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108766.log 2023-11-08T19:33:44.325 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92958.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92958.log.gz 2023-11-08T19:33:44.326 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113232.log 2023-11-08T19:33:44.327 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105941.log 2023-11-08T19:33:44.328 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108766.log.gz 2023-11-08T19:33:44.329 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103891.log 2023-11-08T19:33:44.329 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113232.log.gz 2023-11-08T19:33:44.330 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105941.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105941.log.gz 2023-11-08T19:33:44.331 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96438.log 2023-11-08T19:33:44.332 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120762.log 2023-11-08T19:33:44.333 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103891.log.gz 2023-11-08T19:33:44.333 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119738.log 2023-11-08T19:33:44.333 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96438.log.gz 2023-11-08T19:33:44.335 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113376.log 2023-11-08T19:33:44.335 INFO:teuthology.orchestra.run.smithi119.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120762.log.gz 2023-11-08T19:33:44.336 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104057.log 2023-11-08T19:33:44.337 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119738.log.gz 2023-11-08T19:33:44.338 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113376.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113376.log.gz 2023-11-08T19:33:44.339 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102512.log 2023-11-08T19:33:44.341 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104057.log: 89.2%gzip/var/log/ceph/ceph-client.admin.102512.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.121140.log 2023-11-08T19:33:44.341 INFO:teuthology.orchestra.run.smithi119.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102512.log.gz 2023-11-08T19:33:44.342 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104057.log.gz 2023-11-08T19:33:44.342 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93342.log 2023-11-08T19:33:44.343 INFO:teuthology.orchestra.run.smithi119.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-08T19:33:44.345 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.121140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101294.log 2023-11-08T19:33:44.346 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93342.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121140.log.gz 2023-11-08T19:33:44.346 INFO:teuthology.orchestra.run.smithi119.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93342.log.gz 2023-11-08T19:33:44.347 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-08T19:33:44.347 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101294.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95010.log 2023-11-08T19:33:44.347 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101294.log.gz 2023-11-08T19:33:44.348 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114613.log 2023-11-08T19:33:44.349 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102942.log 2023-11-08T19:33:44.349 INFO:teuthology.orchestra.run.smithi119.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95010.log.gz 2023-11-08T19:33:44.349 INFO:teuthology.orchestra.run.smithi119.stderr: 86.5% -- replaced with /var/log/ceph/ceph.log.gz/var/log/ceph/ceph-client.admin.114613.log: 2023-11-08T19:33:44.350 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115086.log 2023-11-08T19:33:44.350 INFO:teuthology.orchestra.run.smithi119.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.114613.log.gz 2023-11-08T19:33:44.351 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113800.log 2023-11-08T19:33:44.351 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102942.log.gz 2023-11-08T19:33:44.352 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115086.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115086.log.gz 2023-11-08T19:33:44.352 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-08T19:33:44.353 INFO:teuthology.orchestra.run.smithi119.stderr:gzip/var/log/ceph/ceph-client.admin.113800.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98531.log 2023-11-08T19:33:44.354 INFO:teuthology.orchestra.run.smithi119.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.113800.log.gz 2023-11-08T19:33:44.354 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94881.log 2023-11-08T19:33:44.356 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96186.log 2023-11-08T19:33:44.356 INFO:teuthology.orchestra.run.smithi119.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98531.log.gz 2023-11-08T19:33:44.357 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117282.log 2023-11-08T19:33:44.358 INFO:teuthology.orchestra.run.smithi119.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94881.log.gz 2023-11-08T19:33:44.359 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111350.log 2023-11-08T19:33:44.359 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.96186.log.gz 2023-11-08T19:33:44.360 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117282.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-08T19:33:44.360 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117282.log.gz 2023-11-08T19:33:44.361 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.111350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115393.log 2023-11-08T19:33:44.362 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111350.log.gz 2023-11-08T19:33:44.363 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101844.log 2023-11-08T19:33:44.364 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115393.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115393.log.gz 2023-11-08T19:33:44.365 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93576.log 2023-11-08T19:33:44.368 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.101844.log: 87.8%gzip -5 --verbose --/var/log/ceph/ceph-client.admin.93576.log: /var/log/ceph/ceph-client.admin.112902.log 2023-11-08T19:33:44.368 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101844.log.gz 2023-11-08T19:33:44.368 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93576.log.gz 2023-11-08T19:33:44.370 INFO:teuthology.orchestra.run.smithi119.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-08T19:33:44.372 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117222.log 2023-11-08T19:33:44.373 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112902.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112902.log.gz 2023-11-08T19:33:44.374 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117324.log 2023-11-08T19:33:44.375 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94523.log 2023-11-08T19:33:44.375 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117222.log.gz 2023-11-08T19:33:44.377 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.117324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92831.log 2023-11-08T19:33:44.377 INFO:teuthology.orchestra.run.smithi119.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117324.log.gz 2023-11-08T19:33:44.378 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94523.log: gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94523.log.gz -5 2023-11-08T19:33:44.379 INFO:teuthology.orchestra.run.smithi119.stderr: --verbose -- /var/log/ceph/ceph-client.admin.97661.log 2023-11-08T19:33:44.380 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105055.log 2023-11-08T19:33:44.380 INFO:teuthology.orchestra.run.smithi119.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.92831.log.gz 2023-11-08T19:33:44.381 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97661.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104183.log 2023-11-08T19:33:44.381 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.97661.log.gz 2023-11-08T19:33:44.382 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115671.log 2023-11-08T19:33:44.383 INFO:teuthology.orchestra.run.smithi119.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.105055.log.gz 2023-11-08T19:33:44.383 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104183.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104183.log.gz 2023-11-08T19:33:44.384 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113674.log 2023-11-08T19:33:44.385 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95695.log 2023-11-08T19:33:44.386 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115671.log.gz 2023-11-08T19:33:44.386 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92490.log 2023-11-08T19:33:44.387 INFO:teuthology.orchestra.run.smithi119.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113674.log.gz 2023-11-08T19:33:44.388 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95695.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93300.log 2023-11-08T19:33:44.388 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95695.log.gz 2023-11-08T19:33:44.390 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.92490.log: gzip -5 --verbose 89.6% -- -- replaced with /var/log/ceph/ceph-client.admin.92490.log.gz /var/log/ceph/ceph-client.admin.106380.log 2023-11-08T19:33:44.390 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:44.391 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.93300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118858.log 2023-11-08T19:33:44.392 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93300.log.gz 2023-11-08T19:33:44.392 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106380.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106380.log.gz 2023-11-08T19:33:44.393 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102638.log 2023-11-08T19:33:44.394 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118858.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118858.log.gz 2023-11-08T19:33:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113064.log 2023-11-08T19:33:44.395 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102638.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102638.log.gz 2023-11-08T19:33:44.396 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96312.log 2023-11-08T19:33:44.397 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113064.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112555.log 2023-11-08T19:33:44.398 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.113064.log.gz 2023-11-08T19:33:44.399 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104271.log 2023-11-08T19:33:44.400 INFO:teuthology.orchestra.run.smithi119.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96312.log.gz 2023-11-08T19:33:44.400 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112555.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112555.log.gz 2023-11-08T19:33:44.402 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118117.log 2023-11-08T19:33:44.402 INFO:teuthology.orchestra.run.smithi119.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104271.log.gz 2023-11-08T19:33:44.403 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113758.log 2023-11-08T19:33:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.118117.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.118117.log.gz 2023-11-08T19:33:44.404 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116886.log 2023-11-08T19:33:44.405 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.113758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94565.log 2023-11-08T19:33:44.406 INFO:teuthology.orchestra.run.smithi119.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113758.log.gz 2023-11-08T19:33:44.409 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103026.log 2023-11-08T19:33:44.410 INFO:teuthology.orchestra.run.smithi119.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.116886.log.gz 2023-11-08T19:33:44.410 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94565.log: gzip 91.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-08T19:33:44.411 INFO:teuthology.orchestra.run.smithi119.stderr: 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.98066.log 2023-11-08T19:33:44.411 INFO:teuthology.orchestra.run.smithi119.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94565.log.gz 2023-11-08T19:33:44.415 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105899.log 2023-11-08T19:33:44.415 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103026.log.gz 2023-11-08T19:33:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98066.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98066.log.gz 2023-11-08T19:33:44.416 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120804.log 2023-11-08T19:33:44.417 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108462.log 2023-11-08T19:33:44.417 INFO:teuthology.orchestra.run.smithi119.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105899.log.gz 2023-11-08T19:33:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120804.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120804.log.gz 2023-11-08T19:33:44.418 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109018.log 2023-11-08T19:33:44.419 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.108462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98153.log 2023-11-08T19:33:44.419 INFO:teuthology.orchestra.run.smithi119.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.108462.log.gz 2023-11-08T19:33:44.423 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109018.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109018.log.gz 2023-11-08T19:33:44.424 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95094.log 2023-11-08T19:33:44.424 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.98153.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98153.log.gz 2023-11-08T19:33:44.425 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120846.log 2023-11-08T19:33:44.425 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.95094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107313.log 2023-11-08T19:33:44.426 INFO:teuthology.orchestra.run.smithi119.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.95094.log.gz 2023-11-08T19:33:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.120846.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120846.log.gz 2023-11-08T19:33:44.426 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110031.log 2023-11-08T19:33:44.427 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107313.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107313.log.gz 2023-11-08T19:33:44.427 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116224.log 2023-11-08T19:33:44.428 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.110031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106762.log 2023-11-08T19:33:44.428 INFO:teuthology.orchestra.run.smithi119.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110031.log.gz 2023-11-08T19:33:44.432 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.116224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119948.log 2023-11-08T19:33:44.433 INFO:teuthology.orchestra.run.smithi119.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.116224.log.gz 2023-11-08T19:33:44.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.106762.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106762.log.gz 2023-11-08T19:33:44.434 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103590.log 2023-11-08T19:33:44.434 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.119948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94812.log 2023-11-08T19:33:44.434 INFO:teuthology.orchestra.run.smithi119.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119948.log.gz 2023-11-08T19:33:44.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.103590.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103590.log.gz 2023-11-08T19:33:44.435 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104659.log 2023-11-08T19:33:44.435 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94812.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94812.log.gz 2023-11-08T19:33:44.435 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97829.log 2023-11-08T19:33:44.436 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.104659.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104659.log.gz 2023-11-08T19:33:44.436 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105671.log 2023-11-08T19:33:44.437 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.97829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105189.log 2023-11-08T19:33:44.438 INFO:teuthology.orchestra.run.smithi119.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97829.log.gz 2023-11-08T19:33:44.438 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105671.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105671.log.gz 2023-11-08T19:33:44.438 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107145.log 2023-11-08T19:33:44.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.105189.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.105189.log.gz 2023-11-08T19:33:44.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107481.log 2023-11-08T19:33:44.439 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107145.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107145.log.gz 2023-11-08T19:33:44.439 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96018.log 2023-11-08T19:33:44.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.107481.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107481.log.gz 2023-11-08T19:33:44.441 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94171.log 2023-11-08T19:33:44.441 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.96018.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-08T19:33:44.441 INFO:teuthology.orchestra.run.smithi119.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96018.log.gz 2023-11-08T19:33:44.442 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.94171.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94171.log.gz 2023-11-08T19:33:44.442 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112284.log 2023-11-08T19:33:44.443 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102418.log 2023-11-08T19:33:44.443 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.112284.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112284.log.gz 2023-11-08T19:33:44.444 INFO:teuthology.orchestra.run.smithi119.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109989.log 2023-11-08T19:33:44.444 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.102418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115567.log 2023-11-08T19:33:44.445 INFO:teuthology.orchestra.run.smithi119.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.102418.log.gz 2023-11-08T19:33:44.446 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.109989.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109989.log.gz 2023-11-08T19:33:44.447 INFO:teuthology.orchestra.run.smithi119.stderr:/var/log/ceph/ceph-client.admin.115567.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115567.log.gz 2023-11-08T19:33:44.494 INFO:teuthology.orchestra.run.smithi119.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-08T19:33:44.823 INFO:teuthology.orchestra.run.smithi188.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-08T19:33:44.865 INFO:teuthology.orchestra.run.smithi188.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-08T19:33:44.971 INFO:teuthology.orchestra.run.smithi188.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-08T19:33:44.981 INFO:teuthology.orchestra.run.smithi188.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-08T19:33:44.985 INFO:teuthology.orchestra.run.smithi188.stderr: 2023-11-08T19:33:44.986 INFO:teuthology.orchestra.run.smithi188.stderr:real 0m1.361s 2023-11-08T19:33:44.986 INFO:teuthology.orchestra.run.smithi188.stderr:user 0m5.415s 2023-11-08T19:33:44.989 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m0.559s 2023-11-08T19:33:45.260 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-08T19:33:45.437 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-08T19:33:45.439 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-08T19:33:45.629 INFO:teuthology.orchestra.run.smithi119.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-08T19:33:45.633 INFO:teuthology.orchestra.run.smithi119.stderr: 2023-11-08T19:33:45.633 INFO:teuthology.orchestra.run.smithi119.stderr:real 0m2.014s 2023-11-08T19:33:45.634 INFO:teuthology.orchestra.run.smithi119.stderr:user 0m5.603s 2023-11-08T19:33:45.634 INFO:teuthology.orchestra.run.smithi119.stderr:sys 0m2.082s 2023-11-08T19:33:45.635 INFO:tasks.ceph:Archiving logs... 2023-11-08T19:33:45.636 DEBUG:teuthology.misc:Transferring archived files from smithi119:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/remote/smithi119/log 2023-11-08T19:33:45.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T19:33:46.988 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/remote/smithi188/log 2023-11-08T19:33:46.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T19:33:47.843 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-08T19:33:47.852 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-08T19:33:47.853 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T19:33:47.862 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T19:33:48.020 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-08T19:33:48.030 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-08T19:33:48.031 DEBUG:teuthology.orchestra.run.smithi119:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:33:48.041 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T19:33:48.090 INFO:teuthology.orchestra.run.smithi119.stderr:bash: line 1: ntpq: command not found 2023-11-08T19:33:48.099 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2023-11-08T19:33:48.100 INFO:teuthology.orchestra.run.smithi119.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:33:48.100 INFO:teuthology.orchestra.run.smithi119.stdout:=============================================================================== 2023-11-08T19:33:48.100 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 +1953us[+1923us] +/- 84ms 2023-11-08T19:33:48.101 INFO:teuthology.orchestra.run.smithi119.stdout:^* hv02.front.sepia.ceph.com 3 6 377 13 -165us[ -194us] +/- 72ms 2023-11-08T19:33:48.101 INFO:teuthology.orchestra.run.smithi119.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -1272us[-1272us] +/- 64ms 2023-11-08T19:33:48.101 INFO:teuthology.orchestra.run.smithi119.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:33:48.109 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T19:33:48.109 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2023-11-08T19:33:48.109 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 +1913us[+1924us] +/- 84ms 2023-11-08T19:33:48.110 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv02.front.sepia.ceph.com 3 6 377 11 -222us[ -210us] +/- 72ms 2023-11-08T19:33:48.110 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 -1306us[-1306us] +/- 64ms 2023-11-08T19:33:48.110 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T19:33:48.111 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-08T19:33:48.125 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-08T19:33:48.126 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-08T19:33:48.136 DEBUG:teuthology.orchestra.run.smithi119:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-08T19:33:48.410 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-08T19:33:48.672 DEBUG:teuthology.orchestra.run.smithi119:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:33:48.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:33:48.783 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T19:33:48.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:33:48.896 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-08T19:33:48.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-08T19:33:48.916 INFO:teuthology.task.internal:Duration was 1327.034330 seconds 2023-11-08T19:33:48.917 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-08T19:33:48.928 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-08T19:33:48.928 DEBUG:teuthology.orchestra.run.smithi119:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T19:33:48.938 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T19:33:49.098 INFO:teuthology.orchestra.run.smithi119.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:33:49.108 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T19:33:49.378 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-08T19:33:49.379 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi119.front.sepia.ceph.com 2023-11-08T19:33:49.379 DEBUG:teuthology.orchestra.run.smithi119:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T19:33:49.449 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2023-11-08T19:33:49.450 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T19:33:49.529 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-08T19:33:49.529 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T19:33:49.539 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T19:33:49.636 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-08T19:33:49.637 DEBUG:teuthology.orchestra.run.smithi119:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T19:33:49.642 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T19:33:49.839 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-08T19:33:49.850 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-08T19:33:49.850 DEBUG:teuthology.orchestra.run.smithi119:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T19:33:49.942 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T19:33:50.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-08T19:33:50.062 DEBUG:teuthology.orchestra.run.smithi119:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:33:50.072 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:33:50.179 INFO:teuthology.orchestra.run.smithi119.stdout:kernel.core_pattern = core 2023-11-08T19:33:50.183 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2023-11-08T19:33:50.233 DEBUG:teuthology.orchestra.run.smithi119:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:33:50.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:33:50.285 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T19:33:50.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T19:33:50.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-08T19:33:50.353 INFO:teuthology.task.internal:Transferring archived files... 2023-11-08T19:33:50.354 DEBUG:teuthology.misc:Transferring archived files from smithi119:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/remote/smithi119 2023-11-08T19:33:50.354 DEBUG:teuthology.orchestra.run.smithi119:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T19:33:50.476 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101/remote/smithi188 2023-11-08T19:33:50.477 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T19:33:50.586 INFO:teuthology.task.internal:Removing archive directory... 2023-11-08T19:33:50.587 DEBUG:teuthology.orchestra.run.smithi119:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T19:33:50.597 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T19:33:50.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-08T19:33:50.671 INFO:teuthology.task.internal:Not uploading archives. 2023-11-08T19:33:50.671 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-08T19:33:50.685 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-08T19:33:50.686 DEBUG:teuthology.orchestra.run.smithi119:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T19:33:50.696 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T19:33:50.749 INFO:teuthology.orchestra.run.smithi119.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 19:33 /home/ubuntu/cephtest 2023-11-08T19:33:50.756 INFO:teuthology.orchestra.run.smithi188.stdout: 524321 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 19:33 /home/ubuntu/cephtest 2023-11-08T19:33:50.759 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-08T19:33:50.770 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-08T19:33:50.793 INFO:teuthology.nuke:Checking targets against current locks 2023-11-08T19:33:50.819 DEBUG:teuthology.nuke:shortname: smithi119 2023-11-08T19:33:50.819 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T19:33:50.843 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi119.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:02:51.385441', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMPBNva1EB0X+Rk9GaiKcTAed2A2VkQxzqBk9e/XzaaFlee/0pzNH2bS7q+Q+rsPjvt+wMPeHsePwojxmU8NtGY='} 2023-11-08T19:33:50.847 DEBUG:teuthology.nuke:shortname: smithi188 2023-11-08T19:33:50.847 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T19:33:50.874 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2023-11-08_19:01:12-fs:functional-main-distro-default-smithi/7452101', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 19:02:51.383928', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBOkKx9IMFOrO+AuaZwDyc1aHT05j90gu48HEtBN8zF7Nk+xxLdFUunqvdnYQFJjED4nXkaViAUM+irEx681lgs='} 2023-11-08T19:33:50.894 INFO:teuthology.orchestra.console.smithi119:Power off 2023-11-08T19:33:50.895 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T19:33:50.919 DEBUG:teuthology.orchestra.console.smithi119:power off output: Chassis Power Control: Down/Off 2023-11-08T19:33:50.920 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:33:50.940 INFO:teuthology.orchestra.console.smithi188:Power off 2023-11-08T19:33:50.941 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T19:33:50.951 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-11-08T19:33:50.964 DEBUG:teuthology.orchestra.console.smithi188:power off output: Chassis Power Control: Down/Off 2023-11-08T19:33:50.965 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:33:50.989 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-08T19:33:54.953 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:33:54.990 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:33:55.065 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-11-08T19:33:55.104 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-08T19:33:59.067 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:33:59.105 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:33:59.180 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is on 2023-11-08T19:33:59.218 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is on 2023-11-08T19:34:03.185 DEBUG:teuthology.orchestra.console.smithi119:pexpect command: ipmitool -H smithi119.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:34:03.219 DEBUG:teuthology.orchestra.console.smithi188:pexpect command: ipmitool -H smithi188.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T19:34:03.297 DEBUG:teuthology.orchestra.console.smithi119:check power output: Chassis Power is off 2023-11-08T19:34:03.331 DEBUG:teuthology.orchestra.console.smithi188:check power output: Chassis Power is off 2023-11-08T19:34:03.399 INFO:teuthology.orchestra.console.smithi119:Power off completed 2023-11-08T19:34:03.431 INFO:teuthology.orchestra.console.smithi188:Power off completed 2023-11-08T19:34:03.537 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 client.1-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snapshots} duration: 1327.034330368042 failure_reason: 'Test failure: test_kill_mdstable (tasks.cephfs.test_snapshots.TestSnapshots)' flavor: default owner: scheduled_pdonnell@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=a9fc6231aec34493b58e938c3b633200 status: fail success: false 2023-11-08T19:34:03.538 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T19:34:03.609 INFO:teuthology.run:FAIL